Understanding the Role of the SYN Flag in Network Troubleshooting with Wireshark

Background

• A couple of days ago during an internship, I was dealing with a bug in a version of the program and fixed it. After the fix, I found that while other machines could run it normally, there was one specific machine that couldn’t receive data from my version. Interestingly, the version with the bug could receive data normally.
• A brief explanation of the program’s function: It uses an RJ45 connection to communicate via TCP within a local area network.

Solution

• Here’s the final solution: Control Panel -> System and Security -> Windows Defender Firewall -> Allow an app or feature through Windows Defender Firewall.
  • Then locate the program and check all three boxes for it. (PS: I’m not quite sure why this works, it’s not what I expected. For more details, see [Mental Journey].) Click OK and restart your program.
    • Related article—Differences between public and private networks in Windows
• The most aggressive approach is to turn off the firewall directly.

Mental Journey

In the initial stage of the problem, I first used Wireshark to capture packets.

• I discovered that the TCP three-way handshake packets sent to the specified port on the local machine were marked as:

[TCP Retransmission][TCP Port numbers reused] 

• [TCP Retransmission] indicates a timeout retransmission, suggesting that the other side is not receiving a response from the local machine.
• The term [TCP Port numbers reused] literally seemed to mean that the port was occupied, so I used relevant commands to check the specified port status. The problem wasn’t resolved.

netstat -ano | findstr "port number"

• After researching, I found that [TCP Port numbers reused] does not mean the port is occupied. Instead, Wireshark marks reused IP+port combinations, and “port numbers reused” is just a label, not an error message.
  • Referenced in the TCP Port numbers reused comments section, which also provides the official documentation explanation—7.5. TCP Analysis.

Set when the SYN flag is set (not SYN+ACK), we have an existing conversation using the same addresses and ports, and the sequence number is different than the existing conversation’s initial sequence number.

After setting the SYN flag (not SYN+ACK), we have an existing session using the same addresses and ports, and the sequence number is different from the initial sequence number of the existing session.

I then moved on to examine the program.

• I continued by trying to monitor and simulate data transmission to the specified port using a TCP/UDP debugging tool based on Qt graphical interface, but there was no response.
• Then, I started installing QT on the machine, attempting to trace the issue from the source code. However, I also felt it wasn’t the code issue. Firstly, the changes I made did not involve data communication code, and secondly, the problem only occurred on this machine while others ran the program fine, indicating the issue was unique to this machine’s settings.
• Despite my doubts, I installed the QT environment. The program was developed in the QT5 environment. It turned out QT6, which was installed using a domestic source, is not backward compatible with 5, leading to many compile-time errors. Ultimately, I couldn’t resolve them, so I gave up.
  • Installing QT with a domestic source—Qt6 Installation Tutorial—Domestic Source
    • Download the appropriate downloader, switch to the folder using PowerShell, run the program command, and add the domestic source suffix. For example: using Tsinghua’s mirror.

.\ xxx.exe --mirror http://mirrors.tuna.tsinghua.edu.cn/qt

By chance, I began looking into the firewall settings.

• For the solution, see the [Solution] section.
• Although the issue was resolved, there were still some nuances that didn’t align with my assumptions. Upon further debugging, I noticed that altering certain settings caused a new entry to appear in the list each time the software was opened.

• Additionally, I thought checking only the private network box was enough, but it turned out not to be the case and I had to check the public network too. Compared to the public network, the private network has higher privileges.
  • Related article—Differences between public and private networks in Windows
  • The explanation from GPT, (PS: I verified it with New Bing, it appears correct.)

• As mentioned in the [Solution] section, if problems persist, turning off the firewall altogether is the ultimate solution.

Summary

• When solving problems, it’s important to clear your thoughts and consider potential causes. Some ideas might fleetingly pass by; it might be worthwhile to jot them down if possible.