1. Symptoms
At a futures exchange, after upgrading the network to Gigabit Ethernet, only one network segment worked correctly, while the other 12 segments experienced issues. Data errors and frequent unexplained disconnections were observed. Initial suspicions were on the platform or software used, but multiple reinstalls and configurations did not resolve the problem. Furthermore, the same system worked fine in other trading networks. Suspicion shifted to potential issues in the cabling system, such as unqualified cables, interference signals, or grounding problems. Each network segment was connected using the existing Cat5 cabling system, which hadn’t undergone major changes. As the cabling system used standard Cat5 cable specifications for Gigabit Ethernet, designed to operate in full duplex with four pairs, 5-level encoding, and a signal bandwidth of precisely 100MHz, Cat5 cables should have been sufficient. Plus, the actual network traffic in the futures trading network was generally not close to the maximum capacity, remaining relatively low. Rigorous certification testing was conducted using professional cable testers, showing compliant parameters and no issues with pulse noise interference or grounding.
The only functional network segment was the one hosting market and trading servers. Most workstations and servers within this segment could access the internet, browse market data, and execute trades, with only a few exceptions. However, all other network segments, while functioning internally with a few exceptions, couldn’t establish reliable connections to the market and trading servers. During the system upgrade, the original cabling was retained and passed testing. The source of the problem remained elusive.
2. Diagnostic Process
Various factors could be causing the connectivity problems, such as issues with network hardware configuration, cabling system performance, platform installation and settings, application software configuration, and software conflicts. Based on user feedback, individual stations within each network segment seemed to work, while connections between network segments were challenging. This led to the suspicion that the fault might be related to network equipment settings and cabling system performance.
Using a network tester (F68X) connected to a segment that could access the servers and trading servers at 100Mbps, network traffic averaged 12% over five minutes, FCS frame check error frames were approximately 11%, and collisions were at 1.7% (within the normal range). The high proportion of FCS frame errors originated from stations in other network segments, accounting for 97% of error frames. The error rates across network segments were similar, making it unlikely that all segments experienced simultaneous issues at various stations. However, testing from the F68X to servers or stations within each network segment showed rapidly increasing FCS frame errors as traffic intensified, consistently across different stations. Enabling ICMP Ping on the network tester, statistics showed that approximately 96% had ICMP Ping fragmentation, and 91% were ICMP Monitor destination unreachable.
Conducting the same tests on other network segments, the results for the routers and switches in the market server and trading server segments mirrored the previous tests. The primary difference was that the other network segments’ network devices showed normal results: ICMP Ping fragmentation was 0%, and all were reachable; ICMP Monitor destination unreachable was 0%. It was apparent that the issue lay in the connectivity links between the market server segment and the other segments. Employing FLUKE’s DSP-4000 cable certification tester and Cat5e cables instead of the original Cat5 cables, the network returned to normal. During peak traffic, the market server segment had an average traffic load of 3%, indicating that the previous 12% of traffic during the issue was largely composed of retransmitted frames.
3. Conclusion
Gigabit Ethernet satisfies network users’ hunger for higher bandwidth applications, making it the next focal point for network development. The designers of Gigabit Ethernet had considered the existing application scope of Cat5 cables and selected a 100MHz physical bandwidth. In theory, Cat5 cables should have been sufficient for Gigabit Ethernet. However, practical statistics revealed that 1% to 5% of users could not connect or experienced intermittent and problematic connections. This means Gigabit Ethernet imposes stricter requirements on the parameters of Cat5 cables. Users who have subjected their Cat5 cabling systems to rigorous certification testing can ensure that most stations will work. For a few stations where parameters have limited margins, problems could arise. Parameters with a significant impact include PS NEXT (Power Sum Near-End Crosstalk), PS FEXT (Power Sum Far-End Crosstalk), ACR (Attenuation to Crosstalk Ratio), ELFEXT (Equal Level Far-End Crosstalk), and RL (Return Loss).
For diagnosing problematic Cat5 cabling systems, the use of Cat5e or higher category cables can be a straightforward solution to enhance network performance at minimal cost. It is important to replace the connectors when upgrading to ensure installation quality.
4. Afterword
Following our recommendations, the futures exchange tested all installed Cat5 cables with the DSP-4000 cable tester using the TIA Cat5e Channel UTP100 standard. The network consisted of 1300 stations, and 21 non-compliant cables were identified. After replacing the problematic cables with Cat5e cables, all stations worked correctly, and network performance remained excellent (with an average traffic peak of only 3%).