A Tisket, a Tasket, I've Lost My TCP Packet

A gentleman today wondered if his expensive leased fibre line was causing packet loss, even though he compared it with an ADSL line from the server to the host. As Dennis Rockwell of BBN pointed out "What you have discovered is that your 2Mbps link is not the bottleneck; that lies elsewhere in your network path. The extra bandwidth of the fiber link cannot help this application".

Dennis is correct. But how do you know where to look to fix the problem? Here's a little story from a manager of international datacenters in Japan and the US to illustrate how complicated the issue can become...

"There was a weak laser in an inexpensive optical ethernet LAN connection used to convey a WAN from one floor of a datacenter in Ariake to the other, resulting in a significant increase in bit error rate. While the laser was within spec for the product, the product was used on a fibre at the edge of the distance of the link - but still also within spec.

The simplest thing to do was to replace it with an optical ethernet WAN connection more suited to the use. Unfortunately, the datacenter insisted that you could only use this inadaquate connection to go between floors. The rule could not be changed to repair this situation. Everyone along the path acknowledged the problem, as they all were blamed, but still the rule 'could not be changed'.

This problem impacted Japanese consumers using one of the most heavily trafficed Japanese websites at the time. The problem persisted for months until we were able to consolodate on one floor only." (from my book on datacenter management and operations).