Posted on 2005-05-05 08:21:52, modified on 2006-01-09 16:29:23
Tags: Networking
We got complaints from people outside our network that the performance of our webservers was very bad. And it was, I didn't get more than 5 to 10 kbps, while I was used to 180 kbps.
The slowness was only when the traffic was going via our primary uplink, not when it was going via our backup uplink, so it wasn't a webserver issue.
I could do all ping-tests I wanted, not a single packet was lost. But bulk transfers just sucked big time.
Tcpdump showed there was about 10 seconds of traffic and 5 seconds of silence, in which the TCP layer was trying to get acknowledgements through.
When reporting it to our internet provider, former Comindico but these days SPTel (or SPT as they announce themselves over the phone), they closed the ticket saying "ping tests show there is no packet loss". Three seconds after my phone call to them explaining that they should read what I described as the problem, with a proper mention of the URL they should try to download, they had reopened the ticket.
After four hours I got called back with the request for more information. I asked them if they had tried to download the file at the URL I had supplied. The person said "No". It took them three times to get the right URL out of my email. And then he downloaded it at about 1.5Mbps. I was stunned.
Why did it work inside their network and not outside their network? What was wrong with me? (don't answer please).
Later that evening I checked of the webserver again, and saw the reason it was so fast: Somebody else within Comindico who had looked at the ticket had already downloaded the whole file once and their proxy server had cached it. And the logfile said: '"GET /~edwin/kiki.asf HTTP/1.0" 304'. 304 means: File not modified. And the proxy then uses the local copy. I'm glad they tested the speed between their proxy and their desktop, but it didn't help me.
(Mental remark: Support personal at ISPs should have the option of not going through a proxy if they want to troubleshoot problems. They often need raw TCP access to figure out what is happening and forcing them to go through proxies will blurr their view. Just like it did here).
So the problem was still there and it got me annoyed. I decided to call Uecomm, our provider of the link towards Comindico. They provide a Layer 2 Ethernet Service and maybe they could help me out with more information. Their NOC answered and accepted the problem. Half an hour later the link to Comindico went down again, and came up. This can't be a coincidence.
I tried again and got a clean 50kbps over it. I tried a file transfer from Freefall.freebsd.org and got a clean 180kbps. I was happy again. And the phone rang, Uecomm again. They had found the problem, one of the ethernet switches of them had the link configured as half-duplex. After they had set it to full-duplex again (and the link bounced) it was all back again. Life is beautiful again!
| Share on Facebook | Share on Twitter