Strange HTTP Performance Variations Between OS’s
In a recent talk at VelocityConf, John Rauser explained the effect of TCP Slow Start and Congestion Control on web performance. He pointed out that RFC 1122 states:
Recent work by Jacobson on Internet congestion and
TCP retransmission stability has produced a transmission
algorithm combining “slow start” with “congestion
avoidance”. A TCP MUST implement this algorithm.
While examining the impact of these with my new HTTP performance testing tool (LatentSee) I noticed that the charts generated on my Mac & Windows machines didn’t seem to match the theory. Usually we would expect to receive 3 packets (< 4500 bytes) in the first segment. Instead I am seeing up to 67KB on the Mac and around 35KB on Windows 7.
Effect of file size on retrieval time from slicehost.latentsee.com
Originals (Theory | Ubuntu | Windows 7 | Mac OS X)
When I ran the same tests against a different hosting provider (Brightbox), Windows and Mac OSX behaved more like my Ubuntu box.
Effect of file size on retrieval time from brightbox.latentsee.com
Originals (Theory | Ubuntu | Windows 7 | Mac OS X)
I’m very curious about these differences. It takes the same time for my Mac to retrieve any file up to 67KB in size from Slicehost. Have they tuned their TCP stack differently? Why then does Ubuntu behave similarly against both Slicehost and Brightbox? Is everyone conforming to the RFCs?
I’d love to hear if people can reproduce these results.
You can experiment with LatentSee using your web browser.
USA: slicehost.latentsee.com
UK: brightbox.latentsee.com
AUS: ultraserve.latentsee.com
You can also put latentsee.php on your Apache webserver to test your own servers (be sure to disable compression on the vhost).
I am curious what the results are on a FreeBSD or NetBSD system. Since the POSIX part of OS X is mostly taken from BSD, I wonder if the implementation change is trully because of BSD or OS X
Comment by Bodaniel Jeanes — July 27, 2010 @ 8:43 am
I get the same “big step” results with *both* Mac OS X and Vista.
I suspect this may be a firewall doing DPI, or perhaps DDoS protection ?
Comment by Michael T — July 27, 2010 @ 11:37 am
There are oodles of RFCs and internet drafts that modify TCP after RFC1122, and each OS has adopted a different set (delayed ack, sack, alternate congestion algorithms like cubic and reno, window scaling, ecn, rfc2581, etc)
On OSX, try turning off lots of the tcp options exposed via systl and seeing if you can get closer to RFC1122 behaviour.
To get a list of the options, run:
sysctl -a | grep tcp
Comment by James Healy — July 28, 2010 @ 4:58 am