making bbPress (and WordPress) work better!

CentOS 6.4 update finally supports IW10 for tcp/ip tuning

iw10Google and others have published several whitepapers showing benchmarks where tuning tcp/ip on servers can really help web browsing performance.

One of these important tweaks is known as IW10 which improves the tcp/ip congestion window by reducing ACKs (initally sending 10 packets instead of only 3) and is enabled by increasing initcwnd and initrwnd.

Unfortunately those on CentOS (one of the most popular linux production distributions) were not able to take advantage of this – until this weekend when CentOS 6.4 was shipped.

CentOS 6.3 could change initcwnd but NOT initrwnd (RWIN) which requires a kernel > 2.6.33

CentOS 6.4 is 2.6.32-358.0.1.el6 (Red Hat 4.4.7-3) so the feature has been backported

Here’s how to take advantage of IW10 on CentOS 6.4

This is obviously for power users with root access to their VPS or dedicated servers (this will NOT work on Virtuozzo VZ vps!)

First of course do a yum update to get Centos 6.4

Then take a look at your current settings via

ip route show

You are looking for the last line that will say something like

default via 4.3.2.1 dev eth0

Then just take that line and add two parts to it

ip route change default via 4.3.2.1 dev eth0 initcwnd 10 initrwnd 10

And bingo, you now have IW10 support.
If something goes wrong, it will just complain and not make the change.

Note these changes aren’t permanent, you will have to add the line to /etc/rc.local
or similar, to make it happen on each reboot

There are other important things to tune with tcp/ip, but one of the other most important is to disable “slow start after idle”

sysctl -a | grep tcp_slow_start_after_idle
sysctl -w net.ipv4.tcp_slow_start_after_idle=0

It is possible to actually examine the performance improvements of making this change on a packet level but it requires the use of wireshark and and counting ACKs which is a bit tedious so I will skip that for now.

Unfortunately it might be YEARS before CentOS gets things like TCP Fast Open which has only appeared a few months ago in linux kernel > 3.5 but I am still investigating if it’s possible in CentOS 6.4 to easily reduce the RTT initial timeout from 3 seconds to 1 second which is another recommended tcpip tuning improvement.

6 responses

  1. Anonymous

    first and foremost… thanks for the heads up that this was backported to 6.4 even tho its running 2.6.32… we were about to dive deep into shit and start compiling a custom kernel for our centos 6.3 box.

    initially, we thought we could just upgrade the kernel to -358 and have this available.

    not so.

    doing yum update kernel* and a reboot is not sufficient.. a full update to 6.4 was required… even tho we had 6.3 running kernel -358, it would complain about “initcwnd is garbage”. sorry, but i didnt copy the string route dumped, but something along those lines. im guessing the route command (or some other parts) were updated to support this…

    regardless, thanks again!

    March 19, 2013 at 5:24 am

    • Anonymous

      btw just a quick heads up… after running this… we did see our overseas latencies drop by about 15%+ as was written in the google paper on this. at least as reported by http://www.webpagetest.org, which i know isnt the most accurate metric and all… but initial connection / time to first byte did drop without a doubt.

      March 19, 2013 at 5:28 am

  2. Pingback: 調整 Linux 下的 Initial Window (IW) | ATI的硬體&攝影網誌

  3. If I have a host with 2 ethernet connections, ie public and private, can i do this on both interfaces? Any change to the command syntax?

    August 27, 2013 at 2:51 pm

  4. Anonymous

    Just to be sure, it the TCP fast open and the RTT initial timeout the same thing or are it 2 different things?

    September 30, 2013 at 3:28 am

  5. Pingback: TCP’s Initial Congestion Window – IW10 | Vincent ROBERT Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 45 other followers