Topic: Network Throughput Problem?

[dhalliday]

Is the max network throughput limit per process? If so where is this configured?

I have just moved from a server with 100Meg ports to one with 1Gig ports and see the following:

1) On the new GigE equipped server.
   I perform a wget and I only get about 21M bytes/s
   I can run about 5 wget's and get a total of 100M bytes/s (20 M/s each).

2) On another test machine running vanilla Centos 5.6
    The single wget will perform at a full 100M bytes/s
    2 wgets would give about 50M bytes each
    and so on

3) On my old SME Server that was only equipped with a 100Meg interfaces
    I can do a single wget for (10M bytes/s).

So it is looking like the new machine has a limit of about 20M bytes/s per process.

Hope someone has some insight into this.

Many thanks,

Dave.

[CharlieBrady]

Is the max network throughput limit per process?

No.

[dhalliday]

OK it looks like this is a Tcp tuning issue.

I can get to 100M bytes+ by setting

 echo "4096 87380 4194304" > /proc/sys/net/ipv4/tcp_rmem
 echo "4096 16384 4194304" > /proc/sys/net/ipv4/tcp_wmem

While these allow the speed to go up above 100M they are choppy, in that the speed does drop to 50ish then work its way back up then drop again... etc.

Dave.

[dhalliday]

OK, I think the choppiness was a limit of writing the downloaded file to the local disk! If I send it with -O /dev/null I now get a full GigE download.

[gregswallow]

File a bug report.  I googled a bit about network throughput tuning on CentOS and checked a standard CentOS 5.6 install and /etc/sysctl.conf has some settings that are not in the templated /etc/sysctl.conf:

/sbin/sysctl -p
...shows you what is being overridden in the /etc/sysctl.conf file on each server. 

This is what is on the standard CentOS server:
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456

I think some of those settings might account for the difference you're seeing from SME to CentOS, but better to discuss that in the bug report.

[dhalliday]

7.x is still based on the older Centos 4.x so you need to compare to a vanilla 4.x install

On the vanilla Centos 4.x install the buffers are set to

net.ipv4.tcp_rmem = 4096        87380   174760
net.ipv4.tcp_wmem = 4096        16384   131072

This matches SME Servers 7.x settings but are way to low for GigE

On Centos 5 they were upped to

net.ipv4.tcp_rmem = 4096        87380   4194304
net.ipv4.tcp_wmem = 4096        16384   4194304

This will presumably match the SME Server 8.x if and when it is ever released.

For even higher network speeds (10Gig)

I found the following recommended settings.

http://fasterdata.es.net/fasterdata/host-tuning/linux/

I'm not sure this is a bug per se, as SME matches the upstream but I suppose it could be seen that way and would be easy enough to fix in a patch update.

Dave.

[thomasc73]

Dave,

My 7.5 is on Gigabit too.
I want to fine-tune the throughput to the max

Please share a how-to.
thanks.

tom