osgserv01.slac.stanford.edu is the SLAC OSG gatekeeper and gridftp server.
osgserv04.slac.stanford.edu is a gridftp server dedicated for ATLAS DDM operation.

They are both

SUN v20z with 2 AMD Opteron 244 processor (single core, 1.8Ghz), 2 GB memory, 
and 2 Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02). The 
first NIC is linked to switch at 1Gb.

RHEL4-32 bit kernel 2.6.9-55.0.2 ELsmp and tg3 ethernet driver.

For turning, we did the following:

1. Add the following to /etc/sysctl.conf and then run "sysctl -p":

net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

net.core.rmem_default = 110592
net.core.wmem_default = 110592
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 2500

2. run "ethtool -K eth0 tso on" to turn on "tcp segmentation offload: on". 
This improves the TX performance by using NIC hardware functions. The 
original state was "off".

3. run "ifconfig eth0 txqueuelen 10000". The original TX queue length is 1000

We suspect that firewalls at SLAC limits the single stream performance to ~ 800 Mbit/sec at both directions. And the competing traffic from other SLAC network activities contributes significant fluctuation to the RX and TX measurements using iperf. However, dramatic iperf performance improvement has been made.

The first two turning items significantly improve the RX and TX performance, by at least 10x. We do not expect the last one to improve the performance as it only make sense to 10Gb links.

-- WeiYang - 12 Oct 2007