NetworkPerformanceUTA < Admins

r14 - 10 Jun 2008 - 10:37:02 - RobertGardnerYou are here: TWiki >

Admins Web > NetworkPerformanceUTA

Tracepath output to dct00.usatlas.bnl.gov
ifconfig for eth0
ethtool
More information on eth0
First test
Second test (10 streams )
statistics difference pre/post reverse iperf test
GK03

Tracepath output to dct00.usatlas.bnl.gov

 1:  ndt.swt2.uta.edu (129.107.255.9)                       0.131ms pmtu 1500
 1:  swt2msfc.swt2.uta.edu (129.107.255.251)                2.057ms 
 2:  as16905_m20.uta.edu (129.107.35.33)                  asymm  3   3.507ms 
 3:  ntg-gw2-dllstx-vl-403-ge-0-0-0.northtexasgigapop.org (206.223.141.138) asymm  4   3.365ms 
 4:  dlls-ntg-newnet-lt-0-3-0-1-layer3.tx-learn.net (74.200.187.21) asymm  5   3.165ms 
 5:  hstn-dlls-newnet-ge-1-0-3-309-layer3.tx-learn.net (74.200.187.17) asymm  6   8.908ms 
 6:  hstn-hstn-newnet-layer3.tx-learn.net (74.200.187.5)  asymm  7  10.504ms 
 7:  64.57.28.56 (64.57.28.56)                            asymm  8  24.531ms 
 8:  so-1-2-0.0.rtr.chic.net.internet2.edu (64.57.28.37)  asymm  9  49.046ms 
 9:  chi-sl-10ge-abilene.es.net (198.125.140.229)          35.712ms 
10:  chislsdn1-chislmr1.es.net (134.55.219.25)            asymm 11  35.164ms 
11:  chiccr1-chislsdn1.es.net (134.55.207.33)             asymm 12  35.712ms 
12:  aofacr1-chicsdn1.es.net (134.55.218.94)              asymm 13  62.371ms 
13:  bnlmr1-aoacr1.es.net (134.55.217.57)                  64.216ms 
14:  bnlsite-bnlmr1.es.net (198.124.216.178)               64.419ms 
15:  dct00.usatlas.bnl.gov (192.12.15.8)                   65.911ms reached
     Resume: pmtu 1500 hops 15 back 15

ifconfig for eth0

eth0    Link encap:Ethernet  HWaddr 00:14:22:1A:94:5D  
          inet addr:129.107.255.9  Bcast:129.107.255.255  Mask:255.255.255.0
          inet6 addr: fe80::214:22ff:fe1a:945d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:321882 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26560 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:21353233 (20.3 MiB)  TX bytes:2745999 (2.6 MiB)
          Base address:0xdcc0 Memory:dfae0000-dfb00000

ethtool

knoppix@0[~]$ ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off
knoppix@0[~]$ ethtool -i eth0
driver: e1000
version: 7.3.15-k2-NAPI
firmware-version: N/A
bus-info: 0000:06:07.0

More information on eth0

knoppix@0[~]$ ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256
RX Mini:        0
RX Jumbo:       0
TX:             256

knoppix@0[~]$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:14:22:1A:94:5D  
          inet addr:129.107.255.9  Bcast:129.107.255.255  Mask:255.255.255.0
          inet6 addr: fe80::214:22ff:fe1a:945d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1165844 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2730994 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:80691219 (76.9 MiB)  TX bytes:4106711419 (3.8 GiB)
          Base address:0xdcc0 Memory:dfae0000-dfb00000

First test

knoppix@0[~]$ iperf -c dct00.usatlas.bnl.gov -w4M -i2 -t60
------------------------------------------------------------
Client connecting to dct00.usatlas.bnl.gov, TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 129.107.255.9 port 58540 connected with 192.12.15.8 port 5001
[  3]  0.0- 2.0 sec  5.90 MBytes  24.7 Mbits/sec
[  3]  2.0- 4.0 sec  2.08 MBytes  8.72 Mbits/sec
[  3]  4.0- 6.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  6.0- 8.0 sec  2.01 MBytes  8.42 Mbits/sec
[  3]  8.0-10.0 sec  2.03 MBytes  8.52 Mbits/sec
[  3] 10.0-12.0 sec  2.00 MBytes  8.39 Mbits/sec
[  3] 12.0-14.0 sec  4.05 MBytes  17.0 Mbits/sec
[  3] 14.0-16.0 sec  7.95 MBytes  33.4 Mbits/sec
[  3] 16.0-18.0 sec  9.98 MBytes  41.9 Mbits/sec
[  3] 18.0-20.0 sec  11.9 MBytes  50.0 Mbits/sec
[  3] 20.0-22.0 sec  11.9 MBytes  50.1 Mbits/sec
[  3] 22.0-24.0 sec  11.9 MBytes  50.0 Mbits/sec
[  3] 24.0-26.0 sec  12.0 MBytes  50.2 Mbits/sec
[  3] 26.0-28.0 sec  14.1 MBytes  58.9 Mbits/sec
[  3] 28.0-30.0 sec  12.7 MBytes  53.2 Mbits/sec
[  3] 30.0-32.0 sec  13.4 MBytes  56.2 Mbits/sec
[  3] 32.0-34.0 sec  14.0 MBytes  58.8 Mbits/sec
[  3] 34.0-36.0 sec  4.10 MBytes  17.2 Mbits/sec
[  3] 36.0-38.0 sec  6.07 MBytes  25.5 Mbits/sec
[  3] 38.0-40.0 sec  8.04 MBytes  33.7 Mbits/sec
[  3] 40.0-42.0 sec  8.02 MBytes  33.7 Mbits/sec
[  3] 42.0-44.0 sec  8.08 MBytes  33.9 Mbits/sec
[  3] 44.0-46.0 sec  9.94 MBytes  41.7 Mbits/sec
[  3] 46.0-48.0 sec  8.06 MBytes  33.8 Mbits/sec
[  3] 48.0-50.0 sec  9.98 MBytes  41.9 Mbits/sec
[  3] 50.0-52.0 sec  9.96 MBytes  41.8 Mbits/sec
[  3] 52.0-54.0 sec  9.86 MBytes  41.4 Mbits/sec
[  3] 54.0-56.0 sec  10.0 MBytes  42.0 Mbits/sec
[  3] 56.0-58.0 sec  12.0 MBytes  50.1 Mbits/sec
[  3] 58.0-60.0 sec  9.98 MBytes  41.8 Mbits/sec
[  3]  0.0-60.0 sec    252 MBytes  35.2 Mbits/sec

Second test (10 streams )

knoppix@0[~]$ iperf -c dct00.usatlas.bnl.gov -w4M -i2 -t60 -P 10 | grep -i sum
[SUM]  0.0- 2.0 sec  78.8 MBytes    330 Mbits/sec
[SUM]  2.0- 4.0 sec  20.1 MBytes  84.1 Mbits/sec
[SUM]  4.0- 6.0 sec  39.8 MBytes    167 Mbits/sec
[SUM]  6.0- 8.0 sec  39.8 MBytes    167 Mbits/sec
[SUM]  8.0-10.0 sec  57.7 MBytes    242 Mbits/sec
[SUM] 10.0-12.0 sec  42.0 MBytes    176 Mbits/sec
[SUM] 12.0-14.0 sec  57.8 MBytes    242 Mbits/sec
[SUM] 14.0-16.0 sec  55.8 MBytes    234 Mbits/sec
[SUM] 16.0-18.0 sec  62.0 MBytes    260 Mbits/sec
[SUM] 18.0-20.0 sec  59.9 MBytes    251 Mbits/sec
[SUM] 20.0-22.0 sec  73.9 MBytes    310 Mbits/sec
[SUM] 22.0-24.0 sec  71.8 MBytes    301 Mbits/sec
[SUM] 24.0-26.0 sec  60.4 MBytes    253 Mbits/sec
[SUM] 26.0-28.0 sec  46.0 MBytes    193 Mbits/sec
[SUM] 28.0-30.0 sec  53.7 MBytes    225 Mbits/sec
[SUM] 30.0-32.0 sec  55.8 MBytes    234 Mbits/sec
[SUM] 32.0-34.0 sec  56.0 MBytes    235 Mbits/sec
[SUM] 34.0-36.0 sec  61.8 MBytes    259 Mbits/sec
[SUM] 36.0-38.0 sec  61.9 MBytes    260 Mbits/sec
[SUM] 38.0-40.0 sec  58.0 MBytes    243 Mbits/sec
[SUM] 40.0-42.0 sec  57.2 MBytes    240 Mbits/sec
[SUM] 42.0-44.0 sec  62.6 MBytes    263 Mbits/sec
[SUM] 44.0-46.0 sec  68.0 MBytes    285 Mbits/sec
[SUM] 46.0-48.0 sec  72.0 MBytes    302 Mbits/sec
[SUM] 48.0-50.0 sec  72.1 MBytes    303 Mbits/sec
[SUM] 50.0-52.0 sec  78.1 MBytes    328 Mbits/sec
[SUM] 52.0-54.0 sec  82.0 MBytes    344 Mbits/sec
[SUM] 54.0-56.0 sec  63.9 MBytes    268 Mbits/sec
[SUM] 56.0-58.0 sec  74.0 MBytes    311 Mbits/sec
[SUM] 58.0-60.0 sec  76.2 MBytes    320 Mbits/sec
[SUM]  0.0-60.7 sec  1.78 GBytes    252 Mbits/sec

statistics difference pre/post reverse iperf test

run iperf as server with: iperf -s -w 4MB

root@0[~]# diff /tmp/stats_pro /tmp/stats_pre 
2,6c2,6
<      rx_packets: 5124086
<      tx_packets: 4718311
<      rx_bytes: 6088470957
<      tx_bytes: 4246538145
<      rx_broadcast: 298780
---
>      rx_packets: 1166033
>      tx_packets: 2731146
>      rx_bytes: 80711701
>      tx_bytes: 4106772163
>      rx_broadcast: 298663
41,42c41,42
<      rx_long_byte_count: 6088470957
<      rx_csum_offload_good: 4818476
---
>      rx_long_byte_count: 80711701
>      rx_csum_offload_good: 860546

GK03

initial sysctl parameters

net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.core.rmem_max = 8388608 net.core.wmem_max = 8388608 net.core.rmem_default = 65536 net.core.wmem_default = 65536 net.ipv4.tcp_rmem = 4096 131072 8388608 net.ipv4.tcp_wmem = 4096 65536 8388608 net.core.netdev_max_backlog = 300

ethtool output

[root@gk03 root]# ethtool -k eth1
Offload parameters for eth1:
Cannot get device tcp segmentation offload settings: Operation not supported
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off


[root@gk03 root]# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             256
RX Mini:        0
RX Jumbo:       0
TX:             256

iperf -c dct100

[root@gk03 root]# iperf -c dct00.usatlas.bnl.gov -w 4MB -i 4 -t 60 
------------------------------------------------------------
Client connecting to dct00.usatlas.bnl.gov, TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 129.107.255.4 port 36860 connected with 192.12.15.8 port 5001
[  3]  0.0- 4.0 sec  14.4 MBytes  30.1 Mbits/sec
[  3]  4.0- 8.0 sec  9.88 MBytes  20.7 Mbits/sec
[  3]  8.0-12.0 sec  11.9 MBytes  25.0 Mbits/sec
[  3] 12.0-16.0 sec  13.9 MBytes  29.1 Mbits/sec
[  3] 16.0-20.0 sec  15.9 MBytes  33.3 Mbits/sec
[  3] 20.0-24.0 sec  16.0 MBytes  33.7 Mbits/sec
[  3] 24.0-28.0 sec  18.0 MBytes  37.8 Mbits/sec
[  3] 28.0-32.0 sec  20.0 MBytes  42.0 Mbits/sec
[  3] 32.0-36.0 sec  20.0 MBytes  41.9 Mbits/sec
[  3] 36.0-40.0 sec  21.9 MBytes  46.0 Mbits/sec
[  3] 40.0-44.0 sec  24.1 MBytes  50.5 Mbits/sec
[  3] 44.0-48.0 sec  24.0 MBytes  50.4 Mbits/sec
[  3] 48.0-52.0 sec  27.9 MBytes  58.5 Mbits/sec
[  3] 52.0-56.0 sec  27.9 MBytes  58.5 Mbits/sec
[  3] 56.0-60.0 sec  12.1 MBytes  25.3 Mbits/sec
[  3]  0.0-61.5 sec    278 MBytes  37.9 Mbits/sec

sysctl changes

net.ipv4.tcp_rmem = 4096 87380 20000000 net.ipv4.tcp_wmem = 4096 87380 20000000 # maximum receive socket buffer size, default 131071 net.core.rmem_max = 20000000 # maximum send socket buffer size, default 131071 net.core.wmem_max = 20000000 net.core.netdev_max_backlog = 2500

-- DantongYu - 04 Oct 2007

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.

Attachments

Admins

USATLAS Site Admin Topics
Admin Support Home
Site Setup Help
Search

Facility Integration Program
Integration home
Proof-Xrootd

Site Admin External Links
dCache
GUMS

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback