NetworkTestBetweenBNLandMWT2

Test Between BNL and University of Chicago.
- Tuning on 21 Sep 2007

Test Between BNL and University of Chicago.

Tuning on 21 Sep 2007

Network test between BNL and University of Chicago.

-- Main.DantongYu - 21 Sep 2007
[root@dct00 ~]# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on


[root@dct00 ~]# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             512
RX Mini:        0
RX Jumbo:       0
TX:             512

[root@dct00 ~]#  iperf -s -w 4MB -i 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  4] local 192.12.15.8 port 5001 connected with 128.135.158.238 port 34433
[  4]  0.0- 4.0 sec  25.3 MBytes  53.1 Mbits/sec
[  4]  4.0- 8.0 sec  26.9 MBytes  56.4 Mbits/sec
[  4]  8.0-12.0 sec  26.9 MBytes  56.4 Mbits/sec
[  4] 12.0-16.0 sec  26.6 MBytes  55.7 Mbits/sec
[  4] 16.0-20.0 sec  26.7 MBytes  55.9 Mbits/sec
[  4] 20.0-24.0 sec  27.2 MBytes  57.0 Mbits/sec
[  4] 24.0-28.0 sec  26.8 MBytes  56.1 Mbits/sec
[  4] 28.0-32.0 sec  26.7 MBytes  56.0 Mbits/sec
[  4] 32.0-36.0 sec  26.9 MBytes  56.4 Mbits/sec
[  4] 36.0-40.0 sec  26.9 MBytes  56.4 Mbits/sec
[  4] 40.0-44.0 sec  27.4 MBytes  57.4 Mbits/sec
[  4] 44.0-48.0 sec  27.6 MBytes  58.0 Mbits/sec
[  4] 48.0-52.0 sec  27.5 MBytes  57.7 Mbits/sec
[  4] 52.0-56.0 sec  28.0 MBytes  58.8 Mbits/sec
[  4] 56.0-60.0 sec  27.2 MBytes  57.1 Mbits/sec
[  4]  0.0-60.0 sec    405 MBytes  56.5 Mbits/sec

After UC made network performance tuning. 

[  4] local 192.12.15.8 port 5001 connected with 128.135.158.238 port 34814
[  4]  0.0- 4.0 sec    370 MBytes    777 Mbits/sec
[  4]  4.0- 8.0 sec    280 MBytes    587 Mbits/sec
[  4]  8.0-12.0 sec    364 MBytes    764 Mbits/sec
[  4] 12.0-16.0 sec    449 MBytes    941 Mbits/sec
[  4] 16.0-20.0 sec    330 MBytes    691 Mbits/sec
[  4] 20.0-24.0 sec    365 MBytes    765 Mbits/sec
[  4] 24.0-28.0 sec    449 MBytes    941 Mbits/sec
[  4] 28.0-32.0 sec    332 MBytes    697 Mbits/sec
[  4] 32.0-36.0 sec    449 MBytes    941 Mbits/sec
[  4] 36.0-40.0 sec    365 MBytes    766 Mbits/sec
[  4] 40.0-44.0 sec    449 MBytes    941 Mbits/sec
[  4] 44.0-48.0 sec    336 MBytes    704 Mbits/sec
[  4] 48.0-52.0 sec    276 MBytes    578 Mbits/sec
[  4] 52.0-56.0 sec    393 MBytes    825 Mbits/sec
[  4] 56.0-60.0 sec    307 MBytes    643 Mbits/sec
[  4]  0.0-60.2 sec  5.39 GBytes    769 Mbits/sec

[root@dct00 networktest]# diff ethtool-S-eth1 ethtool-S-eth1-after 
2,5c2,5
<      rx_packets: 336906150
<      tx_packets: 472018579
<      rx_bytes: 422471354
<      tx_bytes: 1127743202
---
>      rx_packets: 341048254
>      tx_packets: 473242704
>      rx_bytes: 2413624457
>      tx_bytes: 1214142946
9c9
<      multicast: 4821048
---
>      multicast: 4821235
36,37c36,37
<      rx_long_byte_count: 249530574522
<      rx_csum_offload_good: 328437561
---
>      rx_long_byte_count: 255816694921
>      rx_csum_offload_good: 332579363


[root@dct00 networktest]# iperf -c  uct2-dc1.uchicago.edu  -w 4MB -t 60 -i 4
------------------------------------------------------------
Client connecting to uct2-dc1.uchicago.edu, TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 192.12.15.8 port 40401 connected with 128.135.158.238 port 5001
[  3]  0.0- 4.0 sec    256 MBytes    538 Mbits/sec
[  3]  4.0- 8.0 sec    444 MBytes    932 Mbits/sec
[  3]  8.0-12.0 sec    448 MBytes    940 Mbits/sec
[  3] 12.0-16.0 sec    449 MBytes    941 Mbits/sec
[  3] 16.0-20.0 sec    448 MBytes    940 Mbits/sec
[  3] 20.0-24.0 sec    450 MBytes    944 Mbits/sec
[  3] 24.0-28.0 sec    449 MBytes    943 Mbits/sec
[  3] 28.0-32.0 sec    448 MBytes    940 Mbits/sec
[  3] 32.0-36.0 sec    448 MBytes    940 Mbits/sec
[  3] 36.0-40.0 sec    449 MBytes    941 Mbits/sec
[  3] 40.0-44.0 sec    449 MBytes    942 Mbits/sec
[  3] 44.0-48.0 sec    451 MBytes    945 Mbits/sec
[  3] 48.0-52.0 sec    448 MBytes    940 Mbits/sec
[  3] 52.0-56.0 sec    449 MBytes    941 Mbits/sec
[  3] 56.0-60.0 sec    448 MBytes    940 Mbits/sec
[  3]  0.0-60.0 sec  6.38 GBytes    914 Mbits/sec

Test Between BNL and Indiana University


[root@dct00 ~]# tracepath  149.165.225.214
 1:  dct00.usatlas.bnl.gov (192.12.15.8)                    0.181ms pmtu 1500
 1:  hsrp.usatlas.bnl.gov (192.12.15.24)                    0.444ms 
 2:  bnlmr1-bnlsite.es.net (198.124.216.177)                0.418ms 
 3:  aoacr1-bnlmr1.es.net (134.55.217.58)                 asymm  4   2.316ms 
 4:  chiccr1-aofacr1.es.net (134.55.218.93)               asymm  5  29.279ms 
 5:  chislsdn1-chicr1.es.net (134.55.207.34)              asymm  6  29.285ms 
 6:  chislmr1-chislsdn1.es.net (134.55.219.26)             29.407ms 
 7:  star-esnet.abilene.iu.edu (198.125.140.230)          asymm  8  29.660ms 
 8:  ge-0-0-0.2063.rtr.ictc.indiana.gigapop.net (149.165.254.2) asymm  9  34.156ms 
 9:  149.165.254.250 (149.165.254.250)                     45.525ms 
10:  no reply
11:  no reply
12:  no reply
13:  no reply
14:  no reply
15:  no reply
16:  no reply
17:  no reply
18:  no reply



[  4] local 192.12.15.8 port 5001 connected with 149.165.225.214 port 42583
[  4]  0.0- 4.0 sec  22.5 MBytes  47.3 Mbits/sec
[  4]  4.0- 8.0 sec  23.3 MBytes  48.8 Mbits/sec
[  4]  8.0-12.0 sec  23.6 MBytes  49.4 Mbits/sec
[  4] 12.0-16.0 sec  23.3 MBytes  48.8 Mbits/sec
[  4] 16.0-20.0 sec  23.4 MBytes  49.0 Mbits/sec
[  4] 20.0-24.0 sec  23.1 MBytes  48.4 Mbits/sec
[  4] 24.0-28.0 sec  23.4 MBytes  49.2 Mbits/sec
[  4] 28.0-32.0 sec  23.3 MBytes  49.0 Mbits/sec
[  4] 32.0-36.0 sec  23.1 MBytes  48.5 Mbits/sec
[  4] 36.0-40.0 sec  23.4 MBytes  49.0 Mbits/sec
[  4] 40.0-44.0 sec  23.2 MBytes  48.7 Mbits/sec
[  4] 44.0-48.0 sec  23.4 MBytes  49.2 Mbits/sec
[  4] 48.0-52.0 sec  23.3 MBytes  49.0 Mbits/sec
[  4] 52.0-56.0 sec  23.4 MBytes  49.1 Mbits/sec
[  4] 56.0-60.0 sec  23.0 MBytes  48.2 Mbits/sec
[  4]  0.0-60.0 sec    349 MBytes  48.8 Mbits/sec

After IU made network performance, we saw a significant performance change.

[root@dct00 ~]#  iperf -s -w 4MB -i 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  4] local 192.12.15.8 port 5001 connected with 149.165.225.214 port 42619
[  4]  0.0- 4.0 sec    330 MBytes    691 Mbits/sec
[  4]  4.0- 8.0 sec    151 MBytes    316 Mbits/sec
[  4]  8.0-12.0 sec    379 MBytes    795 Mbits/sec
[  4] 12.0-16.0 sec    397 MBytes    833 Mbits/sec
[  4] 16.0-20.0 sec    298 MBytes    624 Mbits/sec
[  4] 20.0-24.0 sec    318 MBytes    668 Mbits/sec
[  4] 24.0-28.0 sec    379 MBytes    794 Mbits/sec
[  4] 28.0-32.0 sec    408 MBytes    856 Mbits/sec
[  4] 32.0-36.0 sec    372 MBytes    780 Mbits/sec
[  4] 36.0-40.0 sec    292 MBytes    612 Mbits/sec
[  4] 40.0-44.0 sec    382 MBytes    800 Mbits/sec
[  4] 44.0-48.0 sec    408 MBytes    855 Mbits/sec
[  4] 48.0-52.0 sec    267 MBytes    559 Mbits/sec
[  4] 52.0-56.0 sec    380 MBytes    796 Mbits/sec
[  4] 56.0-60.0 sec    397 MBytes    833 Mbits/sec
[  4]  0.0-60.0 sec  5.04 GBytes    721 Mbits/sec

We send data from BNL to IU. It stabilizes at above 900mbs.

[root@dct00 networktest]# iperf -c   iut2-dc3.iu.edu  -w 4MB -t 60 -i 4
------------------------------------------------------------
Client connecting to iut2-dc3.iu.edu, TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 192.12.15.8 port 40402 connected with 149.165.225.214 port 5001
[  3]  0.0- 4.0 sec    143 MBytes    300 Mbits/sec
[  3]  4.0- 8.0 sec    276 MBytes    579 Mbits/sec
[  3]  8.0-12.0 sec    433 MBytes    908 Mbits/sec
[  3] 12.0-16.0 sec    431 MBytes    904 Mbits/sec
[  3] 16.0-20.0 sec    449 MBytes    941 Mbits/sec
[  3] 20.0-24.0 sec    448 MBytes    940 Mbits/sec
[  3] 24.0-28.0 sec    448 MBytes    940 Mbits/sec
[  3] 28.0-32.0 sec    450 MBytes    944 Mbits/sec
[  3] 32.0-36.0 sec    449 MBytes    942 Mbits/sec
[  3] 36.0-40.0 sec    448 MBytes    940 Mbits/sec
[  3] 40.0-44.0 sec    448 MBytes    940 Mbits/sec
[  3] 44.0-48.0 sec    448 MBytes    940 Mbits/sec
[  3] 48.0-52.0 sec    450 MBytes    944 Mbits/sec
[  3] 52.0-56.0 sec    449 MBytes    942 Mbits/sec
[  3] 56.0-60.0 sec    448 MBytes    939 Mbits/sec
[  3]  0.0-60.0 sec  6.07 GBytes    870 Mbits/sec
[root@dct00 networktest]#

-- JohnDeStefano - 25 Sep 2007: copied from topic "BNL".

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.

Attachments

Admins

USATLAS Site Admin Topics
Admin Support Home
Site Setup Help
Search

Facility Integration Program
Integration home
Proof-Xrootd

Site Admin External Links
dCache
GUMS

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback