BNL-OUNetworkTest

Test Between BNL and University of Chicago.
- Tuning on 21 Sep 2007
  - Network test between BNL and University of Chicago.

Test Between BNL and University of Chicago.

Tuning on 21 Sep 2007

Network test between BNL and University of Chicago.


[root@dct00 ~]# ping tier2-02.ochep.ou.edu
PING tier2-02.ochep.ou.edu (129.15.40.251) 56(84) bytes of data.
64 bytes from tier2-02.ochep.ou.edu (129.15.40.251): icmp_seq=0 ttl=52 time=57.5 ms
64 bytes from tier2-02.ochep.ou.edu (129.15.40.251): icmp_seq=1 ttl=52 time=52.5 ms
64 bytes from tier2-02.ochep.ou.edu (129.15.40.251): icmp_seq=2 ttl=52 time=55.2 ms
64 bytes from tier2-02.ochep.ou.edu (129.15.40.251): icmp_seq=3 ttl=52 time=56.9 ms
64 bytes from tier2-02.ochep.ou.edu (129.15.40.251): icmp_seq=4 ttl=52 time=52.4 ms

[root@dct00 ~]# tracepath tier2-02.ochep.ou.edu
 1:  dct00.usatlas.bnl.gov (192.12.15.8)                    0.164ms pmtu 1500
 1:  hsrp.usatlas.bnl.gov (192.12.15.24)                    2.060ms 
 2:  bnlmr1-bnlsite.es.net (198.124.216.177)                2.072ms 
 3:  aoacr1-bnlmr1.es.net (134.55.217.58)                 asymm  4   4.111ms 
 4:  chiccr1-aofacr1.es.net (134.55.218.93)               asymm  5  30.003ms 
 5:  chislsdn1-chicr1.es.net (134.55.207.34)              asymm  6  31.105ms 
 6:  chislmr1-chislsdn1.es.net (134.55.219.81)             29.894ms 
 7:  star-esnet.abilene.iu.edu (198.125.140.230)          asymm  8 296.813ms 
 8:  so-4-3-0.0.rtr.kans.net.internet2.edu (64.57.28.36)  asymm  9  41.959ms 
 9:  ks-42-tenge4-1-110.r.greatplains.net (164.113.238.194)  40.699ms 
10:  164.113.238.210 (164.113.238.210)                    asymm  8  45.146ms 
11:  164.58.245.129 (164.58.245.129)                      asymm  9  45.150ms 
12:  164.58.244.173 (164.58.244.173)                      asymm 10  50.142ms 
13:  164.58.244.165 (164.58.244.165)                      asymm 10  51.622ms 
14:  164.58.244.166 (164.58.244.166)                      asymm 10  52.512ms 
15:  164.58.10.98 (164.58.10.98)                          asymm 11  52.654ms 
16:  no reply
17:  tier2-02.ochep.ou.edu (129.15.40.251)                asymm 13  54.048ms reached
     Resume: pmtu 1500 hops 17 back 13

[root@dct00 ~]# iperf -s -w 4MB -i 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  4] local 192.12.15.8 port 5001 connected with 129.15.40.251 port 40103
[  4]  0.0- 4.0 sec  36.2 MBytes  75.9 Mbits/sec
[  4]  4.0- 8.0 sec  53.5 MBytes    112 Mbits/sec
[  4]  8.0-12.0 sec  31.0 MBytes  65.0 Mbits/sec
[  4] 12.0-16.0 sec  38.3 MBytes  80.4 Mbits/sec
[  4] 16.0-20.0 sec  39.6 MBytes  83.0 Mbits/sec
[  4] 20.0-24.0 sec  33.1 MBytes  69.3 Mbits/sec
[  4] 24.0-28.0 sec  39.7 MBytes  83.3 Mbits/sec
[  4] 28.0-32.0 sec  40.3 MBytes  84.5 Mbits/sec
[  4] 32.0-36.0 sec  53.6 MBytes    112 Mbits/sec
[  4] 36.0-40.0 sec  50.5 MBytes    106 Mbits/sec
[  4] 40.0-44.0 sec  41.2 MBytes  86.5 Mbits/sec
[  4] 44.0-48.0 sec  48.8 MBytes    102 Mbits/sec
[  4] 48.0-52.0 sec  45.8 MBytes  95.9 Mbits/sec
[  4] 52.0-56.0 sec  48.5 MBytes    102 Mbits/sec
[  4] 56.0-60.0 sec  45.7 MBytes  95.9 Mbits/sec
[  4]  0.0-60.1 sec    646 MBytes  90.3 Mbits/sec

[root@dct00 ~]# iperf -c tier2-02.ochep.ou.edu -w 4MB -t 60 -i 4 -p 64000
------------------------------------------------------------
Client connecting to tier2-02.ochep.ou.edu, TCP port 64000
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 192.12.15.8 port 58818 connected with 129.15.40.251 port 64000
[  3]  0.0- 4.0 sec  18.7 MBytes  39.3 Mbits/sec
[  3]  4.0- 8.0 sec  15.9 MBytes  33.4 Mbits/sec
[  3]  8.0-12.0 sec  56.4 MBytes    118 Mbits/sec
[  3] 12.0-16.0 sec  58.5 MBytes    123 Mbits/sec
[  3] 16.0-20.0 sec  48.4 MBytes    102 Mbits/sec
[  3] 20.0-24.0 sec  44.1 MBytes  92.4 Mbits/sec
[  3] 24.0-28.0 sec  50.4 MBytes    106 Mbits/sec
[  3] 28.0-32.0 sec  52.3 MBytes    110 Mbits/sec
[  3] 32.0-36.0 sec  38.3 MBytes  80.3 Mbits/sec
[  3] 36.0-40.0 sec  6.03 MBytes  12.6 Mbits/sec
[  3] 40.0-44.0 sec  24.2 MBytes  50.7 Mbits/sec
[  3] 44.0-48.0 sec  58.3 MBytes    122 Mbits/sec
[  3] 48.0-52.0 sec  56.6 MBytes    119 Mbits/sec
[  3] 52.0-56.0 sec  50.6 MBytes    106 Mbits/sec
[  3] 56.0-60.0 sec  58.3 MBytes    122 Mbits/sec
[  3]  0.0-60.1 sec    637 MBytes  88.9 Mbits/sec

[root@dct00 ~]# iperf -s -w 4MB -i 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  4] local 192.12.15.8 port 5001 connected with 129.15.40.251 port 40462
[  4]  0.0- 4.0 sec    103 MBytes    215 Mbits/sec
[  4]  4.0- 8.0 sec    128 MBytes    269 Mbits/sec
[  4]  8.0-12.0 sec    129 MBytes    271 Mbits/sec
[  4] 12.0-16.0 sec    119 MBytes    250 Mbits/sec
[  4] 16.0-20.0 sec    113 MBytes    237 Mbits/sec
[  4] 20.0-24.0 sec  94.0 MBytes    197 Mbits/sec
[  4] 24.0-28.0 sec  91.4 MBytes    192 Mbits/sec
[  4] 28.0-32.0 sec  86.2 MBytes    181 Mbits/sec
[  4] 32.0-36.0 sec  87.9 MBytes    184 Mbits/sec
[  4] 36.0-40.0 sec  27.5 MBytes  57.6 Mbits/sec
[  4] 40.0-44.0 sec  20.4 MBytes  42.9 Mbits/sec
[  4] 44.0-48.0 sec  23.4 MBytes  49.2 Mbits/sec
[  4] 48.0-52.0 sec  59.8 MBytes    125 Mbits/sec
[  4] 52.0-56.0 sec    100 MBytes    210 Mbits/sec
[  4] 56.0-60.0 sec  83.1 MBytes    174 Mbits/sec
[  4]  0.0-60.2 sec  1.24 GBytes    177 Mbits/sec

After we tuned the network, the performance does not have too much improvement.

[root@dct00 ~]# iperf -s -w 16MB -i 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 32.0 MByte (WARNING: requested 16.0 MByte)
------------------------------------------------------------
[  4] local 192.12.15.8 port 5001 connected with 129.15.40.247 port 43121
[  4]  0.0- 4.0 sec  5.12 MBytes  10.7 Mbits/sec
[  4]  4.0- 8.0 sec  51.7 MBytes    109 Mbits/sec
[  4]  8.0-12.0 sec  70.3 MBytes    147 Mbits/sec
[  4] 12.0-16.0 sec  69.9 MBytes    147 Mbits/sec
[  4] 16.0-20.0 sec  61.6 MBytes    129 Mbits/sec
[  4] 20.0-24.0 sec  63.3 MBytes    133 Mbits/sec
[  4] 24.0-28.0 sec  70.7 MBytes    148 Mbits/sec
[  4] 28.0-32.0 sec    112 MBytes    236 Mbits/sec
[  4] 32.0-36.0 sec    110 MBytes    230 Mbits/sec
[  4] 36.0-40.0 sec  61.0 MBytes    128 Mbits/sec
[  4] 40.0-44.0 sec  71.7 MBytes    150 Mbits/sec
[  4] 44.0-48.0 sec  60.6 MBytes    127 Mbits/sec
[  4] 48.0-52.0 sec  46.8 MBytes  98.2 Mbits/sec
[  4] 52.0-56.0 sec  50.1 MBytes    105 Mbits/sec
[  4] 56.0-60.0 sec  56.2 MBytes    118 Mbits/sec
[  4]  0.0-60.5 sec    971 MBytes    135 Mbits/sec

We tested the network performance from BNL to OU:

[root@dct00 ~]# iperf -c  129.15.40.247  -w 16MB -i 4 -t 60 -p 64000
------------------------------------------------------------
Client connecting to 129.15.40.247, TCP port 64000
TCP window size: 32.0 MByte (WARNING: requested 16.0 MByte)
------------------------------------------------------------
[  3] local 192.12.15.8 port 58821 connected with 129.15.40.247 port 64000
[  3]  0.0- 4.0 sec  31.4 MBytes  65.9 Mbits/sec
[  3]  4.0- 8.0 sec  0.00 Bytes  0.00 bits/sec
[  3]  8.0-12.0 sec  0.00 Bytes  0.00 bits/sec
[  3] 12.0-16.0 sec  56.5 MBytes    119 Mbits/sec
[  3] 16.0-20.0 sec  90.8 MBytes    191 Mbits/sec
[  3] 20.0-24.0 sec  82.3 MBytes    173 Mbits/sec
[  3] 24.0-28.0 sec  89.5 MBytes    188 Mbits/sec
[  3] 28.0-32.0 sec  98.7 MBytes    207 Mbits/sec
[  3] 32.0-36.0 sec  90.0 MBytes    189 Mbits/sec
[  3] 36.0-40.0 sec  40.2 MBytes  84.2 Mbits/sec
[  3] 40.0-44.0 sec  40.2 MBytes  84.3 Mbits/sec
[  3] 44.0-48.0 sec    135 MBytes    282 Mbits/sec
[  3] 48.0-52.0 sec    126 MBytes    265 Mbits/sec
[  3] 52.0-56.0 sec    143 MBytes    299 Mbits/sec
[  3] 56.0-60.0 sec    118 MBytes    248 Mbits/sec
[  3]  0.0-60.2 sec  1.11 GBytes    159 Mbits/sec

Then we increased the number of TCP streams to see whether the performance increased: We first tried five TCP streams, and then we tried 10 TCP streams.

[root@dct00 ~]# iperf -s -w 2MB -i 4 
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 4.00 MByte (WARNING: requested 2.00 MByte)
------------------------------------------------------------
 [root@dct00 ~]# iperf -s -w 2MB -i 4  | grep -i SUM
[SUM]  0.0- 4.0 sec    184 MBytes    387 Mbits/sec
[SUM]  4.0- 8.0 sec    228 MBytes    477 Mbits/sec
[SUM]  8.0-12.0 sec    223 MBytes    468 Mbits/sec
[SUM] 12.0-16.0 sec    297 MBytes    623 Mbits/sec
[SUM] 16.0-20.0 sec    359 MBytes    753 Mbits/sec
[SUM] 20.0-24.0 sec    334 MBytes    699 Mbits/sec
[SUM] 24.0-28.0 sec    286 MBytes    599 Mbits/sec
[SUM] 28.0-32.0 sec    329 MBytes    691 Mbits/sec
[SUM] 32.0-36.0 sec    340 MBytes    714 Mbits/sec
[SUM] 36.0-40.0 sec    429 MBytes    899 Mbits/sec
[SUM] 40.0-44.0 sec    325 MBytes    681 Mbits/sec
[SUM] 44.0-48.0 sec    376 MBytes    789 Mbits/sec
[SUM] 48.0-52.0 sec    361 MBytes    756 Mbits/sec
[SUM] 52.0-56.0 sec    420 MBytes    880 Mbits/sec
[SUM] 56.0-60.0 sec    364 MBytes    763 Mbits/sec
[SUM]  0.0-60.4 sec  4.77 GBytes    678 Mbits/sec

Then we used 10 TCP streams. The performance reached the wire speed of local network card.

[root@dct00 ~]# iperf -s -w 2MB -i 4  | grep -i SUM
[SUM]  0.0- 4.0 sec    322 MBytes    676 Mbits/sec
[SUM]  4.0- 8.0 sec    334 MBytes    701 Mbits/sec
[SUM]  8.0-12.0 sec    444 MBytes    932 Mbits/sec
[SUM] 12.0-16.0 sec    427 MBytes    894 Mbits/sec
[SUM] 16.0-20.0 sec    346 MBytes    725 Mbits/sec
[SUM] 20.0-24.0 sec    351 MBytes    736 Mbits/sec
[SUM] 24.0-28.0 sec    428 MBytes    897 Mbits/sec
[SUM] 28.0-32.0 sec    441 MBytes    925 Mbits/sec
[SUM] 32.0-36.0 sec    428 MBytes    897 Mbits/sec
[SUM] 36.0-40.0 sec    426 MBytes    893 Mbits/sec
[SUM] 40.0-44.0 sec    446 MBytes    936 Mbits/sec
[SUM] 44.0-48.0 sec    406 MBytes    851 Mbits/sec
[SUM] 48.0-52.0 sec    437 MBytes    917 Mbits/sec
[SUM] 52.0-56.0 sec    409 MBytes    857 Mbits/sec
[SUM] 56.0-60.0 sec    444 MBytes    932 Mbits/sec
[SUM]  0.0-60.9 sec  6.02 GBytes    848 Mbits/sec

Then we tested the network performance from BNL to OU. We started with five TCP streams. Here are the results with five TCP streams.

[root@dct00 ~]# iperf -c  129.15.40.247  -w 2MB -i 4 -t 60 -p 64000 -P 5 | grep -i SUM
[SUM]  0.0- 4.0 sec    142 MBytes    298 Mbits/sec
[SUM]  4.0- 8.0 sec    147 MBytes    308 Mbits/sec
[SUM]  8.0-12.0 sec    167 MBytes    351 Mbits/sec
[SUM] 12.0-16.0 sec    169 MBytes    353 Mbits/sec
[SUM] 16.0-20.0 sec    108 MBytes    227 Mbits/sec
[SUM] 20.0-24.0 sec    167 MBytes    350 Mbits/sec
[SUM] 24.0-28.0 sec    193 MBytes    404 Mbits/sec
[SUM] 28.0-32.0 sec    176 MBytes    368 Mbits/sec
[SUM] 32.0-36.0 sec    176 MBytes    370 Mbits/sec
[SUM] 36.0-40.0 sec    184 MBytes    387 Mbits/sec
[SUM] 40.0-44.0 sec    180 MBytes    378 Mbits/sec
[SUM] 44.0-48.0 sec    168 MBytes    352 Mbits/sec
[SUM] 48.0-52.0 sec    175 MBytes    367 Mbits/sec
[SUM] 52.0-56.0 sec    176 MBytes    369 Mbits/sec
[SUM]  0.0-60.2 sec  2.45 GBytes    349 Mbits/sec
[root@dct00 ~]#

Then we tested with 10 TCP streams. We did not see any performance improvement.

[root@dct00 ~]# iperf -c  129.15.40.247  -w 2MB -i 4 -t 60 -p 64000 -P 10 | grep -i SUM
[SUM]  0.0- 4.0 sec    172 MBytes    360 Mbits/sec
[SUM]  4.0- 8.0 sec    170 MBytes    358 Mbits/sec
[SUM]  8.0-12.0 sec    146 MBytes    307 Mbits/sec
[SUM] 12.0-16.0 sec  66.9 MBytes    140 Mbits/sec
[SUM] 16.0-20.0 sec    130 MBytes    273 Mbits/sec
[SUM] 20.0-24.0 sec    133 MBytes    279 Mbits/sec
[SUM] 24.0-28.0 sec    172 MBytes    361 Mbits/sec
[SUM] 28.0-32.0 sec    177 MBytes    370 Mbits/sec
[SUM] 32.0-36.0 sec    213 MBytes    446 Mbits/sec
[SUM] 36.0-40.0 sec    204 MBytes    427 Mbits/sec
[SUM] 40.0-44.0 sec    174 MBytes    366 Mbits/sec
[SUM] 44.0-48.0 sec    172 MBytes    361 Mbits/sec
[SUM] 48.0-52.0 sec    174 MBytes    364 Mbits/sec
[SUM] 52.0-56.0 sec    195 MBytes    410 Mbits/sec
[SUM] 56.0-60.0 sec    186 MBytes    390 Mbits/sec
[SUM]  0.0-60.5 sec  2.43 GBytes    344 Mbits/sec
[root@dct00 ~]#

[root@dct00 ~]# iperf -c  129.15.40.247  -w 2MB -i 4 -t 60 -p 64000 -P 20 | grep -i SUM
[SUM]  0.0- 4.0 sec    247 MBytes    519 Mbits/sec
[SUM]  4.0- 8.0 sec    205 MBytes    430 Mbits/sec
[SUM]  8.0-12.0 sec    191 MBytes    401 Mbits/sec
[SUM] 12.0-16.0 sec    171 MBytes    359 Mbits/sec
[SUM] 16.0-20.0 sec    161 MBytes    337 Mbits/sec
[SUM] 20.0-24.0 sec    186 MBytes    391 Mbits/sec
[SUM] 24.0-28.0 sec    223 MBytes    468 Mbits/sec
[SUM] 28.0-32.0 sec    216 MBytes    454 Mbits/sec
[SUM] 32.0-36.0 sec    174 MBytes    365 Mbits/sec
[SUM] 36.0-40.0 sec    212 MBytes    445 Mbits/sec
[SUM] 40.0-44.0 sec    221 MBytes    464 Mbits/sec
[SUM] 44.0-48.0 sec    224 MBytes    469 Mbits/sec
[SUM] 48.0-52.0 sec    141 MBytes    295 Mbits/sec
[SUM] 52.0-56.0 sec    151 MBytes    317 Mbits/sec
[SUM]  0.0-60.7 sec  2.82 GBytes    399 Mbits/sec

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.

Attachments

Admins

USATLAS Site Admin Topics
Admin Support Home
Site Setup Help
Search

Facility Integration Program
Integration home
Proof-Xrootd

Site Admin External Links
dCache
GUMS

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback