Home Using iperf to measure the network bandwidth
Post
Cancel

Using iperf to measure the network bandwidth

iPerf is a tool for measurements of the maximum achievable bandwidth on IP network. The following iperf option can be used on the client side to saturate the network bandwidth if single client thread is not sufficient.

  • -P, –parallel # number of parallel client threads to run

Benchmark a 100GbE network

  1. Single thread test

    Start iperf on the server side:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    
     $ iperf -v
     iperf version 2.0.13 (21 Jan 2019) pthreads
    
     $ iperf -s
     ------------------------------------------------------------
     Server listening on TCP port 5001
     TCP window size:  128 KByte (default)
     ------------------------------------------------------------
     [  4] local 192.168.1.240 port 5001 connected with 192.168.1.245 port 44292
     [ ID] Interval       Transfer     Bandwidth
     [  4]  0.0-10.0 sec  37.7 GBytes  32.4 Gbits/sec
    

    Run iperf benchmark on the client side:

    1
    2
    3
    4
    5
    6
    7
    8
    
     $ iperf -c 192.168.1.240
     ------------------------------------------------------------
     Client connecting to 192.168.1.240, TCP port 5001
     TCP window size: 2.44 MByte (default)
     ------------------------------------------------------------
     [  3] local 192.168.1.245 port 44292 connected with 192.168.1.240 port 5001
     [ ID] Interval       Transfer     Bandwidth
     [  3]  0.0-10.0 sec  37.7 GBytes  32.4 Gbits/sec
    

    With the default single thread, the achievable bandwidth is 32.4Gb/s which is far less than the available 100Gb/s bandwidth.

  2. Multi-threads test

    We can increase the number of iperf client threads until the maximum bandwidth is hit.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    
     $ iperf -c 192.168.1.240 -P 1
     ------------------------------------------------------------
     Client connecting to 192.168.1.240, TCP port 5001
     TCP window size: 4.00 MByte (default)
     ------------------------------------------------------------
     [  3] local 192.168.1.245 port 44436 connected with 192.168.1.240 port 5001
     [ ID] Interval       Transfer     Bandwidth
     [  3]  0.0-10.0 sec  38.8 GBytes  33.4 Gbits/sec
    
     $ iperf -c 192.168.1.240 -P 2
     ------------------------------------------------------------
     Client connecting to 192.168.1.240, TCP port 5001
     TCP window size: 1.13 MByte (default)
     ------------------------------------------------------------
     [  4] local 192.168.1.245 port 44440 connected with 192.168.1.240 port 5001
     [  3] local 192.168.1.245 port 44438 connected with 192.168.1.240 port 5001
     [ ID] Interval       Transfer     Bandwidth
     [  4]  0.0-10.0 sec  35.3 GBytes  30.4 Gbits/sec
     [  3]  0.0-10.0 sec  37.6 GBytes  32.3 Gbits/sec
     [SUM]  0.0-10.0 sec  72.9 GBytes  62.6 Gbits/sec
    
     $ iperf -c 192.168.1.240 -P 4
     ------------------------------------------------------------
     Client connecting to 192.168.1.240, TCP port 5001
     TCP window size:  366 KByte (default)
     ------------------------------------------------------------
     [  6] local 192.168.1.245 port 44448 connected with 192.168.1.240 port 5001
     [  5] local 192.168.1.245 port 44446 connected with 192.168.1.240 port 5001
     [  4] local 192.168.1.245 port 44444 connected with 192.168.1.240 port 5001
     [  3] local 192.168.1.245 port 44442 connected with 192.168.1.240 port 5001
     [ ID] Interval       Transfer     Bandwidth
     [  6]  0.0-10.0 sec  26.0 GBytes  22.3 Gbits/sec
     [  5]  0.0-10.0 sec  26.3 GBytes  22.6 Gbits/sec
     [  4]  0.0-10.0 sec  26.3 GBytes  22.6 Gbits/sec
     [  3]  0.0-10.0 sec  26.4 GBytes  22.6 Gbits/sec
     [SUM]  0.0-10.0 sec   105 GBytes  90.1 Gbits/sec
    
     $ iperf -c 192.168.1.240 -P 8
     ------------------------------------------------------------
     Client connecting to 192.168.1.240, TCP port 5001
     TCP window size:  518 KByte (default)
     ------------------------------------------------------------
     [ 16] local 192.168.1.245 port 44464 connected with 192.168.1.240 port 5001
     [  3] local 192.168.1.245 port 44452 connected with 192.168.1.240 port 5001
     [  7] local 192.168.1.245 port 44456 connected with 192.168.1.240 port 5001
     [  4] local 192.168.1.245 port 44450 connected with 192.168.1.240 port 5001
     [  6] local 192.168.1.245 port 44454 connected with 192.168.1.240 port 5001
     [  9] local 192.168.1.245 port 44462 connected with 192.168.1.240 port 5001
     [  8] local 192.168.1.245 port 44460 connected with 192.168.1.240 port 5001
     [  5] local 192.168.1.245 port 44458 connected with 192.168.1.240 port 5001
     [ ID] Interval       Transfer     Bandwidth
     [ 16]  0.0-10.0 sec  15.0 GBytes  12.9 Gbits/sec
     [  3]  0.0-10.0 sec  9.06 GBytes  7.78 Gbits/sec
     [  7]  0.0-10.0 sec  15.1 GBytes  12.9 Gbits/sec
     [  4]  0.0-10.0 sec  15.1 GBytes  13.0 Gbits/sec
     [  6]  0.0-10.0 sec  15.1 GBytes  12.9 Gbits/sec
     [  9]  0.0-10.0 sec  15.1 GBytes  12.9 Gbits/sec
     [  8]  0.0-10.0 sec  8.97 GBytes  7.70 Gbits/sec
     [  5]  0.0-10.0 sec  15.1 GBytes  12.9 Gbits/sec
     [SUM]  0.0-10.0 sec   108 GBytes  93.1 Gbits/sec
    

    We can verify the network throughput with the command sar -n DEV 2.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    
     03:26:05 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
     03:26:07 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00
     03:26:07 PM      eth0     15.00      0.50      0.91      0.33      0.00      0.00      0.00
     03:26:07 PM      eth3      0.00      0.00      0.00      0.00      0.00      0.00      0.00
     03:26:07 PM      eth1 8045242.00 252491.50 11895015.32  16273.87      0.00      0.00      0.00
     03:26:07 PM      eth2      0.00      0.00      0.00      0.00      0.00      0.00      0.00
    
     03:26:07 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
     03:26:10 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00
     03:26:10 PM      eth0     10.50      1.00      0.65      0.41      0.00      0.00      0.00
     03:26:10 PM      eth3      0.00      0.00      0.00      0.00      0.00      0.00      0.00
     03:26:10 PM      eth1 8046641.50 251968.50 11897084.51  16240.16      0.00      0.00      0.00
     03:26:10 PM      eth2      0.00      0.00      0.00      0.00      0.00      0.00      0.00
    
This post is licensed under CC BY 4.0 by the author.

How to deploy MySQL and phpMyAdmin with Docker

Access a sharedv4 volume outside of a Portwrox cluster

Comments powered by Disqus.