Iperf 2 / Discussion / General Discussion: Multicast test performance

I have a machine that can drive 50Gbps on the NIC and 10Mpps with a 5Gbps limit per flow. When I try to stress test it with multicast I can't. I performed the following tests.

For 200, 250, 300, 500 and 700 Bytes Unicast I am CPU bound and understandably cannot drive the 5Gbps per flow, on 1000, 1500 and 8000bytes I can achieve the 5Gbps.
However with Multicast I cannot get close to the unicast numbers and CPU does not hit 100%. Can someone explain this behavior?

###
### 200 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 200 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42869 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 200 byte datagrams, IPG target: 0.32 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 50362 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:44:08.583 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  2.30 GBytes   988 Mbits/sec  12352412/0   617621 pps
[  1] Sent 12352413 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
WARN: Reporter thread may be too slow, await counter=1235, consider increasing NUM_REPORT_STRUCTS
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 200 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42877 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 200 byte datagrams, IPG target: 0.32 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 47497 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:45:06.715 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  1.19 GBytes   509 Mbits/sec  6363107/0   318153 pps
[  1] Sent 6363109 datagrams
###
### 250 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 250 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42882 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 250 byte datagrams, IPG target: 0.40 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 38572 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:45:59.982 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  2.88 GBytes  1.24 Gbits/sec  12354408/0   617720 pps
[  1] Sent 12354409 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
WARN: Reporter thread may be too slow, await counter=1235, consider increasing NUM_REPORT_STRUCTS
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 250 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42886 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 250 byte datagrams, IPG target: 0.40 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 58790 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:46:51.854 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  1.44 GBytes   620 Mbits/sec  6204731/0   310237 pps
[  1] Sent 6204733 datagrams
###
### 300 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 300 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42893 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 300 byte datagrams, IPG target: 0.48 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 42124 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:48:27.434 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  3.45 GBytes  1.48 Gbits/sec  12351747/0   617587 pps
[  1] Sent 12351748 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
WARN: Reporter thread may be too slow, await counter=1235, consider increasing NUM_REPORT_STRUCTS
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 300 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42897 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 300 byte datagrams, IPG target: 0.48 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 39680 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:49:20.606 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  1.79 GBytes   768 Mbits/sec  6403652/0   320180 pps
[  1] Sent 6403654 datagrams
###
### 500 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 500 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42904 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 500 byte datagrams, IPG target: 0.80 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 44167 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:50:41.484 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  5.75 GBytes  2.47 Gbits/sec  12351860/0   617593 pps
[  1] Sent 12351861 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
WARN: Reporter thread may be too slow, await counter=1235, consider increasing NUM_REPORT_STRUCTS
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 500 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42908 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 500 byte datagrams, IPG target: 0.80 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 38793 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:51:27.444 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  2.94 GBytes  1.26 Gbits/sec  6319679/0   315982 pps
[  1] Sent 6319681 datagrams
###
### 700 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 700 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42912 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 700 byte datagrams, IPG target: 1.12 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 36802 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:52:17.429 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  8.06 GBytes  3.46 Gbits/sec  12357590/0   617879 pps
[  1] Sent 12357591 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
WARN: Reporter thread may be too slow, await counter=1235, consider increasing NUM_REPORT_STRUCTS
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 700 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42916 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 700 byte datagrams, IPG target: 1.12 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 40059 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:52:54.095 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  4.13 GBytes  1.77 Gbits/sec  6328553/0   316426 pps
[  1] Sent 6328555 datagrams
###
### 1000 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 1000 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42919 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 1000 byte datagrams, IPG target: 1.60 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 57456 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:53:41.781 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  11.2 GBytes  4.80 Gbits/sec  11989499/0   599475 pps
[  1] Sent 11989500 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 1000 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42923 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 1000 byte datagrams, IPG target: 1.60 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 48536 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:54:21.305 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  5.87 GBytes  2.52 Gbits/sec  6302025/0   315100 pps
[  1] Sent 6302027 datagrams
###
### 1500 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 1500 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42931 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 1500 byte datagrams, IPG target: 2.40 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 51157 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:55:08.733 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  11.3 GBytes  4.87 Gbits/sec  8108660/0   405431 pps
[  1] Sent 8108662 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 1500 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42935 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 1500 byte datagrams, IPG target: 2.40 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 58491 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:55:49.700 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  8.62 GBytes  3.70 Gbits/sec  6169914/0   308495 pps
[  1] Sent 6169916 datagrams
###
### 8000 Bytes
###
root@ip-10-0-164-135:~# iperf -c 10.0.165.218 -u -e -b 5g -l 8000 -t 20 -T 2
------------------------------------------------------------
Client connecting to 10.0.165.218, UDP port 5001 with pid 42940 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 8000 byte datagrams, IPG target: 12.80 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 38545 connected with 10.0.165.218 port 5001 (sock=3) on 2023-03-15 23:56:52.792 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  11.6 GBytes  4.98 Gbits/sec  1554775/0    77739 pps
[  1] Sent 1554777 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.
################################
root@ip-10-0-164-135:~# iperf -c 224.1.2.1 -u -e -b 5g -l 8000 -t 20 -T 2
------------------------------------------------------------
Client connecting to 224.1.2.1, UDP port 5001 with pid 42944 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 8000 byte datagrams, IPG target: 12.80 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 10.0.164.135%ens5 port 52781 connected with 224.1.2.1 port 5001 (sock=3) on 2023-03-15 23:57:30.862 (UTC)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-20.00 sec  11.6 GBytes  4.98 Gbits/sec  1554774/0    77739 pps
[  1] Sent 1554776 datagrams

Robert McMahon - 2023-03-16

I don't see the multicast output. Do you have server side?

Also, it might be good to increase the number of traffic threads via -P to match the core count of the machine this is running on.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Diego Soares - 2023-03-17
  
  Hi Robert,
  
  On the Multicast side there is no ACK from the server, correct?
  The Client is just pushing the traffic into the network and the server does
  not matter. If this is not the case, how multiple servers (receivers) are
  handled?
  And for -P doesn't help because I the unicast scenario since the server
  becomes the bottleneck and -P is not supported for MCAST.
  
  On Thu, Mar 16, 2023 at 7:41 PM Robert McMahon rjmcmahon@users.sourceforge.net wrote:
  
  I don't see the multicast output. Do you have server side?
  
  Also, it might be good to increase the number of traffic threads via -P to
  match the core count of the machine this is running on.
  
  Multicast test performance
  https://sourceforge.net/p/iperf2/discussion/general/thread/d3357de162/?limit=25#51e5
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/iperf2/discussion/general/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  alternate
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Diego Soares - 2023-03-16

Hi Robert,

On the Multicast side there is no ACK from the server, correct?
The Client is just pushing the traffic into the network and the server side does not matter. If this is not the case, how multiple servers (receivers) are handled?
And for -P doesn't help. In the unicast scenario the server becomes the bottleneck since the server thread becomes CPU constrained. But the unicast piece is not a problem because I can achieve my machine limits with several iperf server threads and cliente. On multicast -P is not supported.

My test is trying to identify single flow limits. I know on the network is 5Gbps, and CPU is ~600K pps, I can achieve these with unicast.

On the multicast side I can achieve the 5Gbps with large packets (8000Bytes) so I know the network is throttling the flow, but when I use small packets I cannot get close to 600K pps and core CPU never hits 100%.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

hmm, no reason to not support -P with multicast. I fixed that. It's in master. I can get about 3.8M pps for multicast with 256 byte payloads

[rjmcmahon@ryzen3950 iperf2-code]$ src/iperf -c 239.1.1.1 -u -e -b 1mpps -i 1 -l 256 -t 3 -P 1
------------------------------------------------------------
Client connecting to 239.1.1.1, UDP port 5001 with pid 320707 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 256 byte datagrams, IPG target: 1.00 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.69%enp4s0 port 54279 connected with 239.1.1.1 port 5001 (sock=3) on 2023-03-16 23:13:02.812 (PDT)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-1.00 sec  93.6 MBytes   786 Mbits/sec  383581/0   383582 pps
[  1] 1.00-2.00 sec  93.0 MBytes   780 Mbits/sec  381094/0   381094 pps
[  1] 2.00-3.00 sec  93.1 MBytes   781 Mbits/sec  381265/0   381265 pps
[  1] 0.00-3.00 sec   280 MBytes   782 Mbits/sec  1145943/0   381980 pps
[  1] Sent 1145945 datagrams
[rjmcmahon@ryzen3950 iperf2-code]$ src/iperf -c 239.1.1.1 -u -e -b 1mpps -i 1 -l 256 -t 3 -P 16
------------------------------------------------------------
Client connecting to 239.1.1.1, UDP port 5001 with pid 320716 (16 flows)
TOS set to 0x0 (Nagle on)
Sending 256 byte datagrams, IPG target: 1.00 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  2] local 192.168.1.69%enp4s0 port 45567 connected with 239.1.1.1 port 5001 (sock=4) on 2023-03-16 23:13:08.708 (PDT)
[  1] local 192.168.1.69%enp4s0 port 44225 connected with 239.1.1.1 port 5001 (sock=5) on 2023-03-16 23:13:08.708 (PDT)
[ 15] local 192.168.1.69%enp4s0 port 50394 connected with 239.1.1.1 port 5001 (sock=8) on 2023-03-16 23:13:08.708 (PDT)
[  7] local 192.168.1.69%enp4s0 port 56723 connected with 239.1.1.1 port 5001 (sock=6) on 2023-03-16 23:13:08.708 (PDT)
[  6] local 192.168.1.69%enp4s0 port 48103 connected with 239.1.1.1 port 5001 (sock=13) on 2023-03-16 23:13:08.708 (PDT)
[ 12] local 192.168.1.69%enp4s0 port 42513 connected with 239.1.1.1 port 5001 (sock=9) on 2023-03-16 23:13:08.708 (PDT)
[  3] local 192.168.1.69%enp4s0 port 39970 connected with 239.1.1.1 port 5001 (sock=14) on 2023-03-16 23:13:08.708 (PDT)
[  9] local 192.168.1.69%enp4s0 port 60020 connected with 239.1.1.1 port 5001 (sock=7) on 2023-03-16 23:13:08.708 (PDT)
[ 16] local 192.168.1.69%enp4s0 port 34678 connected with 239.1.1.1 port 5001 (sock=3) on 2023-03-16 23:13:08.708 (PDT)
[  4] local 192.168.1.69%enp4s0 port 59645 connected with 239.1.1.1 port 5001 (sock=29) on 2023-03-16 23:13:08.708 (PDT)
[ 13] local 192.168.1.69%enp4s0 port 36145 connected with 239.1.1.1 port 5001 (sock=31) on 2023-03-16 23:13:08.708 (PDT)
[  5] local 192.168.1.69%enp4s0 port 50465 connected with 239.1.1.1 port 5001 (sock=27) on 2023-03-16 23:13:08.708 (PDT)
[ 10] local 192.168.1.69%enp4s0 port 33493 connected with 239.1.1.1 port 5001 (sock=12) on 2023-03-16 23:13:08.708 (PDT)
[  8] local 192.168.1.69%enp4s0 port 33518 connected with 239.1.1.1 port 5001 (sock=10) on 2023-03-16 23:13:08.708 (PDT)
[ 11] local 192.168.1.69%enp4s0 port 57662 connected with 239.1.1.1 port 5001 (sock=30) on 2023-03-16 23:13:08.708 (PDT)
[ 14] local 192.168.1.69%enp4s0 port 33526 connected with 239.1.1.1 port 5001 (sock=11) on 2023-03-16 23:13:08.708 (PDT)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[ 13] 0.00-1.00 sec  58.2 MBytes   488 Mbits/sec  238388/0   238389 pps
[ 12] 0.00-1.00 sec  59.1 MBytes   496 Mbits/sec  242262/0   242263 pps
[  8] 0.00-1.00 sec  58.7 MBytes   492 Mbits/sec  240391/0   240392 pps
[ 10] 0.00-1.00 sec  58.9 MBytes   494 Mbits/sec  241198/0   241199 pps
[ 15] 0.00-1.00 sec  58.5 MBytes   491 Mbits/sec  239534/0   239535 pps
[  7] 0.00-1.00 sec  57.9 MBytes   485 Mbits/sec  236965/0   236966 pps
[  2] 0.00-1.00 sec  60.1 MBytes   504 Mbits/sec  246334/0   246335 pps
[  5] 0.00-1.00 sec  59.8 MBytes   501 Mbits/sec  244809/0   244822 pps
[  4] 0.00-1.00 sec  58.5 MBytes   491 Mbits/sec  239675/0   239743 pps
[  1] 0.00-1.00 sec  60.6 MBytes   508 Mbits/sec  248081/0   248082 pps
[ 16] 0.00-1.00 sec  60.6 MBytes   508 Mbits/sec  248066/0   248133 pps
[ 14] 0.00-1.00 sec  59.0 MBytes   495 Mbits/sec  241568/0   241570 pps
[  6] 0.00-1.00 sec  59.5 MBytes   499 Mbits/sec  243606/0   243607 pps
[  9] 0.00-1.00 sec  57.6 MBytes   483 Mbits/sec  235794/0   235795 pps
[ 11] 0.00-1.00 sec  58.0 MBytes   487 Mbits/sec  237744/0   237745 pps
[  3] 0.00-1.00 sec  58.8 MBytes   493 Mbits/sec  240855/0   240856 pps
[SUM] 0.00-1.00 sec   944 MBytes  7.92 Gbits/sec  3865270/0 3865286 pps
[ 13] 1.00-2.00 sec  59.5 MBytes   499 Mbits/sec  243877/0   243877 pps
[ 12] 1.00-2.00 sec  59.8 MBytes   501 Mbits/sec  244765/0   244765 pps
[  8] 1.00-2.00 sec  59.5 MBytes   499 Mbits/sec  243822/0   243822 pps
[ 10] 1.00-2.00 sec  59.7 MBytes   501 Mbits/sec  244519/0   244519 pps
[ 15] 1.00-2.00 sec  59.1 MBytes   496 Mbits/sec  242063/0   242063 pps
[  7] 1.00-2.00 sec  58.4 MBytes   490 Mbits/sec  239217/0   239217 pps
[  2] 1.00-2.00 sec  59.2 MBytes   497 Mbits/sec  242608/0   242647 pps
[  5] 1.00-2.00 sec  59.2 MBytes   497 Mbits/sec  242658/0   242647 pps
[  4] 1.00-2.00 sec  59.2 MBytes   497 Mbits/sec  242446/0   242378 pps
[  1] 1.00-2.00 sec  58.8 MBytes   494 Mbits/sec  240976/0   240976 pps
[ 16] 1.00-2.00 sec  59.2 MBytes   497 Mbits/sec  242436/0   242372 pps
[ 14] 1.00-2.00 sec  59.1 MBytes   496 Mbits/sec  242273/0   242273 pps
[  6] 1.00-2.00 sec  59.5 MBytes   499 Mbits/sec  243762/0   243762 pps
[  9] 1.00-2.00 sec  59.1 MBytes   496 Mbits/sec  241955/0   241955 pps
[ 11] 1.00-2.00 sec  59.0 MBytes   495 Mbits/sec  241825/0   241825 pps
[  3] 1.00-2.00 sec  59.2 MBytes   497 Mbits/sec  242499/0   242565 pps
[SUM] 1.00-2.00 sec   948 MBytes  7.95 Gbits/sec  3881701/0 3880618 pps
[ 13] 2.00-3.00 sec  59.7 MBytes   501 Mbits/sec  244393/0   244397 pps
[ 12] 2.00-3.00 sec  59.6 MBytes   500 Mbits/sec  244046/0   244119 pps
[  8] 2.00-3.00 sec  59.8 MBytes   502 Mbits/sec  244985/0   244985 pps
[ 10] 2.00-3.00 sec  59.9 MBytes   502 Mbits/sec  245147/0   245147 pps
[ 15] 2.00-3.00 sec  59.1 MBytes   496 Mbits/sec  242145/0   242173 pps
[  7] 2.00-3.00 sec  59.4 MBytes   498 Mbits/sec  243104/0   243104 pps
[  2] 2.00-3.00 sec  59.0 MBytes   495 Mbits/sec  241850/0   241811 pps
[  5] 2.00-3.00 sec  59.2 MBytes   496 Mbits/sec  242381/0   242381 pps
[  4] 2.00-3.00 sec  59.4 MBytes   498 Mbits/sec  243216/0   243216 pps
[  1] 2.00-3.00 sec  58.3 MBytes   489 Mbits/sec  238905/0   238905 pps
[ 16] 2.00-3.00 sec  58.4 MBytes   490 Mbits/sec  239151/0   239151 pps
[ 14] 2.00-3.00 sec  59.5 MBytes   499 Mbits/sec  243543/0   243553 pps
[  6] 2.00-3.00 sec  59.5 MBytes   499 Mbits/sec  243621/0   243621 pps
[  9] 2.00-3.00 sec  58.8 MBytes   493 Mbits/sec  240789/0   240811 pps
[ 11] 2.00-3.00 sec  59.1 MBytes   496 Mbits/sec  242037/0   242037 pps
[  3] 2.00-3.00 sec  59.3 MBytes   497 Mbits/sec  242797/0   242730 pps
[SUM] 2.00-3.00 sec   948 MBytes  7.95 Gbits/sec  3882110/0 3881043 pps
[ 13] 0.00-3.00 sec   177 MBytes   496 Mbits/sec  726660/0   242220 pps
[ 13] Sent 726662 datagrams
[  8] 0.00-3.00 sec   178 MBytes   498 Mbits/sec  729201/0   243067 pps
[  8] Sent 729203 datagrams
[ 15] 0.00-3.00 sec   177 MBytes   494 Mbits/sec  723744/0   241234 pps
[ 15] Sent 723746 datagrams
[  2] 0.00-3.00 sec   178 MBytes   499 Mbits/sec  730795/0   243599 pps
[  2] Sent 730797 datagrams
[  4] 0.00-3.00 sec   177 MBytes   495 Mbits/sec  725340/0   241780 pps
[  4] Sent 725342 datagrams
[ 16] 0.00-3.00 sec   178 MBytes   498 Mbits/sec  729655/0   243218 pps
[ 16] Sent 729657 datagrams
[  6] 0.00-3.00 sec   178 MBytes   499 Mbits/sec  730993/0   243664 pps
[  6] Sent 730995 datagrams
[ 11] 0.00-3.00 sec   176 MBytes   493 Mbits/sec  721610/0   240537 pps
[ 11] Sent 721612 datagrams
[  3] 0.00-3.00 sec   177 MBytes   496 Mbits/sec  726154/0   242051 pps
[  3] Sent 726156 datagrams
[ 10] 0.00-3.00 sec   178 MBytes   499 Mbits/sec  730867/0   243622 pps
[ 10] Sent 730869 datagrams
[  5] 0.00-3.00 sec   178 MBytes   498 Mbits/sec  729850/0   243284 pps
[  5] Sent 729852 datagrams
[ 14] 0.00-3.00 sec   178 MBytes   497 Mbits/sec  727386/0   242444 pps
[ 14] Sent 727388 datagrams
[ 12] 0.00-3.00 sec   178 MBytes   499 Mbits/sec  731075/0   243689 pps
[ 12] Sent 731077 datagrams
[  1] 0.00-3.00 sec   178 MBytes   497 Mbits/sec  727964/0   242655 pps
[  1] Sent 727966 datagrams
[  7] 0.00-3.00 sec   176 MBytes   491 Mbits/sec  719289/0   239763 pps
[  7] Sent 719291 datagrams
[  9] 0.00-3.00 sec   175 MBytes   490 Mbits/sec  718540/0   239497 pps
[  9] Sent 718542 datagrams
[SUM] 0.00-3.00 sec  2.77 GBytes  7.94 Gbits/sec  11629123/0 3875973 pps
[SUM-16] Sent 11629155 datagrams

and similar for unicast

[rjmcmahon@ryzen3950 iperf2-code]$ src/iperf -c 192.168.1.72 -u -e -b 1mpps -i 1 -l 256 -t 3 -P 1
------------------------------------------------------------
Client connecting to 192.168.1.72, UDP port 5001 with pid 320802 (1 flows)
TOS set to 0x0 (Nagle on)
Sending 256 byte datagrams, IPG target: 1.00 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.69%enp4s0 port 45152 connected with 192.168.1.72 port 5001 (sock=3) on 2023-03-16 23:14:09.190 (PDT)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  1] 0.00-1.00 sec   112 MBytes   942 Mbits/sec  459716/0   459717 pps
[  1] 1.00-2.00 sec   127 MBytes  1.06 Gbits/sec  518239/0   518239 pps
[  1] 2.00-3.00 sec   124 MBytes  1.04 Gbits/sec  508144/0   508144 pps
[  1] 0.00-3.00 sec   363 MBytes  1.01 Gbits/sec  1486102/0   495366 pps
[  1] Sent 1486104 datagrams
read failed: Connection refused
read failed: Connection refused
[  3] WARNING: did not receive ack of last datagram after 10 tries.
[rjmcmahon@ryzen3950 iperf2-code]$ src/iperf -c 192.168.1.72 -u -e -b 1mpps -i 1 -l 256 -t 3 -P 16
------------------------------------------------------------
Client connecting to 192.168.1.72, UDP port 5001 with pid 320817 (16 flows)
TOS set to 0x0 (Nagle on)
Sending 256 byte datagrams, IPG target: 1.00 us (kalman adjust)
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.69%enp4s0 port 34427 connected with 192.168.1.72 port 5001 (sock=4) on 2023-03-16 23:14:28.151 (PDT)
[  2] local 192.168.1.69%enp4s0 port 48530 connected with 192.168.1.72 port 5001 (sock=6) on 2023-03-16 23:14:28.151 (PDT)
[ 13] local 192.168.1.69%enp4s0 port 54866 connected with 192.168.1.72 port 5001 (sock=11) on 2023-03-16 23:14:28.151 (PDT)
[  8] local 192.168.1.69%enp4s0 port 47907 connected with 192.168.1.72 port 5001 (sock=13) on 2023-03-16 23:14:28.151 (PDT)
[  9] local 192.168.1.69%enp4s0 port 49010 connected with 192.168.1.72 port 5001 (sock=8) on 2023-03-16 23:14:28.151 (PDT)
[ 10] local 192.168.1.69%enp4s0 port 55661 connected with 192.168.1.72 port 5001 (sock=12) on 2023-03-16 23:14:28.151 (PDT)
[ 15] local 192.168.1.69%enp4s0 port 58354 connected with 192.168.1.72 port 5001 (sock=33) on 2023-03-16 23:14:28.151 (PDT)
[ 14] local 192.168.1.69%enp4s0 port 53102 connected with 192.168.1.72 port 5001 (sock=10) on 2023-03-16 23:14:28.151 (PDT)
[ 16] local 192.168.1.69%enp4s0 port 39795 connected with 192.168.1.72 port 5001 (sock=31) on 2023-03-16 23:14:28.151 (PDT)
[ 12] local 192.168.1.69%enp4s0 port 49815 connected with 192.168.1.72 port 5001 (sock=27) on 2023-03-16 23:14:28.151 (PDT)
[  5] local 192.168.1.69%enp4s0 port 55379 connected with 192.168.1.72 port 5001 (sock=5) on 2023-03-16 23:14:28.151 (PDT)
[  4] local 192.168.1.69%enp4s0 port 57122 connected with 192.168.1.72 port 5001 (sock=7) on 2023-03-16 23:14:28.151 (PDT)
[  3] local 192.168.1.69%enp4s0 port 51864 connected with 192.168.1.72 port 5001 (sock=3) on 2023-03-16 23:14:28.151 (PDT)
[ 11] local 192.168.1.69%enp4s0 port 54094 connected with 192.168.1.72 port 5001 (sock=9) on 2023-03-16 23:14:28.151 (PDT)
[  6] local 192.168.1.69%enp4s0 port 38966 connected with 192.168.1.72 port 5001 (sock=29) on 2023-03-16 23:14:28.151 (PDT)
[  7] local 192.168.1.69%enp4s0 port 51537 connected with 192.168.1.72 port 5001 (sock=25) on 2023-03-16 23:14:28.151 (PDT)
[ ID] Interval        Transfer     Bandwidth      Write/Err  PPS
[  3] 0.00-1.00 sec  63.6 MBytes   533 Mbits/sec  260421/0   260422 pps
[  5] 0.00-1.00 sec  63.9 MBytes   536 Mbits/sec  261794/0   261795 pps
[ 16] 0.00-1.00 sec  63.7 MBytes   534 Mbits/sec  260733/0   260734 pps
[ 12] 0.00-1.00 sec  62.8 MBytes   527 Mbits/sec  257197/0   257226 pps
[ 15] 0.00-1.00 sec  31.8 MBytes   267 Mbits/sec  130348/0   130443 pps
[  2] 0.00-1.00 sec  64.0 MBytes   537 Mbits/sec  262282/0   262327 pps
[  8] 0.00-1.00 sec  63.4 MBytes   531 Mbits/sec  259504/0   259505 pps
[  6] 0.00-1.00 sec  63.0 MBytes   529 Mbits/sec  258142/0   258143 pps
[  7] 0.00-1.00 sec  63.4 MBytes   532 Mbits/sec  259644/0   259645 pps
[ 14] 0.00-1.00 sec  62.4 MBytes   523 Mbits/sec  255519/0   255520 pps
[  9] 0.00-1.00 sec  62.5 MBytes   525 Mbits/sec  256186/0   256187 pps
[ 13] 0.00-1.00 sec  62.6 MBytes   525 Mbits/sec  256328/0   256363 pps
[  4] 0.00-1.00 sec  62.4 MBytes   523 Mbits/sec  255492/0   255523 pps
[ 11] 0.00-1.00 sec  62.9 MBytes   528 Mbits/sec  257657/0   257658 pps
[ 10] 0.00-1.00 sec  63.5 MBytes   533 Mbits/sec  260134/0   260135 pps
[  1] 0.00-1.00 sec  32.3 MBytes   271 Mbits/sec  132111/0   132149 pps
[SUM] 0.00-1.00 sec   948 MBytes  7.95 Gbits/sec  3883492/0 3883508 pps
[  3] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258718/0   258781 pps
[  5] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258892/0   258933 pps
[ 16] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258900/0   258900 pps
[ 12] 1.00-2.00 sec  63.1 MBytes   529 Mbits/sec  258380/0   258433 pps
[ 15] 1.00-2.00 sec  31.7 MBytes   266 Mbits/sec  129794/0   129701 pps
[  2] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258796/0   258753 pps
[  8] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258702/0   258714 pps
[  6] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258801/0   258801 pps
[  7] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258851/0   258851 pps
[ 14] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258776/0   258776 pps
[  9] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258844/0   258844 pps
[ 13] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258874/0   258840 pps
[  4] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258817/0   258854 pps
[ 11] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258906/0   258906 pps
[ 10] 1.00-2.00 sec  63.2 MBytes   530 Mbits/sec  258851/0   258851 pps
[  1] 1.00-2.00 sec  31.5 MBytes   264 Mbits/sec  129047/0   129068 pps
[SUM] 1.00-2.00 sec   948 MBytes  7.95 Gbits/sec  3881949/0 3879156 pps
[  3] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258840/0   258777 pps
[  5] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258798/0   258803 pps
[ 16] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258825/0   258825 pps
[ 12] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258786/0   258705 pps
[ 15] 2.00-3.00 sec  31.6 MBytes   265 Mbits/sec  129397/0   129458 pps
[  2] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258764/0   258838 pps
[  8] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258798/0   258786 pps
[  6] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258786/0   258786 pps
[  7] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258810/0   258810 pps
[ 14] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258726/0   258805 pps
[  9] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258826/0   258830 pps
[ 13] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258751/0   258815 pps
[  4] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258761/0   258693 pps
[ 11] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258763/0   258779 pps
[ 10] 2.00-3.00 sec  63.2 MBytes   530 Mbits/sec  258830/0   258865 pps
[  1] 2.00-3.00 sec  31.6 MBytes   265 Mbits/sec  129437/0   129383 pps
[SUM] 2.00-3.00 sec   948 MBytes  7.95 Gbits/sec  3881898/0 3880276 pps
[  3] 0.00-3.00 sec   190 MBytes   531 Mbits/sec  777981/0   259327 pps
[  3] Sent 777983 datagrams
[ 16] 0.00-3.00 sec   190 MBytes   531 Mbits/sec  778462/0   259487 pps
[ 16] Sent 778464 datagrams
[ 15] 0.00-3.00 sec  95.1 MBytes   266 Mbits/sec  389541/0   129837 pps
[ 15] Sent 389543 datagrams
[  8] 0.00-3.00 sec   190 MBytes   530 Mbits/sec  777010/0   259002 pps
[  8] Sent 777012 datagrams
[  7] 0.00-3.00 sec   190 MBytes   531 Mbits/sec  777308/0   259103 pps
[  7] Sent 777310 datagrams
[ 14] 0.00-3.00 sec   189 MBytes   528 Mbits/sec  773023/0   257674 pps
[ 14] Sent 773025 datagrams
[ 13] 0.00-3.00 sec   189 MBytes   528 Mbits/sec  773955/0   257978 pps
[ 13] Sent 773957 datagrams
[ 11] 0.00-3.00 sec   189 MBytes   529 Mbits/sec  775328/0   258424 pps
[ 11] Sent 775330 datagrams
[  1] 0.00-3.00 sec  95.4 MBytes   267 Mbits/sec  390597/0   130181 pps
[  1] Sent 390599 datagrams
[ 12] 0.00-3.00 sec   189 MBytes   529 Mbits/sec  774367/0   258122 pps
[ 12] Sent 774369 datagrams
[  6] 0.00-3.00 sec   189 MBytes   530 Mbits/sec  775732/0   258577 pps
[  6] Sent 775734 datagrams
[  9] 0.00-3.00 sec   189 MBytes   528 Mbits/sec  773859/0   257953 pps
[  9] Sent 773861 datagrams
[ 10] 0.00-3.00 sec   190 MBytes   531 Mbits/sec  777817/0   259257 pps
[ 10] Sent 777819 datagrams
[  2] 0.00-3.00 sec   190 MBytes   532 Mbits/sec  779844/0   259944 pps
[  2] Sent 779846 datagrams
[  4] 0.00-3.00 sec   189 MBytes   528 Mbits/sec  773073/0   257691 pps
[  4] Sent 773075 datagrams
[  5] 0.00-3.00 sec   190 MBytes   532 Mbits/sec  779486/0   259819 pps
[  5] Sent 779488 datagrams
[SUM] 0.00-3.00 sec  2.78 GBytes  7.95 Gbits/sec  11647383/0 3882319 pps
[SUM-16] Sent 11647415 datagrams

Last edit: Robert McMahon 2023-03-17

Robert McMahon - 2023-03-17

Also, one can use port ranges too, e.g. -p 5001-5008

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Multicast test performance

A means to measure network responsiveness and throughput

Forums

Help

Multicast test performance

Multicast test performance

A means to measure network responsiveness and throughput

Forums

Help

Multicast test performance document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Multicast test performance