Enable bounceback measurements with asymetric dataload

[Andreas]

While conducting responsiveness tests using the bounceback option, I noticed that the server consistently sends back the same amount of data that it receives from the client. We have been utilizing iperf for throughput measurements across mobile communication networks, which often have unique characteristics in their uplink and downlink directions. This makes it essential to conduct individual tests on unidirectional client-server connections.

The question I have is whether or not there is an argument that I may have unintentionally overlooked in my testing that could force bounceback to generate data solely in one direction (from client to server . I am aware that the server must send back some data to the client to communicate timing information). If such an argument does not currently exist, are there any plans to support unidirectional throughput with the bounceback option?

[Jean Tourrilhes]

Hi,
The way I do that is with --bounceback-request / --bounceback-reply.
If you configure a large request size and a small reply size, you will get an asymmetrical test, with very low bandwidth need on the return path. If you are limited by the number of packets on the return path, you can use large request sizes spanning multiple MSSs/MTUs to create more packets on the request path.
Those are not unidirectional, but fairly close to it.
Regards,
Jean

[Robert McMahon]

The latest in master has the following bounceback options. The reply & request sizes need to be large enough to carry the bounceback payload with timestamps.

--bounceback[=n]
run a TCP bounceback or rps test with optional number writes in a burst per value of n. The default is ten writes every period and the default period is one second (Note: set size with --bounceback-request). See NOTES on clock unsynchronized detections.
--bounceback-hold n
request the server to insert a delay of n milliseconds between its read and write (default is no delay)
--bounceback-no-quickack
request the server not set the TCP_QUICKACK socket option (disabling TCP ACK delays) during a bounceback test (see NOTES)
--bounceback-period[=n]
request the client schedule its send(s) every n seconds (default is one second, use zero value for immediate or continuous back to back)
--bounceback-request n
set the bounceback request size in units bytes. Default value is 100 bytes.
--bounceback-reply n
set the bounceback reply size in units bytes. This supports asymmetric message sizes between the request and the reply. Default value is zero, which uses the value of --bounceback-request.
--bounceback-txdelay n
request the client to delay n seconds between the start of the working load and the bounceback traffic (default is no delay)

struct bb_ts {
    uint32_t sec;
    uint32_t usec;
};
struct bounceback_hdr {
    uint32_t flags;
    uint32_t bbsize;
    uint32_t bbid;
    uint16_t bbflags;
    uint16_t tos;
    uint32_t bbRunTime; //units 10 ms
    struct bb_ts bbclientTx_ts;
    struct bb_ts bbserverRx_ts;
    struct bb_ts bbserverTx_ts;
    uint32_t bbhold; // up to here is mandatory
    uint32_t bbrtt;
    struct bb_ts bbread_ts;
    uint32_t bbreplysize;
};

[Andreas]

Thank you, Robert, for your swift answer and for pointing me to the relevant implementation. Your insights made me realize that a minimum packet size is necessary to transmit bounceback data effectively.
To elaborate more on my intentions, I'm trying to measure latency under high uplink or downlink load conditions. Initially, my idea was to use the bounceback parameter n (--bounceback[=n]) to increase the number of packets sent per interval, or parameter -l to expand the packet size, aiming to create a significant load in one direction. This is when I noticed that in the backhaul the number and size of data blocks are just the same as in forward drirection resulting in equal throughput in both directions.
However, now I understand that the correct approach for achieving my use case involves combining --bounceback with --working-load[=up|down|bidir][,n] argument, which steers me in the correct direction (see attachment).
Despite this, I have observed a discrepancy while looking at the TCP throughput from the working-load stream in combination with --bounceback. The data indicates 481 Mbits/sec, whereas the IP connection's actual capacity should be around 124 Gbit/s:

user@93c9152d0b01:/mnt/hgfs/iperf/iperf-2.1.9/src$ ./iperf -c 192.168.138.103 -i 1 -t 3
------------------------------------------------------------
Client connecting to 192.168.138.103, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  1] local 172.17.0.2 port 53970 connected with 192.168.138.103 port 5001 (icwnd/mss/irtt=14/1448/41)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.00-1.00 sec  13.5 GBytes   116 Gbits/sec
[  1] 1.00-2.00 sec  15.0 GBytes   129 Gbits/sec
[  1] 2.00-3.00 sec  14.8 GBytes   127 Gbits/sec
[  1] 0.00-3.01 sec  43.3 GBytes   124 Gbits/sec

vs.

user@93c9152d0b01:/mnt/hgfs/iperf/iperf-2.1.9/src$ ./iperf -c 192.168.138.103 --bounceback -i 1 -t 3 --working-load=down
------------------------------------------------------------
Client connecting to 192.168.138.103, TCP port 5001 with pid 930 (1 flows)
Write buffer size:  100 Byte
Bursting:  100 Byte writes 10 times every 1.00 second(s)
Bounce-back test (size= 100 Byte) (server hold req=0 usecs & tcp_quickack)
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  2] local 172.17.0.2%eth0 port 56308 connected with 192.168.138.103 port 5001 (prefetch=16384) (reverse) (sock=3) (qack) (icwnd/mss/irtt=14/1448/35) (ct=0.06 ms) on 2024-02-23 12:11:17.822 (UTC)
[  1] local 172.17.0.2%eth0 port 56324 connected with 192.168.138.103 port 5001 (bb w/quickack len/hold=100/0) (sock=4) (icwnd/mss/irtt=14/1448/28) (ct=0.05 ms) on 2024-02-23 12:11:17.822 (UTC)
[ ID] Interval        Transfer    Bandwidth         BB cnt=avg/min/max/stdev         Rtry  Cwnd/RTT    RPS
[  1] 0.00-1.00 sec  1.95 KBytes  16.0 Kbits/sec    10=0.105/0.035/0.536/0.152 ms    0   14K/30 us    9497 rps
[ ID] Interval        Transfer    Bandwidth       Reads=Dist
[ *2] 0.00-1.00 sec  58.0 MBytes   486 Mbits/sec  607687=0:0:0:0:0:0:0:0
[  1] 1.00-2.00 sec  1.95 KBytes  16.0 Kbits/sec    10=0.312/0.290/0.375/0.024 ms    0   14K/130 us    3210 rps
[ *2] 1.00-2.00 sec  55.8 MBytes   468 Mbits/sec  585389=0:0:0:0:0:0:0:0
[  1] 2.00-3.00 sec  1.95 KBytes  16.0 Kbits/sec    10=0.083/0.062/0.158/0.036 ms    0   14K/67 us    12107 rps
[ *2] 2.00-3.00 sec  58.1 MBytes   488 Mbits/sec  609683=0:0:0:0:0:0:0:0
[  1] 0.00-3.03 sec  7.81 KBytes  21.1 Kbits/sec    40=0.136/0.028/0.536/0.131 ms    0   14K/4005 us    7346 rps
[  1] 0.00-3.03 sec BB8(f)-PDF: bin(w=100us):cnt(40)=1:26,2:3,3:3,4:7,6:1 (5.00/95.00/99.7%=1/4/6,Outliers=0,obl/obu=0/0)
[ *2] 0.00-3.03 sec   174 MBytes   481 Mbits/sec  1821224=0:0:0:0:0:0:1:0
[ CT] final connect times (min/avg/max/stdev) = 0.053/0.057/0.062/6.364 ms (tot/err) = 2/0

Interestingly, omitting --bounceback in the final test boosts the throughput back up to 124 Gbit/sec. I'm curious, do you have any insights as to why the throughput for the working-load stream would significantly drop when --bounceback is additionally utilized?
By the way, I also experimented with augmenting the number of streams in conjunction with the working-load argument. This technique boosted the throughput to approximately 13 Gbit/s with about 24 streams in use. However, this is still significantly less than the channel's actual capabilities.

[Jean Tourrilhes]

This is when I noticed that in the backhaul the number and size of data blocks are just the
same as in forward drirection resulting in equal throughput in both directions.

As I said in my previous message, a workaround to to use larger requests, spanning multiple MSSs/MTUs.
Jean

[Robert McMahon]

I would probably use a newer version of iperf. There has been a lot of bug fixes in this area since 2.1.9. I plan to release a new version soon. Until then, you'll need to build from the master branch.

Sorry I missed your message until now.

[Andreas]

No worries, I've been away for a while and didn't have a chance to follow up on this. Now, after updating to iperf 2.2 I see a lot of improvements in the bounceback area, most importantly -working-load down|up seems to work withoug any problems. Thanks! Great work!