[Tcpreplay-devel] A problem with tcpreplay and limited output buffers

[Alex M. <mal...@gm...>]

Hi

I have a rather unconventional use for tcpreplay which involves sending
traffic through limited queues for QoS measurement on linux. Because of
that, to me, the timing of the sent packets is actually more important  than
the packets themselves. This use of tcpreplay caused me some problems which
I'd like to share with you (and hopefully with anyone else having a similar
problem).

The problem is that tcpreplay silently attempts to retransmit all failed
packets, even if the failure is due to a lack of buffer space (ENOBUFS). The
first problem this causes is that any failed transmission attempt by
tcpreplay (e.g. ENOBUFS) will be considered as a dropped packet by iproute2,
even if a retransmission later succeeds. This is not really a bug, but I
feel like it should be documented somewhere. The second, more serious,
problem is that tcpreplay doesn't drop packets when the output buffer is
full, thus limiting its throughput and skewing any performance measurements.
Even if the output buffer is one packet long tcpreplay will send the entire
trace as if nothing is wrong. This may be the default desired behavior but
for my use case it's simply catastrophic. The proper solution to this is
adding a user level option for skipping packets if the buffer cannot
currently hold them. I think that this feature should be added to future
versions.

Meanwhile, my solution was to patch sendpacket(sendpacket_t *sp, const
u_char *data, size_t len) in common/sendpacket.c to prevent retransmissions
in case of ENOBUFS by replacing the "goto TRY_SEND_AGAIN;" lines with
"retcode=(int)len;" to make the rest of the code think that the packet was
sent successfully ( sp->retry_enobufs now holds the number of packets
dropped due full buffers). This patch seems to work well except it strangely
causes problems when exiting using ctrl+c (don't know why, haven't really
tinkered with it much).