Iperf 2 Files

See doc/DESIGN_NOTES for some description around the software design
See doc/CLOCKSYNC_NOTES for some description around synchronizing the
realtime clocks

2.2.2 (as of August 2nd, 2026)
------------------------------
o feat: --working-load pause/resume now covers every direction and both
  transports - SIGUSR1 to the client process pauses, SIGUSR2 resumes,
  without tearing the connection down. Intended for a technician manually
  correlating a bounceback probe's own RPS/latency trace against load
  on/off transitions, without needing to restart the whole test (and
  re-cold-start UDP-L4S's own PragueCC congestion state) at every
  transition.
  - Up (and bidir's own upload half): the client's own writer thread is
    gated directly - new FLAG_WORKING_LOAD_THREAD (Settings.hpp) marks a
    spawned thread as itself being a working-load stream, distinct from
    the main flow's own isWorkingLoadUp/Down request, so the pause only
    ever affects the working-load thread's own writes, never the main
    traffic thread's. Checked in all three of a working-load writer's
    possible loops (RunTCP, RunWriteEventsTCP, RunUDPL4S); UDP-L4S also
    skips its own "no ACK for 1s" peer-close check while paused so a
    multi-second pause doesn't trip a teardown meant for a genuinely dead
    peer.
  - Down (and bidir's own download half): this writer is server-spawned,
    a different process a signal to the client can't reach directly - but
    the client still locally reverses roles for this direction
    (clientside_client_reverse), so it owns the *reader* instead. TCP
    pauses via pure backpressure (the reader just stops calling recv();
    TCP's own flow control stalls the remote writer, no wire changes at
    all). UDP-L4S has no backpressure, so the reader instead relays a new
    level-triggered L4S_PAUSE bit (payloads.h) on every ACK while
    locally paused, which the real writer gates its send loop on; a
    0.2s timer-driven heartbeat ACK (independent of data receipt) keeps
    that same writer's dead-peer check from firing during an extended
    pause, since a paused writer would otherwise give the reader nothing
    left to acknowledge.
  Root cause of an early "pause does nothing at all" symptom on Android:
  SIGUSR1/SIGUSR2 were blocked in the process's inherited signal mask
  (masks survive exec(), unlike handler dispositions - Android execs this
  binary as a child of the app's own JVM/ART process, which blocks
  various signals for its own purposes); sigaction() registration
  succeeded and the signal reached the process with no permission error,
  but the handler was simply never invoked. Fixed with an explicit
  sigprocmask(SIG_UNBLOCK, ...) at startup. Two other real bugs found
  and fixed getting this usable end-to-end: a TCP-write CSV row landing
  on exactly the same field count as a bounceback row (both 26 fields),
  silently misread as a garbage bounceback sample whenever
  --working-load=up|bidir ran alongside --bounceback; and a pause
  outlasting one report interval corrupting that interval's own CSV row
  by skipping the Reporter thread's normal interval-boundary bookkeeping.
  New t/t42-t44 regression tests (up direction; reversed TCP; reversed
  UDP-L4S, including a pause held well past the UDP-L4S dead-peer
  threshold specifically to cover the heartbeat fix)
o fix(android): three follow-on bugs found actually using the
  above pause/resume feature through the app rather than the CLI -
  the Pause/Resume button was unconditionally hidden whenever the
  configured direction was Download, a leftover condition from before
  reversed-direction pause existed; pause-vs-real-stall detection
  depended on a native CSV marker that only ever existed for TCP's
  up-direction shape (never UDP-L4S, never the reversed/down direction
  at all), so a genuine pause on any other combination showed a
  misleading flat zero instead of a chart gap - now keyed off the
  app's own isWorkingLoadPaused state instead (it already knows when
  it's paused, since it's the one that sent the signal), covering
  every direction/transport uniformly; and the "Zoom Tail" histogram
  button had no enabled gate at all, staying clickable the whole time
  its own chart was still showing "Waiting for data..." - now gated on
  the histogram actually being present, matching the row's other three
  overlay buttons
o fix: a connected UDP socket surfaces a queued ICMP Destination-Port-
  Unreachable as ECONNREFUSED on its next read - FATALUDPREADERR treated
  this as fatal (unlike FATALUDPWRITERR, which already excluded it),
  misreading a transient refusal as the peer gracefully closing and
  permanently ending the read loop. Most exposed by --full-duplex UDP-L4S,
  which starts two independent new connections to the same server
  microseconds apart, regularly racing Listener::udp_accept()'s brief
  "socket handed off, not yet re-listening" window (a pre-existing,
  documented tradeoff - see ticket 357) - this is what the "Connection
  refused" KNOWN ISSUE below was. Fixed by excluding ECONNREFUSED from
  FATALUDPREADERR, matching the write-side convention, plus a small
  stagger between --full-duplex's two connects as defense in depth.
  t/t39_udp_l4s_full_duplex.sh is un-stubbed and passing. Also fixed, as
  a side effect: the same macro is used by UDP bounceback's own
  fatal-read path, and on POSIX a fatal error there only stops the one
  thread (thread_stop()/pthread_exit(), not a process exit) - so a
  bounceback flow dying this way while an independent --working-load
  stream kept running left the whole client hanging indefinitely past
  its own -t deadline, worse than a clean failure. That was the other
  half of the --bounceback + --working-load-transport udp-l4s KNOWN
  ISSUE below; the "same-port dispatch collision" it described turned
  out not to be a real bug once investigated further - the regression
  test's own server invocation had accidentally used -P 1 (permanently,
  not just transiently, blocking the second of the two flows this
  scenario needs). t/t40_bounceback_workingload_udpl4s.sh is un-stubbed
  and passing; verified with 8 concurrent clients of each scenario under
  CPU stress, 0 failures, 0 hangs
o test: new t/t31_bounceback_ipv6.sh - --bounceback's own accept/parse
  path had no IPv6 coverage before (only plain TCP and UDP did)
o docs: corrected several stale/broken spots in the man page, its HTML
  mirror, and the user manual - wrong --permit-key community-edition
  gating text (claimed both client and server are blocked; only the
  server is), --NUM_REPORT_STRUCTS default said 5000 (actual 10000),
  --histograms bincount defaults off by 10x on both the server and
  client side, three broken option names that are hard CLI failures
  rather than typos (--fileinput/--stdin/--peerdetect ->
  --file_input/--stdin_input/--peer-detect), a duplicate/wrong
  -Z entry, a broken --udp-l4s=80211 reference, an inaccurate
  "professional edition required for UDP L4S" claim, a stale one-line
  --working-load description on the server side, and two duplicate .TP
  macros rendering as blank man page entries
o chore: removed remaining traces of an experimental, never-finished
  802.11/concentrator-telemetry wire-format extension for UDP-L4S
  (payloads.h structs, an orphaned CLI flag with zero callers, a design
  doc) - design-only scaffolding, never wired into any actual send/
  receive path, cleaned up ahead of this release rather than shipped
  half-finished
o feat: UDP bounceback (RPS/responsiveness test) now runs over UDP-L4S,
  not just TCP - a discrete stop-and-wait exchange (struct
  bounceback_udp_hdr) reusing the same ECT(1)/CE marking as the
  continuous UDP-L4S stream, with its own bounded per-request timeout
  (a lost datagram costs one round trip instead of stalling the rest of
  the run) rather than TCP's retransmission-backed reliability. Also
  carries CE-mark stats through in the human-readable/CSV output for
  both TCP and UDP-L4S bounceback
o feat: UDP bounceback per-direction loss attribution - the server
  echoes its own running count of valid requests received
  (bounceback_udp_hdr.bbRequestsRx) on every reply, letting the client
  tell request-direction loss (never reached the server) apart from
  reply-direction loss (received and answered, but the answer never
  came back) instead of only seeing an aggregate round-trip gap.
  Reported per-interval and as a lifetime total in both -y C CSV
  (bbrequestssent/bbrepliesreceived/bbserverreceived/bbreqloss/
  bbreplyloss) and human-readable ("BB Loss Req=x/x Reply=x/x") output
o feat: --trip-times one-way-delay (OWD) reporting extended to -y C CSV
  output for bounceback (previously human-readable only) - per-direction
  OWD/asymmetry stats (owdto/owdfro/asym, mean/min/max/stdev) plus a
  running count of clock-sync-impossible samples. Also adds a clock-
  offset-tolerant relative-OWD variant (owdtorel/owdfrorel): each sample
  minus that direction's own lifetime running minimum, so a constant
  (but unknown) client/server clock offset cancels out and only genuine
  OWD variation above the best-observed sample is reported - useful when
  the two hosts' clocks aren't disciplined against each other, where the
  absolute OWD numbers alone aren't trustworthy
o fix: a UDP client thread pushes a packetID=-1 "not a real event"
  sentinel through the reporter pipeline at end-of-test (EndJob());
  the generic UDP packet handler had no guard against it, letting the
  sentinel silently overwrite the real final PacketID right before the
  last report reads it - corrupting that row's own datagram-count/
  errpercent fields (e.g. a genuine 4-error run printing "errpercent
  -400.000"). Universal, pre-existing bug on any UDP client run with -e;
  only ever visible as a dramatic garbage value on UDP-L4S bounceback +
  working-load (the one write-role path with a genuinely nonzero error
  count) rather than plain UDP's always-zero write-side error count
o fix: --udp-l4s --full-duplex shared one socket between its two traffic
  directions, an assumption carried over from TCP's independent read/
  write streams that doesn't hold for UDP - recv() on a shared fd
  returns whichever datagram arrives next regardless of which logical
  thread wants it, starving one direction. Each direction now gets its
  own independent connection, mirroring the existing plain --reverse
  socket setup (see this file's own top entry for the follow-on
  "Connection refused" issue this surfaced, and its fix)
o feat: --cca is now dual-purpose - the existing literal TCP algorithm
  name (--tcp-cca's own long-standing behavior, kept as a deprecated
  alias) when the test isn't UDP-L4S, or a prague/reno choice for UDP-L4S
  congestion response when it is. In Reno mode, PragueCC's CE-triggered
  alpha-proportional decrease is suppressed - only the loss-triggered
  flat halving stays active, matching how a real Classic/legacy
  congestion-controlled flow behaves sharing an L4S-capable queue.
  --working-load-cca is the same dual-purpose choice for an independent
  working-load stream. New --working-load-transport <tcp|udp-l4s> lets a
  working-load stream run as its own UDP-L4S flow instead of always
  being forced to plain TCP. New --dual-transport (server-only) opens a
  second listener one port off the main one, for whichever of TCP/UDP-L4S
  the main listener isn't already serving, so one server process can
  serve a main test and a working-load stream regardless of which
  protocol each side picks (see this file's own top entry for the
  follow-on hang issue combining --bounceback with
  --working-load-transport udp-l4s surfaced, and its fix)
o fix: a --permit-key-file server under --dual-transport could race
  between its main and auxiliary listener threads both reloading the
  key file at once
o fix: UDP bounceback's end-of-test EINTR (a blocking reply read
  interrupted by the test's own one-shot end-of-test SIGALRM landing
  while the last reply was still in flight) was being treated as a read
  failure instead of retried/allowed to fall through to a clean exit
o fix: histogram_print_csv()/--histogram-baseline's EMD comparison used
  a histogram's raw bin-width count as-is rather than converting it to
  genuine microseconds first - invisible for bounceback's own default
  histogram (always microsecond-scaled already), but corrupted the
  printed binwidthus field, the EMD-against-baseline comparison, and the
  --histogram-baseline-save file's own copy of the row for any histogram
  actually configured with a different scale (e.g. a trip-time histogram,
  or an explicit --histograms=...m)
o feat: --txdelay-time (a relative start-delay, as opposed to
  --txstart-time's absolute epoch) now relays to the server for -R/
  --full-duplex, so the server's own reverse-writer thread also holds
  back before sending. Unlike an absolute epoch, a relative delay needs
  no client/server clock synchronization to land both ends' start within
  about one connection-setup round trip of each other - intended as a
  primitive for external orchestration (a script or app launching
  several iperf invocations to sequence timed load phases), not any new
  scheduling logic in the binaries themselves
o test: 7 new regression tests (t35-t41) covering all of the above -
  bounceback OWD/loss CSV shape, the --txdelay-time relay (verified via
  real wall-clock timing, the only way to actually prove the server held
  back rather than silently ignoring the relayed value), --dual-transport,
  --working-load-transport, and full-duplex UDP-L4S (stubbed skip, see
  above). Also adds IPERF_TEST_BB_DROP_REPLY_MODULO, a permanent test-
  only debug hook (not a real CLI feature - no flag, no man page entry)
  that silently withholds every Nth bounceback reply, and a new test
  using it to verify the loss-attribution counters actually respond
  correctly under real induced loss - previously every loss-related test
  in the suite only checked CSV row shape on a clean, loss-free loopback
  run
o fix: bounceback's one-time permit-key handshake was gated on
  burst_id == 1 (the scheduling tick, shared by every request within a
  --bounceback-burst window), not a real one-shot flag - the key got
  resent on every request within the first tick's burst instead of once,
  corrupting the fixed-size framing the server expects until the
  connection desynced and reset a few intervals in (the "bounceback
  fails after a few seconds" symptom reported from IETF 126 testing).
  Also fixes a separate wire-compatibility bug found while investigating:
  client_udp_testhdr's permitkey field was wrapped in #if
  HAVE_PROFESSIONAL, so a professional-edition client and a community-
  edition UDP server disagreed on this header's byte layout
o feat(bounceback): kernel RTT min (Linux tcpi_min_rtt) added to CSV/
  human-readable output and the Android RPS chart alongside a
  theoretical RPS ceiling (1000/rtt_min_ms) - bounceback is strictly
  serialized, so this is a real, principled ceiling letting a reader
  tell a pacing-limited run from a path-limited one
o fix(bounceback): --tcp-cca never reached the server (bounceback's own
  wire handshake didn't carry a CCA field or parse plain TCP's extend-
  header CCA flags) - added as a one-time suffix on the first request,
  same convention already used for the bounceback permit-key
o feat: bounceback RTT histogram support for -y C CSV output (previously
  human-readable only) plus a native --histogram-baseline name=path/
  --histogram-baseline-save name=path pair for Earth Mover's Distance
  comparison against a saved reference histogram, printed as a new
  trailing CSV field - generalized beyond bounceback to any histogram in
  the same pass (renamed reportCSV_client_bb_histogram_* ->
  reportCSV_client_histogram_*)
o fix(darwin): macOS/iOS's TCP_CONNECTION_INFO getsockopt path left
  in-flight/notsent/rtt_min fields as uninitialized memory instead of
  the explicit -1/"unavailable" sentinel every other unavailable field
  in this codebase uses - groundwork for an iOS client. Unverified (no
  Darwin toolchain available to compile/test this specific branch)

o fix: a server with no --permit-key/--permit-key-file configured was
  rejecting any client that presented a key anyway (Listener.cpp's
  apply_client_settings_udp/apply_client_settings_tcp/bounceback key-check
  branches all explicitly rejected on HEADER_KEYCHECK when the server's own
  isPermitKey() was false - one had a comment confirming this was
  deliberate) - a professional client with its own default/silent key
  talking to an unrelated server that doesn't gate on permit-key at all was
  silently refused. Fixed to still decode the key (consuming its bytes off
  the wire so TCP framing stays in sync, and recording it for display) but
  accept the connection regardless when the server has no requirement of
  its own to fail against; the connection-report line (report_peer vs
  report_peer_fail, ReportOutputs.c, keyed off server->mKeyCheck) is forced
  back to the normal "connected" banner in this case too, since it
  previously misreported these as "[drop] ... (permit key fail)" even once
  the connection itself was being accepted. Verified locally: no-key-server
  + keyed client now succeeds; a permit-key-gated server's own wrong-key/
  no-key rejection and correct-key acceptance are both unchanged
o fix (log/report hygiene): setTransferID() (Reports.c) was embedding the
  raw --permit-key value itself into mTransferIDStr, the "[srcip key(id)] "
  prefix used on every report/error line for a permit-key-gated connection
  - printing the secret in plaintext to the console/any captured logs. The
  function's own existing comment already explains that srcip alone (not
  the key) is what disambiguates simultaneous clients sharing the same
  static key, so the key was redundant there; removed from both format
  strings (forward and REVERSED/role-reversal branches), prefix is now just
  "[srcip(id)] "
o feat(android): remaining bar-graph meters (TCP Write Block Time/Retries,
  UDP-L4S Packets/sec, CE marks/sec, Marking Probability, Loss Probability)
  converted to the same compact-dial + correlated MiniSparkline layout
  already used for Throughput/RTT/Load Delay/CE Duration, so people can see
  each metric's own recent trend and correlate it against the others
  in-place rather than only ever reading one collapsed bar reading at a
  time. CWND/inFlight/NotSent and Window/Inflight/Queued each also gain a
  combined three-line mini chart (their three series sharing one axis)
  paired beside the existing bar. The old separate RTT/Load-Delay/floor-
  stability charts are consolidated into one RTT-smoothed-vs-RTT-min chart
  per mode, with on-canvas current-value callouts instead of a below-chart
  text line
o fix: server startup banner ("Server listening on...", both the plain and
  pid/portrange variants in Locale.c) now inserts a "(professional edition) "
  tag right after "Server " on professional builds (HAVE_PROFESSIONAL), e.g.
  "Server (professional edition) listening on TCP port 5201 with pid 315934";
  community builds are unchanged (empty tag, not a runtime check) - see
  IPERF_SERVER_EDITION_TAG in include/version.h
o fix: --bounceback (RPS/responsiveness test) completely bypassed the
  --permit-key handshake on both client and server - Client.cpp's
  StartSynch() explicitly excluded bounceback from SendFirstPayload(), and
  Settings_GenerateClientHdr() short-circuited before reaching its own
  key-embedding logic for bounceback, while Listener.cpp's HEADER_BOUNCEBACK
  branch never called test_permit_key() at all; a permit-key-gated server
  was silently accepting unauthenticated bounceback traffic. Since
  bounceback_hdr.bbsize is negotiated once and then reused as a fixed
  per-round-trip size for the life of the connection, the key can't be
  spliced into that region without desyncing later bounces - fixed by having
  the client send the key as a separate suffix (2-byte length + value,
  standard permitKey wire format) immediately after the first bounce's own
  bbsize-counted write, signaled via HEADER_KEYCHECK in the same leading
  flags word Listener.cpp already decodes pre-dispatch; the server validates
  it once at connection setup, reusing test_permit_key() unmodified. This is
  a wire-protocol change for bounceback; verified locally across all 4
  key/no-key combinations and live against a production permit-key server
  (both rejection with no key and acceptance with a valid key)
o fix: fullduplex EndJob() (Reporter.c) decided which of a fullduplex thread
  pair tears down a shared SumReport using fullduplex_stop_barrier()'s own
  racy return value, which could destroy the SumReport's mutex while the
  other thread was still waiting on it (bionic FORTIFY: "pthread_mutex_lock
  called on a destroyed mutex"); reproducible with --bounceback
  --working-load=bidir,1. Fixed by using SumReport's existing reference
  counter (Incr/DecrSumReportRefCounter) to gate teardown instead of the
  barrier's return value
o feat(android): new Bounceback (RPS) test mode - live requests-per-second
  meter, working-load up/down byte-in-flight bars sharing one scale, and a
  rolling-window sparkline (Welford's online algorithm, 80-sample window,
  monotonic-deque O(1) min/max) showing current mean/stdev band plus
  running min/max; excludes the CSV "Sum" aggregate row (transferid=-1),
  which shares bounceback's own field layout and would otherwise
  intermittently overwrite real samples
o fix: --permit-key-file now refuses to start the server (rather than WARN
  and continue) when it can't be opened or contains no usable keys -
  previously a misconfigured/unreadable file left a permit-key-gated
  server listening and looking perfectly healthy (process up, port open)
  while silently rejecting every single connection
o fix: recvn()'s fatal-read-error paths (e.g. ECONNRESET, a truncated
  handshake read) set the process-wide sInterupted flag instead of the
  per-thread tInterupted one, so a single client resetting its connection
  could silently take down the entire listener - a clean exit, so
  Restart=on-failure wouldn't recover it. Same class of cross-connection
  bug as the per-thread test-duration timer fix below, just not fully
  applied to this path when that fix went in; seen in production taking
  down a --permit-key-timeout-gated server hours into what should have
  been a week-long run
o feat: android/compute_permit_key.py can auto-detect --version-info from
  a connected, authorized adb device (finds the app's own bundled
  libiperf.so via `dumpsys package` + `find`, runs it with -v) instead of
  requiring a manual adb run-as/pm-path incantation pasted in by hand;
  --version-info remains available as a manual override
o fix: --permit-key/--permit-key-timeout/--permit-key-file now require the
  professional edition (community was never intended to support this
  feature) - a community edition build refuses to start (client or
  server) with any of these options set, rather than silently running
  with an access-control scheme the operator believes is enforced
o feat: --permit-key-file [=<path>] loads a reloadable (SIGHUP, no
  restart) list of valid permit keys, in addition to (or instead of)
  the single --permit-key value - lets a key be rotated in (e.g. a new
  Android app build deriving a new key, see doc/PERMIT_KEY_NOTES) with
  an overlap window instead of a hard cutover that drops already-
  connected clients using the still-valid old key. File format is
  systemd EnvironmentFile=-compatible, so the same file already used
  for a systemd unit's own EnvironmentFile= can be pointed at directly.
  Defaults to /etc/iperf2/permit-keys.env when no path is given.
o feat(android): live meters now also show a running min/avg/max per
  metric (throughput, rtt/jitter, bytes-in-flight/pps), computed in the
  app itself since iperf2's own final report only averages bytes/speed,
  not rtt/cwnd/bytes-in-flight; the first sample of a run is excluded
  from these stats (still shown live) since its interval still has data
  sitting in the send buffer that hasn't crossed the network yet.
o fix: the --reverse-mode bytes/packets-in-flight relay (Server.cpp's
  burst_info parsing, Reporter.c's reporter_handle_packet_server_tcp, and
  ReportOutputs.c's tcp_output_read_enhanced_reverse{,_csv}) read/wrote
  those two tcpstats fields unconditionally, but they only exist in
  struct iperf_tcpstats when HAVE_TCP_STATS is set - true on Linux where
  this shipped, but not on platforms without TCP_INFO support (e.g.
  Windows/mingw), where it failed to compile at all; found via an
  overnight cross-compile regression sweep (native Linux, mingw32/64,
  OpenWrt ath79-musl/x86_64-musl, Android arm64-v8a/armeabi-v7a/x86_64)
o fix: per-thread test-duration timer (was a single process-wide clock via
  setitimer()/ITIMER_REAL plus one global interrupt flag) - on a server
  handling overlapping connections, whichever connection's own timer fired
  first could truncate a different, unrelated, still-running connection
  (seen in production as "FAIL: writen errno = 104" / "shutdown failed"
  well short of that connection's own configured duration); now isolated
  per-connection via timer_create()+SIGEV_THREAD_ID (Linux/glibc)
o fix: a --permit-key server's listener accept-loop lifetime was tied to
  -t (armed relative to when the server PROCESS started), so a persistent
  server given -t for connection safety would stop accepting *any new*
  connections once that -t elapsed, ignoring a much longer
  --permit-key-timeout; -t on a --permit-key server now only bounds each
  accepted connection's own duration, with --permit-key-timeout (or -P N)
  governing the listener's own lifetime instead. A plain (non-permit-key)
  server keeps -t bounding the whole listener lifetime, unchanged, for
  compatibility with single-session server invocations
o fix: a client's own -t request in reverse/full-duplex/dual-test modes
  could force the server to send for longer than the server's own -t
  allowed, with no cap; now clamped to whichever is shorter whenever the
  server itself was given its own -t
o feat: server startup with -e/--enhanced now also lists the TCP
  congestion control algorithms available on the host (Linux only, via
  /proc/sys/net/ipv4/tcp_available_congestion_control), so a client
  picking -Z <algo> can check the server's own log to confirm that
  algorithm is actually loaded there
o add make check tests t28_listener_forever, t29_reverse_duration_clamp,
  t30_plain_server_time_compat
o feat(android): Host/Port fields now offer a dropdown of the last 5
  values actually used (not every keystroke), cleared by "Reset to
  defaults"; Port field is now numeric-only (no lookups apply to a port
  number, unlike Host)
o feat(android): simple LED-style connection indicator - green
  "Connected" once real data is flowing, red "Connect failed" on the
  same conditions as before; mutually exclusive, nothing shown while
  idle or still connecting (the Start button's own "Connecting..." text
  already covers that phase)
o feat: server also relays packets-in-flight in --reverse mode's TCP_INFO
  relay (previously just bytes-in-flight/rtt) - new tcppktsinflight
  CSV/human report column, using the last previously-unused reserved
  field in TCP_burst_payload
o feat(android): persist test config (host/port/toggles/CCA/etc.) across
  app restarts, explicit subprocess cleanup on ViewModel teardown
  (previously could orphan a running test in the background if the
  Activity was torn down mid-test), a connect watchdog that gives up
  after a few seconds instead of waiting on the OS's own TCP SYN-retry
  timeout for an unreachable host, live "Connecting..." progress plus a
  red connect-failure indicator, and a "Reset to defaults" action
o fix(android): Port field snapped back to its old value when cleared to
  retype it; Start button now also validates port/duration/interval
  ranges before enabling instead of only checking host non-blank
o fix(android): parallel-streams (-P) temporarily removed from the UI -
  RTT/bytes-in-flight only reflect one stream's TCP_INFO and don't work
  correctly yet with more than one; revisit together later
o feat: in --reverse mode, the server relays its own bytes-in-flight/rtt
  back to the client (new read-side enhanced report/CSV columns), since
  the client only receives data and can never observe these sender-side-
  only TCP_INFO stats locally otherwise; uses the same otherwise-unused
  TCP_burst_payload reserved fields --trip-times already rides on
o fix: the above reverse-mode relay never sampled a real value when
  --tcp-write-prefetch was also active, since Client::Run() dispatches to
  RunWriteEventsTCP() (a separate write loop) instead of RunTCP() in that
  case, and only RunTCP()'s burst branch had the new sampling call; every
  --reverse + --tcp-write-prefetch client (e.g. this app's own default)
  got a permanent "not valid" sentinel instead of real data
o feat: --with-max-threads-per-client=N configure flag caps concurrent
  traffic threads per client IP on the server (0/default is uncapped),
  enforced independently of -P/Listener::mCount
o fix: connection-setup summary line's icwnd value was printed with no
  unit at all (e.g. "icwnd/mss/irtt=213/1448/22"); it's actually KB, same
  as the periodic enhanced report's cwnd field which already shows a K
  suffix - now labeled consistently (e.g. "icwnd/mss/irtt=213K/1448/22")
o feat: transfer-ID prefix now includes the source IP when --permit-key
  is in use, since all clients share the same static permit-key value and
  couldn't otherwise be told apart in the server's own output
o fix: -B/-c IPv6 addresses with link-local scope (fe80::/10 unicast or
  ffx2::/16 multicast) failed bind()/connect() with EINVAL/EADDRNOTAVAIL
  because the %<iface> device suffix was only ever used for the later
  multicast group join, never applied to sin6_scope_id on the address
  itself; both server bind and client connect now resolve the interface
  via if_nametoindex() and set the scope id
o add make check tests t18_ipv6_mcast_bind and t19_ipv6_mcast_client_scope
  covering the above fix
o fix: macOS/BSD server multicast bind emitted a spurious "Invalid argument"
  SO_BINDTODEVICE warning (t18 regression); that belt-and-suspenders call is
  Linux-specific (sk_bound_dev_if workaround for inet6_bind()) and is now
  gated to __linux__, since BSD/Darwin's bind() already honors sin6_scope_id
  and SO_BINDTODEVICE is declared there but not functionally implemented
o --send-delay replaces deprecated --tcp-tx-delay; works for both TCP and UDP
o SO_TXTIME socket support for UDP with --send-delay
o --omit support to exclude initial intervals from reporting (TCP only)
o --set-rand-seed to set the random number generator seed for reproducible -b variance
o TCP -b 0 support for unlimited (capacity-seeking) rate
o --enable-professional umbrella configure flag with community/professional versioning
o Markov chain traffic pattern support (experimental, client and server side)
o mBuf grows automatically when the peer requests a larger packet size (reverse mode)
o port range displayed in client settings report when used
o show first packet sequence number when greater than one in connect message
o default UDP read buffer size increased to 128 KBytes
o UDP L4S: CE (Congestion Experienced) bit counting on both client and server
o UDP L4S: CE duration output on client
o UDP L4S: capacity-seeking behavior is now the default for --udp-l4s and --reverse
o UDP L4S: -b rate limiting now supported with --udp-l4s (ticket 338)
o UDP L4S: ECT-1 set on first UDP packet (ticket 369)
o UDP L4S: client timeout and L4S_PKT_FIN handling
o UDP L4S: autoconf/automake support for --enable-udp-l4s / --disable-udp-l4s
o --trip-times and --tcp-write-times now auto-enable enhanced (-e) reporting
o --sendmmsg #<1-1024> (client) and --recvmmsg #<1-1024> (server) to batch UDP datagrams
  via sendmmsg(2)/recvmmsg(2), reducing per-datagram syscall overhead at high packet rates
  (Linux only, requires HAVE_SENDMMSG/HAVE_RECVMMSG from autoconf)
o Write and Read columns report actual sendmmsg/recvmmsg syscall counts (not datagram counts)
  when mmsg is active; ratio of PPS to Write/Read gives average datagrams per syscall
o fix --sendmmsg and --recvmmsg to use required_argument so space-separated values are
  accepted (e.g. --recvmmsg 8); previously the count was silently ignored
o --sendmmsg/--recvmmsg now treat a value of 0 (or negative) as "use the regular
  msg API instead of mmsg", rather than silently substituting 8
o fix prevPacketTime tracking in RunUDPBurstMMSG so interval PPS is reported correctly
o fix ReadCnt increment to use isRcvMMsgs flag so Read column counts syscalls not datagrams
o fix WritePacketID type error in RunUDPL4S (pass &mBuf_UDP->seqno_ts, not mBuf_UDP)
o add make check test t17_udp_mmsg for sendmmsg/recvmmsg (skips when not compiled in)
o five correctness bugs fixed in histogram.c
o three buffer overrun bugs fixed in Settings, Thread, and igmp_querier
o strncpy bound exceeded fix in test_permit_key
o memory leaks fixed in Settings_Destroy()
o fix heap buffer overflow: --tcp-cca/--tcp-congestion/--working-load-cca
  names longer than 32 chars overflowed the wire header's fixed-size CCA
  field; now bounded and truncated with a warning
o fix client test-exchange header length decode using too-narrow a mask,
  which could silently wrap a header length over 255 bytes instead of
  rejecting it
o UDP reverse mode: skip Condition_TimedWait on receive (prevents spurious hang)
o UDP reverse mode: use local address as flow key in active_hosts
o reporter thread: hold ReportCond mutex when signaling on thread exit
o listener: hold mutex when setting consumerdone; add missing Condition_Lock
o packet ring: add acquire/release memory ordering to the lock-free producer/
  consumer index handoff, fixing a potential data race on weak memory-model
  architectures (ARM, RISC-V, POWER); no effect on x86
o packet ring: fix off-by-one in the enqueue full-check that let a producer
  outrunning the reporter thread silently overwrite unread reports instead
  of blocking, with no warning
o packet ring: batch producer wakeups via a low-water mark instead of
  signaling only when fully empty (could stall an already-resumable
  producer for up to a second) or on every dequeue (thrashes the condvar
  and the ring's cache lines); cuts signaling ~180x under sustained load
  with no added latency
o packet ring: fix a busy-spin (hang) when the ring fills with no consumer
  condition variable to wait on (-U/--single_udp mode); now drops the
  newest report with a rate-limited warning instead of spinning forever
o Windows: fix WSAECONNRESET after ICMP port unreachable on UDP
o honor --txstart-time for server-reverse sender for both UDP and TCP (ticket 390)
o ticket 389: fix compilation failure with --configure --disable-ipv6
o ticket 376: refactor UDP test exchange with --reverse for correct -P > 1 behavior
o fix small window size warning to apply to TCP client only
o fix mBuf size regression with --reverse
o fix minsize check for --trip-times to apply correctly to both TCP and UDP
o fix UDP --trip-times with small (64 byte) packets
o fix precision in --sum-only trip-time outputs
o remove %g from percentage outputs to avoid unwanted exponential formatting
o fix omit accounting with partial intervals
o volatile qualifier added to sInterrupted for correct signal-context visibility across threads
o fix recvn with MSG_DONTWAIT to distinguish peer close from no-data
o inet_pton4 and inet_pton6 converted from K&R to ANSI C function definitions
o buildroot linux compile fix (ticket 342)
o do not default bounceback to one-second interval reporting
o automake support for interleaved assembly listings; cleaned by make clean
o simplified version string format: "iperf VERSION (DATE) THREADS (BRANCH)"
o remove unused ax_create_stdint_h; move Socklen_t discovery to dast.m4
o fix: server-side guard against unrealistic peer-supplied mBufLen in UDP v1
  header and BounceBack bbsize/bbreplysize; oversized values now emit a
  warning through the Reporter thread and abort the connection instead of
  attempting a huge allocation
o fix: UDP v1 header packet size guard rejects truncated packets (e.g. port
  scanner probes) before any header field is accessed
o fix: unrealistic mBufLen in UDP v1 header now correctly aborts the
  connection; previously the warning was emitted but processing continued,
  consuming an extra transfer ID for a reverse client that would never connect
o fix: UDP v1 header mPort values exceeding 65535 are rejected as garbage
  rather than silently truncated by the unsigned-short cast
o fix: apply_client_settings_udp diagnostic output now routed through the
  reporter thread via PostReport/InitStringReport instead of direct
  fprintf(stdout) from the listener thread
o fix: MCAST_JOIN_GROUP calls in iperf_multicast_api.c passed
  sizeof(struct group_source_req) instead of sizeof(struct group_req),
  over-reading the stack-allocated request and risking EINVAL on
  platforms that validate optlen exactly; also fixed SSM v6 join casting
  the group/source pointers as sockaddr_in instead of sockaddr_in6
o fix: Listener socket fd leaks on rejection paths - the UDP peer/IPv6
  mismatch path only closed the accepted socket for TCP, and the L2
  checks setup failure path didn't close the socket (or the dropped
  AF_INET socket) at all; L2_setup() also leaked the original socket
  when the new AF_PACKET socket() call failed
o refactor: Listener::L2_setup() converted to single-exit (goto DONE)
  to match apply_client_settings_udp/tcp, centralizing cleanup instead
  of duplicating it across each early return
o fix: iperf_formattime() used gmtime()/localtime(), which return a
  pointer into a shared static buffer (glibc shares it between the two
  calls); iperf_formattime() is called directly from per-connection
  threads (e.g. TCP connect error/timeout reporting), so concurrent
  -P > 1 failures could race and corrupt the formatted timestamp; now
  uses gmtime_r/localtime_r (gmtime_s/localtime_s on Windows, since
  MinGW/MSVC only expose the POSIX _r variants behind
  _POSIX_THREAD_SAFE_FUNCTIONS, which this project doesn't define)
o fix: mingw/Windows cross-compile breakage - server->mSockDrop was
  referenced unconditionally in Listener::Run() but only exists in
  thread_Settings under HAVE_LINUX_FILTER_H && HAVE_AF_PACKET
  (Linux-only AF_PACKET/BPF L2 support); verified against a real
  x86_64-w64-mingw32 configure + full link
o fix: three printf/format-string argument-count mismatches between
  Locale.c and ReportOutputs.c (these can never be caught by -Wformat
  since the format strings are never literals at their call sites):
  report_l2statistics was missing a trailing %s so the --omit
  indicator was silently dropped; the !(HAVE_TCP_STATS) variant of
  report_write_enhanced_write_format had a stray unfed %s (a leftover
  netpower field with no valid RTT data source on that build path,
  verified via a real x86_64-w64-mingw32 build) that misaligned every
  argument after it; report_peer_dev's !HAVE_IPV6 branch passed local
  address twice, shifting two port numbers into %s slots (verified via
  a real --disable-ipv6 build with --permit-key -e -B ip%dev)
o add make check test t20_tcp_write_times for --tcp-write-times output
o refactor: remove report_connection/report_settings/report_statistics/
  report_serverstatistics typedefs and their connection_reports[]-style
  extern array declarations from Reporter.h; unreferenced dead code
  from an apparently-abandoned array-dispatch design
o fix: three more printf argument-count mismatches, found by
  t/check_report_format_strings.py (see below): udp_output_read_triptime_isoch's
  suppressed-isoch branch passed a spurious extra pps-shaped argument that
  bumped the --omit indicator off the end of the vararg list;
  udp_output_sumcnt_enhanced was missing its pps argument entirely, feeding
  the omit-indicator string into a %8.0f slot and leaving the final %s
  reading uninitialized memory; tcp_output_write_bb_csv's !HAVE_TCP_STATS
  current-interval branch supplied 4 tcpstats placeholder zeros instead of
  5, one fewer than its "final" sibling, shifting cntTxBytes/cntRxBytes/rps
  into the wrong CSV columns
o add t/check_report_format_strings.py and make check test
  t21_lint_report_formats: cross-checks printf/fprintf/snprintf argument
  counts against Locale.c's report_* format-string %-specifiers, across
  every relevant #if/#ifdef build variant (by running the real preprocessor
  once per variant, including a real out-of-tree mingw ./configure when
  a cross-compiler is available, rather than approximating the
  preprocessor by hand); this class of bug is invisible to -Wformat since
  these format strings are never literals at their call sites, and is
  what found all of this release's printf argument-count fixes
o add make check tests t22_csv_output, t23_isochronous, t24_bounceback,
  t25_sum_report, and t26_udp_l4s (skips when not compiled in) covering
  output_handler paths (CSV, isochronous, bounceback, sum reports,
  UDP L4S) that previously had no test coverage at all
o t26_udp_l4s found a real bug (filed on the tracker, not yet fixed): the
  UDP client's relayed "Server Report" summary can show a garbage huge
  Lost count under --udp-l4s -e; the server's own directly-printed
  output for the same run is correct, so this looks like the client's
  final-ack-wait read() consuming a different/stale packet rather than
  the server's actual stats ack. t26 works around a related test-harness
  false positive: the server's best-effort final-ack retry budget
  (write_UDP_AckFIN) can legitimately warn "ack of last datagram failed"
  under this same bursty timing even when the test completes fine, which
  used to trip run_iperf's blanket failure grep in base.sh; t26 now does
  its own targeted output checks instead of using run_iperf
o fix: three bugs found in a review of Client.cpp -
  1) AwaitServerFinPacket() read the server's final stats relay packet
     with a hardcoded 1470-byte bound, but thread_Settings::mBuf is only
     guaranteed to be MINMBUFALLOCSIZE bytes, which could be smaller for
     small -l values (e.g. -l 64), risking a heap buffer overflow if an
     oversized datagram landed on the socket during that window; fixed
     by tying MINMBUFALLOCSIZE's floor to the same kDefault_UDPTxBufLen
     constant the read uses, removing Client.cpp's redundant local
     MAXUDPBUF define so the invariant can't drift apart again
  2) myWriten()'s !HAVE_TCP_STATS variant ignored its inSock/inBuf
     arguments, always writing from mySocket/mSettings->mBuf instead;
     harmless at 3 of 4 call sites (which already passed those same
     values) but corrupted TCP bounceback's partial-write retry, which
     writes from an advancing mSettings->mBuf+write_offset
  3) AwaitServerCloseEvent()'s `rc = recv(...) > 0` parsed per C/C++
     precedence as `rc = (recv(...) > 0)`, collapsing rc to 0/1 and
     making the `if (rc < 0)` error warning permanently dead code
     (loop/close-detection behavior was unaffected)
o fix: Server::ClientReverseFirstRead()'s non-trip-time fallback set
  sent_time.tv_sec twice (a copy/paste typo) instead of setting
  tv_sec then tv_usec, clobbering tv_sec with a microseconds value and
  leaving tv_usec unset; every consumer of sent_time.tv_sec is gated
  on isTripTime, which is false whenever this branch runs, so currently
  dormant, but a real landmine for any future/untraced reader
o scale the reporter consumption-detector's delay proportionally to the
  packet-ring shortfall (REPORTERDELAY_MIN/MAX bounded 1000-16000us)
  instead of always sleeping the fixed 16000us duration
o fix: SetSumHandlers()'s TCP client branch checked isSumOnly() in a
  standalone if, not chained via else-if into the following
  isBounceBack/isFullDuplex/isEnhanced/else block, so the sum-only
  output handler was silently overwritten right after being set
  whenever none of those three conditions applied (e.g. --sum-only
  without -e, or --sum-only -y C without -e, on a TCP client)
o fix: reduce Linux hrtimer slack (via PR_SET_TIMERSLACK) for traffic
  threads not using -z (realtime scheduling); the default 50us slack
  systematically biases delay_loop()'s clock_nanosleep()-based IPG
  pacing delays long, which is otherwise uncorrected on Linux since
  the Kalman fallback in compat/delay.c only compiles in when
  clock_nanosleep() is unavailable
o build: promote reporter function-pointer vector signature mismatches
  (output_handler, transfer_protocol_handler, etc. in Reporter.h) from
  an easily-missed warning to a hard build failure via
  -Werror=incompatible-pointer-types
o fix: all 26 make check t/*.sh scripts hardcoded #!/bin/bash -e; on
  FreeBSD (confirmed on OPNsense 14.3-RELEASE) there is no /bin/bash in
  the base system, and even after installing the bash package it lands
  at /usr/local/bin/bash, so every script failed to exec at all
  (exit 127), failing all 26 tests uniformly regardless of iperf itself
  working correctly; switched to #!/usr/bin/env bash (PATH-resolved)
  with a separate `set -e` line
o fix: professional builds' "(compiled by X)" version tag printed
  "(compiled by )" with nothing in the parens on FreeBSD/bmake, since
  IPERF_BUILD_USER was filled in via make's $(shell whoami ...), a GNU
  Make extension that silently expands to empty under BSD make; moved
  the whoami/id -un lookup into configure.ac (portable /bin/sh at
  configure time) and baked it into config.h instead
o TCP client enhanced CSV output (-c ... -e -y C) now includes
  tcppktsinflight/tcpbytesinflight columns when HAVE_TCP_INFLIGHT, mirroring
  the InF(pkts) already shown in the human-readable enhanced write report;
  previously only available in text form, not CSV
o fix: TCP client enhanced CSV's tcppktsinflight/tcpbytesinflight columns
  showed a plain 0 (indistinguishable from a genuine zero-in-flight
  measurement) on any path where inflight data isn't actually available
  (missing HAVE_TCP_INFLIGHT, sum reports, or builds without TCP_INFO);
  now consistently -1, matching this same function's existing convention
  for other not-applicable fields (writecnt/writeerr under !HAVE_TCP_STATS)

2.2.1 (as of Oct 26th, 2024)
------------------------------
o man page updates
o support (alpha level) for --udp-l4s (linux only, requires ./configure --enable-udp-l4s)
o buffer overflow fixes (multiple places)
o support of udp summing enhanced outputs in sum reports with -P > 1
o SETABSTIME fix
o connection report errors ouput to stderr (vs stdout)
o fixes to --connectly-only regressions
o remove --tcp-cca and --reverse restriction
o fixes to ip_tos and cmsg
o remove setsockopt for ip tos, use sendmsg and ancillary messages instead
o sample tcp_info in the middle of a -i interval
o multiple fixes for TCP_TX_DELAY
o don't autoset --tcp-write-prefetch with --trip-times, warn instead
o Add Android NDK example, add mingw64 example
o set smallest prefetch to 256K
o print wait time on server side with --tx-starttime
o fix header code #if mismatch of (HAVE_DECL_SO_TIMESTAMP) && (HAVE_DECL_MSG_CTRUNC) per ticket 328
o fix client side bb summing
o fix format error in timestamps
o 1) Support CSV for isochronous, both UDP and TCP 2) Reorganise CSV report assignement to be more logical.
o support milliseconds and microseconds with iperf_formattime, also make sure the leading zeros are printed per the field width
o fix multiple pps regressions
o csv patches per ticket 320 and 322
o add per direction byte counts with bounceback on client (server side code yet to be done)
o fix summing init code per ticket 324
o tcp working load should use full capacity seeking behaviors
o fix csv compile breakage on MAC
o use append for --ouput vs w, ticket 321
o use --ipg units of seconds
o fix settings calculations when -b is given for --burst options
o improve port range / traffic thread count (-P) warning
o add transferid to recvmsg warning
o compute packet pps accounts for interval crossing using that timestamp vs packet timestamp
o use object setnow() method to set lastPackeTime in first packet delay
o pps calculation needs to include partial gap value with IPGsum ahead of PPS output
o minor fixes for DEBUG_PPS support
o fix initial udp write delay and reporting
o Rerun autoconf
o Remove unused ax_create_stdint_h
o move Socklen_t discovery entirely into dast.m4
o Remove unused DAST_REPLACE macros
o fix udp regression with high pps, sosndtimer needs to be set, don't use write select
o Remove obsolete web100 makefile support
o fix for windows enhanced writes with summing
o don't mix typecast with format specifiers, use %ld for (long) typecast
o use typecast for time_val seconds for portability, fix windows 64 cross compile
o Set default compiler and load flags without overiding user choice.
o Move packet and tuntap checks later so they don't execute before compiler checks.
o Use HAVE_GETIFADDRS instead of HAVE_IFADDRS_H for Android and any other OS that has the header file but might not enable the actual feature
o Remove code that is unused and deprecated
o Update defuns to comply with autoconf-2.72
o use netinet vs linux for ip.h and udp.h, add configure.ac checks for the header files, update dscp.h for new defines not in netienet per WFA/WMM inputs
o disable SO_REUSEPORT for server side
o various fixes in ReportOutputs including divide by zero test, segv fixes, and spacings
o fix for UDP 64b seq number detection per packet header
o fix configure.ac to support Win64 compiles
o ticket 314: eliminate the udp accept race between listener and server thread using a conditional signal
o ticket 313: remove support for configure --enable-static-bin, never worked and only misleads
o ticket 312: regression, fix for segv in UDP summing enhance outputs
o ticket 311: histogram worst timestamp needs proper formatting, didn't print leading zeros for the usec portion
o ticket 310: regression on client setting reports, too many with things like -P > 1
o ticket 309 (regression): udp summing is broken. Need to reset sum event counters after interval reports
o ticket 308: -P should be order independent
o ticket 305: add support for --skip-rx-copy w/tcp. Set recv flags to MSG_TRUNC when the payload isn't needed by iperf
o ticket 157 (regression since 2.1.4): remove AM_CONDITIONAL for TUN & TAP. Move the AF_PACKET AM_CONDITIONAL to its own scope. Rerun autoreconf automake
o fix output for tos on server side
o remove settings report for client threads when P > 1
o fix timeval outputs to use %06ld for usecs

2.2.0 (as of April 9th, 2024)
------------------------------
o new ./configure --enable-summing-debug option to help with summing debug
o select ahead of writes slow down UDP performance. support ./configure --disable-write-select
o support fo -b 0 with UDP, unlimited load or no delay between writes
o support for --sync-transfer-id so client and server will match the ids and give a remap message
o support --dscp command line option
o support for application level retries and minimum retry interval of the TCP connect() syscall via --connect-retry-time and --connect-retry-timer, repsectively
o support for --ignore-shutdown so test will end on writes vs the BDP drain and TCP close/shutdown, recommended not to use this but in rare cases
o support for --fq-rate-step and --fq-rate-step-interval
o CCAs per --tcp-cca, --tcp-congestion, etc neeed to be case sensitive
o support for both packets and bytes inflight taken from tcp_info struct amd pkt calc of (tcp_info_buf.tcpi_unacked - tcp_info_buf.tcpi_sacked - tcp_info_buf.tcpi_lost + tcp_info_buf.tcpi_retrans)
o man page updates and -h to reflect new options, better descriptions
o lots of work around summing with parallel threads, new implementation based on interval or slot counters, hopefully should work reliably
o --bounceback tests are much more reliable and robust
o Improve event handling around select timeouts, helps with larger -P values and summing
o use the getsockopt IP_TOS for the displayed output, warn when set and get don't match
o better tos byte output, include dscp and ecn fields individually
o better tos setting code for both v6 and v4, so they behave the same around checks and warnings
o much better NULL events to help with reporter processing even when traffic is not flowing
o support for a new string report
o python flows work around CDF based tests
o rate limit fflush calls to a max of one every millisecond or 1000 per sec
o remove superfulous fflush calls
o reports when P = 1 and --sum-only need sum outputs
o enable summing with --incr-dstip
o add macro TIME_GET_NOW to set a struct timeval in a portable manner
o code readability improvements with enums, bools, etc.
o fix for TCP rate limited and -l less than min burst size
o only use linux/tcp.h when absolutely needed, otherwise use netinet/tcp.h
o print bounceback OWD tx/rx in interval reports
o add flows Makefiles for tarball or make dist-all
o support interval reports for bounceback histograms
o support for TCP working loads and UDP primary flows, including UDP isochronous, per ticket 283
o fix working-load with isoch so working-load streams are capacity seeking
o exit when CCA not supported or read of the current CCA doesn't match requested CCA
o add more make check tests
o add support for omit string (omit code not ready for this release)
o pyflows qdisc settings and outputs
o add first send pacing with --tx-starttime so listener threads udp_accept has time to perform udp_accept() between the client threads
o adjust the sender time per the client delay and the client first write, i.e. subtract out this delay in the calculations
o fixes for small packets and --tx-starttime
o use more modern multicast socket options (now in src/iperf_multicast_api.c)
o warn on bind port not sent with --incr-srcport
o display fq-rate values in outputs when --fq-rate is used
o add support for --test-exchange-timeout
o fixes around wait_tick
o add support for TCP_TX_DELAY via --tcp-tx-delay <val ms> option on both client and server
o pass the CCA from client to server
o support burst-size with different write sizes and don't require --burst-period
o output traffic thread send scheduling error stats in final ouput
o output clock unsync stats with --bounceback
o add warn message on MSG_CTRUNC
o UDP select fixes
o enable TCP_NOTSENTLOWAT and set to a default small value with --tcp-write-times
o default histogram max binning to 10 seconds
o add a max timestamp to histogram outputs so user can find packets in pcaps or equivalent
o autoconf change for struct ip_mreqn
o print errno on writen fail

2.1.9 (as of February 13th, 2023)
------------------------------
o fixed traffic setitimer to use uintmax_t vs int, supporting large values
o --bounceback officially supported (including Windows) for repsonsiveness test scenarios
o deprecated --bounceback-congest introduced in 2.1.8, replaced by --working-loads
o --working-loads support generalized; works with --bounceback, --connect-only & --burst-period
o default TCP_NOTSENT_LOWAT with the --working-loads concurrent traffic
o add support for GMT time formatting via --utc option
o --trip-times will auto set TCP_NOTSENT_LOWAT
o CSV output fixes for reverse
o CSV output regressions fixed per sum outputs using negative transfer ids
o CSV output support with --enhanced
o Fix to isoch wait_tick with Windows
o fix support for --txstart-time with --bounceback
o Add support for summing histograms in histogram sum outputs
o Multiple sum report fixes per threading & needing mutex protections
o Jitter packet IPG calcluations ignore inter frame gaps
o Isoch jitter output to use running value vs sampled value
o Add support for --jitter-histograms
o man page content updates
o output isoch scheduling errors at end of isoch run
o PRIdMAX fix for ARM systems
o better work around in isochronous with Windows per early return of WaitForSingleObject()
o fix SO_BINDTODEVICE regression
o fix v6 source port parsing with -B and brackets
o fix malloc error with --hideips
o fixes for rate limited TCP with --trip-times
o add support for TCL_NOTSENT_LOWAT with rate limited TCP
o permit key now supports -P using listen() with a backlog, no longer single thread limited
o fixes for zero valued permit-key
o fixes for multiple permit-key regressions
o fix token bucket delay with TCP await write
o fix isMulticast test for ipv4 - previous logic indicate true for 240.x.x.x which is not multicast
o fix regression on jitter calc - starts on second transit time
o add cmsg for loop with UDP rx timestamp, cmsg processing best to use loop w/test
o use stdout and exit(0) for -h and -v (vs stderr and exit(1))
o add python facetime scripts
o Fix single thread compile breakage
o fix windows cross compile
o multiple spelling error fixes in comments and man page


2.1.8 (as of August 5th, 2022)
------------------------------
o Add support for --bounceback to perform a repsonsiveness test (see man page for other options)
o add support for working loads with --bounceback
o Fix to wait_tick with Mac OS X
o Various python pyflows commits
o add support for client side tcp-write-time histograms and mean/min/max
o add support for human readable dscp or -T values (see man page)
o udp_accept no longer accepts packets from a previous run as a new connection, this can occur with long network delays
o multiple isoch bug fixes for both UDP and TCP
o isoch server provides mean/min/max/stdev for both frames and packets
o UDP max MTU discovery, requires configure.ac will support --enable-discover-defaultlen prior to compile


2.1.7 (as of April 5th, 2022)
----------------------------
o Support for tcp bounceback test
o Code clean up
o Regression fixes (see git-log)

2.1.6
-----
o Fix to major 2.1.5 regeressions

2.1.5 change set as of (December 3, 2021)
--------------------------------------
o fix some HAVE_IPV6 conditional changes
o fix SO_TIMEOUT regressiony
o ren sockets.c to socket_io.c
o fix compile breakage per abs() returning an int instead of float
o support for gettcpinfo on Mac OS X (tested on both M1 and x86 silicon)
o move setsock_blocking from sockets into PerfSocket.cpp
o don't require -V for v6, instead try v6 when v4 hostname lookup fails, client only
o add assert in writen
o add tcp RTT variance to client output
o use setsockopt to get the nagle status
o show Nagle and TOS settings on client
o more on connect-only testing
o sample and output the initial rtt and cwnd in the connect report
o fix multiple fullduplex regressions
o fix for HAVE_TCP_STATS in configure, then linux compile
o writen can have more than one write, fix accounting when this occurs
o fix tos with --reverse and --full-duplex
o add support for --tos-override <value> on server
o add support for  --tcp-drain, add mmm stats, histograms - experimental feature
o multiple man page updates
o fix partial histogram print to not show (f)
o some new scripts in python flows
o fixes to incr-srcport
o fixes for --incr-dstport
o fix regression on very first UDP packet having transit latency of zero
o fix --reverse and --isochronous when --trip-times not set
o fix client_init regression, pull out tcp_shutdown
o fix reporter startup race and one second delay by setting the threads ready predicate and issuing the signal under a lock
o fix first send accounting for small -n
o fix configure.ac to use '=' instead of '=='


2.1.4 change set as of (August 12, 2021)
--------------------------------------
o fix TCP isoch regression
o fix regression in UDP header exchange for tests like --reverse
o Add support for TCP_NOTSENT_LOWAT vi --tcp-write-prefetch and select() before write()
o Add support for TCP_WINDOW_CLAMP
o Rework recvn() and writen() for when SO_SNDTIMEO and SO_RCVTIMEO are enabled
o Add support for --histograms on select with --tcp-write-prefetch
o Add support for bind to device on the listener, i.e. iperf -s -i 1 -e -B 0.0.0.0%eth0, will only accept/receive on the eth0 interface
o Add support for virtual/tap interfaces
o Add support for --hide-ips (don't show the ip addresses in the report outputs)
o Fix units of -pps with --reverse, --fullduplex, -r and -d
o Remove use of MSG_PEEK by moving the mBuf buffer from client/server object to settings context
o Use MSG_WAITALL in recvn (collided with MSG_PEEK on Windows)

2.1.3 change set as of (July 13, 2021)
--------------------------------------
o relax cli errors a bit to WARN instead of ERROR
o fix TCP read fatal error macro
o fix UDP server to not fatal error on EINTR, use macro
o handle and warn on failed read of tcp test flags
o redesign of tcp retry (2.1.2 fix was incomplete)
o thread exit signals reporter thread condition var for timely exits of the tool


2.1.2 change set (as of June 25th, 2021)
----------------------------------------
o fix TCP retry regression per interval reporting

2.1.1 change set (as of June 23rd, 2021)
----------------------------------------
o isochronous bug fix
o -P and -B src port will increment for unique quintuple
o support for port ranges, e.g. -p 6000-6008
o double free fix per memory corruption when -l is less than 244
o don't use pthread_join on the client --reverse, symptom hung client
o fixes for --trip-times and small 64 byte packets
o udp fail on reverse should exit
o support for low duty cycle bursts (--burst-period and --burst-size)
o final report fixes
o full duplex ouput fixex
o support for --incr-scrip
o multicast setsockopt fixes


2.1.0 change set (as of January 5th, 2021)
----------------------------------------
o scaling improvements for -P, i.e. improved support for large numbers of traffic threads
o major code refactoring (see doc/DESIGN_NOTES) for maintainability, extensibilty, performance, scaling, memory usage
o support for full duplex traffic using --full-duplex
o support for reverse traffic using --reverse
o support for role-reversal character of asterisk in the transfer id
o transfer id now an incrementing integer and no longer the socket id
o support for TCP connect only tests with --connect-only
o isochronous support compiled in by default, must use config to disable
o support --isochronous for both UDP or TCP traffic to simulate video streams
o support for low duty cycle traffic patterns via --burst-period and --burst-size
o use of clock_nanosleep when supported to schedule isochronous burst starts, otherwise use nanosleep delay
o support for --trip-times indicating the client and server clocks are synchronized to an accuracy sufficient, note: consider the use of precision time protocol as well as ask your data center to provide access to a GPS disciplined reference time source
o support for --trip-times with -d and -r bidirectional tests
o output TCP connect times (3WHS) in connect reports
o support for application level tcp connect retries via --connect-retries n
o rate-limited options of -b and --fq-rate supported for unidirectional, full duplex and reverse traffic
o reporter thread designed to automatically cause packet reports to aggregate - mitigating and hopefully removing thread thrashing
o support for frame or burst based reporting or sampling vs time based via -i [f|F] (experimental)
o support for UDP traffic only from client to server with --no-udp-fin
o support for write to read latencies (UDP and TCP) with --trip-times
o support for sum only outputs with --sum-only
o support for little's law calculations in --trip-time outputs
o support for --txstart-time <epoch-time> to schedule client traffic start, timestamp support microseconds, e.g. unix $(expr $(date +%s) + 1).$(date +%N)
o support for --txdelay-time to insert delay between TCP three way handshake (3WHS) and data transfer
o support for --no-connect-sync which disables transmit traffic start synchronization when -P is used, defaults to synchronized
o option of --full-duplex implementation uses a barrier on the client side to synchronize full duplex traffic
o no limits to group sum reports, i.e. all clients will get its own sum report per a server
o improved report timestamps, e.g. end to end or client and server based timestamps with --trip-times
o improved settings messaging
o improved messaging for --tcp-congestion or -Z
o re-implemented -U for single UDP server with minimal threading interactions
o re-implemented -1 or --singleclient where server will serialize traffic runs
o warning message if the test were likely CPU bound instead of network i/o bound
o fix the case when -P <value> is set on the server such that summing output is displayed
o multicast listener will autoset -U (single server), e.g -P > 1 not supported for multicast
o multicast listener no longer busy drops multicast packets during traffic test, i.e. only server thread receives them
o immediate bail out on mutually exclusive command line options
o getaddrinfo bug with -static linkage workaround and DNS lookup one time in setttings context vs twice in Settings and client traffic thread
o fix -o or --output using freopen to redirect stdout and stderr to a file
o support for --local-only which sets SO_DONTROUTE on a socket to limit traffic to local hosts (default is off)
o support compile time option of --local-only to set on by default via ./configure --enable-default-localonly
o support for date and time of in connect messages,
  e.g. [  0] local 192.168.1.108%eth0 port 5001 connected with 192.168.1.62 port 36724 (MSS=453) (sock=5) on 2020-12-22 19:43:42 (PST)
o support for feature of --permit-key and permit-key-timeout (defaults to 20 seconds.) The permit-key must match for the server to accpet the client's traffic. It also sets the transfer id. TCP only.
o support for experimental feature of --near-congestion (tcp only)
o man page updates with examples
o tested with 1000's of traffic streams, WiFi, 10G and 100G

2.0.13 change set (as of January 22, 2019)
----------------------------------------
o Set the listening socket backlog to a large value, let os control the max, better operations with FreeBSD
o Fix breakage to -r and -d options
o Fix so ctrl-c works with -r and -d
o Fix ctrl-c on server to have a graceful exit including print report
o Fix freebsd compile breakage per multicast support
o The UDP final server report may be larger than the client's -l length, fix the client to read maximum packet length for that instead of using -l as the read size
o Minor code clean up around write errors
o Added netpower to TCP client enhanced output (throughput/RTT)
o Display TCP client connect time in the connected to message
o use IPV6_TCLASS for ipv6 (-V) dscp/tos (-S) if available
o Add support for --txstart-time <value> where value is epoch/unix format, e.g. iperf -c 192.168.100.33 --txstart-time 1536090358.515
o Add support for socket option SO_MAX_PACING_RATE using --fq-rate
o Add configure support for --enable-fastsampling, allowing 100 microsecond report intervals
o Add support for --trip-time on the client.  Measures client's 3WHS done to client's fin+fin-ack, reported on the server requires -e and synchronized clocks
o Support for 64 bit sequence numbers on by default, no longer requires --udp-counters-64bit, inter-operates with 2.0.5 32 bit sequence numbers
o Integer fixes for 64 bit, int max, printf,  etc.
o Obsolete the need for include/slim_headers.h
o Update man page to have some example usages

2.0.12 change set (as of June 25th 2018)
----------------------------------------
o Change the unicast TTL default value from 1 to the system default (to be compatible with previous versions.)   Multicast still defaults to 1.
o adaptive formatting bug fix: crash occurs when values exceed 1 Tera.  Add support for Tera and Peta and eliminate the potential crash condition
o configure default compile to include isochronous support (use configure --disable-isochronous to remove support)
o replace 2.0.11's --vary-load option with a more general -b option to include <mean>,<stdev>, e.g. -b 100m,40m, which will pull from a log normal distribution every 0.1 seconds
o fixes for windows cross compile (using mingw32)
o compile flags of -fPIE for android
o configure --enable-checkprograms to compile ancillary binaries used to test things such as delay, isoch, pdf generation
o compile tests when trying to use 64b seq numbers on a 32b platform
o Fix GCC ver 8 warnings


2.0.11 change set (as of May 24th, 2018)
----------------------------------------
o support for -b on server (read rate limiting)
o honor -T (ttl) for unicast.  (Note: the default value is 1 so this will impact unicast tests that require routing)
o support for --isochronous traffic with optional frames per second, mean and variance uses a log normal distribution (requires configure w/-enable-isochronous and compile)
o support for --udp triggers (requires configure w/ --enable-udptriggers, early code with very limited support)
o support for --udp-histogram with optional bin width and number of bins (default is 1 millisecond bin width and 1000 bins)
o support for  frame (burst) latency histograms when --isochronous is set
o support for --tx-sync with -P for synchonrized writes.  Initial use is for WiFi OFDMA latency testing.
o support for --incr-dstip with -P for simultaneous flows to multiple destinations (use case is for OFDMA)
o support for --vary-load with optional weight, uses log normal distribution (requires -b to set the mean)
o support for --l2checks to detect L2 length errors not detected by v4 or v6 payload length errors (requires linux, berkeley packet filters BPFs and AF_PACKET socket support)
o support for server joining mulitcast source specific multicast (S,G) and (*,G) for both v4 and v6 on platforms that support it
o improved write counters (requires -e)
o accounting bug fix on client when write fails, this bug was introduced in 2.0.10
o slight restructure client/server traffic thread code for maintainability
o python: flow example script updates
o python: ssh node object using asyncio
o python: histograms in flows with plotting (assumed gnuplot available)
o python: hierarchical clustering of latency histograms (early code)
o man pages updates
o Note: latency histograms require client and server system clock synchronization.   A GPS disciplined oscillator using Precision Time Protocol works well for this.

2.0.10 change set (as of August 11, 2017)
-----------------------------------------
o clean up help and man page for -V option
o UDP IPv6 : Default the mBuf size to 1450 for the client, default the Listener/server to 1470
o Display read/write buffer size in the report header (only when -e enhanced)
o Add -S and --tos to man page and help, and format options to same
o Reject --ipv6_domain (-V) option if HAVE_IPV6 is not defined, rather than silently ignoring it.
o Clean up settings/command line parsing code including removing oder dependencies
o Use strtok (instead of strtok_r) for better portability
o Fix 'format string is not a string literal' warnings on Mac OS X
o Use SOCKET type on Windows instead of int to clean up signed/unsigned warnings
o Fix WIN32 timeout paste errors
o man page updates to describe format characters, improve BUGS sections
o Fix autoconf IPV6 check for case where configure is not called from srcdir
o Add support for v6 link local, e.g. 'iperf -V -c fe80::428d:5cff:fef7:5a73%eno1'
o Fix ClientHdrXchange when gettimeofday() is used
o Fix for -l, -W and -n to support format characters, also fix -n for -d and -r
o Fixes -for -d (dual) honoring -b on remote
o Fix for -t and -r
o Support for 64 bit sequence numbers (--udp-counters-64bit)
o Update config to support new platforms, eg --host=aarch64-linux
o Fixes to header xchange, length messages, and packing
o Server thread mbuf length checks
o remove trailing whitespace across all files in the git repository
o Support peer version detection and exchange with -X option
o Add UDP buffer size minimum checks and warnings prior to peer test exchange
o 2.0.5 interop testing and fixes (assume lots of 2.0.5 servers in the field)
o Fix thread settings per hdr xhcange fixes
o Fix for -P and -d used together
o Error on client and -D
o Fix local port binding on client to support both v4 and v6
o Fix more on use clock_gettime() over gettimeofday() when available
o Treat ENOBUFS as transient, i.e. don't exit on this error
o Python: add flows directory and skeleton flows code using asyncio
o Python: Support flows stats in dictionary for pandas

2.0.9 change set (as of June 2016)
----------------------------------
o Apply SO_SNDTIMEO for both UDP and TCP per both -t and -i
o Server (and listener) threads won't block forever when -t set (uses select() with non-blocking accept())
o Remove need for <cmath> to improve portability
o Fix report interval bug so single threaded mode no works
o Skeleton code for Python based flows which will be use Pythnon 3's asyncio module
o configure script check for struct tcp_info code fix and uses netinet/tcp.h

2.0.8 change set (as of 1/12/2015)
----------------------------------
o Fix portability, compile and test with Linux, Win10, Win7, WinXP, MacOS and Android
o Client now requires -u for UDP (no longer defaults to UDP with -b)
o Maintain legacy report formats
o Support for -e to get enhanced reports
o Support TCP rate limited streams (via the -b) using token bucket
o Support packets per second (UDP) via pps as units, (e.g. -b 1000pps)
o Display PPS in both client and server reports (UDP)
o Support realtime scheduler as a command line option (--realtime or -z)
o Improve client tx code path so actual tx offerred rate will converge to the -b value
o Improve accuracy of microsecond delay calls (in platform independent manner)
o (Use of Kalman filter to predict delay errors and adjust delays per predicted error)
o Display target loop time in initial client header (UDP)
o Fix final latency report sent from server to client (UDP)
o Include standard deviation in latency output
o Suppress unrealistic latency output (-/-/-/-)
o Support SO_SNDTIMEO on send so socket write won't block beyond -t (TCP)
o Use clock_gettime if available (preferred over gettimeofday())
o TCP write and error counts (TCP retries and CWND for linux)
o TCP read count, TCP read histogram (8 bins)
o Server will close the socket after -t seconds of no traffic

2.0.7 change set (rjmcmahon@rjmcmahon.com) August 2014
------------------------------------------------------
o Linux only version which supports end/end latency (assumes clocks synched)
o Support for smaller report interval  (5 milliseconds or greater)
o End/end latency with UDP (mean/min/max), display in milliseconds with resolution of microseconds
o Socket read timeouts (server only) so iperf reports occur regardless of no received packets
o Report timestamps now display millisecond resolution
o Local bind supports port value using colon as delimeter (-B 10.10.10.1:60001)
o Use linux realtime scheduler and packet level timestamps for improved latency accuracy
o Suggest PTP on client and server to synch clocks to microsecond
o Suggest a quality reference for the PTP grandmaster such as a GPS disciplined oscillator from companies like Spectracom


2.0.6 change set (rjmcmahon@rjmcmahon.com) March 2014
-----------------------------------------------------
o Increase the shared memory for report headers reducing mutex contention.  Needed to increase performance.  Minor code change that should be platform/os independent