“The file copy to the NAS crawls” and “the new site-to-site link isn’t giving us what we pay for” are two tickets with the same first question: what can the network itself move, with disks, SMB and application quirks taken out of the picture? iPerf3 answers that by generating traffic in memory between two endpoints you control. It’s a small command-line tool from ESnet, BSD-licensed, and the numbers it produces are the kind a carrier or a colleague will take seriously — provided you ran the test properly.
A reminder before you start: iPerf3 saturates links by design. Run it only between hosts you own or are authorised to test, and warn whoever shares the link, especially on WAN circuits.
What you need
- Two hosts, one on each side of the path you want to measure.
- iPerf3 on both. ESnet’s official platforms are Linux, FreeBSD and macOS; most Linux distributions package it (
apt install iperf3,dnf install iperf3). ESnet publishes no binaries itself, so Windows builds come from third parties — see where to get tools safely and check hashes. - TCP port 5201 (the default) open from client to server, both TCP and UDP if you’ll test UDP.
The current release at the time of writing is 3.21. Versions from 3.16 onward run parallel streams in separate threads, which matters above a few Gbit/s; if you’re testing 10G or faster, use a recent build on both ends.
Step 1: Start the server
On the far-side host:
iperf3 -sUseful variations:
# listen on a specific interface address and a non-default port
iperf3 -s -B 10.20.0.5 -p 5301
# serve a single test then exit (handy for scripted or ad-hoc runs)
iperf3 -s -1An iPerf3 server handles one test at a time. If a second client connects mid-test it gets a “server is busy” error, so for several simultaneous tests start several servers on different ports.
Step 2: Run a baseline TCP test
On the near-side host:
iperf3 -c 10.20.0.5That sends TCP from client to server for 10 seconds, reporting every second. Read the sender and receiver summary lines at the bottom — the receiver line is the one that counts. Also look at the Retr (retransmits) column: a steady trickle of retransmits on a LAN points at errors, duplex, or buffer problems.
Step 3: Test the other direction and both at once
Asymmetric problems are common: a bad cable pair, a policer on one side, a misconfigured shaper.
# server sends to client (inbound traffic from the client's viewpoint)
iperf3 -c 10.20.0.5 -R
# both directions simultaneously
iperf3 -c 10.20.0.5 --bidirIf -R is dramatically slower than the default direction, you’ve localised the problem to one direction without touching a single switch.
Step 4: Use parallel streams and a longer run
One TCP stream is limited by window size and round-trip time. On a high-latency WAN, a single stream can underreport a perfectly healthy circuit.
# 4 parallel streams, 30 seconds, ignore the first 3 seconds of slow start
iperf3 -c 10.20.0.5 -P 4 -t 30 -O 3Rough rule: if -P 4 gives much more than a single stream, the link is fine and the limit is per-flow (window size × RTT). That’s a tuning discussion, not an ISP ticket. You can test the effect of a larger socket buffer with -w, for example -w 4M, within whatever the OS allows.
Step 5: Measure jitter and loss with UDP
TCP hides loss by retransmitting. UDP shows it. UDP mode defaults to just 1 Mbit/s, so always set a bitrate:
# 50 Mbit/s of UDP for 30 seconds
iperf3 -c 10.20.0.5 -u -b 50M -t 30The receiver summary shows jitter in milliseconds and lost/total datagrams. For voice-heavy sites, a UDP run at the expected call load is more informative than any TCP number. Push -b upward in steps; the point where loss starts climbing is the practical ceiling for real-time traffic.
Step 6: Save the evidence
iperf3 -c 10.20.0.5 -P 4 -t 60 -J --logfile iperf-wan-$(date +%F-%H%M).json-J produces JSON with per-interval data, which is easy to graph or attach to a ticket. For a quick human-readable log, drop -J and add --forceflush so each interval is written as it happens. Record which host was server and client, the time, and the exact command line — someone will ask.
Interpreting results
- Gigabit LAN: a healthy wired path typically shows a little over 930 Mbit/s of TCP goodput; protocol overhead accounts for the rest. Around 90-something Mbit/s often means a link negotiated at 100 Mbit/s somewhere.
- 10G LAN: single-stream results are often limited by the CPU or NIC offload settings of the test hosts. Try
-P 4or more before blaming the switch. - WAN: compare against the committed rate and check both directions. Run at different times of day if you suspect congestion.
Common mistakes
- Testing from a VM or laptop that can’t push the rate. A slow endpoint measures itself, not the network. Check CPU during the test.
- Wi-Fi at either end. Fine if Wi-Fi is what you’re measuring; misleading otherwise.
- Forgetting the firewall. A client that hangs at “Connecting to host” usually means 5201 is blocked or the server is bound to another address.
- UDP with no
-b. You’ll get a flawless 1 Mbit/s and learn nothing. - Reading the sender line. The sender can report more than was actually delivered; trust the receiver.
- Testing through the thing you’re testing to. If the server is a busy file server, you’re competing with production.
- Confusing iPerf2 and iPerf3. They aren’t wire-compatible and use different default ports (iPerf2 uses 5001). Use the same major version on both ends.
Where this fits
Throughput testing answers “how much”, not “where”. If throughput is poor and you suspect a lossy hop, switch to path analysis with finding where packet loss starts. If the numbers are fine but an application still misbehaves, capture the traffic with Wireshark. The iPerf3 review covers its limits in more detail, and related tools are grouped under Throughput & Packet Inspection. For continuous rather than one-off measurement between sites, see Obkio.