esnet / esnet/iperf

iperf3 single-stream low bandwidth with small message sizes (1KB, 1500B, 2000B, etc.)

Open
#1,078 14 comments 0 reactions 0 assignees Claimed by @davidBar-On View on GitHub
Dominant language
C
Stars
8.8k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Description

# Context

* Version of iperf3: iperf 3.5

* Hardware:
Nvidia ConnectX-6 Dx 100GbE
Intel XXV710 25GbE

* Operating system (and distribution, if any):
Red Hat Enterprise Linux Server release 7.5 (Maipo) - kernel 5.10.0-rc4
CentOS Linux release 8.1.1911 (Core) - kernel 5.2.9

# Bug Report
We are observing low performance with iperf3 while sending single-stream traffic with small message sizes.
Other benchmarks reports significant better results for same scenario, we assume it's Iperf3 that's limiting the performance we observe.

For example:
Running `iperf3 -c fd12:fd07::200:105 -t 10 -l1500B` we see ~7.9Gbps.
![iperf3_1stream_1500B](https://user-images.githubusercontent.com/17932324/100090710-44bc6b80-2e5c-11eb-8a3d-19f2b09da1c2.jpg)

Running the same test with 1 stream and 1500B with **iperf2** we see 23Gbps which is what we would expect with current system.
cmd: `iperf -c fd12:fd07::200:105 -V -l1500 -P1`
![iperf2_1stream_1500B](https://user-images.githubusercontent.com/17932324/100090915-95cc5f80-2e5c-11eb-9dc1-db7bc4126637.jpg)

* Expected Behavior

Iperrf3 should be able to achieve at least the same performance reported by iperf2 for a single-stream.

* Actual Behavior

Iperf3 small message size performance is low. Iperf3 is the limiting factor.

* Steps to Reproduce

Send traffic with message size 1500B (On a high performance NIC over 10/25Gbps).

* Possible Solution

I've noted iperf2 suffer a similiar issue when running with '-i' flag, might be related to the reporting/results gathering flow?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.