TCP bandwidth drops to zero

The behavior occurs on Linux 4.19.124-RT,
UDP works fine, on TCP the bandwidth drops to zero after few seconds.
Ethernet driver is virtual driver between to PCIe machines.
The driver is quiet stable, and no observed problems with software (scp, ssh, etc)
No packet drops on the Ethernet driver.

I would love to understand why iperf3 stops sending packets, is there a way to enable more debugging, or somewhere I could look to debug it further ?

* Version of iperf3:
3.7
* Hardware:
MIPS based Embedded Linux board
* Operating system (and distribution, if any):
Buildroot 20.02

* Other relevant information (for example, non-default compilers,
libraries, cross-compiling, etc.):
Cross compiler:
mips-img-linux-gnu-gcc (Codescape.GNU.Tools.Package.2019.02-01.for.MIPS.IMG.Linux.CentOS-6.x86_64.tar.gz)

# Bug Report

* Expected Behavior
Bandwidth is stable
* Actual Behavior
Bandwidth drops to zero, no packets are sent from iperf.
* Steps to Reproduce
Running on the HW