google / google/compute-virtual-ethernet-dpdk
Segmented Sends Blocking Tx Queues
- Dominant language
- C
- Stars
- 14
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
### Test Environment:
* Machine type `c3-standard-8`
* CPU platform `Intel Sapphire Rapids`
* Architecture `x86/64`
* Networking NIC type `gVNIC`
My team is currently on DPDK version `23.11` and we use segmented packets in various flows of our application. We were seeing packet leaks so I applied the patch from this [PR](https://github.com/google/compute-virtual-ethernet-dpdk/pull/8/files). That fixed a few of our flows, but introduced another error. We upgraded to version `24.07-rc4` as another test, but found that the behavior stayed the same.
### Test Program
```
dpdk-v24.07-rc4
sudo ./build/app/dpdk-testpmd -l 0-3 -n 2 -- --txpkts= --port-topology=chained --nb-cores 1 --txq 1 --rxq 1 --mbcache=512 --txd=1024 --forward-mode=txonly --stats-period 1 --tx-ip=10.0.0.1,10.0.0.2 --eth-peer=0,00:00:00:00:00:00 --tx-udp=5555
```
### Behavior
There are a variety of packet chains that can be created that will cause the queues to block.
* Any chain with a zero length mbuf will result in the queues getting blocked. E.g `--txpkts=100,0,100`
Other `--txpkts` configurations required experimentation, but `--txpkts=100,100,13` will block while `--txpkts=100,100,14` will run as expected.
```
--txpkts=100,100,13
Port statistics ====================================
######################## NIC statistics for port 0 ########################
RX-packets: 0 RX-missed: 0 RX-bytes: 0
RX-errors: 0
RX-nombuf: 0
TX-packets: 1013 TX-errors: 11 TX-bytes: 215769
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
Telling cores to stop...
Waiting for lcores to finish...
---------------------- Forward statistics for port 0 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 1013 TX-dropped: 211829782 TX-total: 211830795
----------------------------------------------------------------------------
+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 1013 TX-dropped: 211829782 TX-total: 211830795
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
```
```
--txpkts=100,100,14
Port statistics ====================================
######################## NIC statistics for port 0 ########################
RX-packets: 0 RX-missed: 0 RX-bytes: 0
RX-errors: 0
RX-nombuf: 0
TX-packets: 45100373 TX-errors: 1868843 TX-bytes: 9651479822
Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 13296903 Tx-bps: 22764299064
############################################################################
Telling cores to stop...
Waiting for lcores to finish...
---------------------- Forward statistics for port 0 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 58405153 TX-dropped: 29676702 TX-total: 88081855
----------------------------------------------------------------------------
+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 58405153 TX-dropped: 29676702 TX-total: 88081855
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
```
Another example is `--txpkts=100,100,13,13` compared to `--txpkts=100,100,13,13,13` not working.
After adding some logging on my side, I found that the following check will always fail once it enters this state.
https://github.com/google/compute-virtual-ethernet-dpdk/blob/main/gve_tx_dqo.c#L24
```c
if (compl_desc->generation != txq->cur_gen_bit)
return;
```
Contributor guide
Assessment
This issue has not been assessed yet.