google / google/compute-virtual-ethernet-dpdk

Segmented Sends Blocking Tx Queues

Open
#10 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
14
Forks
0
PR merge metrics
No merged PRs in 30d

Description

### Test Environment:

* Machine type `c3-standard-8`
* CPU platform `Intel Sapphire Rapids`
* Architecture `x86/64`
* Networking NIC type `gVNIC`

My team is currently on DPDK version `23.11` and we use segmented packets in various flows of our application. We were seeing packet leaks so I applied the patch from this [PR](https://github.com/google/compute-virtual-ethernet-dpdk/pull/8/files). That fixed a few of our flows, but introduced another error. We upgraded to version `24.07-rc4` as another test, but found that the behavior stayed the same.

### Test Program

```
dpdk-v24.07-rc4
sudo ./build/app/dpdk-testpmd -l 0-3 -n 2 -- --txpkts= --port-topology=chained --nb-cores 1 --txq 1 --rxq 1 --mbcache=512 --txd=1024 --forward-mode=txonly --stats-period 1 --tx-ip=10.0.0.1,10.0.0.2 --eth-peer=0,00:00:00:00:00:00 --tx-udp=5555
```

### Behavior
There are a variety of packet chains that can be created that will cause the queues to block.

* Any chain with a zero length mbuf will result in the queues getting blocked. E.g `--txpkts=100,0,100`

Other `--txpkts` configurations required experimentation, but `--txpkts=100,100,13` will block while `--txpkts=100,100,14` will run as expected.
```
--txpkts=100,100,13
Port statistics ====================================
######################## NIC statistics for port 0 ########################
RX-packets: 0 RX-missed: 0 RX-bytes: 0
RX-errors: 0
RX-nombuf: 0
TX-packets: 1013 TX-errors: 11 TX-bytes: 215769

Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 0 Tx-bps: 0
############################################################################
Telling cores to stop...
Waiting for lcores to finish...

---------------------- Forward statistics for port 0 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 1013 TX-dropped: 211829782 TX-total: 211830795
----------------------------------------------------------------------------

+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 1013 TX-dropped: 211829782 TX-total: 211830795
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
```
```
--txpkts=100,100,14
Port statistics ====================================
######################## NIC statistics for port 0 ########################
RX-packets: 0 RX-missed: 0 RX-bytes: 0
RX-errors: 0
RX-nombuf: 0
TX-packets: 45100373 TX-errors: 1868843 TX-bytes: 9651479822

Throughput (since last show)
Rx-pps: 0 Rx-bps: 0
Tx-pps: 13296903 Tx-bps: 22764299064
############################################################################
Telling cores to stop...
Waiting for lcores to finish...

---------------------- Forward statistics for port 0 ----------------------
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 58405153 TX-dropped: 29676702 TX-total: 88081855
----------------------------------------------------------------------------

+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 58405153 TX-dropped: 29676702 TX-total: 88081855
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.
```

Another example is `--txpkts=100,100,13,13` compared to `--txpkts=100,100,13,13,13` not working.

After adding some logging on my side, I found that the following check will always fail once it enters this state.
https://github.com/google/compute-virtual-ethernet-dpdk/blob/main/gve_tx_dqo.c#L24
```c
if (compl_desc->generation != txq->cur_gen_bit)
return;
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.