intel / intel/ethernet-linux-ice

ice: raw (protocol-agnostic) FDIR rules added by a VF return flow_id=0, making every delete fail with RULE_NONEXIST

Open
#71 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
60
Forks
21
PR merge metrics
No merged PRs in 30d

Description

## Summary

A VF adds protocol-agnostic ("raw") FDIR rules via `VIRTCHNL_OP_ADD_FDIR_FILTER`.
The rules **are programmed and demonstrably steer traffic to the intended
queue**. However the `flow_id` returned to the VF is `0` for every rule, so
every subsequent `VIRTCHNL_OP_DEL_FDIR_FILTER` is rejected by the PF:

```
ice 0000:81:00.1: VF 4: FDIR invalid flow_id:0x0
```

Because the VF never receives a usable handle, **rules that work can never be
deleted**. They accumulate until the VF's VSI is destroyed (`sriov_numvfs=0`).
VF FLR, PF `devlink driver_reinit` and PF PCIe FLR do **not** release them.

This makes raw FDIR unusable for any application that restarts: each restart
permanently consumes filter resources on that VF.

Reproduced with stock `dpdk-testpmd` — no application code required.

## Environment

| item | value |
|---|---|
| ice driver | 2.3.10 (out-of-tree). **Also verified present in v2.6.7 / `main` by source inspection** |
| adapter | E810, PCI ID `[8086:1592]`, subsystem `[8086:0002]` |
| firmware / NVM | `fw 7.9.1 api 1.7.11 nvm 4.90 0x80020ef2 1.3863.0` |
| DDP | ICE COMMS Package 1.3.57.0 (`fw.app.bundle_id 0xc0000002`), loaded successfully at probe |
| fw.netlist | 4.4.2000-3.28.0 (build 0x2723426e) |
| host kernel | 6.8.12-15-pve (SMP PREEMPT_DYNAMIC) |
| guest kernel | 7.0.0-29-generic |
| VF driver | DPDK 25.11.0 `iavf` PMD via `vfio-pci` |

## Reproducer

VF bound to `vfio-pci`:

```bash
dpdk-testpmd -l 6-7 -n 4 -a 0000:05:00.0 --file-prefix=t -m 1024 \
-- -i --rxq=4 --txq=4 --forward-mode=rxonly
```

Create raw rules. The spec is a complete 79-byte Ethernet/IPv4/UDP frame; the
mask selects only the EtherType, IP protocol, UDP destination port, and two
bytes of UDP payload (a 1-bit flag and a 4-bit selector at a fixed offset).
Each rule targets a different queue by matching a different selector value:

```
# queue 0
flow create 0 ingress pattern raw pattern spec aabbccddeeff11223344556608004500004112340000401142640a0909010a090902303901bb002d000040000000000000000000000000000000000000000000000000000000000000000000000000 pattern mask 000000000000000000000000ffff000000000000000000ff000000000000000000000000ffff00000000803c0000000000000000000000000000000000000000000000000000000000000000000000 / end actions mark id 233472 / queue index 0 / end

# queue 1 (payload byte 43 = 0x04 instead of 0x00)
flow create 0 ingress pattern raw pattern spec aabbccddeeff11223344556608004500004112340000401142640a0909010a090902303901bb002d000040040000000000000000000000000000000000000000000000000000000000000000000000 pattern mask 000000000000000000000000ffff000000000000000000ff000000000000000000000000ffff00000000803c0000000000000000000000000000000000000000000000000000000000000000000000 / end actions mark id 233473 / queue index 1 / end
```

Repeat for 16 rules, then delete them one at a time:

```
flow destroy 0 rule 0
flow destroy 0 rule 1
...
```

### Observed

All creates report success. **Every delete fails:**

```
iavf_fdir_del(): Failed to delete rule request due to this rule doesn't exist
iavf_flow_destroy(): Failed to destroy flow
port_flow_complain(): Caught PMD error type 2 (flow rule (handle)):
Failed to delete filter rule.: Operation not permitted
```

With PF-side dynamic debug enabled:

```bash
echo 'file ice_virtchnl_fdir.c +p' > /sys/kernel/debug/dynamic_debug/control
```

the PF logs, once per delete attempt:

```
ice 0000:81:00.1: VF 4: FDIR invalid flow_id:0x0
```

### Expected

Each delete succeeds — or, failing that, the preceding add reports failure.

### Side observation in the same run

The **first** `flow create` of a fresh session fails, and subsequent identical
creates succeed:

```
port_flow_complain(): Caught PMD error type 2 (flow rule (handle)):
Failed to create parser engine.: Invalid argument
```

In the attached log 15 of 16 rules were created, the single failure being the
first. This is reproducible and looks like first-use initialisation of the
parser profile; it is reported here for completeness rather than as the main
issue, but may share a root cause with the `flow_id` handling.

### Attachments

- `repro-raw-fdir-delete.sh` — standalone reproducer (testpmd only)
- `testpmd-raw-fdir-delete-failure.log` — full testpmd session showing the
creates succeeding and all deletes failing
- `pf-dmesg-excerpt.txt` — PF-side driver/firmware/DDP provenance and the
`FDIR invalid flow_id:0x0` lines

### Evidence the rules really are programmed in hardware

With a full rule set (16 raw rules, 4 queues) installed on a freshly created VF,
the device steers **every** packet to the intended queue:

| | rules | RX packets | delivered to wrong queue |
|---|---|---|---|
| no rules (baseline) | 0 | 6,106,479 | 4,971,215 — **81.4%** |
| **16 raw rules** | 16 | **3,451,435** | **0 — none** |

All 16 rules installed first try, zero retries. Every received packet was
delivered to the queue its matched rule selected; the cross-queue counter did
not increment once. Sustained throughput 501 Mbps at **0.0000% loss**
(0 of 7,614,267 packets).

A second run with a partial rule set (8 rules covering 2 of the 4 packet shapes)
steered 47.5% — consistent with the uncovered shapes having no rule to match,
and confirming the rules act exactly where they are installed and nowhere else.

**These rules work perfectly. They still cannot be deleted.** That is the whole
of this report: the filters are correctly programmed and demonstrably steering
millions of packets, while the `flow_id` returned to the VF is 0, so there is no
handle with which to remove them.

## Analysis

### 1. Blocker — `flow_id` returned as 0 for rules that were programmed

`ice_vc_del_fdir_fltr()` rejects at its first step:

```c
conf = ice_vc_fdir_lookup_entry(vf, fltr->flow_id);
if (!conf) {
stat->status = VIRTCHNL_FDIR_FAILURE_RULE_NONEXIST;
dev_dbg(dev, "VF %d: FDIR invalid flow_id:0x%X", vf->vf_id, fltr->flow_id);
```

On its success path `ice_vc_add_fdir_raw()` does `return 0` **without sending a
response**; the reply is emitted asynchronously by `ice_vc_add_fdir_fltr_post()`,
which is the only site that sets `resp->flow_id = conf->flow_id`. A raw add that
programs hardware but whose reply carries `flow_id = 0` implies that completion
either did not run for the raw path, or ran against a stale `ctx->conf`.

### 2. VF reset orphans raw rules in hardware

`ice_vf_fdir_exit()` runs on every VF reset. Its rule-clearing step is
software-only:

```c
static void ice_vc_fdir_flush_entry(struct ice_vf *vf)
{
list_for_each_entry_safe(desc, temp, &vf->fdir.fdir_rule_list, fltr_node) {
list_del(&desc->fltr_node);
kfree(conf); /* no hardware write anywhere */
}
}
```

It never calls `ice_vc_fdir_add_del_raw(vf, conf, false)`. The classic FDIR path
*does* reach hardware (`ice_vc_fdir_rem_prof_all()` → `ice_rem_prof_id_flow()` /
`ice_flow_rem_entry()`); the raw path has no equivalent, and
`vf->fdir_prof_info[ptg]` is touched by no cleanup path in the tree. Hardware
entries survive while the `conf` holding their handle is freed.

**This function is byte-identical in v2.6.7 and `main`.**

### 3. `0` is a legal flow_id

`ice_vc_fdir_insert_entry()` uses
`idr_alloc(&vf->fdir.fdir_rule_idr, conf, 0, ICE_FDIR_MAX_FLTRS, GFP_KERNEL)`,
so id 0 is allocatable and the PF cannot distinguish "id 0" from "unset".
Unchanged in v2.6.7.

## Status in the latest release

`src/ice_virtchnl_fdir.c` at tag `v2.6.7` is byte-identical to `main`. Diffed
against 2.3.10 the whole file changes by 102 lines, and **no changed line
touches `flow_id`, `fdir_prof_info`, or `idr_alloc`**. So items 1–3 above are
all still present in the latest release.

One related fix has landed since 2.3.10 and is acknowledged: the
`ice_flow_set_hw_prof()` failure path in `ice_vc_add_fdir_raw()` now sets
`stat->status = VIRTCHNL_FDIR_FAILURE_RULE_NORESOURCE` rather than falling
through with a `kzalloc`'d stat (which previously reported success with
`flow_id = 0`). That is a real improvement, but it does not address the case
reported here, where the rule **is** successfully programmed.

## Requested fixes, in priority order

1. **Return the real `flow_id` in the raw add response.** This is the blocker.
2. `ice_vc_fdir_flush_entry()` should delete each entry from hardware before
freeing its `conf`, so a VF reset does not orphan working rules.
3. `ice_vf_fdir_exit()` should release `vf->fdir_prof_info[]`.
4. Reserve `0` as an invalid `flow_id`.

## Ruled out

- **Not a DPDK bug** — the `iavf` PMD returns to the PF exactly the `flow_id`
the PF supplied. Reproduced identically under a second, unrelated DPDK
application.
- **Not resource exhaustion** — reproduced immediately after
`sriov_numvfs=0` teardown and recreate, on a PF with no other FDIR users.
- **Not the action set** — fails identically with `queue` alone and with
`mark` + `queue`.
- **Not DDP** — `devlink dev info` reports ICE COMMS Package 1.3.57.0, which
provides raw/protocol-agnostic pattern support, loaded successfully at probe.

[pf-dmesg-excerpt.txt](https://github.com/user-attachments/files/31223933/pf-dmesg-excerpt.txt)
[repro-raw-fdir-delete.sh](https://github.com/user-attachments/files/31223935/repro-raw-fdir-delete.sh)
[testpmd-raw-fdir-delete-failure.log](https://github.com/user-attachments/files/31223934/testpmd-raw-fdir-delete-failure.log)

Contributor guide

Open the contributing guide

Research direction

Start in src/ice_virtchnl_fdir.c and trace ice_vc_add_fdir_raw() through ice_vc_add_fdir_fltr_post(), then inspect ice_vc_fdir_flush_entry() and ice_vf_fdir_exit() for reset cleanup. Run repro-raw-fdir-delete.sh with the attached logs as a baseline; done means successful raw adds return usable nonzero flow IDs, deletes succeed, and VF reset does not leave hardware rules orphaned.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
networking, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.