intel / intel/ethernet-linux-ice

ice: VF infinite reset loop caused by VLAN validation race in ice_vc_validate_{add,del}_vlan_filter_list() - ice-2.6.4

Open
#63 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
60
Forks
21
PR merge metrics
No merged PRs in 30d

Description

## Description

When using Intel E810 NICs with SR-IOV, enabling VFs and bringing up the VF interface triggers a continuous iavf reset loop. The VF never becomes operational — it constantly removes/re-probes itself every ~1 second.

The root cause is a race condition in the VIRTCHNL_OP_DEL_VLAN_V2 (opcode 53) and VIRTCHNL_OP_ADD_VLAN_V2 (opcode 52) validation paths in `ice_virtchnl.c`.

## Environment

- PF Driver: ice 2.6.4 (OOT, from ice-2.6.4.tar.gz)
- VF Driver: iavf 4.13.27 (from iavf-4.13.27.tar.gz)
- Kernel: 6.18.34 .2 SMP PREEMPT_RT Wed Jun 3 09:20:37 EDT 2026 x86_64
- Hardware: Intel E810-C series — device ID 8086:1889 (VF)
- DDP Package: ICE OS Default Package 1.3.59.0
- SR-IOV: 2 VFs enabled (65 vectors, 16 queues per VF)

## Steps to Reproduce

1. Load ice 2.6.4 and iavf 4.13.27 on system with E810 NIC
2. Enable SR-IOV VFs:
```bash
echo 2 > /sys/class/net/ens4f1/device/sriov_numvfs
```
3. Wait for VF to initialize, then observe dmesg:
```bash
dmesg -w | grep -E "opcode 53|Removing device|enabling device"
```

The loop starts immediately upon VF initialization — sometimes even without explicit `ip link set up`.

## Expected Behavior

VF interface initializes and remains stable.

## Actual Behavior

The VF enters an infinite reset loop cycling every ~1.1 seconds:

```
[276.028823] ice 0000:b4:00.1: Enabling 2 VFs
[276.028840] ice 0000:b4:00.1: Enabling 2 VFs with 65 vectors and 16 queues per VF
[276.161903] iavf 0000:b4:09.0: enabling device (0000 -> 0002)
[276.347007] iavf 0000:b4:09.0: Multiqueue Enabled: Queue pair count = 16
[276.347079] iavf 0000:b4:09.0: MAC address: 56:10:ee:2f:0f:7c
[280.984915] 8021q: adding VLAN 0 to HW filter on device ens4f1v0
[280.999399] iavf 0000:b4:09.0: removing PTP clock iavf-0000:b4:09.0-clk
[281.079969] ice 0000:b4:00.1: PF failed to honor VF 0, opcode 53, error -22
[281.079973] iavf 0000:b4:09.0: PF returned error -5 (VIRTCHNL_STATUS_ERR_PARAM) to our request 53 (VIRTCHNL_OP_DEL_VLAN_V2)
[281.175367] iavf 0000:b4:09.0: Removing device
[281.460496] iavf 0000:b4:09.0: enabling device (0000 -> 0002)
... (repeats indefinitely, sustained for 100+ seconds / 80+ cycles in this capture)
```

This was sustained for over 100 seconds / 80+ reset cycles in our test (see full dmesg attached).

Occasional `Expected response X from PF, received Y` errors also appear, indicating virtchnl message ordering is being corrupted by the rapid resets.

## Root Cause Analysis

In `ice_virtchnl.c`:

### `ice_vc_validate_del_vlan_filter_list()`

```c
if (!vfl->num_elements ||
vfl->num_elements > ice_vsi_num_non_zero_vlans(vsi))
return false;
```

During VF reset, the VSI VLAN count is transiently 0. The iavf driver sends `VIRTCHNL_OP_DEL_VLAN_V2` to clean up VLAN 0 as part of its normal teardown/replay sequence. Since `num_elements (1) > ice_vsi_num_non_zero_vlans(vsi) (0)`, the request is rejected with `-EINVAL`.

### `ice_vc_validate_add_vlan_filter_list()`

```c
num_req_fltr = ice_vsi_num_non_zero_vlans(vsi) + vfl->num_elements;
if (num_req_fltr > vfc->max_filters)
return false;
```

During iavf state replay after reset, iavf re-sends all VLANs. Some may still be cached on the VSI from the previous incarnation, causing double-counting and rejection.

### The loop mechanism

1. iavf sends DEL_VLAN_V2 during normal operation
2. PF rejects it because VSI VLAN count is 0 (transient state)
3. iavf receives `VIRTCHNL_STATUS_ERR_PARAM` and resets the VF
4. Reset clears VSI state → VLAN count goes to 0
5. iavf replays state, sends DEL_VLAN_V2 again → goto 2

## Suggested Fix

Replace the dynamic VSI-based bounds with the stable `max_filters` capability negotiated at VF init time:

```c
static bool
ice_vc_validate_del_vlan_filter_list(struct virtchnl_vlan_filtering_caps *vfc,
struct virtchnl_vlan_filter_list_v2 *vfl)
{
if (!vfl->num_elements || vfl->num_elements > vfc->max_filters)
return false;

return ice_vc_validate_vlan_filter_list(vfc, vfl);
}

static bool
ice_vc_validate_add_vlan_filter_list(struct virtchnl_vlan_filtering_caps *vfc,
struct virtchnl_vlan_filter_list_v2 *vfl)
{
if (!vfl->num_elements || vfl->num_elements > vfc->max_filters)
return false;

return ice_vc_validate_vlan_filter_list(vfc, vfl);
}
```

This is safe because:
- Per-element deletion is already idempotent: `ice_vsi_del_vlan()` treats `-ENOENT`/`-EBUSY` as success
- Per-element add is already idempotent: `ice_vsi_add_vlan()` treats `-EEXIST` as success
- Real HW capacity exhaustion is propagated from `ice_fltr_add_vlan()` on a per-element basis

The `vsi` parameter can be removed from both functions.

## Impact

- VFs are completely unusable — they never stabilize
- Each reset cycle takes ~1.1 seconds, consuming PCI bus bandwidth and PF resources
- The virtchnl message ordering corruption (wrong opcode responses) can affect other VFs on the same PF
- Affects any deployment using SR-IOV with E810 (cloud, telco, HPC)

## References

- Driver sources: ice-2.6.4.tar.gz, iavf-4.13.27.tar.gz from https://downloadmirror.intel.com/920657/Release_31.2.zip
- VIRTCHNL_OP_DEL_VLAN_V2 = opcode 53
- VIRTCHNL_OP_ADD_VLAN_V2 = opcode 52
- Relevant source: `src/ice_virtchnl.c`, functions `ice_vc_validate_del_vlan_filter_list()` and `ice_vc_validate_add_vlan_filter_list()`

Reproducer script:

[test-vf-vlan-race.sh](https://github.com/user-attachments/files/31020576/test-vf-vlan-race.sh)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.