NVIDIA / NVIDIA/daqiri

[BUG] DPDK VLAN-push flow is rejected by mlx5 under default HWS mode

Open
#305 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
36
Forks
11
Avg merge
3d 4h
Merged PRs (30d)
25

Description

Describe the bug

A raw DPDK configuration containing a TX vlan_push action cannot initialize on
ConnectX-7 under DAQIRI's default dv_flow_en=2 hardware-steering mode.

DAQIRI completes configuration parsing, memory registration, ethdev setup, and TX
queue creation. The mlx5 driver then rejects the VLAN-push action while creating the
TX flow. daqiri_init() returns failure and the benchmark cannot start.

Steps/Code to reproduce bug

Build a local container from main commit
57f3d8ebdd2e5410f654edea2e25b211cbca17de:

IMAGE_TAG=daqiri:vlan-repro \
BASE_TARGET=dpdk \
DAQIRI_ENGINE="dpdk ibverbs" \
scripts/build-container.sh

Start it with access to the GPU, NIC, and hugepages:

docker run --rm -it \
  --privileged \
  --gpus all \
  --network host \
  -v /dev/hugepages:/dev/hugepages \
  -v "$PWD:/workspace/daqiri" \
  -w /workspace/daqiri \
  daqiri:vlan-repro

Inside the container, save this as /tmp/vlan-dpdk.yaml:

%YAML 1.2
---
daqiri:
  cfg:
    version: 1
    stream_type: raw
    engine: dpdk
    master_core: 3
    debug: false
    log_level: info
    loopback: ''
    memory_regions:
    - name: Data_TX_GPU
      kind: device
      affinity: 0
      num_bufs: 16384
      buf_size: 1042
    - name: Data_RX_GPU
      kind: device
      affinity: 0
      num_bufs: 16384
      buf_size: 1042
    interfaces:
    - name: tx_port
      address: '0005:03:00.0'
      tx:
        queues:
        - name: tx_q_0
          id: 0
          batch_size: 1024
          cpu_core: 4
          memory_regions: [Data_TX_GPU]
          offloads: [tx_eth_src]
        flows:
        - name: vlan_encap
          id: 101
          actions:
          - type: vlan_push
            vlan_id: 100
            pcp: 0
            dei: 0
            ethertype: 33024
          match:
            udp_src: 48100
            udp_dst: 48100
    - name: rx_port
      address: '0005:03:00.1'
      rx:
        flow_isolation: true
        queues:
        - name: rx_q_0
          id: 0
          cpu_core: 5
          batch_size: 1024
          memory_regions: [Data_RX_GPU]
        flows:
        - name: vlan_decap
          id: 101
          match: {}
          actions:
          - type: vlan_pop
          - type: queue
            id: 0
bench_rx:
- interface_name: rx_port
  queue_id: 0
  cpu_core: 7
bench_tx:
- interface_name: tx_port
  queue_id: 0
  cpu_core: 6
  batch_size: 1024
  payload_size: 1000
  header_size: 42
  eth_dst_addr: 48:b0:2d:f4:04:24
  ip_src_addr: 1.1.1.1
  ip_dst_addr: 2.2.2.2
  udp_src_port: 48100
  udp_dst_port: 48100

Run:

/opt/daqiri/bin/daqiri_bench_raw_gpudirect \
  /tmp/vlan-dpdk.yaml --seconds 2

The DPDK arguments confirm that hardware steering is selected:

-a 0005:03:00.0,txq_inline_max=0,dv_flow_en=2
-a 0005:03:00.1,txq_inline_max=0,dv_flow_en=2

Flow creation then fails:

[INFO] ... Adding TX flow vlan_encap
mlx5_net: [mlx5dr_action_root_build_attr]: Found unsupported action type: 14
mlx5_net: [mlx5dr_bwc_rule_create_root]: BWC rule: failed creating rule on root tbl
[CRITICAL] ... Failed to create TX flow 'vlan_encap' on port 0:
                 fail to create rte flow (Invalid argument)
[CRITICAL] ... Failed to initialize DPDK
daqiri_init failed

Expected behavior

DAQIRI documents VLAN push/pop as supported by the raw DPDK engine, so the VLAN
flow should install under the engine's default steering configuration.

If mlx5 requires a different steering mode for this action, DAQIRI should either:

  1. select the compatible mode for interfaces containing VLAN actions, or
  2. reject the configuration before partial initialization with a clear message
    identifying the required mode.

Whether dv_flow_en=1 works on this hardware has not yet been tested.

Environment overview (please complete the following information)

  • Environment location: Docker on bare-metal IGX
  • Method of DAQIRI install: locally built Docker image
    • No docker pull command was used.
    • Build and run commands are included above.

Environment details

  • Host OS: Ubuntu 22.04.5 LTS
  • Host kernel: 5.15.0-1039-nvidia-tegra-igx, aarch64
  • Container base: CUDA 13.1 development image on Ubuntu 24.04
  • DAQIRI version: 2026.7.0
  • DAQIRI commit: 57f3d8ebdd2e5410f654edea2e25b211cbca17de
  • DPDK version: 25.11
  • Hardware: NVIDIA IGX with discrete RTX 6000 Ada GPU
  • NIC: NVIDIA ConnectX-7, ports 0005:03:00.0 and 0005:03:00.1
  • NIC firmware: 28.39.3004
  • mlx5 kernel driver: 24.07-0.6.1
  • MLNX_OFED/rdma-core: 24.07
  • NVIDIA GPU driver: 535.274.02

Additional context

VLAN/tunnel actions were introduced in PR #194 as the implementation for #196.
The recorded PR checks did not compile or exercise the VLAN flow on DPDK hardware.

The rejected action is emitted as RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN. Memory
registration and queue setup finish before this error, isolating the observed failure
to TX flow creation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce with scripts/build-container.sh and the provided /tmp/vlan-dpdk.yaml, then trace daqiri_init() through creation of the TX flow vlan_encap. Compare the default dv_flow_en=2 behavior with the compatible steering mode on the stated hardware. Done means VLAN push installs successfully, or initialization rejects the configuration before partial setup with a clear mode requirement.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.