mudler / mudler/vllm.cpp

ROCm: DFlash2 reaches host fallbacks and hangs gfx1151

Open
#2,377 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-ROCM

Qwen3.8-27B with the published DFlash2 safetensors draft does not complete on strix:gpu0 (gfx1151, ROCm 7.2.4).

The plain target completes 32 greedy tokens. The DFlash2 arm reaches three CPU reference-tier ops and then the GPU reports a hang:

[vt reference-tier] op=IndexSelect device=rocm has NO native kernel
[vt reference-tier] op=IndexCopy device=rocm has NO native kernel
[vt reference-tier] op=DFlashGroupedConv device=rocm has NO native kernel
HW Exception by GPU node-1 ... reason :GPU Hang

The DFlash2 process exits 134 before its first completion. test_dflash2_runner_reach is also red on the same HIP build. Its D9 case reports zero differing blocks after changing the selector scalars. One repeated run also failed the W11 call-count and output-size checks.

GetOp calls Backend::FlushPending() before a CPU reference-tier kernel reads managed device memory. RocmBackend does not override that method, so it inherits the no-op. Metal and Vulkan both override it. This is the first root-cause candidate for the hang and the unstable runner output.

A safe run is not enough for a performance result. The production DFlash2 chain also needs native ROCm implementations for every op that the full provider census reports. The known static set includes row gather/scatter, grouped convolution, top-k pairs, selector edges, path walk, and the block-attention variants.

Acceptance:

  • A device test proves that RocmBackend::FlushPending() drains queued HIP work before a host fallback.
  • The real Qwen3.8-27B plus DFlash2 arm completes greedy generation without a GPU hang.
  • The focused DFlash2 suites pass on gfx1151.
  • VT_OP_PROVIDER_STATS=1 reports zero CPU reference-tier hits for the measured DFlash2 workload.
  • A repeated local-checkpoint benchmark records clock state and warm-leg throughput.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at RocmBackend::FlushPending and GetOp, then inspect the device test coverage around host fallbacks. Run test_dflash2_runner_reach and the focused DFlash2 suites on gfx1151. Done means queued HIP work is drained, Qwen3.8-27B DFlash2 completes without a hang, tests pass, and provider stats show no CPU reference-tier hits.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning, performance, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.