ROCm / ROCm/rocm-cli

vLLM serve fails on MI300X (gfx942 / TheRock 7.13 & 7.14 nightly): _C ABI mismatch — get_cuda_view_from_cpu_tensor / undefined HIP symbols

Open
#257 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
40
Forks
9
Avg merge
4d 20h
Merged PRs (30d)
59

Description

Summary

rocm serve <model> --engine vllm fails to reach ready on MI300X (gfx942) for
BOTH managed runtimes installed by rocm install sdk:

  • release wheel gfx94X-dcgpu 7.13.0 (torch 2.11.0+rocm7.13.0)
  • nightly wheel gfx94X-dcgpu 7.14.0a20260611 (torch 2.9.1+rocm7.14.0a...)

The vLLM resolved for the managed TheRock runtime is ABI-incompatible with that
runtime's torch: the compiled _C / _rocm_C extensions fail to import with
undefined HIP/c10 symbols, and the V1 EngineCore then dies calling a native op
that isn't present. rocm engines list still reports vllm runtime: ready,
which is misleading.

Environment
  • GPU: AMD Instinct MI300X — detected_gfx_target gfx943, family gfx94X-dcgpu
  • OS: Ubuntu 22.04.5 LTS, kernel 6.5.0-45-generic
  • CPU: AMD EPYC 9654; RAM ~1.5 TiB
  • Driver: amdgpu available, /dev/kfd present
  • vLLM version at startup: 0.23.0
  • Runtimes (both status: ready):
    • release-wheel-gfx94x-dcgpu-7-13-0
    • nightly-wheel-gfx94x-dcgpu-7-14-0a20260611
Steps to reproduce
  1. rocm install sdk
  2. rocm install sdk --channel nightly
  3. rocm examine
  4. rocm runtimes activate release-wheel-gfx94x-dcgpu-7-13-0
  5. rocm serve qwen --engine vllm --verbose
    (resolves to Qwen/Qwen3-4B-Instruct-2507, endpoint 127.0.0.1:11435)
Actual result — RELEASE 7.13.0

WARNING [rocm.py:44] Failed to import from vllm._C ... undefined symbol:
_ZN3c103hip19getCurrentHIPStreamEa
Fatal during init_device:
torch.ops._C.get_cuda_view_from_cpu_tensor(cpu_tensor)
AttributeError: '_OpNamespace' '_C' object has no attribute 'get_cuda_view_from_cpu_tensor'
RuntimeError: Engine core initialization failed.
Error: vLLM server process exited before becoming ready (exit status: 1)

Actual result — NIGHTLY 7.14.0a20260611

WARNING Failed to import from vllm._C ... undefined symbol:
_ZN3c103hip28c10_hip_check_implementationEiPKcS2_jb
WARNING Failed to import from vllm._C_stable_libtorch ... undefined symbol:
torch_get_mutable_data_ptr
WARNING Failed to import from vllm._rocm_C ... undefined symbol:
_ZNK3c1010TensorImpl15decref_pyobjectEv
ModuleNotFoundError: No module named 'xgrammar'
Error: vLLM server process exited before becoming ready (exit status: 1)

Expected result

rocm serve <model> --engine vllm reaches ready and serves an OpenAI-compatible
endpoint on a freshly installed managed runtime on gfx942, without requiring a
manual from-source vLLM build.

Notes
  • docs/vllm.md says the adapter does not install vLLM automatically and
    recommends building from source against TheRock torch (gfx942/7.13 GPTQ
    compat.cuh patch). If from-source is the only supported path on gfx942, could
    rocm serve / rocm engines list detect the ABI mismatch and surface
    actionable guidance instead of a raw AttributeError deep in EngineCore?
  • Missing xgrammar dependency on the nightly runtime looks like a packaging gap.
  • Minor UX: rocm services stop --all errors; working form is
    rocm services stop -- --all --yes.
Questions
  1. Is a prebuilt vLLM expected to work out-of-the-box on gfx942 for the released
    gfx94X-dcgpu runtime, or is build-from-source currently required?
  2. Is the missing xgrammar dependency on nightly a known packaging gap?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/vllm.md and reproduce rocm serve on both named managed runtimes using the commands in the issue. Compare the runtime readiness reported by rocm engines list with the vLLM extension imports and EngineCore failure. Done means the supported gfx942 path either reaches a ready endpoint or reports the ABI and missing-dependency requirements with actionable guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.