vLLM serve fails on MI300X (gfx942 / TheRock 7.13 & 7.14 nightly): _C ABI mismatch — get_cuda_view_from_cpu_tensor / undefined HIP symbols
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 40
- Forks
- 9
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 59
Description
Summary
rocm serve <model> --engine vllm fails to reach ready on MI300X (gfx942) for
BOTH managed runtimes installed by rocm install sdk:
- release wheel gfx94X-dcgpu 7.13.0 (torch 2.11.0+rocm7.13.0)
- nightly wheel gfx94X-dcgpu 7.14.0a20260611 (torch 2.9.1+rocm7.14.0a...)
The vLLM resolved for the managed TheRock runtime is ABI-incompatible with that
runtime's torch: the compiled _C / _rocm_C extensions fail to import with
undefined HIP/c10 symbols, and the V1 EngineCore then dies calling a native op
that isn't present. rocm engines list still reports vllm runtime: ready,
which is misleading.
Environment
- GPU: AMD Instinct MI300X — detected_gfx_target gfx943, family gfx94X-dcgpu
- OS: Ubuntu 22.04.5 LTS, kernel 6.5.0-45-generic
- CPU: AMD EPYC 9654; RAM ~1.5 TiB
- Driver: amdgpu available, /dev/kfd present
- vLLM version at startup: 0.23.0
- Runtimes (both status: ready):
- release-wheel-gfx94x-dcgpu-7-13-0
- nightly-wheel-gfx94x-dcgpu-7-14-0a20260611
Steps to reproduce
- rocm install sdk
- rocm install sdk --channel nightly
- rocm examine
- rocm runtimes activate release-wheel-gfx94x-dcgpu-7-13-0
- rocm serve qwen --engine vllm --verbose
(resolves to Qwen/Qwen3-4B-Instruct-2507, endpoint 127.0.0.1:11435)
Actual result — RELEASE 7.13.0
WARNING [rocm.py:44] Failed to import from vllm._C ... undefined symbol:
_ZN3c103hip19getCurrentHIPStreamEa
Fatal during init_device:
torch.ops._C.get_cuda_view_from_cpu_tensor(cpu_tensor)
AttributeError: '_OpNamespace' '_C' object has no attribute 'get_cuda_view_from_cpu_tensor'
RuntimeError: Engine core initialization failed.
Error: vLLM server process exited before becoming ready (exit status: 1)
Actual result — NIGHTLY 7.14.0a20260611
WARNING Failed to import from vllm._C ... undefined symbol:
_ZN3c103hip28c10_hip_check_implementationEiPKcS2_jb
WARNING Failed to import from vllm._C_stable_libtorch ... undefined symbol:
torch_get_mutable_data_ptr
WARNING Failed to import from vllm._rocm_C ... undefined symbol:
_ZNK3c1010TensorImpl15decref_pyobjectEv
ModuleNotFoundError: No module named 'xgrammar'
Error: vLLM server process exited before becoming ready (exit status: 1)
Expected result
rocm serve <model> --engine vllm reaches ready and serves an OpenAI-compatible
endpoint on a freshly installed managed runtime on gfx942, without requiring a
manual from-source vLLM build.
Notes
- docs/vllm.md says the adapter does not install vLLM automatically and
recommends building from source against TheRock torch (gfx942/7.13 GPTQ
compat.cuh patch). If from-source is the only supported path on gfx942, could
rocm serve/rocm engines listdetect the ABI mismatch and surface
actionable guidance instead of a raw AttributeError deep in EngineCore? - Missing
xgrammardependency on the nightly runtime looks like a packaging gap. - Minor UX:
rocm services stop --allerrors; working form is
rocm services stop -- --all --yes.
Questions
- Is a prebuilt vLLM expected to work out-of-the-box on gfx942 for the released
gfx94X-dcgpu runtime, or is build-from-source currently required? - Is the missing xgrammar dependency on nightly a known packaging gap?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/vllm.md and reproduce rocm serve on both named managed runtimes using the commands in the issue. Compare the runtime readiness reported by rocm engines list with the vLLM extension imports and EngineCore failure. Done means the supported gfx942 path either reaches a ready endpoint or reports the ABI and missing-dependency requirements with actionable guidance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100