Build the pinned vLLM oracle for ROCm/gfx1200 in Docker, with a benchmarking/profiling harness against vllm.cpp
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: ORACLE-VLLM-ROCM-GFX1200-DOCKER
What this is
A standing Docker project that builds the pinned PRIMARY oracle
(e126687a9a, vLLM 0.28.1rc1.dev132) for ROCm on gfx1200, alongside
vllm-gguf-plugin, and runs benchmarking/profiling comparisons against
vllm.cpp's own HIP build on the same box (RX 9060 XT).
This mirrors the measurement BACKEND-GATE-ROCM-LLAMACPP already did by hand
on gfx1151 (#2740,
docs/bench-evidence/oracle-vllm-gfx1151-20260903.md), but packages the
recipe as reusable infrastructure instead of a one-off rc job, and targets
gfx1200 instead of gfx1151.
Why
.agents/oracles/vllm.md's device-scoped gateability table is empty at the
CURRENT pin (e126687a9a) on every device. Nobody has built or run the
primary oracle on ROCm at this pin yet.- gfx1200 is confirmed in
HIP_SUPPORTED_ARCHSand the default
PYTORCH_ROCM_ARCHat this exact commit (checked directly against
CMakeLists.txt:52anddocker/Dockerfile.rocm_base:109at
e126687a9a), so the prerequisite that blocked earlier attempts on other
devices does not obviously apply here — unmeasured, not assumed. - This box's own GPU (RX 9060 XT, gfx1200) is the only local hardware
available, so a CUDA oracle image is fleet-only; a ROCm image is what
makes apples-to-apples comparison possible without a lease.
Scope
Dockerfilebuilding the pinned vLLM commit for
VLLM_TARGET_DEVICE=rocm PYTORCH_ROCM_ARCH=gfx1200, plus
vllm-gguf-pluginat its recorded pin, plus the five packages
oracle-vllm-gfx1151-20260903.mdfound missing from a bare ROCm image
(python3-dev,rocm-libs,libdrm-dev, ROCmtorchvision,amdsmi).- A platform-identity smoke test baked into the image (mirrors
gen_rocm.py's asserts):HSA_OVERRIDE_GFX_VERSIONunset,
RocmPlatformresolved, notUnspecifiedPlatform. - A benchmarking/profiling script comparing the containerized vLLM oracle
against a host-builtvllm.cppbinary: same prompts, same token counts,
same GPU (device passthrough), GPU-clock-state discipline per
.agents/benchmarking.md.
Out of scope
- Advancing the vLLM parity pin or any
src//include/change. - A performance verdict of any kind before the declared token gate passes
(perAGENTS.md§Gates); this row builds the tool, it does not itself
license a benchmark claim. - CUDA oracle packaging (fleet-only, separate row if wanted).
Blocker to record
This session's user does not have docker group membership on the box the
work is being done on, so the image could not be built or run to verify
inside this session. sudo usermod -aG docker <user> plus a new login
session unblocks it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .agents/oracles/vllm.md, AGENTS.md §Gates, .agents/benchmarking.md, and docs/bench-evidence/oracle-vllm-gfx1151-20260903.md, then inspect the pinned vLLM commit and existing Dockerfile conventions. The work is done when a gfx1200 ROCm image builds with the specified pins and platform smoke test, and its harness compares it with the host vllm.cpp build under the declared benchmarking discipline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker, python
- Domain
- ai, devops, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100