mudler / mudler/vllm.cpp

fix(BACKEND-GATE-ROCM-SGLANG): query shared memory through pinned HIP driver

Open
#3,072 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-GATE-ROCM-SGLANG

Owner: current Strix benchmark campaign operator. Found during #3053 restoration. The reviewed preparation CLI at c8d019447 queries torch.cuda.get_device_properties(0).shared_memory_per_block, but the pinned AMD Torch 2.9.1 wheel does not expose that Python property. After repairing MIOpen, Strix job c13a24ee-d1ad-49a7-ba08-e96e814273c0 fails identity-dependencies with AttributeError before kernel compilation. Evidence: /mnt/nas_share/rc/strix-four-engine-3053.X94a3J/prepare-c8d019447-rebuilt-02/logs/identity-dependencies.log.

The pinned Triton AMD backend already returns max_shared_mem from HIP hipGetDeviceProperties: triton/backends/amd/driver.c lines 199-209; driver.py lines 161-177 and 700-705. Validate that executing path on Strix, then use the measured HIP value without changing the Torch/Triton pin, hard-coding 65536 as a measurement, or weakening the existing 64KiB identity gate. Add a regression that executes the production identity probe with Torch properties lacking the unsupported member. Preserve failure behavior for missing, invalid, or insufficient device limits. A committed design supplement precedes implementation; a fresh implementer and independent mutation reviewer are required. #3053 owns downstream preparation, model startup, and benchmarks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with triton/backends/amd/driver.c lines 199-209 and driver.py lines 161-177 and 700-705, then trace the production identity probe used by the preparation CLI. Validate the HIP shared-memory path on Strix and add a regression where Torch properties lack the unsupported member. Done means valid HIP limits are used without changing pins or the 64KiB gate, while missing, invalid, or insufficient limits still fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.