mudler / mudler/vllm.cpp

fix(BACKEND-ROCM-QUANT-GATHER): reject unqualified model captures

Open
#3,113 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-ROCM-QUANT-GATHER

The bounded model comparator in MR #3097 accepts incomplete or unqualified primary records. Fresh review of commit 75d92614e84ff428eab49324f3b4fb2aba183ac3 reproduced the failures in tools/rocm_quant_gather/compare_models.py.

A synthetic validator corpus uses the actual native and secondary captures. Changing one primary record to PENDING with a RuntimeError after generation still returns PASS. Wrong primary and plugin pins, a wrong runtime version, and an FP8 cache request also pass. The separate memory record accepts PENDING, a wrong primary pin, and different sampling or logical capacity. A nonzero cache view storage offset is also accepted. These synthetic mutations test the validator and are not oracle measurements.

The driver records tokens before its second memory observation. Its exception path preserves those tokens. Comparing tokens alone therefore cannot establish successful execution. The operator separately checks actual process exits and sealed inputs; the comparator must enforce its own documented qualification contract.

Require successful execution without an exception, the pinned primary and plugin identities, the resolved runtime identity, and the complete identical workload in both token and memory records. Validate the cache view offset and its physical bounds. Add focused rejection tests for each demonstrated invalid record and retain the valid actual captures as the positive control.

Evidence: /home/vikash/.cache/rdna3-gather-oracle-review/comparison-input-audit.json records 31 cases, with 22 rejected mutations and nine invalid records accepted. The existing spec .agents/specs/rocm-quant-gather.md owns the correction. A fresh implementer repairs the finding, a different agent reviews the immutable result, and the operator reruns the complete comparison. No model, workload, pin, or performance requirement changes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/rocm_quant_gather/compare_models.py and the correction owned by .agents/specs/rocm-quant-gather.md; review the 31 cases in comparison-input-audit.json and the valid actual captures. Done means focused rejection tests cover each demonstrated invalid record, while the valid captures remain a positive control and the complete comparison passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.