fix(BACKEND-ROCM-QUANT-GATHER): reject unqualified model captures
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM-QUANT-GATHER
The bounded model comparator in MR #3097 accepts incomplete or unqualified primary records. Fresh review of commit 75d92614e84ff428eab49324f3b4fb2aba183ac3 reproduced the failures in tools/rocm_quant_gather/compare_models.py.
A synthetic validator corpus uses the actual native and secondary captures. Changing one primary record to PENDING with a RuntimeError after generation still returns PASS. Wrong primary and plugin pins, a wrong runtime version, and an FP8 cache request also pass. The separate memory record accepts PENDING, a wrong primary pin, and different sampling or logical capacity. A nonzero cache view storage offset is also accepted. These synthetic mutations test the validator and are not oracle measurements.
The driver records tokens before its second memory observation. Its exception path preserves those tokens. Comparing tokens alone therefore cannot establish successful execution. The operator separately checks actual process exits and sealed inputs; the comparator must enforce its own documented qualification contract.
Require successful execution without an exception, the pinned primary and plugin identities, the resolved runtime identity, and the complete identical workload in both token and memory records. Validate the cache view offset and its physical bounds. Add focused rejection tests for each demonstrated invalid record and retain the valid actual captures as the positive control.
Evidence: /home/vikash/.cache/rdna3-gather-oracle-review/comparison-input-audit.json records 31 cases, with 22 rejected mutations and nine invalid records accepted. The existing spec .agents/specs/rocm-quant-gather.md owns the correction. A fresh implementer repairs the finding, a different agent reviews the immutable result, and the operator reruns the complete comparison. No model, workload, pin, or performance requirement changes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tools/rocm_quant_gather/compare_models.py and the correction owned by .agents/specs/rocm-quant-gather.md; review the 31 cases in comparison-input-audit.json and the valid actual captures. Done means focused rejection tests cover each demonstrated invalid record, while the valid captures remain a positive control and the complete comparison passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100