test_opt_paged_engine: removing greedy_ids.npy deletes the whole SACRED gate into a green at 0 assertions
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: MODEL-TEXT-opt-optfor-causal-lm
#2805 / #2981 closed one mute switch in
tests/vllm/models/test_opt_paged_engine.cpp. Two of the same class survive in
the same file, and one of them is worse than the one that was fixed.
1. Removing greedy_ids.npy deletes the whole SACRED gate into a green
Measured by a fresh reviewer of #2981, on both the pre-fix and post-fix
binaries:
assertions: 0 | 0 passed | 0 failed | Status: SUCCESS! rc 0
The bar itself is behind an existence guard, so removing one committed file
makes the entire token-exact gate assert nothing and report success. The ctest
entry at tests/CMakeLists.txt:3663-3667 carries no
FAIL_REGULAR_EXPRESSION, and the case never returns 77, so nothing outside
the binary notices either.
.agents/specs/sweep-opt-125m.md currently calls this sibling guard "already
correct". That is too strong: it asserts nothing rather than asserting
something unlicensed, which is less bad, not correct. assertions: 0 is a
skip wearing a pass.
2. p3_prompt.i32 absence silently drops a cross-check
Removing it takes the post-fix binary from 43 to 42 assertions, still
SUCCESS, rc 0 -- if (!want_prompt.empty()) CHECK(out.prompt_token_ids == want_prompt) at :304. Same absence-means-skip shape. Pre-existing, inside
the region #2981 left byte-unchanged, so neither introduced nor claimed fixed
there.
What closes this
The golden set is a precondition of the gate, not an option: a missing
greedy_ids.npy or prompt file reds rather than reducing the assertion count.
Red-before for each: remove the file, confirm the gate reds where it currently
passes. A FAIL_REGULAR_EXPRESSION or an assertion-count floor on the ctest
entry would also catch the assertions: 0 shape from outside.
Related: #2987 (the same licence defect in four sibling gates).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tests/vllm/models/test_opt_paged_engine.cpp, especially the greedy_ids.npy guard and the p3_prompt.i32 check around line 304, then inspect the ctest entry in tests/CMakeLists.txt:3663-3667. Remove each golden file to confirm the current green behavior, then make the gate fail when a required file is absent or assertions drop to zero; verify both missing-file cases turn red.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100