Add golden tests for case matrix generation and non-overwrite behavior
- Dominant language
- Rust
- Stars
- 53
- Forks
- 16
- Avg merge
- 4h 22m
- Merged PRs (30d)
- 46
Description
## Problem
`tools/populate_case_matrix.py` now creates hundreds of deterministic baseline cases, but its idempotence and non-overwrite guarantees are only exercised manually. A generator regression could rewrite hand-authored cases or create numbering gaps.
## Scope
- Refactor the generator just enough to accept an isolated cases root/manifest in tests.
- Test first-run creation, second-run idempotence, contiguous numbering, and preservation of hand-written files.
- Cover supported and planned cells plus architecture aliases.
- Avoid invoking a Wave compiler or network service.
## Completion criteria
- Unit tests prove a second run writes zero files.
- Hand-authored content is byte-for-byte unchanged.
- Invalid numbering and unsupported manifest shapes fail clearly.
Related: #457
Contributor guide
Research direction
Start with tools/populate_case_matrix.py and trace how it selects the cases root and manifest. Add isolated tests for first and second runs, numbering, supported and planned cells, architecture aliases, hand-authored files, and invalid inputs; done means the second run writes zero files and preserved content is byte-for-byte unchanged without invoking a compiler or network service.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing, tooling
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100