amirbena / amirbena/code-review-skill
Build shared corpus-hygiene test infrastructure for benchmark and review fixture corpora
- Dominant language
- Python
- Stars
- 7
- Forks
- 2
- Avg merge
- 27m
- Merged PRs (30d)
- 188
Description
## Type
Refactor
## Area
Packaging / Portability
## Priority
P2 — Medium
Contributor-owned: pure test-infrastructure work; no fixture content, reference model, or corpus schema changes.
## Problem
20 files — 15 under `tests/unit/benchmark/*_corpus.py` and 5 under `tests/unit/review/{root_cause,repository_intelligence,specialist_depth}/*_corpus.py` — total 491 tests, each repeating the same generic ~15–20-test "corpus shape" check against a different `CORPUS_DIR` constant: directory exists with a README, sub-corpus stays small, every fixture file parses and validates, filename stem matches case id, case ids are unique, every case records a rationale and tags, every case pins an explicit consistent decision, patch-case anchors occur in the diff/base, no finding carries a severity outside P0/P1/P2, and README links every fixture / states the selection principle / uses the single reference validator. Only each corpus's own semantic assertions (which cases exist, what they specifically assert) are genuinely per-corpus.
## Goal
One shared, parametrizable corpus-hygiene test base (mixin or equivalent) that every corpus file's generic-shape checks delegate to, while every corpus keeps its own domain-specific semantic assertions in its owning file.
## Scope
- Design one shared corpus-hygiene test base/mixin covering the ~15–20 generic checks, parametrized by `CORPUS_DIR` (and any other per-corpus config already present, e.g. required tags or size bounds).
- Migrate all 20 files to use it for their generic-shape assertions only.
- Leave every corpus-specific semantic assertion (e.g. domain-specific case-count expectations, "all flagged cases are distinct defect kinds") in its owning file, unparametrized.
## Non-Goals
- Does not redesign the `benchmark-case/v1` fixture schema, the benchmark matcher, the finding model, or any `tests/reference/**` reference model — infrastructure only.
- Does not change which fixture files exist or their content.
- Does not merge the `tests/unit/benchmark/` and `tests/unit/review/` corpus trees into one directory — each stays under its current package.
## Acceptance Criteria
- [ ] Baseline test count for the 20 scoped files recorded (491).
- [ ] After-change test count recorded and reported.
- [ ] Every generic-shape check still runs, per corpus; no semantic assertion silently dropped.
- [ ] Corpus-specific content stays locally owned and unparametrized.
- [ ] A failing case still identifies which corpus/file broke.
- [ ] Scoped tests and the full suite (`python3 -m unittest discover -s tests -t .`) pass.
## Dependencies
Depends on: none
Blocks: none
Parent: #395
Independent of the other four children (disjoint files, no shared-helper collision) — can be implemented in parallel.
## Validation
- Before/after: `grep -rc "def test_" tests/unit/benchmark/*_corpus.py tests/unit/review/{root_cause,repository_intelligence,specialist_depth}/*_corpus.py`.
- `python3 -m unittest discover -s tests -t .`
Contributor guide
Assessment
This issue has not been assessed yet.