Enforce a coverage ratchet in CI
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 65/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- github-actions, python
- Domain
- ci-cd, developer-experience, testing-qa
Research direction
Start by reading .github/workflows/ci.yml, the [tool.coverage.report] section in pyproject.toml, and the relevant guidance in CONTRIBUTING.md. Run make test locally and inspect the coverage output to establish the current floor, then compare the single gated matrix cell with the other CI jobs. Done means the committed floor matches measured coverage, regressions fail clearly, the policy is documented, and full-matrix runtime does not materially increase.
Written by the indexing model from the issue text.
Description
Summary
CI runs the test suite with --cov=contextweaver --cov-report=term-missing on every matrix cell, but nothing consumes the number: there is no fail_under, no upload/tracking service, and no diff coverage on PRs. Coverage is paid for on every run and can regress silently. This issue adds a ratcheted floor.
Why this matters
With 1,848 tests and branch coverage already configured ([tool.coverage.run] branch = true), the project is one config line away from making coverage a guarantee instead of a scrolling log artifact. A ratchet (set at the current measured level, raised deliberately) prevents the slow erosion that otherwise accompanies rapid feature addition — and several issues from this audit cycle add code in security-grade areas where untested branches are most costly.
Current evidence
.github/workflows/ci.yml— pytest invoked with coverage flags; no consumer of the result.pyproject.toml[tool.coverage.report]—show_missing,exclude_linesconfigured; nofail_under.- No codecov/coveralls integration; the floor-deps job deliberately skips
--cov(fine — it proves resolvability, not behavior).
Proposed implementation
- Measure current line+branch coverage on main (one CI run's
term-missingoutput or a local run). - Set
fail_underin[tool.coverage.report]to the measured value, rounded down to the nearest whole percent. - Run the coverage-gated pytest on a single matrix cell (e.g. the 3.12 cell that already runs the extra steps) to avoid quadrupling the cost; other cells can run with
--no-covfor speed if desired. - Document the ratchet policy in
CONTRIBUTING.md: the floor only moves up; PRs that meaningfully raise coverage may bump it. - Optional follow-up (separate decision): diff coverage on PRs via
diff-coveror a coverage-report job summary ($GITHUB_STEP_SUMMARYworks for fork PRs, unlike sticky comments).
Acceptance criteria
- A PR that drops total coverage below the committed floor fails CI with a clear message.
- The floor matches measured main-branch coverage at merge time (no aspirational number).
- CONTRIBUTING.md documents the policy.
- Full-matrix CI wall time does not increase materially.
Test plan
- Deliberately add an uncovered branch in a scratch branch and confirm the gate fails; revert.
- Confirm
make testlocally reports the same number CI enforces.
Migration notes
Contributor workflow change: PRs adding significant untested code now fail a named gate instead of merging silently. Not expected to require other migration.
Risks and tradeoffs
- A single global number can be gamed by adding trivial tests; it is a floor, not a target — review remains the real gate.
- Coverage-measurement flakiness across Python versions is avoided by gating on one pinned cell.
Suggested labels
testing, developer-experience
- Dominant language
- Python
- Stars
- 9
- Forks
- 17
- Avg merge
- 21h 36m
- Merged PRs (30d)
- 22
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from dgenio/contextweaver
-
complexity:simple priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
dgenio/contextweaver#421 ·
-
adoption investigation priority: high product
Difficulty 5/5 Over a week Newbie friendliness 25/100
dgenio/contextweaver#855 · 2 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
dgenio/contextweaver#846 ·
-
adoption complexity:average investigation priority: high product
Difficulty 5/5 Over a week Newbie friendliness 25/100
dgenio/contextweaver#840 · 3 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
dgenio/contextweaver#834 ·
All issues in dgenio/contextweaver
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100