ContextLab / ContextLab/clustrix

Extract the salvageable pieces from the closed test-coverage epic (PR #128)

Open
#131 1 comment 0 reactions 0 assignees View on GitHub
P2-medium testing
Dominant language
Python
Stars
10
Forks
4
Avg merge
6h 27m
Merged PRs (30d)
9

Description

Part of #108 · Follow-up to the closed PR #128.

PR #128 (branch `epic/test-coverage-90-percent`, **retained on the remote**) was closed without merging — see the review on that PR. A few pieces in it are genuinely worth having. Extract each onto its own branch off `master`, reviewed on its own merits.

Ordered by value:

## 1. `tests/test_loop_analysis_ast.py` + `tests/test_loop_analysis_advanced.py`
61 tests, **zero mocks**, real AST parsing. The best tests in that PR by a wide margin, and the only ones that would survive #117 unchanged.
- [ ] Move into `tests/unit/` so CI actually executes them
- [ ] Confirm they still pass against `master`'s `loop_analysis.py` (they were written against the branch's rewritten 1,712-line version)

## 2. `tests/test_aws_clusters.py:32-36` as the reference pattern for #117
Uses a real `boto3.client` with `botocore.stub.Stubber` — real client, real request/response shapes, no network, no `MagicMock`. This is the pattern #117 should standardise on.
- [ ] Extract as a worked example and reference it from #117

## 3. `scripts/test_discovery.py` — report only
Its output is a real backlog: 1,532 marker-hygiene findings, mostly *"test in integration directory should have `@pytest.mark.integration`"*.
- [ ] Land as a reporting script, not a gate
- [ ] Feed its findings into the marker inventory needed by the `pytest.ini` issue

## 4. `scripts/quality_gates.py` + `.github/workflows/quality_gates.yml` — non-blocking
Per #101/#112 this workflow has never run. In PR #128 it fails its own thresholds (discovery 74.8% vs 95% required; categorization 10.9% vs 80%), and `quality-enforcement` `needs:` two jobs that both fail — so `scripts/quality_gates.py` has **literally never executed**.
- [ ] Run black on it first (not currently black-clean)
- [ ] Land with `continue-on-error: true` or schedule-only
- [ ] Set thresholds to today's measured actuals, then ratchet upward
- [ ] Do **not** make it merge-blocking until the numbers it measures are real

## 5. Coverage plumbing
- [ ] `--cov-branch` / `branch = true`, coverage artifact upload, Codecov step, `.github/scripts/update_coverage_badge.py`, `docs/infrastructure_quality_gates.md`. Pairs with #115.

## 6. `clustrix/cloud_providers/aws.py` `KeyName` fix
Refactored so `KeyName` is omitted when falsy. Needs a decision rather than a straight port: launching a **keyless, SSH-unreachable** EC2 instance silently is arguably worse than raising.
- [ ] Land as its own PR with that decision recorded

## Explicitly NOT extracting
- The five `notebook_magic_{aws,azure,gcp,ssh,enhanced}.py` modules (~2,678 lines, zero importers, and they import the mock shim) — these are what #122 deletes
- The three root `*_config.yml` files
- The `pytest.ini` deletion and `testpaths`/`-n 4` changes — handled deliberately in the config issue instead
- The mock-heavy AWS/executor/widget/utils test files (+695 mock occurrences, ~28% growth on what #117 must remove)

Contributor guide

Open the contributing guide

Research direction

Start with the review on PR #128 and the ordered list in this issue, beginning with tests/test_loop_analysis_ast.py and tests/test_loop_analysis_advanced.py. Check the tests against master’s loop_analysis.py, then review the named AWS, discovery, quality-gate, coverage, and AWS provider files as separate pieces. Done means each selected piece has its own branch and review, while excluded files and changes remain untouched.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, github-actions, python
Domain
ci-cd, cloud, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.