tests: pytest addopts cleanup — drop global --cov/--exitfirst/--capture=no, add --durations, raise --timeout
- Dominant language
- Python
- Stars
- 124
- Forks
- 24
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 60
Description
## Problem
The `[tool.pytest.ini_options]` addopts in `pyproject.toml` apply to **every** pytest invocation (PR fast jobs, slow batches, release gate, coverage job), and several are costing time or hiding information:
- `--cov=. --cov-report=term-missing --cov-report=xml` — instruments every job, but the coverage produced by PR/slow/release jobs is never uploaded anywhere (only `coverage.yml` uploads). Measured overhead: up to ~2.5× on Python-heavy code with the default tracer (+21% with `COVERAGE_CORE=sysmon`). Also `--cov=.` silently **unions** with `coverage.yml`'s CLI `--cov=src/hssm`, defeating its scoping.
- `--exitfirst` — combined with 20–55 min CI iterations, this yields one-failure-per-push debugging; in the coverage job a single late failure aborts a ~2 h run with nothing uploaded.
- `--capture=no` + `log_cli = INFO` — a log firehose that is currently the *only* per-test timing source, because no `--durations` is configured (this analysis had to scrape GH Actions log timestamps).
- `--timeout=360` — signal-based: it does not guard collection (#1061) and it converted the suite's slowest test into a guaranteed timeout+untimed-rerun double-run (see the waic_loo issue).
- `--reruns=2` — masks real flakes; worst case triples a failing slow test.
## Plan
- Replace addopts with: `--durations=50 --durations-min=1 --timeout=600 --reruns=1 --reruns-delay=5`.
- Move coverage flags out of addopts; `coverage.yml` already passes `--cov=src/hssm --cov-report=xml` on its own CLI (verify post-removal; also set `COVERAGE_CORE=sysmon` there).
- Drop `--exitfirst` (use `-x` locally when wanted).
- Drop `--capture=no`; set `log_cli_level = "WARNING"`. If hang forensics are still wanted on slow batches, pass `-o log_cli_level=INFO` only on the slow-path pytest invocation in `run_tests.yml`.
- Fix stale docs in the same PR: the `--runslow` flag referenced in the `slow` marker help text (`pyproject.toml`) and in `CLAUDE.md` **does not exist** (there is no `pytest_addoption` hook — plain `pytest` runs everything, which is also why the coverage job silently runs the full suite). Either implement the flag or correct both references to `-m "not slow"` / `-m slow`.
## Acceptance
- `--durations=50` tables appear in all CI jobs (future audits become free).
- Fast PR job and slow batches measurably faster (record before/after in the PR).
- `coverage.yml` still uploads a valid `coverage.xml` scoped to `src/hssm`.
- No stale `--runslow` references remain.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start with the pytest options in pyproject.toml, then inspect coverage.yml, run_tests.yml, and the --runslow references in CLAUDE.md. Verify the coverage command remains scoped to src/hssm, choose whether to implement or correct the stale flag references, and compare CI timings before and after. Done means all CI jobs show duration tables, coverage.xml uploads successfully, and no stale --runslow references remain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, documentation, testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100