CI: pilot pytest-xdist on one slow batch
- Dominant language
- Python
- Stars
- 124
- Forks
- 24
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 60
Description
## Opportunity
Slow tests are single-core by construction (`cores=1, chains=1, tune=10, draws=10`), so 3 of the runner's 4 vCPUs idle through every slow batch. `pytest-xdist` is already installed but unused. Potential: 2–3× per batch (a balanced ~30-min batch → ~12–16 min), the single largest remaining speedup after batching/rebalancing.
## Known risks (why this is a pilot, not a rollout)
- **Memory:** RSS grows ~0.6 → 4.9 GiB across one serial batch (native JAX/pytensor allocations; measured via the `HSSM_TEST_RSS_LOG` fixture in `tests/conftest.py`). N workers multiply resident sets on a 16 GB runner.
- `log_cli` / capture interactions garble under xdist.
- `pytest-rerunfailures` + xdist have known quirks.
- Per-worker pytensor/JAX compilation may eat into the win.
## Plan
1. Prerequisites: `--durations` telemetry and the batch rebalance must be landed first (see the addopts-cleanup and rebalance issues).
2. Add **one** matrix entry (single batch, single Python) running `-n 4 --dist loadfile`, with `HSSM_TEST_RSS_LOG=1`.
3. Observe for ~a week of main pushes: wall-clock, peak RSS per worker, flake/rerun rate vs. the serial twin batch.
4. Decide and record on this issue: roll out to all batches / reduce to `-n 2` / abandon.
## Acceptance
- A data-backed go/no-go decision with the measurements attached, not a silent adoption.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start with tests/conftest.py to understand HSSM_TEST_RSS_LOG, then inspect the existing CI matrix and serial slow-batch configuration. Add the single specified -n 4 --dist loadfile pilot with RSS logging, compare it with the serial twin for about a week, and attach wall-clock, peak RSS, and flake/rerun measurements before recording a go/no-go decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100