Speed up the unit test suite v2: setUpTestData deep-copies, audit tests, timing runner
- Dominant language
- Python
- Stars
- 30
- Forks
- 3
- Avg merge
- 9h 28m
- Merged PRs (30d)
- 42
Description
🤖 Written by Claude
Follow-up to #1856, which took the suite from 89 s to 65 s wall (203 s to 152 s CPU) on vg-test2 (4 cores, `--keepdb --parallel 4`, 3,212 tests). These are the cost centres that profiling found and #1856 deliberately left alone, roughly in payoff order. Numbers below are from the #1856 profile (8-core laptop, 666 s serial cProfile) unless stated; each wants re-measuring against the current tree before it is worked on.
## 1. `setUpTestData` deep-copies (~23 s)
Django copies every `setUpTestData` class attribute on each access, so a fixture object carrying populated `cached_property` values is expensive to touch: `VariantAnnotationVersion` 22 ms, `Analysis` 14 ms, `GenomeBuild` 13 ms per access. Options worth measuring:
- Store the pk in `setUpTestData` and re-fetch in the test where the object is only an FK target.
- `GenomeBuild` is in `PRODUCTION_CACHED_TABLES` and immutable — a classmethod accessor (`GenomeBuild.grch37()`) rather than a class attribute skips the copy entirely.
- Django exempts an attribute from copying if it is wrapped so it is not a model instance; check what the current Django offers before hand-rolling anything.
This is the largest single item left, and unlike the #1856 work it touches many test modules, so it wants a measurement of one module first to confirm the per-access cost is real here.
## 2. Audit tests (~20 s)
`library/tests/test_decorator_audit.py` AST-parses every source file and `library/tests/test_signal_receiver_registration.py` imports every app. Each blocks one worker for its whole duration, so on 4 workers they set a floor under the wall time even when everything else finishes. Either cache the scan (keyed on a hash of the tree) or move it into lint, where it belongs — it is checking source shape, not behaviour.
## 3. DB round trips (153 s, 207k queries at 0.7 ms)
The single biggest line in the profile, and mostly the honest cost of fixtures writing rows. Worth attacking only where a fixture builder is doing per-row work that could be one `bulk_create`; `snpdb/tests/utils/fake_cohort_data.py` and the `slowly_*` variant builders are the places to look. `assign_permission_to_user_and_groups` (the #1856 item 3) came out of this line and is done.
## 4. Partition DDL (15 s, 1,937 `CREATE TABLE ... INHERITS`)
Now that the runner seeds the annotation versions once per run (#1856), a good share of these are gone — this needs re-measuring before anything is decided. What remains is tests creating their own sub-versions, including the 25 classes that call `retire_seeded_annotation_version` and then build their own VAV.
## 5. `vg tests --timing`
The per-class / per-test timing runner used for the #1856 profile records `setUpClass`, per-test and `TestData` deep-copy time to JSONL. Making it a `VG_TEST_TIMING=` option on `VariantGridTestRunner` would make each of the items above measurable without rebuilding the harness each time. Cheap, and it is the thing that makes the rest honest.
## Not in scope
`AnalysisNode.save` (41 s, 1,223 saves) and the test-client requests (72 s, 935 requests) are production code under test. Making them faster is a production optimisation that happens to show up here, not a test-suite change — worth its own issue if the analysis pages want the work anyway.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by re-measuring the current suite and one module's setUpTestData cost with the existing timing runner described in the issue. Read library/tests/test_decorator_audit.py, library/tests/test_signal_receiver_registration.py, snpdb/tests/utils/fake_cohort_data.py, and the slowly_* variant builders to identify a focused workstream. Done means a measured, reviewed change that reduces the targeted cost without changing test behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- performance, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100