SACGF / SACGF/variantgrid

Speed up the unit test suite v2: setUpTestData deep-copies, audit tests, timing runner

Open
#1,858 0 comments 0 reactions 0 assignees View on GitHub
optimisation
Dominant language
Python
Stars
30
Forks
3
Avg merge
9h 28m
Merged PRs (30d)
42

Description

🤖 Written by Claude

Follow-up to #1856, which took the suite from 89 s to 65 s wall (203 s to 152 s CPU) on vg-test2 (4 cores, `--keepdb --parallel 4`, 3,212 tests). These are the cost centres that profiling found and #1856 deliberately left alone, roughly in payoff order. Numbers below are from the #1856 profile (8-core laptop, 666 s serial cProfile) unless stated; each wants re-measuring against the current tree before it is worked on.

## 1. `setUpTestData` deep-copies (~23 s)

Django copies every `setUpTestData` class attribute on each access, so a fixture object carrying populated `cached_property` values is expensive to touch: `VariantAnnotationVersion` 22 ms, `Analysis` 14 ms, `GenomeBuild` 13 ms per access. Options worth measuring:

- Store the pk in `setUpTestData` and re-fetch in the test where the object is only an FK target.
- `GenomeBuild` is in `PRODUCTION_CACHED_TABLES` and immutable — a classmethod accessor (`GenomeBuild.grch37()`) rather than a class attribute skips the copy entirely.
- Django exempts an attribute from copying if it is wrapped so it is not a model instance; check what the current Django offers before hand-rolling anything.

This is the largest single item left, and unlike the #1856 work it touches many test modules, so it wants a measurement of one module first to confirm the per-access cost is real here.

## 2. Audit tests (~20 s)

`library/tests/test_decorator_audit.py` AST-parses every source file and `library/tests/test_signal_receiver_registration.py` imports every app. Each blocks one worker for its whole duration, so on 4 workers they set a floor under the wall time even when everything else finishes. Either cache the scan (keyed on a hash of the tree) or move it into lint, where it belongs — it is checking source shape, not behaviour.

## 3. DB round trips (153 s, 207k queries at 0.7 ms)

The single biggest line in the profile, and mostly the honest cost of fixtures writing rows. Worth attacking only where a fixture builder is doing per-row work that could be one `bulk_create`; `snpdb/tests/utils/fake_cohort_data.py` and the `slowly_*` variant builders are the places to look. `assign_permission_to_user_and_groups` (the #1856 item 3) came out of this line and is done.

## 4. Partition DDL (15 s, 1,937 `CREATE TABLE ... INHERITS`)

Now that the runner seeds the annotation versions once per run (#1856), a good share of these are gone — this needs re-measuring before anything is decided. What remains is tests creating their own sub-versions, including the 25 classes that call `retire_seeded_annotation_version` and then build their own VAV.

## 5. `vg tests --timing`

The per-class / per-test timing runner used for the #1856 profile records `setUpClass`, per-test and `TestData` deep-copy time to JSONL. Making it a `VG_TEST_TIMING=` option on `VariantGridTestRunner` would make each of the items above measurable without rebuilding the harness each time. Cheap, and it is the thing that makes the rest honest.

## Not in scope

`AnalysisNode.save` (41 s, 1,223 saves) and the test-client requests (72 s, 935 requests) are production code under test. Making them faster is a production optimisation that happens to show up here, not a test-suite change — worth its own issue if the analysis pages want the work anyway.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by re-measuring the current suite and one module's setUpTestData cost with the existing timing runner described in the issue. Read library/tests/test_decorator_audit.py, library/tests/test_signal_receiver_registration.py, snpdb/tests/utils/fake_cohort_data.py, and the slowly_* variant builders to identify a focused workstream. Done means a measured, reviewed change that reduces the targeted cost without changing test behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.