SACGF / SACGF/variantgrid

N+1 queries in snpdb/annotation/variantopedia grids and views

Open
#1,730 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
30
Forks
3
Avg merge
9h 28m
Merged PRs (30d)
42

Description

🤖 Written by Claude.

snpdb/annotation/variantopedia half of the N+1 sweep in #1723 (the classification half is tracked separately). All of these are fixed by adding `select_related`/`prefetch_related` or by hoisting a per-row query out of a loop.

Structural note: `DatabaseTableView.prepare_results` (`snpdb/views/datatable_view.py:509`) iterates `qs.values(...)`, so `select_related` has no effect there — every N+1 in a DataTable comes from a `renderer=` callback issuing its own query per row. Page size goes up to 100. There is a `pre_render(qs)` hook (line 359) that is the natural place for bulk fetches, currently used only by `ClinVarExportColumns`.

### Grids

Both of these do one fetch per row and are fixable with the `pre_render` hook:
- `snpdb/grids.py:686` — AlleleLiftover grids fetch the same `Allele` twice per row, then two more queries via `variant_for_build_optional`.
- `variantopedia/grids.py:372` — `can_write` instantiates a `VariantTag` per row (the docstring already flags this).

### Hot views

- `snpdb/variant_sample_information.py:392` — `latest_for_user` applies no `select_related`, so each row refetches `Classification` (wide `evidence` JSONB) plus `lab`. Line 444 fetches `Variant` rows without their sequences, and `str(v)` then costs three queries each via `locus.contig.name`, `locus.ref.seq`, `alt.seq`.
- `annotation/transcripts_annotation_selections.py:278` — `gene_version` is a lazy FK over a loop that pulls every transcript version for the variant's gene symbols (often 50-300 rows). Used by the variant page, classification autopopulate and the classification detail view.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with DatabaseTableView.prepare_results and its pre_render hook in snpdb/views/datatable_view.py, then inspect the renderer callbacks at snpdb/grids.py:686 and variantopedia/grids.py:372. Trace the related queries in snpdb/variant_sample_information.py:392 and :444 and annotation/transcripts_annotation_selections.py:278. Done means these grids and views use bulk related-object fetching or hoisted queries instead of per-row N+1 queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, database, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.