SACGF / SACGF/variantgrid

Analysis - DamageNode optimisations

Open
#1,535 1 comment 0 reactions 0 assignees View on GitHub
Analysis optimisation
Dominant language
Python
Stars
30
Forks
3
Avg merge
9h 22m
Merged PRs (30d)
40

Description

Split out from #574

* We check for not variant class variant_class" = 'SN' - which makes an inner query - could try and optimise
* We check for Consequence::text like splice or splice regions is not null - we preprocess this?
* We check if any of 6 splice scores are above a threshold - we could preprocess and store max_score for all of them and only check that

--------

**variant class != SN**

* Original inner query - 4.3s
* Tried ref/alt length > 1 - 4.2s
* List indel variant classes (same result but not sure this is exactly the same as there is also ALU insert) - 3.7
* __ne not equal lookup - 3.8s

For the other stuff, since it requires a schema migration, I might do it along with the split up table work

```
# These splicing fields are pre-calculated to make DamageNode queries faster
max_splice_score = models.FloatField(null=True, blank=True) # Of spliceAI and dbscsnv_ada_score/dbscsnv_rf_score
# (Q(consequence__contains='splice')|Q(splice_region__isnull=False)) & Q(variantannotation__variant_class__ne="SN")
splice_indel = models.BooleanField(default=False, blank=True)
```

The only reference to in SA Path prod is in a Qase test case

```
FilterNodeItem.objects.filter(field__icontains='spliceai').values("filter_node__analysis")
```

http://frgeneseq02.imvs.sa.gov.au:92/analysis/18907/

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names FilterNodeItem, the Qase lookup, and the proposed max_splice_score and splice_indel model fields; begin by tracing those references and the existing DamageNode filters. Measure the listed variant-class and splice predicates before deciding whether preprocessing or a schema migration belongs with the split-up table work. Done means the agreed query path is faster without changing results, with the relevant migration and checks covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.