Structural variant effects() is very slow at some loci and for large typed SVs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 87
- Forks
- 28
- Avg merge
- 3h 27m
- Merged PRs (30d)
- 27
Description
Annotating every SV in the four esvee somatic VCFs from the public osteosarc.com dataset (1,797 SVs after loading, GRCh38 / Ensembl 115) with Variant.effects() took about 21 minutes. A handful of breakends dominate:
| SV | Time |
|---|---|
| BND chr20:58909683 ↔ 58909577 (GNAS) | 178 s |
| BND chr19:54155308 ↔ 54153839 (CNOT3) | 71 s |
| BND chr19:54153839 ↔ 54155308 (CNOT3, other half) | 69 s |
| BND chr20:36148058 ↔ 36148629 | 64 s |
A native stack sample during the run was dominated by SQLite (sqlite3VdbeExec, B-tree reads), i.e. pyensembl queries, rather than Python-side work.
Separately, breakend pairs that callers label SVTYPE=DEL/DUP/INV now load as typed SVs spanning the event, so effects() visits every transcript in the span. Some esvee events span tens of megabases (e.g. a 37 Mb INV on chr2), which makes full-span annotation expensive.
Ideas:
- profile
StructuralVariantAnnotator.annotate_on_transcriptand the cryptic-exon / splice-outcome enumeration it runs per effect; - cache per-transcript exon and cDNA lookups across effects;
- skip splice / cryptic enumeration for transcripts wholly inside a span, where no breakpoint touches them;
- offer a breakpoint-only mode for very large SVs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at StructuralVariantAnnotator.annotate_on_transcript and profile the cryptic-exon and splice-outcome enumeration, along with the pyensembl SQLite queries. Reproduce the four osteosarc.com VCF benchmark and inspect the listed slow breakends and large typed SVs. Done means establishing a faster effects() path while preserving annotation results for these cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- bioinformatics, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100