scverse / scverse/rustar-aligner
Align pipeline stops scaling past ~4-8 threads: decode and write are each a single thread
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 75
- Forks
- 7
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
Measurement
Synthetic 20 Mb genome, 2 M single-end 100 bp reads, --outSAMtype BAM Unsorted, macOS aarch64,
release build (with the batch-size fix from #222 already in, so this is the remaining ceiling):
--runThreadN |
wall | user CPU | speedup |
|---|---|---|---|
| 1 | 8.81s | 11.67s | 1.00x |
| 2 | 4.82s | 12.36s | 1.83x |
| 4 | 2.80s | 13.49s | 3.15x |
| 8 | 2.43s | 15.79s | 3.63x |
| 12 | 2.44s | 17.19s | 3.61x |
Throughput stops improving after ~8 threads while user CPU keeps climbing and sys roughly doubles
(1.96s → 4.45s). On a 32- or 64-core machine the extra cores are being paid for and not used.
Where it goes
Same input, 8 threads, output disabled:
| output | wall |
|---|---|
BAM Unsorted |
2.43s |
None |
1.60s |
So the writer stage alone is ~34% of wall at 8 threads, and it is one thread: SAM record → BAM
encode → BGZF deflate, serially, for the whole run.
Gzipped input costs on top of that (8 threads, BAM out): 2.74s from .fq.gz vs 2.43s from plain
FASTQ, i.e. ~13%, again on a single decode thread.
Why
The align pipelines are decode(1 thread) → align(N rayon workers) → write(1 thread), joined by
bounded channels. The two ends are serial, so Amdahl caps the whole thing regardless of
--runThreadN. The profile agrees: semaphore_wait_trap + __psynch_cvwait were ~16% of all
samples, i.e. workers parked waiting on the ends.
Options, in payoff order
- Multithreaded BGZF writer.
noodles-bgzf0.50+ runs its multithreaded writer on its own
local pool with a supportedwith_worker_count, so it no longer competes with rayon's global
pool. Biggest single lever, and the version bump is already in #207 / #211. Needs care: BGZF
block boundaries must stay byte-identical, or the change must be documented. - Parallel or chunked decode. Related to #95 (paraseq, built for exactly this) and #97
(parallel readers). For.gzspecifically, note that a BGZF-compressed FASTQ can be inflated in
parallel with thenoodles-bgzfwe already depend on, with no new dependency. - Only then consider reshaping the stage handoff itself (work-stealing over batches rather
than fixed producer/consumer threads). Do not start here: it is the largest change and the
smallest measured share.
Note
The numbers above are from a synthetic genome on a laptop; the shape (serial ends capping the
scaling) is architectural, but the exact crossover point should be re-measured on a real genome and
a real machine before sizing any of the fixes. That needs the benchmark harness from the
dependency sweep.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the benchmark harness from the dependency sweep and reproduce the decode(1 thread) → align(N workers) → write(1 thread) measurements on a real genome and machine. Read the current pipeline ends and the noodles-bgzf version changes referenced in #207 and #211, along with related issues #95 and #97. Done should include a measured improvement in scaling beyond roughly 4–8 threads without regressing output behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100