scverse / scverse/rustar-aligner
dev-deps: add a benchmark harness (divan or criterion) — there is no benches/ today
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 75
- Forks
- 7
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
There is no benches/ directory and no benchmark harness in Cargo.toml. Several open dependency
questions (SIMD crates, packed-seq, external sort, sufr vs caps-sa, #162) all start with
"measure first", and each currently has to invent its own measurement. One harness, once, unblocks
all of them.
Candidates
| Crate | Version | License | Note |
|---|---|---|---|
divan |
0.1.21 |
MIT OR Apache-2.0 | Simple API, generic benchmarks, allocation counting |
criterion |
0.8.2 |
Apache-2.0 OR MIT | Incumbent, statistics and regression reports, heavier |
Both are dev-dependencies only, so nothing ships in the binary and the supply-chain argument in
CONTRIBUTING.md is much weaker than for a runtime dep.
What to benchmark
The paths the open questions actually care about:
genomeGeneratesuffix-array construction (src/index/sa_build.rs), in-mem and ext-mem arms- Seed extension /
find_stop(src/align/simd_scan.rs) - FASTQ decode, plain and gzipped (
src/io/fastq.rs) - BAM write and coordinate sort (
src/io/bam.rs) - End-to-end align on a fixed small fixture, as a coarse regression signal
Constraints
- Benchmarks must not gate CI correctness; a slow machine must never fail the build.
- Numbers are machine-dependent. Useful as A/B within one machine and one session, not as absolute
thresholds checked into tests. - Keep the fixture small enough to run locally, large enough not to be dominated by process startup.
Checklist
- Pick one harness, not both
- Add the five benches above
- Document in
CONTRIBUTING.mdhow to run them and how to report an A/B in a PR - Link this issue from the dependency issues that are gated on measurement
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading Cargo.toml and CONTRIBUTING.md, then inspect the listed paths: src/index/sa_build.rs, src/align/simd_scan.rs, src/io/fastq.rs, and src/io/bam.rs. Choose either divan or criterion, add the five requested benchmarks with a small fixed fixture, and document local execution and A/B reporting in CONTRIBUTING.md; done also includes linking the dependency issues.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100