scverse / scverse/rustar-aligner

dev-deps: add a benchmark harness (divan or criterion) — there is no benches/ today

Open
#204 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dependencies
Dominant language
Rust
Stars
75
Forks
7
Avg merge
8m
Merged PRs (30d)
1

Description

There is no benches/ directory and no benchmark harness in Cargo.toml. Several open dependency
questions (SIMD crates, packed-seq, external sort, sufr vs caps-sa, #162) all start with
"measure first", and each currently has to invent its own measurement. One harness, once, unblocks
all of them.

Candidates

Crate Version License Note
divan 0.1.21 MIT OR Apache-2.0 Simple API, generic benchmarks, allocation counting
criterion 0.8.2 Apache-2.0 OR MIT Incumbent, statistics and regression reports, heavier

Both are dev-dependencies only, so nothing ships in the binary and the supply-chain argument in
CONTRIBUTING.md is much weaker than for a runtime dep.

What to benchmark

The paths the open questions actually care about:

  1. genomeGenerate suffix-array construction (src/index/sa_build.rs), in-mem and ext-mem arms
  2. Seed extension / find_stop (src/align/simd_scan.rs)
  3. FASTQ decode, plain and gzipped (src/io/fastq.rs)
  4. BAM write and coordinate sort (src/io/bam.rs)
  5. End-to-end align on a fixed small fixture, as a coarse regression signal

Constraints

  • Benchmarks must not gate CI correctness; a slow machine must never fail the build.
  • Numbers are machine-dependent. Useful as A/B within one machine and one session, not as absolute
    thresholds checked into tests.
  • Keep the fixture small enough to run locally, large enough not to be dominated by process startup.

Checklist

  • Pick one harness, not both
  • Add the five benches above
  • Document in CONTRIBUTING.md how to run them and how to report an A/B in a PR
  • Link this issue from the dependency issues that are gated on measurement

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading Cargo.toml and CONTRIBUTING.md, then inspect the listed paths: src/index/sa_build.rs, src/align/simd_scan.rs, src/io/fastq.rs, and src/io/bam.rs. Choose either divan or criterion, add the five requested benchmarks with a small fixed fixture, and document local execution and A/B reporting in CONTRIBUTING.md; done also includes linking the dependency issues.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.