scverse / scverse/rustar-aligner
Dependency discussion: sufr/libsufr for suffix-array construction (vs the caps-sa incumbent)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 75
- Forks
- 7
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
Dependency discussion per CONTRIBUTING.md ("New dependencies need prior discussion"). Sibling of
#162 (libsais), same subsystem, different candidate.
Incumbent
caps-sa 0.6 drives src/index/sa_build.rs: the segmented arm (caps_sa::SegmentedText +
StarSegmentedText), build_ext_mem for production-scale genomes with build_in_memory below the
16 MB threshold, streaming filter+pack into the PackedArray so the full SA is never materialized
twice. Byte-for-byte STAR-compatible, verified by the
segmented_arm_matches_sentinel_arm_byte_for_byte_* differential tests.
Candidate
| Crate | Version | License | Last release |
|---|---|---|---|
sufr / libsufr |
0.7.12 |
BSD-3-Clause | 2025-02-27 |
Wheeler Lab. Parallel SA + LCP construction, explicitly "inspired by CaPS-SA" and the C++ CaPS-SA
implementation, i.e. the same algorithm family as the incumbent. Extras it advertises:
low-memory and very-low-memory modes (suffixes from disk, then text from disk too), a DNA mode
with softmask/ambiguity handling, configurable sequence delimiters, an LCP array, and a
count/locate/extract search API over its own .sufr file format.
Why this probably ends in "declined", and what would change that
Almost every sufr differentiator is either already solved here or unwanted:
- Ext-mem —
caps_sa::build_ext_memalready bounds peak RAM at ~O(text + n/p). - DNA mode, delimiters, sentinels — the sentinel/segmented transform in
sa_build.rsexists
precisely because STAR's generalized-SA order is not the naive DNA-mode order. A crate's own
DNA handling is a liability here, not a feature. - LCP array,
.sufrformat, search API — unused; we need the permutation only. - Maintenance — last release 2025-02-27, ~18 months ago.
What would flip the verdict: a measured win on genomeGenerate wall time or peak RSS versus
caps-sa on a real genome, at equal output bytes. Nothing else.
Checklist
- Confirm the SA it produces can be made byte-identical to today's packed SA (the segmented /
spacer-as-largest convention is the crux, same question as for any external SACA) - Benchmark vs
caps-saext-mem and in-mem on a real genome: wall time and peak RSS - Enumerate transitive deps; BSD-3-Clause is a third license to carry alongside MIT
- Cross-check against #162 so the three candidates (caps-sa, libsais, sufr) are decided together
rather than pairwise - Record the outcome in writing (see the DEPENDENCIES sub-issue) so this is not re-surveyed
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with CONTRIBUTING.md and src/index/sa_build.rs, then inspect the segmented_arm_matches_sentinel_arm_byte_for_byte_* differential tests. Benchmark sufr/libsufr against caps-sa in-memory and external-memory builds on a real genome, comparing wall time, peak RSS, and equal output bytes; enumerate transitive dependencies and record the decision with the other candidates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- bioinformatics, build-system, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100