scverse / scverse/rustar-aligner

Dependency discussion: sufr/libsufr for suffix-array construction (vs the caps-sa incumbent)

Open
#202 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dependencies
Dominant language
Rust
Stars
75
Forks
7
Avg merge
8m
Merged PRs (30d)
1

Description

Dependency discussion per CONTRIBUTING.md ("New dependencies need prior discussion"). Sibling of
#162 (libsais), same subsystem, different candidate.

Incumbent

caps-sa 0.6 drives src/index/sa_build.rs: the segmented arm (caps_sa::SegmentedText +
StarSegmentedText), build_ext_mem for production-scale genomes with build_in_memory below the
16 MB threshold, streaming filter+pack into the PackedArray so the full SA is never materialized
twice. Byte-for-byte STAR-compatible, verified by the
segmented_arm_matches_sentinel_arm_byte_for_byte_* differential tests.

Candidate

Crate Version License Last release
sufr / libsufr 0.7.12 BSD-3-Clause 2025-02-27

Wheeler Lab. Parallel SA + LCP construction, explicitly "inspired by CaPS-SA" and the C++ CaPS-SA
implementation, i.e. the same algorithm family as the incumbent. Extras it advertises:
low-memory and very-low-memory modes (suffixes from disk, then text from disk too), a DNA mode
with softmask/ambiguity handling, configurable sequence delimiters, an LCP array, and a
count/locate/extract search API over its own .sufr file format.

Why this probably ends in "declined", and what would change that

Almost every sufr differentiator is either already solved here or unwanted:

  • Ext-memcaps_sa::build_ext_mem already bounds peak RAM at ~O(text + n/p).
  • DNA mode, delimiters, sentinels — the sentinel/segmented transform in sa_build.rs exists
    precisely because STAR's generalized-SA order is not the naive DNA-mode order. A crate's own
    DNA handling is a liability here, not a feature.
  • LCP array, .sufr format, search API — unused; we need the permutation only.
  • Maintenance — last release 2025-02-27, ~18 months ago.

What would flip the verdict: a measured win on genomeGenerate wall time or peak RSS versus
caps-sa on a real genome, at equal output bytes. Nothing else.

Checklist

  • Confirm the SA it produces can be made byte-identical to today's packed SA (the segmented /
    spacer-as-largest convention is the crux, same question as for any external SACA)
  • Benchmark vs caps-sa ext-mem and in-mem on a real genome: wall time and peak RSS
  • Enumerate transitive deps; BSD-3-Clause is a third license to carry alongside MIT
  • Cross-check against #162 so the three candidates (caps-sa, libsais, sufr) are decided together
    rather than pairwise
  • Record the outcome in writing (see the DEPENDENCIES sub-issue) so this is not re-surveyed

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CONTRIBUTING.md and src/index/sa_build.rs, then inspect the segmented_arm_matches_sentinel_arm_byte_for_byte_* differential tests. Benchmark sufr/libsufr against caps-sa in-memory and external-memory builds on a real genome, comparing wall time, peak RSS, and equal output bytes; enumerate transitive dependencies and record the decision with the other candidates.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
bioinformatics, build-system, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.