scverse / scverse/rustar-aligner
Dependency discussion: portable SIMD crate (wide/pulp) vs the hand-rolled intrinsics in simd_scan.rs
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 75
- Forks
- 7
- Avg merge
- 8m
- Merged PRs (30d)
- 1
Description
Dependency discussion per CONTRIBUTING.md. Gated on the benchmark-harness issue — this should
not turn into a PR before there is a repeatable measurement.
What exists today
Hand-rolled, and deliberately minimal:
src/align/simd_scan.rs—find_stop, a 16-byte chunked "does this chunk contain a stop"
reduction in SSE2 / NEON intrinsics, with a scalar re-scan to locate the exact position. The
module comment states the design rationale: keep the arch-specific surface to a trivial boolean
reduction, do position extraction in portable scalar code.src/cpu.rs— runtime feature detection, the build-flavour guard, andprefetch_read
(_mm_prefetch/prfm).
That is a good design and the bar for replacing it is correspondingly high.
Candidates
| Option | Version | License | Note |
|---|---|---|---|
wide |
1.6.0 |
Zlib OR Apache-2.0 OR MIT | Mature, fixed-width vector types, SSE2/AVX2/NEON/WASM. No multiversioning |
pulp |
0.22.3 |
MIT | Built-in multiversioning, proven by faer; native-width only, so code must handle variable-width chunks |
std::simd |
— | — | Nightly-only; the crate pins rust-version = "1.89" stable, so it is out |
The actual questions
- Does a portable crate buy wider vectors? Today's scan is fixed at 16 bytes, i.e. SSE2/NEON
width. AVX2 (32 B) or AVX-512 would halve or quarter the chunk count, but only with runtime
dispatch — which is exactly whatwidedoes not do andpulpdoes. - Five platforms, including Windows.
CONTRIBUTING.mdcalls this out. Any candidate has to
build cleanly everywhere, including the aarch64 macOS path. - Results must be bit-identical across dispatch paths. A scalar fallback, an SSE2 path and an
AVX2 path must return the samefind_stopindex. That needs a test exercising all compiled
paths, not just the native one. - Is
find_stopeven hot enough to matter? Answer with a profile before touching it.
Checklist
- Profile
find_stop's share of align wall time (blocked on the benchmark harness) - Try widening the existing hand-rolled scan to AVX2 with
is_x86_feature_detected!first,
since that costs zero dependencies and answers question 1 directly - Only if that is unsatisfying: prototype with
pulp(multiversioning is the differentiator) - Cross-path identity test: scalar vs each vector path
- Verify Windows + macOS aarch64 builds
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read CONTRIBUTING.md and the design comments in src/align/simd_scan.rs and src/cpu.rs. Wait for the benchmark harness, then profile find_stop, try the AVX2 widening experiment, and only prototype pulp if needed. Done means measured results, cross-path index identity tests, and verified Windows and aarch64 macOS builds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100