rust-lang / rust-lang/portable-simd
Consider enabling other indices than usize
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 1.1k
- Forks
- 108
- Avg merge
- 22h 53m
- Merged PRs (30d)
- 3
Description
This would minimize contortions for casting between various index sizes.
See: https://github.com/rust-lang/rust/issues/89193#issuecomment-92627025
Since there is a bit of chatter here on the use cases for performing a scatter or gather for indexes, I can give a little bit of context, since I do think it is somewhat informative.
In this case I am working on a columnar store. You can kind of think of the use case as a secondary index over columnar data. For just a single primary index, you have a list of row_ids Simd<usize, LANES> and you can perform a simd gather instruction to get all of the values at those indexes. However, if you have a secondary index the simd gather from that secondary index needs to produce a Simd<usize, LANES>, so that can be passed directly into the second of the gather functions over the primary keys. In this way you go from needed 2*LANES loads to perform a secondary index lookup over data to only needing 2 simd operations.
Alternatively it would be nice if the Simd libraries were updated to perform gathers using indexes as Simd<u32, LANES> or Simd<u64, LANES>. That would dodge this issue altogether and it would mean less copies in our columnar store. Many times columnar stores you keep offsets as u32 (or even smaller) to keep things compressed. And it is nice to be able to act directly on those indices rather than performing intermediate conversions to usize.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no repository files, tests, or entry points. Start with the linked Rust issue and its discussion, then inspect the gather and scatter APIs mentioned in the report. Before implementation, establish which non-usize index types are supported and define a concrete acceptance criterion; the issue currently does not provide one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100