Support inputs with more than `u32::MAX` elements
- Dominant language
- Rust
- Stars
- 49
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Inputs with more than `u32::MAX` elements are currently not supported, because the [underlying implementation](https://github.com/gendx/paralight/blob/0.0.6/src/core/range.rs#L398) packs a `start..end` range of items as a `(u32, u32)` pair into an [`AtomicU64`](https://doc.rust-lang.org/std/sync/atomic/struct.AtomicU64.html). Being able to use an atomic type is essential for the algorithm to work, but the Rust standard library [doesn't offer](https://doc.rust-lang.org/std/sync/atomic/index.html) atomic types larger than `u64`, even though common CPUs support 128-bit atomics (e.g. [`cmpxchg16b`](https://doc.rust-lang.org/stable/core/arch/x86_64/fn.cmpxchg16b.html) on x86-64).
Possible ways forward:
- Detect CPU support and allow larger inputs on platforms that offer them.
- Depending on a crate such as [`portable_atomic`](https://docs.rs/portable-atomic/) to provide `AtomicU128` for us.
- If the input is larger than 2^32, process it into fewer than 2^32 chunks of 2^k items each, with k as small as possible.
Contributor guide
Research direction
Start in src/core/range.rs around line 398 and inspect how the start..end range is packed into AtomicU64. Compare CPU-detection, portable_atomic, and chunking approaches against the algorithm's atomic requirements. Done should mean inputs exceeding u32::MAX are supported without breaking the parallel range-processing behavior, with validation on relevant platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100