Consider recalibrating how bits are divided in Span
- Dominant language
- Rust
- Stars
- 934
- Forks
- 135
- PR merge metrics
- No merged PRs in 30d
Description
Currently fallback spans store a pair of 32-bit low and high character indices.
https://github.com/dtolnay/proc-macro2/blob/fecb02df0e2966c7eb39e020dbf650fa8bafd0c5/src/fallback.rs#L491-L496
A span in which `lo > hi` is malformed, so right off the bat, approximately half of possible Span bit patterns are wasted.
Separately, tokens are usually small compared to the total amount of input parsed by a thread. If we switch to storing `lo` and `hi - lo` instead of `lo` and `hi`, then an even split of 32 bits each may not be the wisest allocation. For example, we could decide to give 36 bits to `lo` (supporting 64 GB input size) and 28 bits to `hi - lo` (limiting token size to 256 MB). Or some other uneven split.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.