killercup / killercup/simd-utf8-check
Figure out a way to speed up ASCII detection
- Dominant language
- Rust
- Stars
- 14
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
The SIMD version is quite fast when looking at text that uses many multi-byte code points, but the std impl is way faster at checking ASCII-only or heavily ASCII-favoring text. I have yet to understand all the SIMD code as well as the code in std, but it might be interesting to see how expensive the trade-off between "early exit when ASCII" and "branchless SIMD code" is.
cc @bluss who wrote the current std::str::from_utf8 implementation
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the SIMD implementation and comparing it with std::str::from_utf8, focusing on ASCII-only, ASCII-heavy, and multi-byte inputs. Measure the trade-off between an ASCII early exit and branchless SIMD processing; done means identifying and implementing a faster approach without regressing the other input patterns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100