killercup / killercup/simd-utf8-check

Figure out a way to speed up ASCII detection

Open
#2 2 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Rust
Stars
14
Forks
1
PR merge metrics
No merged PRs in 30d

Description

The SIMD version is quite fast when looking at text that uses many multi-byte code points, but the std impl is way faster at checking ASCII-only or heavily ASCII-favoring text. I have yet to understand all the SIMD code as well as the code in std, but it might be interesting to see how expensive the trade-off between "early exit when ASCII" and "branchless SIMD code" is.

cc @bluss who wrote the current std::str::from_utf8 implementation

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the SIMD implementation and comparing it with std::str::from_utf8, focusing on ASCII-only, ASCII-heavy, and multi-byte inputs. Measure the trade-off between an ASCII early exit and branchless SIMD processing; done means identifying and implementing a faster approach without regressing the other input patterns.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.