mlc-ai/tokenizers-cpp
View on GitHubUniversal cross-platform tokenizers binding to HF and sentencepiece
- Stars
- 512
- Forks
- 132
- Open beginner issues
- 0
- Indexed issues
- 26
- Dominant language
- C++
- License
- Apache-2.0
- Last GitHub push
- May 20, 2026
- Latest indexed
- Sep 19, 2026
- Contributing guide
- No contributing guide
- Code of conduct
- No code of conduct
- Beginner labels
- No beginner labels indexed
- PR merge metrics
- No merged PRs in 30d
2 beginner-friendly issues open
Loading issues
-
Missing Cargo.lock Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mlc-ai/tokenizers-cpp#96 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
mlc-ai/tokenizers-cpp#88 · 2 reactions ·