mlc-ai/tokenizers-cpp
View on GitHubUniversal cross-platform tokenizers binding to HF and sentencepiece
- Stars
- 512
- Forks
- 132
- Open beginner issues
- 0
- Indexed issues
- 26
- Dominant language
- C++
- License
- Apache-2.0
- Last GitHub push
- May 20, 2026
- Latest indexed
- Sep 19, 2026
- Contributing guide
- No contributing guide
- Code of conduct
- No code of conduct
- Beginner labels
- No beginner labels indexed
- PR merge metrics
- No merged PRs in 30d
-
Missing Cargo.lock Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mlc-ai/tokenizers-cpp#96 · 1 reaction ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
mlc-ai/tokenizers-cpp#95 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#94 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#91 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#89 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
mlc-ai/tokenizers-cpp#88 · 2 reactions ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
mlc-ai/tokenizers-cpp#84 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 38/100
mlc-ai/tokenizers-cpp#83 ·
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
mlc-ai/tokenizers-cpp#82 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#74 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#72 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#71 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#63 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 30/100
mlc-ai/tokenizers-cpp#61 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#58 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#56 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
mlc-ai/tokenizers-cpp#54 ·
-
Building for size Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#51 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
mlc-ai/tokenizers-cpp#50 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 32/100
mlc-ai/tokenizers-cpp#48 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
mlc-ai/tokenizers-cpp#45 · 6 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 28/100
mlc-ai/tokenizers-cpp#40 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#39 ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
mlc-ai/tokenizers-cpp#35 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
mlc-ai/tokenizers-cpp#32 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
mlc-ai/tokenizers-cpp#31 · 3 comments ·