BySense tokenization details
Open
documentation
- Dominant language
- C++
- Stars
- 9.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
Hello!
Could you please provide more details on how the `BySense` tokenization method is implemented? Are any rules or NLP models being used? The documentation has no information about it.
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.