huggingface / huggingface/swift-transformers
How to use customized tokenizer?
Open
tokenization
- Dominant language
- Swift
- Stars
- 1.4k
- Forks
- 209
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 4
Description
Hello. I am writing this post because I have a question about loading the tokenizer model. I am trying to use a pre-trained tokenizer in a Swift environment. After training, how do I apply the byproduct .model and .vocab files so that I can use the tokenizer I trained in Swift while using the swift-transformer API? I would appreciate it if you could answer.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.