microsoft / microsoft/onnxruntime-genai
Unsupported tokenizer class for microsoft/OptiMind-SFT
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
I bumped the same issue ``Unsupported tokenizer class: TokenizersBackend``.
I created the following model ``python -m onnxruntime_genai.models.builder -m microsoft/OptiMind-SFT -o microsoft/OptiMind -p int4 -e cuda -c cache`` and tries it with following code:
```python
import onnxruntime_genai as og
model = og.Model(".")
tokenizer = og.Tokenizer(model)
stream = tokenizer.create_stream()
# Set the max length to something sensible by default,
# since otherwise it will be set to the entire context length
search_options = {}
search_options['max_length'] = 2048
search_options['batch_size'] = 1
chat_template = '<|user|>\n{input} <|end|>\n<|assistant|>'
text = "dummy example for somebody willing to learn python"
prompt = f'{chat_template.format(input=text)}'
input_tokens = tokenizer.encode(prompt)
params = og.GeneratorParams(model)
params.set_search_options(**search_options)
generator = og.Generator(model, params)
print("Output: ", end='', flush=True)
generator.append_tokens(input_tokens)
while not generator.is_done():
generator.generate_next_token()
new_token = generator.get_next_tokens()[0]
print(stream.decode(new_token), end='', flush=True)
print()
del generator
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the `python -m onnxruntime_genai.models.builder` command for `microsoft/OptiMind-SFT`, then run the shown `og.Tokenizer` and generation snippet. Trace the tokenizer entry point where `TokenizersBackend` is rejected; the issue is done when the built model loads and the example can generate output without the unsupported-tokenizer error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100