microsoft / microsoft/onnxruntime-genai
CodeQwen running error
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
Hello guys, much appreciate your work here.
I have some problems with running a CodeQwen model. There is no errors in converting script and the onnx models appears in the output folder. However when I am trying to launch the `model-generate.py` it throwns an Error without any really helpfull information in a traceback.
I've tried both release version and build from source, the outcome is the same.
Will appreciate any suggestions on how to debug this error properly.
**To Reproduce**
Steps to reproduce the behavior:
1. Download official CodeQwen 1.5 weights from [HF](https://huggingface.co/Qwen/CodeQwen1.5-7B)
2. `python3 onnxruntime-genai/src/python/py/models/builder.py -i models/CodeQwen1.5-7B -o models/CodeQwen1.5-7B-onnx -p fp32 -e cpu` (using the `python3 -m onnxruntime_genai.models.builder` makes no difference)
3. Run the test `python3 onnxruntime-genai/examples/python/model-generate.py -m models/CodeQwen1.5-7B-onnx -pr "# Binary search on Python" -l 64` throws an IndexError with this traceback:
```
Prompt #0: # Binary search on Python
Traceback (most recent call last):
File "/home/***/onnxruntime-genai/examples/python/model-generate.py", line 75, in
main(args)
File "/home/***/onnxruntime-genai/examples/python/model-generate.py", line 51, in main
print(tokenizer.decode(output_tokens[i]))
IndexError: map::at
```
**Desktop:**
- OS: Ubuntu 22.04
**Additional context**
Please note that using the very same commands in the same environment I was able to convert the `codegemma-1.1-2b` to the onnx format and run the test. The response was accurate (correct binary search implementation) and without any errors at all. So I am expecting there is nothing wrong with my environment.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the commands in onnxruntime-genai/src/python/py/models/builder.py and examples/python/model-generate.py on Ubuntu 22.04, then inspect the traceback at tokenizer.decode(output_tokens[i]). Compare the CodeQwen and codegemma conversion and generation results; done means the CodeQwen example provides useful failure information or completes generation without the IndexError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100