huggingface / huggingface/optimum-intel
Phi-3.5-vision-instruct invalid EOS token
- Dominant language
- Jupyter Notebook
- Stars
- 620
- Forks
- 270
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 22
Description
Hello,
I have a problem with exporting the model microsoft/Phi-3.5-vision-instruct.
Model loads and generates correctly, but it doesn't stop after answering and generates random output unrelated to the prompt.
After some troubleshooting I found that EOS token id in generation_config.json is 2, while it should be 32000. After changing it manually it works correctly.
Export command:
```
optimum-cli export openvino --model microsoft/Phi-3.5-vision-instruct --weight-format int4 --sym --ratio 1.0 --group-size -1 --trust-remote-code models\microsoft\Phi-3.5-vision-instruct
```
I've used optimum from main with transformers==4.53.3 (https://github.com/huggingface/optimum-intel/issues/1573).
Python: 3.10.7
Kind regards
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.