michaelfeil / michaelfeil/infinity
Getting Errors during ONNX translation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### System Info
Hey there!
Tried installing a 1 year old project and the container won't start anymore. Getting it on multiple models.
Can somebody explain me why/whats going on?
Running on
- Linux
- x86
- Intel(R) Core(TM) Ultra 7 255U
Infinity is the latest cpu image.
```
The ONNX file model_quantized_optimized.onnx is not a regular name used in optimum.onnxruntime, the ORTModel might not behave as expected.
2025-12-23 22:51:12.139294021 [E:onnxruntime:, inference_session.cc:2280 operator()] Exception during initialization: /onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:79 std::shared_ptr onnxruntime::openvino_ep::OVCore::ReadModel(std::string&&, const std::string&) [OpenVINO-EP] [OpenVINO-EP] Exception while Reading network: Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:164:
FrontEnd API failed with GeneralFailure:
Errors during ONNX translation:
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '':
Unsupported input bias type, accepted FP32 but got: dynamic
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '':
Unsupported input bias type, accepted FP32 but got: dynamic
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '':
Unsupported input bias type, accepted FP32 but got: dynamic
```
### Information
- [x] Docker + cli
- [ ] pip + cli
- [ ] pip + usage of Python interface
### Tasks
- [x] An officially supported CLI command
- [ ] My own modifications
### Reproduction
I get it when trying to start a container with following:
```
infinity:
image: michaelf34/infinity:latest-cpu
command: >
v2
--engine optimum
--model-id michaelfeil/bge-small-en-v1.5
--port ${PORT:-7997}
ports:
- "${PORT:-7997}:${PORT:-7997}"
volumes:
- "sail-infinity:/app/.cache"
networks:
- app-network
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the documented Docker Compose command and inspect the reported ONNX Runtime/OpenVINO translation errors for the bge-small-en-v1.5 model. Done means identifying the model or runtime compatibility cause and documenting a verified way to start the container.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100