michaelfeil / michaelfeil/infinity

Getting Errors during ONNX translation

Open
#652 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### System Info

Hey there!
Tried installing a 1 year old project and the container won't start anymore. Getting it on multiple models.
Can somebody explain me why/whats going on?

Running on
- Linux
- x86
- Intel(R) Core(TM) Ultra 7 255U

Infinity is the latest cpu image.

```
The ONNX file model_quantized_optimized.onnx is not a regular name used in optimum.onnxruntime, the ORTModel might not behave as expected.

2025-12-23 22:51:12.139294021 [E:onnxruntime:, inference_session.cc:2280 operator()] Exception during initialization: /onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:79 std::shared_ptr onnxruntime::openvino_ep::OVCore::ReadModel(std::string&&, const std::string&) [OpenVINO-EP] [OpenVINO-EP] Exception while Reading network: Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:164:

FrontEnd API failed with GeneralFailure:

Errors during ONNX translation:

Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:

While validating ONNX node '':

Unsupported input bias type, accepted FP32 but got: dynamic

Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:

While validating ONNX node '':

Unsupported input bias type, accepted FP32 but got: dynamic

Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:

While validating ONNX node '':

Unsupported input bias type, accepted FP32 but got: dynamic

```

### Information

- [x] Docker + cli
- [ ] pip + cli
- [ ] pip + usage of Python interface

### Tasks

- [x] An officially supported CLI command
- [ ] My own modifications

### Reproduction

I get it when trying to start a container with following:

```
infinity:
image: michaelf34/infinity:latest-cpu
command: >
v2
--engine optimum
--model-id michaelfeil/bge-small-en-v1.5
--port ${PORT:-7997}
ports:
- "${PORT:-7997}:${PORT:-7997}"
volumes:
- "sail-infinity:/app/.cache"
networks:
- app-network
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the documented Docker Compose command and inspect the reported ONNX Runtime/OpenVINO translation errors for the bge-small-en-v1.5 model. Done means identifying the model or runtime compatibility cause and documenting a verified way to start the container.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.