openvinotoolkit / openvinotoolkit/model_server

OVMS load error with HETERO and Qwen3-32B

Open
#3,696 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
931
Forks
277
Avg merge
2d 13h
Merged PRs (30d)
68

Description

Describe the bug
When attempting to load Qwen3-32B with a hetero config, the model_server fails to do so.

To Reproduce
Steps to reproduce the behavior:

  1. Launch ovms with models in directory converted from export_model.py
  2. fail.

Expected behavior
OVMS should load the model into VRAM.

Logs
[2025-10-12 04:26:45.371][367240][modelmanager][debug][modelmanager.cpp:490] Mediapipe graph:Qwen/Qwen3-32B was not loaded so far. Triggering load
[2025-10-12 04:26:45.371][367240][modelmanager][debug][mediapipegraphdefinition.cpp:129] Started validation of mediapipe: Qwen/Qwen3-32B
[2025-10-12 04:26:45.371][367240][modelmanager][debug][mediapipe_utils.cpp:84] setting input stream: input packet type: UNKNOWN from: HTTP_REQUEST_PAYLOAD:input
[2025-10-12 04:26:45.371][367240][modelmanager][debug][mediapipe_utils.cpp:84] setting output stream: output packet type: UNKNOWN from: HTTP_RESPONSE_PAYLOAD:output
[2025-10-12 04:26:45.371][367240][serving][info][mediapipegraphdefinition.cpp:421] MediapipeGraphDefinition initializing graph nodes
[2025-10-12 04:26:45.371][367240][modelmanager][info][servable_initializer.cpp:424] Initializing Language Model Continuous Batching servable
[2025-10-12 04:27:41.099][367240][serving][error][servable_initializer.cpp:145] Error during llm node initialization for models_path: /media/models/Qwen/Qwen3-32B/./ exception: Exception from src/inference/src/cpp/core.cpp:109:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/hetero/src/compiled_model.cpp:36:
Standard exception from compilation library: Exception from src/inference/src/dev/plugin.cpp:53:
Check 'false' failed at src/plugins/intel_gpu/src/plugin/program_builder.cpp:163:
[GPU] ProgramBuilder build failed!
Exception from src/plugins/intel_gpu/src/runtime/ocl/ocl_common.hpp:40:
[GPU] clEnqueueNDRangeKernel, error code: -52 CL_INVALID_KERNEL_ARGS

Configuration

  1. OVMS version - built from latest.
  2. 11600KF, 3 x A770's with latest drivers.

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Qwen3-32B load with the hetero configuration and models produced by export_model.py. Use the logged path through servable_initializer.cpp, compiled_model.cpp, and program_builder.cpp to investigate the CL_INVALID_KERNEL_ARGS failure. Done means OVMS loads the model successfully into VRAM on the reported GPU setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.