openvinotoolkit

openvinotoolkit/model_server

A scalable inference server for models optimized with OpenVINO™

Stars: 931
Forks: 277
Open beginner issues: 0
Indexed issues: 55
Avg merge: 2d 13h
Merged PRs (30d): 68

Dominant language: C++
License: Apache-2.0
Last GitHub push: Sep 18, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted

Beginner friendly Everything

Open All

55 open issues indexed

OVMS tools call for Gemma 4 E4B failed Open

Difficulty 3/5 1-2 days Newbie friendliness 55/100

openvinotoolkit/model_server#4586 · Opened Sep 18, 2026
OVMS requiring manual WEIGHTS_PATH Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

openvinotoolkit/model_server#4580 · 1 comment · Opened Sep 18, 2026
stou32() accepts partial parses, unlike the other stringutils converters Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

openvinotoolkit/model_server#4554 · Opened Sep 12, 2026
Transcription temperature: malformed values are silently reinterpreted instead of rejected Open

openvinotoolkit/model_server#4552 · 1 assignee · Opened Sep 11, 2026
Download progress bar divides by zero without Content-Length and prints ~2.1 billion spaces per tick Open

openvinotoolkit/model_server#4550 · 1 assignee · Opened Sep 11, 2026
--source_loras misreads a URL port as the :alpha suffix and refuses to start Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

openvinotoolkit/model_server#4545 · Opened Sep 11, 2026
readMp3() never down-mixes stereo - returns interleaved samples where the API contract says mono Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

openvinotoolkit/model_server#4543 · Opened Sep 11, 2026
Add dimensions parameter support for embeddings endpoint (Matryoshka) Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 68/100

openvinotoolkit/model_server#4542 · Opened Sep 11, 2026
[Regression] 2026.3.1-gpu hangs with 200% CPU in sched_yield / libopencl-clang on dynamic shape embeddings (working fine in 2026.3-gpu) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#4510 · Opened Sep 4, 2026
Continuous batching: a GPU lane can reach a state where the servable stays up and every request hangs indefinitely until restart Open

Difficulty 5/5 Over a week Newbie friendliness 32/100

openvinotoolkit/model_server#4503 · Opened Sep 3, 2026
OpenAI-compatible Chat Completions: top-level reasoning_effort ignored, only chat_template_kwargs controls reasoning Open

bug

openvinotoolkit/model_server#4495 · 8 comments · 1 assignee · Opened Sep 1, 2026
Publish full YYYY.MINOR.PATCH Docker tags for .0 OVMS releases Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 65/100

openvinotoolkit/model_server#4488 · Opened Aug 29, 2026
Permanent LLMExecutor wedge after CL_OUT_OF_RESOURCES on discrete GPU: servable stays AVAILABLE but every request returns 400 until restart (VLM legacy pipeline, public model) Open

bug

openvinotoolkit/model_server#4469 · 8 comments · 1 assignee · Opened Aug 24, 2026
OVMS 2026.3 exits 139 (SIGSEGV) with no error message when loading an IR that requires a newer OpenVINO — reproduces on CPU Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

openvinotoolkit/model_server#4461 · 1 comment · 1 reaction · Opened Aug 18, 2026
can't run qwen3.6 27b in pipeline parallel Open

bug

openvinotoolkit/model_server#4452 · 7 comments · 1 assignee · Opened Aug 14, 2026
MTP and Dflash Open

openvinotoolkit/model_server#4447 · 1 comment · 1 assignee · Opened Aug 13, 2026
SIGSEGV in continuous batching when a streaming client disconnects mid-generation (block_manager.hpp:633 assertion, GPU, 2026.2.1) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

openvinotoolkit/model_server#4428 · 17 comments · Opened Aug 4, 2026
Enable_prefix_caching: Suspected cache inaccuracy during prefix reuse in Linear Attention Models Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

openvinotoolkit/model_server#4376 · Opened Jul 15, 2026
Baremetal ovms_ubuntu24 binary fails on newer distros — libxml2.so.2 (+ ICU 74) is a runtime dep but not bundled Open

openvinotoolkit/model_server#4362 · 3 comments · 1 assignee · Opened Jul 9, 2026
Windows Manager: Updates page and staged package upgrade workflow Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#4351 · Opened Jul 1, 2026
[RFC] Standalone Windows installer + tray Manager for OVMS (install / repair / uninstall / update) Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

openvinotoolkit/model_server#4347 · 1 reaction · Opened Jun 30, 2026
Qwen3.5-35B-A3B-int4-ov gibberish output during high memory pressure Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

openvinotoolkit/model_server#4345 · 4 comments · Opened Jun 30, 2026
Windows build fails on Visual Studio 2026 / newer MSVC toolsets (hardcoded VS2022 + toolset) Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

openvinotoolkit/model_server#4333 · Opened Jun 27, 2026
/chat/completions fails ("Chat template not loaded correctly") for VL-derived / omni text decoders — Python-Jinja path leaves template null; MINJA mode would fix it but is not in the 2026.2.x release Open

openvinotoolkit/model_server#4322 · 1 comment · 1 assignee · Opened Jun 24, 2026
CPU_RUNTIME_CACHE_CAPACITY is unsupported in OVMS plugin_config but works in raw OpenVINO Runtime Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

openvinotoolkit/model_server#4299 · 2 comments · Opened Jun 19, 2026
Gemma-4-26b-a4b-it-int4-ov gibberish output Open

bug

openvinotoolkit/model_server#4298 · 7 comments · 1 assignee · Opened Jun 19, 2026
Feature request: CPU offload / HETERO:GPU,CPU support for models exceeding VRAM in continuous batching Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#4263 · 4 comments · Opened May 29, 2026
generation_config.json path override per LLM node Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#4233 · 5 comments · Opened May 22, 2026
OVMS 2026.1 JSON request parsed successfully but pipeline input text sent to LLM says otherwise Open

bug

openvinotoolkit/model_server#4181 · 5 comments · 1 assignee · Opened May 5, 2026
OVMS 2026.1-gpu fails to load OpenVINO/gemma-4-E4B-it-int4-ov with Unsupported 'gemma4' VLM model type Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

openvinotoolkit/model_server#4178 · 5 comments · Opened May 5, 2026
Unload model from memory/GPU when idle for set period Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#4141 · 3 comments · 6 reactions · Opened Apr 15, 2026
gguf_tensor_to_f16 failed when loading Qwen3.5-9B GGUF model Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

openvinotoolkit/model_server#4046 · 4 comments · Opened Mar 8, 2026
OVMS GPU Memory Allocation Issue - Crashes on model load Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

openvinotoolkit/model_server#4035 · 6 comments · Opened Mar 4, 2026
OVMS refuses to load models into second or third GPUs. Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

openvinotoolkit/model_server#3816 · 3 comments · 6 reactions · Opened Nov 25, 2025
Failure to load HETERO models across multi GPU setup. Open

bug

Difficulty 5/5 Over a week Newbie friendliness 25/100

openvinotoolkit/model_server#3812 · 1 comment · Opened Nov 24, 2025
OVMS load error with HETERO and Qwen3-32B Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

openvinotoolkit/model_server#3696 · 13 comments · 1 reaction · Opened Oct 12, 2025
OVMS model loading time is much longer than openvino model loading time and crashes from time to time Open

bug

openvinotoolkit/model_server#3539 · 5 comments · 1 assignee · Opened Jul 29, 2025
Using openvino-quantized embedder and reranker from huggingface hub Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

openvinotoolkit/model_server#3493 · 3 comments · Opened Jul 10, 2025
I hope the official will add sample code for grpc's java, Android, and web clients. The server deployment is very easy, but the grpc client is not easy. Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

openvinotoolkit/model_server#3204 · 1 comment · Opened Apr 7, 2025
Significant Inference Time Increase with Multiple Models in OpenVINO Model Server Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

openvinotoolkit/model_server#3136 · 6 comments · Opened Mar 20, 2025
build from source Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

openvinotoolkit/model_server#3112 · 2 comments · Opened Mar 10, 2025
Error during llm node initialization for models_path Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

openvinotoolkit/model_server#2912 · 2 comments · Opened Dec 13, 2024
Got an error from OVMS: rpc error: code = ResourceExhausted desc = Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

openvinotoolkit/model_server#2871 · 3 comments · Opened Nov 22, 2024
GLiNER Inference Issue Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

openvinotoolkit/model_server#2858 · 2 comments · Opened Nov 19, 2024
Model server inference using Python Mediapipe Open

help wanted

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#2768 · 4 comments · Opened Oct 26, 2024
high latency Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

openvinotoolkit/model_server#2721 · 10 comments · Opened Oct 7, 2024
OVMS Docker image for ARM? Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

openvinotoolkit/model_server#2648 · 8 comments · Opened Aug 28, 2024
Queue policy Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

openvinotoolkit/model_server#2543 · 1 comment · Opened Jul 5, 2024
Model Server hangs when inference on Intel GPU Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

openvinotoolkit/model_server#2336 · 3 comments · Opened Feb 23, 2024
vehicle-detection-0202 responds to queries when running on CPU, gives an error when running on GPU Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

openvinotoolkit/model_server#2337 · 3 comments · Opened Feb 19, 2024

A short digest of beginner-friendly GitHub issues.