openvinotoolkit / openvinotoolkit/model_server

CPU_RUNTIME_CACHE_CAPACITY is unsupported in OVMS plugin_config but works in raw OpenVINO Runtime

Open
#4,299 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
931
Forks
277
Avg merge
2d 13h
Merged PRs (30d)
68

Description

The config key CPU_RUNTIME_CACHE_CAPACITY is valid and functional when used directly with OpenVINO Runtime:

config = {
    "CPU_RUNTIME_CACHE_CAPACITY": "0",
    "PERFORMANCE_HINT": "THROUGHPUT"
}

However, if I put this key inside the plugin_config block of OVMS model config JSON, the server fails to start and throws an error log in terminal:
CPU_RUNTIME_CACHE_CAPACITY not found in supported config keys for device: CPU.

OVMS config snippet I used:

"config": {
  "name": "vocos",
  "base_path": "./models/vocos",
  "target_device": "CPU",
  "plugin_config": {
    "CPU_RUNTIME_CACHE_CAPACITY": "0",
    "PERFORMANCE_HINT": "THROUGHPUT"
  }
}

Is there an equivalent supported configuration in OVMS to disable the CPU runtime cache (set CPU_RUNTIME_CACHE_CAPACITY to 0)?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No implementation file or test is named. Start by tracing OVMS model configuration handling for the plugin_config block and the supported-config-key validation for the CPU device, then compare it with the direct OpenVINO Runtime behavior described here. Done means establishing a supported way to set CPU_RUNTIME_CACHE_CAPACITY to 0 in OVMS and covering the result with an appropriate test or documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.