openvinotoolkit / openvinotoolkit/model_server
CPU_RUNTIME_CACHE_CAPACITY is unsupported in OVMS plugin_config but works in raw OpenVINO Runtime
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 931
- Forks
- 277
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 68
Description
The config key CPU_RUNTIME_CACHE_CAPACITY is valid and functional when used directly with OpenVINO Runtime:
config = {
"CPU_RUNTIME_CACHE_CAPACITY": "0",
"PERFORMANCE_HINT": "THROUGHPUT"
}
However, if I put this key inside the plugin_config block of OVMS model config JSON, the server fails to start and throws an error log in terminal:
CPU_RUNTIME_CACHE_CAPACITY not found in supported config keys for device: CPU.
OVMS config snippet I used:
"config": {
"name": "vocos",
"base_path": "./models/vocos",
"target_device": "CPU",
"plugin_config": {
"CPU_RUNTIME_CACHE_CAPACITY": "0",
"PERFORMANCE_HINT": "THROUGHPUT"
}
}
Is there an equivalent supported configuration in OVMS to disable the CPU runtime cache (set CPU_RUNTIME_CACHE_CAPACITY to 0)?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No implementation file or test is named. Start by tracing OVMS model configuration handling for the plugin_config block and the supported-config-key validation for the CPU device, then compare it with the direct OpenVINO Runtime behavior described here. Done means establishing a supported way to set CPU_RUNTIME_CACHE_CAPACITY to 0 in OVMS and covering the result with an appropriate test or documented limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100