ModelEngine-Group / ModelEngine-Group/unified-cache-management
[Feature]:
Open
Nobody has claimed this yet.
feature request
- Dominant language
- C++
- Stars
- 334
- Forks
- 119
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 82
Description
🚀 The feature, motivation and pitch
是否有对某个/某些请求的单独清理kv cache的运维措施，能否增加。 情况如下： 目前发现可能有这个问题 ,某些kv cache是异常数据，需要单独刷新这些kv cache ，否则基本上每次都异常了，但又想完全重新刷所有kv cache , 并且只知道一些请求的 message body 。
Alternatives
No response
Additional context
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named in the issue, so begin by locating the KV-cache management and request lookup paths in the repository. Done should provide an operational way to clear cache entries for selected requests using their message bodies, without clearing all KV cache.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100