kvcache-ai / kvcache-ai/Mooncake
[Bug]: 使用内置mooncake client方式（ModeA）卸载大量请求的KV到DRAM和SSD后，VLLM PD实例重启，重启后重新压测之前的请求，无法在DRAM和SSD中命中，导致Prefix Cache命中率降低ttft增加
Open
bug
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
### Bug Report
[Bug]: 使用内置mooncake client方式（ModeA）卸载大量请求的KV到DRAM和SSD后，VLLM PD实例重启，重启后重新压测之前的请求，无法在DRAM和SSD中命中，导致Prefix Cache命中率降低ttft增加
### Before submitting...
- [ ] Ensure you searched for relevant issues and read the [documentation]
Contributor guide
Research direction
Reproduce the ModeA workflow by offloading a large request set to DRAM and SSD, restarting the vLLM PD instance, and repeating the same load test. Trace the restart and cache lookup entry points to determine why previous KV data is not hit; done means the post-restart requests recover the expected Prefix Cache hit rate and TTFT, with a regression test covering the scenario.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100