LMCache / LMCache/lmcache-tests
Why does blend_kv.py show limited optimization compared to the original CacheBlend implementation?
- Dominant language
- Python
- Stars
- 10
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
### Description:
I ran two experiments to evaluate the optimization effectiveness:
Using examples/blend_kv/blend_kv.py.
Running the full prefill directly with vLLM.
In both cases, I found the optimization results were not significant. However, when using the original CacheBlend repository code, the results were excellent under the same dataset and setup.
What could be the reasons for this discrepancy? Are there any additional configurations or dependencies in blend_kv.py that I might have missed?
Looking forward to your insights!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with examples/blend_kv/blend_kv.py and compare its configuration and dependencies with the original CacheBlend repository code. Reproduce the reported comparison against full prefill in vLLM using the same dataset and setup, then identify and document the configuration or dependency difference that explains the optimization gap.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100