LMCache / LMCache/lmcache-tests

Why does blend_kv.py show limited optimization compared to the original CacheBlend implementation?

Open
#32 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10
Forks
8
PR merge metrics
No merged PRs in 30d

Description

### Description:

I ran two experiments to evaluate the optimization effectiveness:

Using examples/blend_kv/blend_kv.py.
Running the full prefill directly with vLLM.
In both cases, I found the optimization results were not significant. However, when using the original CacheBlend repository code, the results were excellent under the same dataset and setup.

What could be the reasons for this discrepancy? Are there any additional configurations or dependencies in blend_kv.py that I might have missed?

Looking forward to your insights!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with examples/blend_kv/blend_kv.py and compare its configuration and dependencies with the original CacheBlend repository code. Reproduce the reported comparison against full prefill in vLLM using the same dataset and setup, then identify and document the configuration or dependency difference that explains the optimization gap.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.