ByteDance-Seed / ByteDance-Seed/ShadowKV
[vLLM support?]
Open
- Dominant language
- Python
- Stars
- 313
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
Support for vLLM with ShadowKV
**Describe the solution you'd like**
Can ShadowKV eviction be integrated with vLLM serving to improve the decode throughput?
Contributor guide
Assessment
This issue has not been assessed yet.