vllm-project / vllm-project/aibrix
where to get the customized vineyard and vLLM which used for distributed kv cache?
Open
kind/support
triage/accepted
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
as the distributed kv cache document said, aibrix use a customized version of vineyard and an internal version of vLLM, I want to know some detail of the distributed kv cache implementaion, such as:
1. What protocol is used for cross-node KV cache transmission?
2. What storage to use to store the kv cache in vineyard?
So I was wondering if I could find the customized code for vineyard and vLLM.
Contributor guide
Assessment
This issue has not been assessed yet.