vllm-project / vllm-project/aibrix
how to set affinity for KVCache and could the cache pods use the same rdma nic with engine pod?
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
hi,
I am trying the KVCache offloading L2 Cache example here https://aibrix.readthedocs.io/latest/features/kvcache-offloading.html#l2-cache-example,and I have two questions about this crd
1、does the KVCache support nodeAffinity and how to set affinity,we try cache pod is ok,but others does not work
2、can the cache pods use rdma nics that have been allocated by vllm engine pod?if not,I think we need to serve some cpu servers with rdma nics for cache pods specially?only use rdma nic in gpu server will waste gpu, that is right?
Contributor guide
Research direction
Start with the linked KVCache offloading L2 Cache example and inspect the CRD configuration used by the cache and engine pods. Determine whether nodeAffinity is supported and whether RDMA NIC allocation can be shared; done means documenting the supported configuration or defining the required scheduling and network changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100