vllm-project / vllm-project/aibrix

how to set affinity for KVCache and could the cache pods use the same rdma nic with engine pod?

Open
#1,492 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

hi,
I am trying the KVCache offloading L2 Cache example here https://aibrix.readthedocs.io/latest/features/kvcache-offloading.html#l2-cache-example,and I have two questions about this crd

1、does the KVCache support nodeAffinity and how to set affinity,we try cache pod is ok,but others does not work

2、can the cache pods use rdma nics that have been allocated by vllm engine pod?if not,I think we need to serve some cpu servers with rdma nics for cache pods specially?only use rdma nic in gpu server will waste gpu, that is right?

Contributor guide

Open the contributing guide

Research direction

Start with the linked KVCache offloading L2 Cache example and inspect the CRD configuration used by the cache and engine pods. Determine whether nodeAffinity is supported and whether RDMA NIC allocation can be shared; done means documenting the supported configuration or defining the required scheduling and network changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.