vllm-project / vllm-project/aibrix

Implement PrisKVConnector for vLLM-Omni KV Communication

Open
#2,010 0 comments 1 reaction 1 assignee View on GitHub

@bob-021206 is already working on this.

Since Mar 17, 2026.

Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🚀 Feature Description and Motivation

This issue proposes adding a PrisKV-based connector for vLLM-Omni so that vLLM-Omni can use the PrisKV client for KV communication across pipeline stages.

Currently, vLLM-Omni supports pluggable connectors for inter-stage KV transfer, but it does not yet provide a native connector for PrisKV. Since AIBrix uses PrisKV as a core distributed KV communication and storage component, this integration is required to enable efficient KV transfer within the AIBrix multi-modal serving stack.

This work includes:
- building and validating the PrisKV dependency in the vLLM-Omni environment,
- implementing a `PrisKVConnector` in vLLM-Omni,
- wiring the connector to use the PrisKV client APIs for KV put/get communication,
- ensuring the connector can serve as a functional backend for KV transfer in omni-modal pipelines.

This feature will allow vLLM-Omni to communicate KV states through PrisKV directly, reducing integration gaps between vLLM-Omni and AIBrix.

Part of #1966

### Use Case

We need vLLM-Omni to communicate KV data through PrisKV so that intermediate KV states can be written and retrieved through a native `PrisKVConnector` backed by the PrisKV client.

This would make it possible to run vLLM-Omni with AIBrix infrastructure and provide the foundation for later optimizations.

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.