cncf / cncf/mentoring

[CNCF LFX Proposal] Volcano/Kthena Build a KVCache-aware scheduler E2E test suite

Open
#2,008 22 comments 1 reaction 0 assignees View on GitHub
2026 CNCF Approved Exported lfx mentorship Maintainer/Contribex Approved Mentors Confirmed Proposal Term 3: Sept-Nov Validation Passed
Dominant language
JavaScript
Stars
3.1k
Forks
816
Avg merge
12h 32m
Merged PRs (30d)
8

Description

### CNCF Project

Volcano

### Term

2026 Term 3 (Sep-Nov)

### Program Name

Build a KVCache-aware scheduler E2E test suite

### Program Description

## Description:
Kthena router already has a scheduler plugin E2E framework covering prefix-cache, least-request, least-latency, lora-affinity, random, and gpu-usage. The KVCache-aware scheduler plugin also exists, but lacks robust end-to-end coverage for the full runtime path involving the router, mock backends, tokenization, ZMQ events, and Redis.

This project aims to build a reusable KVCache-aware scheduler E2E test suite on top of the existing router plugin framework. In addition to validating routing behavior, the suite should provide reusable setup, cache-state injection, observation, and assertion utilities for future KVCache-aware test scenarios.

## Expected Outcome:

1. Build a reusable KVCache-aware E2E test suite under `test/e2e/router/`, including shared fixtures and helpers for llm-d-sim, the ZMQ bridge, Redis, cache-state injection, request generation, and routing assertions.
2. Validate the complete data path: dispatch KV-cache ownership events through ZMQ, verify the resulting Redis mappings, and confirm that the router uses them when scoring replicas.
3. Provide a focused command or Make target for running the suite independently, and document how contributors can add future KVCache-aware scenarios.
4. Integrate the suite into CI with deterministic cleanup and useful failure diagnostics for router, Redis, ZMQ, and backend state.

### Technologies

Go, Kubernetes, Kind-based E2E testing, Redis, ZMQ, observability/metrics, and distributed-system debugging.

### Skills same as Technologies?

- [x] Yes, the required skills are the same as the technologies listed above.

### Required/Desirable Skills

_No response_

### Mentors

Jinyu Zhou | @FAUST-BENCHOU | 2319109590@qq.com | benchou8
Jprakash | @katara-Jayprakash | katarajayprakash@icloud.com | lfxkatara1

### Upstream Issue URL

https://github.com/volcano-sh/kthena/issues/1328

### Application Prerequisites

- [x] Resume
- [x] Cover Letter
- [ ] School Enrollment Verification
- [ ] Participation Permission from school or employer
- [ ] Coding Challenge
- [ ] Custom Prerequisite (fill in details below)

### Coding Challenge URL

_No response_

### Custom Prerequisite Name

_No response_

### Custom Prerequisite Description

_No response_

### Custom Prerequisite — File Upload

- [ ] Yes — completion of this task requires the mentee to submit a file.

---
**LFX program:** [CNCF - Volcano: Build a KVCache-aware scheduler E2E test suite (2026 Term 3)](https://mentorship.lfx.linuxfoundation.org/project/16e5821d-2ae3-4145-a729-1b0c755b613b)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing scheduler plugin E2E framework under test/e2e/router/ and the upstream issue linked in the proposal. The work is complete when reusable fixtures and helpers cover llm-d-sim, ZMQ, Redis, cache-state injection, requests, and routing assertions; the suite has an independent command, CI integration, cleanup, diagnostics, and contributor documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, redis
Domain
devops, distributed-systems, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.