vllm-project / vllm-project/agentic-api

[Claude Code - Stage 4] Normalize Messages session identity for cache-aware routing

Open
#159 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Part of #113 (Claude Code integration). Related to #18 and vllm-project/vllm#48049.

Problem

The Messages path currently forwards Claude Code's x-claude-code-session-id, but agentic-api does not normalize or act on it. Anthropic Messages remains stateless with respect to transcript persistence, but clients can still provide a stable session coordinate for routing, retention, and cache lifecycle policy.

Without one internal contract, agentic-api, llm-d, and vLLM must each understand client-specific headers or lose the session signal.

Proposed contract

  • Claude Code: x-claude-code-session-idsession_id
  • Generic Messages clients: X-Session-IDsession_id
  • Neither header present: leave session_id unset
  • Both headers present with different values: return a clear 4xx error
  • Scope the normalized value to the authenticated tenant/workspace
  • Preserve the original client headers for protocol compatibility and observability
  • Do not derive session identity from cache_control, message content, prompt hashes, or request IDs

Ownership

  • agentic-api normalizes protocol-specific identity.
  • llm-d can use the normalized coordinate for soft affinity, routing, retention, and lifecycle policy.
  • vLLM can receive it as typed request metadata once the upstream contract lands.
  • Exact KV reuse is still determined by rendered token IDs, canonical block keys, and event-confirmed residency. session_id is not KV block identity and does not prove residency.

Messages remains stateless in the transcript sense: this issue does not add message persistence, rehydration, or a Responses-style stored-response object.

Acceptance criteria

  • /v1/messages recognizes x-claude-code-session-id.
  • /v1/messages recognizes provider-neutral X-Session-ID.
  • Matching values normalize to one internal session coordinate.
  • Conflicting values return a deterministic 4xx error.
  • Missing values leave the internal session coordinate unset.
  • The normalized value is tenant/workspace scoped.
  • Original Messages headers and body remain protocol-faithful.
  • The canonical session coordinate can be forwarded to llm-d and bridged to vLLM's typed session_id.
  • Tests cover Claude Code, generic Messages, missing, matching, conflicting, and cross-tenant cases.

Out of scope

  • Persisting Anthropic Messages transcripts
  • Hard replica pinning
  • Treating session identity as exact KV/cache-block identity
  • Implementing llm-d or vLLM lifecycle policy in this repository

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the /v1/messages entry point and trace how request metadata can reach the llm-d and vLLM bridge. Add tests covering Claude Code, generic, missing, matching, conflicting, and cross-tenant headers; done means the accepted values normalize consistently while original headers and the stateless Messages body remain intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.