feature: Optimize vLLM production-stack for agentic workflows (BeeAI, MCP) via KV-cache reuse and context-aware routing

Describe the feature

Problem Statement
Agentic workflows (e.g., multi-agent systems like IBM BeeAI or frameworks using protocols like Anthropic MCP) involve frequent agent-to-agent communication and shared context (task states, knowledge bases, etc.). Current LLM serving systems, including vLLM production-stack, focus on isolated requests and lack optimizations for cross-agent context reuse. This results in redundant KV-cache recomputation and higher latency for agentic workloads.

Proposed Optimizations
To better support agentic frameworks, vLLM production-stack could:

1. Context/KV-Cache Sharing Across Agents
   • Allow agents in the same workflow to reuse precomputed KV caches for shared context (e.g., task descriptions, shared knowledge).
   • Extend the existing KV-cache storage (via LMCache) to track agent-session relationships.
2. Agent-Aware Request Routing
   • Enhance the router to recognize agent workflows (e.g., via session_id or workflow metadata) and route requests to nodes with cached context.
   • Add support for MCP-like metadata to annotate requests with agent roles or workflow stages.
3. Observability for Agent Workflows
   • Expose metrics like cross-agent KV-cache hit rate, inter-agent token latency, and workflow-level TTFT/TBT.

Why This Matters for vLLM

• Performance Gains: Agentic workflows often reuse prompts/context across multiple steps (e.g., task decomposition → execution → validation). Reusing KV caches could reduce inference latency by 3–10x for these workloads.
• Ecosystem Growth: Native support for agent frameworks like BeeAI or MCP-based systems would position vLLM as the go-to engine for next-gen AI agent deployments.
• Synergy with Existing Features: This aligns with production-stack’s existing strengths (KV-cache sharing, prefix-aware routing) while expanding its use cases.

Integration Points

1. IBM BeeAI Framework:
   • BeeAI’s Model Context Protocol (MCP) Tool could be adapted to interface with vLLM’s KV-cache storage.
   • Workflow telemetry could feed into production-stack’s Grafana dashboards.
2. Anthropic MCP:
   • Map MCP’s structured communication (metadata/payload) to vLLM’s router for context-aware scheduling.

Next Steps

• Open a discussion with the BeeAI/LMCache teams to explore joint optimization opportunities.
• Add a workflow_id field to the vLLM production-stack API for tagging related requests.
• Benchmark agentic workloads with/without these optimizations (we’d be happy to collaborate on testing!).

Additional context

• BeeAI Framework (multi-agent workflows + MCP support)
• Anthropic MCP Overview (structured agent communication)

The issue does not name files, tests, or a specific entry point. Start by reviewing the existing LMCache storage and prefix-aware router described in the issue, then examine how workflow_id, agent metadata, and Grafana telemetry could fit together. Done would require an agreed scope, implementation, and benchmarks comparing agentic workloads with and without the proposed optimizations.