stacklok / stacklok/toolhive

Add proper SSE support for vMCP (beyond streamable-http session affinity)

Open
#4,974 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement scalability vmcp
Dominant language
Go
Stars
2.2k
Forks
300
Avg merge
1d 15h
Merged PRs (30d)
184

Description

Background

The current vMCP implementation supports streamable-http transport and handles session affinity by storing pod assignment in Redis. This works well for streamable-http because any request can land on any vMCP pod and be routed correctly.

However, SSE (Server-Sent Events) has different requirements:

  • SSE is a long-lived connection — the client connects once and receives a stream of events identified by event IDs.
  • It is fundamentally session-oriented: the client expects to reconnect to the same backend and resume from a known event ID.
  • Features like elicitation (and other push-based MCP patterns) rely on SSE even when the primary transport is streamable-http.

Problem

The current Redis-based session affinity approach solves pod restarts and horizontal scaling for streamable-http, but does not fully support SSE:

  • Session affinity by client IP (at the load balancer level) does not survive pod restarts or rescheduling.
  • The Redis routing layer can preserve routing after pod restarts, but SSE connections are terminated when the pod dies — there is no reconnect/resume mechanism.

What needs to be done

  • Investigate the SSE reconnection/resume requirements in the MCP spec (event IDs, Last-Event-ID header).
  • Determine what changes are needed in the vMCP proxy layer to support SSE reliably in a multi-pod environment.
  • Consider whether SSE needs a separate persistence/replay mechanism or whether graceful drain on pod shutdown is sufficient.
  • Ensure elicitation and any other SSE-dependent MCP features work correctly end-to-end.

References

  • Current session affinity implementation uses Redis with GETEX + TTL.
  • Streamable-http routing is tested with sessionAffinity: none.
  • SSE path has not been fully tested/validated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the current Redis GETEX + TTL session-affinity implementation and the streamable-http routing test using sessionAffinity: none; the payload does not identify files or entry points. Compare the proxy behavior with MCP SSE event IDs and the Last-Event-ID header, then validate elicitation and reconnection across pod restarts, with reliable replay or graceful draining as the documented outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, redis
Domain
api, backend, distributed-systems, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.