agentic-community / agentic-community/mcp-gateway-registry

MCP proxy hop breaks session affinity with multiple backend replicas

Open
#1,101 0 comments 0 reactions 1 assignee Claimed by @aarora79 View on GitHub
nginx performance
Dominant language
Python
Stars
912
Forks
234
Avg merge
1d 11h
Merged PRs (30d)
62

Description

## Summary

The mcp-proxy architecture (PR #1026) breaks MCP streamable-http session affinity when a backend MCP server has multiple replicas behind Service Connect or a load balancer.

## Root cause

Auth server's `mcp_proxy()` creates a new `httpx.AsyncClient` for each request. When the upstream service (e.g. mcpgw) has multiple tasks, Service Connect round-robins between them. The MCP `initialize` request creates a session on task A, but subsequent requests (`tools/list`, `tools/call`) may land on task B which returns "Session not found".

## Impact

Any MCP server registered with multiple replicas will intermittently fail with "Session not found" errors. Currently worked around by scaling mcpgw to 1 replica.

## Proposed solutions (pick one)

1. **Lua-based selective routing (preferred):** Only route `tools/list` through the mcp-proxy hop. All other methods go directly nginx -> backend (restoring session affinity for 99% of traffic). The lua body capture script already parses the JSON body.

2. **Session-aware connection pooling in auth server:** Maintain a persistent httpx connection per `Mcp-Session-Id`, reusing the same TCP connection for all requests in a session. Complex, adds state to auth server.

3. **Sticky sessions via Service Connect:** Configure Service Connect with session affinity based on a header (e.g. `Mcp-Session-Id`). Requires AWS support for header-based routing in Service Connect (not currently available).

4. **Disable mcp-proxy for non-tools/list:** Auth server checks the method in the request body and returns a redirect or passthrough for non-`tools/list` methods. Simpler than lua but still adds latency.

## Current workaround

Scale mcpgw to 1 replica (`aws ecs update-service --desired-count 1`).

## Files affected

- `auth_server/server.py` (`mcp_proxy()` function)
- `docker/lua/capture_body.lua` (for option 1)
- `registry/core/nginx_service.py` (for option 1, conditional proxy_pass)

Refs: #1026, #1023

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.