agentic-community / agentic-community/mcp-gateway-registry
MCP proxy hop breaks session affinity with multiple backend replicas
- Dominant language
- Python
- Stars
- 912
- Forks
- 234
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 62
Description
## Summary
The mcp-proxy architecture (PR #1026) breaks MCP streamable-http session affinity when a backend MCP server has multiple replicas behind Service Connect or a load balancer.
## Root cause
Auth server's `mcp_proxy()` creates a new `httpx.AsyncClient` for each request. When the upstream service (e.g. mcpgw) has multiple tasks, Service Connect round-robins between them. The MCP `initialize` request creates a session on task A, but subsequent requests (`tools/list`, `tools/call`) may land on task B which returns "Session not found".
## Impact
Any MCP server registered with multiple replicas will intermittently fail with "Session not found" errors. Currently worked around by scaling mcpgw to 1 replica.
## Proposed solutions (pick one)
1. **Lua-based selective routing (preferred):** Only route `tools/list` through the mcp-proxy hop. All other methods go directly nginx -> backend (restoring session affinity for 99% of traffic). The lua body capture script already parses the JSON body.
2. **Session-aware connection pooling in auth server:** Maintain a persistent httpx connection per `Mcp-Session-Id`, reusing the same TCP connection for all requests in a session. Complex, adds state to auth server.
3. **Sticky sessions via Service Connect:** Configure Service Connect with session affinity based on a header (e.g. `Mcp-Session-Id`). Requires AWS support for header-based routing in Service Connect (not currently available).
4. **Disable mcp-proxy for non-tools/list:** Auth server checks the method in the request body and returns a redirect or passthrough for non-`tools/list` methods. Simpler than lua but still adds latency.
## Current workaround
Scale mcpgw to 1 replica (`aws ecs update-service --desired-count 1`).
## Files affected
- `auth_server/server.py` (`mcp_proxy()` function)
- `docker/lua/capture_body.lua` (for option 1)
- `registry/core/nginx_service.py` (for option 1, conditional proxy_pass)
Refs: #1026, #1023
Contributor guide
Assessment
This issue has not been assessed yet.