[Feature Request]: Migrate Scaffolding MCP clients and examples to Streamable HTTP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
Motivation
Scaffolding still uses the legacy MCP HTTP+SSE transport in its contrib client, core workers, and example servers. This prevents these clients from directly interoperating with servers that expose only the standard Streamable HTTP endpoint.
The MCP transport specification defines Streamable HTTP as the replacement for the 2024-11-05 HTTP+SSE transport. This does not mean SSE framing itself is removed: Streamable HTTP can still use SSE responses/streams.
Existing implementation
Inspected upstream source at b941c5c15:
tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py: client/session setup usesmcp.client.sse.sse_client.tensorrt_llm/scaffolding/worker.py: coreMCPWorkerandApiaryMCPWorkerdirectly manage legacy SSE connections.- MCP example servers, including Coder/Apiary, use legacy SSE routes/transports; example/default URLs use
/sse.
Requested change
- Use the official Python SDK Streamable HTTP client/server APIs.
- Centralize core MCP session initialization and cleanup in a shared helper used by both worker types.
- Migrate the contrib client and MCP example servers/default URLs to
/mcp. - Preserve tool discovery/calls, worker queueing, per-scope connection lifetimes, Coder authentication, explicit
client_idsandbox identity, image selection, and sandbox attach/detach behavior. - Document the transport migration and SDK requirements. No automatic legacy-server fallback is proposed in this change.
Validation
Cover initialization, tool discovery, tool invocation, errors, and session teardown with local client/server smoke tests and worker tests. Verify Coder authentication/client identity and sandbox lifecycle across GET/POST/DELETE requests. GPU/inference integration coverage should run in project CI.
Implementation: #19055. Per the maintainer coordination reported by the contributor, that PR will also incorporate the separate KV-cache fixes tracked in #19281 and #19319 so the combined Scaffolding changes can share a CI run. These remain separate issues; they are not the same root cause.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Implementation is tracked in #19055. Start with tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py and tensorrt_llm/scaffolding/worker.py, then inspect the MCP example servers and existing worker tests. Done means Streamable HTTP uses /mcp while tool calls, authentication, client identity, sandbox lifecycle, teardown, and requested smoke-test coverage remain working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100