NVIDIA / NVIDIA/TensorRT-LLM

[Feature Request]: Migrate Scaffolding MCP clients and examples to Streamable HTTP

Open
#19,329 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Scaffolding
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

Motivation

Scaffolding still uses the legacy MCP HTTP+SSE transport in its contrib client, core workers, and example servers. This prevents these clients from directly interoperating with servers that expose only the standard Streamable HTTP endpoint.

The MCP transport specification defines Streamable HTTP as the replacement for the 2024-11-05 HTTP+SSE transport. This does not mean SSE framing itself is removed: Streamable HTTP can still use SSE responses/streams.

Existing implementation

Inspected upstream source at b941c5c15:

  • tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py: client/session setup uses mcp.client.sse.sse_client.
  • tensorrt_llm/scaffolding/worker.py: core MCPWorker and ApiaryMCPWorker directly manage legacy SSE connections.
  • MCP example servers, including Coder/Apiary, use legacy SSE routes/transports; example/default URLs use /sse.

Requested change

  • Use the official Python SDK Streamable HTTP client/server APIs.
  • Centralize core MCP session initialization and cleanup in a shared helper used by both worker types.
  • Migrate the contrib client and MCP example servers/default URLs to /mcp.
  • Preserve tool discovery/calls, worker queueing, per-scope connection lifetimes, Coder authentication, explicit client_id sandbox identity, image selection, and sandbox attach/detach behavior.
  • Document the transport migration and SDK requirements. No automatic legacy-server fallback is proposed in this change.

Validation

Cover initialization, tool discovery, tool invocation, errors, and session teardown with local client/server smoke tests and worker tests. Verify Coder authentication/client identity and sandbox lifecycle across GET/POST/DELETE requests. GPU/inference integration coverage should run in project CI.

Implementation: #19055. Per the maintainer coordination reported by the contributor, that PR will also incorporate the separate KV-cache fixes tracked in #19281 and #19319 so the combined Scaffolding changes can share a CI run. These remain separate issues; they are not the same root cause.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Implementation is tracked in #19055. Start with tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py and tensorrt_llm/scaffolding/worker.py, then inspect the MCP example servers and existing worker tests. Done means Streamable HTTP uses /mcp while tool calls, authentication, client identity, sandbox lifecycle, teardown, and requested smoke-test coverage remain working.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.