stacklok / stacklok/toolhive

vMCP elicitation: fast-fail without a standalone stream, and cross-pod origination

Open
#5,975 2 comments 0 reactions 1 assignee View on GitHub

@JAORMX is already working on this.

Since Jul 24, 2026.

Dominant language
Go
Stars
2.2k
Forks
300
Avg merge
1d 15h
Merged PRs (30d)
184

Description

Part of #5743. Surfaced during the #5742 regression-gate audit (matrix §11).

Problem

Mid-tools/call elicitation from vMCP to a downstream client has two unaddressed failure modes under the current go-sdk-backed shim (toolhive-core v0.0.32):

1. Advertised-but-no-stream hangs to timeout (single-pod)

The shim runs Streamable HTTP with JSONResponse on, so go-sdk routes a server→client elicitation to the client's standalone SSE ("") stream, not the tools/call response. A client MUST hold that stream open (transport.WithContinuousListening()); the default is DisableStandaloneSSE: true.

Today the only fast-fail is the capability gate: if the client never advertised elicitation at initialize, go-sdk's Elicit rejects immediately (covered by TestForwarding_Elicitation_NoDownstreamCapability). But a client that advertised the capability yet holds no open standalone stream is a runtime condition with no guard — the elicitation is handed to ss.Elicit and blocks until the caller's context deadline. That is a hang-to-timeout, not a clean fail-fast.

Delivery path: composer → vmcp.ElicitationRequesterpkg/vmcp/server/sdk_elicitation_adapter.go → shim RequestElicitationgo-sdk ServerSession.Elicit.

Design question: a pre-dispatch guard belongs in pkg/vmcp/server, keyed on the downstream client's advertised capabilities recorded on MultiSession at initialize (NOT context values — see the vMCP anti-patterns rule). The hard part is detecting "no open standalone stream": that state lives in go-sdk/shim stream lifecycle and is not currently surfaced upward. Reaching into SDK internals would violate the adapter boundary (anti-pattern #5), so this likely needs an upstream mcpcompat accessor for stream presence before a clean in-repo guard can exist.

2. Cross-pod origination is undeliverable

clientSession.RequestElicitation loads the go-sdk session bound to this pod. A server→client request can only be delivered by the pod currently holding the client's standalone stream. If the tools/call executes on pod A but the client's GET stream is on pod B, an elicitation from A cannot reach the client. The shim's cross-replica tests cover keep-alive/notification delivery on a rehydrated stream, not server-initiated request/response (elicitation/sampling) origination from the non-stream-holding replica. This needs session affinity (LB pins the standalone stream and tool calls to the same replica) or an MRTR-era redesign (2026-07-28 replaces server-initiated requests with client-polled MRTR — see #5743).

Proposed work

  1. Upstream: add a mcpcompat accessor exposing whether a session holds an open standalone stream.
  2. In-repo: pre-dispatch fast-fail in the vMCP server when elicitation is requested but the downstream advertised the capability without an open stream — return a clean error, not a hang. Add the regression test (TestForwarding_Elicitation_AdvertisedButNoStream_FastFails) that the #5742 gate leaves skipped/documented today.
  3. Design + document the cross-pod story (session affinity requirement, or defer to MRTR). Update docs/arch/10-virtual-mcp-architecture.md.

Notes

  • Freeze investment appropriately: the 2026-07-28 revision replaces server-initiated requests with MRTR, so the cross-pod fix should be shaped MRTR-first rather than doubling down on the SSE standalone-stream model.

Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.