stacklok / stacklok/toolhive

Add regression tests gating the mcp-go to go-sdk migration

Open
#5,742 1 comment 0 reactions 1 assignee View on GitHub

@JAORMX is already working on this.

Since Jul 8, 2026.

Dominant language
Go
Stars
2.2k
Forks
300
Avg merge
1d 15h
Merged PRs (30d)
184

Description

Summary

#5729 migrates ToolHive from mark3labs/mcp-go to the go-sdk-backed mcpcompat shim (toolhive-core v0.0.27). An audit found functional regressions in the shim — tracked with fixes in stacklok/toolhive-core#156. To guarantee the migration lands with zero functional loss, ToolHive needs regression tests that pin the current (mcp-go) behavior: they should fail against shim v0.0.27 and pass once #5729 bumps to the fixed toolhive-core release.

These tests are the acceptance gate for toolhive-core#156 and merge-blocking evidence for #5729.

Test matrix

1. stdio bridge notification forwarding (end-to-end)

Backend emits → client of the bridge receives, over pkg/transport/bridge.go:

  • notifications/progress (with progressToken correlation)
  • notifications/message (logging)
  • notifications/tools/list_changed — and a subsequent tools/list through the bridge reflects the changed tool set
  • notifications/resources/list_changed, notifications/prompts/list_changed

Currently ALL of these are dropped end-to-end under the shim (client half never registers progress/logging handlers; server half cannot send list_changed).

2. vMCP session lifecycle
  • Per-request session validation: terminate a session out-of-band (auth-failure path or cross-pod Terminate via shared Redis storage), then verify the origin pod rejects the next request on that Mcp-Session-Id (today the shim skips Validate for local sessions).
  • DELETE termination still evicts cache, closes backend connections, and returns the spec status.
  • Session identity binding: a second principal reusing a session ID is rejected.
3. Per-session tool projection (security-relevant)

Runtime validation that the shim's per-session go-sdk Server + live tool reconciliation preserves vMCP behavior:

  • Two concurrent sessions with different authz outcomes see different tools/list results.
  • A tool filtered for session A cannot be invoked by session A via tools/call while remaining callable in session B.
  • SetSessionTools reconciliation mid-session is reflected in the next tools/list.
4. SSE keep-alive
  • An idle long-lived text/event-stream response from the vMCP server carries periodic keep-alive bytes (SSE comments or pings) at the configured WithHeartbeatInterval cadence (currently a silent no-op under the shim).
5. Pagination behavior
  • A vMCP server exposing >1000 tools: verify downstream clients receive the complete set (go-sdk paginates at 1000; mcp-go returned everything). Either the server page size is raised or the test documents/exercises cursor-following.
6. Local proxy Host-header behavior
  • Localhost-bound proxy accessed with a non-localhost Host header behaves as before the migration (go-sdk's DNS-rebinding protection defaults on and returns 403; decide and pin the intended behavior).
7. Remote auth error mapping
  • 401 from a remote backend surfaces ErrUnauthorized/ErrAuthorizationRequired sentinels consumed by pkg/vmcp/client auth discovery.
  • 403-on-initialize maps to the legacy-SSE fallback path as it did under mcp-go.

Notes

  • Prefer e2e tests where the behavior crosses process/transport boundaries (bridge, sessions), unit/integration where a fake backend suffices (Ginkgo/Gomega per repo conventions; t.Context() in unit tests).
  • Land the tests before or together with the toolhive-core bump in #5729 so the fixes are demonstrably verified.
  • Related: stacklok/toolhive-core#156 (shim fixes), #5729 (migration PR).

Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.