modelcontextprotocol / modelcontextprotocol/conformance

Add conformance test (2025-06-18+): servers MUST reject JSON-RPC batch requests

Open
#378 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
127
Forks
101
Avg merge
6d 1h
Merged PRs (30d)
7

Description

Describe the bug

The conformance suite does not test whether servers reject JSON-RPC batch requests (HTTP POST bodies that are a JSON array of request objects). MCP removed batch support in 2025-06-18 (modelcontextprotocol/modelcontextprotocol#416); the draft spec continues with single-message JSONRPCMessage only.

All harness helpers (sendStatelessRequest, ctx.connect()conn.request()) send single JSON-RPC objects. A server or SDK that accepts batch arrays (or crashes on them) passes conformance today with no signal in tier-check results.

This gap applies only to spec versions where batching was removed: 2025-06-18, 2025-11-25, and draft (2026-07-28). It must not run for 2025-03-26, which requires batch support — a rejection check there would false-fail compliant servers.

To Reproduce

Steps to reproduce the behavior:

  1. Run the server conformance suite against any reference SDK, e.g. node dist/index.js server --url '<sdk url>' --suite all.
  2. Observe that no scenario POSTs a batch body such as:
    [
      { "jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {} },
      { "jsonrpc": "2.0", "id": 2, "method": "ping", "params": {} }
    ]
    
  3. Search emitted checks — there is no check ID for batch rejection (e.g. nothing matching json-rpc-batch-rejected).
  4. (Optional) Point the suite at a server that incorrectly accepts batch arrays; it still passes with no failure on this requirement.

Expected behavior

For --spec-version values ≥ 2025-06-18 (2025-06-18, 2025-11-25, draft):

  • A scenario should POST a JSON-RPC batch array via raw fetch (same pattern as dns-rebinding.ts / stateless.ts negative probes).
  • The server under test should reject it (likely HTTP 400 + JSON-RPC -32600 Invalid Request — confirm exact wording against the spec diff).
  • Conformance should emit a check (e.g. json-rpc-batch-rejected) with FAILURE if the server accepts or mishandles the batch.
  • The reference SDK / everything-server should SUCCESS on applicable versions.
  • A deliberately broken fixture that accepts batches should fail the check in vitest (per AGENTS.md negative-test pattern).

For --spec-version 2025-03-26: the check should not run (gate via introducedIn: '2025-06-18' or specVersionAtLeast). Do not attach this only to draft-only scenarios like server-stateless, or 2025-06-18 and 2025-11-25 runs would miss it.

Logs

N/A — this is a missing test, not a failing run. No check is emitted today.

Additional context

  • Wire-level negative tests with precedent: server-stateless, http-standard-headers, dns-rebinding; vitest broken fixtures in src/scenarios/server/negative.test.ts.
  • SDKs already reject batches by design: go-sdk#21, python-sdk#934. kotlin-sdk#82 still tracks 2025-03-26 batch support (backcompat only).
  • No open issue in this repo currently tracks batch rejection tests.
  • Optional follow-up (separate issue): positive batch-acceptance test for --spec-version 2025-03-26 backcompat only.
  • Stdio batch rejection could follow if stdio conformance lands (#258).
  • Spec refs to verify before implementing: 2025-06-18 changelog, 2025-03-26 transports (contrast), draft lifecycle/transports.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the wire-level negative-test patterns in the server-stateless, http-standard-headers, and dns-rebinding scenarios, then read src/scenarios/server/negative.test.ts and AGENTS.md. Run the relevant vitest tests and verify that batch rejection emits a failure when accepted, succeeds for applicable spec versions, and is skipped for 2025-03-26.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
api, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.