modelcontextprotocol / modelcontextprotocol/conformance
Add conformance test (2025-06-18+): servers MUST reject JSON-RPC batch requests
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 127
- Forks
- 101
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 7
Description
Describe the bug
The conformance suite does not test whether servers reject JSON-RPC batch requests (HTTP POST bodies that are a JSON array of request objects). MCP removed batch support in 2025-06-18 (modelcontextprotocol/modelcontextprotocol#416); the draft spec continues with single-message JSONRPCMessage only.
All harness helpers (sendStatelessRequest, ctx.connect() → conn.request()) send single JSON-RPC objects. A server or SDK that accepts batch arrays (or crashes on them) passes conformance today with no signal in tier-check results.
This gap applies only to spec versions where batching was removed: 2025-06-18, 2025-11-25, and draft (2026-07-28). It must not run for 2025-03-26, which requires batch support — a rejection check there would false-fail compliant servers.
To Reproduce
Steps to reproduce the behavior:
- Run the server conformance suite against any reference SDK, e.g.
node dist/index.js server --url '<sdk url>' --suite all. - Observe that no scenario POSTs a batch body such as:
[ { "jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {} }, { "jsonrpc": "2.0", "id": 2, "method": "ping", "params": {} } ] - Search emitted checks — there is no check ID for batch rejection (e.g. nothing matching
json-rpc-batch-rejected). - (Optional) Point the suite at a server that incorrectly accepts batch arrays; it still passes with no failure on this requirement.
Expected behavior
For --spec-version values ≥ 2025-06-18 (2025-06-18, 2025-11-25, draft):
- A scenario should POST a JSON-RPC batch array via raw
fetch(same pattern asdns-rebinding.ts/stateless.tsnegative probes). - The server under test should reject it (likely HTTP 400 + JSON-RPC
-32600Invalid Request — confirm exact wording against the spec diff). - Conformance should emit a check (e.g.
json-rpc-batch-rejected) withFAILUREif the server accepts or mishandles the batch. - The reference SDK / everything-server should
SUCCESSon applicable versions. - A deliberately broken fixture that accepts batches should fail the check in vitest (per AGENTS.md negative-test pattern).
For --spec-version 2025-03-26: the check should not run (gate via introducedIn: '2025-06-18' or specVersionAtLeast). Do not attach this only to draft-only scenarios like server-stateless, or 2025-06-18 and 2025-11-25 runs would miss it.
Logs
N/A — this is a missing test, not a failing run. No check is emitted today.
Additional context
- Wire-level negative tests with precedent:
server-stateless,http-standard-headers,dns-rebinding; vitest broken fixtures insrc/scenarios/server/negative.test.ts. - SDKs already reject batches by design: go-sdk#21, python-sdk#934. kotlin-sdk#82 still tracks 2025-03-26 batch support (backcompat only).
- No open issue in this repo currently tracks batch rejection tests.
- Optional follow-up (separate issue): positive batch-acceptance test for
--spec-version 2025-03-26backcompat only. - Stdio batch rejection could follow if stdio conformance lands (#258).
- Spec refs to verify before implementing: 2025-06-18 changelog, 2025-03-26 transports (contrast), draft lifecycle/transports.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the wire-level negative-test patterns in the server-stateless, http-standard-headers, and dns-rebinding scenarios, then read src/scenarios/server/negative.test.ts and AGENTS.md. Run the relevant vitest tests and verify that batch rejection emits a failure when accepted, succeeds for applicable spec versions, and is skipped for 2025-03-26.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- api, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100