modelcontextprotocol / modelcontextprotocol/conformance
input-required-result: sampling/roots scenarios vs SEP-2577, and capability-check accepting no inputRequests
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 127
- Forks
- 101
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 7
Description
Referee 0.2.0-alpha.10, server mode, --spec-version 2026-07-28 --suite all.
Three scenarios in the SEP-2322 set that a server following SEP-2577 cannot pass, plus two separate observations about input-required-result-capability-check, which is the fourth. The first item may well be a deliberate call about what the suite is scoring, so that part is a question rather than a bug report. The last one looks like a plain defect.
1. Three scenarios require the server to emit a deprecated request
input-required-result-basic-sampling, input-required-result-basic-list-roots and input-required-result-multiple-input-requests pass only if the server under test puts a sampling/createMessage or roots/list entry in inputRequests.
In schema/draft/schema.ts, CreateMessageRequest, CreateMessageResult, SamplingMessage, ListRootsRequest, ListRootsResult, Root and the ClientCapabilities.sampling / .roots slots all carry @deprecated Deprecated as of protocol version 2026-07-28 (SEP-2577). ElicitRequest is the one member of the InputRequest union left unmarked. SEP-2577 says new implementations "should not add support for them", and the changelog's Deprecated section repeats it.
A server that takes that guidance is then required by the suite to fail. The capabilities side compounds it: ClientCapabilities.sampling and .roots carry the same deprecation, so a client written to this revision declares neither, and the ability these three scenarios test has no conformant counterparty to exercise it against.
This may be intentional. An argument for the current behaviour: the suite scores the spec surface as written, deprecated members included, and a deprecated feature is still a valid feature for the twelve months SEP-2596 guarantees. If that is the intent then the failures are correct and we will keep them baselined on our side.
If it is not, the suite already has the shape this would want. The SEP-2575 client scenario reads _meta['io.modelcontextprotocol/clientCapabilities'] and records SKIPPED for an absent capability, and json-schema-2020-12 gates on spec version the same way. Either would let these three skip rather than fail for a server that omits the deprecated half of the union.
2. input-required-result-capability-check rejects -32021
The scenario sends _meta['io.modelcontextprotocol/clientCapabilities'] = {sampling: {}} and expects inputRequests naming sampling only.
A server whose only input-request type is elicitation cannot serve that. The 2026-07-28 revision defines MissingRequiredClientCapabilityError (-32021) for this exact case: "Returned when processing a request requires a capability the client did not declare in clientCapabilities". The check treats any JSON-RPC error as a failure, so that response scores FAILURE.
Is -32021 meant to be an acceptable outcome here? It is the only answer the schema offers a server that has nothing it may ask of the capabilities the client did declare.
3. input-required-result-capability-check passes a result carrying no inputRequests
Independent of the above, and this one looks like a plain bug. Lifting the check body out of dist/index.js and running it directly:
| Server returns | Verdict |
|---|---|
{"resultType":"input_required","inputRequests":{"a":{"method":"sampling/createMessage"}}} |
SUCCESS (intended) |
{"resultType":"input_required","inputRequests":{"a":{"method":"elicitation/create"}}} |
FAILURE (intended) |
| a complete result | FAILURE (intended) |
{"resultType":"input_required","requestState":"abc"} |
SUCCESS |
{"resultType":"input_required"} |
SUCCESS |
The cause is the else if (F(r) && r.inputRequests) arm. With inputRequests absent the chain falls through to the trailing else, where I(r) is false for an input_required result, so no violation is recorded and the check passes. A server that asks for nothing at all satisfies a check whose description is "Server only includes inputRequests for declared client capabilities".
Asserting inputRequests is present and non-empty before the loop would close it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the input-required-result-capability-check implementation, including its generated form in dist/index.js, and the related scenarios named in the issue. Compare behavior for missing, empty, and populated inputRequests, then run the conformance checks to confirm the intended verdicts. Separately inspect schema/draft/schema.ts and the SEP-2575 and json-schema-2020-12 scenario handling before deciding how deprecated sampling and roots cases should be scored.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100