modelcontextprotocol / modelcontextprotocol/conformance
WIP: generic scenario steering — plumbing-only client scenarios shouldn't need per-SDK fixture handlers
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 127
- Forks
- 101
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 7
Description
Parking a design note; details to be fleshed out (there may be an earlier exploration of this on a branch elsewhere — to be dug up and linked).
Pain (fresh example from #335)
All five Tier 1 SDK conformance clients failed the new json-schema-2020-12-preservation scenario with "Unknown scenario" — yet when we prototyped the missing handler in each fixture (10-35 lines each), every SDK passed 9/9. The scenario needs no SDK behavior change at all; the per-scenario fixture handler is pure dispatch plumbing, duplicated across N repos, and it rots (see #345 for in-repo drift of the same kind).
Idea
Scenarios of this shape declare their client-side steps as data over the context-passing channel from #51 (implemented), e.g. steps like "tools/list", "call tool X with the schema you got for tool Y". Each SDK fixture implements one small generic interpreter, once; after that, new plumbing-only scenarios need zero fixture changes anywhere. Scenarios needing real client behavior (elicitation decisions, sampling, auth flows) keep bespoke handlers.
Related: #51 (context channel), #345 (handler drift), #335 (motivating case, with working handler patches for all five SDKs).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the context-passing channel described in #51, then inspect the working per-SDK handlers from #335 and the drift noted in #345. Define the generic scenario-step data and interpreter boundary, while preserving bespoke handlers for scenarios requiring real client behavior. Done means plumbing-only scenarios can run across SDK fixtures without per-scenario handler changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100