modelcontextprotocol / modelcontextprotocol/conformance
tier-audit docs coverage: evaluate the feature list against the spec revision an SDK targets
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 127
- Forks
- 101
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 7
Description
The tier audit's documentation check scores SDKs against the canonical 48-feature list in .claude/skills/mcp-sdk-tier-audit/references/feature-list.md. The list carries no lifecycle metadata, and the rubric in docs-coverage-prompt.md allows only PASS, PARTIAL, or FAIL, so a feature the spec itself has removed still scores FAIL when an SDK neither implements nor documents it.
As of the 2026-07-28 revision, 11 of the 48 rows are removed or deprecated: 14 and 15 (resources/subscribe/unsubscribe, removed by SEP-2575), 23 (sampling, deprecated by SEP-2577), 29 (notifications/elicitation/complete, absent from the final 2026-07-28 schema), 30 and 31 (roots, deprecated by SEP-2577), 32 and 33 (logging, SEP-2577/SEP-2575), 36 (ping, removed by SEP-2575), and 39 and 40 (legacy HTTP+SSE, deprecated in 2025-03-26). An SDK that tracks the current revision is therefore capped at 37/48 and can never meet Tier 1's "all non-experimental features documented with examples", even when every feature it ships is documented with examples. An SDK that keeps deprecated surface scores higher, which inverts the incentive the SEP-2596 lifecycle is meant to create.
The referee already handles this correctly for conformance: scenario filtering by removedIn at --spec-version is what keeps a 2026-07-28 SDK from being failed on initialize or logging/setLevel. The docs rubric is the one check still scoring against the union of all revisions.
Proposal:
- Add lifecycle columns to
feature-list.md(introduced/deprecated/removed, with the SEP that did it), mirroring the scenarios'removedInmetadata. - Add a rule to
docs-coverage-prompt.md: when the audited SDK targets spec revision R, a feature removed at or before R scores N/A and leaves the denominator. A feature deprecated at or before R that the SDK deliberately declines also scores N/A when the omission is documented with a stated reason.
Observed in practice auditing an SDK that targets 2026-07-28 only: 37/48 with zero PARTIAL rows and all 11 FAILs being removed or deprecated features, each already accounted for in the SDK's own feature index with the SEP that retired it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .claude/skills/mcp-sdk-tier-audit/references/feature-list.md and docs-coverage-prompt.md, then compare their current rubric with the referee's scenario filtering by removedIn at --spec-version. Add lifecycle metadata and revision-aware N/A guidance for removed or deliberately declined deprecated features, preserving the documented SEP references. Done means the 2026-07-28 audit no longer counts retired features in its denominator.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- documentation, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100