Description and schema changes are tool changes: a digest field, a Rule 2 wording, and two observations from a nightly record

Following up on Vandana's invitation on X to bring the data here.

Summary

The framework treats a server's tool list as the thing that changes between reviews: "tool inventory diff", "new tools added to an existing server", Rule 2 "Re-classify when tools change". In practice the more common change is to the text of an existing tool: its description or its input schema changes in a release while the tool list stays the same. The description is what the model reads and follows, so a changed description changes what the tool does in practice without a new tool appearing. Three small additions would close this, all in existing sections, plus two observations from a nightly record that may be useful as evidence.

1. Chapter 4, Required fields: a tool definition digest next to Version

Add a row after "Version":

Field	Description	Example	Why it matters
Tool definition digest	Hash over the tool names, descriptions and input schemas as read at the approved version	sha256:9f2a...	Makes a later release comparable to the reviewed one without re-reading every tool by hand; an identical digest means nothing the model reads has changed, so the review can stand

The "Tools / actions exposed" field records identities and read/write class; the digest records the text the model sees. In the record below, 3,058 of 4,756 new versions (64.3%) carried an identical digest to the version before: a reviewer with the digest on file skips two thirds of releases with confidence and spends the time on the third that changed.

2. Chapter 5, Rule 2, and Chapter 4, Inventory Maintenance: description and schema changes are tool changes

Rule 2 currently reads "Adding a write tool to a Tier 1 server triggers immediate re-classification." Suggested wording:

A new tool, or a change to an existing tool's description or input schema in a new release, triggers re-classification. The description is the instruction the model follows; a changed description can change what a tool does in practice without a new tool appearing. Compare the release's tool definition digest with the one recorded at approval; a match means no re-review is needed for this rule.

In "Owners must report within 5 business days", add: "A release that changes any tool's description or input schema". Since the owner usually does not know a dependency's description changed, this pairs with the automated check below rather than replacing it.

A vocabulary for triaging the changed text that has held up in practice, three tiers by what the added words do rather than by intent:

• exfiltration: the new text names a secret-bearing file (~/.ssh, ~/.aws, .env), a secret next to a verb that moves it, a move verb with a named address, or the vocabulary of exfiltration itself;
• override: the new text tells the agent to hide something from the user or to ignore prior instructions;
• sensitive terms: a destination named in ordinary product terms ("posts the result to the webhook you configured"), which describes a feature and needs a human read rather than an alarm.

The first two justify immediate re-classification; the third is a note for the reviewer. Setup lines ("export API_KEY=") and warnings ("never send the key anywhere") should not trigger the first tier; both appear constantly in benign text.

3. Appendix: the definition diff as evidence

• Evidence Pack, "Tool inventory": add "with each tool's description and input schema as reviewed, or their digest".
• Automating Evidence Collection, worked example step 4: "tool inventory diff" becomes "tool definition diff (names, descriptions, schemas)".
• "Declared versus observed" already covers this in its third case, "Declared bound, contract since mutated". Worth naming the most common instance in that bullet: the tool description or input schema reviewed at approval differs from the one the client now receives, with no tool added or removed. In the record below that is the shape of 470 of the 1,541 releases that changed in the last 30 days.

4. Two observations for "Declared versus observed", from asking remote servers directly

A remote server's declared tool list can be checked without running anything: one initialize and one tools/list over the published endpoint, nothing called. Doing this across the official registry's remote entries: of 34,017 asked, 19,203 (56.5%) returned a tool list without a sign-in, 3,893 (11.4%) required authentication before listing tools, 9,356 (27.5%) did not answer at all, and 1,542 answered with something that was not MCP. Two consequences for the framework:

• "Observed" is only possible for about half of remote servers without credentials. The Evidence Pack could say which of the two it holds for a remote entry, declared or observed, so a reviewer knows whether the tool list was ever confirmed.
• A registry entry whose endpoint does not answer is still listed. An inventory that counts listed entries as reviewed servers overstates coverage; counting listed and observed separately keeps the number honest.

Data

The public registries show how often this happens. I keep a nightly record of the tool definitions of every version of the MCP servers, agent skills and plugins on twelve public sources (78,680 entries; 25,132 servers with their tool definitions read as of 14 September 2026), and diff them release to release, reading source statically in eight languages and asking remote servers for their list without calling anything. In the last 30 days:

• 1,541 releases changed a tool list, a tool description, an input schema, or a skill's instruction file.
• 470 of those changed the description or schema of an existing tool with no tool added or removed: invisible to an inventory diff.
• 99 changes across 66 entries graded in the first two tiers above: 60 added text naming a secret-bearing file, a named address or the vocabulary of exfiltration, and 39 added an instruction to hide something from the user or to ignore prior instructions.

Every figure is counted from the record and dated: https://smallprint.dev/stats (JSON at https://smallprint.dev/api/stats). The diffs and the rule behind each grade are public at https://smallprint.dev/drift and https://smallprint.dev/how-we-grade, so any example can be checked against the source. Happy to turn this into a PR with the wording above if that is easier to review.

Start with Chapter 4's Required fields and Inventory Maintenance sections, then review Chapter 5 Rule 2 and the Appendix sections named in the issue. Incorporate the proposed digest, wording, vocabulary, and declared-versus-observed clarifications, while preserving the supplied evidence and links; done means each requested change appears in the relevant existing section.