modelcontextprotocol / modelcontextprotocol/inspector

--verify reports outcome "verified" and exits 0 for a skill whose resources are "dynamic", with nothing hashed

Open
#2,405 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug v2
Dominant language
TypeScript
Stars
10.9k
Forks
1.5k
Avg merge
6h 17m
Merged PRs (30d)
151

Description

--verify gives a resources: "dynamic" skill the same verdict as a skill whose every file hashed clean: "outcome":"verified", "ok":true, exit 0. The report states both things in one line, that integrity cannot be verified and that the outcome is verified.

Installed @modelcontextprotocol/inspector 2.7.0 from npm. This is not new in 2.7.0: 2.6.0's --verify help text, its dynamic-resources message and its summary headline are identical strings.

Minimal stdio server, 2026-07-28 era, one skill served three ways (source at the bottom):

mcp-inspector --cli --config cfg.json --server <name> \
  --protocol-era modern --method skills/list --verify

Case 1, resources: "dynamic"

{"uri":"file:///skills/weather-lookup/SKILL.md","name":"weather-lookup","conformance":[{"code":"dynamic-resources","severity":"warning","message":"resources is \"dynamic\": the file set is generated, so no digest is advertised and integrity cannot be verified."}],"frontmatter":[],"files":[],"ok":true,"outcome":"verified"}

stderr: Verified 1 skill and 0 files: no conformance errors. exit 0

Case 2, a one-entry manifest with a wrong digest

"files":[{"uri":"...","status":"mismatch","actualDigest":"sha256:5983bb8a...","expectedDigest":"sha256:0000...","expectedSize":129,"actualSize":129}],"ok":false,"outcome":"failed"

stderr: 1 of 1 skill failed verification (1 digest/size mismatch across 1 file). exit 7

Case 3, the same manifest with the correct digest

"files":[{"uri":"...","status":"verified","actualDigest":"sha256:5983bb8a...","expectedDigest":"sha256:5983bb8a...","expectedSize":129,"actualSize":129}],"ok":true,"outcome":"verified"

stderr: Verified 1 skill and 1 file: no conformance errors. exit 0

Case 1 and case 3 are the same event to a CI job. Same exit code, same ok, same outcome. The difference is carried only by the file count in the stderr headline and a warning row a consumer has to go looking for.

Why this looks like a gap rather than the documented behaviour

clients/cli/README.md defines the value the dynamic case returns:

| verified | 0 | Everything was checked and everything passed. |

Nothing was checked. files is [].

EXIT_CODES.SKILL_INCOMPLETE already rules on this shape, for the read-bounds case:

It is still non-zero, because reporting success for a manifest whose unread entries were never fetched is a false pass. A CI job that wants to tolerate oversized catalogs can allow 8 and still fail on 7.

A catalog advertising no digests at all is the same false pass by a different route. No MUST broken, nothing fetched, nothing cleared of anything.

One qualification on "nothing checked": the dynamic run does call resources/read, and my server served the entry file. It is parsed for the frontmatter cross-check, then dropped without being hashed, because the manifest declined to say what the hash should be.

What I am not asking for

Not exit 7, and not ok: false. The README is right that "dynamic" is a conforming wire form and that "failing CI for it would tell server authors their valid skill is broken". That argument is about failed. It does not reach verified, and exit 8 already exists as the non-zero answer for a server that broke no MUST.

SEP-2640 has a word for this state and it is not "verified". Resources: the marker exists "so that a host can tell a deliberately unverifiable skill from a malformed entry". Integrity and verification: a fetched SKILL.md is "digest-verified when the entry's resources is an array, and unverifiable when it is "dynamic"". The spec also expects the policy split a flag would express: "Hosts MAY decline to load such skills, and server authors SHOULD expect that some hosts will."

Three shapes, not mutually exclusive
  1. A fourth outcome value, unverifiable, which is the spec's own word for it, so a consumer can separate checked-and-clean from nothing-to-check without parsing conformance by severity. #2293 notes the union is switched on through a Record, so a new arm is a type error rather than a silent gap.
  2. A headline that says so: Verified 1 skill and 0 files: 1 skill advertised no digests and was not checked.
  3. An opt-in --require-digests, so a CI job standing in for a host that declines unverifiable skills can say so, without changing what every other job sees.

3 alone would leave the default report still saying verified for a skill it did not verify, which is the part I would fix first.

Repro server (srv.mjs), run as node srv.mjs dynamic | manifest | good
const send = (o) => process.stdout.write(JSON.stringify(o) + "\n");
const SKILL_MD = "---\nname: weather-lookup\ndescription: Looks up the weather for a city.\n---\n\nAsk the user for a city, then call the weather tool.\n";
const MODE = process.argv[2] || "dynamic";

let buf = "";
process.stdin.setEncoding("utf8");
process.stdin.on("data", (c) => {
  buf += c; let i;
  while ((i = buf.indexOf("\n")) >= 0) {
    const line = buf.slice(0, i); buf = buf.slice(i + 1);
    if (!line.trim()) continue;
    let msg; try { msg = JSON.parse(line); } catch { continue; }
    handle(msg);
  }
});

async function handle(msg) {
  const { id, method } = msg;
  const meta = { "io.modelcontextprotocol/protocolVersion": "2026-07-28" };
  if (method === "server/discover") {
    return send({ jsonrpc: "2.0", id, result: {
      supportedVersions: ["2026-07-28"],
      serverInfo: { name: "skill-test-server", version: "0.0.1" },
      capabilities: { resources: {}, extensions: { "io.modelcontextprotocol/skills": {} } },
      _meta: meta } });
  }
  if (method === "skills/list") {
    const fm = { name: "weather-lookup", description: "Looks up the weather for a city." };
    const uri = "file:///skills/weather-lookup/SKILL.md";
    const entry = MODE === "dynamic"
      ? { uri, name: "weather-lookup", frontmatter: fm, resources: "dynamic" }
      : { uri, name: "weather-lookup", frontmatter: fm, resources: [ { uri,
          digest: MODE === "good"
            ? "sha256:" + (await import("crypto")).createHash("sha256").update(SKILL_MD).digest("hex")
            : "sha256:" + "0".repeat(64),
          size: Buffer.byteLength(SKILL_MD) } ] };
    return send({ jsonrpc: "2.0", id, result: { resultType: "complete", skills: [entry],
      ttlMs: 0, cacheScope: "private", _meta: meta } });
  }
  if (method === "resources/read") {
    return send({ jsonrpc: "2.0", id, result: { resultType: "complete", ttlMs: 0,
      cacheScope: "private",
      contents: [ { uri: msg.params?.uri, mimeType: "text/markdown", text: SKILL_MD } ],
      _meta: meta } });
  }
  if (id !== undefined) send({ jsonrpc: "2.0", id,
    error: { code: -32601, message: "Method not found: " + method } });
}

cfg.json has three mcpServers entries, dyn / bad / good, each running node srv.mjs with the matching argument.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the --verify entry point and its outcome, summary, and EXIT_CODES.SKILL_INCOMPLETE handling; read clients/cli/README.md and issue #2293 for the existing contract. Add coverage for dynamic resources alongside the mismatch and correctly hashed cases, then settle the intended outcome and headline behavior; done means unverifiable skills are distinguishable from verified ones without changing failed or incomplete handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
cli, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.