anthropics / anthropics/skills

Skills referencing tools that don't exist on the running platform fail silently

Offen
#1,154 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
176k
Forks
20.8k
Ø Merge
7 Std. 21 Min.
Gemergte PRs (30 T.)
5

Beschreibung

When a skill's `SKILL.md` instructs Claude to call a tool that doesn't exist on the running platform, no error surfaces. Claude either falls back to prose or silently substitutes a similar-named tool with a different schema. This makes cross-platform skill development effectively untestable from one platform — the skill author has no signal that the skill ran differently than intended.

**Reproduction A — Claude.ai → Claude Code (silent fallback to prose):**

A skill instructing Claude to call `ask_user_input_v0` was run on Claude Code (where that tool doesn't exist). Claude Code didn't raise a "tool not available" error. It produced a prose response instead, as if the skill had never asked for a structured question. Same outcome was observed for `image_search`, `conversation_search`, `recent_chats`, `present_files`, and `visualize:show_widget` — 6 of 7 Claude.ai-only tools tested produced no error.

**Reproduction B — Claude Code → Claude.ai (silent tool substitution):**

A skill instructing Claude to call `Write` was run on Claude.ai (where the tool is `create_file`). Claude.ai silently substituted `create_file` and proceeded. The two tools have *opposite overwrite semantics* — `Write` overwrites existing files; `create_file` refuses with "File already exists". A skill author testing on one platform observes one behavior; the same skill on the other platform produces the opposite without any error. Same pattern for `AskUserQuestion` → `ask_user_input_v0`, `Read` → `view`, `Edit` → `str_replace`, `Bash` → `bash_tool` — fields that the analogue doesn't support are silently discarded.

**Why this matters for skill authors**

CI or smoke tests for a skill must assert that the *expected tool was actually called*, not just that the conversation produced plausible output. Otherwise a skill that "works" on the development platform will silently degrade or behave differently on other platforms, and the author has no signal until a user reports a confusing failure.

**Suggested fix:** when a `SKILL.md` references a tool name that doesn't exist in the current platform's inventory, surface a visible signal — either an error to the user, a system-reminder to Claude noting the mismatch, or a flag in the skill-discovery layer that warns the author at install time.

Full set of reproductions and a cross-platform tool-inventory matrix at https://github.com/livlign/claude-skills-pitfalls.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.