anthropics / anthropics/skills

Skills referencing tools that don't exist on the running platform fail silently

Abierto
#1,154 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
176k
Forks
20.9k
Merge medio
7 h 21 min
PR fusionados (30 d)
5

Descripción

When a skill's `SKILL.md` instructs Claude to call a tool that doesn't exist on the running platform, no error surfaces. Claude either falls back to prose or silently substitutes a similar-named tool with a different schema. This makes cross-platform skill development effectively untestable from one platform — the skill author has no signal that the skill ran differently than intended.

**Reproduction A — Claude.ai → Claude Code (silent fallback to prose):**

A skill instructing Claude to call `ask_user_input_v0` was run on Claude Code (where that tool doesn't exist). Claude Code didn't raise a "tool not available" error. It produced a prose response instead, as if the skill had never asked for a structured question. Same outcome was observed for `image_search`, `conversation_search`, `recent_chats`, `present_files`, and `visualize:show_widget` — 6 of 7 Claude.ai-only tools tested produced no error.

**Reproduction B — Claude Code → Claude.ai (silent tool substitution):**

A skill instructing Claude to call `Write` was run on Claude.ai (where the tool is `create_file`). Claude.ai silently substituted `create_file` and proceeded. The two tools have *opposite overwrite semantics* — `Write` overwrites existing files; `create_file` refuses with "File already exists". A skill author testing on one platform observes one behavior; the same skill on the other platform produces the opposite without any error. Same pattern for `AskUserQuestion` → `ask_user_input_v0`, `Read` → `view`, `Edit` → `str_replace`, `Bash` → `bash_tool` — fields that the analogue doesn't support are silently discarded.

**Why this matters for skill authors**

CI or smoke tests for a skill must assert that the *expected tool was actually called*, not just that the conversation produced plausible output. Otherwise a skill that "works" on the development platform will silently degrade or behave differently on other platforms, and the author has no signal until a user reports a confusing failure.

**Suggested fix:** when a `SKILL.md` references a tool name that doesn't exist in the current platform's inventory, surface a visible signal — either an error to the user, a system-reminder to Claude noting the mismatch, or a flag in the skill-discovery layer that warns the author at install time.

Full set of reproductions and a cross-platform tool-inventory matrix at https://github.com/livlign/claude-skills-pitfalls.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.