vercel / vercel/vercel-plugin

Skill injection matches the whole Bash command string: heredoc bodies, read-only commands, and even grepping the skill's own name force a MANDATORY injection

Open
#158 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
287
Forks
58
Avg merge
1d 1h
Merged PRs (30d)
17

Description

Summary

pretooluse-skill-inject.mjs matches skill trigger patterns against the entire raw Bash command string, with no attempt to exclude non-code regions. As a result, a skill is force-injected (with a MANDATORY: ... You must run the Skill(...) tool banner) whenever its trigger word appears anywhere in the command text — including inside heredoc bodies, string literals, and even a grep for the skill's own name.

Version: vercel-plugin@0.24.0, Claude Code on macOS (darwin 25.5.0).

Root cause

hooks/pretooluse-skill-inject.mjs:300

const toolTarget = toolName === "Bash" ? toolInput.command || "" : toolInput.file_path || "";

The whole command string becomes the match target, and trigger patterns are word-boundary regexes (\bagent-browser\b etc.). Nothing distinguishes "the command does this" from "the command contains this text".

Reproduction

All four of these fired during a single session of ordinary backend work (a Next.js API route + tests). None of them involved a browser, a dev server, or Vercel env mutation.

1. Heredoc body — a string literal in a test fixture

cat > src/lib/__tests__/foo.test.ts <<'TS'
it('does not serve on localhost', () => {
  vi.stubEnv('NEXT_PUBLIC_APP_URL', 'http://localhost:3000')
  expect(resolve()).toEqual({ served: false })
})
TS

→ injected agent-browser ("dev server started, verify it visually"). Nothing was started; localhost:3000 is test data being written to disk.

2. Read-only listing

vercel env ls --scope myteam

→ injected env-vars. This command only lists variable names and targets; it prints no values and changes nothing.

3. Self-referential grep — the clearest case

grep -rlE "agent-browser" "$PLUGIN_DIR/hooks" 2>/dev/null | head -3

→ injected agent-browser. I was grepping the plugin's own hook directory to investigate this bug, and the investigation triggered the bug.

4. Background task completion notice

A <task-notification> for a finished background command whose text contained the word "workflow" injected the workflow skill (Vercel Workflow DevKit) via the UserPromptSubmit path, in a session with no workflow code.

Example of the emitted metadata for case 3:

{"version":1,"toolName":"Bash","matchedSkills":["agent-browser"],"injectedSkills":["agent-browser"],
 "reasons":{"agent-browser":{"trigger":"full","reasonCode":"pattern-match"}}}

Why this matters

The injected block is not a hint — it is worded as a hard directive:

MANDATORY: Your training data for these libraries is OUTDATED and UNRELIABLE. ... You MUST open and read the official docs linked below BEFORE writing ANY code. ... You must run the Skill(agent-browser) tool.

So each false positive costs real context budget and pushes the agent toward an irrelevant tool. Because the strongest signal (MANDATORY) is attached to the least reliable trigger (substring presence), the practical outcome is that an agent learns to ignore the banner — which also degrades the true positives this feature exists for.

Suggested fixes

Roughly in order of value-per-effort:

  1. Do not match inside heredoc bodies and quoted strings. A minimal version: strip <<'EOF' ... EOF / <<EOF ... EOF blocks from toolTarget before matching. This alone removes case 1, which is the most common shape (writing test fixtures and docs).
  2. Match on the command head, not the whole line. Most true positives are about what is being executed (npm run dev, vercel env pull), so matching the first token(s) of each pipeline segment would be far more precise than a whole-string scan. isDevServerCommand() at :217 already does something closer to this — the generic pattern path does not.
  3. Exclude read-only invocations for mutation-oriented skills. vercel env ls / vercel env --help should not trigger the same guidance as vercel env add.
  4. Skip when the match is the skill's own name in a path or a search pattern (grep, rg, find, ls arguments) — case 3.
  5. Reserve MANDATORY for high-confidence triggers. Consider emitting a one-line "skill available" note for reasonCode: "pattern-match" and keeping the imperative wording for explicit user requests or verified command matches.

Happy to test a patch against the reproductions above if that helps.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in hooks/pretooluse-skill-inject.mjs around lines 217 and 300, and trace both the Bash matching path and the UserPromptSubmit path. Run the four reproductions from the issue, then verify that ordinary heredocs, read-only commands, self-referential searches, and background notices no longer cause false-positive mandatory injections while genuine triggers remain covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, typescript
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.