vercel / vercel/vercel-plugin

matchPromptWithReason: phrases/allOf/anyOf use unbounded .includes() while noneOf uses \b — short phrases substring-match inside unrelated words

Open Beginner friendly
#51 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
287
Forks
58
Avg merge
1d 1h
Merged PRs (30d)
17

Description

Summary

hooks/src/prompt-patterns.mts scores skills with an asymmetric matcher: the noneOf branch uses a word-boundary regex, but the positive branches (phrases, allOf, anyOf) use plain String.prototype.includes(). Any skill declaring a short phrase ("PPR", "stale", "use", "deploy", etc.) force-injects itself whenever the prompt contains a larger English word that happens to embed that substring.

This is a narrow, mechanical bug. It's distinct from (though aggravated by) the architectural issue in #38 and the over-broad-patterns issue in #19: even if both of those ship, "ppr" will still match "appropriate" inside a legitimate Vercel repo because the matcher itself is wrong.

Plugin version: 0.32.5.

Minimal repro

echo '{"prompt":"please plan a fix to avoid this if appropriate","cwd":"/tmp","session_id":"repro","hook_event_name":"UserPromptSubmit"}' \
  | node hooks/user-prompt-submit-skill-inject.mjs

Actual:

[vercel-plugin] Best practices auto-suggested based on prompt analysis:
  - "next-cache-components" matched: phrase "ppr" +6
…
You must run the Skill(next-cache-components) tool.

Expected: no match. The prompt has nothing to do with Next.js or Partial Prerendering. "ppr" matched the substring appropriate. skills/next-cache-components/SKILL.md declares phrases: ["PPR"], which normalizes to "ppr".

The same class of false positive triggers on any short phrase embedded in an ordinary word — approach, approval, appropriate, etc.

Root cause

hooks/src/prompt-patterns.mts, matchPromptWithReason:

// noneOf — CORRECT (uses word boundaries)
for (const term of compiled.noneOf) {
  const escaped = term.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  const re = new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`);
  if (re.test(normalizedPrompt)) { /* suppress */ }
}

// phrases — BROKEN (plain substring)
for (const phrase of compiled.phrases) {
  if (normalizedPrompt.includes(phrase)) {        // ← no word boundary
    score += 6;
  }
}

// allOf — BROKEN
for (const group of compiled.allOf) {
  const allMatch = group.every(t => normalizedPrompt.includes(t));  // ←
  if (allMatch) score += 4;
}

// anyOf — BROKEN
for (const term of compiled.anyOf) {
  if (normalizedPrompt.includes(term)) anyOfScore += 1;              // ←
}

findMatchedPhrases (same file) has the same bug.

The asymmetry — noneOf using \b, everything else using .includes() — looks like an oversight rather than an intentional design choice.

Proposed fix

Extract the noneOf regex into a small helper and use it for all four sites. Multi-word phrases like "partial prerendering" and "use cache" still match, because the boundaries sit on the outer edges only, not between interior words.

function phraseMatchesWordBoundary(normalizedPrompt: string, phrase: string): boolean {
  if (!phrase) return false;
  const escaped = phrase.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  return new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`).test(normalizedPrompt);
}

Then replace the four .includes() sites in matchPromptWithReason and findMatchedPhrases.

Verification

Using a locally patched prompt-patterns.mjs:

Prompt Before After
please plan a fix to avoid this if appropriate phrase "ppr" +6 → injected no match
i need to approach this approval process with appropriate care phrase "ppr" +6 × 3 → injected no match
help me add partial prerendering to this next.js 16 app injected (legit) still injected via "partial prerendering" and "next.js"
how do I use cacheTag in next.js 16 injected (legit) still injected

No regressions on legitimate multi-word phrase matches.

Related

  • #38 — "project context is advisory, not authoritative". Somewhat downstream of that issue: #38 would prevent the matcher from running at all in non-Vercel repos. But .includes() is still wrong in-matcher and would still misfire inside a legitimate Vercel repo.
  • #19 — "overly generic skill trigger patterns". Orthogonal: #19 is about the patterns declared in SKILL.md; this is about the matcher's interpretation of any pattern.

Happy to open a PR with the helper plus a unit test in hooks/prompt-patterns.test.ts if that's useful.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in hooks/src/prompt-patterns.mts at matchPromptWithReason and findMatchedPhrases, then read hooks/prompt-patterns.test.ts. Reproduce the short-phrase false positive from the issue and verify that matching uses boundaries consistently while legitimate multi-word phrases still match. Run the prompt-patterns tests and confirm the listed regressions are prevented.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.