microsoft / microsoft/playwright

[BUG] bundled skills grant Bash(npx:*) / Bash(npm:*), far wider than anything they invoke

Open Beginner friendly
#42,745 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
96.3k
Forks
6.5k
Avg merge
1d 6h
Merged PRs (30d)
180

Description

### What's going on?

**Version**: `@playwright/cli` 0.1.20 / `packages/playwright-core/src/tools/skills`

Two of the three bundled skills declare `allowed-tools` grants much broader than the commands they actually use.

```yaml
# skills/playwright-cli/SKILL.md
allowed-tools: Bash(playwright-cli:*) Bash(npx:*) Bash(npm:*)

# skills/playwright-trace/SKILL.md
allowed-tools: Bash(npx:*)
```

In Claude Code `allowed-tools` is a *grant*, not a restriction: the listed commands run without prompting for the turn that invokes the skill. `npx` is an environment runner that executes its arguments, and the permissions docs call out this exact shape as a hazard: a rule like `Bash(npx:*)` matches whatever follows, so it is a no-prompt pre-approval to fetch and execute any package on the npm registry.

Neither skill needs that. Grepping every `npx`/`npm` occurrence across both skills and their reference files, no invocation targets any package other than `playwright`:

| Skill | Every invocation found |
|---|---|
| playwright-trace | `npx playwright trace ` only |
| playwright-cli | `npx playwright test [...]`, `npx playwright cli [...]`, `npx --no-install playwright --version` |

The only `npm` uses in playwright-cli are one-time bootstrap (`npm install -g @playwright/cli@latest`, `npm init playwright@latest`) plus one illustrative `npm run special-test-command` example. A prompt on a once-ever install costs nothing, so `Bash(npm:*)` buys no day-to-day convenience in exchange for pre-approving every npm subcommand.

**Suggested fix** - narrow both to the inner commands actually used:

```yaml
# skills/playwright-cli/SKILL.md
allowed-tools: Bash(playwright-cli:*) Bash(npx playwright:*) Bash(npx --no-install playwright:*)

# skills/playwright-trace/SKILL.md
allowed-tools: Bash(npx playwright trace:*)
```

(`--no-install` needs its own rule because a leading flag does not prefix-match `npx playwright`.)

A deny rule is not a usable workaround: deny and ask rules override `allowed-tools` but match any subcommand, so `deny: Bash(npx:*)` would also block `npx playwright test`.

Editing the installed skill does not hold either. `install --skills` copies `SKILL.md` into the user's repo, and the CLI then compares that copy against the bundled one character for character, after normalising line endings, on every run (as observed in `skillCheck.js` in the published `@playwright/cli` 0.1.20 tarball). A narrowed `allowed-tools` no longer matches, so the CLI warns on every run that the skill does not match the tool version and tells the user to re-run `playwright-cli install --skills`, which copies the broad grant back over their edit. That leaves users choosing between a permanent warning and losing the fix, so it needs fixing in the package.

I'd like to work on this; happy to send the two-line PR if approved.

### Version

0.1.20

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/playwright-core/src/tools/skills/playwright-cli/SKILL.md and packages/playwright-core/src/tools/skills/playwright-trace/SKILL.md, then review the skill comparison behavior described in skillCheck.js. Narrow the allowed-tools entries to the Playwright commands shown in the issue and verify that the bundled skills remain consistent with installed copies without broad npx or npm grants.

Written by the indexing model from the issue text.

Assessment

Domain
cli, security
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.