microsoft / microsoft/skill-recorder
[High] Skill install can broaden `allowed-tools` beyond the human-approved plan
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 4k
- Forks
- 413
- Avg merge
- 3h 16m
- Merged PRs (30d)
- 10
Description
Severity
High — trust-boundary defect: privilege escalation past the human-in-the-loop review gate.
Location
electron/skillbuilder/builder.ts:181:
allowedTools: submission.allowedTools.length ? submission.allowedTools : plan.allowedTools,
The comment just above (lines 175–177) asserts "allowed-tools may be tightened by the agent to the final steps, but never emptied below what the plan declared" — but nothing enforces that invariant.
Mechanism
The human reviews and approves a plan with a specific allowedTools set. The builder's create turn is an LLM call returning submission.allowedTools. When non-empty, that value is written verbatim into the installed SKILL.md allowed-tools frontmatter — with no check that it is a subset of plan.allowedTools. The model can therefore add tools the reviewer never approved.
Impact
An installed skill can be granted capabilities (e.g. shell/network tools) outside what the human approved, defeating the purpose of the plan-review gate.
Suggested fix
Enforce the subset invariant the comment already claims:
const finalAllowed = submission.allowedTools.filter((t) => plan.allowedTools.includes(t));
allowedTools: finalAllowed.length ? finalAllowed : plan.allowedTools,
Allow tightening/removal; reject additions; log any dropped tools.
Regression test to add (npm test)
Unit test the create/merge logic with submission.allowedTools containing a tool not in plan.allowedTools; assert the final allowed-tools ⊆ plan.allowedTools. (The skillbuilder is currently not unit-tested at all.) Optionally add one adversarial evals/skillbuilder scenario that tries to broaden tools — but the deterministic unit test is the real guard, since the current skillbuilder eval scores only build() (the plan), never create() (the install turn where this bug lives).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in electron/skillbuilder/builder.ts around line 181 and trace the create/merge logic that writes the installed SKILL.md frontmatter. Add deterministic unit coverage for a submission containing an unapproved tool, then run npm test; done means the final allowed-tools never exceeds the approved plan and approved tightening still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- electron, typescript
- Domain
- security, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100