microsoft / microsoft/agent-governance-toolkit
[Feature]: optionally scrub hidden instruction carriers from tool output
- Dominant language
- Python
- Stars
- 6.3k
- Forks
- 1.1k
- Avg merge
- 5d 11h
- Merged PRs (30d)
- 142
Description
## Package
Other — `agent-governance-opencode`
## Problem Statement
`tool.execute.after` scans tool output for secrets but does not handle instruction-shaped content in hidden carriers. Poisoned README content, issue bodies, tool results, or pull-request descriptions can therefore reach the model verbatim.
Two deterministic carrier classes are especially useful to address because they are invisible or misleading in normal human review:
- HTML comments and hidden script/style/remote-asset blocks;
- zero-width and bidirectional control characters.
## Impact
In-process output transformation is one of the OpenCode plugin's strongest capabilities because it can rewrite content before the model sees it. A surgical scrubber caught all hidden-carrier payloads in our deterministic harness without changing ordinary code, diffs, or markup.
## Proposed Solution
- Add an optional tool-output scrubbing pass, separate from secret detection.
- Strip or visibly neutralize HTML comments, zero-width/bidirectional controls, script/style blocks, and configurable remote-asset tags.
- Preserve source-location/audit metadata without logging removed payloads.
- Add a false-positive corpus covering source code, diffs, Markdown, HTML examples, and legitimate international text.
- Make the transformation and limitations visible in status and audit output.
## Alternatives Considered
- Rely only on model-side prompt-injection detection. That is probabilistic and occurs after the hidden carrier reaches the model.
- Perform scrubbing in every host integration independently. This duplicates a control AGT can provide centrally.
## Priority
Important
## Contribution
- [x] I would be willing to submit a PR for this feature
## Coordination status
No implementation PR is currently linked (checked 2026-08-11). #3665 and #3674 cover secret redaction in advisory mode, while #3667 covers configurable secret patterns; neither implements the optional hidden-instruction scrubbing proposed here.
Contributors are welcome to propose an implementation. Please comment here before starting, search open PRs for overlapping work, and include `Closes #3670` in the PR description.
Contributor guide
Research direction
Start at the tool.execute.after output path and inspect the existing deterministic harness for secret scanning. Trace how transformed output, source-location metadata, status, and audit information are represented before defining the optional scrubbing boundary. Done means the listed hidden carriers are handled without changing the false-positive corpus, ordinary code, diffs, Markdown, HTML examples, or legitimate international text.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100