microsoft / microsoft/agent-governance-toolkit

[Feature]: optionally scrub hidden instruction carriers from tool output

Open
#3,670 1 comment 0 reactions 0 assignees View on GitHub
needs-review:MEDIUM
Dominant language
Python
Stars
6.3k
Forks
1.1k
Avg merge
5d 11h
Merged PRs (30d)
142

Description

## Package

Other — `agent-governance-opencode`

## Problem Statement

`tool.execute.after` scans tool output for secrets but does not handle instruction-shaped content in hidden carriers. Poisoned README content, issue bodies, tool results, or pull-request descriptions can therefore reach the model verbatim.

Two deterministic carrier classes are especially useful to address because they are invisible or misleading in normal human review:

- HTML comments and hidden script/style/remote-asset blocks;
- zero-width and bidirectional control characters.

## Impact

In-process output transformation is one of the OpenCode plugin's strongest capabilities because it can rewrite content before the model sees it. A surgical scrubber caught all hidden-carrier payloads in our deterministic harness without changing ordinary code, diffs, or markup.

## Proposed Solution

- Add an optional tool-output scrubbing pass, separate from secret detection.
- Strip or visibly neutralize HTML comments, zero-width/bidirectional controls, script/style blocks, and configurable remote-asset tags.
- Preserve source-location/audit metadata without logging removed payloads.
- Add a false-positive corpus covering source code, diffs, Markdown, HTML examples, and legitimate international text.
- Make the transformation and limitations visible in status and audit output.

## Alternatives Considered

- Rely only on model-side prompt-injection detection. That is probabilistic and occurs after the hidden carrier reaches the model.
- Perform scrubbing in every host integration independently. This duplicates a control AGT can provide centrally.

## Priority

Important

## Contribution

- [x] I would be willing to submit a PR for this feature

## Coordination status

No implementation PR is currently linked (checked 2026-08-11). #3665 and #3674 cover secret redaction in advisory mode, while #3667 covers configurable secret patterns; neither implements the optional hidden-instruction scrubbing proposed here.

Contributors are welcome to propose an implementation. Please comment here before starting, search open PRs for overlapping work, and include `Closes #3670` in the PR description.

Contributor guide

Open the contributing guide

Research direction

Start at the tool.execute.after output path and inspect the existing deterministic harness for secret scanning. Trace how transformed output, source-location metadata, status, and audit information are represented before defining the optional scrubbing boundary. Done means the listed hidden carriers are handled without changing the false-positive corpus, ordinary code, diffs, Markdown, HTML examples, or legitimate international text.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.