github / github/gh-aw

Feature: Content redaction for public-facing outputs

Open
#34,906 0 comments 1 reaction 2 assignees Claimed by @pelikhan View on GitHub
ai-inspected community
Dominant language
Go
Stars
5.1k
Forks
541
Avg merge
5h 46m
Merged PRs (30d)
760

Description

## Feature: Content redaction for public-facing outputs

### Motivation

Agent output posted as PR/issue comments, reviews, and discussions is publicly visible. Corporate policies typically require:

- **Inclusive language** — avoiding gendered, racially insensitive, or discriminatory phrasing
- **PII protection** — no names, emails, or internal identifiers leaking into public output
- **Security disclosure discipline** — an agent that discovers "SQL injection in `auth.go`" must not post that publicly before the vulnerability is fixed
- **Corporate communication standards** — brand voice, legal disclaimers, no internal codenames

Today the only mechanism is embedding policy in the agent's main prompt. This is unreliable:

1. **Context overflow** — instructions get lost when the LLM's context is compacted or truncated
2. **No guarantee of execution** — the primary LLM can skip the check under prompt pressure or competing instructions
3. **Shared context** — the agent has full access to the vulnerability details it's supposed to suppress, making accidental disclosure a single token-generation error away

**Vs. threat detection:** Threat detection is a pass/fail gate (block or allow). Content redaction **rewrites**. "I found a SQL injection in auth.go" becomes "I found a potential issue in auth.go that should be reviewed privately."

### Proposal

A **content redaction step**: a platform-enforced job in the safe-outputs pipeline that the compiler wires into the dependency chain `agent → detection → content_redaction → safe_outputs`, so the primary agent cannot bypass it. A fresh-context subagent — **no tools, no repo access, no MCP** — reviews all text-bearing output items, seeing only the text and the policy. It rewrites non-compliant text, or blocks items that cannot be made compliant.

### Configuration

The `agent` field is a list of policy sources — **URLs** (fetched at runtime via curl), **repo-relative paths** (read from checkout), or **inline strings** — concatenated in order into the redaction agent's system prompt.

#### Full schema (all fields)

```yaml
safe-outputs:
add-comment:
max: 3
content-redaction:
agent: # required — string or string[]
- "https://corp.example.com/policy.md" # URL: fetched at runtime via curl
- ".github/policies/redaction.md" # path: read from repo checkout
- "Never disclose CVE IDs before fix" # inline: literal prompt text
model: "gpt-4o-mini" # optional — override redaction model
on-failure: block # optional — "block" (default) | "warn"
scope: # optional — defaults to all text-bearing types
- add-comment
- create-issue
runs-on: ubuntu-latest # optional — runner override
continue-on-error: false # optional — defaults to false
```

#### Minimal — single inline policy

```yaml
safe-outputs:
add-comment:
max: 1
content-redaction:
agent: "Do not disclose security vulnerabilities in public comments"
```

#### Corporate URL + cost-effective model

```yaml
safe-outputs:
add-comment:
max: 5
content-redaction:
agent: "https://corp.example.com/ai-content-policy.md"
model: "gpt-4o-mini"
```

#### Array shorthand (the list *is* the agent)

```yaml
safe-outputs:
add-comment:
max: 1
content-redaction:
- "https://corp.example.com/content-policy.md"
- ".github/policies/inclusive-language.md"
- "Never mention internal project codenames"
```

#### Conditional — runtime toggle with full config

```yaml
safe-outputs:
add-comment:
max: 3
content-redaction:
enabled: "${{ inputs.enable-content-redaction }}"
agent:
- "https://corp.example.com/content-policy.md"
- ".github/policies/inclusive-language.md"
model: "gpt-4o-mini"
on-failure: block
```

Job is always compiled in but skipped at runtime when `inputs.enable-content-redaction` is falsy.

Implementation notes (for implementer)

Follows the **threat-detection** pattern with key differences below.

#### Pipeline position & job wiring

- Slots between detection and safe_outputs: `activation → agent → detection → content_redaction → safe_outputs → conclusion`
- Built in `compiler_safe_output_jobs.go` → `buildSafeOutputsJobs()`
- `content_redaction.needs:` = `agent`, `activation`, and (if enabled) `detection`; `safe_outputs.needs:` conditionally includes `content_redaction`

#### Fail-closed by default

Opposite of threat detection (`continue-on-error: true`): if the redaction agent crashes/times out, safe_outputs does not run. Controlled via `on-failure: block|warn`; `continue-on-error: true` is an alias for `on-failure: warn`.

#### Sandboxing, engine & model

- Redaction `WorkflowData` built with `Tools: map[string]any{}`, `SafeOutputs: nil`, no MCP, no bash; `Agent`, `PermissionMode`, `MaxTurns` from main engine config explicitly excluded
- Inherits workflow's engine provider (copilot/claude/codex) and `APITarget` (GitHub Enterprise)
- `model` overrides for cost; falls back to engine's default detection model

#### Policy source assembly

Resolved at **Actions runtime**, not compile time:

- **URLs**: Go job builder emits `curl` steps downloading to `/tmp/gh-aw/content-redaction/policies/policy_0.md`, `policy_1.md`, … HTTPS-only validation at compile time; download at runtime so policies update centrally without recompile.
- **Paths**: Checkout step runs if any source is a repo-relative path; JS assembler reads from `$GITHUB_WORKSPACE/`.
- **Inline**: Passed via `CONTENT_REDACTION_AGENT` env var as `{type: "inline", value: "..."}`.

`assemble_redaction_prompt.cjs` concatenates all sources into `/tmp/gh-aw/content-redaction/redaction_prompt.txt`.

#### Agent output format

Structured `CONTENT_REDACTION_RESULT` block with per-item verdicts: `pass`, `rewrite` (replacement text + reason), or `block` (reason). Parsed by `parse_content_redaction_results.cjs` → `redacted_agent_output.json`, which downstream safe_outputs reads instead of the original agent output when redaction succeeded.

#### Scope filtering

Defaults to all 12 text-bearing safe-output types (comments, issues, PRs, discussions, reviews, project updates, memory). `scope` restricts to specific types; values use underscore form (`add_comment`, not `add-comment`) — validated at compile time.

#### Conditional compilation

When `enabled: "${{ ... }}"` is set, compiler always emits `content_redaction` with an `if:` condition. A `redaction_guard` step evaluates the expression at runtime and gates all downstream steps (curl fetch, prompt assembly, engine execution, results parsing).

#### Feature flag & error handling

Gated behind `content-redaction` feature flag during development:
```yaml
features:
content-redaction: true
```
Results parser uses ADR-29031 resilience: outer try/catch ensures outputs always set; failure reasons categorized (`agent_failure`, `parse_error`, `no_content`) and surfaced as job outputs for downstream conditions.

#### Key files

| Area | File |
|---|---|
| Config type & parsing | `pkg/workflow/content_redaction.go` |
| Job compilation | `pkg/workflow/content_redaction_job.go` |
| Compile-time validation | `pkg/workflow/content_redaction_validation.go` |
| Prompt assembly (JS) | `actions/setup/js/assemble_redaction_prompt.cjs` |
| Results parser (JS) | `actions/setup/js/parse_content_redaction_results.cjs` |
| JSON schema | `pkg/parser/schemas/main_workflow_schema.json` |
| Constants | `pkg/constants/constants.go` |
| Fixtures | `pkg/cli/workflows/test-copilot-content-redaction-*.md` |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.