[gh-aw draft] Host-side follow-ups after detector structural-eligibility work lands

> Created by **GitHub Ace** · [View Session](https://ace.githubnext.com/sessions/01M0GJ1RW9BVMPRG6X0HCKXZ7C)

> **Draft for `github/gh-aw`** — the body below is written from gh-aw's viewpoint and is ready to copy/paste into that repo. Delete this blockquote when filing. Tracked here only because it cannot be filed cross-repo from this environment.

---

**Based on:** github/gh-aw-threat-detection#916, github/gh-aw-threat-detection#954, github/gh-aw-threat-detection#955 merging and a promoted detector release.

## Context

`gh-aw-threat-detection` is landing a change that makes threat verdicts subject to **structural eligibility**: a category can only be raised if the artifacts actually contain a channel it could have come from. `malicious_patch` with zero patch files is now rejected as an unusable report rather than recorded as a finding. This was motivated by four false positives in our own runs — [32201512271](https://github.com/github/gh-aw/actions/runs/32201512271), [32203115530](https://github.com/github/gh-aw/actions/runs/32203115530), [32239956539](https://github.com/github/gh-aw/actions/runs/32239956539), [32305540872](https://github.com/github/gh-aw/actions/runs/32305540872) — where the detector scored instruction non-compliance and framework-rejected validation errors as security threats.

Eligibility is computed from declared **channels**, each carrying two independent signals: `present` (content was found) and `uninspectable` (the channel may hold content the detector could not read). Either one grants eligibility, so the check fails open — a staging fault can never suppress a real threat.

That design has a direct consequence for us: **the detector can only see what gh-aw stages.** Everything below is host-side work the detector cannot do for itself.

## 1. Stage evidence for channels the detector currently cannot see

This is the substantive item.

The detector's eligibility is scoped to the artifact bundle. MCP tool results, fetched web content, and the agent's own engine transcript are **never staged as artifacts**, so an injection delivered only through those channels leaves no trace in the analyzed inputs. The detector now documents this limitation explicitly rather than implying coverage it does not have.

The detector side of closing the gap is cheap — adding a channel is a few lines:

```go
{
name: "an MCP tool result recorded during the run",
present: len(arts.MCPToolResults) > 0,
uninspectable: arts.HasWarningForField("mcp_tool_results"),
}
```

But it stays inert until we write that data into the artifacts directory. The ask is to decide what is feasible to stage, in rough order of value:

- **MCP tool results** — the highest-value gap. A tool result is untrusted content that reaches the instruction channel by design, which is exactly the shape `prompt_injection` is meant to catch.
- **Fetched web content** — same argument, if we can capture it at the fetch boundary.
- **Outbound request metadata** — even destinations without bodies would let the detector reason about exfiltration attempts it currently cannot observe at all.

Two constraints are worth settling up front, because they shape the format more than the content does:

- **Size.** These can be large and are wholly attacker-influenced. They need bounding at the staging boundary, the way `aw_info.json` is already allowlisted and size-capped.
- **Provenance.** The detector distinguishes trusted template text from untrusted input. Staged tool results must be unambiguously marked untrusted, or they will degrade the trusted/untrusted analysis rather than improve it.

**Scope note.** Detection is a *gate on downstream actions*, not a post-hoc session audit. A secret exfiltrated mid-run has already left before detection runs — network boundaries and MCP constraints remain the defense for that, and this work does not change it. The value of staging tool results is catching *injection that shaped the staged output*, not retroactively catching exfiltration.

## 2. Plumb `GH_AW_DETECTION_CONTINUE_ON_WARNING`

Depends on github/gh-aw-threat-detection#955.

The detector is gaining a second gate, separate from `CONTINUE_ON_ERROR`: when set to `false`, it refuses to return a clean verdict from a bundle it could not fully inspect, exiting `2` with `reason=config_error`. The default is `true`, so nothing changes for existing workflows.

The compiler already emits the sibling knob as a literal in the detection job — e.g. in `smoke-copilot-standalone.lock.yml`:

```yaml
env:
GH_AW_DETECTION_CONTINUE_ON_ERROR: "false"
```

The new variable needs the same treatment: a frontmatter option, emitted into both the detection and conclude steps.

**Keep the two flags separate.** They express different host policies. `CONTINUE_ON_ERROR=false` means "the primary inputs must be staged correctly" — it is about our staging contract. `CONTINUE_ON_WARNING=false` means "I will not accept a clean verdict from a partially-read bundle" — it is about assurance strength. A host may reasonably want one without the other, and folding them together would silently tighten every workflow already running strict mode.

Recommend landing github/gh-aw-threat-detection#954 first and reading the resulting `warnings: []` data to see how often this actually fires before enabling it anywhere.

## 3. Audit comment-memory and patch staging

The detector previously described an unreadable channel to the model as *empty* — identical wording to a channel that genuinely holds nothing. The model then reported clean about content nobody looked at, and the run exited `0`. That is now fixed: such channels are described as unexamined, and two previously-silent refusal paths in comment-memory loading emit warnings.

The practical consequence for us is that **staging faults that were previously invisible will start producing `ERR_VALIDATION` warnings.** Two are worth checking proactively:

- A `comment-memory` path that is not a real directory. The detector refuses to follow a symlink here deliberately — following it would let the run under analysis point the detector at markdown outside the artifacts tree. If we ever stage this as a link, it will now warn.
- `HAS_PATCH=true` with no readable, non-empty `aw-*.patch` / `aw-*.bundle` reaching the artifacts directory.

Neither is a new failure; both are newly *visible*. Expect some noise on first rollout, and treat it as pre-existing staging bugs surfacing rather than as a regression in the detector.

## Explicitly not needed

**No change is required for `warnings: []` rendering.** `actions/setup/sh/conclude_threat_detection.sh` on `main` already delegates to `threat-detect conclude` for everything except reporting its own absence from `PATH`:

```bash
exec threat-detect conclude \
--result-file "${RESULT_FILE}" \
--detection-log "${DETECTION_LOG_FILE}"
```

So the ⚠️ warnings block from github/gh-aw-threat-detection#954 will appear in compiled workflows' job logs automatically once a detector release carrying it is promoted. (This supersedes older notes in the detector repo claiming the script reads `detection_result.json` directly — that was true before the delegation landed.)

**No recompile is needed to pick up the detector change.** The locks emit the literal `latest`, which `install_threat_detect_binary.sh` resolves at run time to the newest non-prerelease detector release. Promoting the release is sufficient.

## Checklist

- [ ] Decide which additional evidence channels are feasible to stage, and their bounded format
- [ ] Stage MCP tool results into the artifacts directory, marked untrusted and size-bounded
- [ ] Add `GH_AW_DETECTION_CONTINUE_ON_WARNING` frontmatter option; emit into detection + conclude steps
- [ ] Audit `comment-memory` and patch staging against the newly-visible warnings
- [ ] Confirm the ⚠️ warnings block renders in a real compiled workflow after promotion

Start by tracing the host-side staging and detection workflow, including actions/setup/sh/conclude_threat_detection.sh, install_threat_detect_binary.sh, and the emitted settings in smoke-copilot-standalone.lock.yml. Determine which evidence channels can be staged with bounded size and untrusted provenance, then follow the frontmatter option through the detection and conclude steps. Done means the feasible channels are staged, the warning policy is plumbed separately, staging warnings are audited, and a compiled workflow shows the warnings block.

Written by the indexing model from the issue text.