github / github/gh-aw

[aw-compat] Daily compat audit 2026-05-08: 4/10 repos fail strict-mode compile (sandbox internal fields, agent:false)

Open
#30,971 0 comments 0 reactions 0 assignees View on GitHub
automation compatibility gh-aw
Dominant language
Go
Stars
5.1k
Forks
541
Avg merge
5h 46m
Merged PRs (30d)
760

Description

### Summary

- **Run date:** 2026-05-08 · build `b5ad3e1`
- **Repos audited:** 10 (code-search rate-limited; target was 20)
- **Compiled successfully after `fix --write`:** 6/10
- **Still failing after fix:** 4/10
- **Stars in failing repos:** 200 combined

> Note: `gh search code` hit a secondary rate limit throughout this run. Discovery was limited to repos found via topic search and direct lock-file verification.

### Critical Issues

#### Cluster A — `sandbox.mcp.container` / `sandbox.mcp.version` rejected in strict mode (2 repos, 193 stars)

`github/gh-aw-mcpg` (125 stars) and `github/gh-aw-firewall` (68 stars) fail because their workflows set `sandbox.mcp.container` and `sandbox.mcp.version` frontmatter fields. Strict mode now rejects these as internal implementation details. **`fix --write` does not resolve these errors.**

Example errors:
```
.github/workflows/smoke-allowonly.md:17:1: error: strict mode: 'sandbox.mcp.container' is not allowed
because it is an internal implementation detail.
.github/workflows/smoke-claude.md:16:1: error: strict mode: 'sandbox.mcp.version' is not allowed
because it is an internal implementation detail.
```

**Impact:** 11 failing workflow files across 2 repos. Fix is manual (remove fields or add `strict: false`).

#### Cluster B — `sandbox.agent: false` rejected in strict mode (1 repo, 7 stars)

`githubnext/gh-aw-test` (7 stars) has 16 test workflows that intentionally set `sandbox.agent: false` to disable the firewall for integration testing. Strict mode now blocks this. **`fix --write` does not resolve these errors.**

Example error:
```
.github/workflows/test-copilot-nosandbox-create-issue.md:9:1: error: strict mode: 'sandbox.agent: false'
is not allowed because it disables the agent sandbox firewall.
```

**Impact:** 16 failing workflow files. These are intentional test workflows — they need `strict: false` added to frontmatter.

#### Cluster C — Secrets in steps section (1 repo, 125 stars)

`github/gh-aw-mcpg` has one workflow (`smoke-proxy-github-script.md`) using a secret expression inline in steps:
```
${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
```
Strict mode requires moving secrets to `env:` bindings or a separate job.

#### Cluster D — Include resolution failure (1 repo, 0 stars)

`ymd65536/gh-aw-test` fails because a workflow includes `agentics/shared/include-link.md` from a repository requiring GitHub authentication unavailable in the audit environment. This is an environment constraint, not a product regression.

View per-repo results table

| Repository | Stars | Before | After fix | Changed files | Notes |
|---|---|---|---|---|---|
| github/gh-aw | 4429 | pass | pass | 197 | 31 warnings |
| github/gh-aw-mcpg | 125 | fail | fail | 26 | 8x sandbox.mcp.container + 1x secrets-in-steps |
| github/gh-aw-firewall | 68 | fail | fail | 34 | 3x sandbox.mcp.version |
| verkyyi/github-agent-runner | 9 | pass | pass | 7 | |
| githubnext/gh-aw-test | 7 | fail | fail | 84 | 16x sandbox.agent: false |
| rshade/gh-aw-fleet | 2 | pass | pass | 7 | |
| zircote/github-agentic-workflows | 2 | pass | pass | 7 | |
| AgentMeter/agentmeter-action | 2 | pass | pass | 3 | |
| verkyyi/agent-team-playground | 1 | pass | pass | 7 | |
| ymd65536/gh-aw-test | 0 | fail | fail | 2 | Include auth failure (env constraint) |

View recurring warnings (non-blocking)

**Single quotes in bash tools crash Copilot CLI** (github/gh-aw, github/gh-aw-mcpg)

31+ warnings where bash tool commands with single-quoted globs trigger Copilot CLI parse failures:
```
bash tool "find pkg -name '*.go'" contains single quotes that crash Copilot CLI;
truncated to safe prefix "find pkg -name" for shell() prefix-matching.
```
Compilation succeeds but shell() prefix-matching is truncated.

**Domain names instead of ecosystem identifiers** (gh-aw-mcpg, gh-aw-firewall)

Warnings to replace `api.github.com` with `github`, `proxy.golang.org` with `go`, `crates.io` with `rust`, etc. Currently warnings only.

### Recommendations

1. **Add a codemod to `fix --write`** to remove `sandbox.mcp.container` and `sandbox.mcp.version` from frontmatter (see `#30972`).
2. **For `githubnext/gh-aw-test`:** Add `strict: false` to nosandbox test workflows.
3. **Improve `fix --write` output** to explicitly report errors it could not auto-fix, rather than exiting 0 when errors remain.

**References:**
- [§25546910642](https://github.com/github/gh-aw/actions/runs/25546910642)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.