githubnext / githubnext/gh-aw-test
Suggested test: copilot/sandbox-runtime-profile
- Dominant language
- Shell
- Stars
- 8
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
## Motivation
github/gh-aw#57692 — "Replace legacy sandbox security settings with runtime profiles" removed `sandbox.agent.legacy-security` / `sandbox.agent.sudo` in favor of explicit `sandbox.agent.runtime` profiles (e.g. `docker-sudo-iptables`). No test in `gh-aw-test` exercises the new `sandbox.agent.runtime` frontmatter field directly (existing `nosandbox` tests use `dangerously-disable-sandbox-agent`, a different mechanism).
## Proposed test
- **Workflow file**: `test-copilot-sandbox-runtime-profile.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (simple confirmation that the agent ran successfully under the named runtime profile)
- **Variant**: standard
## Minimal test prompt sketch
Set `sandbox: { agent: { runtime: docker-sudo-iptables } }` in frontmatter and have the agent create a confirmation issue with a `samples:` block, verifying the workflow compiles and runs successfully with the new explicit runtime profile field (replacing the removed legacy fields).
## New fixtures or secrets needed
None.
## Notes
Low risk/low novelty from the harness's point of view (same create-issue validation path as other tests) but adds coverage for a frontmatter field with a hard compiler-level behavior change (legacy fields removed). Worth confirming `docker-sudo-iptables` is the correct/available profile name in the target gh-aw ref before compiling, since profile names could evolve.
> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/33609783863) · copilot · auto · 83.2 AIC · ⌖ 8.02 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with test-copilot-sandbox-runtime-profile.md and compare the existing create-issue validation path. Add the workflow_dispatch copilot standard variant with the stated runtime frontmatter and samples block, then compile and run it against the target gh-aw ref; done means the workflow compiles and creates the confirmation issue without new fixtures or secrets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100