githubnext / githubnext/gh-aw-test

Suggested test: copilot/sandbox-runtime-profile

Open Beginner friendly
#13,717 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

github/gh-aw#57692 — "Replace legacy sandbox security settings with runtime profiles" removed `sandbox.agent.legacy-security` / `sandbox.agent.sudo` in favor of explicit `sandbox.agent.runtime` profiles (e.g. `docker-sudo-iptables`). No test in `gh-aw-test` exercises the new `sandbox.agent.runtime` frontmatter field directly (existing `nosandbox` tests use `dangerously-disable-sandbox-agent`, a different mechanism).

## Proposed test

- **Workflow file**: `test-copilot-sandbox-runtime-profile.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (simple confirmation that the agent ran successfully under the named runtime profile)
- **Variant**: standard

## Minimal test prompt sketch

Set `sandbox: { agent: { runtime: docker-sudo-iptables } }` in frontmatter and have the agent create a confirmation issue with a `samples:` block, verifying the workflow compiles and runs successfully with the new explicit runtime profile field (replacing the removed legacy fields).

## New fixtures or secrets needed

None.

## Notes

Low risk/low novelty from the harness's point of view (same create-issue validation path as other tests) but adds coverage for a frontmatter field with a hard compiler-level behavior change (legacy fields removed). Worth confirming `docker-sudo-iptables` is the correct/available profile name in the target gh-aw ref before compiling, since profile names could evolve.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/33609783863) · copilot · auto · 83.2 AIC · ⌖ 8.02 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with test-copilot-sandbox-runtime-profile.md and compare the existing create-issue validation path. Add the workflow_dispatch copilot standard variant with the stated runtime frontmatter and samples block, then compile and run it against the target gh-aw ref; done means the workflow compiles and creates the confirmation issue without new fixtures or secrets.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, testing
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.