githubnext / githubnext/gh-aw-test

Suggested test: copilot/sandbox-exclude-env

Open
#7,607 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR or issue: github/gh-aw#46076 — fix: exclude job-output credential env vars from agent sandbox (`--exclude-env`)

## Proposed test

- **Workflow file**: `test-copilot-sandbox-exclude-env.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `add-comment` or `command`
- **Variant**: standard (sandbox enabled)

## Minimal test prompt sketch

Create a workflow that sets a job-output env var containing a credential-like value, then invokes the agent. The agent attempts to read the excluded env var inside the sandbox and reports what it sees (or cannot see). The harness asserts the agent run succeeds without leaking the credential into the sandbox environment, confirming the `--exclude-env` fix works end-to-end.

## New fixtures or secrets needed

A dummy (non-secret) env var name to act as the credential placeholder — no real secrets needed.

## Notes

Existing `test-copilot-network-isolation.md` covers firewall/network isolation but not env-var exclusion. This is a regression test for the sandbox leak fixed in PR #46076.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/29569068059) · 26.7 AIC · ⌖ 5.61 AIC · ⊞ 5.6K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the existing test-copilot-network-isolation.md workflow and the proposed test-copilot-sandbox-exclude-env.md fixture. Review how the workflow_dispatch trigger, copilot engine, safe output, and standard sandbox variant are expressed, then run the new workflow through the existing harness. Done means the agent run succeeds and the dummy credential-like environment variable is not visible inside the sandbox.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.