githubnext / githubnext/gh-aw-test

Suggested test: copilot/runtimes-elixir

Closed Beginner friendly
#6,210 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#44283 — fix: expose erl in AWF sandbox when using runtimes.elixir

PR #44283 added Elixir runtime support to the AWF sandbox: it emits a `Capture ERLANG_HOME` step after `erlef/setup-beam` and prepends `ERLANG_HOME/bin` to PATH inside the chroot, so `mix` can find `erl`. There is no E2E test exercising the Elixir runtime path.

## Proposed test

- **Workflow file**: `test-copilot-runtimes-elixir.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue`
- **Variant**: standard (sandbox enabled)

## Minimal test prompt sketch

Declare `runtimes.elixir: true` in the frontmatter. Ask the agent to run `elixir -e "IO.puts(:os.type())"` via a `command` or `gh-steps` block to verify Elixir is accessible inside the sandbox, then create an issue reporting the output. The test passes if the issue body contains the expected Elixir output (e.g. `{:unix, :linux}`).

## New fixtures or secrets needed

None — the Elixir runtime is provided by the AWF sandbox machinery (`erlef/setup-beam`). No additional secrets or repos are required.

## Notes

The existing `test-copilot-gh-steps.md` provides a model for how to exercise shell commands inside the sandbox and capture output to a safe-output. The Elixir test would be similar but specifically validates the `ERLANG_HOME` injection from #44283. If `runtimes.elixir` is a rarely-used feature, a `nosandbox` variant may not be needed.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/29009122314) · 97.4 AIC · ⌖ 8.19 AIC · ⊞ 5.6K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading test-copilot-gh-steps.md and the proposed test-copilot-runtimes-elixir.md entry point. Mirror the existing sandbox command pattern, enable runtimes.elixir in frontmatter, and run the workflow to confirm the safe issue output contains the expected Elixir result such as {:unix, :linux}.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir, github-actions
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.