githubnext / githubnext/gh-aw-test

Suggested test: copilot/features-gh-aw-detection-flag

Closed Beginner friendly
#10,152 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#49071 — "Add gh-aw-detection: true to 7 high-frequency workflows lacking anomaly/prompt-injection detection". This surfaces the `features.gh-aw-detection: true` frontmatter field, which enables the threat/anomaly detection engine job in the compiled lockfile. No existing test exercises this feature flag or asserts the detection job is present in the compiled output.

## Proposed test

- **Workflow file**: `test-copilot-gh-aw-detection.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (or `noop`) — the detection feature itself doesn't require a specific safe-output, so pick the simplest
- **Variant**: standard

## Minimal test prompt sketch

Add `features:\n gh-aw-detection: true` to frontmatter and have the agent emit a simple safe output (e.g. create-issue via `samples:`). The primary assertion is that `gh aw compile` succeeds and the generated `.lock.yml` contains the detection/anomaly-scan job, confirming the flag compiles and runs end-to-end without breaking the normal safe-output flow.

## New fixtures or secrets needed

None — reuses existing fixtures and Copilot credentials.

## Notes

No open suggestion or existing test currently covers `features.gh-aw-detection`. This is a compiler/runtime plumbing check rather than a security-outcome test — it verifies the feature flag compiles cleanly and doesn't interfere with normal safe-output delivery.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/30530172057) · aut00 · 33.4 AIC · ⌖ 6.57 AIC · ⊞ 8.3K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the existing test fixtures and the proposed test file, test-copilot-gh-aw-detection.md. Add the workflow_dispatch Copilot fixture with the gh-aw-detection frontmatter and a simple safe output, then run gh aw compile. Done means compilation succeeds and the generated .lock.yml contains the detection or anomaly-scan job without disrupting safe-output delivery.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
devtools, testing-qa
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.