githubnext / githubnext/gh-aw-test

Suggested test: copilot/models-inventory-validation-flag

Closed
#13,851 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

github/gh-aw#57958 — Added opt-in `--models` validation to `gh aw compile`, which warns (without blocking compilation) when a workflow references a model absent from the active `gh aw models` inventory/alias catalog.

## Proposed test

- **Workflow file**: `test-copilot-models-inventory-validation.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue`
- **Variant**: standard

## Minimal test prompt sketch

Compile a workflow that declares an intentionally unknown/misspelled model alias with `gh aw compile --models`, assert a warning is emitted (non-fatal), and confirm the workflow still compiles and runs successfully via dispatch.

## New fixtures or secrets needed

None — uses the existing `gh aw models` catalog already available to the CLI.

## Notes

This is primarily a `gh aw compile` CLI-flag test rather than a runtime workflow behavior test, so it may fit better as a step added to `e2e.sh`'s compile-sanity checks (§16 in AGENTS.md) rather than a new `test-*.md` file — flagging for triage on whether a `.md` test or a harness-level compile assertion is the right home.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/33734461438) · copilot · auto · 19.8 AIC · ⌖ 6.58 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with AGENTS.md §16 and the existing compile-sanity checks in e2e.sh, then compare the proposed test-copilot-models-inventory-validation.md workflow shape. Exercise gh aw compile --models with an unknown model alias and verify the warning is non-fatal; done means the workflow still compiles and runs successfully via dispatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
ci-cd, cli, testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.