githubnext / githubnext/gh-aw-test

Suggested test: copilot/engine-model-with-max-ai-credits

Closed Beginner friendly
#9,783 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#48482 — Fix imported `engine.model` silently dropped when `max-ai-credits` is set

When a workflow sets `max-ai-credits` at the top level and also pins `engine.model` (whether directly or via `@import`), the model pin was silently discarded before this fix. No regression test covers the combination of `max-ai-credits` + explicit model selection.

## Proposed test

- **Workflow file**: `test-copilot-engine-model-with-max-ai-credits.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (minimal, to confirm the workflow ran without falling back to the default model expression)
- **Variant**: standard

## Minimal test prompt sketch

Write a workflow that sets both `max-ai-credits: 50` and an explicit `engine.model` pin in frontmatter. Instruct the agent to create a trivial issue. The compiled lockfile should embed the pinned model literal (not the `${{ vars.GH_AW_MODEL_AGENT_COPILOT || ... }}` fallback expression), which the test runner can verify by inspecting the generated `.lock.yml`.

## New fixtures or secrets needed

None. Standard repo is sufficient; model pin can be any valid model string (e.g. `claude-sonnet-4.6`).

## Notes

The lockfile assertion (checking that the compiled model expression is not the dynamic vars fallback) could be done in `e2e.sh` as a compile-time check rather than a live-run check, making this test cheap to run. This complements the existing `test-copilot-dispatch-workflow` and `test-copilot-call-workflow` coverage.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/30346216775) · sonnet46 · 24.7 AIC · ⌖ 5.9 AIC · ⊞ 6.1K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the proposed test-copilot-engine-model-with-max-ai-credits.md workflow and compare it with test-copilot-dispatch-workflow and test-copilot-call-workflow. Use e2e.sh to see how compiled .lock.yml files are checked. Done means the workflow sets max-ai-credits and an explicit copilot engine.model, and the lockfile contains the pinned model literal rather than the dynamic fallback expression.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.