githubnext / githubnext/gh-aw-test

Suggested test: copilot/skills-frontmatter-nonsha-ref

Closed
#11,362 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#51455 — Allow non-SHA refs (branch/tag names) in skills frontmatter, which the compiler now resolves and pins to a commit SHA at compile time.

## Proposed test

- **Workflow file**: `test-copilot-skills-frontmatter-nonsha-ref.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue`
- **Variant**: standard

## Minimal test prompt sketch

Reuse the existing `test-copilot-skills-frontmatter.md` pattern but declare the skill using a branch or tag ref (e.g. `mattpocock/skills/diagnosing-bugs@main`) instead of a full commit SHA. Have the agent use the installed skill and create an issue confirming it was available, same as the existing test. The key assertion is that `gh aw compile` succeeds and the generated `.lock.yml` shows the ref resolved/pinned to a commit SHA rather than the branch name.

## New fixtures or secrets needed

None — reuses the same `mattpocock/skills` fixture already referenced by `test-copilot-skills-frontmatter.md`.

## Notes

Existing `test-copilot-skills-frontmatter.md` only exercises the already-pinned-SHA case, so it doesn't cover the new resolve-and-pin-at-compile-time behavior. Worth double-checking during triage that the compiler output format (where the resolved SHA appears in the lock file) is stable enough to assert on in `e2e.sh`, or whether just confirming successful compile + runtime pass is sufficient.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/31304130291) · auto · 23.3 AIC · ⌖ 2.44 AIC · ⊞ 8.3K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with test-copilot-skills-frontmatter.md and the proposed test-copilot-skills-frontmatter-nonsha-ref.md workflow details, then inspect e2e.sh to see how compile and runtime assertions are handled. Use the existing mattpocock/skills fixture with a branch or tag ref; done means gh aw compile succeeds, the lock file pins a commit SHA, and the agent creates the confirming issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.