githubnext / githubnext/gh-aw-test

Suggested test: copilot/update-release-missing-release-error

Closed
#15,804 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link: github/gh-aw#60845 — Clarify missing GitHub Release safe-output errors. The `update_release` handler now distinguishes "no GitHub Release for this tag" from generic API 404s, and gives a direct release-creation URL, instead of a misleading generic error.

## Proposed test

- **Workflow file**: `test-copilot-update-release-missing.md`
- **Trigger**: `workflow_dispatch` (tag input, no `release_tag` required to already have a Release — just a Git tag)
- **Engine**: copilot
- **Safe output**: `update-release` (expected to fail cleanly) or `report-incomplete`
- **Variant**: standard

## Minimal test prompt sketch

Harness pushes a lightweight Git tag (no GitHub Release) before dispatch. Agent calls `update_release` with that tag; expect the handler to reject with the new diagnostic message distinguishing "tag exists but no Release" from an API error, and the workflow completes without a false-positive success.

## New fixtures or secrets needed

None — reuse existing tag/release helpers in `e2e.sh` (`create_test_release`), but create only a Git tag (e.g. via `git tag`/`git push`) and skip `gh release create`.

## Notes

Complements existing `test-copilot-update-release.md` (happy path with a real Release). Confirms the new error-classification code path has regression coverage.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/34948148565) · copilot · auto · 151.2 AIC · ⌖ 5.93 AIC · ⊞ 8.4K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with test-copilot-update-release-missing.md and compare it with test-copilot-update-release.md, then inspect the existing tag and release helpers in e2e.sh. Run the workflow with a pushed Git tag but no GitHub Release. Done means update_release fails cleanly with the missing-Release diagnostic and the workflow has no false-positive success.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, shell
Domain
ci-cd, testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.