githubnext / githubnext/gh-aw-test

Suggested test: copilot/maintenance-disabled-jobs

Closed
#5,448 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#43300 — Add `maintenance.disabled_jobs` to selectively omit agentics-maintenance jobs and permissions

The compiler gained support for a `maintenance.disabled_jobs` key in `aw.json` that lets repos omit specific generated maintenance jobs (e.g. `close-expired-entities`, `apply_safe_outputs`). When all label-triggered jobs are disabled the `issues:labeled` trigger is also suppressed. There is no E2E test verifying that a workflow compiled from a source that declares `maintenance.disabled_jobs` actually omits those jobs at runtime.

## Proposed test

- **Workflow file**: `test-copilot-maintenance-disabled-jobs.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `noop` (the test validates compiler output shape, not a safe-output action)
- **Variant**: standard

## Minimal test prompt sketch

The workflow is triggered via `workflow_dispatch`. The agent reads the generated `agentics-maintenance.yml` (via a `gh-steps` or `command` safe output) and asserts that the job(s) listed in `maintenance.disabled_jobs` are absent from the compiled lockfile. Alternatively, this could be a compile-time assertion baked into the test via a custom `samples:` block that the harness validates structurally.

## New fixtures or secrets needed

This test requires an `aw.json` fixture with `maintenance.disabled_jobs` configured, or a way to pass that configuration at compile time in the test source. Open question: should the test live in a subdirectory fixture or use an inline `config:` frontmatter block if gh-aw supports that? Needs triage before implementation.

## Notes

- This is primarily a compiler correctness test rather than a runtime agent test. If the E2E harness cannot easily assert lockfile shape, it may be better as a unit test in `github/gh-aw` itself rather than an integration test here.
- The `issues:labeled` trigger suppression when all label-jobs are disabled is the most observable runtime effect.
- Overlap: `test-copilot-dispatch-workflow.md` and `test-copilot-call-workflow.md` cover workflow dispatch, but nothing covers selective maintenance job omission.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/28701594992) · 66.9 AIC · ⌖ 6.82 AIC · ⊞ 5.6K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading test-copilot-dispatch-workflow.md and test-copilot-call-workflow.md, then inspect how the harness handles workflow_dispatch, samples, and aw.json configuration. Determine whether test-copilot-maintenance-disabled-jobs.md can validate the generated agentics-maintenance.yml or whether the assertion belongs in github/gh-aw. Done means disabled maintenance jobs, and the issues:labeled trigger when applicable, are verified absent.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.