githubnext / githubnext/gh-aw-test

Suggested test: copilot/update-project

Open
#7,434 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

Link to the gh-aw PR: github/gh-aw#45825 — fix(update_project): parse string-typed `fields` instead of iterating over characters

A bug was fixed where `Object.entries()` iterated over character indices when `fields` was a JSON-encoded string rather than a plain object, polluting project boards with junk fields. There is no regression test for `update_project` safe output at all.

## Proposed test

- **Workflow file**: `test-copilot-update-project.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `update-project` (set a single-select field, e.g. `Status` or `Priority`, on an existing project item linked to a known test issue)
- **Variant**: standard

## Minimal test prompt sketch

Ask the agent to update a GitHub Project field on a specific item in the `githubnext/gh-aw-test` project board. The agent should call `update_project` with a plain object `fields` value (not a JSON-encoded string) to set a named single-select field. Verify via sample that the output shape is correct.

## New fixtures or secrets needed

Requires a GitHub Project (Projects v2) linked to `githubnext/gh-aw-test` with at least one item and one single-select field (e.g. `Status`). A project ID and item node ID would need to be documented or injected as workflow inputs.

## Notes

The bug in #45825 is silent — the tool call succeeds but creates garbage fields. A sample-based test would at minimum assert the tool is invoked with an object-typed `fields`, guarding against regressions. No overlap with any current test.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/29486105201) · 16.6 AIC · ⌖ 5.36 AIC · ⊞ 5.6K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with test-copilot-update-project.md and the existing sample-based workflow tests. Configure or inject a Projects v2 ID and item node ID for the githubnext/gh-aw-test board, then run the workflow_dispatch test with the copilot engine. Done means update_project is called with an object-typed fields value and the sample verifies the expected safe-output shape without creating garbage fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, markdown
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.