githubnext / githubnext/gh-aw-test

Suggested test: copilot/mcp-github-remote

Open
#4,419 0 comments 0 reactions 0 assignees View on GitHub
suggested new test
Dominant language
Shell
Stars
8
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Motivation

github/gh-aw#41864 — `Fix invalid JSON escapes in GitHub remote MCP gateway config generation` — fixed a bug where GitHub remote MCP env placeholders were emitted with a leading backslash (`\${...}`), producing invalid JSON. No E2E test currently exercises `type: remote` MCP servers; all existing MCP tests use `type: stdio` container MCP (e.g. `mcp/time` in `test-copilot-mcp.md`).

## Proposed test

- **Workflow file**: `test-copilot-mcp-github-remote.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (or `add-comment`)
- **Variant**: standard

## Minimal test prompt sketch

Configure a GitHub remote MCP server (type: remote, using the GitHub MCP server endpoint) and ask the agent to list the open issues in `githubnext/gh-aw-test` via MCP, then create an issue (or add a comment) summarising the count. This exercises the remote-type MCP config path, validates that the JSON gateway config is generated without escape errors, and verifies the gateway can authenticate and serve requests.

## New fixtures or secrets needed

Likely requires a `GITHUB_MCP_SERVER_TOKEN` secret (or reuse of `GH_AW_TEST_PAT`) for the remote MCP server authentication. The remote MCP endpoint URL (e.g. `https://api.githubcopilot.com/mcp/`) and token need to be confirmed before the test can be written. If the token is already available as `GH_AW_TEST_PAT`, no new secrets are needed.

## Notes

The bug in #41864 only manifests in the generated lock file (bad JSON escapes make the gateway fail to start). A compiled test with `type: remote` in `mcp-servers:` would provide a regression test for the JSON generation path as well as an integration test for the remote MCP connection. Open question: which remote MCP endpoint and auth scheme to use — this needs confirmation before writing the `.md` file.

> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/28318008590) · 110.8 AIC · ⌖ 11.6 AIC · ⊞ 5.9K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing test-copilot-mcp.md and confirm the remote MCP endpoint and authentication scheme, including whether GH_AW_TEST_PAT can be reused. Add test-copilot-mcp-github-remote.md with a workflow_dispatch copilot test using type: remote, then verify the generated lock-file JSON, remote MCP connection, and safe issue or comment output.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, github-actions
Domain
ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.