githubnext / githubnext/gh-aw-test
Suggested test: copilot/playwright-cli-mode
- Dominant language
- Shell
- Stars
- 8
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
## Motivation
Link to the gh-aw PRs: github/gh-aw#57521 (Remove built-in Playwright MCP support, default to CLI mode; `mode: mcp` now rejected) and github/gh-aw#57594 (Add AWF-aware Playwright CLI policy prompt enforcing loopback-only servers and Squid-routed egress). No existing test exercises `tools: playwright` in CLI mode at all.
## Proposed test
- **Workflow file**: `test-copilot-playwright-cli.md`
- **Trigger**: `workflow_dispatch`
- **Engine**: copilot
- **Safe output**: `create-issue` (report a short result after running a local Playwright CLI check)
- **Variant**: standard
## Minimal test prompt sketch
Configure `tools: { playwright: { mode: cli } }` with the sandbox/firewall enabled. Ask the agent to start a local static server bound to `127.0.0.1`, use the Playwright CLI to take a screenshot or snapshot of it, then create an issue summarizing success and confirming no external navigation occurred.
## New fixtures or secrets needed
None — uses existing repo content for a static page; no new fixtures.
## Notes
Verify `mode: mcp` truly fails to compile (could be asserted with `gh aw compile` in a lint-only check rather than an E2E dispatch test, since compile-time rejection isn't something `workflow_dispatch` proves). The CLI-mode happy path is the part suited to E2E; the mode:mcp rejection is more of a compiler unit-test concern.
> Generated by [🔍 Suggest New E2E Tests](https://github.com/githubnext/gh-aw-test/actions/runs/33488119915) · copilot · auto · 29.7 AIC · ⌖ 4.64 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-test+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-test%2Fsuggest-new-e2e-tests%22&type=issues)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the proposed test-copilot-playwright-cli.md workflow and review existing repository content used by similar tests. Configure the workflow_dispatch copilot test for Playwright CLI mode with the sandbox and firewall enabled, then verify it can inspect a 127.0.0.1 page, create an issue with the result, and make the mode:mcp rejection a separate compile or lint check if supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, playwright
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100