freeCodeCamp / freeCodeCamp/classroom

test: integrate Playwright API and browser E2E checks in CI

Open
#614 0 comments 0 reactions 1 assignee Claimed by @NewtonLC View on GitHub
Dominant language
JavaScript
Stars
155
Forks
138
PR merge metrics
No merged PRs in 30d

Description

## Summary

Add Playwright Test as the repository's integration and browser end-to-end test runner, while retaining the current Jest suite for fast unit/component coverage. The objective is to make pull requests prove that Classroom's own authenticated APIs, Prisma/PostgreSQL persistence, and critical browser workflows work together in an isolated environment.

## What this will accomplish

- Exercise the actual Next.js API routes against an isolated PostgreSQL database, rather than only mocked unit boundaries.
- Cover critical browser workflows in a real browser: teacher creates/edits/deletes a classroom, student follows an invite and joins, teacher views dashboard/progress, and admin updates a user.
- Run an inexpensive Chromium smoke suite on every pull request, in addition to the existing lint, build, and Jest checks.
- Retain actionable failure evidence: Playwright HTML report, per-test screenshots/video/traces, browser console/network evidence in traces, plus application and database logs.
- Add a broader scheduled/manual run for Firefox and WebKit and additional non-destructive regression coverage.
- Ensure test environments use seeded synthetic accounts and a dedicated database, never production credentials, production URLs, or real freeCodeCamp actions.

## Proposed implementation

### Test layout

- Keep `__tests__/` and `npm run test` for existing Jest unit/component tests.
- Add Playwright configuration and scripts for:
- API integration specs for `/api/create_class_teacher`, `/api/editclass`, `/api/deleteclass`, `/api/student_email_join`, and `/api/modifyuser`, including authorization and error-path assertions.
- Browser E2E specs for the initial money paths above.
- Use deterministic test fixtures and an explicitly named test database. Each run migrates/seeds its own database and removes only data it created.
- Mock the external freeCodeCamp API at the test boundary; do not use `TPA_API_BEARER_TOKEN` or call freeCodeCamp production from PR tests.
- Introduce a test-only auth/session fixture or controlled provider strategy so tests do not require interactive GitHub/Auth0 login or OAuth client secrets.

### Pull-request CI shape

The existing Node 24 workflow remains responsible for lint, build, and Jest. Add a separate integration/E2E job that:

1. checks out the PR and installs dependencies with `npm ci`;
2. starts a PostgreSQL 16 service and a local/mock freeCodeCamp dependency;
3. supplies only non-production test environment values, including a dedicated `DATABASE_URL` and generated test `NEXTAUTH_SECRET`;
4. runs Prisma generate/migrations and deterministic test seeding;
5. installs the Playwright Chromium browser and dependencies;
6. starts the built Next.js app locally;
7. runs the API integration suite and Chromium E2E smoke suite, with a single retry for transient browser failures;
8. uploads test artifacts on failure (and optionally reports on success) with a bounded retention period.

Use concurrency cancellation for superseded commits on the same pull request. Do not expose secrets to forked PRs; the PR suite must be able to run solely from safe, checked-in test configuration and locally mocked dependencies.

### Scheduled/manual coverage

Add a nightly `schedule` run and `workflow_dispatch` entry point on the default branch. It should run the full Playwright suite across Chromium, Firefox, and WebKit, including the broader dashboard and failure-path coverage. A pre-release/manual invocation can reuse this configuration before deployment. Browser jobs may be sharded only after the suite duration justifies it.

## Failure evidence and reporting

Configure Playwright to retain screenshots and video on failure and traces on the first retry (or retain them on failure). Upload:

- `playwright-report/` HTML report;
- `test-results/` (traces, screenshots, videos, and result metadata);
- Next.js application logs; and
- PostgreSQL container logs.

The Playwright trace viewer includes action history, DOM snapshots, console output, network requests, and source locations, making a CI failure reproducible without rerunning it locally.

## Cost and infrastructure impact

- Playwright Test is open-source and has no license or per-seat cost.
- PostgreSQL runs as a GitHub Actions service on the hosted runner, so no standing database infrastructure is required for PR checks.
- Incremental cost is GitHub Actions runner minutes, browser download/cache bandwidth, and uploaded artifact storage. Actual cost depends on this repository's GitHub plan and retention settings; start with Chromium on PRs and short failure-only artifact retention (for example, 14 days) to bound it.
- Firefox/WebKit are reserved for nightly/manual runs to avoid multiplying normal PR duration and minutes.
- No Cypress Cloud, BrowserStack/Sauce Labs, staging environment, or paid test-management service is required for this initial implementation. Those would be optional future costs only if the project needs hosted dashboards, physical-device coverage, or remote-grid capacity.

## Acceptance criteria

- [ ] Playwright is installed and documented with local commands for API integration and E2E tests.
- [ ] PR CI runs isolated API integration tests plus Chromium smoke E2E tests after the existing quality checks.
- [ ] The workflow uses PostgreSQL 16 and synthetic, deterministic data only.
- [ ] No PR test needs Auth0/GitHub OAuth credentials, freeCodeCamp production credentials, or access to production services.
- [ ] Failed runs upload HTML report, traces, screenshots, videos, and app/database logs.
- [ ] A nightly/manual workflow runs the broader suite across Chromium, Firefox, and WebKit.
- [ ] The README or contributor documentation explains prerequisites, local execution, artifact inspection, and the production-safety boundary.

## References

- [Playwright API testing](https://playwright.dev/docs/api-testing)
- [Playwright CI guidance](https://playwright.dev/docs/ci)
- [Playwright trace viewer](https://playwright.dev/docs/trace-viewer)
- [GitHub Actions artifacts](https://docs.github.com/en/actions/tutorials/store-and-share-data)
- [GitHub Actions scheduled workflows](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#schedule)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.