nebari-dev / nebari-dev/llm-serving-pack
Add Playwright for E2E and Accessibility Testing
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3
- Forks
- 3
- Avg merge
- 16h 31m
- Merged PRs (30d)
- 11
Description
Summary
Add Playwright to frontend/ for browser-level end-to-end coverage of the API key management flows, plus automated accessibility checks via @axe-core/playwright. Adds a new e2e-frontend job to .github/workflows/test.yaml so both suites gate every PR alongside the existing Vitest, Go, and Helm jobs.
Motivation
- Current frontend coverage is Vitest + Testing Library only (
src/**/*.test.tsx) — jsdom, no real browser, no real navigation, no real Keycloak bootstrap path. - The multi-step key lifecycle (create → one-time secret reveal in
KeyCreatedDialog→ revoke) spans several components and is only verified in isolation today. src/auth/keycloak.tsalready exposes awindow.__PW_E2E_AUTH__injection hook explicitly reserved for Playwright — the auth seam exists but nothing uses it.- No accessibility testing exists at all; the app ships custom dialogs, a theme toggle, and Base UI/Radix primitives whose focus trapping, labeling, and contrast are currently unverified.
- Both light and dark themes (
ThemeProvider) plus the branding/theming work merged in #145 need contrast regression protection. - Real-browser tests catch class of bugs jsdom cannot: focus management, scroll locking, portal rendering, CSS-dependent visibility.
Acceptance Criteria
-
@playwright/testand@axe-core/playwrightadded tofrontend/devDependencies -
frontend/playwright.config.tscreated with awebServerblock that builds/serves the app (npm run build && npm run preview) and abaseURLmatching the preview port - Chromium project configured at minimum;
trace: "on-first-retry"andretriesset for CI - E2E specs live under
frontend/e2e/and are excluded from the Vitestincludeglob invite.config.tssonpm testdoes not pick them up - Auth is stubbed via
page.addInitScriptsettingwindow.__PW_E2E_AUTH__(no live Keycloak in CI) - Backend
/apicalls stubbed with Playwrightpage.routefixtures — no livekey-managerdependency in CI - E2E spec: keys list renders, including the empty state
- E2E spec: create a key via
CreateKeyDialog, assert the secret is shown once inKeyCreatedDialogand is copyable - E2E spec: revoke a key via
KeyRowActions→RevokeKeyDialog, assert the row updates - E2E spec: API failure path renders
ErrorBanner - E2E spec: theme toggle switches light/dark and persists across reload
- A11y spec runs
AxeBuilderagainst the main view and each open dialog, asserting zero violations forwcag2a,wcag2aa, andwcag21aatags - A11y assertions run in both light and dark themes (color-contrast included, not disabled)
- Any pre-existing violations are either fixed or recorded in a documented, time-boxed exclusion list rather than silently suppressed
-
npm run e2eandnpm run e2e:uiscripts added tofrontend/package.json -
e2e-frontendjob added to.github/workflows/test.yaml: checkout →actions/setup-node(usingfrontend/.node-version, npm cache) →npm ci→npx playwright install --with-deps chromium→npm run e2e - Playwright browser binaries cached in CI (keyed on the resolved Playwright version) to keep job time reasonable
-
playwright-report/,test-results/, andblob-report/added tofrontend/.gitignore - HTML report uploaded via
actions/upload-artifacton failure, with the same pinned-SHA convention used by the other workflow steps - Biome config updated so
frontend/e2e/lints clean undernpm run check -
README.md(ordocs/) documents how to run E2E locally, including running against a realkey-managerwithVITE_DEV_NO_AUTH=true
Out of Scope
- Component-level and hook-level unit tests — already covered by Vitest + Testing Library in
src/ - Go test coverage for
operator/andkey-manager/— handled by the existingtest-operatorandtest-key-managerjobs - Live Keycloak login flow testing (real IdP redirect + PKCE); this issue stubs auth at the
__PW_E2E_AUTH__seam - Cross-browser matrix (Firefox/WebKit) and mobile viewports — Chromium only for now, expandable later
- Visual regression / screenshot snapshot testing
- Manual accessibility audit (screen reader walkthroughs, keyboard-only review) — axe catches automated violations only, roughly 30–40% of WCAG issues
- Load or performance testing of the vLLM serving path
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading frontend/package.json, vite.config.ts, src/auth/keycloak.ts, and .github/workflows/test.yaml, then run the existing frontend checks. Add the Playwright configuration, mocked E2E and AxeBuilder specs under frontend/e2e/, scripts, ignore rules, and CI job described in the acceptance criteria. Done means the Chromium E2E and light/dark accessibility suites pass locally and in every PR.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, playwright, typescript, vite
- Domain
- accessibility, ci-cd, documentation, frontend, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100