test(webui): gate bundled UX, security, and real-model integration
- Dominant language
- Rust
- Stars
- 467
- Forks
- 54
- Avg merge
- 4h 25m
- Merged PRs (30d)
- 310
Description
## Parent / dependencies
Parent: #1834. Depends on #1838, #1837, #1839, #1840, #1841, #1842, #1843, #1844, #1845, #1846, #1847. Final integration owner; individual children remain responsible for their own unit/contract tests.
## Ownership
Own `tests/webui_*`, frontend Playwright/accessibility/visual integration suite, shared verification script and CI integration, final cross-surface regression fixes coordinated with module owners. Browser mock fixtures do not replace a real bundled Rust server.
## Test matrix and release gate
- Build and launch BOTH server commands in UI-on/UI-off and single-model/empty-router modes, feature-on/off, nonempty API prefix, loopback generated key, explicit key/TLS and supported reverse proxy configuration. Verify root health, public shell/private data, asset/cache/CSP behavior and offline operation outside source tree. Keep required MLX runtime resources available; single bundled WebUI does not imply static linking of MLX.
- Hermetic browser flow against the real router and fake model/downloader traits: login → empty library → rescan/download → inspect → load → chat → cancel → unload/drain → remove. Exercise bad state transitions, stale multi-tab changes, unknown POST result, event gaps, auth expiration, server restart, download error/cancel, and corrupted/missing assets. Security attacks target legacy routes as well as UI adapters.
- Hardware gate with actual checkpoints: one dense plus one hybrid/MoE family from the project's available regression set, one supported VLM image input, and an opt-in small public download. Record binary SHA/features, checkpoint path and revision, hardware/OS, command, generated output, operation order and before/after resource observations. Set `MLXCEL_REQUIRE_MODELS=1` or equivalent required-fixture assertion; unavailable hardware/checkpoint is a reported blocker, not a green skip.
- Run Rust workspace/test-fast gate with Metal+accelerate and relevant workspace/all-target clippy/fmt; feature-disabled build and CUDA server smoke on its supported host. Do not parallelize GPU test binaries. No quantization/kernel arithmetic changes belong here; if unavoidable follow the teacher-forced logit trace conditions.
- Playwright Chromium/Firefox/WebKit plus actual Safari on macOS 27 (WebKit CI alone is not Safari validation); 390/1024/1440 widths, light/dark/tinted/opaque/high-contrast, 200% zoom, reduced motion, keyboard/IME and manual VoiceOver. Capture approved deterministic screenshots; failed baselines need review rather than blanket regeneration.
- Initial JS ≤200 KiB gzip, total JS ≤700 KiB gzip, bundle ≤5 MiB; local cold shell usable ≤2 s, action feedback ≤100 ms excluding backend work, no layout shifts from status updates. Record reference host/browser/dataset and startup RSS/time separately from model load. Test 1,000 catalog entries and 10,000-token transcript; observation overhead follows the Activity child's paired-run budget.
## Acceptance criteria
- [ ] One `make verify-webui` (or documented equivalent) orchestrates lint/type/unit/contract/bundle/browser checks; actual hardware checks are clearly separate required release evidence, never masked by mock success.
- [ ] Compatibility endpoints/CLI manifests and UI-disabled regression pass. CI tests the installed artifact, not only the Vite dev server.
- [ ] Shared requirement-to-test matrix accounts for every epic acceptance criterion, including security, state completion and no-model/no-network guarantees. No enabled placeholder or misleading metric survives final review.
- [ ] Publish test evidence and remaining limitations; do not close the epic while a required gate remains unverified.
Contributor guide
Research direction
Start with tests/webui_* and the existing frontend Playwright, accessibility, and visual suites, then inspect the shared verification script and CI integration. Run the documented server, browser, workspace, and hardware gates across the listed modes. Done means make verify-webui or its equivalent covers the acceptance matrix, installed-artifact checks pass, and required hardware evidence and limitations are published.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, playwright, rust
- Domain
- ci-cd, frontend, performance, security, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100