lablup / lablup/mlxcel

test(webui): gate bundled UX, security, and real-model integration

Closed
#1,848 0 comments 0 reactions 0 assignees View on GitHub
area:architecture priority:high status:in-progress type:test
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Parent / dependencies

Parent: #1834. Depends on #1838, #1837, #1839, #1840, #1841, #1842, #1843, #1844, #1845, #1846, #1847. Final integration owner; individual children remain responsible for their own unit/contract tests.

## Ownership

Own `tests/webui_*`, frontend Playwright/accessibility/visual integration suite, shared verification script and CI integration, final cross-surface regression fixes coordinated with module owners. Browser mock fixtures do not replace a real bundled Rust server.

## Test matrix and release gate

- Build and launch BOTH server commands in UI-on/UI-off and single-model/empty-router modes, feature-on/off, nonempty API prefix, loopback generated key, explicit key/TLS and supported reverse proxy configuration. Verify root health, public shell/private data, asset/cache/CSP behavior and offline operation outside source tree. Keep required MLX runtime resources available; single bundled WebUI does not imply static linking of MLX.
- Hermetic browser flow against the real router and fake model/downloader traits: login → empty library → rescan/download → inspect → load → chat → cancel → unload/drain → remove. Exercise bad state transitions, stale multi-tab changes, unknown POST result, event gaps, auth expiration, server restart, download error/cancel, and corrupted/missing assets. Security attacks target legacy routes as well as UI adapters.
- Hardware gate with actual checkpoints: one dense plus one hybrid/MoE family from the project's available regression set, one supported VLM image input, and an opt-in small public download. Record binary SHA/features, checkpoint path and revision, hardware/OS, command, generated output, operation order and before/after resource observations. Set `MLXCEL_REQUIRE_MODELS=1` or equivalent required-fixture assertion; unavailable hardware/checkpoint is a reported blocker, not a green skip.
- Run Rust workspace/test-fast gate with Metal+accelerate and relevant workspace/all-target clippy/fmt; feature-disabled build and CUDA server smoke on its supported host. Do not parallelize GPU test binaries. No quantization/kernel arithmetic changes belong here; if unavoidable follow the teacher-forced logit trace conditions.
- Playwright Chromium/Firefox/WebKit plus actual Safari on macOS 27 (WebKit CI alone is not Safari validation); 390/1024/1440 widths, light/dark/tinted/opaque/high-contrast, 200% zoom, reduced motion, keyboard/IME and manual VoiceOver. Capture approved deterministic screenshots; failed baselines need review rather than blanket regeneration.
- Initial JS ≤200 KiB gzip, total JS ≤700 KiB gzip, bundle ≤5 MiB; local cold shell usable ≤2 s, action feedback ≤100 ms excluding backend work, no layout shifts from status updates. Record reference host/browser/dataset and startup RSS/time separately from model load. Test 1,000 catalog entries and 10,000-token transcript; observation overhead follows the Activity child's paired-run budget.

## Acceptance criteria

- [ ] One `make verify-webui` (or documented equivalent) orchestrates lint/type/unit/contract/bundle/browser checks; actual hardware checks are clearly separate required release evidence, never masked by mock success.
- [ ] Compatibility endpoints/CLI manifests and UI-disabled regression pass. CI tests the installed artifact, not only the Vite dev server.
- [ ] Shared requirement-to-test matrix accounts for every epic acceptance criterion, including security, state completion and no-model/no-network guarantees. No enabled placeholder or misleading metric survives final review.
- [ ] Publish test evidence and remaining limitations; do not close the epic while a required gate remains unverified.

Contributor guide

Open the contributing guide

Research direction

Start with tests/webui_* and the existing frontend Playwright, accessibility, and visual suites, then inspect the shared verification script and CI integration. Run the documented server, browser, workspace, and hardware gates across the listed modes. Done means make verify-webui or its equivalent covers the acceptance matrix, installed-artifact checks pass, and required hardware evidence and limitations are published.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, playwright, rust
Domain
ci-cd, frontend, performance, security, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.