lablup / lablup/backend.ai-webui

feat(WebUI Smoke CLI): post-process report with summary.json and diagnostic block

Open
#7,383 0 comments 0 reactions 0 assignees View on GitHub
phase-2
Dominant language
TypeScript
Stars
133
Forks
81
Avg merge
1d 11h
Merged PRs (30d)
344

Description

## Parent

- Epic: [FR-2871](https://lablup.atlassian.net/browse/FR-2871)
- Spec Task: [FR-2872](https://lablup.atlassian.net/browse/FR-2872)
- Spec: `.specs/FR-2871-webui-smoke-cli/spec.md` § FR-E

## Goal

After Playwright finishes, post-process the JSON reporter output into a self-contained report directory with `summary.json`, `environment.json`, failure-only traces/videos/HARs, and a copy-pasteable diagnostic block on each failed case.

## Acceptance criteria

- [ ] Inside `--output`, produce:

```

smoke-report-/

├── index/ # Playwright html reporter output

│ └── index.html

├── summary.json # machine-readable {total, passed, failed, skipped, durationMs, byCategory: {...}}

├── environment.json # {endpoint, role, cliVersion, webuiSha, os, playwrightVersion, chromiumVersion}

├── traces/ # failed cases only

├── videos/ # failed cases only (passing cases pruned)

└── logs/

├── console-\*.log

└── network-\*.har # failures only

```

- [ ] Each failed case includes a copy-pasteable diagnostic block: endpoint, role, webui SHA, last 20 console lines, 4xx/5xx network summary.
- [ ] The runner does **not** write any custom Playwright reporter; post-processing reads the JSON reporter output and augments it.
- [ ] `summary.json` round-trips through `JSON.parse` and contains all fields above.
- [ ] After a forced failure, `traces/` and `videos/` for that case are non-empty; for passing cases, `videos/` is empty.

## In scope

- `packages/backend.ai-webui-smoke-cli/src/report/{summary,environment,prune,diagnostic}.ts`

## Out of scope

- Authoring a custom Playwright reporter (explicitly out of scope per spec)
- HTML report styling changes — we use Playwright's bundled HTML reporter

## Test plan

- Force one test to fail and one to pass; assert resulting directory shape, file presence/absence (passing video pruned, failing video retained), and `summary.json` schema.

JIRA Issue: FR-2879

[FR-2871]: https://lablup.atlassian.net/browse/FR-2871?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[FR-2872]: https://lablup.atlassian.net/browse/FR-2872?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.