web-infra-dev / web-infra-dev/midscene
[Bug]: HTML report shows Playground demo data (SauceDemo) instead of real UI context after aiAssert
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 14.9k
- Forks
- 1.2k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 96
Description
Version
- @midscene/core: 1.12.0
- @midscene/web: 1.12.0
- Playwright: 1.62.1
- Node.js: 24.12.0
- macOS (darwin)
Describe the bug
When running Midscene.js with Playwright (via @midscene/web/playwright PlaywrightAiFixture), AI actions (aiAssert) execute correctly against the real page — the AI model's analysis text is correct (e.g. "a gray toast ... displayed in the center of the page" → StatementIsTruthy: true). The screenshot is also captured successfully at runtime (screenshotBase64 begin/end in web-page.log).
However, the generated HTML report does not contain the real UI context (screenshot + DOM). Instead, the report shows Playground demo data — a SauceDemo page (Swag Labs, Username, Login, etc.) as the UI context preview.
Steps to reproduce
- Set up Playwright + Midscene (PlaywrightAiFixture).
- Write a test that calls
aiAssert(...)on any page (e.g. mobile dispatch page). - Run the test, let aiAssert execute successfully.
- Open the generated HTML report under
midscene_run/report/.
Expected behavior
The report should show the real screenshot + DOM context captured at runtime.
Actual behavior
The report shows SauceDemo demo data instead of the real page context.
Root cause analysis (from source)
packages/visualizer/src/component/context-preview/index.tsxrendersNo UI context+ demo data when no real UI context is provided to the report.- The demo data is
packages/visualizer/src/component/playground/playground-demo-ui-context.json(SauceDemoSwag Labs). - AI runtime DOES capture the screenshot (confirmed in
midscene_run/log/web-page.log:screenshotBase64 begin/end), and the model's analysis text references the real screenshot correctly (midscene_run/log/ai-call.log). - But the report data pipeline seems to lose the UI context (screenshot) before it reaches the HTML reporter, so the report falls back to demo data.
Additional notes
midscene_run/log/cache.logshowsno cache file foundevery run — cache is not the cause.- The report file is freshly generated (current timestamp), not stale.
- This affects
@midscene/web/playwrightplaywright reporterseparatemode with defaultsingle-htmloutput.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/visualizer/src/component/context-preview/index.tsx and the PlaywrightAiFixture report path, then reproduce the issue using the steps and compare web-page.log, ai-call.log, and the generated report. Trace where the captured screenshot and DOM are lost before HTML generation. Done means the report shows the real page context instead of the SauceDemo data from playground-demo-ui-context.json.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100