Updating snapshot tests is impossible per Developing locally
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 7h 27m
- Merged PRs (30d)
- 222
Description
Bug description
The Developing locally guide's section Testing - Frontend is incorrect/outdated and makes it impossible to update snapshots.
The guide mentions:
To update all visual regression test snapshots, make sure Storybook is running on your machine (you can start it with pnpm storybook in a separate Terminal tab).
Trying to update snapshots with Storybook in dev mode as proposed will fail with a timeout with no obvious reason. This is due to the hot module replacement (HMR) running in the background, preventing await page.waitForLoadState('networkidle'); from succeeding, causing the timeout.
Looking at the .github/workflows/storybook-chromatic.yml file, the right way is to:
pnpm install http-server
pnpm run build-storybook
pnpm exec http-server storybook-static --port 6006
One way to fix this would be to include the above instructions in the guide. Although this would add additional complexity to the instructions. It might be worth extracting the commands into a package.json script and reuse it in both the guide and the workflow.
How to reproduce
- Prepare the app as per Developing locally
- Start storybook in one terminal:
pnpm storybook - Run visual tests in another terminal:
pnpm test:visual
Expected: tests pass
Actual: tests fail due to a timeout
Note that this happens only for chromium, not webkit.
Additional context
- Tested on Ubuntu 24.04
- related issue on storybookjs/test-runner
Debug info
- [ ] PostHog Cloud, Debug information: [please copy/paste from https://us.posthog.com/settings/project-details#variables or https://eu.posthog.com/settings/project-details#variables]
- [x] PostHog Hobby self-hosted with `docker compose`, version/commit: a2dc60846d904de647eb4c54624cae0ce7bca530
- [ ] PostHog self-hosted with Kubernetes (deprecated, see [`Sunsetting Kubernetes support`](https://posthog.com/blog/sunsetting-helm-support-posthog)), version/commit: [please provide]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Testing - Frontend section of the Developing locally guide and compare its commands with .github/workflows/storybook-chromatic.yml. Reproduce the timeout with pnpm storybook and pnpm test:visual, then verify the guide documents a working snapshot-update flow and any shared package.json or workflow instructions remain consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, typescript
- Domain
- documentation, frontend, testing-qa
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100