opensafely-core / opensafely-core/opencodelists
Make screenshots reproducible if possible
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 60
- Forks
- 16
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 17
Description
Why are we doing this?
It may be useful to rebuild the screenshots on a scheduled basis, or add a check to validate that screenshots are up-to-date in CI checks for branches.
Checking a binary diff is easy with git diff, but maybe the screenshots change, because of, for example:
- differences in Playwright behaviour across different operating systems
- differences in browser rendering behaviour following a Playwright update
- differences in test conditions (for example, dates/times — controllable, and maybe other aspects of codelist creation, such as version IDs)
Another comparison would be pixel comparison, rather than binary comparison, but again may also differ.
How will we know when it's done?
If we make the screenshot build reproducible, or we discover that this isn't possible, or it's not worth the effort.
What are we doing?
- Changing the way the screenshots work to remove variation between runs
- Investigating if the screenshots are reproducible
- Maybe trying to run taking screenshot in Docker, and maybe only via a CI workflow
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the existing screenshot build, Playwright test setup, and CI workflow. Compare screenshots across repeated runs and operating-system or Playwright changes, then evaluate Docker as a controlled environment. Done means screenshot generation is reproducible, or the investigation documents why it is not feasible or worthwhile.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, playwright, python
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100