Run visual tests in a container
- Dominant language
- TypeScript
- Stars
- 38
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
### Description
When running visual tests, it needs to set `headless: false` to maintain snapshot quality, especially for map-related tests (for example: when running in headless mode, polygons and other map elements don’t fully render).
However, running in non-headless mode requires GPU features, and the Docker base images used so far have limited GPU settings. Attempts to manually enable GPU support haven’t succeeded yet. I experimented with multiple configurations (see the 3 dockerfiles below as small sample of my tries), but still getting the same error:
```
[visual-chromium] › e2e/test-cases/visual/all-question-types.test.ts:129:2 › All Question Types - Geolocation permission denied › select from map displays error when zooming to current location
Error: browserType.launch: Target page, context or browser has been closed
Browser logs:
Looks like you launched a headed browser without having a XServer running.
Set either 'headless: true' or use 'xvfb-run ' before running Playwright.
```
Dockerfile sample 1
```
FROM node:22.12.0-slim AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y \
python3 \
make \
g++ \
curl \
git \
cmake \
xz-utils \
openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production
COPY . .
RUN yarn install --frozen-lockfile
RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && \
./emsdk install latest && \
./emsdk activate latest
RUN cd /emsdk && \
. ./emsdk_env.sh && \
cd /app && \
yarn build --force
# ---- #
FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.48.0-focal
RUN curl -fsSL https://deb.nodesource.com/node_20.x/pool/main/n/nodejs/nodejs_20.19.3-1nodesource1_amd64.deb -o /tmp/nodejs.deb && \
dpkg -i /tmp/nodejs.deb && rm /tmp/nodejs.deb && \
npm i -g yarn@1.22.22
WORKDIR /app
COPY --from=builder /app /app
RUN yarn playwright install
CMD ["xvfb-run", "--auto-servernum", "--server-args=-screen 0 1280x720x24", "bash", "-c", "echo 'Starting tests...'; yarn workspace @getodk/web-forms test:e2e:visual-chromium --reporter=html"]
```
Dockerfile sample 2
```
FROM node:22.12.0-slim AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y \
python3 \
make \
g++ \
curl \
git \
cmake \
xz-utils \
openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production
COPY . .
RUN yarn install --frozen-lockfile
RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && \
./emsdk install latest && \
./emsdk activate latest
RUN cd /emsdk && \
. ./emsdk_env.sh && \
cd /app && \
yarn build --force
FROM ubuntu:22.04
RUN apt-get update && \
apt-get install -y curl gnupg && \
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
apt-get install -y nodejs && \
npm install -g yarn && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=builder /app /app
RUN yarn playwright install --with-deps
RUN apt-get update && \
apt-get install -y xvfb xauth libgl1 libxrender1 libxtst6 && \
rm -rf /var/lib/apt/lists/*
CMD ["bash", "-c", "Xvfb :99 -ac +extension GLX +render -screen 0 1280x720x24 -nolisten tcp > /dev/null 2>&1 & sleep 2; export DISPLAY=:99; PWDEBUG=1 yarn workspace @getodk/web-forms test:e2e:visual-chromium"]
```
Dockerfile sample 3
```
FROM node:20.19.3-slim AS builder
WORKDIR /app
RUN apt-get update && apt-get install -y \
python3 make g++ curl git cmake xz-utils openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production
COPY . .
RUN yarn install --frozen-lockfile
RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && ./emsdk install latest && ./emsdk activate latest
RUN cd /emsdk && . ./emsdk_env.sh && cd /app && yarn build --force
FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.48.0-focal
RUN apt-get update && \
apt-get install -y ca-certificates curl gnupg && \
mkdir -p /etc/apt/keyrings && \
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \
gpg --dearmor --batch --yes -o /etc/apt/keyrings/nodesource.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
apt-get update && \
apt-get install -y nodejs=20.19.3-1nodesource1 && \
npm i -g yarn@1.22.22 && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=builder /app /app
RUN yarn playwright install
CMD ["bash", "-c", "\
Xvfb :99 -ac +extension GLX +render -screen 0 1280x720x24 -nolisten tcp > /dev/null 2>&1 & \
sleep 3; \
export DISPLAY=:99; \
PWDEBUG=1 yarn workspace @getodk/web-forms test:e2e:visual-chromium --reporter=html \
"]
```
---
### Checklist
- **Does it need UI/UX design?**
- [ ] Yes. Describe the UI/UX requirements or attach a Figma link in the "Design" section.
- [x] No
- **Does it need API design?**
- [ ] Yes. Specify the endpoints, methods, or data structures needed.
- [x] No
- **Does it need design around state flow?**
- [ ] Yes. Outline the state flow requirements.
- [x] No
- **Does it need a test plan?**
- [ ] Yes. Add the test plan as a comment in this task or attach the document.
- [x] No
---
### User Stories
As a developer,
I want to run visual tests inside Docker with full rendering,
so that snapshots remain accurate and consistent with CI results regardless of the OS
---
### Related Issues
It will resolve the following issue without workarounds:
- https://github.com/getodk/web-forms/issues/549
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the visual test command `yarn workspace @getodk/web-forms test:e2e:visual-chromium` and compare the three Dockerfile samples in the issue, focusing on headed Playwright execution and Xvfb. Done means the map-related visual tests run inside Docker without the XServer error and produce accurate, consistent snapshots.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, playwright, typescript
- Domain
- devops, infrastructure, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100