getodk / getodk/web-forms

Run visual tests in a container

Open
#554 0 comments 0 reactions 0 assignees View on GitHub
test-automation
Dominant language
TypeScript
Stars
38
Forks
22
PR merge metrics
No merged PRs in 30d

Description

### Description
When running visual tests, it needs to set `headless: false` to maintain snapshot quality, especially for map-related tests (for example: when running in headless mode, polygons and other map elements don’t fully render).

However, running in non-headless mode requires GPU features, and the Docker base images used so far have limited GPU settings. Attempts to manually enable GPU support haven’t succeeded yet. I experimented with multiple configurations (see the 3 dockerfiles below as small sample of my tries), but still getting the same error:

```
[visual-chromium] › e2e/test-cases/visual/all-question-types.test.ts:129:2 › All Question Types - Geolocation permission denied › select from map displays error when zooming to current location

Error: browserType.launch: Target page, context or browser has been closed

Browser logs:

Looks like you launched a headed browser without having a XServer running.
Set either 'headless: true' or use 'xvfb-run ' before running Playwright.
```

Dockerfile sample 1

```
FROM node:22.12.0-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y \
python3 \
make \
g++ \
curl \
git \
cmake \
xz-utils \
openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*

COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production

COPY . .

RUN yarn install --frozen-lockfile

RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && \
./emsdk install latest && \
./emsdk activate latest

RUN cd /emsdk && \
. ./emsdk_env.sh && \
cd /app && \
yarn build --force

# ---- #

FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.48.0-focal

RUN curl -fsSL https://deb.nodesource.com/node_20.x/pool/main/n/nodejs/nodejs_20.19.3-1nodesource1_amd64.deb -o /tmp/nodejs.deb && \
dpkg -i /tmp/nodejs.deb && rm /tmp/nodejs.deb && \
npm i -g yarn@1.22.22

WORKDIR /app
COPY --from=builder /app /app

RUN yarn playwright install

CMD ["xvfb-run", "--auto-servernum", "--server-args=-screen 0 1280x720x24", "bash", "-c", "echo 'Starting tests...'; yarn workspace @getodk/web-forms test:e2e:visual-chromium --reporter=html"]

```

Dockerfile sample 2

```

FROM node:22.12.0-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y \
python3 \
make \
g++ \
curl \
git \
cmake \
xz-utils \
openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*

COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production

COPY . .

RUN yarn install --frozen-lockfile

RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && \
./emsdk install latest && \
./emsdk activate latest

RUN cd /emsdk && \
. ./emsdk_env.sh && \
cd /app && \
yarn build --force

FROM ubuntu:22.04

RUN apt-get update && \
apt-get install -y curl gnupg && \
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
apt-get install -y nodejs && \
npm install -g yarn && \
rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY --from=builder /app /app

RUN yarn playwright install --with-deps

RUN apt-get update && \
apt-get install -y xvfb xauth libgl1 libxrender1 libxtst6 && \
rm -rf /var/lib/apt/lists/*

CMD ["bash", "-c", "Xvfb :99 -ac +extension GLX +render -screen 0 1280x720x24 -nolisten tcp > /dev/null 2>&1 & sleep 2; export DISPLAY=:99; PWDEBUG=1 yarn workspace @getodk/web-forms test:e2e:visual-chromium"]

```

Dockerfile sample 3

```
FROM node:20.19.3-slim AS builder
WORKDIR /app

RUN apt-get update && apt-get install -y \
python3 make g++ curl git cmake xz-utils openjdk-17-jre-headless \
&& rm -rf /var/lib/apt/lists/*

COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --production
COPY . .
RUN yarn install --frozen-lockfile

RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk && \
cd /emsdk && ./emsdk install latest && ./emsdk activate latest

RUN cd /emsdk && . ./emsdk_env.sh && cd /app && yarn build --force

FROM --platform=linux/amd64 mcr.microsoft.com/playwright:v1.48.0-focal

RUN apt-get update && \
apt-get install -y ca-certificates curl gnupg && \
mkdir -p /etc/apt/keyrings && \
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \
gpg --dearmor --batch --yes -o /etc/apt/keyrings/nodesource.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
apt-get update && \
apt-get install -y nodejs=20.19.3-1nodesource1 && \
npm i -g yarn@1.22.22 && \
rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY --from=builder /app /app

RUN yarn playwright install

CMD ["bash", "-c", "\
Xvfb :99 -ac +extension GLX +render -screen 0 1280x720x24 -nolisten tcp > /dev/null 2>&1 & \
sleep 3; \
export DISPLAY=:99; \
PWDEBUG=1 yarn workspace @getodk/web-forms test:e2e:visual-chromium --reporter=html \
"]

```

---

### Checklist
- **Does it need UI/UX design?**
- [ ] Yes. Describe the UI/UX requirements or attach a Figma link in the "Design" section.
- [x] No
- **Does it need API design?**
- [ ] Yes. Specify the endpoints, methods, or data structures needed.
- [x] No
- **Does it need design around state flow?**
- [ ] Yes. Outline the state flow requirements.
- [x] No
- **Does it need a test plan?**
- [ ] Yes. Add the test plan as a comment in this task or attach the document.
- [x] No
---

### User Stories

As a developer,
I want to run visual tests inside Docker with full rendering,
so that snapshots remain accurate and consistent with CI results regardless of the OS

---

### Related Issues

It will resolve the following issue without workarounds:

- https://github.com/getodk/web-forms/issues/549

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the visual test command `yarn workspace @getodk/web-forms test:e2e:visual-chromium` and compare the three Dockerfile samples in the issue, focusing on headed Playwright execution and Xvfb. Done means the map-related visual tests run inside Docker without the XServer error and produce accurate, consistent snapshots.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, playwright, typescript
Domain
devops, infrastructure, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.