collective / collective/pas-plugins-identity

make test-docker runs the federation tests against a stale image

Open
#65 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
0
Avg merge
39m
Merged PRs (30d)
38

Description

## What happens

`make test-docker` runs the federation tests against whatever
`ghcr.io/collective/pas-plugins-identity-demo:latest` happens to be on the
machine. Nothing in the local path builds it:

- `backend/Makefile`: `test-docker` is `pytest -m docker`, and
`demo-image-build` is a separate target nobody depends on.
- `tests/federation/conftest.py:126` brings the stack up with
`_compose("up", "-d")` — no `--build`.
- `tests/federation/docker-compose.yml` names an `image:` and declares no
`build:` section, so there is nothing for compose to build even if it were
asked.

CI is correct: the `docker` job in `.github/workflows/backend.yml` runs
`make demo-image-build` and then `make test-docker`. Only local runs are
exposed.

## Why it matters

The failure is silent in both directions. A green run can be green against
code that is not the working tree, and a red run can be red for something
already fixed. Tonight (2026-09-09, #48) two full runs — about twelve minutes
— were spent on a stale image before anybody thought to check when it was
built.

## What it should do

Bring the stack up with the image built from the working tree.

`pytest-docker` already does exactly this for the Dex and Keycloak stacks: its
default `docker_setup` is

```python
["up --build --wait"]
```

and it is documented as a fixture to override — *"Override this fixture in your
tests if you need to change setup actions."*

Two ways to get the same for the federation stack:

1. **Move it onto `pytest-docker`**, overriding `docker_setup`,
`docker_compose_file` and `docker_compose_project_name` in
`tests/federation/conftest.py`. `--wait` would also replace the hand-written
300-second probe loop with compose's own health gate.

The docstring at the top of that conftest says why it drives compose
directly today: `docker_services` is session-scoped and built from a single
`docker_compose_file`, so a second compose file "would either be ignored or
would fight the first for that fixture". Whether a per-directory override
settles that cleanly is the thing to establish first — that is the question
this issue turns on.

2. **Keep the hand-driven manager** and add `--build` to the `up`, which needs
the compose file to grow a `build:` section beside its `image:` (context
`backend/`, `Dockerfile.demo`). `image:` plus `build:` is valid compose and
names the tag to build, so `DEMO_IMAGE` keeps working for a run that wants a
prebuilt one.

Either way, the CI step that builds the image first becomes a cache hit and
could stay as documentation of intent, or go.

## Not this issue

A second thing made those runs unreadable, and it is a different problem: the
readiness probe accepts any 200 on `http://id.localhost:8080`, so a Plone
belonging to another project listening on `127.0.0.1:8080` answers instead of
the container. The stack then times out after 300 seconds reporting that it
never came up, while the container's own healthcheck was returning 200
throughout. The discovery document names its `issuer`, so one comparison would
tell "not up yet" from "that is not our server".

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with backend/Makefile, tests/federation/conftest.py, and tests/federation/docker-compose.yml, then compare the pytest-docker fixture setup used by the Dex and Keycloak tests. Run make test-docker to reproduce the stale-image behavior and establish whether per-directory fixture overrides work cleanly or whether the hand-driven compose path is required. Done means local federation tests use an image built from the working tree.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, python
Domain
backend, devops, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.