Make the Docker-backed test recipes runnable from a git worktree with deterministic, non-colliding ports
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Description
None of the Docker-backed test recipes can be used from a git worktree today. Every one of them
pins host ports and container names to fixed values, so a second checkout — or a running
just dev-run — collides with it. Developers working one worktree per issue currently cannot run
these suites at all without stopping whatever else is up.
just test-postman Experiments_Resource # tomcat 8080, wiremock 50505, debug 5005
just test-integration -Dit.test=MyTest # db 5437, es 9207, os-upgrade 9201
just test-karate # tomcat 8080, management 8090
just test-e2e # tomcat 8080
Why it matters. Measured on PR run 32875684455: from push to knowing whether the experiment
Postman group passed is 50 minutes (the job itself is 17 min; the rest is the upstream build
phase), and a backend PR fans out 11 Postman jobs alongside the integration and Karate
batteries. That is a fine merge gate and a terrible authoring loop. Writing or debugging a single
collection or a single integration test locally should be minutes, and today it is blocked outright
inside a worktree.
What is pinned, per recipe
| Recipe | Tomcat | DB | OpenSearch | Other fixed |
|---|---|---|---|---|
test-postman |
8080 (dotcms-postman/pom.xml:35) |
dynamic | dynamic | wiremock 50505, JDWP 5005 (via -Pdebug, always passed) |
test-integration |
dynamic | 5437 (dotcms-integration/pom.xml:65) |
9207 (:66) |
os-upgrade 9201 (:67) |
test-karate |
8080 (test-karate/pom.xml:24) |
dynamic | dynamic | management 8090 (:25) |
test-e2e |
8080 (core-web/apps/dotcms-ui-e2e/pom.xml:41) |
dynamic | dynamic | — |
dev-run |
dynamic | dynamic | dynamic | — (already correct; dotCMS/pom.xml:44 sets fabric8.dynamic-port=true and leaves db.port/es.port commented out) |
Root cause — one mechanism, four local patches
docker.run.context is ${context.parent}_${context.module}_${context.name}
(parent/pom.xml:194), and container names follow ${docker.run.context}_%a (:199). Because
context.module is the artifactId, the modules do not collide with each other — the collision is
always the same module run from two different checkouts, because context.name resolves to
ext.default.context.name = default (environments/environment.properties) and nothing ever
derives it. Volumes (data-local, data-shared, db, es) hang off the same context and clash the
same way.
The ports have no automatic derivation either. parent/pom.xml declares them in fabric8's
dynamic-port form (tomcat.port:8080, db.port:5432, es.port:9200, tomcat.ssl.port:8443,
management.port:8090), which assigns a free host port unless the property is set — and each
test module sets the ones it cares about, for its own historical reason. dev-run is the one place
that leaves them alone, and it is consequently the only recipe that already behaves.
So this is not a Postman problem. It is one missing derivation, patched around in four places.
Approach
- Derive in one place.
context.nameand a port offset are computed once — inparent/pom.xml
or a sharedjustfilehelper — and every Docker-backed recipe consumes them. No per-module
reimplementation. - Outside a worktree, nothing changes. The main checkout keeps every current value
(8080/8443/8090/50505/5005,5437/9207/9201) and the current container and volume
names. The existing pins stay as the defaults. - Inside a worktree, detect it and derive deterministically. The recipe recognises a worktree
(e.g.git rev-parse --git-common-dirdiffering from--git-dir) and derives the offset and
context.namefrom a stable key — the worktree directory name or branch. Same worktree, same
ports on every run, so a debugger attach or a bookmarked URL keeps working; different worktrees
never overlap. - An explicitly passed port always wins, over both the derived worktree value and the default.
At the Maven level this already works — a command-line-Dtomcat.port=9090overrides the pinned
property. What is missing is a passthrough in thejustrecipes, which today take only a
collection or test name.
Deliberately out of scope: rewriting the Postman collections. dotcms-postman/pom.xml:34
carries the comment that some tests "currently fail unless docker external port is same as
internal", naming ApiToken_Resource — it holds a collection variable remoteTokenPort = 8080,
and it lives in the default group. Thirteen other collection files contain a literal :8080, but
most of those sit in descriptions or rawUrl fields rather than real requests, and
Experiments_Resource contains none. So this issue derives the ports and verifies which
collections actually break on a non-8080 host port; fixing any that do is a separate, scoped
follow-up. Until then -Dtomcat.port=8080 is the documented escape hatch.
Acceptance Criteria
Default path — no behavior change
- Run from the main checkout (not a worktree), each of
test-postman,test-integration,
test-karateandtest-e2euses the same host ports, container names and volume names as
today — no flags required, no observable difference.
Worktree path — deterministic isolation
- The derivation of
context.nameand the port offset lives in one place and is consumed by
every Docker-backed recipe. No module reimplements it. - Run from a git worktree, each of the four recipes completes without a port binding error and
without reusing another checkout's containers or volumes. - Every host port a recipe binds is derived from the worktree's stable key — Tomcat, Tomcat SSL,
management, WireMock, JDWP debug, Postgres, OpenSearch and os-upgrade. No fixed value survives
into the worktree path. - The derived values are deterministic: the same worktree produces the same ports and the
samecontext.nameon every run. - Two different worktrees run the same recipe at the same time and both complete; neither
reports a port conflict, a container-name clash, nor picks up the other's volumes. - A worktree recipe runs while
just dev-runis up in the main checkout, and neither disturbs
the other. - The resolved server URL and ports are printed at the start of the run, so a developer can
attach a debugger or open the instance without reading the Maven log.
Escape hatches and failure modes
- Each recipe accepts an explicit port — e.g.
just test-postman Experiments_Resource 9090—
and that value wins over both the derived worktree value and the default. - Explicit Maven flags still win too:
-Dtomcat.port,-Dcontext.name,-Ddebug.port,
-Dwiremock.port,-Ddb.port,-Des.porton the command line override the derived values.
(This already works at the Maven level today; the recipes must not break it.) - If a derived port is already taken, the run fails with a message naming the port and how to
override it — not a raw Docker bind error. -
just postman-stopandjust test-integration-stopstop only the containers belonging to the
current worktree, andcleanup-at-startdoes not remove another worktree's containers or
volumes.
Cleanup surfaced while scoping this
-
test-karate/pom.xml:23carries theApiToken_Resource.postman_collectioncomment
copy-pasted from the Postman module, but Karate runs no Postman collections. Remove or correct
it. -
core-web/apps/dotcms-ui-e2e/pom.xmldeclares<tomcat.port>8080</tomcat.port>twice in the
same<properties>block (lines 41 and 52). Collapse to one.
Verification and follow-up
- Verified end to end from a worktree for each recipe, including two worktrees running the same
recipe concurrently, and from the main checkout withjust dev-runup. - Each Postman collection group is run once on a derived (non-8080) port and the result
recorded, so we know exactly which groups depend on host port == container port. Any that
break get a separate follow-up issue rather than a fix here. - The behavior is documented where a developer will find it — the
justfilerecipe comments
plusdocs/testing/, including-Dtomcat.port=8080as the escape hatch for the affected
Postman groups.
Priority
Medium
Additional Context
-Pdebugis passed unconditionally by thetest-postmanrecipe, sodebug.port=5005is always
bound — a worktree run collides with any other debuggable dotCMS instance before Tomcat is even
reached.dev-runis the reference implementation: it leaves the fabric8 dynamic-port syntax alone and
therefore already survives a second checkout. The goal is for the test recipes to reach the same
place without losing determinism.- Related: #34722 (development environment setup — shared services, worktree integration).
environments/dev/user-dev.propertiesis gitignored and already supports per-developer overrides
with theext.prefix; it is a candidate location for a manual opt-out.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with parent/pom.xml and the justfile recipes for test-postman, test-integration, test-karate, and test-e2e; compare them with dotCMS/pom.xml and the module POM locations listed in the issue. Run the four recipes from the main checkout and a worktree to establish current ports and names. Done means deterministic isolated runs, explicit-port overrides, safe cleanup, printed endpoints, and documented behavior are verified across two concurrent worktrees.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, git, java
- Domain
- build-system, developer-experience, devops, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100