dotCMS / dotCMS/core

with-redis compose example sets DOT_DOT_PUBSUB_PROVIDER_OVERRIDE (double prefix) and likely never enables Redis pub/sub

Open
#36,804 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : Clustering OKR : Customer Support Team : Maintenance Type : Defect
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Problem Statement

docker/docker-compose-examples/with-redis sets the pub/sub provider using a doubled DOT_ prefix, which almost certainly resolves to a property nothing reads — meaning the example silently does not enable Redis pub/sub and both nodes stay on the default JDBCPubSubImpl:

# docker-compose-node-1.yml:48  and  docker-compose-node-2.yml:26
DOT_DOT_PUBSUB_PROVIDER_OVERRIDE: 'com.dotcms.dotpubsub.RedisPubSubImpl'

The property key is already DOT_PUBSUB_PROVIDER_OVERRIDE (DotPubSubProviderLocator.java:10), and:

  • Config.readEnvironmentVariables() collects env vars starting with DOT_ and stores them verbatim, with no prefix stripping (Config.java:321-329). So DOT_DOT_PUBSUB_PROVIDER_OVERRIDE becomes a property literally named DOT_DOT_PUBSUB_PROVIDER_OVERRIDE.
  • Config.envKey() returns keys that already begin with DOT_ unchanged (:332-343), so the lookup is for DOT_PUBSUB_PROVIDER_OVERRIDE — which was never set.
  • No entrypoint rewrites these: setenv.sh only assigns defaults to DOT_-prefixed env vars (it does not translate DOT_X-DX), and 10-import-env.sh is purely DB/asset import.

So the correct env form is the single-prefix DOT_PUBSUB_PROVIDER_OVERRIDE.

Why this matters beyond a docs typo. Spike #36544 found that the default JDBCPubSubImpl draws its permanently-held LISTEN connection from the shared JDBC pool, which caused a production startup crash-loop. RedisPubSubImpl is one of the recommended mitigations precisely because it avoids that pool. If this example never actually activated Redis pub/sub, then:

  1. anyone who followed it believes they are on Redis pub/sub while still exposed to #36544; and
  2. RedisPubSubImpl is far less exercised in the field than the presence of this example suggests.

Also worth noting: REDIS_SESSION_* is a different keyspace from REDIS_LETTUCECLIENT_URLS, so an install using Redis for session storage is not thereby using Redis for cluster notifications — a related and consequential misreading.

Status: derived from static analysis, not yet confirmed at runtime. All three plausible resolution paths were traced and none strip the extra prefix, but this has not been observed in a running container. Confirming that is the first acceptance criterion.

Steps to Reproduce

cd docker/docker-compose-examples/with-redis
docker-compose -f docker-compose-node-1.yml up

Then check which provider actually started:

docker compose logs | grep -E "PGListener listening|Running Listener Loop|RedisPubSub"

Expected: evidence of RedisPubSubImpl subscribing.
Actual (predicted): PGListener listening : cluster_actions and Running Listener Loop every 500ms from JDBCPubSubImpl — i.e. the Postgres provider, not Redis.

Contrast with the single-prefix form, which should select Redis:

DOT_PUBSUB_PROVIDER_OVERRIDE: 'com.dotcms.dotpubsub.RedisPubSubImpl'

Acceptance Criteria

  • Confirm at runtime which provider the current with-redis example actually starts, and record the finding on this issue.
  • If confirmed: correct both docker-compose-node-1.yml:48 and docker-compose-node-2.yml:26 to the single-prefix DOT_PUBSUB_PROVIDER_OVERRIDE, and verify from the logs that RedisPubSubImpl is selected.
  • Audit every docker-compose-examples/ file and the k8s manifest examples for other doubled-DOT_ env vars, since the same mistake is easy to repeat with any config key that natively begins with DOT_ (e.g. DOT_PUBSUB_USE_QUEUE).
  • Document the rule where developers will meet it: a config key already starting with DOT_ is used as-is as the env var name — do not add another DOT_ prefix. with-redis/README.md is a reasonable home, alongside the existing cluster docs.
  • Consider a startup log line stating which pub/sub provider was resolved, so a mis-set override is visible immediately rather than inferred from downstream listener logging. (DotPubSubProviderLocator currently logs nothing about its choice.)
  • Consider whether an unrecognised or ignored DOT_DOT_* env var should produce a startup warning — a misspelled override currently fails completely silently.

dotCMS Version

Present on main. The with-redis example has carried this form since Redis pub/sub was added to it; relevant to every version where JDBCPubSubImpl is the default (24.01.26 onward, plus LTS 22.03.14, 23.01.11, 23.10.24 v3).

Severity

Medium - Some functionality impacted

Links

  • Discovered while researching mitigations for spike #36544 — see docs/core/incidents/36544-pubsub-connection-churn.md (item 4 under "Needs measurement") on branch issue-36544-pubsub-connection-churn-spike
  • Parent epic: #34837
  • Affected files: docker/docker-compose-examples/with-redis/docker-compose-node-1.yml:48, docker-compose-node-2.yml:26
  • Resolution logic: Config.java:321-343, DotPubSubProviderLocator.java:10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in docker/docker-compose-examples/with-redis by running the provided compose command and checking the listed logs for the selected provider. Compare both compose files with Config.java and DotPubSubProviderLocator.java, then audit the compose and Kubernetes examples for doubled DOT_ variables. Done means the runtime behavior is recorded, affected configuration is corrected if confirmed, and the env-var rule is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, java, kubernetes, redis
Domain
devops, distributed-systems, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.