Docker Swarm Deployment with secrets results in challenges with server and worker connecting to postgres

**Describe your question/**
I am attempting to deploy a working authentik solution using docker compose, docker secrets in swarm mode

**Relevant info**
Docker running in swarm mode
the postgres password and authentik key are both docker secrets. I'm deploying the stack via portainer

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Logs**
output from worker/server

INF | event=PostgreSQL connection failed, retrying... (connection failed: connection to server at "10.0.1.137", port 5432 failed: FATAL: password authentication failed for user "authentik") logger=authentik.lib.config timestamp=1740367506.2970626
INF | event=PostgreSQL connection failed, retrying... (connection failed: connection to server at "10.0.1.137", port 5432 failed: FATAL: password authentication failed for user "authentik") logger=authentik.lib.config timestamp=1740367507.3094716

postgres i was seeing this (so there was connectivity)

2025-02-24 01:42:21.439 UTC [103] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:21.439 UTC [103] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"
2025-02-24 01:42:22.451 UTC [104] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:22.451 UTC [104] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"
2025-02-24 01:42:23.467 UTC [105] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:23.467 UTC [105] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"

the password is correct in /var/run/secrets but somehow its not working properly between the worker/server and postgres its self.

docker secrets output

jdunn@container-manager:~$ sudo docker secret ls
ID NAME DRIVER CREATED UPDATED
liaullk9vfcxh4x39la49hf7n AUTHENTIK_PG_PASS 12 seconds ago 12 seconds ago
btuzygv0avtttyiioek6zqkwd AUTHENTIK_SECRET_KEY 7 days ago 7 days ago

**Version and Deployment (please complete the following information):**

version: "3.8"

services:
postgresql-authentik:
image: docker.io/library/postgres:16-alpine
deploy:
replicas: 1
restart_policy:
condition: on-failure
healthcheck:
test: ["CMD-SHELL", "pg_isready -d authentik -U authentik"]
interval: 30s
retries: 5
start_period: 30s
timeout: 5s
volumes:
- database:/var/lib/postgresql/data
secrets:
- AUTHENTIK_PG_PASS
environment:
POSTGRES_PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
POSTGRES_USER: authentik
POSTGRES_DB: authentik
networks:
- traefik-overlay

redis:
image: docker.io/library/redis:alpine
command: --save 60 1 --loglevel warning
deploy:
replicas: 1
restart_policy:
condition: on-failure
healthcheck:
test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
interval: 30s
retries: 5
start_period: 20s
timeout: 3s
volumes:
- redis:/data
networks:
- traefik-overlay

server:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.12.3}
deploy:
replicas: 1
restart_policy:
condition: on-failure
labels:
- "traefik.enable=true"
- "traefik.http.routers.authentik.rule=Host(`auth.dunnnetwork.com`)"
- "traefik.http.routers.authentik.entrypoints=websecure"
- "traefik.http.routers.authentik.tls=true"
- "traefik.http.routers.authentik.tls.certresolver=myresolver"
- "traefik.http.routers.authentik.tls.domains[0].main=dunnnetwork.com"
- "traefik.http.routers.authentik.tls.domains[0].sans=*.dunnnetwork.com"
- "homepage.group=Utilities"
- "homepage.name=Authentik"
- "homepage.icon=authentik.svg"
- "homepage.href=https://auth.dunnnetwork.com"
- "homepage.description=Authentication Manager"
command: server
secrets:
- AUTHENTIK_SECRET_KEY
- AUTHENTIK_PG_PASS
environment:
AUTHENTIK_REDIS__HOST: "tasks.redis"
AUTHENTIK_POSTGRESQL__HOST: "tasks.postgresql-authentik"
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
AUTHENTIK_SECRET_KEY: "/run/secrets/AUTHENTIK_SECRET_KEY"
volumes:
- media:/media
- certs:/certs
- templates:/templates
networks:
- traefik-overlay

worker:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.12.3}
deploy:
replicas: 1
restart_policy:
condition: on-failure
command: worker
secrets:
- AUTHENTIK_SECRET_KEY
- AUTHENTIK_PG_PASS
environment:
AUTHENTIK_REDIS__HOST: "tasks.redis"
AUTHENTIK_POSTGRESQL__HOST: "tasks.postgresql-authentik"
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
AUTHENTIK_SECRET_KEY: "/run/secrets/AUTHENTIK_SECRET_KEY"
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- media:/media
- certs:/certs
- templates:/templates
networks:
- traefik-overlay

volumes:
database:
driver: local
redis:
driver: local
media:
driver: local
certs:
driver: local
templates:
driver: local

networks:
traefik-overlay:
external: true

secrets:
AUTHENTIK_SECRET_KEY:
external: true
AUTHENTIK_PG_PASS:
external: true

**Additional context**
Add any other context about the problem here.