zalando / zalando/postgres-operator
Sidecar without image specification causes all PostgreSQL pods to shutdown
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 3
Description
- Which image of the operator are you using? ghcr.io/zalando/postgres-operator:v1.14.0
- Where do you run it - cloud or metal? Kubernetes or OpenShift? Bare Metal K8s
- Are you running Postgres Operator in production? yes
- Type of issue? Bug report
It's possible to add a sidecar specification without providing any image (by mistake) for that additional container.
This leads to a cascading failure where all cluster instances are shut down sequentially during sync intervals (30 minutes) until all pods are terminated, resulting in complete PostgreSQL unavailability.
New pods cannot start due to the Kubernetes error: "create Pod ... in StatefulSet ... failed error: Pod ... is invalid: spec.containers[2].image: Required value"
The operator logs show this -
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"performing rolling update","pkg":"cluster","time":"2025-10-20T09:10:42Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"there are 2 pods in the cluster to recreate","pkg":"cluster","time":"2025-10-20T09:10:42Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"subscribing to pod \"dev-mdb-mq-clusters/test-no-image-1\"","pkg":"cluster","time":"2025-10-20T09:10:42Z","worker":2}
...
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"unsubscribing from pod \"dev-mdb-mq-clusters/test-no-image-1\" events","pkg":"cluster","time":"2025-10-20T09:20:52Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"error","msg":"could not sync statefulsets: could not recreate pods: could not recreate replica pod \"dev-mdb-mq-clusters/test-no-image-1\": pod label wait timeout","pkg":"cluster","time":"2025-10-20T09:20:52Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"syncing roles","pkg":"cluster","time":"2025-10-20T09:20:52Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"db ready status: true","pkg":"cluster","time":"2025-10-20T09:20:52Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"closing database connection","pkg":"cluster","time":"2025-10-20T09:20:52Z","worker":2}
Next operator sync kills last available pod -
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"found rolling update flag on pod \"test-no-image-0\"","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"1 / 1 pod(s) still need to be rotated","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"performing rolling update","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"there are 1 pods in the cluster to recreate","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"warning","msg":"cannot perform switch over before re-creating the pod: no replicas","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"info","msg":"recreating old master pod \"dev-mdb-mq-clusters/test-no-image-0\"","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"subscribing to pod \"dev-mdb-mq-clusters/test-no-image-0\"","pkg":"cluster","time":"2025-10-20T09:36:13Z","worker":2}
...
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"debug","msg":"unsubscribing from pod \"dev-mdb-mq-clusters/test-no-image-0\" events","pkg":"cluster","time":"2025-10-20T09:46:24Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"warning","msg":"error while syncing cluster state: could not sync statefulsets: could not recreate pods: could not recreate old master pod \"dev-mdb-mq-clusters/test-no-image-0\": pod label wait timeout","pkg":"cluster","time":"2025-10-20T09:46:24Z","worker":2}
{"cluster-name":"dev-mdb-mq-clusters/test-no-image","level":"error","msg":"could not sync cluster: could not sync statefulsets: could not recreate pods: could not recreate old master pod \"dev-mdb-mq-clusters/test-no-image-0\": pod label wait timeout","pkg":"controller","time":"2025-10-20T09:46:25Z","worker":2}
The operator should validate the StatefulSet configuration before triggering updates.
At minimum, the sidecar image field should be marked as required in the CRD validation schema.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the CRD validation schema and the StatefulSet configuration or update path mentioned in the report. Reproduce a sidecar with no image, verify that validation rejects it before a rolling update, and add or update coverage for the invalid configuration and the resulting safe behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, postgresql
- Domain
- databases, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100