zalando / zalando/postgres-operator
Error in postgres-cluster Upgrade - error while syncing cluster state: could not sync statefulsets:
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 3
Description
Hi Team,
We are trying to perform upgrade for the postgres cluster . Below is the operator image and the cluster image being used in deployment:
operator tag: v1.5.0-44-gab95eaa-dirty
cluster Image: docker-registry-proxy.corp.amdocs.com/acid/spilo-13:2.0-p4
When the upgrade is performed we could see, Out of 2 cluster pods only one pod is terminated and restarted and the below error:
ERROR
time="2021-03-02T11:25:58Z" level=warning msg="error while syncing cluster state: could not sync statefulsets: could not recreate pods: postpone pod recreation until next Sync: recreation is unsafe because pods are being initialized" cluster-name=nts-dev-ocp-install-neelasha1/postgres-operator-cluster pkg=cluster
time="2021-03-02T11:25:58Z" level=error msg="could not sync cluster: could not sync statefulsets: could not recreate pods: postpone pod recreation until next Sync: recreation is unsafe because pods are being initialized" cluster-name=nts-dev-ocp-install-neelasha1/postgres-operator-cluster pkg=controller worker=0
time="2021-03-02T11:25:58Z" level=info msg="Event(v1.ObjectReference{Kind:\"postgresql\", Namespace:\"nts-dev-ocp-install-neelasha1\", Name:\"postgres-operator-cluster\", UID:\"b511b98e-1507-4bed-b3c2-aa3270c9e56e\", APIVersion:\"acid.zalan.do/v1\", ResourceVersion:\"640594780\", FieldPath:\"\"}): type: 'Warning' reason: 'Sync' could not sync cluster: could not sync statefulsets: could not recreate pods: postpone pod recreation until next Sync: recreation is unsafe because pods are being initialized"
Attached is the custom values used for postgres-operator and postgres-cluster deployment.
Could you please let us know how do we overcome this error during upgrade or this error is expected ?
Please note: After resync, the cluster is up and running fine .
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the attached postgres-cluster.yaml.txt and postgres-operator.yaml.txt files alongside the reported synchronization logs. Reproduce the upgrade and inspect the operator's statefulset and pod-recreation handling; the issue does not identify a source file, test, or agreed fix, so completion criteria are not defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, postgresql
- Domain
- databases, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100