zalando / zalando/postgres-operator
Replica in archive recovery mode
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 3
Description
This morning, for the first time, and an unnown reason, I saw this log:
2025-10-16 06:01:10,984 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:20,972 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:30,977 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:41,011 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:50,975 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:01,053 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:11,008 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:20,977 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:30,978 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:40,982 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:51,482 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:03:01,482 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:03:11,487 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311174984}, "timeline": 52, "replication": [{"usename": "standby", "application_name": "entre-vos-mains--de-pg-0", "client_addr": "100.64.29.244", "state": "streaming", "sync_state": "async", "sync_priority": 0}], "dcs_last_seen": 1760594590, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}
2025-10-16 06:03:11,489 WARNING: Primary (entre-vos-mains--de-pg-1) is still alive
server signaled
2025-10-16 06:03:11,500 INFO: following a different leader because i am not the healthiest node
2025-10-16 06:03:20,987 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311174984}, "timeline": 52, "dcs_last_seen": 1760594600, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}
2025-10-16 06:03:20,988 WARNING: Primary (entre-vos-mains--de-pg-1) is still alive
2025-10-16 06:03:21,041 INFO: following a different leader because i am not the healthiest node
2025-10-16 06:03:30,985 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311252208}, "timeline": 52, "dcs_last_seen": 1760594610, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}
Patronictl list gave for the replica:
in archive recovery
I wonder what happened.
restarting the pod worked.
I noticed it with increase cpu usage on our object storage ingress.
I think it would be nice to show the CRD status to something like degraded in this case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the operator observes Patroni's "in archive recovery" state and updates the cluster CRD status; the issue names no source files or tests. Reproduce or inspect the reported log sequence, then determine what status should be exposed when a replica remains in that state and add coverage for the observed transition.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, postgresql
- Domain
- databases, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100