zalando / zalando/postgres-operator
How to disable BACKUP_NUM_TO_RETAIN while keeping scheduled base backups?
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 3
Description
Hi!
We are currently using the scheduled basebackup mechanism with BACKUP_NUM_TO_RETAIN as described here: https://postgres-operator.readthedocs.io/en/latest/administrator/#wal-archiving-and-physical-basebackups.
We are running into some issues with the built-in backup cleanup logic and are evaluating an alternative approach. Based on the comment below, we are considering delegating retention entirely to S3 lifecycle rules:
Number of backups is set by BACKUP_NUM_TO_RETAIN. Spilo sets it to 5 by default, no? You can override it in the manifest in the env section of a cluster. You can also define lifecycle rules on S3 buckets. This is how we clean up old Postgres log files for example.
— @FxKu in #2942
Setting BACKUP_NUM_TO_RETAIN to a very large number works as a workaround, but it isn't ideal.
Setting a very large BACKUP_NUM_TO_RETAIN value technically works but feels like a workaround. Is there a proper way to completely disable Spilo's cleanup logic while still keeping the scheduled basebackup feature enabled?
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the WAL archiving and physical basebackups documentation linked in the issue, then trace BACKUP_NUM_TO_RETAIN and the scheduled basebackup mechanism in the operator and Spilo configuration. Done means documenting or identifying a supported way to keep scheduled base backups enabled while delegating retention to S3 lifecycle rules.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, postgresql
- Domain
- cloud, databases, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100