zalando / zalando/postgres-operator
Allow cloning with s3 credentials from secret
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 3
Description
- Which image of the operator are you using?: registry.opensource.zalan.do/acid/postgres-operator:v1.10.0
- Where do you run it - cloud or metal? Kubernetes or OpenShift? OKD (OpenShift) on bare metal UPI.
- Are you running Postgres Operator in production? yes
- Type of issue? feature request/bug report
I have a custom s3 endpoint and s3 creds that I have in secrets.
Unfortunately only the following spec works correctly. Note that the credentials are in plain text and the configuration is duplicated. The spec below (without it) does not find any backups (because it cannot access the bucket).
kind: postgresql
metadata:
annotations:
labels:
team: team
name: team-recovery
spec:
clone:
cluster: team-production
uid: ...
timestamp: 2023-06-21T11:30:00+02:00
s3_endpoint: https://s3.example.com
s3_wal_path: s3://a-bucket-id/spilo/db-team-production/wal/14
s3_force_path_style: true
s3_access_key_id: ...
s3_secret_access_key: ...
env:
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: WAL_S3_BUCKET
value: a-bucket-id
- name: WALG_S3_BUCKET
value: a-bucket-id
- name: AWS_ENDPOINT
value: https://s3.example.com
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: AWS_SECRET_ACCESS_KEY
name: team-production-wal
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: AWS_ACCESS_KEY_ID
name: team-production-wal
numberOfInstances: 1
kind: postgresql
metadata:
annotations:
labels:
team: team
name: team-recovery
spec:
clone:
cluster: team-production
uid: ...
timestamp: 2023-06-21T11:30:00+02:00
s3_endpoint: https://s3.example.com
s3_wal_path: s3://a-bucket-id/spilo/db-team-production/wal/14
env:
- name: USE_WALG_BACKUP
value: "true"
- name: USE_WALG_RESTORE
value: "true"
- name: WAL_S3_BUCKET
value: a-bucket-id
- name: WALG_S3_BUCKET
value: a-bucket-id
- name: AWS_ENDPOINT
value: https://s3.example.com
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: AWS_SECRET_ACCESS_KEY
name: team-production-wal
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: AWS_ACCESS_KEY_ID
name: team-production-wal
numberOfInstances: 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the postgresql spec's clone fields and the env entries using valueFrom secretKeyRef, then trace how clone configuration reaches the backup and restore process. Done means cloning can use the referenced S3 credentials without duplicating them as plaintext clone fields, including the custom endpoint and bucket shown in the examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes, postgresql
- Domain
- databases, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100