cloudnative-pg / cloudnative-pg/plugin-barman-cloud
need a proper way to change the bucket
- Dominant language
- Go
- Stars
- 191
- Forks
- 72
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 21
Description
Good day!
I tried to make the next experiment.
I had a working ObjectStore with a proper schedule.
```yaml
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: keycloakdb-daily
namespace: keycloak
spec:
backupOwnerReference: self
cluster:
name: keycloakdb
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
schedule: 0 2 * * *
```
```yaml
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: keycloak
namespace: keycloak
spec:
configuration:
data:
compression: gzip
jobs: 2
destinationPath: s3://[mybucketname]/
endpointURL: https://storage.yandexcloud.net
s3Credentials:
accessKeyId:
key: AWS_ACCESS_KEY_ID
name: s3-yandex
secretAccessKey:
key: AWS_SECRET_ACCESS_KEY
name: s3-yandex
wal:
compression: zstd
maxParallel: 8
```
Now I want to change:
- bucketname to another one
- and appropriately change the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
So I naively changing the content of the `s3-yandex` secret and apply the corrected `ObjectStore`. But.... nothing happens.
I see the next in the logs of barman:
```
{"level":"info","ts":"2025-06-13T11:30:07.452298887Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T11:35:11.141188431Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T11:40:14.628910791Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T11:45:18.580604564Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T11:50:21.684530075Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T11:55:13.513036839Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000AF","options":["--zstd","--endpoint-url","https://storage.yandexcloud.net","--cloud-provider","aws-s3","s3://testcloudnative/","keycloakdb","/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000AF"]}
{"level":"info","ts":"2025-06-13T11:55:16.342916796Z","msg":"Archived WAL file","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000AF","startTime":"2025-06-13T11:55:13.512903992Z","endTime":"2025-06-13T11:55:16.34287209Z","elapsedWalTime":2.829968224}
{"level":"info","ts":"2025-06-13T11:55:25.139734503Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:00:13.96310529Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B0","options":["--zstd","--endpoint-url","https://storage.yandexcloud.net","--cloud-provider","aws-s3","s3://testcloudnative/","keycloakdb","/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B0"]}
{"level":"info","ts":"2025-06-13T12:00:16.812934587Z","msg":"Archived WAL file","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B0","startTime":"2025-06-13T12:00:13.963073964Z","endTime":"2025-06-13T12:00:16.812896472Z","elapsedWalTime":2.849822661}
{"level":"info","ts":"2025-06-13T12:00:28.490192702Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:05:13.279134066Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B1","options":["--zstd","--endpoint-url","https://storage.yandexcloud.net","--cloud-provider","aws-s3","s3://testcloudnative/","keycloakdb","/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B1"]}
{"level":"info","ts":"2025-06-13T12:05:15.905317755Z","msg":"Archived WAL file","logging_pod":"keycloakdb-1","walName":"/var/lib/postgresql/data/pgdata/pg_wal/0000000100000003000000B1","startTime":"2025-06-13T12:05:13.278914875Z","endTime":"2025-06-13T12:05:15.905254787Z","elapsedWalTime":2.62633958}
{"level":"info","ts":"2025-06-13T12:05:32.282742702Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:10:35.701574771Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:15:38.925811524Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:20:42.222088847Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:20:44.612536625Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250612230200"}
{"level":"info","ts":"2025-06-13T12:20:44.620135745Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613000200"}
{"level":"info","ts":"2025-06-13T12:20:44.626424912Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613010200"}
{"level":"info","ts":"2025-06-13T12:20:44.633155459Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613020200"}
{"level":"info","ts":"2025-06-13T12:20:44.638740297Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613030200"}
{"level":"info","ts":"2025-06-13T12:20:44.646144456Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613040200"}
{"level":"info","ts":"2025-06-13T12:20:44.652175408Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613050200"}
{"level":"info","ts":"2025-06-13T12:20:44.659974535Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613060200"}
{"level":"info","ts":"2025-06-13T12:20:44.665997349Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613070200"}
{"level":"info","ts":"2025-06-13T12:20:44.671859722Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613080200"}
{"level":"info","ts":"2025-06-13T12:20:44.678399927Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613090200"}
{"level":"info","ts":"2025-06-13T12:20:44.685202587Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613100200"}
{"level":"info","ts":"2025-06-13T12:20:44.692265438Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613110200"}
{"level":"info","ts":"2025-06-13T12:20:44.699563098Z","msg":"Deleting backup not in the catalog","logging_pod":"keycloakdb-1","backup":"keycloakdb-daily-20250613120200"}
{"level":"info","ts":"2025-06-13T12:25:44.754412291Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:30:47.517659989Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
{"level":"info","ts":"2025-06-13T12:35:50.152164082Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"keycloakdb-1"}
```
My expectations: that the backups would be streamed to a new bucket.
Contributor guide
Research direction
Start with the ScheduledBackup and ObjectStore manifests in the issue, then compare the applied ObjectStore and secret values with the barman-cloud WAL archive command shown in the logs. Determine how an ObjectStore destination and credentials are refreshed, and verify completion by observing WAL archives use the new bucket and credentials.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, postgresql
- Domain
- databases, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100