canonical / canonical/postgresql-operator

Cannot have backups when using s3-integrator 2/edge

Open
#1,043 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
20
Forks
36
Avg merge
4d 15h
Merged PRs (30d)
30

Description

## Steps to reproduce

```bash
juju deploy postgresql --channel 16/stable --constraints="cores=4 mem=4G"
juju deploy s3-integrator --base=ubuntu@24.04 --channel 2/edge --revision 167
secret=$(juju add-secret minio-postgres access-key=admin secret-key=password)
juju grant-secret minio-postgres s3-integrator
juju config s3-integrator \
endpoint="https://10.240.246.4:9000" \
bucket=foobar \
path=/postgresql \
s3-uri-style="path" \
s3-api-version="" \
region="" \
tls-ca-chain="$(base64 -w0 cert.pem)" \
credentials=$secret
juju integrate s3-integrator postgresql
```

## Expected behavior

Postgres detects the bucket that is pre-created in Minio, and then it writes the required files to setup a backup location.

## Actual behavior

The backup init is starting successfully but a warning message is thrown in the mid of the process. It seems that the postgres is setting a bucket name in the relation databag which is the relation id: `relation-17`. Then the s3-integrator is consuming it, and it is changing the bucket location. This bucket does not exist in the s3 location, so backup init is stuck.

## Versions

Operating system: Ubuntu 24.04

Juju CLI: 3.6.8-genericlinux-amd64

Juju agent: 3.6.8

Charm revision: 843

LXD: 5.21.3 LTS

## Log output

Juju debug log:

```
unit-s3-integrator-1: 08:28:23 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:29:14 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:30:18 INFO juju.worker.uniter.operation ran "s3-credentials-relation-created" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:30:19 INFO juju.worker.uniter.operation ran "s3-credentials-relation-joined" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:30:19 INFO unit.s3-integrator/1.juju-log s3-credentials:17: On storage-connection-info-requested
unit-postgresql-1: 08:30:20 INFO juju.worker.uniter.operation ran "s3-parameters-relation-created" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:30:20 INFO juju.worker.uniter.operation ran "s3-credentials-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-1: 08:30:39 INFO unit.postgresql/1.juju-log s3-parameters:17: backups:Bucket foobar exists.
unit-postgresql-1: 08:30:40 INFO unit.postgresql/1.juju-log s3-parameters:17: backups:Reading content from bucket=foobar, path=postgresql/model-uuid.txt
unit-postgresql-1: 08:30:40 INFO unit.postgresql/1.juju-log s3-parameters:17: backups:No such object to read from S3 bucket=foobar, path=postgresql/model-uuid.txt
unit-postgresql-1: 08:31:00 INFO unit.postgresql/1.juju-log s3-parameters:17: backups:Uploading content to bucket=foobar, path=postgresql/model-uuid.txt
unit-postgresql-1: 08:31:01 INFO juju.worker.uniter.operation ran "s3-parameters-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-1: 08:31:02 INFO unit.postgresql/1.juju-log s3-parameters:17: root:Starting rotate logs process
unit-postgresql-1: 08:31:02 INFO unit.postgresql/1.juju-log s3-parameters:17: root:Started rotate logs process with PID 26749
unit-postgresql-1: 08:31:03 INFO juju.worker.uniter.operation ran "s3-parameters-relation-joined" hook (via hook dispatching script: dispatch)
unit-s3-integrator-1: 08:31:03 INFO unit.s3-integrator/1.juju-log s3-credentials:17: On storage-connection-info-requested
unit-s3-integrator-1: 08:31:04 INFO juju.worker.uniter.operation ran "s3-credentials-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-1: 08:31:23 WARNING unit.postgresql/1.juju-log s3-parameters:17: backups:Bucket relation-17 doesn't exist or you don't have access to it.
unit-postgresql-1: 08:31:24 INFO juju.worker.uniter.operation ran "s3-parameters-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-1: 08:31:48 WARNING unit.postgresql/1.juju-log s3-parameters:17: backups:Bucket relation-17 doesn't exist or you don't have access to it.
unit-postgresql-1: 08:31:49 INFO juju.worker.uniter.operation ran "s3-parameters-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-1: 08:32:08 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch)
```

## Additional context

```
ubuntu@maas-bastion:/root$ jhack show-relation s3-integrator postgresql
relation data v0.6
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ relation (id: 17) ┃ s3-integrator ┃ postgresql ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ type │ regular │ = │
│ interface │ s3 │ = │
│ model │ the current model │ = │
│ relation ID │ 17 │ = │
│ role │ provider │ requirer │
│ endpoint │ s3-credentials │ s3-parameters │
│ leader unit │ 1 │ 1 │
├───────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ application data │ ╭──────────────────────────────────────────────────────────────────────────────────────────╮ │ ╭──────────────────────────────────────────────────────────────────────────────────────────╮ │
│ │ │ │ │ │ │ │
│ │ │ access-key admin │ │ │ bucket relation-17 │ │
│ │ │ bucket relation-17 │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │ │ endpoint https://10.240.246.4:9000 │ │ │
│ │ │ path /postgresql │ │ │
│ │ │ s3-uri-style path │ │ │
│ │ │ secret-key password │ │ │
│ │ │ tls-ca-chain ["-----BEGIN │ │ │
│ │ │ CERTIFICATE-----\nMIIFkzCCA3ugAwIBAgIUU7AVlPx6DAsIVHkHEwYaR6KdqNIwDQYJK… │ │ │
│ │ │ CERTIFICATE-----", "-----BEGIN │ │ │
│ │ │ CERTIFICATE-----\nMIIEkzCCAnugAwIBAgIUZ05+9Kt3AmDhQRbBrgFfXDvxHQswDQYJK… │ │ │
│ │ │ CERTIFICATE-----"] │ │ │
│ │ ╰──────────────────────────────────────────────────────────────────────────────────────────╯ │ │
│ unit data │ ╭─ s3-integrator/1* ─╮ │ ╭─ postgresql/1* ─╮ │
│ │ │ │ │ │ │ │
│ │ ╰────────────────────╯ │ ╰─────────────────╯ │
└───────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────┘
```

```bash
mc ls --insecure myminio/foobar/postgresql
[2025-07-14 08:31:00 UTC] 36B STANDARD model-uuid.txt
[2025-07-14 08:39:48 UTC] 0B archive/
[2025-07-14 08:39:48 UTC] 0B backup/
```

- If I use 1/stable channel of s3-integrator, postgres is also setting the relation databag, but it is simply ignored. I attach a screenshot from an experiment I conducted.

- This behavior to set the relation id in the databag is part of the s3 relation library: https://github.com/canonical/data-platform-libs/blob/main/lib/charms/data_platform_libs/v0/s3.py#L666

Image

Contributor guide

Open the contributing guide

Research direction

Reproduce the deployment using the listed Juju commands and inspect the s3-parameters and s3-credentials relation events in the log. Compare the 2/edge behavior with 1/stable, focusing on why the relation data changes the bucket to relation-17. Done means backup initialization continues using the configured foobar bucket and creates the expected PostgreSQL backup paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.