zalando / zalando/postgres-operator

Problems setting up backups to s3

Open
#1,965 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
5.2k
Forks
1.1k
Avg merge
2d 16h
Merged PRs (30d)
3

Description

Please, answer some short questions which should help us to understand your problem / question better?

  • **Which image of the operator are you using? registry.opensource.zalan.do/acid/postgres-operator:v1.8.1
  • **Where do you run it - cloud or metal? Kubernetes or OpenShift? Bare Metal K8s
  • **Are you running Postgres Operator in production? no
  • **Type of issue? question

I am attempting to set up backups/wal archiving to an AWS s3 bucket but I cannot get it to work.

The current problem is that the wal-e env directlry does not seem to be getting set up:

envdir "/run/etc/wal-e.d/env" wal-g backup-list
chpst: fatal: unable to switch to directory: /run/etc/wal-e.d/env: file does not exist

I have been adding config values one at a time starting from the ones listed in the docs here:
Using AWS S3 or compliant services
Just adding those ones does not seem to do anything.

I have used a configmap and also set values in the helm chart:

apiVersion: v1
kind: ConfigMap
metadata:
  name: pod-env-overrides
  namespace: postgres-operator
data:
  # Any env variable used by spilo can be added
  AWS_REGION: eu-west-1
  kube_iam_role: pg-backup-role
  wal_s3_bucket: pg-backup-bucket
  AWS_ACCESS_KEY_ID: [VALUE]
  AWS_SECRET_ACCESS_KEY: [VALUE]
  additional_secret_mount: aws-bucket-key
  additional_secret_mount_path: /var/secrets/aws
  BACKUP_NUM_TO_RETAIN: "30"
  BACKUP_SCHEDULE: 0 02 * * *
  USE_WALG_BACKUP: "false"
  USE_WALG_RESTORE: "false"
  WAL_BUCKET_SCOPE_PREFIX: ""
  AWS_S3_FORCE_PATH_STYLE: "true"
  WALG_DISABLE_S3_SSE: "true"
  # STANDBY
  STANDBY_additional_secret_mount: aws-bucket-key
  STANDBY_additional_secret_mount_path: /var/secrets/aws
  STANDBY_AWS_REGION: eu-west-1
  STANDBY_AWS_ACCESS_KEY_ID: [VALUE]
  STANDBY_AWS_SECRET_ACCESS_KEY: [VALUE]
  STANDBY_WAL_S3_BUCKET: pg-backup-bucket 

And the relevant section in the helm release yaml:

configAwsOrGcp:
      AWS_REGION: eu-west-1
      kube_iam_role: pg-backup-role
      wal_s3_bucket: pg-backup-bucket
      additional_secret_mount: aws-bucket-key
      additional_secret_mount_path: /var/secrets/aws
      STANDBY_AWS_REGION: eu-west-1
      STANDBY_AWS_ACCESS_KEY_ID: [VALUE]
      STANDBY_AWS_SECRET_ACCESS_KEY: [VALUE]

It seems to be populating the env variables on the pods but not setting up that env directory or trying to run wal-e/g

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the “Using AWS S3 or compliant services” administrator documentation and compare its settings with the pod-env-overrides ConfigMap and Helm configAwsOrGcp values shown here. Reproduce the envdir "/run/etc/wal-e.d/env" wal-g backup-list failure and trace why the directory is absent; done means identifying the required configuration or documenting that the setup is unsupported.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, postgresql
Domain
cloud, database, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.