zalando / zalando/postgres-operator

Postgres operator 1.8.0: Pooler schema does not appear to be created on new databases

Open
#1,913 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
5.2k
Forks
1.1k
Avg merge
2d 16h
Merged PRs (30d)
3

Description

Please, answer some short questions which should help us to understand your problem / question better?

  • Which image of the operator are you using? e.g. registry.opensource.zalan.do/acid/postgres-operator:v1.8.1
    im using: registry.opensource.zalan.do/acid/postgres-operator:v1.8.0
    installed the postgres operator with the helm chart.

  • Where do you run it - cloud or metal? Kubernetes or OpenShift? [AWS K8s | GCP ... | Bare Metal K8s]
    Kubernetes on a k3d cluster

  • Are you running Postgres Operator in production? [yes | no]
    no

  • Type of issue? [Bug report, question, feature request, etc.]
    bug report

  • Description

I have created a database cluster using the "postgresql.acid.zalan.do/v1" ressource, and enabled the connection pooler using the "enableConnectionPooler" property, the database cluster is then created, and I can connect using the pooler service to the preexisting default "postgres" database.

The issue seems to arise when I create a new database in the database cluster using the postgres user rather than the prepared databases approach. When I try to connect to the new database using psql, then an error is returned: (where dbhost is the pooler service and dbname is the newly created database.)
psql -U 'dbname' -h dbhost sslmode=require
psql: error: FATAL: bouncer config error

The log of the pooler also contains the same error: ( someIP contains an IP, but I have excluded it here. )
2022-06-01 08:06:12.810 UTC [1] LOG C-0x55954b4c0b50: (nodb)/(nouser)@someIP:39712 registered new auto-database: db=dbname
2022-06-01 08:06:22.826 UTC [1] WARNING C-0x55954b4c0b50: dbname/(nouser)@someIP:39712 pooler error: bouncer config error

When I then look in the postgres log, for the database I tried to connect to, then there is the following entry: (SQL query for the log was: SELECT message, log_time, user_name, connection_from FROM public.postgres_log where database_name = 'dbname' ORDER BY log_time desc; )
message: schema "pooler" does not exist
log_time: 2022-06-01 08:06:22.826+00
user_name: pooler
(excluded the connection_from)

Looking at the database I created, it indeed does not have the pooler schema. I tried to wait for the resync, but that does not appear to add the schema to the database, I can also find no error logs in the postgres operator.

However, I did find one way in which I could have the postgres operator install the schema, and that was to restart the postgres operator. After restarting the operator, and seeing that it had created the schema, then I tried another login using psql, which went through.

The postgres operator also logs that it is installing the lookup function in the database, after it has been restarted: (dbClusterName is the excluded database clustername)

time="2022-06-01T10:37:50Z" level=info msg="install pooler lookup function into database 'dbname'" cluster-name=postgresdb/dbClusterName pkg=cluster
time="2022-06-01T10:37:50Z" level=debug msg="closing database connection" cluster-name=postgresdb/dbClusterName pkg=cluster

I did find an older issue, that appears to be similar in the sense that the pooler schema is missing, but the issue appears to be closed, and the release note for 1.7.0 indicates that the issue is fixed: https://newreleases.io/project/github/zalando/postgres-operator/release/v1.7.0
https://github.com/zalando/postgres-operator/issues/1060

Let me know, if there is more information that you'd like.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the report on a Kubernetes cluster: create a database through the PostgreSQL resource with the connection pooler enabled, then connect to the new database through the pooler service. Trace the operator's pooler schema installation and resync behavior; done means the schema is installed without restarting the operator and the connection succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, helm, kubernetes, postgresql
Domain
database, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.