hapostgres / hapostgres/pg_auto_failover
pg_auto_failover tunes postgres parameters based on the VM size instead of the container resources
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 142
- Avg merge
- 5h 8m
- Merged PRs (30d)
- 1
Description
**Setup:**
We are running postgres server in containers on Kubernetes cluster.
**Issue:**
While looking at the pg_auto_failover debug logs, found an interesting log snippet related to pgtuning:
```
17:15:05 105 DEBUG pgtuning.c:83 Detected 8 CPUs and 29 GB total RAM on this server
17:15:05 105 DEBUG pgtuning.c:223 Setting autovacuum_max_workers to 3
17:15:05 105 DEBUG pgtuning.c:228 Setting shared_buffers to 7523 MB
17:15:05 105 DEBUG pgtuning.c:231 Setting work_mem to 24 MB
17:15:05 105 DEBUG pgtuning.c:235 Setting maintenance_work_mem to 512 MB
17:15:05 105 DEBUG pgtuning.c:239 Setting effective_cache_size to 22 GB
17:15:05 105 INFO pgctl.c:923 Contents of "/pgsql/data/postgresql-auto-failover.conf" have changed, overwriting
17:15:05 105 DEBUG pgctl.c:940 Wrote file "/pgsql/data/postgresql-auto-failover.conf" with content:
# Settings by pg_auto_failover
shared_preload_libraries = pg_stat_statements
listen_addresses = '*'
port = 5432
max_wal_senders = 12
max_replication_slots = 12
wal_level = 'replica'
wal_log_hints = on
wal_sender_timeout = '30s'
hot_standby_feedback = on
hot_standby = on
synchronous_commit = on
logging_collector = on
log_destination = stderr
log_directory = log
log_min_messages = info
log_connections = off
log_disconnections = off
log_lock_waits = on
password_encryption = 'scram-sha-256'
ssl = on
ssl_ca_file = '/etc/postgres_ssl/..2023_02_09_17_14_46.186169050/ca.crt'
ssl_cert_file = '/etc/postgres_ssl/..2023_02_09_17_14_46.186169050/tls.crt'
ssl_key_file = '/etc/postgres_ssl/..2023_02_09_17_14_46.186169050/tls.key'
ssl_ciphers = 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384'
wal_keep_size = '8 GB'
# basic tuning computed by pg_auto_failover
track_functions = pl
shared_buffers = '7523 MB'
work_mem = '24 MB'
maintenance_work_mem = '512 MB'
effective_cache_size = '22 GB'
autovacuum_max_workers = 3
autovacuum_vacuum_scale_factor = 0.08
autovacuum_analyze_scale_factor = 0.02
```
The container itself is restricted to use only 560MB whereas the underlying node/VM this Kubernetes pod/container is scheduled on has a capacity of 30GB.
Looks like pgaf is reading the VM resources (sysinfo.totalmem) instead of getting the container resources.
This could lead to issues where postgres is trying to use more resources than what's allocated to the container, thus, leading the container to be killed due to out-of-memory.
**Expected:**
pg_auto_failover should have used container resources to compute and not the underlying VM resources.
cc @SMiller70
Contributor guide
Assessment
This issue has not been assessed yet.