canonical / canonical/postgresql-operator

pg_exporter_last_scrape_error returns 1 on replica unit of the standby cluster when async-replication is enabled

Open
#1,290 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
20
Forks
36
Avg merge
4d 15h
Merged PRs (30d)
30

Description

## Steps to reproduce
- 2 Postgresql clusters of 2 units each
- Enable async-replication with the `create-replication` action
- Enable monitoring with grafana-agent subordinates
- Check alerts in prometheus (COS)

## Expected behavior
Alert `PostgresqlExporterError` is **not** firing

## Actual behavior
Alert `PostgresqlExporterError` is firing
Image

Indeed the `pg_exporter_last_scrape_error` returns 1 on the replica unit of the standby cluster

```
curl -sS http://127.0.0.1:9187/metrics | grep pg_exporter_last_scrape_error
# HELP pg_exporter_last_scrape_error Whether the last scrape of metrics from PostgreSQL resulted in an error (1 for error, 0 for success).
# TYPE pg_exporter_last_scrape_error gauge
pg_exporter_last_scrape_error 1
```

The curl command above shows there is no scrape error.

## Versions

Juju 3.6.11
OS 24.04 Noble
Postgresql machine charm: 16/stable rev 952

## Additional context

This issue seems to be well known at upstream level: https://github.com/prometheus-community/postgres_exporter/issues/957

https://discuss.prometheus.io/t/pg-exporter-last-scrape-error-provides-1-on-crunchy-cluster-replica/1981

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.