prometheus-community / prometheus-community/postgres_exporter

Add support for pg_stat_replication.(write_lag|flush_lag|replay_lag)

Open
#1,007 3 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.6k
Forks
835
Avg merge
2d 4h
Merged PRs (30d)
10

Description

Add support for pg_stat_replication.(write_lag|flush_lag|replay_lag)

Use case. Why is this important?

For simple primary:replica setups it would be convenient to be able to monitor standby lag in seconds from the primary. pg_stat_replication already includes this information on postgres>=10, but the exporter does not parse it. AFAIK, workarounds would include

  • monitoring byte offset from pg_stat_replication.write_lsn|flush_lsn|replay_lsn), but this does not quite capture issues where the replica has not replayed a business critical change however small in bytes.
  • monitoring delay on the replica, but this would require spinning up a separate monitor only to monitor a single value on the replica.

Notes

I'm assuming the exporter does not support parsing the interval data type, which is why these metrics are marked as DISCARD here. I wonder if the following, explicit approach would be appropriate in this case:

diff --git a/cmd/postgres_exporter/queries.go b/cmd/postgres_exporter/queries.go
index fa0b5c2..e28d7b4 100644
--- a/cmd/postgres_exporter/queries.go
+++ b/cmd/postgres_exporter/queries.go
@@ -53,7 +53,10 @@ var queryOverrides = map[string][]OverrideQuery{
 			SELECT *,
 				(case pg_is_in_recovery() when 't' then null else pg_current_wal_lsn() end) AS pg_current_wal_lsn,
 				(case pg_is_in_recovery() when 't' then null else pg_wal_lsn_diff(pg_current_wal_lsn(), pg_lsn('0/0'))::float end) AS pg_current_wal_lsn_bytes,
-				(case pg_is_in_recovery() when 't' then null else pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)::float end) AS pg_wal_lsn_diff
+				(case pg_is_in_recovery() when 't' then null else pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)::float end) AS pg_wal_lsn_diff,
+				(case pg_is_in_recovery() when 't' then null else  extract(epoch from write_lag) end) as write_lag_seconds,
+				(case pg_is_in_recovery() when 't' then null else  extract(epoch from flush_lag) end) as flush_lag_seconds,
+				(case pg_is_in_recovery() when 't' then null else  extract(epoch from replay_lag) end) as replay_lag_seconds
 			FROM pg_stat_replication
 			`,
 		},

That way the deviation from the official fields is not in conflict with the source data.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cmd/postgres_exporter/queries.go at the pg_stat_replication query override, then inspect the DISCARD handling referenced in cmd/postgres_exporter/postgres_exporter.go. Confirm how the write_lag, flush_lag, and replay_lag interval values are parsed and exposed; done means the three lag metrics are available in seconds for supported PostgreSQL versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
databases, observability
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.