prometheus-community / prometheus-community/postgres_exporter

PostgresCollector emits no metrics at all — not even pg_scrape_collector_success=0 — when the per-scrape connection fails, while pg_up stays 1

Open
#1,387 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.6k
Forks
835
Avg merge
2d 4h
Merged PRs (30d)
10

Description

Version

v0.20.1 (postgres_exporter_build_info{version="0.20.1", revision="867fbcac31cd18c143e244190ea9168cca069827"}), PostgreSQL 17.10.

What happens

When PostgresCollector cannot open its per-scrape connection, it emits nothing at all — not even pg_scrape_collector_success{collector="…"} 0. The failure exists only as a log line.

https://github.com/prometheus-community/postgres_exporter/blob/v0.20.1/collector/collector.go#L183-L195

func (p PostgresCollector) Collect(ch chan<- prometheus.Metric) {
	// copy the instance so that concurrent scrapes have independent instances
	inst := p.instance.copy()

	// Set up the database connection for the collector.
	err := inst.setup()
	defer inst.Close()
	if err != nil {
		p.logger.Error("Error opening connection to database", "err", err)
		return
	}
	p.collectFromConnection(inst, ch)
}

pg_scrape_collector_success is produced by execute(), which is only reached from collectFromConnection. So the return above skips the one metric that is supposed to report a collector problem.

At the same time the other half of the exporter — the Exporter registered alongside PostgresCollector in cmd/postgres_exporter/main.go — holds a persistent connection per server. It therefore keeps reporting pg_up 1 and pg_exporter_last_scrape_error 0 while every --collector.* metric is missing.

Net effect: the exporter reports "up, no scrape errors" while roughly half of its metric surface has silently disappeared. The only in-band signal is the absence of series.

Why it matters (measured)

On 2026-09-07 a shared development PostgreSQL of ours refused new connections for just under three hours (FATAL: sorry, too many clients already, ~39.7k occurrences). Over that window:

Source Behaviour
all 18 --collector.* collectors pg_scrape_collector_success = 0 on 49 samples, plus outright gaps 05:16→07:49 (scrapes where inst.setup() failed and Collect returned early)
pg_up, pg_exporter_last_scrape_error 2400 samples at 15s, no gaps, values 1 and 0 for the entire outage
a --extend.query-path user query 2399 samples at 15s, no gaps

Because collector metrics vanish rather than report a failure, an alert written on them (in our case connection saturation, sum(pg_stat_activity_count) / pg_settings_max_connections > 0.7) evaluates to "no data" and resolves itself in the middle of the incident, exactly when the condition is at its worst. pg_up-based and pg_exporter_last_scrape_error-based alerts do not compensate, because those come from the other registration and stay green.

Connection exhaustion is only the most obvious trigger; anything that makes inst.setup() fail (auth change, pooler saturation, TLS) has the same shape.

Suggestion

Make the failure representable in metrics rather than only in logs. Either of:

  1. On inst.setup() error, emit pg_scrape_collector_success{collector=…} 0 for every enabled collector before returning; or
  2. Add a dedicated gauge, e.g. pg_scrape_connection_error, set to 1 on that path.

(1) needs no new metric name and makes existing dashboards/alerts on pg_scrape_collector_success correct. Happy to send a PR if the maintainers prefer one of the two.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in collector/collector.go around PostgresCollector.Collect, inst.setup(), collectFromConnection, and execute(), then compare the Exporter registration in cmd/postgres_exporter/main.go. Determine which failure signal maintainers prefer, and verify that a failed per-scrape connection reports failure metrics instead of silently omitting collector series while pg_up remains successful.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql, prometheus
Domain
databases, observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.