prometheus-community / prometheus-community/postgres_exporter

Exporter web page hang when one or more databases is down

Open
#388 2 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.6k
Forks
835
Avg merge
2d 4h
Merged PRs (30d)
10

Description

Details:

  • Exporter version 0.8.0
  • Postgres ver 8 - 10

Expected:
Data from other databases, even that one is down.

Observation:
No data in prometheus when monitoring one or multiple database, and a database go down/offline

Reproduce:

  1. start exporter against fake database (simulate that database is down. Same result when testing on real database).
docker run --net=host -e DATA_SOURCE_NAME="postgresql://db.xxx.com/dbname?sslmode=disable" wrouesnel/postgres_exporter
  1. curl or browse http://localhost:9187/metrics and refresh page e.g. 5 - 10 times.

Result:
Browser hangs = no response for 6 sec x amount of refresh/request. Only reply/data when all requests are processed. So prometheus scape will fail. Use default timeout in prometheus (10 sec).

The exporter appears to buffer requests and only comes back with responses when all requests are processed / attempted. Time between each request is approx. 6 sec. (sourcecode: 3 retries) if it cannot access the database(s).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start at the /metrics HTTP scrape entry point and trace the three database retries described in the issue; done means an unavailable database no longer blocks metrics from other databases and scrapes receive a response within the configured timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgres
Domain
databases, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.