prometheus-community / prometheus-community/postgres_exporter
To many connections open when scrape different instances from one exporter setup
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 835
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 10
Description
What did you do?
Configuring a postgres-exporter instance using docker-compose to scrape metrics from different instances open to many connections
postgres=# select count(*), state, usename from pg_stat_activity group by 2,3;
count | state | usename
-------+--------+-------------------
4 | |
2 | active | replication
1 | active | postgres
1 | | postgres
2 | idle | postgres
50 | idle | postgres_exporter
(6 rows)
This is the DATASOURCE definition:
environment:
DATA_SOURCE_NAME: "postgresql://postgres_exporter:XXXX@10.0.2.25:5432/postgres?sslmode=disable,postgresql://postgres_exporter:XXXX@10.0.2.26:5432/postgres?sslmode=disable,postgresql://postgres_exporter:XXXX@10.0.2.27:5432/postgres?sslmode=disable"
docker-compose log:
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Established new database connection to \"10.0.2.25:5432\"." source="postgres_exporter.go:983"
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Semantic Version Changed on \"10.0.2.25:5432\": 0.0.0 -> 10.15.0" source="postgres_exporter.go:1552"
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Established new database connection to \"10.0.2.26:5432\"." source="postgres_exporter.go:983"
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Semantic Version Changed on \"10.0.2.26:5432\": 0.0.0 -> 10.15.0" source="postgres_exporter.go:1552"
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Established new database connection to \"10.0.2.27:5432\"." source="postgres_exporter.go:983"
postgres-exporter | time="2021-05-10T10:22:14Z" level=info msg="Semantic Version Changed on \"10.0.2.27:5432\": 0.0.0 -> 10.15.0" source="postgres_exporter.go:1552"
And if you scrape from only one instance, it does not open such many connections:
environment:
DATA_SOURCE_NAME: "postgresql://postgres_exporter:XXXX@10.0.2.25:5432/postgres?sslmode=disable"
Connections:
postgres=# select count(*), state, usename from pg_stat_activity group by 2,3;
count | state | usename
-------+--------+-------------------
4 | |
2 | active | replication
1 | active | postgres
1 | | postgres
2 | idle | postgres
1 | idle | postgres_exporter
(6 rows)
docker-compose log:
postgres-exporter | time="2021-05-10T10:26:47Z" level=info msg="Established new database connection to \"10.0.2.25:5433\"." source="postgres_exporter.go:983"
postgres-exporter | time="2021-05-10T10:26:47Z" level=info msg="Semantic Version Changed on \"10.0.2.25:5433\": 0.0.0 -> 10.15.0" source="postgres_exporter.go:1552"
What did you expect to see?
postgres-exporter should not open more than one connection per instance even if you use auto-discover-databases.
Check also https://github.com/prometheus-community/postgres_exporter/issues/436
Environment
This is the docker-compose definition:
version: '3'
services:
postgres-exporter:
image: quay.io/prometheuscommunity/postgres-exporter
container_name: postgres-exporter
environment:
DATA_SOURCE_NAME: "postgresql://postgres_exporter:XXXX@10.0.2.25:5432/postgres?sslmode=disable,postgresql://postgres_exporter:XXXX@10.0.2.26:5432/postgres?sslmode=disable,postgresql://postgres_exporter:XXXX@10.0.2.27:5432/postgres?sslmode=disable"
PG_EXPORTER_EXTEND_QUERY_PATH: "/queries/queries.yaml"
PG_EXPORTER_AUTO_DISCOVER_DATABASES: "true"
PG_EXPORTER_EXCLUDE_DATABASES: "template0,template1"
PG_EXPORTER_CONSTANT_LABELS: "cluster_name=my_cluster"
volumes:
- ./postgres-exporter/:/queries
ports:
- 9187:9187
networks:
- monitor
networks:
monitor:
- postgres_exporter version:
v0.9.0
$ docker images | grep postgres-exporter
quay.io/prometheuscommunity/postgres-exporter latest ca5bb120a572 2 months ago 16.9MB
- PostgresSQL version:
version
--------------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 10.15 (Ubuntu 10.15-0ubuntu0.18.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit
(1 row)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the connection setup reported at postgres_exporter.go:983 and the auto-discovery configuration in the docker-compose definition, including PG_EXPORTER_AUTO_DISCOVER_DATABASES. Reproduce with the multi-instance DATA_SOURCE_NAME and queries.yaml setup, then verify that scraping leaves no more than one connection per instance; compare against issue #436.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, go, postgresql
- Domain
- databases, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100