prometheus-community / prometheus-community/postgres_exporter

Database inside the DSN is never excluded

Open
#788 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.6k
Forks
835
Avg merge
2d 4h
Merged PRs (30d)
10

Description

What did you do?

Try to exclude the "master" database on custom queries with auto-discovery enabled (I don't know if it's the right term, but it is the database provided with the DSN).

What did you expect to see?

Even if I exclude the database provided inside the DSN, the exporter execute queries on it

# with DATA_SOURCE_NAME: "postgresql://postgres:password@postgres:5432/postgres?sslmode=disable" and PG_EXPORTER_EXCLUDE_DATABASES: 'postgres'
curl --silent localhost:9187/metrics | grep pg_test | grep -v '#'
pg_test_id{database="db1",server="localhost:5432"} 1
pg_test_id{database="db2",server="localhost:5432"} 1
pg_test_id{database="db3",server="localhost:5432"} 1
pg_test_id{database="db4",server="localhost:5432"} 1
pg_test_id{database="postgres",server="localhost:5432"} 1

What did you see instead? Under which circumstances?

I don't want to see the excluded database

# with DATA_SOURCE_NAME: "postgresql://postgres:password@postgres:5432/postgres?sslmode=disable" and PG_EXPORTER_EXCLUDE_DATABASES: 'postgres'
curl --silent localhost:9187/metrics | grep pg_test | grep -v '#'
pg_test_id{database="db1",server="localhost:5432"} 1
pg_test_id{database="db2",server="localhost:5432"} 1
pg_test_id{database="db3",server="localhost:5432"} 1
pg_test_id{database="db4",server="localhost:5432"} 1

Environment

  • System information:
Linux 6.2.9-200.fc37.x86_64 x86_64
  • postgres_exporter version:
postgres_exporter, version 0.12.0 (branch: HEAD, revision: f9a1edbbc6d629663aa658378e1d989293e6e488)
  build user:       root@463311dfdfb6
  build date:       20230321-23:52:03
  go version:       go1.20.2
  platform:         linux/amd64
  tags:             netgo static_build
  • postgres_exporter flags:

(everything is configured using environment)

DATA_SOURCE_NAME: "postgresql://postgres:password@postgres:5432/postgres?sslmode=disable"
PG_EXPORTER_AUTO_DISCOVER_DATABASES: 'true'
PG_EXPORTER_DISABLE_DEFAULT_METRICS: 'true'
PG_EXPORTER_DISABLE_SETTINGS_METRICS: 'true'
PG_EXPORTER_EXCLUDE_DATABASES: 'postgres'
PG_EXPORTER_EXTEND_QUERY_PATH: /queries.yml
  • PostgreSQL version:
psql (PostgreSQL) 15.2 (Debian 15.2-1.pgdg110+1)
  • Logs:

Nothing relevant about logs

Personal investigation
I made some investigation to find out if I did something wrong, and here is what I found

  • Files used

docker-compose.yml:

version: '3.5'

services:
  postgres:
    image: postgres
    environment:
      POSTGRES_PASSWORD: password
    volumes:
      - postgres:/data/postgres
    ports:
      - "5432:5432"
    restart: unless-stopped

  postgres-exporter:
    image: quay.io/prometheuscommunity/postgres-exporter
    environment:
      DATA_SOURCE_NAME: "postgresql://postgres:password@postgres:5432/postgres?sslmode=disable"
      PG_EXPORTER_AUTO_DISCOVER_DATABASES: 'true'
      PG_EXPORTER_DISABLE_DEFAULT_METRICS: 'true'
      PG_EXPORTER_DISABLE_SETTINGS_METRICS: 'true'
      PG_EXPORTER_EXCLUDE_DATABASES: 'postgres'
      PG_EXPORTER_EXTEND_QUERY_PATH: /queries.yml
    volumes:
      - ./queries.yml:/queries.yml
    command:
      - --log.level=debug
    ports:
      - "9187:9187"
    restart: unless-stopped
    user: root

volumes:
    postgres:

queries.yaml:

pg_test:
  query: "SELECT 1 as id, current_database() as database;"
  metrics:
    - database:
        usage: "LABEL"
        description: "Database name"
    - id:
        usage: "GAUGE"
        description: "Always 1"
  • What I found ?

The default DSN is always added to the list of DSN and never filtered: https://github.com/prometheus-community/postgres_exporter/blob/master/cmd/postgres_exporter/datasource.go#L57 ... it can explain why the given database is never excluded.
My suggestion would be to delete this line, but I don't know why it exists and if there is a good reason for it.

NOTE: this can be related with https://github.com/prometheus-community/postgres_exporter/issues/735 and probably other issues like https://github.com/prometheus-community/postgres_exporter/issues/750, https://github.com/prometheus-community/postgres_exporter/issues/443

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cmd/postgres_exporter/datasource.go around the default DSN addition cited in the issue, then trace how discovered databases are filtered by PG_EXPORTER_EXCLUDE_DATABASES. Reproduce the configuration with auto-discovery and the custom query, and confirm that the excluded DSN database no longer produces metrics; add or update coverage if the repository provides a relevant test location.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, postgresql
Domain
backend, databases, observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.