prometheus-community / prometheus-community/postgres_exporter
autoDiscoverDatabases:true gives error 500 (collected metric "pg_stat_activity_idle_process_seconds" was collected before)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 835
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 10
Description
Copied from prometheus-community/helm-charts
This issue originally was opened here but was unattended. Copied here as is. I experience this issue with postgres_exporter 0.10.1 and latest 0.11.1 as well
Describe the bug a clear and concise description of what the bug is.
With config.autoDiscoverDatabases: true I got error 500 on /metrics endpoint:
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Thu, 14 Jul 2022 11:37:11 GMT
< Transfer-Encoding: chunked
<
An error has occurred while serving metrics:
12 error(s) occurred:
* collected metric "pg_stat_activity_idle_process_seconds" { label:<name:"application_name" value:"" > label:<name:"server" value:"develop-postgresql:5432" > histogram:<sample_count:117 sample_sum:9240 bucket:<cumulative_count:36 upper_bound:1 > bucket:<cumulative_count:42 upper_bound:2 > bucket:<cumulative_count:53 upper_bound:5 > bucket:<cumulative_count:64 upper_bound:15 > bucket:<cumulative_count:65 upper_bound:30 > bucket:<cumulative_count:67 upper_bound:60 > bucket:<cumulative_count:67 upper_bound:90 > bucket:<cumulative_count:79 upper_bound:120 > bucket:<cumulative_count:118 upper_bound:300 > > } was collected before with the same name and label values
* collected metric "pg_stat_activity_idle_process_seconds" { label:<name:"application_name" value:"Patroni" > label:<name:"server" value:"develop-postgresql:5432" > histogram:<sample_count:1 sample_sum:1 bucket:<cumulative_count:0 upper_bound:1 > bucket:<cumulative_count:1 upper_bound:2 > bucket:<cumulative_count:1 upper_bound:5 > bucket:<cumulative_count:1 upper_bound:15 > bucket:<cumulative_count:1 upper_bound:30 > bucket:<cumulative_count:1 upper_bound:60 > bucket:<cumulative_count:1 upper_bound:90 > bucket:<cumulative_count:1 upper_bound:120 > bucket:<cumulative_count:1 upper_bound:300 > > } was collected before with the same name and label values
According to https://github.com/prometheus-community/postgres_exporter/issues/429#issuecomment-850865088 I manually added key master: true to query pg_stat_activity_idle and it fixes error
What's your helm version?
version.BuildInfo{Version:"v3.7.1", GitCommit:"1d11fcb5d3f3bf00dbe6fe31b8412839a96b3dc4", GitTreeState:"clean", GoVersion:"go1.16.9"}
What's your kubectl version?
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:10:43Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Which chart?
prometheus-postgres-exporter
What's the chart version?
3.0.3
What happened?
No response
What you expected to happen?
No response
How to reproduce it?
No response
Enter the changed values of values.yaml?
serviceMonitor:
enabled: true
config:
datasource:
host: develop-postgresql
user: develop_admin
password:
passwordSecret:
name: develop-admin.develop-postgresql.credentials.postgresql.acid.zalan.do
key: password
database: 'template1'
logLevel: info
autoDiscoverDatabases: true
Enter the command that you execute and failing/misfunctioning.
helm install postgres-exporter prometheus-community/prometheus-postgres-exporter --namespace db -f ./postrges_exporter.yaml
Anything else we need to know?
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the /metrics endpoint and the pg_stat_activity_idle query involved in autoDiscoverDatabases. Inspect how that query is handled across discovered databases and compare it with the reported manual master: true change. Done means the exporter serves metrics without duplicate pg_stat_activity_idle_process_seconds errors when autoDiscoverDatabases is enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, helm, kubernetes, postgresql, prometheus
- Domain
- databases, devops, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100