prometheus / prometheus/node_exporter
error gathering metrics: collected metric \"apt_upgrades_pending\" ... was collected before with the same name and label values
Open
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 13.8k
- Forks
- 2.7k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 8
Description
Host operating system:
Linux fra-pg-01 4.4.0-66-generic #87-Ubuntu SMP Fri Mar 3 15:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
node_exporter version:
$ /usr/bin/prometheus-node-exporter --version
node_exporter, version 0.17.0+ds (branch: debian/sid, revision: 0.17.0+ds-2)
build user: pkg-go-maintainers@lists.alioth.debian.org
build date: 20181214-14:09:27
go version: go1.10.4
node_exporter command line flags
$ cat /etc/default/prometheus-node-exporter
# Set the command-line arguments to pass to the server.
# Due to shell escaping, to pass backslashes for regexes, you need to double
# them (\\d for \d). If running under systemd, you need to double them again
# (\\\\d to mean \d), and escape newlines too.
ARGS="--collector.diskstats.ignored-devices=^(ram|loop|fd)\\\\d+$ \
--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|run|var/lib/docker/overlay2/[a-z0-9]{64}/merged|var/lib/docker/aufs/mnt/[a-z0-9]{64}|var/lib/docker/devicemapper/mnt/[a-z0-9]{64}|var/lib/docker/containers/[a-z0-9]{64}/shm|var/lib/docker/containers/[a-z0-9]{64}/mounts/shm|var/lib/docker/plugins/[a-z0-9]{64})($|/) \
--collector.netdev.ignored-devices=^lo$ \
--collector.textfile.directory=/var/lib/prometheus/node-exporter \
--collector.ntp \
--no-collector.hwmon"
Are you running node_exporter in Docker?
No.
What did you do that produced an error?
Didn't do anything particular, just looking at the logs.
What did you expect to see?
No error.
What did you see instead?
The following error:
$ systemctl status prometheus-node-exporter --no-pager -l
● prometheus-node-exporter.service - Prometheus exporter for machine metrics
Loaded: loaded (/lib/systemd/system/prometheus-node-exporter.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2019-05-02 15:32:02 UTC; 3 months 27 days ago
Docs: https://github.com/prometheus/node_exporter
Main PID: 2235 (prometheus-node)
Tasks: 74
Memory: 25.7M
CPU: 21h 7min 57.439s
CGroup: /system.slice/prometheus-node-exporter.service
└─2235 /usr/bin/prometheus-node-exporter --collector.diskstats.ignored-devices=^(ram|loop|fd)\d+$ --collector.filesystem.ignored-mount-points=^/(sys|proc|dev|run|var/lib/docker/overlay2/[a-z0-9]{64}/merged|var/lib/docker/aufs/mnt/[a-z0-9]{64}|var/lib/docker/devicemapper/mnt/[a-z0-9]{64}|var/lib/docker/containers/[a-z0-9]{64}/shm|var/lib/docker/containers/[a-z0-9]{64}/mounts/shm|var/lib/docker/plugins/[a-z0-9]{64})($|/) --collector.netdev.ignored-devices=^lo$ --collector.textfile.directory=/var/lib/prometheus/node-exporter --collector.ntp --no-collector.hwmon
Aug 29 12:05:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:05:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:05:36 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:05:36Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:05:39 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:05:39Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:06:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:06:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:07:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:07:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:08:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:08:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:09:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:09:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:10:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:10:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:11:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:11:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Aug 29 12:12:22 fra-pg-01 prometheus-node-exporter[2235]: time="2019-08-29T12:12:22Z" level=error msg="error gathering metrics: [from Gatherer #2] collected metric \"apt_upgrades_pending\" { label:<name:\"arch\" value:\"Debian/Ubuntu\" > label:<name:\"origin\" value:\"PostgreSQLfor\" > gauge:<value:6 > } was collected before with the same name and label values\n" source="log.go:172"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the node_exporter metric-gathering path for apt_upgrades_pending and reproduce the duplicate metric using the reported Debian/Ubuntu setup. Trace which collector or input produces the same name and label values, then add coverage for the failure case; done means gathering completes without duplicate-series errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100