prometheus / prometheus/node_exporter
node_systemd_unit_state metrics silently ignores units that systemd failed to load
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 13.8k
- Forks
- 2.7k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 8
Description
Systemd units have two status fields, "load state" and "active state". Node Exporter only reports on active state. If a unit has any load state other than "loaded" it is silently ignored (unless debug logging is enabled, but it still does not appear in metrics).
Practical example: if a .timer unit has an invalid OnCalendar value (if like me you mistakenly assumed it would support typical crontab syntax), then the timer will fail to load and never be reported in any metrics by Node Exporter.
This filtering is done here: https://github.com/prometheus/node_exporter/blob/2179f0a34d2d7b6212f3a1c647d5aca44ffa33e5/collector/systemd_linux.go#L494-L499
It was added in this commit: https://github.com/prometheus/node_exporter/commit/0fdc089187c3d5d9fe62e528653986ee6e71325b
I am not sure why this filter was added, but i feel that being able to alert on units that failed to load would be a useful safety feature. Could this filter either be removed or controlled by a CLI flag?
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in collector/systemd_linux.go at the filtering logic around lines 494-499, then read the linked commit that introduced it. Determine whether failed-to-load units should be reported directly or controlled by a CLI flag, and verify that the resulting metrics include units with non-loaded load states.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability, operating-systems
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100