prometheus / prometheus/node_exporter

node_systemd_unit_state metrics silently ignores units that systemd failed to load

Open
#3,347 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.8k
Forks
2.7k
Avg merge
1d 23h
Merged PRs (30d)
8

Description

Systemd units have two status fields, "load state" and "active state". Node Exporter only reports on active state. If a unit has any load state other than "loaded" it is silently ignored (unless debug logging is enabled, but it still does not appear in metrics).

Practical example: if a .timer unit has an invalid OnCalendar value (if like me you mistakenly assumed it would support typical crontab syntax), then the timer will fail to load and never be reported in any metrics by Node Exporter.

This filtering is done here: https://github.com/prometheus/node_exporter/blob/2179f0a34d2d7b6212f3a1c647d5aca44ffa33e5/collector/systemd_linux.go#L494-L499

It was added in this commit: https://github.com/prometheus/node_exporter/commit/0fdc089187c3d5d9fe62e528653986ee6e71325b

I am not sure why this filter was added, but i feel that being able to alert on units that failed to load would be a useful safety feature. Could this filter either be removed or controlled by a CLI flag?
Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in collector/systemd_linux.go at the filtering logic around lines 494-499, then read the linked commit that introduced it. Determine whether failed-to-load units should be reported directly or controlled by a CLI flag, and verify that the resulting metrics include units with non-loaded load states.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability, operating-systems
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.