prometheus / prometheus/node_exporter

Linux mdadm error parsing mdraids - Invalid syntax since version 1.10.0

Open
#3,528 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.8k
Forks
2.7k
Avg merge
1d 23h
Merged PRs (30d)
8

Description

After upgrading to the latest version of node-exporter (from version 1.9.1), we're seeing frequent errors as below

level=ERROR source=collector.go:168 msg="collector failed" name=mdadm duration_seconds=0.000213012 err="error parsing mdraids: strconv.ParseUint: parsing": invalid syntax"

On our commercial Azure Kubernetes Service, this error does not occur.
But in Azure US Government AKS clusters, the error appears.

This is the only major difference I know of.
As it is a managed service, we don't directly control any of the RAID or other underlying disks the nodes run from, so not sure what could be causing this.

We've also never recorded, configured or used any RAID metrics.
Any suggestions on what more information to supply to get this resolved would be appreciated

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at collector.go:168 and trace the mdadm collector's RAID parsing path. Compare the mdraid input from commercial and US Government AKS environments, focusing on the value that causes strconv.ParseUint to report invalid syntax. Done means the triggering input and cause are identified and a verified resolution or actionable reproduction is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go, kubernetes, linux
Domain
observability-sre, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.