prometheus / prometheus/node_exporter

xvda[0-9]+ disks (not partitions) are ignored by default

Open
#2,041 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.8k
Forks
2.7k
Avg merge
1d 23h
Merged PRs (30d)
8

Description

Host operating system: output of uname -a

Linux 5.3.18-24.64-default

node_exporter version: output of node_exporter --version

node_exporter, version (branch: , revision: )
build user:
build date:
go version: go1.14.7

node_exporter command line flags

none

Are you running node_exporter in Docker?

no

What did you do that produced an error?

default config does not collect disk stats

What did you expect to see?

all disk devices should be collected by default

What did you see instead?

No disk stats

Since #13 got merged, xvda[0-9]+ are ignored. However, it is not correct to assume that those are always partitions. Xen allows a machine to inform "virtual partition", which are mapped to dedicated devices (it makes expanding a filesystem trivial).

This is a normal xvda-disk with partitions (possibly on what that PR was based on):

# ls /sys/block/xvda* -l
lrwxrwxrwx 1 root root 0 Mai 17 14:46 /sys/block/xvda -> ../devices/vbd-51712/block/xvda
# grep xvda /proc/diskstats 
 202       0 xvda 26438 52 1897369 17235 4263982 53800 47477992 3542063 0 103104 3557922
 202       1 xvda1 22 0 176 7 0 0 0 0 0 7 7
 202       2 xvda2 118 0 13522 94 129 3 30326 274 0 42 367
 202       3 xvda3 26268 52 1880583 17122 4263853 53797 47447666 3541789 0 103052 3557536

And this is a "virtual-partitions" machine:

# ls /sys/block/xvd* -d1
/sys/block/xvda1
/sys/block/xvda2
# grep xvda /proc/diskstats 
 202       1 xvda1 66701 3 13340088 570748 1833 254 308680 72 0 32440 674096
 202       2 xvda2 101 0 4448 0 0 0 0 0 0 4 4

xvda[0-9]+ should not be excluded by default only based on their name. At least in Linux, this is the easy but wrong way to check if a device is a partition or a disk. node_exporter could check if /sys/block/ exists (not partition) of if /sys/dev/block/:/partition file exists (not a disk).

If collector.diskstats.ignored-devices, by default, only excludes partitions, I think a new option like:
"collector.diskstats.ignore-partitions (default: true)" should be used instead, leaving "ignored-devices" empty by default.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the diskstats collector and its collector.diskstats.ignored-devices defaults, then compare /proc/diskstats with /sys/block and /sys/dev/block partition information described in the report. Done means virtual xvda[0-9]+ devices are collected while actual partitions remain excluded by the default behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, linux
Domain
observability-sre, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.