prometheus / prometheus/node_exporter

Mounstats collector ignores NFS mount even if metrics are different

Open
#993 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.8k
Forks
2.7k
Avg merge
1d 23h
Merged PRs (30d)
8

Description

Host operating system: output of uname -a

Linux debian 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

node_exporter, version 0.16.0 (branch: master, revision: ac5a98176129e86c69f664e632ff273eee6f67bd)
build user: root@debian
build date: 20180706-16:39:05
go version: go1.10.3

node_exporter command line flags

--collector.mountstats --log.level=debug

Are you running node_exporter in Docker?

No

What did you do that produced an error?

I mounted the same device twice, both mounts were on the same mountpoint. The first mount was done using nfs version 3 and the other was done using nfs version 4.
Command: mount -t nfs -o vers=3 127.0.0.1:/var/nfs /mnt/nfs
Command: mount -t nfs -o vers=4 127.0.0.1:/var/nfs /mnt/nfs

What did you expect to see?

The mounts have some differences(e.g age, port, events stats). Considering that these metrics are different I expected to see a separate line for each nfs mount in node_exporter metrics.

What did you see instead?

I saw metrics for only one mount.
node_mountstats_nfs_age_seconds_total{export="127.0.0.1:/var/nfs"} 499
# HELP node_mountstats_nfs_event_jukebox_delay_total Number of times the NFS server indicated EJUKEBOX; retrieving data from offline storage.
# HELP node_mountstats_nfs_event_vfs_read_page_total Number of pages read directly via mmap()'d files.
# TYPE node_mountstats_nfs_event_vfs_read_page_total counter
node_mountstats_nfs_event_vfs_read_page_total{export="127.0.0.1:/var/nfs"} 0
# HELP node_mountstats_nfs_event_vfs_read_pages_total Number of times a group of pages have been read.
# TYPE node_mountstats_nfs_event_vfs_read_pages_total counter
node_mountstats_nfs_event_vfs_read_pages_total{export="127.0.0.1:/var/nfs"} 0
# HELP node_mountstats_nfs_event_vfs_update_page_total Number of updates (and potential writes) to pages.
# TYPE node_mountstats_nfs_event_vfs_update_page_total counter
node_mountstats_nfs_event_vfs_update_page_total{export="127.0.0.1:/var/nfs"} 0

My current approach to solving this problem is to have the mounstats_linux.go compare the device name, mountpoint and port. If all those values are the same then the collector will skip it as a duplicate, if not then it will show the metrics. Do you think this is a good solution? I can make a pull request if the approach seems fine.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read mountstats_linux.go to trace how mounts are identified and why mounts sharing an export and mountpoint are treated as duplicates. Compare the device name, mountpoint, and port as proposed, then verify that separate metrics are emitted for the NFS v3 and v4 mounts.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, linux
Domain
observability-sre, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.