prometheus / prometheus/node_exporter

node_disk_filesystem_info missing via container

Open
#2,774 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
13.8k
Forks
2.7k
Avg merge
1d 23h
Merged PRs (30d)
8

Description

I have tried without host and the entire line which made no difference.

- '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'

If I install node-exporter via package manager I get the missing stats.

Host operating system: output of uname -a

Linux controller 6.4.10-200.fc38.aarch64 #1 SMP PREEMPT_DYNAMIC Fri Aug 11 12:41:15 UTC 2023 aarch64 GNU/Linux

node_exporter version: output of node_exporter --version
node_exporter, version 1.6.1 (branch: HEAD, revision: 4a1b77600c1873a8233f3ffb55afcedbb63b8d84)
  build user:       root@586879db11e5
  build date:       20230717-12:11:23
  go version:       go1.20.6
  platform:         linux/arm64
  tags:             netgo osusergo static_build
node_exporter command line flags
...
services:
  node-exporter:
    image: docker.io/prom/node-exporter:latest
    privileged: true
    container_name: node-exporter
    restart: unless-stopped
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro,rslave
    command:
      - '--log.level=debug'
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
    expose:
      - 9100
    ports:
      - 9100:9100
    networks:
      - monitoring
node_exporter log output
ts=2023-08-15T13:28:54.335Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.6.1, branch=HEAD, revision=4a1b77600c1873a8233f3ffb55afcedbb63b8d84)"
ts=2023-08-15T13:28:54.335Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.20.6, platform=linux/arm64, user=root@586879db11e5, date=20230717-12:11:23, tags=netgo osusergo static_build)"
ts=2023-08-15T13:28:54.336Z caller=node_exporter.go:186 level=debug msg="Go MAXPROCS" procs=1
ts=2023-08-15T13:28:54.340Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(sys|proc|dev|host|etc)($|/)
ts=2023-08-15T13:28:54.340Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$
ts=2023-08-15T13:28:54.341Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$
ts=2023-08-15T13:28:54.341Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data
ts=2023-08-15T13:28:54.341Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
ts=2023-08-15T13:28:54.341Z caller=node_exporter.go:117 level=info collector=arp
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=bcache
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=bonding
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=btrfs
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=conntrack
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=cpu
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=cpufreq
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=diskstats
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=dmi
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=edac
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=entropy
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=fibrechannel
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=filefd
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=filesystem
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=hwmon
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=infiniband
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=ipvs
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=loadavg
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=mdadm
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=meminfo
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=netclass
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=netdev
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=netstat
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=nfs
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=nfsd
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=nvme
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=os
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=powersupplyclass
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=pressure
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=rapl
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=schedstat
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=selinux
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=sockstat
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=softnet
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=stat
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=tapestats
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=textfile
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=thermal_zone
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=time
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=timex
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=udp_queues
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=uname
ts=2023-08-15T13:28:54.342Z caller=node_exporter.go:117 level=info collector=vmstat
ts=2023-08-15T13:28:54.343Z caller=node_exporter.go:117 level=info collector=xfs
ts=2023-08-15T13:28:54.343Z caller=node_exporter.go:117 level=info collector=zfs
ts=2023-08-15T13:28:54.345Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100
ts=2023-08-15T13:28:54.345Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100
Are you running node_exporter in Docker?

Using rootless podman

What did you do that produced an error?

N/A

What did you expect to see?
# HELP node_disk_filesystem_info Info about disk filesystem.
# TYPE node_disk_filesystem_info gauge
node_disk_filesystem_info{device="dm-0",type="xfs",usage="filesystem",uuid="89014dd8-f24f-45ff-98d2-03adfe894544",version=""} 1
node_disk_filesystem_info{device="mmcblk0p1",type="vfat",usage="filesystem",uuid="6E0C-FE6D",version="FAT32"} 1
node_disk_filesystem_info{device="mmcblk0p2",type="xfs",usage="filesystem",uuid="f9ef8131-4212-4194-bbcb-3f04626f34a4",version=""} 1
node_disk_filesystem_info{device="mmcblk0p3",type="LVM2_member",usage="raid",uuid="7IeeNH-xPfj-gtdm-q63k-MkND-Mqg0-WRg5Ni",version="LVM2 001"} 1
What did you see instead?

Missing the above exported settings

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the filesystem collector and the path settings shown in the node_exporter command, using the filesystem_common.go log entry as an entry point. Compare the container's rootless Podman mounts with the package-manager installation and inspect why the expected node_disk_filesystem_info metrics are absent. Done means the listed filesystem metrics are emitted in the container setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre, operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.