prometheus / prometheus/node_exporter
textfile collector won't send any metric to prometheus server
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 13.8k
- Forks
- 2.7k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 8
Description
Host operating system: output of uname -a
Linux ... 4.14.287-215.504.amzn2.x86_64 #1 SMP Wed Jul 13 21:34:43 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
node_exporter version: output of node_exporter --version
v1.5.0
node_exporter command line flags
version: "3"
services:
node_exporter:
image: quay.io/prometheus/node-exporter:v1.5.0
volumes:
- "/proc:/host/proc:ro"
- "/sys:/host/sys:ro"
- "/:/rootfs:ro"
command:
- "--path.procfs=/host/proc"
- "--path.rootfs=/rootfs"
- "--path.sysfs=/host/sys"
- "--collector.disable-defaults"
- "--web.disable-exporter-metrics"
# - "--collector.loadavg"
# - "--collector.meminfo"
# - "--collector.filesystem"
# - "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc|mnt|run)($$|/)"
- "--collector.textfile"
- "--collector.textfile.directory=/rootfs/prom_files"
ports:
- 9010:9100
deploy:
mode: global
node_exporter log output
ts=2023-06-01T06:41:38.332Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)"
ts=2023-06-01T06:41:38.332Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)"
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:186 level=debug msg="Go MAXPROCS" procs=1
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:110 level=info msg="Enabled collectors"
ts=2023-06-01T06:41:38.333Z caller=node_exporter.go:117 level=info collector=textfile
ts=2023-06-01T06:41:38.333Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100
ts=2023-06-01T06:41:38.333Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100
ts=2023-06-01T06:41:45.478Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:41:45.479Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000507488
ts=2023-06-01T06:42:00.476Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:42:00.477Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000264999
ts=2023-06-01T06:42:15.476Z caller=node_exporter.go:78 level=debug msg="collect query:" filters="unsupported value type"
ts=2023-06-01T06:42:15.476Z caller=collector.go:173 level=debug msg="collector succeeded" name=textfile duration_seconds=0.000256308
Are you running node_exporter in Docker?
Yes, especially in docker swarm installation.
I have deployed node exporter only on manager nodes.
What did you do that produced an error?
I just added textfile collector metrics from a file that is updated by cronjob every minute. The file is correct and updated correctly every 1 minute.
The data weren't appeared in prometheus, but after several reloads it is. So, we live with that bug for weeks, but didn't find any solution to make node exporter stable to send metrics from textfile collector.
What did you expect to see?
Textfile collector sends data from the file correctly after any node exporter reload.
What did you see instead?
After node exporter reload, node exporter won't send any textfile collector metric to prometheus server, only after several more reloads.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the supplied Docker Swarm configuration with the textfile collector and its mounted /rootfs/prom_files directory, then inspect the collector's scrape behavior and debug logs after a reload. Done means metrics from the updated textfile are exposed consistently after every node_exporter reload, rather than only after repeated reloads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- devops, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100