infiniband collector: device-exclude does not prevent reading excluded devices' counters (sysfs reads still issued; kernel/firmware errors persist)

What did you do?

On hosts with NVIDIA ConnectX-7 adapters whose ports are firmware-managed (restricted PF — same hardware class as #3434; counter reads on those ports fail per #3265), I used the device filter added in v1.12.0 to stop the collector from touching those devices:

prometheus-node-exporter --collector.infiniband.device-exclude=^mlx5_(12|13|14|15)$ ...

• node_exporter version: 1.12.1 (revision 6044da783597cc3b57aef7580ddcdcff58a4ee99)
• OS/kernel: Ubuntu, 6.5.0-45-generic, NVIDIA OFED/DKMS ib_core / mlx5_ib
• Problem devices: mlx5_12..15 (ConnectX-7, MT4129, PCI 0000:a8:00.0-.3)

What did you expect to see?

Excluded devices are skipped before any sysfs access — that is the point of the filter on this hardware class: reading ports/*/counters/* on these ports triggers a firmware command that the device rejects, which floods the kernel log (rate-limited):

mlx5_core 0000:a8:00.0: mlx5_cmd_out_err:839:(pid …): ACCESS_REG(0x805) op_mod(0x1) failed,
status bad operation(0x2), syndrome (0x9a6171), err(-22)

What did you see instead?

The filter works for the emitted metrics (excluded devices disappear from /metrics), but the collector still opens and reads the excluded devices' counter files on every scrape, so the kernel errors continue (~40 lines/min/host after the flag was applied).

Evidence (reproducible):

1. Flag present in the running process argv; excluded devices absent from /metrics.
2. strace -f --seccomp-bpf -e trace=openat -p <node_exporter pid> for 30 s:
   120 opens of /sys/class/infiniband/mlx5_12/ports/1/counters/* (and 120 for mlx5_13) — all excluded devices.
3. ftrace kprobe on mlx5_cmd_out_err (fires only on failed firmware commands): 100% of failures come from the node_exporter task, with stack:

   mlx5_cmd_out_err ← mlx5_access_reg ← mlx5_core_access_reg ← query_ib_ppcnt
   ← mlx5_ib_process_mad ← get_perf_mad ← show_pma_counter ← port_attr_show
   ← sysfs_kf_seq_show ← kernfs_seq_show ← seq_read_iter ← kernfs_fop_read_iter
   ← vfs_read ← ksys_read
   

Note this is not the collector-abort problem from #3265 — failed reads are tolerated (EINVAL is skipped) and node_scrape_collector_success{collector="infiniband"} stays 1; the remaining problem is the reads themselves (firmware churn + kernel log spam).

Workaround that fully stops it (for reference): replacing the device's counters/ dir with an empty dir in the node_exporter service's mount namespace:

[Service]
BindReadOnlyPaths=/var/lib/node-exporter-hidden/empty:/sys/class/infiniband/mlx5_12/ports/1/counters
# ... one line per port

After that: zero kernel errors, collector success=1, all non-excluded metrics intact.

Request

Please apply the device include/exclude check before any sysfs access for that device (e.g. at the top of the per-device loop in collector/infiniband_linux.go), or document the current behavior if it is intentional. Happy to test a patch — we have reproducible hardware for this.

Start in collector/infiniband_linux.go and locate the per-device loop and device include/exclude handling. Reproduce the issue with strace against the node_exporter process, then verify that excluded devices cause no sysfs counter opens while non-excluded metrics and collector success remain intact.