influxdata / influxdata/telegraf
[inputs.win_perf_counters] warnings although IgnoredErrors is set
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Relevant telegraf.conf
```toml
[[inputs.win_perf_counters]]
IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]
[[inputs.win_perf_counters.object]]
ObjectName = "Process"
Instances = ["w3wp","w3wp#1","w3wp#2","w3wp#3"]
Counters = [
"% Processor Time",
"Private Bytes",
"IO Read Bytes/sec",
"IO Write Bytes/sec",
"Page Faults/sec"
]
Measurement = "win_proc"
IncludeTotal = false
```
### Logs from Telegraf
```text
With IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]
2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\% Processor Time", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\IO Read Bytes/sec", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#2)\\IO Read Bytes/sec", instance: w3wp#2, will skip metric: A counter with a negative value was detected.
Without IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]
2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\% Processor Time", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#2)\\% Processor Time", instance: w3wp#2, will skip metric: A counter with a negative value was detected.
2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#3)\\% Processor Time", instance: w3wp#3, will skip metric: A counter with a negative value was detected.
```
### System info
Telegraf 1.33.0, Windows Server 2012R2
### Docker
_No response_
### Steps to reproduce
1. Added IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"] to Telgraf config
2. Restart Telegraf Service
3. Removed IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"] from Telegraf config
4. Restart Telegraf Service.
### Expected behavior
I would expect that with ` IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]` set no log entry will appear, because I want to ignore such errors by the win_perf_counters plugin.
If it only means to output a warning instead of an error, that we might need to have another option like `IgnoreWarnings = ["PDH_CALC_NEGATIVE_VALUE"]` as well.
The code line `m.Log.Warnf("Error while getting value for counter %q, instance: %s, will skip metric: %v", metric.counterPath, metric.instance, err)` is in my opinion a bit misleading, because it is outputting a warning with a message starting with "Error ..."
### Actual behavior
I am trying to understand the behavior of Telegraf with or without the “IgnoredErrors” list set. I occasionally get log entries with the message “A counter with a negative value was detected.” I am trying to limit the log entries to only show real errors, becasue we are ingesting them with Grafana Alloy and Loki. I have therefore put some PDH errors in the IgnoredErrors list. Unfortunately, this makes no difference to the output.
If I understand the Go code in win_perf_counters.go line 536 correctly, then a message is only output if err is not nil and the function isKnownCounterDataError decides whether the message should be outputted as an error or as a warning.
The logs of both cases (with or without IgnoredErrors) shows the message severies as "W!" which means "warning".
### Additional info
I also tried it with `IgnoredErrors = ["PDH_NO_DATA", "PDH_INVALID_DATA", "PDH_CSTATUS_INVALID_DATA", "PDH_CALC_NEGATIVE_VALUE"]` which also makes no difference.
Contributor guide
Research direction
Start in the win_perf_counters.go code around line 536 and compare how IgnoredErrors is used with the warning path shown in the issue logs. Reproduce the plugin with and without PDH_CALC_NEGATIVE_VALUE in the configuration, then verify that the resulting log behavior matches the documented meaning of IgnoredErrors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre, operating-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100