influxdata / influxdata/telegraf

[inputs.win_perf_counters] warnings although IgnoredErrors is set

Open
#16,673 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Relevant telegraf.conf

```toml
[[inputs.win_perf_counters]]
IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]

[[inputs.win_perf_counters.object]]
ObjectName = "Process"
Instances = ["w3wp","w3wp#1","w3wp#2","w3wp#3"]
Counters = [
"% Processor Time",
"Private Bytes",
"IO Read Bytes/sec",
"IO Write Bytes/sec",
"Page Faults/sec"
]
Measurement = "win_proc"
IncludeTotal = false
```

### Logs from Telegraf

```text
With IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]

2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\% Processor Time", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\IO Read Bytes/sec", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:14:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#2)\\IO Read Bytes/sec", instance: w3wp#2, will skip metric: A counter with a negative value was detected.

Without IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]

2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#1)\\% Processor Time", instance: w3wp#1, will skip metric: A counter with a negative value was detected.
2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#2)\\% Processor Time", instance: w3wp#2, will skip metric: A counter with a negative value was detected.
2025-03-21T10:54:30+01:00 W! [inputs.win_perf_counters] Error while getting value for counter "\\Process(w3wp#3)\\% Processor Time", instance: w3wp#3, will skip metric: A counter with a negative value was detected.

```

### System info

Telegraf 1.33.0, Windows Server 2012R2

### Docker

_No response_

### Steps to reproduce

1. Added IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"] to Telgraf config
2. Restart Telegraf Service
3. Removed IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"] from Telegraf config
4. Restart Telegraf Service.

### Expected behavior

I would expect that with ` IgnoredErrors = ["PDH_CALC_NEGATIVE_VALUE"]` set no log entry will appear, because I want to ignore such errors by the win_perf_counters plugin.

If it only means to output a warning instead of an error, that we might need to have another option like `IgnoreWarnings = ["PDH_CALC_NEGATIVE_VALUE"]` as well.

The code line `m.Log.Warnf("Error while getting value for counter %q, instance: %s, will skip metric: %v", metric.counterPath, metric.instance, err)` is in my opinion a bit misleading, because it is outputting a warning with a message starting with "Error ..."

### Actual behavior

I am trying to understand the behavior of Telegraf with or without the “IgnoredErrors” list set. I occasionally get log entries with the message “A counter with a negative value was detected.” I am trying to limit the log entries to only show real errors, becasue we are ingesting them with Grafana Alloy and Loki. I have therefore put some PDH errors in the IgnoredErrors list. Unfortunately, this makes no difference to the output.

If I understand the Go code in win_perf_counters.go line 536 correctly, then a message is only output if err is not nil and the function isKnownCounterDataError decides whether the message should be outputted as an error or as a warning.

The logs of both cases (with or without IgnoredErrors) shows the message severies as "W!" which means "warning".

### Additional info

I also tried it with `IgnoredErrors = ["PDH_NO_DATA", "PDH_INVALID_DATA", "PDH_CSTATUS_INVALID_DATA", "PDH_CALC_NEGATIVE_VALUE"]` which also makes no difference.

Contributor guide

Open the contributing guide

Research direction

Start in the win_perf_counters.go code around line 536 and compare how IgnoredErrors is used with the warning path shown in the issue logs. Reproduce the plugin with and without PDH_CALC_NEGATIVE_VALUE in the configuration, then verify that the resulting log behavior matches the documented meaning of IgnoredErrors.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre, operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.