influxdata / influxdata/telegraf

Add option to add additional info to internal statistics

Open
#17,962 10 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Use Case

Internal statistics related to an input plugin only add `alias` or `_id` as additional info to identify the plugin when they are set in the config. This is mostly not enough to query them afterwards in the database.

An option to also add the labels or tags configured on the plugin to these statistics would help.

#### Example

I have an `inputs.snmp` for each device I collect info from. They are grouped by addition of a `role` tag. I also added the device name as alias value to have distinct logs and `internal_gather` metrics.

```toml
[[inputs.snmp]]
alias = "fw001"
agents = ["10.0.0.1:161"]

[inputs.snmp.tags]
role = "firewall"
```

When I want to see error counters from all the firewalls, then I need to do the following:

1. Gather all hostnames where role equals firewall.
2. Prepare a huge query with all the retrieved names: (pseudo code)
```sql
SELECT errors FROM internal_gather
WHERE input = 'snmp' AND REGEXP_LIKE(alias, '^(fw001|fw002|fw003|...)$');
```

You can imagine this could be a lot more performant when a query like this would be possible instead:

```sql
SELECT errors FROM internal_gather
WHERE input = 'snmp' AND role = 'firewall';
```

### Expected behavior

A global config option to enable addition of plugin-level defined tags or labels:

```diff
- internal_gather,alias=fw001,input=snmp,version=1.36.4 errors=0i,gather_time_ns=0i,gather_timeouts=0i,metrics_gathered=0i
+ internal_gather,alias=fw001,input=snmp,role=firewall,version=1.36.4 errors=0i,gather_time_ns=0i,gather_timeouts=0i,metrics_gathered=0i
```

### Actual behavior

No additional info on internal plugin metrics.

### Additional info

Related spec [TSD-011 Internal plugin statistics collection](https://github.com/influxdata/telegraf/blob/master/docs/specs/tsd-011-internal-plugin-statistics.md) says:

> The injected collector instance must add all relevant model-level information such as an optional `alias` setting or `tags` settings.

However, it was requested in https://github.com/influxdata/telegraf/pull/17749#discussion_r2414293656 to remove tags copy, rendering the current implementation to not adhere to the spec.

Contributor guide

Open the contributing guide

Research direction

Start with docs/specs/tsd-011-internal-plugin-statistics.md and the discussion in pull request 17749, then locate the internal plugin statistics collector and its tests. Confirm how a global option should control copying plugin tags or labels, and verify that enabled statistics include those fields while existing output remains unchanged when it is disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.