influxdata / influxdata/telegraf

Add support for Jolokia2 plugin property-key filtering

Open
#7,497 1 comment 0 reactions 0 assignees View on GitHub
area/jolokia feature request
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

## Enhancement
Jolokia2 plugin takes MBean "object names" in order to determine which metrics to collect from the Jolokia agent. There is a notation that looks like `={||}` which explicitly filters by specific property keys so as to not pull more data than is necessary.

An example MBean name that employs this filtering:
`kafka.network:type=RequestMetrics,name=RequestsPerSec,request={Produce|FetchConsumer},version={0|1|2}`

**Example configuration:**

```
[[inputs.jolokia2_agent.metric]]
name = "kafka_network"
mbean = "kafka.network:type=RequestMetrics,name=ErrorsPerSec,request={CreateTopics|Metadata|Produce},error=*"
paths = ["MeanRate","OneMinuteRate","FiveMinuteRate","FifteenMinuteRate","Count"]
tag_keys = ["error","request"]
```

**Desired output:**
```
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=CreateTopics Count_Count=1 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=UpdateMetadata Count_Count=19 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=Produce Count_Count=61881 1589354866000000000
> kafka_network,error=STALE_CONTROLLER_EPOCH,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=LeaderAndIsr Count_Count=104 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=LeaderAndIsr Count_Count=13 1589354866000000000
```

**Actual output:**
No data

**Using wildcard as the value for `request` (in config) results in data but includes unwanted data:**
```
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=CreateTopics Count_Count=1 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=ApiVersions Count_Count=11 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=Metadata Count_Count=824 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=Produce Count_Count=61881 1589354866000000000
> kafka_network,error=STALE_CONTROLLER_EPOCH,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=LeaderAndIsr Count_Count=104 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=LeaderAndIsr Count_Count=13 1589354866000000000
> kafka_network,error=NONE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=UpdateMetadata Count_Count=19 1589354866000000000
> kafka_network,error=UNKNOWN_TOPIC_OR_PARTITION,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=Metadata Count_Count=1 1589354866000000000
> kafka_network,error=LEADER_NOT_AVAILABLE,host=Sams-MacBook-Pro.local,jolokia_agent_url=http://localhost:8778/jolokia,request=Metadata Count_Count=1 1589354866000000000
```

## Business Need
Jolokia2 plugin is a really useful way for our Java users to collect metrics from their JVM applications. That said, not all metrics are necessary all the time. Allowing for filtering by MBean properties to cherrypick the metrics one wants -- and we already support this somewhat with the `paths` config field -- is crucial for a) saving on costs and b) making the data set more palatable to the human eye.

This plugin is great for monitoring all things JVM for which we don't already have out-of-the-box plugins: JVM itself, Kafka, Spark, Flink, Storm, Flume, NiFi, AirFlow, etc.

Contributor guide

Open the contributing guide

Research direction

Start with the Jolokia2 plugin's MBean matching and configuration handling, using the example [[inputs.jolokia2_agent.metric]] configuration to reproduce the missing data. Verify that property-key notation selects only the listed request values while wildcard matching continues to work and the output contains the requested metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.