influxdata / influxdata/telegraf
New plugin type: discovery plugins
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
When it is required to have certain Input plugin for each X of something currently it has to be done in the configuration management tool. This can be too limiting when running in a more dynamic environment such as clouds or kubernetes.
For instance there is a `postgres` Input plugin which requires database connection string. On the AWS cloud RDS databases can come and go and if task is to monitor all of them it is not straightforward how to achieve it.
### Expected behavior
Config option to instantiate plugins dynamically based on the output of some discovery process.
### Actual behavior
-
### Additional info
I didn't give much thought how exactly to achieve it, but as a rough idea:
Every discovery plugin returns equivalent of JSON array of maps. Each map contains configuration values to be used by plugins
```
[[discovery.exec]]
alias = "example"
# simple discovery plugin which just prints discovery result.
command = ["/bin/sh", "-c", "echo '[{host: \"xyz\", interface: \"eth0\"}, {host: \"abc\", interface: \"eth1\"}]'"]
```
Then plugins can be wired to the discovery plugin. When wired that can use map values from discovery:
```
[[input.ping]]
discovery = discovery.exec.example # identifier of the discovery plugin used to instantiate this plugin
alias = "${discovery.host}"
urls = ["${discovery.host}"]
interface = "${discovery.interface}"
```
Should "spawn" 2 `ping` plugins dynamically.
Discovery plugin runs on the the interval as any other plugins and plugins it spawns ideally should stay up unless discovery plugin changes output.
Contributor guide
Research direction
Start by reviewing Telegraf's existing plugin lifecycle and the proposed discovery.exec and input.ping configuration examples. Define how discovery results instantiate and update plugins, including interval handling and retaining unchanged plugins. Done means a concrete design and implementation plan exists for dynamic plugin creation from discovery output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100