influxdata / influxdata/telegraf
Redis Cluster Node Discovery
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
A Redis cluster node `info` provides metrics that are mostly specific to itself, not the cluster. To get a perspective on the whole cluster, you must collect `info` results from each node in the cluster.
This is trivial in a e.g. EC2/self-hosted Redis where you can install the Telegraf agent locally and have the Redis plugin connect to `localhost`.
It is challenging, however, when using a Managed Redis cluster such as AWS ElasiCache or GCP Memorystore, where you're presented with a primary endpoint, and the cluster members will change potentially dynamically as the cluster scales in/out.
The desired feature is for the Telegraf Redis plug-in have a node discovery feature. Similar to a Redis client, the initial connection will perform a `cluster nodes` command to understand the cluster topology:
```
10.64.82.45:11006> cluster nodes
4ce37a099986f2d0465955e2e66937d6893aa0e1 10.64.82.45:11006@16379 myself,master - 0 1713471635000 5 connected 5462-10922
d6eb119a1f050982cc901ae663e7448867e49f7c 10.64.82.46:11005@16379 master - 0 1713471639016 4 connected 10923-16383
3a386fb6930d8f6c1a6536082071eb2f32590d31 10.64.82.46:11007@16379 master - 0 1713471638009 6 connected 0-5461
```
It will then connect to each node to collect the `info` or custom command metrics.
### Expected behavior
With the following example configuration, the Redis plug-in will connect to the initially provided server and perform node/topology discovery. It will then connect to each node in the cluster to collect metrics.
```
[[inputs.redis]]
servers = ["tcp://10.64.82.45:11006"]
node_discovery = true
```
This will result in effectively the following configuration:
```
servers = ["tcp://10.64.82.45:11006","10.64.82.46:11005","10.64.82.46:11007"]
```
and metrics will be pulled for each node.
tls configuration, authentication, custom commands, etc will all be assumed to be the same for each discovered node.
### Actual behavior
Currently, the following configuration:
```
[[inputs.redis]]
servers = ["tcp://10.64.82.45:11006"]
```
will only collect metrics for the specified node.
### Additional info
_No response_
Contributor guide
Research direction
Start with Telegraf's Redis input plugin and its existing server configuration, then review how the initial connection can issue the Redis cluster nodes command. Done means node discovery connects to each discovered cluster member while preserving the configured TLS, authentication, and custom-command settings; validate against the configuration and topology examples in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, redis
- Domain
- databases, distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100