influxdata / influxdata/telegraf

Redis Cluster Node Discovery

Open
#15,199 1 comment 0 reactions 0 assignees View on GitHub
area/redis feature request help wanted size/m
Dominant language
Go
Stars
17.8k
Forks
5.8k
Avg merge
1d 20h
Merged PRs (30d)
161

Description

### Use Case

A Redis cluster node `info` provides metrics that are mostly specific to itself, not the cluster. To get a perspective on the whole cluster, you must collect `info` results from each node in the cluster.

This is trivial in a e.g. EC2/self-hosted Redis where you can install the Telegraf agent locally and have the Redis plugin connect to `localhost`.

It is challenging, however, when using a Managed Redis cluster such as AWS ElasiCache or GCP Memorystore, where you're presented with a primary endpoint, and the cluster members will change potentially dynamically as the cluster scales in/out.

The desired feature is for the Telegraf Redis plug-in have a node discovery feature. Similar to a Redis client, the initial connection will perform a `cluster nodes` command to understand the cluster topology:

```
10.64.82.45:11006> cluster nodes
4ce37a099986f2d0465955e2e66937d6893aa0e1 10.64.82.45:11006@16379 myself,master - 0 1713471635000 5 connected 5462-10922
d6eb119a1f050982cc901ae663e7448867e49f7c 10.64.82.46:11005@16379 master - 0 1713471639016 4 connected 10923-16383
3a386fb6930d8f6c1a6536082071eb2f32590d31 10.64.82.46:11007@16379 master - 0 1713471638009 6 connected 0-5461
```

It will then connect to each node to collect the `info` or custom command metrics.

### Expected behavior

With the following example configuration, the Redis plug-in will connect to the initially provided server and perform node/topology discovery. It will then connect to each node in the cluster to collect metrics.

```
[[inputs.redis]]
servers = ["tcp://10.64.82.45:11006"]
node_discovery = true
```

This will result in effectively the following configuration:

```
servers = ["tcp://10.64.82.45:11006","10.64.82.46:11005","10.64.82.46:11007"]
```

and metrics will be pulled for each node.

tls configuration, authentication, custom commands, etc will all be assumed to be the same for each discovered node.

### Actual behavior

Currently, the following configuration:

```
[[inputs.redis]]
servers = ["tcp://10.64.82.45:11006"]
```

will only collect metrics for the specified node.

### Additional info

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with Telegraf's Redis input plugin and its existing server configuration, then review how the initial connection can issue the Redis cluster nodes command. Done means node discovery connects to each discovered cluster member while preserving the configured TLS, authentication, and custom-command settings; validate against the configuration and topology examples in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, redis
Domain
databases, distributed-systems, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.