influxdata / influxdata/telegraf
Redis plugin: cannot use EVAL using custom commands
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
Feature Request
The following PR #8196 allows the user to specify commands that are run in redis capturing their output as a metric.
The problem with it is that we cannot use EVAL to run some lua code that gathers those metrics. The usage of EVAL would be really convenient because you can actually script the metrics gathering logic. For example if you want to gather some metrics dynamically (let's say run a KEYS something and then for each key run other commands to gather some specific metrics) then your best option is to script it with Lua that is run through EVAL in redis.
This use case was also discussed in the issue #1027 that lead to the PR above.
Specifically the problem is that the required second argument for EVAL is an integer with the number of arguments. This is an example of the simplest EVAL command you could run
EVAL 'return 1' 0
If you put it in the commands section something like:
[[inputs.redis.commands]]
command = ["EVAL", "return 1", 0]
You're going to receive the error (because you can't mix types in array in toml)
2021-04-08T09:39:41Z E! [telegraf] Error running agent: Error loading config file /etc/telegraf/telegraf.conf: error parsing redis, line 67: (redis.RedisCommand.Command) array can't contain multiple types
if you try to fix it by using a string as last element like this:
[[inputs.redis.commands]]
command = ["EVAL", "return 1", "0"]
you'll have the following error (because EVAL expects a number and not a string as second argument)
2021-04-08T09:42:02Z E! [inputs.redis] Error in plugin: redis: unexpected type=int64 for String
Proposal:
I don't think there is a strong reason to maintain it as array of single statements that compose a command. It looks like it has been done mainly because that's the input expected by redis client, but IMO it doesn't feel natural from a configuration point of view.
My proposal is to deprecate the command array, create another parameter where you can actually specify the whole command in one string.
Something like this:
[[inputs.redis.commands]]
commands = ["EVAL 'return 1' 0"]
This would overcome the "cannot contain multiple types" and allow the consumer to write the script in one script as it would normally be expected.
NOTE: also, we would need to be able to configure to interpret the output not as a metric, but as influx lines. This will close the loop since the lua output could be directly the influx lines (that's the use case when you run a script dynamically gathering N metrics)
Current behavior:
We're not able to run EVAL commands in redis plugin custom commands due to the limitation explained above
Desired behavior:
We should be able to run EVAL commands in redis plugin custom commands
Use case:
Gather custom metrics with Lua logic run in redis through redis plugin custom commands.
Workaround:
Do not use redis plugin and go with [[inputs.exec]] executing a custom bash script using redis-cli EVAL 'your-lua-code' 0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Redis plugin custom-command handling in plugins/inputs/redis/redis.go and review how the command array is parsed and executed. Define the configuration and output behavior needed for EVAL, including the proposed influx-line handling; done means Redis Lua commands can run without TOML type errors and their output is handled as configured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, lua, redis
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100