influxdata / influxdata/telegraf
[parser.enum] regex support
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
Use Case
Using telegraf to parse logs using [inputs.tail] and grok and then having to normalise data, especially constants, I am finding I need to write long lists of regex using [processor.regex]. It would be real nice if I could just use [processor.enum] to do all the normalisation using a regex - nice and clean. This would have to be an optional flag on the enum processor so as not to break backwards compatibility.
Expected behavior
Something like
[processors.enum.mapping.value_mappings]
"^72-20." = "ConfigAgent"
.
.
.
Actual behavior
Having regex on enum would replace long lists of this and make it much more readable:
# License and configuration events (72-20x, 21-*, 05-*, 51-*, 04-*)
[[processors.regex.fields]]
key = "process"
pattern = '^72-20.*'
replacement = "ConfigAgent"
# Port and interface events (62-*, 72-*, 20-*)
[[processors.regex.fields]]
key = "process"
pattern = '^62-.*|^72-.*|^20-.*'
replacement = "Ebra"
# Authentication and authorization events (46-*, 24-*)
[[processors.regex.fields]]
key = "process"
pattern = '^46-.*|^24-.*'
replacement = "Aaa"
# Configuration management events (21-*, 05-*, 51-*, 04-*)
[[processors.regex.fields]]
key = "process"
pattern = '^21-.*|^05-.*|^51-.*|^04-.*'
replacement = "ConfigAgent"
# Performance monitoring events (80-*)
[[processors.regex.fields]]
key = "process"
pattern = '^80-.*'
replacement = "Tca"
.
.
.
Additional info
This will only work for me if I can do inplace field replacements
[[processors.enum]]
[[processors.enum.mapping]]
field = "status"
dest = "status" # same as source
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the processors.enum and processors.regex implementations and their configuration handling. Compare enum mapping behavior with regex replacement, including the requested same-source destination, and define how an optional regex mode preserves existing mappings. Done means regex mappings can normalize fields in place without changing default enum behavior, with coverage for the examples described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100