fluent / fluent/fluentd

Regular expression support for record_transformer's keep_keys and remove_keys

Open
#3,219 0 comments 5 reactions 0 assignees View on GitHub
enhancement
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Check [CONTRIBUTING guideline](https://github.com/fluent/fluentd/blob/master/CONTRIBUTING.md) first and here is the list to help us investigate the problem.

**Is your feature request related to a problem? Please describe.**

I have a use-case where I am trying to filter out thousands of fields from log records. The fields-to-be-discarded have a fairly predictable name pattern, e.g., `my-field-$N`.

**Describe the solution you'd like**

Add regular expression support to record_transformer's keep_keys and remove_keys.

Following the example above, one could use remove keys as such:

```
remove_keys /^my-field-[0-9]+$/
```

**Describe alternatives you've considered**

In this case, the fields I wanted to retain numbered around 150, so I ended up using keep_keys instead.

**Additional context**

N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.