fluent / fluent/fluentd

Feature Request: Allow filtering of the record on output emit rather than before

Open
#3,045 1 comment 0 reactions 1 assignee Claimed by @repeatedly View on GitHub
enhancement
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

Check [CONTRIBUTING guideline](https://github.com/fluent/fluentd/blob/master/CONTRIBUTING.md) first and here is the list to help us investigate the problem.

**Is your feature request related to a problem? Please describe.**

We are storing fields that we do not need.

**Describe the solution you'd like**

Would be nice to have a universal, simple way to filter out fields on output. Something built into the Output base rather than in every plugin.

Ideally it would be a simple `remove_keys A,B,C` parameter in output (or in buffer block)

**Describe alternatives you've considered**

An obvious alternative is to run a `` block before the output, but that causes problems because in many case I may need those fields in output to configure the outpuyt

Some examples:
* I want to output to an Elasticsearch to an arbitrary index - i have an `index` field, but I do not need it in my record in ES.
* I want to write to S3 bucket using a field as part of the hostname but not include that field in the record.

**Additional context**

N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.