fluent / fluent/fluentd

Allow compressed data for tcp/udp/syslog inputs

Open
#3,728 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Ruby
Stars
13.6k
Forks
1.4k
Avg merge
1d 3h
Merged PRs (30d)
20

Description

### Is your feature request related to a problem? Please describe.

To reduce network congestion, we want to enabled log compression when forwarding logs to our fluentd server. Fortunately, most of our tools can handle sending compressed syslog or raw data. Unfortunately, fluentd doesn't and takes the data "as-is". It appears in buffer files as strings containing only escaped unicode characters (ex: `=\u0001\u0000`).

### Describe the solution you'd like

It would be great if we could setup tcp, udp or syslog inputs to receive compressed data. Having a `compress` directive accepting as values a boolean (`true/false`) or a compression algorithm (`none/gzip/...`).

```

@type tcp
port 3333
bind 0.0.0.0
tag input.myapp
compress gzip

@type none

```

### Describe alternatives you've considered

So far we managed to activate compression on our data by logging our tools activities into files and by using a local td-agent tailing each file and forwarding their contents to our td-agent central server with compression enabled.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.