fluent / fluent/fluent-bit

Fluent-bit build-in plugin loki output new-line and tab characters

Open
#2,808 21 comments 0 reactions 0 assignees View on GitHub
exempt-stale
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 20h
Merged PRs (30d)
71

Description

## Bug Report

**Describe the bug**
I am sending Java Logs in json format from Fluent-bit to Loki new-line (\n) and tab (\t) characters within record are shown as plain text in Grafana.
Example is \n instead of newline.
It looks like there is an issue with plugin.
The same thing was tested on same instance and application with FluentD. Logs was forwarded from Fluent-bit to FluentD and then to Loki. Grafana shows FluentD json logs correctly.

**To Reproduce**

Just send json exception log message.

**Expected behavior**
Correct view of exception message in Grafana.( new-line, tab instead of `\n` and `\t`)

**Screenshots**
Grafana output of exception from Fluent-bit
![issue](https://user-images.githubusercontent.com/73885977/100366843-04025500-300a-11eb-8c69-c85b4839a409.png)

Grafana output of exception from FluentD (Fluent-bit is collector)
![correct_output](https://user-images.githubusercontent.com/73885977/100366863-0d8bbd00-300a-11eb-837d-5a2e8076173f.png)

**Your Environment**
* Version used: `td-agent-bit.x86_64 1.6.6-1`
* Configuration:
Fluent-bit config:
```
SERVICE]
flush 5
daemon Off
log_level info

# HTTP Server
http_server Off
http_listen 0.0.0.0
http_port 2020

# Storage
storage.metrics on

[INPUT]
name systemd
tag java_app.*
systemd_Filter _SYSTEMD_UNIT=java_app.service

[FILTER]
Name parser
Match java_app.*
Key_Name MESSAGE
Reserve_Data On
Parser json

[OUTPUT]
name loki
match *
host
port 3100
labels job=fluentbit
```

FluentD config (second scenario):
```

@type tdlog
@id output_td
apikey YOUR_API_KEY

auto_create_table

@type file
path /var/log/td-agent/buffer/td


@type file
path /var/log/td-agent/failed_records

@type stdout
@id output_stdout

@type forward
@id input_forward
port 24224
@label @java_app


@type parser
key_name MESSAGE
reserve_data true
remove_key_name_field true

@type json



@type copy

@type loki
url "http://:3100"
extra_labels {"agent":"fluentd"}
line_format json
flush_interval 10s
flush_at_shutdown true
buffer_chunk_limit 1m


@type stdout

```
* Environment name and version (e.g. Kubernetes? What version?):
```
OpenJDK Runtime Environment Corretto-11.0.9.12.1 (build 11.0.9.1+12-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.9.12.1 (build 11.0.9.1+12-LTS, mixed mode)

Grafana v7.3.3 (2489dc4d3a)
```
* Operating System and version:
`Linux ip-.eu-west-.compute.internal 4.14.203-156.332.amzn2.x86_64 # `

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.