Built in support for Multiline JSON
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
**Is your feature request related to a problem? Please describe.**
It is very common to have JSON span multiple lines. This is an example log:
```json
{
"level": "ERROR",
"message": "This is my message",
"timestamp": "06:45:03.376"
}
```
**Describe the solution you'd like**
A built-in multiline parser similar to those for docker, cri, go etc. https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/multiline-parsing.
```
[INPUT]
name tail
path /var/log/app.log
multiline.parser json
```
**Describe alternatives you've considered**
I was unable to get it working using the new multiline core mechanism. Every solution I had resulted in a multiline JSON being sent to the JSON parser that doesn't support it. I was able to get the workarounds discussed here for the old multiline to work: https://github.com/fluent/fluent-bit/issues/2418
```
[PARSER]
Name custom_json_parser
Format json
[PARSER]
Name multiline_json
Format regex
Regex ^(?{.*)
[INPUT]
Name tail
Multiline On
Parser_Firstline multiline_json
path /var/log/app.log
[FILTER]
Name parser
Match *
Key_Name data
Parser custom_json_parser
Unescape_Key On
```
Contributor guide
Assessment
This issue has not been assessed yet.