fluent / fluent/fluent-bit

Multiline Filter can lose logs on shutdown

Open
#4,918 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 20h
Merged PRs (30d)
71

Description

## Bug Report

For context on the multiline filter see: https://github.com/fluent/fluent-bit/issues/4309

On shutdown if there is a pending multiline log in the buffer, it can be lost. I still need to spend some more time double checking exactly in what cases this happens, but I've seen it. Run Fluent Bit, send it multiline logs and use the filter and then send it a SIGTERM, and the last multiline is not always delivered.

I think this is because even if the multiline library flushes the data before shutdown, the in_emitter instance is already paused and so the records are never emitted. I need to investigate this more and check, but that's my hypothesis.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.