Multiline Filter can lose logs on shutdown
Open
enhancement
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
## Bug Report
For context on the multiline filter see: https://github.com/fluent/fluent-bit/issues/4309
On shutdown if there is a pending multiline log in the buffer, it can be lost. I still need to spend some more time double checking exactly in what cases this happens, but I've seen it. Run Fluent Bit, send it multiline logs and use the filter and then send it a SIGTERM, and the last multiline is not always delivered.
I think this is because even if the multiline library flushes the data before shutdown, the in_emitter instance is already paused and so the records are never emitted. I need to investigate this more and check, but that's my hypothesis.
Contributor guide
Assessment
This issue has not been assessed yet.