[Feature Request] provide option to throw_exception/block before buffer queue is full
- Dominant language
- Ruby
- Stars
- 13.6k
- Forks
- 1.4k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 20
Description
### Is your feature request related to a problem? Please describe.
Due to some issues I have have been experiencing, I believe it would make sense to provide options to throw_exception/block before the buffer queue is full.
When preforming tests with file buffering I have noted at least these issues:
https://github.com/fluent/fluent-plugin-opensearch/issues/35
https://github.com/fluent/fluent-plugin-opensearch/issues/34
Even with these issues resolved, I believe providing options to throw_exception/block before the buffer queue is full makes sense.
### Describe the solution you'd like
I would suggest adding to the buffering and/or flushing parameters:
https://docs.fluentd.org/configuration/buffer-section#buffering-parameters
https://docs.fluentd.org/configuration/buffer-section#flushing-parameters
Options to set a high-water mark percentage, similar to chunk_full_threshold, and an action, similar to the overflow_action would be required.
For example:
chunk_highwater_action - throw_exception/block
chunk_highwater_threshold - percentage of chunk size threshold for executing chunk_highwater_action
### Describe alternatives you've considered
N/A
### Additional context
N/A
Contributor guide
Assessment
This issue has not been assessed yet.