Replicate buffer retry configurability from Fluentd
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
**Is your feature request related to a problem? Please describe.**
Currently it is not possible to configure the way output plugins attempt to retry flushing blocks via the scheduler. The default behaviour is to use exponential backoff with jitter, which is great all while you have a constant connection from hosts to the server.
In my use case I'm using Fluent-bit on clients to forward logs to a fluentd central cluster. The client can potentially lose network connection for hours at a time and possibly only reconnect for a short period of time. This means that many blocks would have been trying to flush for a long time and will likely have very large backoff numbers and will not send in the reconnected window.
**Describe the solution you'd like**
Add configuration options similar to that in fluentd which sets values in which the scheduler abides. Here is the configuration in fluentd which would resolve this issue:
retry_wait 5
retry_type periodic
retry_forever true
Documentation for fluentd can be found here:
https://docs.fluentd.org/buffer
Thanks
Contributor guide
Assessment
This issue has not been assessed yet.