FLB_THROTTLE return for outputs
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 58
Description
Fluent Bit's behavior when an output is throttled is not ideal.
If one of my outputs gets a throttling error, I only have 3 choices for what to tell the core pipeline:
* FLB_OK: the entire chunk was successfully sent/handled
* FLB_RETRY: retry the entire chunk up to the max user configured retries
* FLB_ERROR: something went very wrong, don't retry
If I return FLB_RETRY, then that specific chunk will be retried with exponential backoff. However, the pipeline will continue to send my output new chunks. Thus, from the POV of my destination, I am not slowing down and backing off at all, I am continuing to send new data.
Thus, I think we need an FLB_THROTTLE return which would be like a special case for FLB_RETRY. It would tell the core to freeze the output for some period of time, and not send any new chunks. Basically, the core should backoff the entire output instance.
Contributor guide
Assessment
This issue has not been assessed yet.