[feature-request] Add event based processing for input plugins
- Dominant language
- Go
- Stars
- 202
- Forks
- 61
- Avg merge
- 7d 20h
- Merged PRs (30d)
- 2
Description
Currently input plugins run on a schedule and are called once per second. This is not ideal for dealing with streaming / event driven data, as that data may reach the input plugin at an uneven rate. Events should ideally be collected as soon as they are available (I believe this is how other plugins work, like the `tail` plugin).
With the current implementation, the fastest rate data chunks could be consumed would be every second.
I believe this would require a wrapper of `flb_input_set_collector_event` in the plugin API, however from what I can tell, Fluent Bit core would probably need to be updated as well since the proxy plugin which is used to wrap Go plugins is responsible for setting this timer, i.e. by calling `flb_input_set_collector_time` (here: https://github.com/fluent/fluent-bit/blob/master/src/flb_plugin_proxy.c#L151-L155).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.