Enhance kafka commit
Open
help wanted
question
- Dominant language
- Rust
- Stars
- 879
- Forks
- 61
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I think, it could be very useful to enhance "Kafka output" with a kind of timer option and associate it with the actual _kafka_coalesce_ parameter.
```
kafka_coalesce=100
kafka_flush_interval=5s
```
If one of theses conditions is met, commit the batch to Kafka.
Thanks
Contributor guide
Research direction
Start by locating the Kafka output implementation and the handling of kafka_coalesce; trace how batches are committed and where a flush interval could be integrated. Confirm the expected behavior for kafka_flush_interval=5s, then verify that a batch is committed when either the record-count or timer condition is met.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, rust
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100