quickwit-oss / quickwit-oss/quickwit
Update to latest version of `rust-rdkafka` and remove consumer poll loop
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
In version v0.36, rust-rdkafka has moved from the callback API to the event API. In theory, this should allow us to remove the consumer poll loop and handle polling for new messages and rebalance events in the same task.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the current rust-rdkafka consumer callback and poll loop, then review the v0.36 event API and its rebalance handling. Done means the dependency is updated, polling and rebalance handling run in the same task, and consumer behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, rust
- Domain
- backend, distributed-systems, stream-processing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100