quickwit-oss / quickwit-oss/quickwit
Support Kafka's group instance id.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 11.7k
- Forks
- 597
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 37
Description
Kafka has a feature called static consumer membership, associated to a property called group.instance.id.
It allows to declare a specific consumer id to a consumer. When leaving the cluster, the consumer does not advertise it is leaving.
Until its session timeout, it can then rejoin and resume its work were it was.
The benefit is that event that cause a node to rapidly leave and rejoin the cluster will not trigger a rebalance:
- a rolling upgrade
- a node crashing/restarting in a k8s deployment
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating Quickwit’s Kafka consumer integration and tracing how consumer identity, session timeouts, and rejoining are handled. Use the issue’s static-membership behavior as the acceptance criteria: support Kafka’s group.instance.id so brief departures during restarts or rolling upgrades do not trigger a rebalance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, rust
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100