Add a new broker routing strategy to handle Realtime table query results consistency issue
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
For a realtime table with >=2 replicas, the broker routing may hit any CONSUMING segment, which might cause user side to see inconsistent results(results may not be constantly increasing) due to the consuming speed for each server.
To implement this, we can:
1. From controller side, define a mapping for every table with the lead(master) partition to server mapping.
2. From broker side, build the new routing strategy to honor this lead partition to instance mapping for the CONSUMING segment.
Contributor guide
Research direction
Begin in the controller and broker paths for realtime tables, focusing on how CONSUMING segments are selected. Trace the existing replica-routing behavior before designing the lead-partition-to-server mapping. Done means queries consistently honor the lead partition’s instance mapping for these segments, with the relevant behavior verified by the project's tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100