Controller missing events due to disconnect
- Dominant language
- Java
- Stars
- 504
- Forks
- 255
- Avg merge
- 17d 47m
- Merged PRs (30d)
- 2
Description
### Describe the bug
If the controller's ZkHelixManager is disconnected for longer than the retry timeout (_waitForConnectedTimeout , defaults to 10seconds) but shorter than the negotiated ZK session timeout, then the controller will not react to any events that occurred during that disconnected period.
The controller will only pick up this change after another event triggers a data refresh and rebalance.
### To Reproduce
1. Disconnect ZKHelixManager
2. Kill participant node
3. Reconnect ZKHelixManager after _waitForConnectedTimeout (10s) exceeded but before session timeout (60s) reached
4. Observe that dead participant still has assignments even though offline from ZK view (no LiveInstance node)
5. Trigger another event in cluster and see that dead participant now has replicas moved off.
### Expected behavior
Controller should respond to current state of cluster after reconnect
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.