kvcache-ai / kvcache-ai/Mooncake
Regarding Mooncake's HA (High Availability) capability, there are still some issues to be resolved as of now.
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
While reviewing the code on the main branch, we analyzed Mooncake's HA capability and identified several outstanding issues:
1. The oplog only supports etcd mode, and each time it must perform a cold start from seq 0. When the oplog volume is large, this becomes a serious problem.
2. Due to the issue above, there is currently no GC (garbage collection) mechanism for oplog data in etcd.
3. The snapshot is only read once during standby initialization and is never read again afterward. If a failover occurs, this will cause a significant gap between the standby and the leader.
4. Currently, snapshot and oplog operate in a mutually exclusive (either/or) mode. When both are enabled simultaneously, the leader/master will stop writing snapshots entirely.
Could the community share its specific plans to address these issues? At this point, it doesn't appear that the current HA implementation offers a robust solution to mitigate these shortcomings. We would greatly appreciate any feedback or insights!
### Before submitting a new issue...
- [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)
Contributor guide
Research direction
Start by reviewing the HA implementation on the main branch and the linked Mooncake documentation, focusing on oplog recovery, oplog garbage collection, standby snapshot refresh, and simultaneous snapshot/oplog operation. Done would require an agreed plan and implementation that addresses all four reported limitations, but the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100