[server] ZooKeeper Removal and Embedded Raft Metadata Backend
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Motivation
Fluss currently depends on an external Apache ZooKeeper ensemble for cluster coordination, metadata persistence, sequence IDs, TabletServer liveness, dynamic config, and ACLs. Clients never talk to ZooKeeper (`bootstrap.servers` + Admin RPC only), but **every server deployment** still needs a healthy ZK quorum.
This extra stateful system is a major operational burden:
- A production Fluss cluster cannot start without ZooKeeper (quorum, tick time, session timeout, SASL/ACL, backups).
- ZK failures cascade into coordinator leadership, TabletServer registration, and metadata availability.
- Container, Kubernetes, edge, and single-node developer setups pay for a second service that exists only for coordination.
- Extra network hops and a second failure domain sit on the metadata path.
This matches the published architecture direction: ZooKeeper is transitional and should be replaced by **KvStore for metadata** and **Raft for coordination**. Kafka KRaft and similar embedded-journal designs show the same class of change is viable when the consensus group stays small and the data plane stays out of the quorum.
**Goals**
1. Default deploy is Fluss-only: CoordinatorServers form a Raft group; no ZooKeeper process is required.
2. A supported dual-backend window so existing ZK clusters can keep running and migrate with tools, not a flag day.
3. No change to client APIs, log/KV format, or `bootstrap.servers`.
4. A later major (Phase 6) removes ZK mode after soak. That deletion is **not** part of the first release of this work.
Tracking FIP: **FIP-52**.
### Solution
Replace ZooKeeper with an **embedded Apache Ratis** Raft quorum on CoordinatorServers, plus a RocksDB-backed metadata KvStore.
- Only coordinators are Raft voters. TabletServers are observers: they discover a coordinator from `coordinator.seed.nodes`, register/heartbeat over RPC, and consume a pull-based `FetchMetadata` stream.
- Consensus is KRaft-style commit-apply: the store is mutated only after a log entry is committed. The Raft term is the coordinator epoch.
- Backend selection is exclusive:
- `zookeeper.address` only → ZooKeeper (deprecated)
- `coordinator.seed.nodes` only → Raft
- both keys → fail fast
- neither key → **Raft** (default)
- Public client APIs, binary log/KV format, and `bootstrap.servers` stay unchanged. New operator CLI: `fluss raft *` and `fluss zk-to-raft-*`.
- This issue / FIP covers Phases 1–5 (dual-support, testing, migration tools, docs, Raft as default). **Phase 6 (delete ZooKeeper mode and the shaded ZK dependency) is out of the first merge.**
**Related materials**
- FIP-52: [`FIP-52`](https://cwiki.apache.org/confluence/spaces/FLUSS/pages/449282216/FIP-52+ZooKeeper+Removal+and+Embedded+Raft+Metadata+Backend)
- Implementation branch: `feature/zookeeper-removal-raft`
- User docs: `website/docs/install-deploy/deploying-with-raft.md`, `website/docs/install-deploy/migrate-zk-to-raft.md`
### Anything else?
_No response_
### Willingness to contribute
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with FIP-52 and the implementation branch feature/zookeeper-removal-raft, then read the referenced deployment and migration docs: website/docs/install-deploy/deploying-with-raft.md and website/docs/install-deploy/migrate-zk-to-raft.md. This work is done when Phases 1–5 provide dual-backend support, migration tools and documentation, with Raft as the default; Phase 6 removal of ZooKeeper is excluded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100