Compress replication commands when serializing
Open
Nobody has claimed this yet.
improvement
- Dominant language
- Java
- Stars
- 668
- Forks
- 135
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 7
Description
When a user creates a large amount of big commits in a replicated environment, a replica might suffer from heap shortage, because we serialize commits (PushAsIsCommand to be precise) into JSON and store it into ZooKeeper.
To reduce the risk of running out of memory, we could compress the serialized replication command (zstd?)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how PushAsIsCommand is serialized to JSON and stored in ZooKeeper, then review the corresponding replication read path. Evaluate zstd or another compression approach and verify that compressed commands can be restored correctly without changing replication behavior or worsening heap usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100