[improve][server]Add entry digest in journal log file
- Dominant language
- Java
- Stars
- 2k
- Forks
- 976
- Avg merge
- 6d 15h
- Merged PRs (30d)
- 7
Description
Hi, I recently read BookKeeper journal source code, and I found that BookKeeper journal didn't use any digest to guarante journal data's integrity and consistency.
https://github.com/apache/bookkeeper/blob/2789316c18e12cbb6d17fa4a023410dbad6593a0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java#L811-L885
Due to the lack of digest validation, we don't know whether the journal log data is corrupted or not. So we can only just do replay scan until while loop breaks out or an IOException happens.
https://github.com/apache/bookkeeper/blob/2789316c18e12cbb6d17fa4a023410dbad6593a0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java#L822-L882
If journal data is corrupted(disk data corruption or modified by mistake), and scanning incorrect journal log data happens to produce some incorrect entries(entris that already exists in this bookie), the old entries will be replaced both in RocksDB and entryLog file, which may cause data loss.
https://github.com/apache/bookkeeper/blob/2789316c18e12cbb6d17fa4a023410dbad6593a0/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieImpl.java#L598-L607
ZooKeeper txnLog will append a digest after a record. So I think BookKeeper also need to do that.
https://github.com/apache/zookeeper/blob/9e8296f678a9de8d9c30e3bc086a79c0cd39ef65/zookeeper-server/src/main/java/org/apache/zookeeper/server/persistence/FileTxnLog.java#L276-L327
Contributor guide
Assessment
This issue has not been assessed yet.