hiero-ledger / hiero-ledger/hiero-consensus-node
Fast MerkleDb snapshots with no index files
- Dominant language
- Java
- Stars
- 406
- Forks
- 226
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 210
Description
This is a part of ZDT epic: https://github.com/hiero-ledger/hiero-consensus-node/issues/25820
MerkleDb snapshots are relatively fast, but not as fast as needed for ZDT upgrades. With 1B states, database indices may be more than 50Gb in RAM. To write them to disk may take minutes (even with fast local NVMe drives), which is completely unacceptable.
Instead, the idea is to have indices shared between two CN processes, so they don't need to be written / read. Also, there should be a way to make a "fast" snapshot, which only includes data files - they are really fast, since no files are copied, only hard-links created.
Purely theoretical: such fast and lightweight snapshots can still be used outside of ZDT process. Database indexes can be restored by traversing all data files, this is long, but doable. This is not a goal for ZDT, though.
Contributor guide
Assessment
This issue has not been assessed yet.