hiero-ledger / hiero-ledger/hiero-consensus-node

Hashes and leaves can be written to disk on multiple threads

Open
#27,259 0 comments 0 reactions 1 assignee Claimed by @artemananiev View on GitHub
Improvement Performance Platform Platform Virtual Map
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

`MerkleDbDataSource.saveRecords()` splits the process into three parts:

* Write hashes to "hash chunks store"
* Write leaves to "leaves store"
* Write key to path mappings to "keys store"

The third one is heavily multi-threaded. It includes loading object key buckets, updating them, and then writing to disk, all implemented as separate tasks with dependencies. However, the first and the second parts are both single-threaded. They become the bottleneck. All keys/path mappings have been already updated, there are threads to utilize, but hashes and leaves are still being written. It increases total flush time significantly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.