hiero-ledger / hiero-ledger/hiero-consensus-node

Minimize reusable bucket pool overhead in HDHM endWriting

Open
#22,762 1 comment 0 reactions 1 assignee Claimed by @artemananiev View on GitHub
Performance Platform Platform Data Structures Platform Virtual Map
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

During data flushes to disk, all key to object mappings are written to HDHM (HalfDiskHashMap). HDHM is a set of buckets, each containing a number of mappings. To update a single mapping, the whole bucket should be loaded from disk first, then updated, then written to disk (to a new file). This is all done in `HDHM.endWriting()`.

A bucket is a straightforward Java object, which stores all bucket data in a heap byte buffer. To avoid extra byte buffer allocations, buckets are cached in a pool, see `ReusableBucketPool` for details. This pool is pretty efficient, but still not free. Its overhead to `endWriting()` is noticeable. To minimize it, HDHM may use specially crafted threads, each with its own pre-allocated bucket, so we don't have to use a pool, nor any thread locals.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.