hiero-ledger / hiero-ledger/hiero-consensus-node
Minimize reusable bucket pool overhead in HDHM endWriting
- Dominant language
- Java
- Stars
- 406
- Forks
- 226
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 210
Description
During data flushes to disk, all key to object mappings are written to HDHM (HalfDiskHashMap). HDHM is a set of buckets, each containing a number of mappings. To update a single mapping, the whole bucket should be loaded from disk first, then updated, then written to disk (to a new file). This is all done in `HDHM.endWriting()`.
A bucket is a straightforward Java object, which stores all bucket data in a heap byte buffer. To avoid extra byte buffer allocations, buckets are cached in a pool, see `ReusableBucketPool` for details. This pool is pretty efficient, but still not free. Its overhead to `endWriting()` is noticeable. To minimize it, HDHM may use specially crafted threads, each with its own pre-allocated bucket, so we don't have to use a pool, nor any thread locals.
Contributor guide
Assessment
This issue has not been assessed yet.