hiero-ledger / hiero-ledger/hiero-consensus-node

Reduce `VirtualNodeCache` memory footprint

Open
#5,216 1 comment 0 reactions 1 assignee Claimed by @OlegMazurov View on GitHub
Performance Platform Platform Data Structures Platform Virtual Map
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

Memory overhead of `VirtualNodeCache` comes from the following sources:

- `ConcurrentHashMap$Node`s of three maps
- `Mutation` fields (`key`, `value`, `deleted`, `version`)
- three `ConcurrentArray`s keeping extra references to `Mutation`s and forcing them to maintain a separate `key`

An ad hoc implementation of a concurrent map may significantly reduce data redundancy without sacrificing performance.
Although the current design assumes only one writer with multiple concurrent readers, the implementation offloads concurrency control to `ConcurrentHashMap` and is robust against concurrent updates (`VirtualRootNode` employing `VirtualNodeCache` is not). It would be good to preserve that.

Another requirement is implementation of parallel map entry processing without incurring more memory overhead as is the case with `ConcurrentHashMap`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.