hiero-ledger / hiero-ledger/hiero-consensus-node
Reduce reconnect chunk count (taller chunks) and measure wall-clock impact
- Dominant language
- Java
- Stars
- 406
- Forks
- 226
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 210
Description
`TopToBottomTraversalOrder` processes the virtual tree in chunks. Chunk count is a
function of tree height (`lastLeafRank - chunkRootRank`, governed by
`DEFAULT_CHUNK_HEIGHT`); at 300M accounts this yields ~128–256 chunks per reconnect.
Each chunk boundary is a transition where the learner depends on the teacher returning
the next chunk's initial internals before its leaf phase can proceed. More chunks =
more transitions = more per-transition cost (network RTT plus any fixed per-chunk work
on teacher/learner). A single RTT may be worth thousands of nodes
transferred, so fewer-but-taller chunks may reduce reconnect wall-clock.
### Hypothesis
Reducing chunk count reduces reconnect wall-clock by eliminating per-transition cost.
### Plan
1. **Measure first (no code change beyond instrumentation):** sum the inter-chunk
transition gap across a representative floor reconnect — the wall-clock between the
last leaf of chunk K and the first leaf of chunk K+1. This is the ceiling on what
chunk-count reduction can buy.
- If transitions are >~15% of floor wall-clock → proceed to sweep.
- If <~5% → chunk-count reduction won't help; close.
2. **If warranted:** make chunk height/count tunable and sweep (e.g. 128 → 64 → 32).
3. Record reconnect wall-clock (`ReconnectStateLearner` `timeInSeconds`)` and **peak heap** per configuration.
### Cost / risk
Taller chunks grow the initial internal seed (`2^skipRanks`, ~2048 today) and per-chunk
set sizes exponentially with height. **Watch peak heap** on every sweep point — this
knob may trade stall reduction for a memory cost without moving wall-clock.
Contributor guide
Assessment
This issue has not been assessed yet.