IntersectMBO / IntersectMBO/ouroboros-consensus

Investigate increased memory usage while syncing with Genesis in 10.5

Open
#1,545 4 comments 0 reactions 0 assignees View on GitHub
Genesis
Dominant language
Haskell
Stars
67
Forks
43
Avg merge
5d 13h
Merged PRs (30d)
43

Description

@karknu has performed some experiments with full syncs using 10.5 (which is not yet released, but using `ouroboros-consensus-0.27` which is intended for 10.5, with some additional patches), and reports an increase in memory, causing the node to crash with a 28GB heap limit:

> The memory increase only happens with genesis enabled but I can’t tell if it is a space leak or genesis just requires slightly more memory (more active peers => more memory)

The goal of this ticket is to find out whether there is a new serious memory leak (which we must fix) or whether the memory requirement just grew slightly (eg it is expected that #1288 should increase peak memory usage *slightly*), which is not a big problem.

---

We know that 10.4.1 definitely doesn't have a (serious) leak; Nick did a full sync, and the [SDET sync tests](https://tests.cardano.intersectmbo.org/test_results/sync_tests.html) also support this. We do know that Genesis uses a bit more memory than Praos (goes away after a restart once caught-up), which would be nice to fully understand, but it is low priority.

Contributor guide

Open the contributing guide

Research direction

Reproduce a full sync with the 10.5 setup using ouroboros-consensus-0.27, comparing Genesis with Praos and the known 10.4.1 behavior. Review the SDET sync test results and measure whether memory continues growing or settles after catch-up. Done means determining whether this is a serious leak or an expected increase in peak memory, with evidence documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
blockchain, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.