IntersectMBO / IntersectMBO/ouroboros-consensus
Investigate increased memory usage while syncing with Genesis in 10.5
- Dominant language
- Haskell
- Stars
- 67
- Forks
- 43
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 43
Description
@karknu has performed some experiments with full syncs using 10.5 (which is not yet released, but using `ouroboros-consensus-0.27` which is intended for 10.5, with some additional patches), and reports an increase in memory, causing the node to crash with a 28GB heap limit:
> The memory increase only happens with genesis enabled but I can’t tell if it is a space leak or genesis just requires slightly more memory (more active peers => more memory)
The goal of this ticket is to find out whether there is a new serious memory leak (which we must fix) or whether the memory requirement just grew slightly (eg it is expected that #1288 should increase peak memory usage *slightly*), which is not a big problem.
---
We know that 10.4.1 definitely doesn't have a (serious) leak; Nick did a full sync, and the [SDET sync tests](https://tests.cardano.intersectmbo.org/test_results/sync_tests.html) also support this. We do know that Genesis uses a bit more memory than Praos (goes away after a restart once caught-up), which would be nice to fully understand, but it is low priority.
Contributor guide
Research direction
Reproduce a full sync with the 10.5 setup using ouroboros-consensus-0.27, comparing Genesis with Praos and the known 10.4.1 behavior. Review the SDET sync test results and measure whether memory continues growing or settles after catch-up. Done means determining whether this is a serious leak or an expected increase in peak memory, with evidence documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- blockchain, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100