coinbase / coinbase/mesh-bitcoin

Mainnet node takes 64+GB ram to run effectively

Open
#67 2 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Go
Stars
119
Forks
127
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
The btc mainnet fully synced node consumes upward of 64GB ram to run (The recommended ram size on docs is 16GB). We have tried running on small memory machines (<20GB or <40GB) by tweaking the badger config and using swap memory instead but giving any lesser amount of ram causes the indexing/block adding to be extremely slow. Newer blocks would constantly lag behind from raw node by about 10min+. Moreover small memory nodes will constantly crash with "Error EOF" issues and are very slow to get data via Rosetta APIs.

Current badger settings
MaxTableSize = PerformanceMaxTableSize (Also tried the default size but no effect)
MaxLogSize = PerformanceLogSize (Also tried the default size but no effect)
CacheSize = 6GB (This seems to have the biggest impact on reducing memory usage, but smaller memory causes slow indexing)

**To Reproduce**
Run a fully synced mainnet node with swap disabled.

**Expected behavior**
Expecting to be able to run the node in small machines.

**Additional context**
We should either update the docs to reflect the real usage value or some more tuned settings to run this effectively at scale.

Some screenshots of memory usage:
(Note: the sudden drops are Rosetta crashes sadly, even after giving such a big vm)
![image](https://user-images.githubusercontent.com/9801383/104830808-a1209200-58c5-11eb-9c23-b6234998debc.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing a fully synced Bitcoin mainnet node with swap disabled, using the reported Badger settings and memory sizes. Compare memory use, indexing lag, crashes, and Rosetta API responsiveness, then determine whether the result should be a documentation update or tuned settings; done means small-memory operation is effective or the documented requirements are accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.