ChainSafe / ChainSafe/lodestar

JavaScript heap OOM during catch-up sync

Open
#9,376 2 comments 0 reactions 0 assignees View on GitHub
meta-bug
Dominant language
TypeScript
Stars
1.4k
Forks
483
Avg merge
1d 18h
Merged PRs (30d)
156

Description

### Describe the bug

Running Lodestar via the packaged binary appears to hardcode:

--max-old-space-size=8192

which overrides NODE_OPTIONS and prevents increasing the Node.js heap size during sync.

Environment:
* Install method: packaged binary in /usr/local/bin/lodestar
* Network: mainnet
* OS: 24.04
* Deployment: LXC

Systemd service includes:

Environment=NODE_OPTIONS=--max-old-space-size=16384

Verified present in process environment:

cat /proc/$(pgrep -fn lodestar)/environ | tr '\0' '\n' | grep NODE_OPTIONS

Output:

NODE_OPTIONS=--max-old-space-size=16384

However the actual running process is:

ps -fp $(pgrep -fn lodestar)

showing:

/tmp/caxa/applications/lodestar/.../node --max-old-space-size=8192 ...

During catch-up sync the client consistently crashes around ~8 GB heap usage with:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

This suggests the packaged launcher is forcing an 8192 MB heap limit internally and preventing operators from overriding it.

### Expected behavior

able to catch-up sync without crashing

### Steps to reproduce

_No response_

### Additional context

_No response_

### Operating system

Linux

### Lodestar version or commit hash

Lodestar/v1.42.0/443e0a4

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the packaged launcher used by /usr/local/bin/lodestar and the /tmp/caxa/applications/lodestar/.../node command; compare its heap flags with NODE_OPTIONS during catch-up sync. Done means the packaged binary honors the configured heap size instead of enforcing 8192 MB and no longer exits at that heap limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend, blockchain, devops, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.