blockblaz / blockblaz/zeam

crash: integer overflow in SSZ serializedSize during processCachedDescendants (catch-up sync)

Open
#696 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Zig
Stars
97
Forks
39
PR merge metrics
No merged PRs in 30d

Description

## Crash: `integer overflow` in SSZ `serializedSize` during `processCachedDescendants`

**Severity:** Critical — node crashes repeatedly during catch-up sync, preventing participation

## Stack Trace

```
thread 1 panic: integer overflow
ssz-0.0.9/src/lib.zig:45:9: 0x17d5ede in serializedSize__anon_10551 (zeam)
ssz-0.0.9/src/lib.zig:89:43: 0x18e5e50 in serialize__anon_1654536 (zeam)
pkgs/database/src/rocksdb.zig:209:30: 0x18ebff3 in onBlock (zeam)
pkgs/node/src/node.zig:410:57: 0x1a99f30 in processCachedDescendants (zeam)
pkgs/node/src/node.zig:489:42: 0x17f1254 in onInterval (zeam)
pkgs/node/src/utils.zig:42:33: 0x1b6c398 in callback (zeam)
libxev/src/backend/io_uring.zig:806:29: 0x17ba9eb in run (zeam)
pkgs/cli/src/node.zig:426:27: 0x181404a in run (zeam)
pkgs/cli/src/main.zig:780:26: 0x181c8bb in mainInner (zeam)
pkgs/cli/src/main.zig:241:14: 0x17b7fdd in main (zeam)
```

The crash occurs consistently — same stack trace on every restart, hitting the same code path in `processCachedDescendants → onBlock → ssz serialize`.

## Observed Behaviour

1. Node starts with checkpoint sync from slot `10570`.
2. Connects to peers and begins catch-up block sync.
3. During `processCachedDescendants` (triggered by `onInterval`), the SSZ `serializedSize` computation overflows an integer when attempting to serialize a block for RocksDB storage.
4. Process panics and restarts. **This repeats on every restart** — the problematic block is re-fetched and re-encountered on each run.

## Additional Errors (same session)

- `error.UnknownSourceBlock` — repeated across many blocks during attestation validation in `chain.zig`
- `error.OutOfMemory` — `Failed to process cached block` warnings during the same catch-up window

## Likely Root Cause

The SSZ library uses `u32` offsets, so `serializedSize` can overflow if the serialized size of a block exceeds `2^32 - 1` bytes (~4 GiB). The most plausible cause is a block with an abnormally large list field (e.g. attestations, deposits, or BLS changes) — possibly the node is accepting or constructing a malformed/oversized block during sync.

Could also indicate that multiple blocks are being concatenated or a slice is not properly bounded before being passed to `serialize`.

## Reproduction

- Checkpoint sync from `https://leanpoint.leanroadmap.org/lean/v0/states/finalized` (slot `10570`)
- Devnet3 with 9 peers (ethlambda ×5, gean ×1, nlean ×1, qlean ×1, lantern ×1)
- Crash occurs reproducibly within ~1 minute of startup

## Suggested Fix

1. Add a bounds check in `rocksdb.zig:onBlock` before calling `ssz.serialize` — validate block size is within `u32` range and log + skip if not.
2. Investigate why a block in the devnet3 chain has an abnormally large serialized size.
3. Check if `processCachedDescendants` is accumulating blocks/data across iterations without bounds.

## Environment

- zeam devnet3 (v0.3.3)
- Checkpoint sync mode
- `attestation-committee-count`: not set (defaults to spec value)

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the checkpoint-sync crash, then trace pkgs/node/src/node.zig:processCachedDescendants into pkgs/database/src/rocksdb.zig:onBlock and ssz-0.0.9/src/lib.zig:serializedSize and serialize. Check the failing block and the suggested u32 size boundary or accumulation path. Done means catch-up no longer panics on this input and the oversized or malformed data is handled visibly.

Written by the indexing model from the issue text.

Assessment

Tech stack
zig
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.