ChainSafe / ChainSafe/lodestar

research: network processor queuing DoS surface for ePBS gossip types

Open
#9,167 7 comments 0 reactions 0 assignees View on GitHub
spec-gloas
Dominant language
TypeScript
Stars
1.4k
Forks
483
Avg merge
1d 16h
Merged PRs (30d)
150

Description

## Problem

The network processor queues gossip messages (attestations, data columns, envelopes) that arrive before their referenced beacon block is known. In ePBS/Gloas, this creates a new DoS surface because:

1. **Data columns and envelopes are large chunks of data** — unlike attestations, these carry significant payload
2. **No way to verify legitimacy before the beacon block arrives** — the block root is untrusted at queue time, so an attacker can submit messages referencing arbitrary roots
3. **Queue pollution** — an attacker can fill the awaiting queue (`MAX_QUEUED_UNKNOWN_BLOCK_GOSSIP_OBJECTS = 16,384`) with garbage, pushing out legitimate messages

### Current Lodestar behavior

- `onPendingGossipsubMessage()` extracts `{slot, root}` from incoming gossip messages
- If the root is unknown, the message is queued in `awaitingMessagesByBlockRoot` pending block arrival
- Messages are pruned per slot via `onClockSlot()` (buffer ~3 slots)
- Peers with unknown roots are penalized via `UnknownBlockSync`
- **No signature verification is performed before queuing**

### Existing mitigations

- Queue length is bounded (`MAX_QUEUED_UNKNOWN_BLOCK_GOSSIP_OBJECTS = 16,384`)
- Slot-based pruning removes stale entries
- Peer downscoring via UnknownBlockSync for unresolvable roots

### Discussion summary

The team discussed several angles ([Discord thread](https://discord.com/channels/593655374469660673/1372263082415493200/1489554903478042804)):

- **@nflaig**: The core issue is that data columns and envelopes are large and unverifiable before the block. Short queue lengths prevent memory exhaustion but also drop legitimate messages. One mitigation: **verify envelope signatures before queuing** — while we don't know the builder identity without the block, at least we know the signer is an on-chain entity (builder or validator), which massively reduces the attack vector. For envelopes specifically, early signature verification may be feasible (low volume, 1 per slot).

- **@twoeths**: Queuing is intentionally cheap (avoids expensive gossip validation). The risk window is narrow — the message's block root must be valid but other fields invalid. The spec requires queuing attestations in this case. Signature verification before queuing is infeasible for attestations (too many messages, signing roots are cached and batch-verified efficiently), but may be feasible for envelopes. Queue lengths are already bounded. Also noted: peer downscoring for invalid queued messages only happens after dequeue, not at queue time.

- **@nflaig** referenced an [Eth R&D discussion](https://discord.com/channels/595666850260713488/874767108809031740/1478817324978016398) about preventing queue pollution from junk data.

### Related PRs

- #9059 — Generalize NetworkProcessor awaiting-block mechanism for all gossip types
- #9161 — Fix: avoid eager parent sync for pre-gloas beacon blocks
- #9162 — Fix: restore seen root dedupe for column created block inputs
- #9166 — Fix: process fulu data columns without waiting for block

## Proposed research

Investigate how other consensus clients handle pre-block gossip message queuing, specifically:

1. **What messages are queued** before the beacon block is known (attestations, data columns, envelopes, etc.)?
2. **What validation is performed before queuing** (signature checks, proof verification, none)?
3. **Queue bounds and eviction strategies** (per-root limits, global limits, priority, etc.)?
4. **Peer scoring** — do they downscore at queue time or only after validation?

### Clients to check

- [ ] **Prysm** (Go) — likely the only other client with Gloas implementation
- [ ] **Lighthouse** (Rust) — mature queue/reprocessing design
- [ ] **Teku** (Java)
- [ ] **Nimbus** (Nim)
- [ ] **Grandine** (Rust)

> **Note:** As of April 2026, likely only Prysm has a Gloas implementation. For others, reviewing their fulu/PeerDAS queuing patterns is still valuable since the same DoS surface exists for data columns.

## Possible mitigations to evaluate

1. **Early envelope signature verification** — verify BLS signature before queuing (feasible for envelopes, 1 per slot)
2. **Per-topic queue limits** — separate bounds for attestations vs columns vs envelopes
3. **Per-root message limits** — cap messages queued per unknown root to prevent single-root flooding
4. **Priority eviction** — prefer envelopes/blocks over attestations when queue is full
5. **Lightweight proof-of-work or rate limiting** per peer before queue admission

Contributor guide

Open the contributing guide

Research direction

Start with onPendingGossipsubMessage(), onClockSlot(), awaitingMessagesByBlockRoot, and MAX_QUEUED_UNKNOWN_BLOCK_GOSSIP_OBJECTS, then inspect PRs #9059, #9162, and #9166. Compare Prysm, Lighthouse, Teku, Nimbus, and Grandine for queued message types, pre-queue validation, bounds, eviction, and peer scoring. Done means a documented comparison and evidence-based assessment of the proposed mitigations.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java, nim, rust, typescript
Domain
blockchain, networking, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.