ChainSafe / ChainSafe/lodestar

Reorgs Leave Epoch+2 Duty Preparation Stale for Up to One Epoch

Open
#9,963 1 comment 0 reactions 0 assignees View on GitHub
meta-bug
Dominant language
TypeScript
Stars
1.4k
Forks
483
Avg merge
1d 16h
Merged PRs (30d)
150

Description

### Describe the bug

## Background

Lodestar precomputes attester duties two epochs ahead. This is done through `nextShuffling` in `packages/state-transition/src/cache/epochCache.ts` (around L631-760). No other consensus client precomputes this far ahead.

This precompute drives two VC-side preparation steps:

- Which subnets to subscribe to
- Which validators are selected for distributed aggregation

When a reorg occurs, the canonical chain may change. The dependent root used to compute epoch+2 duties may no longer be valid. The subnet subscriptions and aggregation selections built from that root become stale.

## Finding

Lodestar subscribes to head events and refreshes duties on reorg. The refresh behavior differs by epoch distance.

For epoch+1: the refresh is immediate. The new dependent root is processed right away.

For epoch+2: the refresh is deferred. The new root is stored in `pendingDependentRootByEpoch`. It is not acted on until `prepareForNextEpoch` runs at the epoch boundary. There is one exception: if the reorg lands on the last slot of the epoch, `handleAttesterDutiesReorg(...)` runs immediately.

**Code locations** (`packages/validator/src/services/attestationDuties.ts`):

- `onNewHead()` (around L306-357): handles the epoch+1 root immediately; stores the epoch+2 root as pending (around L321-331).
- `prepareForNextEpoch()` (around L131-148): applies the pending root at the epoch boundary.

**Worst-case timeline** (reorg at epoch N, slot 0):

```
slot 0: reorg occurs; epoch N+2 dependent root changes
new root stored as pending, no refresh
slot 1-30: subnet subscriptions and aggregation selection use stale assignments
slot 31: prepareForNextEpoch detects the mismatch and triggers a refresh
```

The stale window lasts up to one full epoch — about 6.4 minutes.

## Impact

Attestation **signing** at epoch+2 is not affected. It still queries fresh BN state. This is not a wrong-chain-vote bug.

The affected part is VC-side preparation derived from the precompute:

- Which subnets to join
- Which validators are selected for aggregation

The cost is missed subnet participation and misdirected aggregation work in epoch+2.

### Concrete Example

Assume a validator prepares duties for epoch N+2 at the start of epoch N.
It uses the committee assignments from the current chain.

At slot 0 of epoch N, a reorg changes the dependent root for epoch N+2.
Lodestar stores the new root, but does not refresh the preparation data.

For the rest of epoch N, the validator may remain subscribed to a subnet that
is not needed on the new chain. It may also select the wrong validator for
distributed aggregation. The validator's epoch N+2 attestation signing still
uses fresh BN state. The stale data affects preparation, not the signed vote.

### Real-World Consequence

If a reorg occurs near the start of epoch N, the stale preparation can remain
active for almost 32 slots. During this time, the validator may listen to the
wrong subnet and miss messages from the subnet required by the new chain. A
validator selected with stale data for distributed aggregation may also do
work that cannot be used by the new chain.

This does not cause an invalid attestation. It can reduce attestation
participation and waste aggregation work. The effect can last for almost one
epoch after a single reorg. The impact is larger when many validators use the
same stale preparation data.

### Expected behavior

## Expected Behavior

A reorg may change the dependent root for epoch+2 duties. Lodestar should promptly invalidate or refresh the affected preparation data. Subnet subscriptions and aggregation selection should not continue using pre-reorg assignments for the rest of the epoch.

## Suggested Direction

The following options close most of the stale window. Each option keeps the anti-jitter rationale intact.

1. Apply the pending root immediately when it differs from the root the epoch+2 duties were computed from. Jitter that does not change the effective root stays free.
2. Bound the deferral to N slots past the boundary instead of the full epoch.
3. Dual-subscribe to both old and new assignments during the deferral window.

### Steps to reproduce

_No response_

### Additional context

_No response_

### Operating system

Linux

### Lodestar version or commit hash

v1.39.0

Contributor guide

Open the contributing guide

Research direction

Start with packages/validator/src/services/attestationDuties.ts, reading onNewHead() and prepareForNextEpoch(), then inspect the nextShuffling flow in packages/state-transition/src/cache/epochCache.ts. Trace how pendingDependentRootByEpoch affects subnet subscriptions and aggregation selection after an epoch+2 reorg. Done means preparation data is promptly invalidated or refreshed without changing the stated fresh-state behavior for attestation signing.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
blockchain
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.