blockblaz / blockblaz/zeam

Add ENR attnets advertisement, discv5 attnets-aware peer filter, and backbone subnet subscriptions

Open
#816 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Zig
Stars
97
Forks
39
PR merge metrics
No merged PRs in 30d

Description

Follow-up to #811. Now that #812 lands selective gossipsub subscribe, the mesh-connectivity problem flagged in #811's "Other gaps" section becomes the dominant remaining risk — covered here as its own issue.

## Problem

With selective subscribe, a node only joins the gossipsub mesh for subnets its validators are assigned to. That's correct for bandwidth, but it means a node now needs *enough peers on those specific subnets* for the mesh to reach `D_low`. Standard gossipsub has no way to find subnet-specific peers — it relies on either:

1. **Topic-aware peer discovery** (find peers on the subnet at the discovery layer), or
2. **Backbone density guarantees** (a fraction of nodes on every subnet regardless of duty).

zeam currently has neither. On a sparse network or an unfortunate connection topology, a subscriber can have zero same-subnet neighbours → mesh size 0 → attestations don't propagate. Diagrammatically this is the "checkerboard" failure mode.

## What the beacon spec mandates

Three normative pieces in the Ethereum consensus spec, all specifically designed to keep the mesh dense per subnet:

### A. ENR `attnets` field — ENR-based subnet discovery

[`specs/phase0/p2p-interface.md` — "Attestation subnet bitfield"](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#attestation-subnet-bitfield):

> An ENR `attnets` entry signifies the attestation subnet bitfield with the following form to more easily discover peers participating in particular attestation gossip subnets.
>
> | Key | Value |
> |:----------|:---------------------------------------------------|
> | `attnets` | SSZ `Bitvector[ATTESTATION_SUBNET_COUNT]` |
>
> If a node's `MetaData.attnets` has any non-zero bit, the ENR MUST include the `attnets` entry with the same value as `MetaData.attnets`. … Bits SHOULD be set to true for a subnet where the validator is intending to continue collecting custodial signed aggregated attestations.

### B. Discv5-level subnet peer discovery

Same section:

> Nodes looking for peers participating in a particular subnet SHOULD search for peers via the discovery mechanism (e.g. ENR-based peer discovery via Discv5) using the `attnets` ENR entry to filter for peers that are participating in the desired subnet(s).

i.e. the spec REQUIRES that ENR `attnets` is populated AND that subnet-targeted peer search consults it. zeam does neither today.

### C. Long-lived (backbone) subnet subscriptions

[`specs/phase0/validator.md` — "Phase 0 attestation subnet stability"](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/validator.md#phase-0-attestation-subnet-stability) (legacy form) and the consolidated/Electra refactor:

> To provide a network resource for finding peers in particular subnets, validators SHOULD subscribe to a long-lived random subnet for a number of epochs equal to `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION = 256` and randomly switch to a new long-lived random subnet at the end of that period.

with constants `RANDOM_SUBNETS_PER_VALIDATOR = 1` and `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION = 256`.

The post-EIP-7917 / consolidated spec replaces the per-validator randomized scheme with a deterministic per-node `compute_subscribed_subnets(node_id, epoch)` returning `SUBNETS_PER_NODE` long-lived subnets, rotated every `EPOCHS_PER_SUBNET_SUBSCRIPTION` epochs. Either form satisfies the backbone-density guarantee.

## Gap analysis

| Spec requirement | zeam status |
|---|---|
| ENR `attnets` populated and updated as subscriptions change (A) | **absent** — no parser, no setter, no call sites |
| Discv5 peer search filtered by `attnets` (B) | **absent** — discovery returns generic candidates |
| Long-lived backbone subnet subscriptions (C) | **absent** — subscriptions are validator-derived only and fixed at startup |

## Proposal

Implement all three. Three pieces:

### 1. ENR `attnets` advertisement
- Add an `attnets` Bitvector[N] type and ENR key handler.
- At `BeamNode.run()`, after the selective subscription set is computed, write the bitfield into the local ENR via the rust libp2p-glue bridge (new export, e.g. `update_enr_attnets`).
- Re-insert whenever the subscription set changes (currently startup-only; will also fire on rotation per #3).

### 2. Discv5 attnets-aware peer filter
- libp2p-glue already pulls in the rust-libp2p `discv5` crate. Add a `find_node_predicate` wrapper, parameterised by the desired subnet bitmap, used when the peer manager wants to grow the mesh on a specific subnet.
- Post-query filter on returned ENRs to drop peers whose `attnets` bit for the target subnet is clear.

### 3. Backbone subnet subscriptions
- Implement either:
- `compute_subscribed_subnets(node_id, epoch)` returning `SUBNETS_PER_NODE` deterministic backbone subnets (consolidated spec form), or
- per-validator random subnet rotation with `RANDOM_SUBNETS_PER_VALIDATOR = 1` and `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION = 256` (legacy form).
- Add the backbone subnets to the selective set at `BeamNode.run()`. Rotate every `EPOCHS_PER_SUBNET_SUBSCRIPTION` epochs — needs a runtime subscribe/unsubscribe path zig→rust (currently the subscribe set is fixed at startup; this would also unblock validator hot-add, see #811).
- Update ENR `attnets` and gossipsub mesh on rotation.

## Out of scope (consider afterwards)

- Sync-committee subnet bitfield (`syncnets`) — analogous to `attnets`, same shape, lower priority.
- ENR `eth2` fork-digest field — orthogonal; useful for peer scoring across forks.
- Coordinating with the leanSpec authors so the three pieces are explicit in the lean spec rather than implicit-via-Ethereum-spec.

## Test plan

- Unit tests for `compute_subscribed_subnets` against published spec test vectors (when available) or a hand-rolled vector.
- Integration test: 3-node simtest with `attestation_committee_count > 1` and validators on disjoint subnets. Assert:
- Each node's ENR `attnets` bitfield reflects its current subscriptions.
- Mesh size on each subnet stays ≥ `D_low` across slots.
- Attestations published on subnet X land at the validator on subnet X within one slot.

## References

- #811 — initial selective-subscribe bug and out-of-scope gap list.
- #812 — landed selective subscribe.
- Phase 0 p2p-interface spec (ENR `attnets` + discv5 search): https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md
- Phase 0 validator spec (legacy `RANDOM_SUBNETS_PER_VALIDATOR`): https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/validator.md
- EIP-7917 / consolidated `compute_subscribed_subnets`: https://eips.ethereum.org/EIPS/eip-7917

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at BeamNode.run() and the rust libp2p-glue bridge, then trace how the selective subscription set from #812 is created and passed to gossipsub. Read the ENR and Discv5 integration points before choosing the backbone-subnet form. Done means the ENR, subnet-aware discovery, rotation, and the proposed 3-node simtest all reflect current subscriptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, zig
Domain
distributed-systems, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.