NethermindEth / NethermindEth/pluto

Converge the coexisting subscription mechanisms

Open
#606 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement rust
Dominant language
Rust
Stars
8
Forks
5
Avg merge
4d 16h
Merged PRs (30d)
37

Description

Summary

The workspace has six mutually incompatible subscription patterns, with divergent semantics for lifetime, drop, lagging, and error propagation:

  1. tokio broadcast + spawned pump — only in the scheduler (the sole broadcast use in the workspace).
  2. Vec<mpsc::Sender<T>> fan-out — SSE listener, with both a builder-time and an actor-round-trip runtime subscribe (sse/mod.rs#L68).
  3. watch::Receiver-returning subscribe() — p2p peers, dkg sync, readiness.
  4. Sync Box<dyn Fn> callback vectors — consensus wrapper/qbft/priority; the prioritiser's is documented "Not thread safe relative to a running instance".
  5. Arc<dyn Fn> -> BoxFuture callback vectors — sigagg, parsigdb, fetcher (a consuming-builder variant), bcast, parsigex, validatorapi. Five different receiver forms (&self, &mut self, async &self, self -> Self, interior Mutex).
  6. Charon-style function-pointer wiring in app/node/wire.rs.

Semantics diverge where it matters:

  • Lagged is handled only in the scheduler, by silently and permanently unsubscribing the lagging subscriber; Closed also breaks silently.
  • Subscriber errors: scheduler logs and continues; sigagg propagates with ?, so one failing subscriber aborts the remaining ones (same in parsigdb) — this asymmetry is load-bearing for duty completion.
  • No subscribe() anywhere returns an unsubscribe handle (scheduler TODO), and the scheduler documents a known leak of detached tasks for stuck handlers.

Related: #84 (make sub calling non-blocking).

Proposed change

This is a design task first: pick one or two blessed shapes (e.g. broadcast/watch channels for events, an async-callback registry with defined error semantics for duty pipeline stages), and document lifetime/drop/lag/error rules. Then migrate the outliers. Concrete decisions to make:

  • Should a subscriber error abort sibling subscribers (current sigagg/parsigdb) or be isolated (current scheduler)? Charon's wireX semantics are the parity reference.
  • What happens on lag: drop events, kill the subscription (current), or apply backpressure?
  • Unsubscribe/drop story: handles vs relying on channel closure.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the subscription implementations named in crates/core/src/scheduler.rs, crates/app/src/sse/mod.rs, crates/core/src/sigagg.rs, and app/node/wire.rs, using Charon's wireX semantics as the parity reference. Define the blessed subscription shapes and lifetime, drop, lag, error, and unsubscribe rules, then migrate the outliers and verify that the documented semantics are consistent across the workspace.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.