NethermindEth / NethermindEth/pluto

Preserve tracing spans across spawns, instrument components, record missing metrics

Open
#631 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement rust
Dominant language
Rust
Stars
8
Forks
5
Avg merge
4d 16h
Merged PRs (30d)
37

Description

Summary

Spans. The workspace has 60 production tokio::spawn/spawn_blocking sites and exactly 2 .instrument(...) calls (zero Span::current()/in_current_span). Every uninstrumented spawn drops the span context — which is also the mechanism behind #588's topic="" problem, since log-topic labels are span fields: spans and topics are the same fix. #[tracing::instrument] appears 33 times, 22 of them in one file (validatorapi/component.rs); scheduler, fetcher, bcast, tracker, dutydb, parsigdb, aggsigdb, consensus/qbft, dkg, parsigex, and priority have none. Highest-value uninstrumented spawns: the scheduler's per-slot detached duty callbacks (scheduler.rs#L173-L184), the actor loops (tracker, aggsigdb, prioritiser), and the duty-workflow wiring in wire.rs.

Defined-but-never-recorded metrics (grep-verified, zero recording sites):

  • app_eth2_using_fallbackmissing entirely, yet the health check that queries it already exists and can never fire. Lands with the eth2api multi-client(#611).
  • p2p_peer_network_{sent,receive}_bytes_total — the BandwidthFactory mechanism exists but only the relay server wires it; Node::new (client path) has no bandwidth parameter (behaviour.rs#L98).
  • p2p_peer_streams, p2p_relay_connections — declared in p2p/src/metrics.rs, recorded nowhere (Charon populates both in p2p.go).
  • core_validatorapi_proxy_request_latency_seconds — the RAII recorder exists, no construction site.

Otherwise name-for-name parity with Charon's metric families is good; app_eth2_* request/error/latency coverage is opt-in per call site (~11 of 76 endpoints) and moves into the client wrapper.

Proposed change

  1. Add a spawn helper attaching Span::current() (the #588 suggestion) and adopt it at the production spawn sites; set one root span per long-running component.
  2. Add #[tracing::instrument] to the duty-pipeline entry points (fetch/propose/participate/broadcast/store paths), with skips to keep fields bounded.
  3. Wire the four dead metric families above (or delete any that are deliberately out of scope, with a note).
  4. While in layers/metrics.rs, cover the topic-resolution layer with tests: the TopicVisitor::record_debug fallback and nearest-enclosing-span resolution are untested (only same-span level filtering is).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/core/src/scheduler.rs, wire.rs, the actor loops, and layers/metrics.rs to map spawn and span behavior; then inspect crates/p2p/src/metrics.rs, crates/app/node/behaviour.rs, and app health checks. Done means production spawns preserve context, the listed metric families are wired or explicitly removed, and topic-resolution tests cover the named fallbacks.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems, observability, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.