celestiaorg / celestiaorg/celestia-node
core(multisource): fast-announcer/slow-server source poisons block ingestion — no health tracking / demotion
- Dominant language
- Go
- Stars
- 996
- Forks
- 1.1k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 34
Description
### Context
Since the multi-source block ingestion redesign ([PR](https://github.com/celestiaorg/celestia-node/pull/5018)), `MultiSource` fans new-height announcements from N core endpoints into one channel, and the Listener fetches each block from **the source that announced the height first** (`GetSignedBlockFrom`). Announce speed is used as a per-height freshness signal for "most responsive peer". There is deliberately no fallback inside the fetch: a failed fetch stores nothing, so the duplicate announcement of the same height from another source retries it (resilience is a property of the fan-in).
This works well when announce speed correlates with serve speed. The hole is a source where it doesn't.
### Problem
A source can be the **fastest to announce** (low RTT to the bridge) while being **slow or unable
to serve blocks** (overloaded, rate-limited, flaky transport). Routing follows the announcement,
so every height is first attempted against the bad source:
t=0.0 source Y announces H (always first - lowest RTT)
t=0.05 source X announces H (duplicate, queued)
t=0.0 listener fetches H from Y ...
t=10.0 ... blockFetchTimeout, nothing stored
t=10.0 listener picks up X's duplicate, fetch succeeds in ~0.2s
→ ~10.2s per height at 6s block time
t=6.0 meanwhile Y has already announced H+1 — first again
The per-height retry works, but **the system never learns**: the next height routes to the same
bad source again. The listener is sequential, so each height costs up to `blockFetchTimeout`
(10s) before the healthy duplicate is even read.
Contributor guide
Research direction
Start by reading the multi-source ingestion redesign in PR 5018 and tracing MultiSource announcements through the Listener and GetSignedBlockFrom. Define how block-fetch failures update source health and how demotion changes subsequent routing, then verify that a fast announcer that cannot serve blocks no longer causes every height to wait for blockFetchTimeout before a healthy duplicate is tried.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100