feat: external block provider for genesis sync
- Dominant language
- Rust
- Stars
- 4
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Nodes running in pruning mode (default) discard old blocks to bound disk growth. To support syncing new nodes from genesis without requiring every node to be a full archive node, we need an external block serving service.
## Motivation
- Block archive storage now prunes by default (retention window of ~100k blocks)
- Chain-index (SQLite) serves RPC/explorer queries independently of the block archive
- Only seed/archive nodes need full block history
- New nodes joining the network need a way to sync from genesis without requiring every peer to store all blocks
## Design
An external block provider service that:
1. Stores all blocks from genesis (potentially using commonware's `immutable::Archive` with freezer for minimal memory + compression)
2. Serves block data over a well-defined API (HTTP/gRPC) to syncing nodes
3. Can be operated independently of the P2P network -- e.g., a centralized service, cloud storage, or a dedicated archive node
### Sync flow
```
New Node Block Provider
| |
|-- request blocks [0, N) --->|
|<--- stream ArchivedBlock ---|
| |
|-- apply blocks to STF --- |
|-- build chain-index --- |
| |
|-- join P2P, continue from |
| latest block |
```
### Considerations
- The provider does not need to be trusted for correctness -- the node validates state roots against the chain's state transition function
- Provider could be backed by object storage (S3/R2) serving pre-compressed block segments
- Protocol should support range requests and resumable sync
- Consider using commonware's `immutable::Archive` (freezer) as the backing store for the provider -- it's designed for exactly this workload (write-once, read-many, minimal memory, compressed)
## Acceptance Criteria
- [ ] Define block provider API (gRPC service definition or HTTP endpoints)
- [ ] Implement block provider server backed by full block archive
- [ ] Implement client-side sync that fetches blocks from provider and applies them
- [ ] Support resumable sync (track last synced block)
- [ ] Node config option to specify block provider URL for initial sync
Contributor guide
Assessment
This issue has not been assessed yet.