nervosnetwork / nervosnetwork/ckb

Ideas for Improving HeaderSync Speed

Open
#4,757 2 comments 0 reactions 1 assignee Claimed by @eval-exec View on GitHub
m:sync stale t:enhancement t:performance
Dominant language
Rust
Stars
1.2k
Forks
266
Avg merge
10d 5h
Merged PRs (30d)
4

Description

## Feature Request

Currently, a node starting from scratch sends a `GetHeaders` request (containing `locator_hashes` and `hash_stop`) to a full node, which responds with 2000 headers from the latest common ancestor determined by `locator_hashes`.

#### Is your feature request related to a problem? Please describe.

#### Describe the solution you'd like
The current CKB header sync process is slow because it requests headers from only one peer at a time, and each peer responds with a maximum of 2000 headers per request. I propose two ideas to improve the efficiency of this process:

**Idea 1: Request Headers from Multiple Peers**

With the recent PR (#4742) that incorporates multiple hardcoded assume valid targets, we can leverage this to dispatch `GetHeaders` requests to multiple peers, thereby speeding up the header sync phase.

For example, if we hardcode 10 assume valid targets: [1,000,000 → 2,000,000 → 3,000,000 → ... → 9,000,000 → 10,000,000], we can distribute requests as follows:

- Send a request to `peer_0` with `1,000,000` as `locator_hashes`
- Send a request to `peer_1` with `2,000,000` as `locator_hashes`
- Send a request to `peer_2` with `3,000,000` as `locator_hashes`
- ...

This approach allows the node to accumulate non-consecutive headers, which can then be incrementally verified. As the number of peers increases, the speed of the HeaderSync phase should improve correspondingly.

**Idea 2: Request More than 2000 Headers at Once**

The current limit of 2000 headers per request is not a heavy bandwidth burden for a CKB peer. By allowing a node to request more than 2000 headers in one go, we can enhance synchronization efficiency.

This can be achieved by trickly using the `hash_stop: Byte32` field in the `GetHeaders` request. Currently, `hash_stop` is underutilized. A node wishing to receive multiple messages at once (e.g., 5 messages totaling 5 × 2000 headers) could encode `5:u8` into the `hash_stop: Byte32` field, ensuring unused bits are set to `0`.

When the full node processes a `GetHeaders` request with an encoded `hash_stop`, it would parse the `u8` data to determine `N` and respond with `N` messages containing a total of `N × 2000` headers.

This proposal maintains backward compatibility with older versions of nodes, as it does not require changes to existing protocols.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.