Ideas for Improving HeaderSync Speed #4757
Labels
m:sync
module: ckb-sync
t:enhancement
Type: Feature, refactoring.
t:performance
Type: Performance tuning
Feature Request
Currently, a node starting from scratch sends a
GetHeaders
request (containinglocator_hashes
andhash_stop
) to a full node, which responds with 2000 headers from the latest common ancestor determined bylocator_hashes
.Is your feature request related to a problem? Please describe.
Describe the solution you'd like
The current CKB header sync process is slow because it requests headers from only one peer at a time, and each peer responds with a maximum of 2000 headers per request. I propose two ideas to improve the efficiency of this process:
Idea 1: Request Headers from Multiple Peers
With the recent PR (#4742) that incorporates multiple hardcoded assume valid targets, we can leverage this to dispatch
GetHeaders
requests to multiple peers, thereby speeding up the header sync phase.For example, if we hardcode 10 assume valid targets: [1,000,000 → 2,000,000 → 3,000,000 → ... → 9,000,000 → 10,000,000], we can distribute requests as follows:
peer_0
with1,000,000
aslocator_hashes
peer_1
with2,000,000
aslocator_hashes
peer_2
with3,000,000
aslocator_hashes
This approach allows the node to accumulate non-consecutive headers, which can then be incrementally verified. As the number of peers increases, the speed of the HeaderSync phase should improve correspondingly.
Idea 2: Request More than 2000 Headers at Once
The current limit of 2000 headers per request is not a heavy bandwidth burden for a CKB peer. By allowing a node to request more than 2000 headers in one go, we can enhance synchronization efficiency.
This can be achieved by trickly using the
hash_stop: Byte32
field in theGetHeaders
request. Currently,hash_stop
is underutilized. A node wishing to receive multiple messages at once (e.g., 5 messages totaling 5 × 2000 headers) could encode5:u8
into thehash_stop: Byte32
field, ensuring unused bits are set to0
.When the full node processes a
GetHeaders
request with an encodedhash_stop
, it would parse theu8
data to determineN
and respond withN
messages containing a total ofN × 2000
headers.This proposal maintains backward compatibility with older versions of nodes, as it does not require changes to existing protocols.
The text was updated successfully, but these errors were encountered: