Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ideas for Improving HeaderSync Speed #4757

Open
eval-exec opened this issue Dec 21, 2024 · 0 comments
Open

Ideas for Improving HeaderSync Speed #4757

eval-exec opened this issue Dec 21, 2024 · 0 comments
Assignees
Labels
m:sync module: ckb-sync t:enhancement Type: Feature, refactoring. t:performance Type: Performance tuning

Comments

@eval-exec
Copy link
Collaborator

eval-exec commented Dec 21, 2024

Feature Request

Currently, a node starting from scratch sends a GetHeaders request (containing locator_hashes and hash_stop) to a full node, which responds with 2000 headers from the latest common ancestor determined by locator_hashes.

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

The current CKB header sync process is slow because it requests headers from only one peer at a time, and each peer responds with a maximum of 2000 headers per request. I propose two ideas to improve the efficiency of this process:

Idea 1: Request Headers from Multiple Peers

With the recent PR (#4742) that incorporates multiple hardcoded assume valid targets, we can leverage this to dispatch GetHeaders requests to multiple peers, thereby speeding up the header sync phase.

For example, if we hardcode 10 assume valid targets: [1,000,000 → 2,000,000 → 3,000,000 → ... → 9,000,000 → 10,000,000], we can distribute requests as follows:

  • Send a request to peer_0 with 1,000,000 as locator_hashes
  • Send a request to peer_1 with 2,000,000 as locator_hashes
  • Send a request to peer_2 with 3,000,000 as locator_hashes
  • ...

This approach allows the node to accumulate non-consecutive headers, which can then be incrementally verified. As the number of peers increases, the speed of the HeaderSync phase should improve correspondingly.

Idea 2: Request More than 2000 Headers at Once

The current limit of 2000 headers per request is not a heavy bandwidth burden for a CKB peer. By allowing a node to request more than 2000 headers in one go, we can enhance synchronization efficiency.

This can be achieved by trickly using the hash_stop: Byte32 field in the GetHeaders request. Currently, hash_stop is underutilized. A node wishing to receive multiple messages at once (e.g., 5 messages totaling 5 × 2000 headers) could encode 5:u8 into the hash_stop: Byte32 field, ensuring unused bits are set to 0.

When the full node processes a GetHeaders request with an encoded hash_stop, it would parse the u8 data to determine N and respond with N messages containing a total of N × 2000 headers.

This proposal maintains backward compatibility with older versions of nodes, as it does not require changes to existing protocols.

@eval-exec eval-exec added the t:enhancement Type: Feature, refactoring. label Dec 21, 2024
@eval-exec eval-exec self-assigned this Dec 24, 2024
@eval-exec eval-exec added m:sync module: ckb-sync t:performance Type: Performance tuning labels Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
m:sync module: ckb-sync t:enhancement Type: Feature, refactoring. t:performance Type: Performance tuning
Projects
None yet
Development

No branches or pull requests

1 participant