Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize parts of transaction verification and conversion #12448

Open
nagisa opened this issue Nov 13, 2024 · 1 comment
Open

Parallelize parts of transaction verification and conversion #12448

nagisa opened this issue Nov 13, 2024 · 1 comment

Comments

@nagisa
Copy link
Collaborator

nagisa commented Nov 13, 2024

Description

When there are a huge number of transactions containing Very Cheap to Process actions in a chunk, we start seeing a significant portion (~35%) of the CPU time being spent in just process_transactions/verify_and_charge_transaction:

  • ~20% in signature verification;
  • ~14% in reading out account and accesskey data.

In principle the functionality here is structured such that a lot of the work within it is parallelizable, with the only exception being its &mut TrieUpdate argument. Mutable trie access is only necessary at the end of the function to update NEAR balances and the access key nonce and doesn't need to happen at the end of each transaction processing, strictly speaking.

Since all operations operate on data that's sharded by AccountID here, the transactions can be grouped by the signed AccountID first and then any changes to nonces and balances would be applied to the state in one go on the main thread once the processing for a group of transactions is complete.

There are also other approaches that may be considered. For instance, we can validate just the signatures in parallel without any prior grouping and let the remaining work continue on the main thread. The processing time ceiling then would be roughly the 14% we see in reading the account/accesskey data...

@nagisa
Copy link
Collaborator Author

nagisa commented Nov 13, 2024

cc @akhi3030 @mooori

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant