Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STR-799: Upgrade CL guest code to prove range of N blocks #563

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

MdTeach
Copy link
Contributor

@MdTeach MdTeach commented Dec 26, 2024

Description

  • Upgrades the current CL guest code and prover manager to support proving the range of CL blocks.
  • Refactor proof job in terms of the block IDs for El, CL and CL_Agg

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor
  • New or updated tests
  • Dependency Update

Notes to Reviewers

Checklist

  • I have performed a self-review of my code.
  • I have commented my code where necessary.
  • I have updated the documentation if needed.
  • My changes do not introduce new warnings.
  • I have added tests that prove my changes are effective or that my feature works.
  • New and existing tests pass with my changes.

Related Issues

@MdTeach MdTeach marked this pull request as ready for review December 26, 2024 06:13
@MdTeach MdTeach requested review from a team as code owners December 26, 2024 06:13
@MdTeach MdTeach force-pushed the STR-702-ol-prove-range-2 branch from befae62 to 5509e46 Compare December 26, 2024 06:22
Copy link

codecov bot commented Dec 26, 2024

Codecov Report

Attention: Patch coverage is 36.94779% with 157 lines in your changes missing coverage. Please review.

Project coverage is 56.51%. Comparing base (214d184) to head (ad3a19b).

Files with missing lines Patch % Lines
bin/prover-client/src/operators/cl_stf.rs 0.00% 80 Missing ⚠️
bin/prover-client/src/operators/checkpoint.rs 0.00% 32 Missing ⚠️
bin/prover-client/src/operators/cl_agg.rs 0.00% 13 Missing ⚠️
bin/prover-client/src/hosts/native.rs 0.00% 7 Missing ⚠️
bin/prover-client/src/hosts/sp1.rs 0.00% 6 Missing ⚠️
bin/strata-client/src/rpc_server.rs 0.00% 5 Missing ⚠️
provers/perf/src/main.rs 0.00% 5 Missing ⚠️
bin/prover-client/src/rpc_server.rs 0.00% 4 Missing ⚠️
bin/prover-client/src/operators/evm_ee.rs 0.00% 2 Missing ⚠️
crates/proof-impl/cl-stf/src/lib.rs 96.42% 2 Missing ⚠️
... and 1 more
@@            Coverage Diff             @@
##             main     #563      +/-   ##
==========================================
- Coverage   56.59%   56.51%   -0.08%     
==========================================
  Files         314      314              
  Lines       31896    31986      +90     
==========================================
+ Hits        18050    18077      +27     
- Misses      13846    13909      +63     
Files with missing lines Coverage Δ
bin/prover-client/src/operators/mod.rs 0.00% <ø> (ø)
crates/primitives/src/proof.rs 36.84% <ø> (ø)
crates/proof-impl/cl-stf/src/prover.rs 100.00% <100.00%> (ø)
crates/rpc/api/src/lib.rs 0.00% <ø> (ø)
crates/rpc/prover-client-api/src/lib.rs 0.00% <ø> (ø)
crates/test-utils/src/evm_ee.rs 100.00% <100.00%> (ø)
crates/zkvm/hosts/src/native.rs 100.00% <100.00%> (ø)
bin/prover-client/src/operators/operator.rs 0.00% <0.00%> (ø)
bin/prover-client/src/operators/evm_ee.rs 0.00% <0.00%> (ø)
crates/proof-impl/cl-stf/src/lib.rs 94.87% <96.42%> (+1.20%) ⬆️
... and 8 more

... and 8 files with indirect coverage changes

Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, main issue is that error handling should be a little more deliberate instead of panicking if things seem wrong. I know a lot of this was written with .expect in places but since we're in here making changes to related code we have good reason to clean it up a bit more.

bin/prover-client/src/operators/checkpoint.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_stf.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/checkpoint.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_agg.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_stf.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_stf.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/checkpoint.rs Outdated Show resolved Hide resolved
functional-tests/fn_prover_cl_dispatch.py Outdated Show resolved Hide resolved
functional-tests/fn_prover_el_dispatch.py Outdated Show resolved Hide resolved
@MdTeach MdTeach force-pushed the STR-702-ol-prove-range-2 branch from 871160f to f75932e Compare December 30, 2024 11:11
@MdTeach MdTeach requested a review from delbonis December 30, 2024 17:32
Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not really happy with the name "operators" it kinda doesn't make sense and it's sorta an overloaded term. A term like "controller" or "dispatcher" or even maybe just "adaptor" makes a lot more sense to me.

bin/prover-client/src/operators/checkpoint.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_agg.rs Show resolved Hide resolved
bin/prover-client/src/operators/cl_stf.rs Outdated Show resolved Hide resolved
bin/prover-client/src/operators/cl_stf.rs Outdated Show resolved Hide resolved
provers/tests/src/provers/el.rs Outdated Show resolved Hide resolved
@MdTeach MdTeach force-pushed the STR-702-ol-prove-range-2 branch from f842bf3 to 391f71c Compare December 31, 2024 09:53
@MdTeach MdTeach force-pushed the STR-702-ol-prove-range-2 branch from 391f71c to 1a74eaa Compare December 31, 2024 10:14
@MdTeach MdTeach force-pushed the STR-702-ol-prove-range-2 branch from 1a74eaa to ad3a19b Compare December 31, 2024 11:04
@MdTeach MdTeach requested a review from delbonis December 31, 2024 11:17
Comment on lines +64 to +88
/// Retrieves the [`L2BlockId`] for the given `block_num`
pub async fn get_l2id(&self, block_num: u64) -> Result<L2BlockId, ProvingTaskError> {
let l2_headers = self
.cl_client
.get_headers_at_idx(block_num)
.await
.inspect_err(|_| error!(%block_num, "Failed to fetch l2_headers"))
.map_err(|e| ProvingTaskError::RpcError(e.to_string()))?;

let headers = l2_headers.ok_or_else(|| {
error!(%block_num, "Failed to fetch L2 block");
ProvingTaskError::InvalidWitness(format!("Invalid L2 block height {}", block_num))
})?;

let first_header: Buf32 = headers
.first()
.ok_or_else(|| {
ProvingTaskError::InvalidWitness(format!("Invalid L2 block height {}", block_num))
})?
.block_id
.into();

Ok(first_header.into())
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be part of CL Operator.

.context();

// Doing the manual block idx to id transformation. Will be removed once checkpoint_info
// include the range interms of block_id.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: interms in terms

Comment on lines +1 to +6
/// Maximum number of blocks allowed in a proving range.
///
/// This constant serves as a safety limit to prevent proving tasks from processing
/// an excessively large number of blocks. If the number of blocks to prove exceeds
/// this limit, the task will be aborted early.
pub const MAX_PROVING_BLOCK_RANGE: u64 = 1024;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense it to be passed as an argument? I don't have a strong opinion on this. Also let's add a rationale for using 1024 instead of other number.

None => return Ok(None),
};

async fn get_cl_block_witness_raw(&self, blkid: L2BlockId) -> RpcResult<Option<Vec<u8>>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should not return an option. It might make sense to return Error when the information is not found.

Comment on lines 44 to +47
async fn get_exec_update_by_id(&self, block_id: L2BlockId) -> RpcResult<Option<RpcExecUpdate>>;

#[method(name = "getCLBlockWitness")]
async fn get_cl_block_witness_raw(&self, index: u64) -> RpcResult<Option<Vec<u8>>>;
async fn get_cl_block_witness_raw(&self, block_id: L2BlockId) -> RpcResult<Option<Vec<u8>>>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. I think this should return error if not found instead of None.

Comment on lines +16 to 17
strata-primitives = { workspace = true }
strata-rpc-types = { workspace = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use
.workspace = true

@@ -52,7 +52,7 @@ impl<H: ZkVmHost> ProofGenerator for CheckpointProofGenerator<H> {

let l2_batch_proof = self
.l2_batch_prover
.get_proof(&(l2_start_height, l2_end_height))
.get_proof(&vec![(l2_start_height, l2_end_height)])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are passing reference, we might not need vec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants