Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sharding of global challenge phase commitment and opcode proving #695

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

lispc
Copy link
Collaborator

@lispc lispc commented Dec 5, 2024

The sharding in first layer circuit is quite simple. While sharding+recursion-circuit will be where most magic happens.

Pending designs:

  1. same shard_size for all opcodes? (simple, but not most efficient?)
  2. same shard_size for all shards? (not now, last shard is auto shrinked to least_pow_of_2).
  3. tweak opcode proof for recursion proving friendliness? like "merge/fold" into 1? how?
  4. Should we refactor the OpcodeProof struct as a vector of shard proofs?
  5. some other misc coding TODOs inside the PR.

Other notes:

  1. i checked log and confirmed the biggest into_mle input is 8xSHARD_SIZE. It is as expected related to some FAN_IN mle parameter.

Perf (MBP M3)

Proving finished.
	Proving time = 43.146s, freq = 267.441khz
	Witgen  time = 1.935s, freq = 5963.201khz
	Total   time = 45.081s, freq = 255.961khz
	thread num: 8```

@lispc lispc changed the title WIP Feat/sharding feat: sharding of global challenge phase commitment and opcode proving Dec 5, 2024
@lispc lispc marked this pull request as ready for review December 5, 2024 05:06
@lispc lispc requested a review from naure December 5, 2024 08:50
@@ -90,33 +92,59 @@ impl<E: ExtensionField, PCS: PolynomialCommitmentScheme<E>> ZKVMProver<E, PCS> {
}
exit_span!(span);

// TODO: is it better to set different size of different opcode?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so. Why choosing 1048576 here?

Copy link
Collaborator

@matthiasgoergens matthiasgoergens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments and questions. I'm still trying to understand what's going on.

@@ -90,33 +92,59 @@ impl<E: ExtensionField, PCS: PolynomialCommitmentScheme<E>> ZKVMProver<E, PCS> {
}
exit_span!(span);

// TODO: is it better to set different size of different opcode?
let shard_size = 1048576;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be easier to read in hex. (Because it's not just a random number.)

// TODO: (1) is it ok to store mle? (2) replace tuple with struct?
#[allow(clippy::type_complexity)]
let mut wits_and_commitments: BTreeMap<
String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the meaning of the key here? A type synonym might be useful here?

let mut commitments = BTreeMap::new();
let mut wits = BTreeMap::new();
// TODO: (1) is it ok to store mle? (2) replace tuple with struct?
#[allow(clippy::type_complexity)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps use some type synonym or so?

@@ -61,10 +61,11 @@ impl<E: ExtensionField, PCS: PolynomialCommitmentScheme<E>> ZKVMVerifier<E, PCS>
does_halt: bool,
) -> Result<bool, ZKVMError> {
// require ecall/halt proof to exist, depending whether we expect a halt.
// TODO: make it less adhoc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Especially once we support more ecalls. Or perhaps we should introduce a specific 'halt-successfully' introduction.

@@ -107,6 +110,42 @@ impl<T: Sized + Sync + Clone + Send + Copy> RowMajorMatrix<T> {
})
.collect()
}

// TODO: should we consume or clone `self`?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's natural to consume self, then do so. The caller can clone.

Copy link
Collaborator

@hero78119 hero78119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sharding looks pretty straightforward and simple 👍

@@ -471,6 +478,17 @@ impl<E: ExtensionField, PCS: PolynomialCommitmentScheme<E>> ZKVMVerifier<E, PCS>
}

// verify zero expression (degree = 1) statement, thus no sumcheck
for (expr, name) in cs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic seems some left over. probably we combine with line 492?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sharding of 1st phase commitment Sharding of opcode proving
4 participants