Skip to content

Commit

Permalink
refactor(indexer-alt): pull out pipeline (MystenLabs#19933)
Browse files Browse the repository at this point in the history
## Descrption

Separate the `pipeline` abstraction from the `handler` abstraction, and
put it into its own module. This is in preparation for:

- Splitting up the batching and committing processes. This will make it
possible to test committing multiple chunks in parallel, and it may be
possible to re-use some of this logic in...
- an `in_order::pipeline` to use for pipelines that need to write out
data strictly in checkpoint order and along checkpoint boundaries.

## Test plan

Existing tests and CI.

## Stack

- MystenLabs#19926 
- MystenLabs#19932 

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
  • Loading branch information
amnn authored Oct 21, 2024
1 parent 54e4738 commit 368fc4b
Show file tree
Hide file tree
Showing 14 changed files with 685 additions and 628 deletions.
2 changes: 1 addition & 1 deletion crates/sui-indexer-alt/src/handlers/ev_emit_mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ impl Handler for EvEmitMod {

type Value = StoredEvEmitMod;

fn handle(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
fn process(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
let CheckpointData {
transactions,
checkpoint_summary,
Expand Down
2 changes: 1 addition & 1 deletion crates/sui-indexer-alt/src/handlers/ev_struct_inst.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ impl Handler for EvStructInst {

type Value = StoredEvStructInst;

fn handle(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
fn process(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
let CheckpointData {
transactions,
checkpoint_summary,
Expand Down
2 changes: 1 addition & 1 deletion crates/sui-indexer-alt/src/handlers/kv_checkpoints.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ impl Handler for KvCheckpoints {

type Value = StoredCheckpoint;

fn handle(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
fn process(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
let sequence_number = checkpoint.checkpoint_summary.sequence_number as i64;
Ok(vec![StoredCheckpoint {
sequence_number,
Expand Down
2 changes: 1 addition & 1 deletion crates/sui-indexer-alt/src/handlers/kv_objects.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ impl Handler for KvObjects {

type Value = StoredObject;

fn handle(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
fn process(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
let deleted_objects = checkpoint
.eventually_removed_object_refs_post_version()
.into_iter()
Expand Down
2 changes: 1 addition & 1 deletion crates/sui-indexer-alt/src/handlers/kv_transactions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ impl Handler for KvTransactions {

type Value = StoredTransaction;

fn handle(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
fn process(checkpoint: &Arc<CheckpointData>) -> Result<Vec<Self::Value>> {
let CheckpointData {
transactions,
checkpoint_summary,
Expand Down
Loading

0 comments on commit 368fc4b

Please sign in to comment.