Skip to content

Commit

Permalink
remove another implementation detail
Browse files Browse the repository at this point in the history
  • Loading branch information
WanYixian committed Nov 21, 2024
1 parent 87b0c80 commit efe6beb
Showing 1 changed file with 2 additions and 6 deletions.
8 changes: 2 additions & 6 deletions delivery/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -139,13 +139,9 @@ There are two primary batching strategies:
- **Batching based on rollover interval**:
For batching based on the time interval, RisingWave checks the threshold each time a chunk is about to be written (`sink_writer.write_batch()`) and when a barrier is encountered (`sink_writer.barrier()`).

The batching condition is relatively coarse-grained. Setting a specific number of rows for batching does not guarantee that the output file will contain exactly that many rows.
The condition for batching is relatively coarse-grained. Setting a specific number of rows for batching does not guarantee that the output file will contain exactly that many rows.

### Sink decoupling with batching strategy

With the batching strategy for file sink, file writing is no longer dependent on the arrival of barriers. `BatchingLogSinkOf` determines when to truncate the log store. Once a file is written, it will be truncated. However, if batching occurs across barriers and no writing has occurred by the time a barrier arrives, the barrier will not trigger truncation.

If no batching strategy is defined, the previous logic will still apply, meaning that a file will be forcefully written upon the arrival of a checkpoint barrier.
If no conditions for batch collection are set, RisingWave will apply a default batching strategy to ensure proper file writing and data consistency.

### File naming rule

Expand Down

0 comments on commit efe6beb

Please sign in to comment.