make the split_by_worker and slpit_by_rank optional

[AlirezaSohofi]

Description

The requirements: merging multiple streams in a single driver, while ensuring data splitting happens at the shard key level. In this case, the user will take care that the correct splitting logic is indeed applied to each stream, still the drivers that includes these streams can't figure out if this is the case. By providing a way to bypass this safety mechanism, power users can build more powerful pipelines.

Fixes # issue

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring including code style reformatting
• Other (please describe):

Checklist:

• I have read the contributing guideline doc (external contributors only)
• Lint and unit tests pass locally with my changes
• I have kept the PR small so that it can be easily reviewed
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• All dependency changes have been reflected in the pip requirement files.

[pzdkn]

Looks solid, but I am not exactly sure which problem this is solving. Is it possible to include an example for combining sources in a driver in the docs and mention how using this configuration is necessary?

[pzdkn]

Thanks Ali, Looks good!

[pzdkn]

lgtm

[github-actions]

This is PR is marked as stale as it has been inactive for 30 days. It will be closed in 7 days.

[pzdkn]

thks, lgtm!