Automatically check "invariants"

[alamb]

Is your feature request related to a problem or challenge?

I extracted this from #13651 so it was more visible

• Part of [EPIC] A collection of items to improve DataFuson stability (reduce effort required to upgrade) #13648
• Inspired by / copied from Introduce LogicalPlan invariants, begin automatically checking them #13651 from @wiedld

During upgrade, downstream systems often experience issues due to implicit changes (not explicit API changes) of LogicalPlans that DataFusion code begins relying on, and which result in unintended consequences when upgrading to a new version of DataFusion (see #13525).

Describe the solution you'd like

The idea is to make the current implicit assumptions ("Invariants" in more formal language)( explict and automatically check them.

Examples of implicit assumptions:

1. Schema column names can't be repeated (this is explicitly mentioned on [DISCUSSION] Making it easier to use DataFusion (lessons from GlareDB) #13525)
2. Inputs to UnionExec must have the same schema
3. ...

Describe alternatives you've considered

I like the approach @wiedld took in #13651 :

• define the invariants
• check the invariants for extensible interfaces (which may be user defined)
• throw the error closer to the problem (rather than weird behavior later)

Additional context

No response

[wiedld]

take

[wiedld]

Possible tasks

• Define API to check LP invariant. PR: Introduce LogicalPlan invariants, begin automatically checking them #13651
• Define API to check physical plan invariants.
• Explore possible API for user-defined invariants, and potential use cases with user-defined plan elements (e.g. physical plan nodes, analyzer and optimizer rules, etc). Refer to conversation here.
• Consider options (e.g. generate of docs) to align the published docs with the invariants encoded.
• Implement more invariants.

[wiedld]

For the physical optimization invariants, we have that the output physical plan schema cannot change; meaning the output results cannot change, but how we get the results can.

What else should be included as a responsibility/check? Maintain input ordering if required?

(The idea is to have a check, perhaps run in debug mode, that would error if a user-defined physical plan or optimization pass fails to maintain the invariant. Throw error closer to the source when debugging.)

[alamb]

For the physical optimization invariants, we have that the output physical plan schema cannot change; meaning the output results cannot change, but how we get the results can.

What else should be included as a responsibility/check? Maintain input ordering if required?

Here are some ideas based on bugs we have hit / my memory of what has changed and caused us pain:

1. Union inputs (can there be more than 2 inputs -- we use such plans in InfluxData but I am not sure what if anything else makes assumptions)
2. Union input schemas (I think initially they need to be 'coercable' and to be executable they need to be identical)
3. Projection_exec can't have zero exprs (is this an invariant?)