Make simplify and lower optional within Expr.optimize

[rjzamora]

I have found that it can sometimes be useful to avoid lowering within an Expr.optimize call, because it becomes a bit easier to inspect the behavior of simplify on an expression graph.

[phofl]

I don’t want to add unnecessary keywords here and this will work differently after #294, so I wouldn’t add control over simplify here

[rjzamora]

Yeah, the motivation isn't really to control options to compute, so I'm happy to roll that back. Thanks for pointing out 294.

[phofl]

Is there an advantage when running computations as well? This makes the logic noticeable more complex for a relatively small benefit.

We don't need everything configurable, I think we agree that there is really no point in not running simplify?

[rjzamora]

Is there an advantage when running computations as well? This makes the logic noticeable more complex for a relatively small benefit.

I included that change because I was using it for other experiments and realized we would probably be interested in making it possible to measure the effects of column-projections etc in a more direct way. However, compute is definitely not my primary interest here, so I'll roll that part back.

We don't need everything configurable, I think we agree that there is really no point in not running simplify?

Well, we must run simplify to guarantee that a low-level graph can be generated. We don't really need to simplify, but it's fine with me if we keep that component simple.

[phofl]

I am a bit uncomfortable with making def optimise more complicated.

Couldn't we use .simplify().combine_similar() instead of adjusting optimise?

[rjzamora]

I am a bit uncomfortable with making def optimise more complicated.

Couldn't we use .simplify().combine_similar() instead of adjusting optimise?

Sure, that seems fair to me. This PR is a much lower priority than #321, so I have no problem closing it.

I'll admit that I was not being completely transparent about my full motivations in my PR description. I was hoping to slowly nudge the library in the direction of establishing a clearer distinction between "abstract" and "dask-specific" expressions.

I was thinking that this would be simpler if lower was optional, but the real blocker is probably the fact that Expr is currently required to support dask-specific attributes like divisions and npartitions. I am starting to feel that this should not be the case at the level of the base Expr class. However, this is obviously a big change (and is somewhat orthogonal to the original goals of this work), so I'll just plan to revisit these thoughts later :)