Feature Request: Migration Guide from Ava 3 to 4 for advanced macro usage

[wessberg]

Hey there.

I've been meaning to migrate to ava 4 for a while now, and I've also made some attempts. However, I'm relying quite heavily on the feature from ava 3 where multiple macros can be passed to test, something that was removed in this PR.

Specifically, I'm maintaining a bunch of libraries and tooling around TypeScript, and for all of these libraries, I'm running the entire test suite against all or most TypeScript versions. For example, a test might look like this:

test("Some test title", withTypeScriptVersions(">=3.8"), async (t, {typescript}) => {
	// Test implementation
});

There, every minor version of TypeScript from v3.8 and up will be passed to the implementation, and I can easily see in the test results which TS versions had problems.

Other tests may receive a different set of macros, corresponding to another set of TypeScript versions.

These tests are typically spread across many files and there are typically a ton of them.

I've read through your original PR in which you state:

Supporting multiple implementations made both the implementation and the type definitions needlessly complex. Using for/of syntax it's easy enough to declare tests (or t.try() attempts) with the same arguments.

However, for my use case, the amount of macros each implementation/test is called with is varying (as seen in the example above). I'm not always passing the same amount of TypeScript versions (macros) through to each test, so I can't just iterate through all of them, or at least I would have to add some guards in front of the respective tests. I do see some ways to implement it, of course, but none that is as clean as what's currently possible.

I'm hoping there's a way to achieve a similar setup with ava 4, and if so, I'd love to see either a migration guide or some hints as to how to achieve it. If you're interested in seeing how I solve it today, one example is the test suite of rollup-plugin-ts, and here is the macro implementation.

I would also ask that you may reconsider supporting multiple macros again, unless there is an obvious and clear migration path forward.
Thanks for your time!

[novemberborn]

I would change withTypeScriptVersions to be a wrapper around test. Roughly:

const withTypeScriptVersions = (range, title, implementation) => {
  for (const version of expandRange(range)) {
    test(`typescript@version — ${title}`, t => implementation(t, {version})
  }
}

Which, probably, you already thought of.

It became hard to correctly type the test function when multiple macros were in play. Removing this feature made the internals a bit simpler, at the cost of needing some wrapping logic for certain test suites.

[wessberg]

Yes, this is akin to what I was doing before I learned about your macro system, but what I particularly like about the macro system is the way I can still use the test interface directly, and import test from ava without any wrappers. In practice, I think wrapping the test interface, as in your example, ends up looking less readable than I'd like, and certainly when I compare it to what I have currently with ava 3.

Thanks for your suggestion, anyway! I think I'm going to stay on ava 3 for the time being, and I'm hoping something like what Sindre proposed in this issue could eventually be a solution for me and enable me to migrate to ava 4.