Automated interoperability tests for new releases

[vasco-santos]

The goal of this issue is to discuss and track the progress of an automatic approach for creating the binaries for libp2p implementations (other than JS), publish them on npm and finally, use them to run the interop tests.

We currently have a completely manual process using libp2p/npm-go-libp2p-dep, where we create the implementation binaries (one for each platform), add them to IPFS and add the hashes to an object of the module. This way, once npm-go-libp2p-dep is installed, the platform in use is identified, as well as the intended version (version in package.json), and the appropriate binary is downloaded from an IPFS public gateway. In npm-go-libp2p-dep README the process is detailed deeper, and it is easy to understand the overhead of the process. Moreover, communication must happen to coordinate it.

After looking at Oli’s work for automatic version & publish when go-ipfs is released ipfs/npm-go-ipfs-dep/pull/35 and a small discussion with him, I identified the differences between the setups for libp2p and ipfs interop / go dependency.

Shortly, the key difference is that we manually create the binaries for each platform using the go-libp2p-daemon, publish and pin them on IPFS and then get them through the public gateway, while in IPFS land, the binaries are also manually created but they are added to the dist.ipfs.io and downloaded from there.

The aimed solution should automatically:

1. identify when a new release of go-libp2p, js-libp2p (and possibly rust-libp2p) happens
2. create appropriate binaries for each platform through npm modules like npm-go-libp2p-dep (non-JS release)
3. add the created binaries to IPFS (consider using IPFS Cluster)
4. publish the new version of the npm module, with the new version of the implementation
5. identify when a new release of an npm module occurs, as well as a new release of js-libp2p
6. create a pull request with the new version of the implementation, so that it can be automatically tested in CI

The first 4 steps would be a responsibility of the repositories containing the binaries of the implementation, while the remaining steps would be a responsibility of the interop repo.

With this solution, the biggest part of the process would be automated and the person responsible for this repo would only have to answer to the created PRs. That is, if the tests get green the PR should be merged, otherwise, an issue should be created and the issue must be fixed.

Current challenges:

• the relation between the daemons and implementation versions. Basically, we use the binary daemons for testing interop, but we currently use the implementation version instead of the daemon version, which seems more correct, but we must have a way of getting the implementation version under the daemon repo (this was part of the human process).
• how to consider release candidates, as those should not be merged, but may be important to know the status of interop before releasing a "final" version.

From what I know, we should be able to take advantage of Github Actions to create all this setup, but I need to look at them first, as I have no prior experience with them.

cc @raulk @jacobheun @olizilla