[tracking issue] Interop test-plans for all existing/developing libp2p transports

[p-shahi]

Part of the libp2p/test-plans roadmap: https://github.com/libp2p/test-plans/blob/master/ROADMAP.md#2-interop-test-plans-for-all-existingdeveloping-libp2p-transports

Done Criteria

Using tooling from #53

• All features of go-libp2p, rust-libp2p, and js-libp2p that should be interoperable have test-plans (i.e. transports (TCP, QUIC, WebRTC, WebTransport), multiplexers (mplex, yamux), secure channels (TLS, Noise), etc.) across versions.
• Features currently in development across implementations (like WebRTC in go-libp2p and rust-libp2p, or QUIC & TLS in rust-libp2p) are not merged without at least manually running these test suites.
• Test suites are run in libp2p/test-plans CI and before releasing a version of go-libp2p, rust-libp2p, and js-libp2p (GitHub workflow added so that these suites run against the master branch on a nightly basis (updating the status check.))

Tasks

• Add fast multidimensional interop tests #97 Add docs for multidim-interop and update readme #101
  • go-libp2p (DRI: @MarcoPolo)
    • TCP, QUIC, WebSocket: Add fast multidimensional interop tests #97
    • WebTransport: (link to PR)
    • WebRTC: blocked on WebRTC Direct transport implementation go-libp2p#2337
  • rust-libp2p (DRI: @MarcoPolo)
    • TCP, QUIC, WebSocket: Add fast multidimensional interop tests #97
    • WebRTC: Add webrtc transport multidim interop #100
    • WebRTC browser: Add Browser WebRTC test #130
  • node.js-libp2p (DRI: @MarcoPolo )
    • TCP, WebSocket: Add fast multidimensional interop tests #97
    • WebTransport: (link to PR) (this is not supported in node right now)
    • WebRTC: (link to PR) (not supported in node)
    • In repo testing test: Add multidimensional interop test js-libp2p#1615
  • browser js-libp2p Use aegir for testing js-libp2p #104
    • WebSocket: Browser WSS testing #144
    • WebTransport: (link to PR)
    • WebRTC: Add Browser WebRTC test #130
    • WebRTC browser to browser Browser <-> Browser over webrtc-private-to-private #152
    • In repo testing test: Add multidimensional interop test js-libp2p#1615
• Update CI configurations in libp2p/test-plans Support other repos using the multidim interop tests #99 and update
  • go-libp2p: CI: Fast multidimensional Interop tests go-libp2p#1991
  • rust-libp2p
  • js-libp2p: test: Add multidimensional interop test js-libp2p#1615
• Blog post about our new interoperability tests blog#70

[John-LittleBearLabs]

I don't have permission to edit this issue, but I believe it was suggested that we add #82 as a task here.

[GlenDC]

In regards to js-libp2p (node/browser) #74, as in accordance with @laurentsenta we agreed on the following approach moving forward:

1. Implement the libp2p ping test using node.js, with just two nodes that can communicate with each other.
2. Add the test to the composer, using @Tanguy (nim-libp2p)'s recent merge as a guide. This will still be a single node.js version added to the test (https://github.com/libp2p/test-plans/pull/70/files)
3. Expand the configuration to include browser testing.
4. Tackle the challenge of dealing with many versions.

It might make sense to first aim for (1-3) using the latest release of libp2p-js, and do (4) in a secondary PR.

[BigLep]

Thanks for the update @GlenDC . I like the plan to do this incrementally. Can 1&2 be the first PR (just using node), then do 3, then do 4?

[GlenDC]

Hi @BigLep. The goal is definitely to do it incrementally, however given we already the browser+node support in testground thanks to the fact that we start from https://github.com/testground/testground/tree/master/plans/example-browser-node, the plan that we created as an example of how to test plans without modification in both a node and browser environment. For now we have to copy from the plan, but thanks to real-world use cases such as this libp2p-js one, we gather insights on all the requirements and catcha's. Based on that knowledge it will be turned into something thats built-in support.

For the time being we will however need to copy the example as a base, which is what I did already in my draft PR (unfinished for now) and which is why the diff is so big. But in reality the things that are unique to this PR are only the testplan found in the /src directory and the added dependencies to the package.json.

As such my proposed steps are:

1. get the libp2p ping test working between two nodejs instances using a static configuration (so not too fancy in how we configure the libp2p node, but just do it using hardcoded noise encryption and websockets transport).
2. with this in place we can also add a test between nodejs and the browser runtimes (chromium, firefox and webkit)

• given libp2p is supposed to already be browser-ready this should just work out of the box without any additional effort beyond adding the testplan toml

3. with all this working we should be able to add the 4 runtimes for libp2p-js (nodejs, chromium, firefox and webkit) to the same composition files as the Nim PR did recently.

Step (1), (2) and (3) are planned by me to be all done in the first PR, which I aim to have ready next week for review. Given that all dev effort goes into (1), it should be ok. To keep things simple this will all be done using the same libp2p-node config (construction) and it will be all done using he latest npm version of libp2p-js.

In a second PR (once the currently discussed PR is approved and merged) I then plan to work on step (4):

4. Tackle the challenge of dealing with many versions.

In that same (second) PR or a different PR we can also look into allowing tests to be run using different configurations. E.g. also support WebRTC / TCP transports, etc... But not sure what the requirements there are. That would essentially be a step (5).

[p-shahi]

@GlenDC @MarcoPolo:
Glen, do you plan to pick up the WebRTC tests and add them to docker compose as well? I've closed #67 and #84

[GlenDC]

Glen, do you plan to pick up the WebRTC tests and add them to docker compose as well

@p-shahi I could and will do with pleasure. Here are my proposed next steps:

• this week: my first step (now that we have JS node test working for ping: https://github.com/libp2p/test-plans/tree/c35de22/multidim-interop/js/v0.41) will be to add decent dual-runtime support (JS+browser) so we can reuse that code for all versions and js tests (not just v0.41 ping), you can expect PR later this week;
• this week or next week: add WebRTC support to ping test (I suppose you mean that?)
• if you mean to port the WebRTC tests that Little Bear Labs developed in a separate repo then I can do that as well (this week or next week, but that might take a bit longer depending how well that porting goes, especially as the JS ones would be te first to add it here?)

[p-shahi]

Thanks @GlenDC your plan sgtm.

add WebRTC support to ping test (I suppose you mean that?)

Yep

if you mean to port the WebRTC tests that Little Bear Labs developed in a separate repo then I can do that as well (this week or next week, but that might take a bit longer depending how well that porting goes, especially as the JS ones would be te first to add it here?)

Great idea here. We definitely want those in test-plans.

[jxs]

Hi @p-shahi I can continue the work on #84 and add WebRTC support to the Rust version, (sorry wrongly assumed this was already done on #97)

[p-shahi]

That sounds good @jxs thank you!

[p-shahi]

If this sounds good to Marco,
@jxs will you be able to take up the Rust complement of libp2p/go-libp2p#1991
@GlenDC for JS?

[GlenDC]

With some guidance @p-shahi , sure. To be done after my current PR in progress, or after everything else we already discussed?

[jxs]

yeah no worries @p-shahi I will look into it 👍

[MarcoPolo]

Sounds good. If folks need any help with getting the GitHub actions integration working lmk. I’ve already spent a fair bit of time to get this working.