From 1a28b97fa4ee33fb0adb2949b9f87a3e76a58820 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Thu, 6 Oct 2022 15:12:22 +0100 Subject: [PATCH 01/18] Roadmap.md: Discuss the near term future of rust-libp2p Here is my (@mxinden) proposal for a roadmap for the next 6 months for the rust-libp2p maintainers (@elenaf9, @thomaseizinger, @jxs and @mxinden) working on the rust-libp2p project. (rust-)libp2p is an open source and open community project. Thus: - I want to make sure we listen to the community and align our work accordingly. I invite **everyone** to leave feedback here. - The below is only planning work for the maintainers, not the community. Folks from the community may suggest and pick up work not prioritized (e.g. [Kademlia efficient querying](#kademlia-efficient-querying)) nor mentioned on this roadmap and we (maintainers) should be supporting them as best as we can. I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it. In my eyes, continued project management is better done via GitHub issues and thus all the information here should eventually make it into the corresponding GitHub issues if not already the case. Also see: - [libp2p roadmap](https://github.com/libp2p/specs/blob/master/ROADMAP.md) - [go-libp2p roadmap](https://github.com/libp2p/go-libp2p/pull/1784) The items in this roadmap are ordered by importance where I define importance as: - Enable others to build with rust-libp2p before building components ourself. - e.g.[cross behaviour communication](#cross-behaviour-communication) over[Kademlia client mode](#kademlia-client-mode) where the former enables the latter - Invest into tools that enable us to work more efficiently - e.g.[Testground](#testground-abstraction) and [release process](#release-process) - Better do one thing well than many things close to right. - e.g. deprioritize [BitSwap implementation](#bitswap-implementation) and continue on [QUIC](#quic-support) even once first ieration is merged. That does not imply that we won't support community members implementing it. - Improve existing components before introducing new ones. - e.g.[Kademlia client mode](#kademlia-client-mode) before [WebTransport](#webtransport) I am curious what folks think and again encourage everyone to share their thoughts. --- ROADMAP.md | 273 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 ROADMAP.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 00000000000..21c75895085 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,273 @@ +# rust-libp2p Roadmap + + +**Table of Contents** + +- [rust-libp2p Roadmap](#rust-libp2p-roadmap) + - [Testground abstraction](#testground-abstraction) + - [TLS support](#tls-support) + - [QUIC support](#quic-support) + - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) + - [Release process](#release-process) + - [Refactor event handling in `Swarm`](#refactor-event-handling-in-swarm) + - [Cross Behaviour communication](#cross-behaviour-communication) + - [swarm/handler: Decouple ConnectionHandler from {In,Out}boundUpgrade #2863](#swarmhandler-decouple-connectionhandler-from-inoutboundupgrade-2863) + - [Generic connection management](#generic-connection-management) + - [Kademlia efficient querying](#kademlia-efficient-querying) + - [Kademlia client mode](#kademlia-client-mode) + - [Hole punching](#hole-punching) + - [Streaming response protocol aka. the ultimate escape hatch](#streaming-response-protocol-aka-the-ultimate-escape-hatch) + - [Improved WASM support](#improved-wasm-support) + - [Muxer handshake optimization](#muxer-handshake-optimization) + - [Bitswap implementation](#bitswap-implementation) + - [WebTransport](#webtransport) + + + +## Testground abstraction + +Status: todo + +Tracking: https://github.com/libp2p/test-plans/pull/49#issuecomment-1267175415 + +Dependencies: + +Dependents: + +- [QUIC](#quic-support) +- [WebRTC](#webrtc-support-browser-to-server) + +We now have the infrastructure to run cross-implementation and cross-version compatilibilty tests +via https://github.com/libp2p/test-plans and Testground. This setup is rather bare-bone, i.e. today +it only runs a single TCP test. Long term we need to come up with the abstractions to test many +dimensions, e.g. implementations (Go, JS, Rust, Nim), version, transport, ... . + +This will enable us to test QUIC and WebRTC against rust-libp2p itself as well as js-libp2p and +go-libp2p. + +## TLS support + +Status: In progress + +Tracking: https://github.com/libp2p/rust-libp2p/pull/2945 + +Dependencies: + +Dependents: +- [QUIC](#quic-support) + +This allows us to secure both TCP and QUIC connections using TLS. This is a requirement for QUIC +support. Running TLS on top of TCP is a nice to have, since we already have noise. + +## QUIC support + +Status: In progress + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2883 + +Dependencies: +- [TLS](#tls-support) +- [Testground](#testground) + +Dependents: + +QUIC has been on the roadmap for a long time. It enables various performance improvements as well as +higher hole punching success rates. We are close to finishing a first version with +https://github.com/libp2p/rust-libp2p/pull/2289. Long term there is lots more to do, see tracking +issue https://github.com/libp2p/rust-libp2p/issues/2883. + +## WebRTC support (browser-to-server) + +Status: In progress + +Tracking: https://github.com/libp2p/specs/pull/412 + +Dependencies: +- [TLS](#tls-support) +- [Testground](#testground) + +Dependents: + +We are currently implementing WebRTC for **browser-to-server** connectivity in +https://github.com/libp2p/rust-libp2p/pull/2622. More specifically the server side. This will enable +browser nodes to connect to rust-libp2p nodes where the latter only have self-signed TLS +certificates. See https://github.com/libp2p/specs/pull/412 for in-depth motivation. + +Long term we should enable rust-libp2p running in the browser via WASM to use the browser's WebRTC +stack. Though that should only happen after improved WASM support, see below. + +## Release process + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2902 + +Dependencies: + +Dependents: + +I think we are doing a decent job at releasing often. That said, automating this will free up +contributor time (less work on PRs), maintainer time (less work on releases) and overall increase +release frequency (better for users). + +## Refactor event handling in `Swarm` + +Status: In progress + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2832 + +Dependencies: + +Dependents: + +- [Cross behaviour communication](#cross-behaviour-communication) + +More specifically replace the `inject_*` methods on `NetworkBehaviour` and `ConnectionHandler` with +consolidated `on_*_event` handlers. See https://github.com/libp2p/rust-libp2p/issues/2832 and +https://github.com/libp2p/rust-libp2p/pull/2867 for details. + +While a rather small change, this will make using rust-libp2p easier. In my eyes this is a +requirement for generic connection management and cross behaviour communication as either would +otherwise introduce too much complexity. + +## Cross Behaviour communication + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2680 + +Dependencies: + +- [Refactor event handling](#refactor-event-handling-in-swarm) + +Dependents: + +- [Kademlia client mode](#kademlia-client-mode) + +## swarm/handler: Decouple ConnectionHandler from {In,Out}boundUpgrade #2863 + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2863 + +Dependencies: + +Dependents: + +I think this will simplify existing implementations and lower the learning curve required to get +started with rust-libp2p. + +## Generic connection management + +See https://github.com/libp2p/rust-libp2p/issues/2824 for motivation. Given that this will enable +downstream users to easier integrate with rust-libp2p, I think this counts as a "improving existing +components" over "introducing a new component". + +First draft is in https://github.com/libp2p/rust-libp2p/pull/2828 + +## Kademlia efficient querying + +Status: in progress + +Tracking: https://github.com/libp2p/rust-libp2p/pull/2712 + +Dependencies: + +Dependents: + +Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency usage of `libp2p-kad`. + +## Kademlia client mode + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2032 + +Dependencies: + +- [Cross behaviour communication](#cross-behaviour-communication) + +Dependents: + +## Hole punching + +Status: todo + +Dependencies: + +Dependents: + +We released hole punching support with [rust-libp2p +`v0.43.0`](https://github.com/libp2p/rust-libp2p/releases/tag/v0.43.0), see also +https://github.com/libp2p/rust-libp2p/issues/2052. We are currently collecting data via the +[punchr](https://github.com/dennis-tra/punchr) project on the hole punching success rate. See also +[call for +action](https://discuss.libp2p.io/t/decentralized-nat-hole-punching-measurement-campaign/1616) in +case you want to help. Based on this data we will likely find many optimizations we can do to our +hole punching stack. + +## Streaming response protocol aka. the ultimate escape hatch + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2657 + +rust-libp2p is very opinionated on how to write peer-to-peer protocols. There are many good reasons +for this, and I think we should not change directions here. That said, the ultimate escape hatch - +allowing users to create a stream and do whatever they want with it - will make it easier for +newcomers to get started. + +## Improved WASM support + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2617 + +Dependencies: + +Dependents: + +- WebRTC browser-to-browser and browser side + +This opens rust-libp2p to hole new set of use-cases. + +## Muxer handshake optimization + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2994 + +I suggest we do not prioritize this. On the performance end, investing into QUIC support seems like +the better bet. Long term this is a wonderful improvement, likely very much appreciated by users +though not thus far demanded. + +## Bitswap implementation + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2632 + +I think this is a common component that many users need to build peer-to-peer applications. In +addition, it is very performance critical and thus likely challenges many of our existing designs in +rust-libp2p. + +I would prioritize it below [Muxer handshake optimization](#muxer-handshake-optimization) following +the convention of improving existing components over introducing new ones. Users have and can +implement their own implementations and are thus not blocked on the rust-libp2p project. + +## WebTransport + +Status: todo + +Tracking: https://github.com/libp2p/rust-libp2p/issues/2993 + +Dependencies: + +- [QUIC](#quic-support) + +Dependents: + +A WebTransport implementation in rust-libp2p will enable browsers to connect to rust-libp2p nodes +where the latter only have a self-signed TLS certificate. Compared to WebRTC, this would likely be +more performant. It is dependent on QUIC support in rust-libp2p. Given that we will support WebRTC +(browser-to-server) this is not a high priority. From 3c04f3e8544d874bc8ae254701fa954eaa2b8534 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Thu, 6 Oct 2022 15:31:03 +0100 Subject: [PATCH 02/18] ROADMAP: Fix wrong tracking issue --- ROADMAP.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 21c75895085..65be3dd8f11 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -28,7 +28,11 @@ Status: todo -Tracking: https://github.com/libp2p/test-plans/pull/49#issuecomment-1267175415 +Tracking: + +Past discussions: + +- https://github.com/libp2p/test-plans/pull/49#issuecomment-1267175415 Dependencies: From 7a2058af751b1fd550d6171222510bc8eb37e37c Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 17:56:13 +0100 Subject: [PATCH 03/18] ROADMAP: Add target completion to each roadmap item --- ROADMAP.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/ROADMAP.md b/ROADMAP.md index 65be3dd8f11..dec1b077160 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -28,6 +28,8 @@ Status: todo +Target completion: Q4/2022 + Tracking: Past discussions: @@ -53,6 +55,8 @@ go-libp2p. Status: In progress +Target completion: Q4/2022 + Tracking: https://github.com/libp2p/rust-libp2p/pull/2945 Dependencies: @@ -67,6 +71,8 @@ support. Running TLS on top of TCP is a nice to have, since we already have nois Status: In progress +Target completion: Q4/2022 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2883 Dependencies: @@ -84,6 +90,8 @@ issue https://github.com/libp2p/rust-libp2p/issues/2883. Status: In progress +Target completion: Q4/2022 + Tracking: https://github.com/libp2p/specs/pull/412 Dependencies: @@ -104,6 +112,8 @@ stack. Though that should only happen after improved WASM support, see below. Status: todo +Target completion: Q4/2022 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2902 Dependencies: @@ -118,6 +128,8 @@ release frequency (better for users). Status: In progress +Target completion: Q4/2022 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2832 Dependencies: @@ -138,6 +150,8 @@ otherwise introduce too much complexity. Status: todo +Target completion: Q1/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2680 Dependencies: @@ -152,6 +166,8 @@ Dependents: Status: todo +Target completion: Q1/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2863 Dependencies: @@ -163,6 +179,8 @@ started with rust-libp2p. ## Generic connection management +Target completion: Q1/2023 + See https://github.com/libp2p/rust-libp2p/issues/2824 for motivation. Given that this will enable downstream users to easier integrate with rust-libp2p, I think this counts as a "improving existing components" over "introducing a new component". @@ -173,6 +191,8 @@ First draft is in https://github.com/libp2p/rust-libp2p/pull/2828 Status: in progress +Target completion: Q1/2023 + Tracking: https://github.com/libp2p/rust-libp2p/pull/2712 Dependencies: @@ -185,6 +205,8 @@ Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this Status: todo +Target completion: Q1/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2032 Dependencies: @@ -197,6 +219,8 @@ Dependents: Status: todo +Target completion: Q1/2023 + Dependencies: Dependents: @@ -214,6 +238,8 @@ hole punching stack. Status: todo +Target completion: Q1/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2657 rust-libp2p is very opinionated on how to write peer-to-peer protocols. There are many good reasons @@ -225,6 +251,8 @@ newcomers to get started. Status: todo +Target completion: Q2/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2617 Dependencies: @@ -239,6 +267,8 @@ This opens rust-libp2p to hole new set of use-cases. Status: todo +Target completion: Q2/2023 + Tracking: https://github.com/libp2p/rust-libp2p/issues/2994 I suggest we do not prioritize this. On the performance end, investing into QUIC support seems like @@ -249,6 +279,8 @@ though not thus far demanded. Status: todo +Target completion: unknown + Tracking: https://github.com/libp2p/rust-libp2p/issues/2632 I think this is a common component that many users need to build peer-to-peer applications. In @@ -263,6 +295,8 @@ implement their own implementations and are thus not blocked on the rust-libp2p Status: todo +Target completion: unknown + Tracking: https://github.com/libp2p/rust-libp2p/issues/2993 Dependencies: From 95c4cfb4c458aadb98a37799c1084fbb699f3593 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 17:57:44 +0100 Subject: [PATCH 04/18] ROADMAP: Document Kademlia work being in progress by external --- ROADMAP.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index dec1b077160..2bd835eeb17 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -199,7 +199,9 @@ Dependencies: Dependents: -Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency usage of `libp2p-kad`. +Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency +usage of `libp2p-kad`. The rust-libp2p maintainers can pick this up unless iroh folks finish the +work before that. ## Kademlia client mode From 53a74578cc9e2bcc2e2873fe7367edb36edc19d4 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:03:05 +0100 Subject: [PATCH 05/18] ROADMAP: Remove release process item --- ROADMAP.md | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 2bd835eeb17..a3510e2539c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -8,7 +8,6 @@ - [TLS support](#tls-support) - [QUIC support](#quic-support) - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) - - [Release process](#release-process) - [Refactor event handling in `Swarm`](#refactor-event-handling-in-swarm) - [Cross Behaviour communication](#cross-behaviour-communication) - [swarm/handler: Decouple ConnectionHandler from {In,Out}boundUpgrade #2863](#swarmhandler-decouple-connectionhandler-from-inoutboundupgrade-2863) @@ -108,22 +107,6 @@ certificates. See https://github.com/libp2p/specs/pull/412 for in-depth motivati Long term we should enable rust-libp2p running in the browser via WASM to use the browser's WebRTC stack. Though that should only happen after improved WASM support, see below. -## Release process - -Status: todo - -Target completion: Q4/2022 - -Tracking: https://github.com/libp2p/rust-libp2p/issues/2902 - -Dependencies: - -Dependents: - -I think we are doing a decent job at releasing often. That said, automating this will free up -contributor time (less work on PRs), maintainer time (less work on releases) and overall increase -release frequency (better for users). - ## Refactor event handling in `Swarm` Status: In progress From be3170c311dc8a9a867f669531df0bba345c4d73 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:06:59 +0100 Subject: [PATCH 06/18] ROADMAP: Reword decoupling ConnectionHandler item --- ROADMAP.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index a3510e2539c..78067fc5a54 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -10,7 +10,7 @@ - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) - [Refactor event handling in `Swarm`](#refactor-event-handling-in-swarm) - [Cross Behaviour communication](#cross-behaviour-communication) - - [swarm/handler: Decouple ConnectionHandler from {In,Out}boundUpgrade #2863](#swarmhandler-decouple-connectionhandler-from-inoutboundupgrade-2863) + - [Decouple ConnectionHandler from {In,Out}boundUpgrade](#decouple-connectionhandler-from-inoutboundupgrade) - [Generic connection management](#generic-connection-management) - [Kademlia efficient querying](#kademlia-efficient-querying) - [Kademlia client mode](#kademlia-client-mode) @@ -145,7 +145,7 @@ Dependents: - [Kademlia client mode](#kademlia-client-mode) -## swarm/handler: Decouple ConnectionHandler from {In,Out}boundUpgrade #2863 +## Decouple ConnectionHandler from {In,Out}boundUpgrade Status: todo From 1153b915dc5bf691ef55ad47b36f44aa4d0ddbb9 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:21:26 +0100 Subject: [PATCH 07/18] ROADMAP: Include security protocol in multiaddr work --- ROADMAP.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 78067fc5a54..85a16be40b1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -17,7 +17,7 @@ - [Hole punching](#hole-punching) - [Streaming response protocol aka. the ultimate escape hatch](#streaming-response-protocol-aka-the-ultimate-escape-hatch) - [Improved WASM support](#improved-wasm-support) - - [Muxer handshake optimization](#muxer-handshake-optimization) + - [Handshake optimizations](#handshake-optimizations) - [Bitswap implementation](#bitswap-implementation) - [WebTransport](#webtransport) @@ -248,17 +248,23 @@ Dependents: This opens rust-libp2p to hole new set of use-cases. -## Muxer handshake optimization +## Handshake optimizations Status: todo Target completion: Q2/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2994 +Tracking: + +- Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 +- Early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 + +Short term, investing into rust-libp2p's QUIC support will likely give us a larger performance win, +thus neither of the two optimizations is planned for 2022. While great to have, it has not been +requested from any rust-libp2p users. -I suggest we do not prioritize this. On the performance end, investing into QUIC support seems like -the better bet. Long term this is a wonderful improvement, likely very much appreciated by users -though not thus far demanded. +Long term, given that this will give us a great performance gain, we should definitely tackle it. It +also allows us to catch up and thus be consistent with go-libp2p. ## Bitswap implementation From 4db089f64ff740ea11631b7fd9fdb4c20b82cfbe Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:25:21 +0100 Subject: [PATCH 08/18] ROADMAP: Rename to optimized hole punching --- ROADMAP.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 85a16be40b1..ee34b5d081c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ - [Generic connection management](#generic-connection-management) - [Kademlia efficient querying](#kademlia-efficient-querying) - [Kademlia client mode](#kademlia-client-mode) - - [Hole punching](#hole-punching) + - [Optimize Hole punching](#optimize-hole-punching) - [Streaming response protocol aka. the ultimate escape hatch](#streaming-response-protocol-aka-the-ultimate-escape-hatch) - [Improved WASM support](#improved-wasm-support) - [Handshake optimizations](#handshake-optimizations) @@ -200,7 +200,7 @@ Dependencies: Dependents: -## Hole punching +## Optimize Hole punching Status: todo From aba46bf96574ab11b466541e74854297ebf113f9 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:31:36 +0100 Subject: [PATCH 09/18] ROADMAP: Expand on WASM priority rational --- ROADMAP.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index ee34b5d081c..d26596a3a0e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -246,7 +246,10 @@ Dependents: - WebRTC browser-to-browser and browser side -This opens rust-libp2p to hole new set of use-cases. +This opens rust-libp2p to hole new set of use-cases. I would love for this to happen earlier. Though +(a) I think we should prioritize improving existing functionality over new functionality and (b) we +don't have high demand for this feature from the community. (One could argue that that demand +follows this roadmap item and not the other way round.) ## Handshake optimizations From 1e5e2d0f57e578c27eaf8b0f24aac0e571a46641 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 9 Oct 2022 18:37:11 +0100 Subject: [PATCH 10/18] ROADMAP: Formatting --- ROADMAP.md | 146 ++++++++++++++++++++++++++--------------------------- 1 file changed, 73 insertions(+), 73 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index d26596a3a0e..675755b09b7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -25,19 +25,19 @@ ## Testground abstraction -Status: todo +**Status:** todo -Target completion: Q4/2022 +**Target completion:** Q4/2022 -Tracking: +**Tracking:** -Past discussions: +**Past discussions:** - https://github.com/libp2p/test-plans/pull/49#issuecomment-1267175415 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** - [QUIC](#quic-support) - [WebRTC](#webrtc-support-browser-to-server) @@ -52,15 +52,15 @@ go-libp2p. ## TLS support -Status: In progress +**Status:** In progress -Target completion: Q4/2022 +**Target completion:** Q4/2022 -Tracking: https://github.com/libp2p/rust-libp2p/pull/2945 +**Tracking:** https://github.com/libp2p/rust-libp2p/pull/2945 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** - [QUIC](#quic-support) This allows us to secure both TCP and QUIC connections using TLS. This is a requirement for QUIC @@ -68,17 +68,17 @@ support. Running TLS on top of TCP is a nice to have, since we already have nois ## QUIC support -Status: In progress +**Status:** In progress -Target completion: Q4/2022 +**Target completion:** Q4/2022 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2883 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2883 -Dependencies: +**Dependencies:** - [TLS](#tls-support) - [Testground](#testground) -Dependents: +**Dependents:** QUIC has been on the roadmap for a long time. It enables various performance improvements as well as higher hole punching success rates. We are close to finishing a first version with @@ -87,17 +87,17 @@ issue https://github.com/libp2p/rust-libp2p/issues/2883. ## WebRTC support (browser-to-server) -Status: In progress +**Status:** In progress -Target completion: Q4/2022 +**Target completion:** Q4/2022 -Tracking: https://github.com/libp2p/specs/pull/412 +**Tracking:** https://github.com/libp2p/specs/pull/412 -Dependencies: +**Dependencies:** - [TLS](#tls-support) - [Testground](#testground) -Dependents: +**Dependents:** We are currently implementing WebRTC for **browser-to-server** connectivity in https://github.com/libp2p/rust-libp2p/pull/2622. More specifically the server side. This will enable @@ -109,15 +109,15 @@ stack. Though that should only happen after improved WASM support, see below. ## Refactor event handling in `Swarm` -Status: In progress +**Status:** In progress -Target completion: Q4/2022 +**Target completion:** Q4/2022 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2832 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2832 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** - [Cross behaviour communication](#cross-behaviour-communication) @@ -131,38 +131,38 @@ otherwise introduce too much complexity. ## Cross Behaviour communication -Status: todo +**Status:** todo -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2680 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2680 -Dependencies: +**Dependencies:** - [Refactor event handling](#refactor-event-handling-in-swarm) -Dependents: +**Dependents:** - [Kademlia client mode](#kademlia-client-mode) ## Decouple ConnectionHandler from {In,Out}boundUpgrade -Status: todo +**Status:** todo -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2863 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2863 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** I think this will simplify existing implementations and lower the learning curve required to get started with rust-libp2p. ## Generic connection management -Target completion: Q1/2023 +**Target completion:** Q1/2023 See https://github.com/libp2p/rust-libp2p/issues/2824 for motivation. Given that this will enable downstream users to easier integrate with rust-libp2p, I think this counts as a "improving existing @@ -172,15 +172,15 @@ First draft is in https://github.com/libp2p/rust-libp2p/pull/2828 ## Kademlia efficient querying -Status: in progress +**Status:** in progress -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Tracking: https://github.com/libp2p/rust-libp2p/pull/2712 +**Tracking:** https://github.com/libp2p/rust-libp2p/pull/2712 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency usage of `libp2p-kad`. The rust-libp2p maintainers can pick this up unless iroh folks finish the @@ -188,27 +188,27 @@ work before that. ## Kademlia client mode -Status: todo +**Status:** todo -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2032 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2032 -Dependencies: +**Dependencies:** - [Cross behaviour communication](#cross-behaviour-communication) -Dependents: +**Dependents:** ## Optimize Hole punching -Status: todo +**Status:** todo -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** We released hole punching support with [rust-libp2p `v0.43.0`](https://github.com/libp2p/rust-libp2p/releases/tag/v0.43.0), see also @@ -221,11 +221,11 @@ hole punching stack. ## Streaming response protocol aka. the ultimate escape hatch -Status: todo +**Status:** todo -Target completion: Q1/2023 +**Target completion:** Q1/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2657 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2657 rust-libp2p is very opinionated on how to write peer-to-peer protocols. There are many good reasons for this, and I think we should not change directions here. That said, the ultimate escape hatch - @@ -234,15 +234,15 @@ newcomers to get started. ## Improved WASM support -Status: todo +**Status:** todo -Target completion: Q2/2023 +**Target completion:** Q2/2023 -Tracking: https://github.com/libp2p/rust-libp2p/issues/2617 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2617 -Dependencies: +**Dependencies:** -Dependents: +**Dependents:** - WebRTC browser-to-browser and browser side @@ -253,11 +253,11 @@ follows this roadmap item and not the other way round.) ## Handshake optimizations -Status: todo +**Status:** todo -Target completion: Q2/2023 +**Target completion:** Q2/2023 -Tracking: +**Tracking:** - Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 - Early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 @@ -271,33 +271,33 @@ also allows us to catch up and thus be consistent with go-libp2p. ## Bitswap implementation -Status: todo +**Status:** todo -Target completion: unknown +**Target completion:** unknown -Tracking: https://github.com/libp2p/rust-libp2p/issues/2632 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2632 I think this is a common component that many users need to build peer-to-peer applications. In addition, it is very performance critical and thus likely challenges many of our existing designs in rust-libp2p. -I would prioritize it below [Muxer handshake optimization](#muxer-handshake-optimization) following -the convention of improving existing components over introducing new ones. Users have and can -implement their own implementations and are thus not blocked on the rust-libp2p project. +I would prioritize it below [handshake optimization](#handshake-optimizations) following the +convention of improving existing components over introducing new ones. Users have and can implement +their own implementations and are thus not blocked on the rust-libp2p project. ## WebTransport -Status: todo +**Status:** todo -Target completion: unknown +**Target completion:** unknown -Tracking: https://github.com/libp2p/rust-libp2p/issues/2993 +**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2993 -Dependencies: +**Dependencies:** - [QUIC](#quic-support) -Dependents: +**Dependents:** A WebTransport implementation in rust-libp2p will enable browsers to connect to rust-libp2p nodes where the latter only have a self-signed TLS certificate. Compared to WebRTC, this would likely be From 538b5b49e437a83abd7cc09902ce6217ee6a1f7f Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 18:48:56 +0100 Subject: [PATCH 11/18] ROADMAP: Remove testground entry This is tracked on the [libp2p test-plans](https://github.com/libp2p/test-plans/pull/44/) see also https://github.com/libp2p/test-plans/issues/53. --- ROADMAP.md | 32 ++------------------------------ 1 file changed, 2 insertions(+), 30 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 675755b09b7..f7116daa316 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,7 +4,6 @@ **Table of Contents** - [rust-libp2p Roadmap](#rust-libp2p-roadmap) - - [Testground abstraction](#testground-abstraction) - [TLS support](#tls-support) - [QUIC support](#quic-support) - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) @@ -23,33 +22,6 @@ -## Testground abstraction - -**Status:** todo - -**Target completion:** Q4/2022 - -**Tracking:** - -**Past discussions:** - -- https://github.com/libp2p/test-plans/pull/49#issuecomment-1267175415 - -**Dependencies:** - -**Dependents:** - -- [QUIC](#quic-support) -- [WebRTC](#webrtc-support-browser-to-server) - -We now have the infrastructure to run cross-implementation and cross-version compatilibilty tests -via https://github.com/libp2p/test-plans and Testground. This setup is rather bare-bone, i.e. today -it only runs a single TCP test. Long term we need to come up with the abstractions to test many -dimensions, e.g. implementations (Go, JS, Rust, Nim), version, transport, ... . - -This will enable us to test QUIC and WebRTC against rust-libp2p itself as well as js-libp2p and -go-libp2p. - ## TLS support **Status:** In progress @@ -76,7 +48,7 @@ support. Running TLS on top of TCP is a nice to have, since we already have nois **Dependencies:** - [TLS](#tls-support) -- [Testground](#testground) +- https://github.com/libp2p/test-plans/issues/53 **Dependents:** @@ -95,7 +67,7 @@ issue https://github.com/libp2p/rust-libp2p/issues/2883. **Dependencies:** - [TLS](#tls-support) -- [Testground](#testground) +- https://github.com/libp2p/test-plans/issues/53 **Dependents:** From 0b071eb3aba20fc7d52a9697044c041a8056fd20 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 18:51:04 +0100 Subject: [PATCH 12/18] ROADMAP: Remove event handling refactoring This is not interesting in itself for stakeholders. Instead it will be tracked with the cross behaviour communication. See also https://github.com/libp2p/rust-libp2p/issues/2832. --- ROADMAP.md | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index f7116daa316..86d99911d4c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -7,7 +7,6 @@ - [TLS support](#tls-support) - [QUIC support](#quic-support) - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) - - [Refactor event handling in `Swarm`](#refactor-event-handling-in-swarm) - [Cross Behaviour communication](#cross-behaviour-communication) - [Decouple ConnectionHandler from {In,Out}boundUpgrade](#decouple-connectionhandler-from-inoutboundupgrade) - [Generic connection management](#generic-connection-management) @@ -79,28 +78,6 @@ certificates. See https://github.com/libp2p/specs/pull/412 for in-depth motivati Long term we should enable rust-libp2p running in the browser via WASM to use the browser's WebRTC stack. Though that should only happen after improved WASM support, see below. -## Refactor event handling in `Swarm` - -**Status:** In progress - -**Target completion:** Q4/2022 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2832 - -**Dependencies:** - -**Dependents:** - -- [Cross behaviour communication](#cross-behaviour-communication) - -More specifically replace the `inject_*` methods on `NetworkBehaviour` and `ConnectionHandler` with -consolidated `on_*_event` handlers. See https://github.com/libp2p/rust-libp2p/issues/2832 and -https://github.com/libp2p/rust-libp2p/pull/2867 for details. - -While a rather small change, this will make using rust-libp2p easier. In my eyes this is a -requirement for generic connection management and cross behaviour communication as either would -otherwise introduce too much complexity. - ## Cross Behaviour communication **Status:** todo @@ -111,7 +88,7 @@ otherwise introduce too much complexity. **Dependencies:** -- [Refactor event handling](#refactor-event-handling-in-swarm) +- https://github.com/libp2p/rust-libp2p/issues/2832 **Dependents:** From af13bb7759d919c72690924c108f26bceb83dc13 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 18:52:33 +0100 Subject: [PATCH 13/18] ROADMAP: Remove removal of {In,Out}boundUpgrade on ConnectionHandler This is not directly relevant for stakeholders. Will be tracked on the project board. https://github.com/orgs/libp2p/projects/4 --- ROADMAP.md | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 86d99911d4c..9418b7a74b3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -94,21 +94,6 @@ stack. Though that should only happen after improved WASM support, see below. - [Kademlia client mode](#kademlia-client-mode) -## Decouple ConnectionHandler from {In,Out}boundUpgrade - -**Status:** todo - -**Target completion:** Q1/2023 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2863 - -**Dependencies:** - -**Dependents:** - -I think this will simplify existing implementations and lower the learning curve required to get -started with rust-libp2p. - ## Generic connection management **Target completion:** Q1/2023 From 8dcee71c7fff478240aa18929d25cc3e3df9c6e5 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 19:02:03 +0100 Subject: [PATCH 14/18] ROADMAP: Track experimental QUIC support only for now --- ROADMAP.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 9418b7a74b3..897bcdd9971 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -5,10 +5,9 @@ - [rust-libp2p Roadmap](#rust-libp2p-roadmap) - [TLS support](#tls-support) - - [QUIC support](#quic-support) + - [Experimental QUIC support](#experimental-quic-support) - [WebRTC support (browser-to-server)](#webrtc-support-browser-to-server) - [Cross Behaviour communication](#cross-behaviour-communication) - - [Decouple ConnectionHandler from {In,Out}boundUpgrade](#decouple-connectionhandler-from-inoutboundupgrade) - [Generic connection management](#generic-connection-management) - [Kademlia efficient querying](#kademlia-efficient-querying) - [Kademlia client mode](#kademlia-client-mode) @@ -32,12 +31,12 @@ **Dependencies:** **Dependents:** -- [QUIC](#quic-support) +- [QUIC](#experimental-quic-support) This allows us to secure both TCP and QUIC connections using TLS. This is a requirement for QUIC support. Running TLS on top of TCP is a nice to have, since we already have noise. -## QUIC support +## Experimental QUIC support **Status:** In progress @@ -53,8 +52,11 @@ support. Running TLS on top of TCP is a nice to have, since we already have nois QUIC has been on the roadmap for a long time. It enables various performance improvements as well as higher hole punching success rates. We are close to finishing a first version with -https://github.com/libp2p/rust-libp2p/pull/2289. Long term there is lots more to do, see tracking -issue https://github.com/libp2p/rust-libp2p/issues/2883. +https://github.com/libp2p/rust-libp2p/pull/2289. + +Long term there is lots more to do, see tracking issue +https://github.com/libp2p/rust-libp2p/issues/2883. Though this will be tackled as a new roadmap +item. ## WebRTC support (browser-to-server) From dd85a8bb02fadc4aa1b9521e70e9d9bc1661dc76 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 19:16:02 +0100 Subject: [PATCH 15/18] ROADMAP: Reformat header fields into tables --- ROADMAP.md | 155 ++++++++++++++++------------------------------------- 1 file changed, 47 insertions(+), 108 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 897bcdd9971..b57c9c2caa3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -22,33 +22,19 @@ ## TLS support -**Status:** In progress +| Status | Target Completion | Tracking | Dependencies | Dependents | +|-------------|-------------------|-------------------------------------------------|------------------------------------|------------| +| In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/pull/2945 | [QUIC](#experimental-quic-support) | | -**Target completion:** Q4/2022 - -**Tracking:** https://github.com/libp2p/rust-libp2p/pull/2945 - -**Dependencies:** - -**Dependents:** -- [QUIC](#experimental-quic-support) This allows us to secure both TCP and QUIC connections using TLS. This is a requirement for QUIC support. Running TLS on top of TCP is a nice to have, since we already have noise. ## Experimental QUIC support -**Status:** In progress - -**Target completion:** Q4/2022 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2883 - -**Dependencies:** -- [TLS](#tls-support) -- https://github.com/libp2p/test-plans/issues/53 - -**Dependents:** +| Status | Target Completion | Tracking | Dependencies | Dependents | +|-------------|-------------------|---------------------------------------------------|---------------------------------------------------------------------|------------| +| In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/issues/2883 | [TLS](#tls-support), https://github.com/libp2p/test-plans/issues/53 | | QUIC has been on the roadmap for a long time. It enables various performance improvements as well as higher hole punching success rates. We are close to finishing a first version with @@ -60,17 +46,10 @@ item. ## WebRTC support (browser-to-server) -**Status:** In progress - -**Target completion:** Q4/2022 - -**Tracking:** https://github.com/libp2p/specs/pull/412 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|-------------|-------------------|------------------------------------------|------------------------------------------------|------------| +| In progress | Q4/2022 | https://github.com/libp2p/specs/pull/412 | https://github.com/libp2p/test-plans/issues/53 | | -**Dependencies:** -- [TLS](#tls-support) -- https://github.com/libp2p/test-plans/issues/53 - -**Dependents:** We are currently implementing WebRTC for **browser-to-server** connectivity in https://github.com/libp2p/rust-libp2p/pull/2622. More specifically the server side. This will enable @@ -82,23 +61,23 @@ stack. Though that should only happen after improved WASM support, see below. ## Cross Behaviour communication -**Status:** todo - -**Target completion:** Q1/2023 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|---------------------------------------------------|-----------------------------------------------| +| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2680 | https://github.com/libp2p/rust-libp2p/issues/2832 | [Kademlia client mode](#kademlia-client-mode) | -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2680 +Today `NetworkBehaviour` implementations like Kademlia, GossipSub or Circuit Relay v2 can not +communicate with each other, i.e. can not make use of information known by another +`NetworkBehaviour` implementation. First step would be to exchange standard information about remote +peers (e.g. supported protocols). -**Dependencies:** - -- https://github.com/libp2p/rust-libp2p/issues/2832 - -**Dependents:** - -- [Kademlia client mode](#kademlia-client-mode) +Long term we might consider a generic approach for `NetworkBehaviours` to exchange data. Though that +would deserve its own roadmap item. ## Generic connection management -**Target completion:** Q1/2023 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|--------------|------------| +| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2824 | | | See https://github.com/libp2p/rust-libp2p/issues/2824 for motivation. Given that this will enable downstream users to easier integrate with rust-libp2p, I think this counts as a "improving existing @@ -108,15 +87,9 @@ First draft is in https://github.com/libp2p/rust-libp2p/pull/2828 ## Kademlia efficient querying -**Status:** in progress - -**Target completion:** Q1/2023 - -**Tracking:** https://github.com/libp2p/rust-libp2p/pull/2712 - -**Dependencies:** - -**Dependents:** +| Status | Target Completion | Tracking | Dependencies | Dependents | +|-------------|-------------------|-------------------------------------------------|--------------|------------| +| in progress | Q1/2023 | https://github.com/libp2p/rust-libp2p/pull/2712 | | | Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency usage of `libp2p-kad`. The rust-libp2p maintainers can pick this up unless iroh folks finish the @@ -124,27 +97,18 @@ work before that. ## Kademlia client mode -**Status:** todo - -**Target completion:** Q1/2023 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2032 - -**Dependencies:** +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|-----------------------------------------------------------------|------------| +| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2032 | [Cross behaviour communication](#cross-behaviour-communication) | | -- [Cross behaviour communication](#cross-behaviour-communication) - -**Dependents:** +Kademlia client mode will enhance routing table health and thus have a positive impact on all +Kademlia operations. ## Optimize Hole punching -**Status:** todo - -**Target completion:** Q1/2023 - -**Dependencies:** - -**Dependents:** +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|----------|--------------|------------| +| todo | Q1/2023 | | | | We released hole punching support with [rust-libp2p `v0.43.0`](https://github.com/libp2p/rust-libp2p/releases/tag/v0.43.0), see also @@ -157,11 +121,9 @@ hole punching stack. ## Streaming response protocol aka. the ultimate escape hatch -**Status:** todo - -**Target completion:** Q1/2023 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2657 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|--------------|------------| +| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2657 | | | rust-libp2p is very opinionated on how to write peer-to-peer protocols. There are many good reasons for this, and I think we should not change directions here. That said, the ultimate escape hatch - @@ -170,17 +132,9 @@ newcomers to get started. ## Improved WASM support -**Status:** todo - -**Target completion:** Q2/2023 - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2617 - -**Dependencies:** - -**Dependents:** - -- WebRTC browser-to-browser and browser side +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|--------------|--------------------------------------------| +| todo | Q2/2023 | https://github.com/libp2p/rust-libp2p/issues/2617 | | WebRTC browser-to-browser and browser side | This opens rust-libp2p to hole new set of use-cases. I would love for this to happen earlier. Though (a) I think we should prioritize improving existing functionality over new functionality and (b) we @@ -189,14 +143,9 @@ follows this roadmap item and not the other way round.) ## Handshake optimizations -**Status:** todo - -**Target completion:** Q2/2023 - -**Tracking:** - -- Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 -- Early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------| +| todo | Q2/2023 | - Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 and early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 | | | Short term, investing into rust-libp2p's QUIC support will likely give us a larger performance win, thus neither of the two optimizations is planned for 2022. While great to have, it has not been @@ -207,11 +156,9 @@ also allows us to catch up and thus be consistent with go-libp2p. ## Bitswap implementation -**Status:** todo - -**Target completion:** unknown - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2632 +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|--------------|------------| +| todo | | https://github.com/libp2p/rust-libp2p/issues/2632 | | | I think this is a common component that many users need to build peer-to-peer applications. In addition, it is very performance critical and thus likely challenges many of our existing designs in @@ -223,17 +170,9 @@ their own implementations and are thus not blocked on the rust-libp2p project. ## WebTransport -**Status:** todo - -**Target completion:** unknown - -**Tracking:** https://github.com/libp2p/rust-libp2p/issues/2993 - -**Dependencies:** - -- [QUIC](#quic-support) - -**Dependents:** +| Status | Target Completion | Tracking | Dependencies | Dependents | +|--------|-------------------|---------------------------------------------------|------------------------------------|------------| +| todo | | https://github.com/libp2p/rust-libp2p/issues/2993 | [QUIC](#experimental-quic-support) | | A WebTransport implementation in rust-libp2p will enable browsers to connect to rust-libp2p nodes where the latter only have a self-signed TLS certificate. Compared to WebRTC, this would likely be From 16bdde7b292cd5499e9633140f2f9cfa6c24399a Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 19:17:54 +0100 Subject: [PATCH 16/18] ROADMAP: Reference project board --- ROADMAP.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index b57c9c2caa3..3576b2bd7e9 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,9 @@ # rust-libp2p Roadmap +Below is a high level roadmap for the rust-libp2p project. + +For a more detailed view see https://github.com/orgs/libp2p/projects/4/views/1. + **Table of Contents** @@ -145,7 +149,7 @@ follows this roadmap item and not the other way round.) | Status | Target Completion | Tracking | Dependencies | Dependents | |--------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------| -| todo | Q2/2023 | - Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 and early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 | | | +| todo | Q2/2023 | Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 and early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 | | | Short term, investing into rust-libp2p's QUIC support will likely give us a larger performance win, thus neither of the two optimizations is planned for 2022. While great to have, it has not been From aed8b29ceff9e963e494b11d27efe61c47f53f91 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 11 Oct 2022 19:28:12 +0100 Subject: [PATCH 17/18] ROADMAP: Mention that items are sorted by priority --- ROADMAP.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 3576b2bd7e9..cd9052def35 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,7 @@ # rust-libp2p Roadmap -Below is a high level roadmap for the rust-libp2p project. +Below is a high level roadmap for the rust-libp2p project. Items are ordered by priority (high to +low). For a more detailed view see https://github.com/orgs/libp2p/projects/4/views/1. From 6fca1a229e8792fc143d6de0deba76b9eae38db5 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 17 Oct 2022 12:48:25 +0100 Subject: [PATCH 18/18] ROADMAP: Introduce category --- ROADMAP.md | 100 ++++++++++++++++++++++++++++------------------------- 1 file changed, 52 insertions(+), 48 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index cd9052def35..f64b1a5bc82 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -27,9 +27,9 @@ For a more detailed view see https://github.com/orgs/libp2p/projects/4/views/1. ## TLS support -| Status | Target Completion | Tracking | Dependencies | Dependents | -|-------------|-------------------|-------------------------------------------------|------------------------------------|------------| -| In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/pull/2945 | [QUIC](#experimental-quic-support) | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|-------------|-------------------|-------------------------------------------------|------------------------------------|------------| +| Connectivity | In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/pull/2945 | [QUIC](#experimental-quic-support) | | This allows us to secure both TCP and QUIC connections using TLS. This is a requirement for QUIC @@ -37,9 +37,9 @@ support. Running TLS on top of TCP is a nice to have, since we already have nois ## Experimental QUIC support -| Status | Target Completion | Tracking | Dependencies | Dependents | -|-------------|-------------------|---------------------------------------------------|---------------------------------------------------------------------|------------| -| In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/issues/2883 | [TLS](#tls-support), https://github.com/libp2p/test-plans/issues/53 | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|-------------|-------------------|---------------------------------------------------|---------------------------------------------------------------------|------------| +| Connectivity | In progress | Q4/2022 | https://github.com/libp2p/rust-libp2p/issues/2883 | [TLS](#tls-support), https://github.com/libp2p/test-plans/issues/53 | | QUIC has been on the roadmap for a long time. It enables various performance improvements as well as higher hole punching success rates. We are close to finishing a first version with @@ -51,9 +51,9 @@ item. ## WebRTC support (browser-to-server) -| Status | Target Completion | Tracking | Dependencies | Dependents | -|-------------|-------------------|------------------------------------------|------------------------------------------------|------------| -| In progress | Q4/2022 | https://github.com/libp2p/specs/pull/412 | https://github.com/libp2p/test-plans/issues/53 | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|-------------|-------------------|------------------------------------------|------------------------------------------------|------------| +| Connectivity | In progress | Q4/2022 | https://github.com/libp2p/specs/pull/412 | https://github.com/libp2p/test-plans/issues/53 | | We are currently implementing WebRTC for **browser-to-server** connectivity in @@ -66,35 +66,38 @@ stack. Though that should only happen after improved WASM support, see below. ## Cross Behaviour communication -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|---------------------------------------------------|-----------------------------------------------| -| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2680 | https://github.com/libp2p/rust-libp2p/issues/2832 | [Kademlia client mode](#kademlia-client-mode) | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|----------------------|--------|-------------------|---------------------------------------------------|---------------------------------------------------|-----------------------------------------------| +| Developer ergonomics | todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2680 | https://github.com/libp2p/rust-libp2p/issues/2832 | [Kademlia client mode](#kademlia-client-mode) | Today `NetworkBehaviour` implementations like Kademlia, GossipSub or Circuit Relay v2 can not communicate with each other, i.e. can not make use of information known by another -`NetworkBehaviour` implementation. First step would be to exchange standard information about remote -peers (e.g. supported protocols). +`NetworkBehaviour` implementation. Users need to write the wiring code by hand to e.g. enable +Kademlia to learn protocols supported by a remote peer from Identify. + +This roadmap item contains exchanging standard information about remote peers (e.g. supported +protocols) between `NetworkBehaviour` implementations. Long term we might consider a generic approach for `NetworkBehaviours` to exchange data. Though that would deserve its own roadmap item. ## Generic connection management -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|--------------|------------| -| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2824 | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|----------------------|--------|-------------------|---------------------------------------------------|--------------|------------| +| Developer Ergonomics | todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2824 | | | -See https://github.com/libp2p/rust-libp2p/issues/2824 for motivation. Given that this will enable -downstream users to easier integrate with rust-libp2p, I think this counts as a "improving existing -components" over "introducing a new component". +Today connection management functionality in rust-libp2p is limited. Building abstractions on top is +cumbersome and inefficient. See https://github.com/libp2p/rust-libp2p/issues/2824. Making connection +management generic allows users to build advanced and efficient abstractions on top of rust-libp2p First draft is in https://github.com/libp2p/rust-libp2p/pull/2828 ## Kademlia efficient querying -| Status | Target Completion | Tracking | Dependencies | Dependents | -|-------------|-------------------|-------------------------------------------------|--------------|------------| -| in progress | Q1/2023 | https://github.com/libp2p/rust-libp2p/pull/2712 | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|-------------|-------------------|-------------------------------------------------|--------------|------------| +| Optimization | in progress | Q1/2023 | https://github.com/libp2p/rust-libp2p/pull/2712 | | | Users of rust-libp2p like [iroh](https://github.com/n0-computer/iroh) need this for low latency usage of `libp2p-kad`. The rust-libp2p maintainers can pick this up unless iroh folks finish the @@ -102,18 +105,18 @@ work before that. ## Kademlia client mode -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|-----------------------------------------------------------------|------------| -| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2032 | [Cross behaviour communication](#cross-behaviour-communication) | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|--------|-------------------|---------------------------------------------------|-----------------------------------------------------------------|------------| +| Optimization | todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2032 | [Cross behaviour communication](#cross-behaviour-communication) | | Kademlia client mode will enhance routing table health and thus have a positive impact on all Kademlia operations. ## Optimize Hole punching -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|----------|--------------|------------| -| todo | Q1/2023 | | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|--------|-------------------|----------|--------------|------------| +| Optimization | todo | Q1/2023 | | | | We released hole punching support with [rust-libp2p `v0.43.0`](https://github.com/libp2p/rust-libp2p/releases/tag/v0.43.0), see also @@ -126,9 +129,9 @@ hole punching stack. ## Streaming response protocol aka. the ultimate escape hatch -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|--------------|------------| -| todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2657 | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|----------------------|--------|-------------------|---------------------------------------------------|--------------|------------| +| Developer ergonomics | todo | Q1/2023 | https://github.com/libp2p/rust-libp2p/issues/2657 | | | rust-libp2p is very opinionated on how to write peer-to-peer protocols. There are many good reasons for this, and I think we should not change directions here. That said, the ultimate escape hatch - @@ -137,20 +140,21 @@ newcomers to get started. ## Improved WASM support -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|--------------|--------------------------------------------| -| todo | Q2/2023 | https://github.com/libp2p/rust-libp2p/issues/2617 | | WebRTC browser-to-browser and browser side | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|----------------------|--------|-------------------|---------------------------------------------------|--------------|--------------------------------------------| +| Developer ergonomics | todo | Q2/2023 | https://github.com/libp2p/rust-libp2p/issues/2617 | | WebRTC browser-to-browser and browser side | -This opens rust-libp2p to hole new set of use-cases. I would love for this to happen earlier. Though -(a) I think we should prioritize improving existing functionality over new functionality and (b) we -don't have high demand for this feature from the community. (One could argue that that demand -follows this roadmap item and not the other way round.) +The project supports WASM already today, though the developer experience is cumbersome at best. +Properly supporting WASM opens rust-libp2p to hole new set of use-cases. I would love for this to +happen earlier. Though (a) I think we should prioritize improving existing functionality over new +functionality and (b) we don't have high demand for this feature from the community. (One could +argue that that demand follows this roadmap item and not the other way round.) ## Handshake optimizations -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------| -| todo | Q2/2023 | Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 and early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|--------------|--------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------| +| Optimization | todo | Q2/2023 | Security protocol in multiaddr https://github.com/libp2p/specs/pull/353 and early muxer negotiation https://github.com/libp2p/rust-libp2p/issues/2994 | | | Short term, investing into rust-libp2p's QUIC support will likely give us a larger performance win, thus neither of the two optimizations is planned for 2022. While great to have, it has not been @@ -161,9 +165,9 @@ also allows us to catch up and thus be consistent with go-libp2p. ## Bitswap implementation -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|--------------|------------| -| todo | | https://github.com/libp2p/rust-libp2p/issues/2632 | | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|----------|--------|-------------------|---------------------------------------------------|--------------|------------| +| | todo | | https://github.com/libp2p/rust-libp2p/issues/2632 | | | I think this is a common component that many users need to build peer-to-peer applications. In addition, it is very performance critical and thus likely challenges many of our existing designs in @@ -175,9 +179,9 @@ their own implementations and are thus not blocked on the rust-libp2p project. ## WebTransport -| Status | Target Completion | Tracking | Dependencies | Dependents | -|--------|-------------------|---------------------------------------------------|------------------------------------|------------| -| todo | | https://github.com/libp2p/rust-libp2p/issues/2993 | [QUIC](#experimental-quic-support) | | +| Category | Status | Target Completion | Tracking | Dependencies | Dependents | +|-----------------------------|--------|-------------------|---------------------------------------------------|------------------------------------|------------| +| Connectivity / optimization | todo | | https://github.com/libp2p/rust-libp2p/issues/2993 | [QUIC](#experimental-quic-support) | | A WebTransport implementation in rust-libp2p will enable browsers to connect to rust-libp2p nodes where the latter only have a self-signed TLS certificate. Compared to WebRTC, this would likely be