[Proof of concept] Async non-blocking IO using task executor preference #2648
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Motivation During the adoption of our new `NIOAsyncChannel` we came across a few issues such as: - The current bootstraps are not providing scoping to the underlying channel which doesn't work nicely with structured concurrency - Unnecessary buffering between the `NIOAsyncChannel` components and the underlying pipeline - Things like HTTP upgrades are inherently tied to the NIO pipeline; hence, they must use `EventLoopFutures`. This leads to API problems higher up the stack where we cannot spell it nicely with async primitives. The first one can be fixed with new bootstraps but the last one is inherently tied to how our current networking stack works. After thinking about this more, I wanted to try if we could respell NIO with purely asynchronous primitives. This became possible via the newly introduced task executor preference feature that allows us to implement thread hops in normal async code. # Modification This PR is mostly a showcase of how the very lowest level could work for a non-blocking networking stack that is written using concurrency primitives such as `TaskExecutor`s and `AsyncSequence`. This PR shows three separate pieces working: 1. An `IOExecutor: TaskExecutor` that spawns a `pthread`. This thread is running a loop to execute `UnownedJob`s and knows how to handle I/O via a selector (kqueue) 2. A `TCPConenction` that uses the `IOExecutor` to handle non blocking reads and writes. It exposes them via an `AsyncSequence` and and `AsyncWriter` 3. A `TCPListener` that can accept new inbound `TCPConnection`s. # TODOs - Make this work in Linux (epoll) - Fix the CoWs inside the executor - Support Cancellation (only done for the listener right now) - Make sure we close everything correctly - Scoped IOExecutor access - Handle all events and make sure we handle them correctly
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This a PoC PR mostly for educational reasons. Don't expect any more work on this but I wanted to showcase that it is possible and it might be interesting for others in the community.
Motivation
During the adoption of our new
NIOAsyncChannelwe came across a few issues such as:NIOAsyncChannelcomponents and the underlying pipelineEventLoopFutures. This leads to API problems higher up the stack where we cannot spell it nicely with async primitives.The first one can be fixed with new bootstraps but the last one is inherently tied to how our current networking stack works. After thinking about this more, I wanted to try if we could respell NIO with purely asynchronous primitives. This became possible via the newly introduced task executor preference feature that allows us to implement thread hops in normal async code.
Modification
This PR is mostly a showcase of how the very lowest level could work for a non-blocking networking stack that is written using concurrency primitives such as
TaskExecutors andAsyncSequence. This PR shows three separate pieces working:IOExecutor: TaskExecutorthat spawns apthread. This thread is running a loop to executeUnownedJobs and knows how to handle I/O via a selector (kqueue)TCPConenctionthat uses theIOExecutorto handle non blocking reads and writes. It exposes them via anAsyncSequenceand andAsyncWriterTCPListenerthat can accept new inboundTCPConnections.TODOs