Download Code Path Refactor Blueprint

[iglesiasbrandon]

Description

This issue aims to create a blueprint for the download code path refactor.
This refactor would eliminate the pause between segment downloads.

Acceptance Criteria:

• A blueprint or tickets are created to tackle this effort.

[iglesiasbrandon]

@zeebo @jtolio We created this ticket, but I'm not entirely sure of the scope of this effort.
Can you shed some light on this for me?

• what are the current pain points?
• what would this refactor solve or help with?

[jtolio]

largely the same things as the upload codepath refactor. we would like to:

• eliminate stalls between segment downloads
• reduce the amount of startup roundtrips
• have dynamic long tail management
• be able to support more concurrency with downloading (largely to support pipelining with stream output, but it would also be useful to support much greater parallelism for seekable writers (nice to have))
• lastly, to enable us to do an uplink-lite (separating encryption from RS, so we can do uplink lite in the browser)

[elek]

• minimize the buffer allocations, decrease the GC pressure
• optimize TTFB (currently full 32k buffer is downloaded and recovered before first byte is server)

[elek]

• support different node selection scheme for downloads (like open 49 connections and use only the 29, where the connection open was the fastest)

[iglesiasbrandon]

We are going to close this ticket because a formal blueprint is not needed. instead what we will do is write a short description of how this is intended to work on the roadmap item itself. @zeebo

storj/roadmap#71

[shaupt131]

Looks like @elek captured our thoughts on the roadmap ticket. Closing this issue.

[iglesiasbrandon]

@shaupt131 we are reopening this issue we need to write a blueprint for the download code path refactor

[iglesiasbrandon]

we created three separate items to complete the overall download code path refactor.
storj/roadmap#79
storj/roadmap#80
storj/roadmap#82