Asynchronous streaming of files

[Lssikkes]

I'm trying to use the asynchronous streaming of a file, but the current bundled example of a HTTP server doesn't seem to work right with files larger than 1 MB (and the cache is extremely inefficient, pulling everything into RAM - I'd rather let Linux kernel worry about that and just use fopen/fread/fclose until it becomes a problem).

So far I came up with the following solution (not hardened for security - will get to that later, want to make it work first):

#define AsyncFileStreamer_BLOCK_SIZE (65536)

struct AsyncFileStreamer {
    std::string root;

    AsyncFileStreamer( std::string root ) : root( root ) {}

    template <bool SSL>
    std::pair<bool, bool> sendBlock( uWS::HttpResponse<SSL> *res, FILE *f, size_t offset, size_t len, size_t &actuallyRead ) {
        int blockSize = AsyncFileStreamer_BLOCK_SIZE;
        char block[AsyncFileStreamer_BLOCK_SIZE];

        fseek( f, offset, SEEK_SET );
        actuallyRead = fread( block, 1, blockSize, f );
        actuallyRead = actuallyRead > 0 ? actuallyRead : 0; // truncate anything < 0

        return res->tryEnd( std::string_view( block, actuallyRead ), len );
    }

    template <bool SSL>
    void streamFile( uWS::HttpResponse<SSL> *res, std::string_view url ) {

        // NOTE: This is very unsafe, people can inject things like ../../bla in the URL and completely bypass the root folder restriction.
        // Make sure to harden this code if it's intended to use it in anything internet facing.
        std::string nview = root + "/" + std::string{url.substr( 1 )};

        FILE *f = fopen( nview.c_str(), "rb" );

        if ( f == nullptr ) {
            res->writeStatus( "404 Not Found" );
            res->writeHeader( "Content-Type", "text/html; charset=utf-8" );
            res->end( "<b>404 Not Found</b>" );
        } else {
            res->writeStatus(uWS::HTTP_200_OK);
            fseek( f, 0, SEEK_END );
            long len = ftell( f );
            fseek( f, 0, SEEK_SET );

            res->onAborted( [f]() {
                fclose( f );
            } );

            auto fillStream = [this, len, res, f]() {
                bool retry;
                bool completed;
                do {
                    size_t actuallyRead;
                    auto tryEndResult = sendBlock( res, f, res->getWriteOffset(), len, actuallyRead );
                    retry = tryEndResult.first /*ok*/ && tryEndResult.second == false /*completed == false*/ ;
                    completed = tryEndResult.second;
                } while ( retry );

                if(completed)
                    fclose(f);
            };

            res->onWritable( [this, fillStream]( uintmax_t offset ) {
                fillStream();
                return true;
            } );

            fillStream();
        }
    }
};

EDIT: Fixed the issue with the socket errors in the code above. Just FYI for anyone looking for a more complete sample on how to stream files directly from disk. Note, actually using synchronous I/O. If your storage backend is prone to blocking/stalling/high latency, consider running in a threadpool or use OS dependent asynchronous I/O instead.

[Lssikkes]

I think I found the problem - I was ignoring the .second field from tryEnd (completely done sending), and forcing another end with ->end({},true).
Updated the code and now it seems to work correctly (even though it still might not be optimal?)
Comments would be welcome!

[ghost]

If you want to stream huge files with ideal performance you need to use KTLS and sendfile, but uWS is not really optimized for that scenario (still beats most servers in that case, though). But ideally you let the kernel handle file streaming including TLS completely in kernel space.

[Yuu6883]

Is it possible to integrate KTLS in uWS/usockets? It seems Nginx added support for it in their recent release. It would be nice if we can leverage the optimized kernel feature without relying on an entire middleware stack. File streaming is a seemingly trivial task but hard to implement efficiently even with this library. There's might be some big market potential for this but I can only speculate because I'm inexperienced.

[ghost]

Yeah of course it can be done but it's not something I plan on doing because it's a minor optimization with lots of moving parts. Sure, if say Netflix would give me 100k to do it I could do it but that's not the case now is it ;P

[ghost]

This solution you've presented in not valid. It performs blocking disk operations mid-handler which is entirely broken and will freeze the entire server every time your disk is not immediately responding.

[Lssikkes]

A valid point - though I'm not sure how to integrate asynchronous disk I/O properly with the usocket loop. For my purposes it's fine because I intend to run the server multiple times to prevent starvation, on a fast NVMe drive (random access latency will be minimal). I also need to transform the data to an event stream, so using sendfile doesn't completely cover the situation.

[Lssikkes]

Looking at how node.js solves the issue, looks like they use libuv for a threadpool to do all blocking I/O on to make them appear async (until the threadpool fills up, and it'd still starve). My guess is the best solution is to somehow add a callback mechanism from the kernel whenever file data becomes available, and then use ::defer to inject it back into the uSocket/uWebSocket loop. I might pick it up later if it turns out to be a problem for me down the line.