HTTP2 without TLS

[WhyNotHugo]

The README explains how to set up HTTP2 with TLS, but there's no indication of how to set it up without TLS.

Just for context: my interest in doing this is because my load balancer already does the TLS termination. There's little sense in me setting up a pipeline to provision certificates to my django instances -- and the overhead of TLS between the load balancer and Django doesn't really make sense.

I've installed the optional dependencies:

pip install -U 'Twisted[tls,http2]'

And daphne indicates it supports HTTP2:

root@e4376f859b81:/app# bash scripts/run-django
[   DEBUG 2020-11-02 12:09:40] django.request:120  Asynchronous middleware django.middleware.clickjacking.XFrameOptionsMiddleware adapted.
[   DEBUG 2020-11-02 12:09:40] asyncio:59  Using selector: EpollSelector
[    INFO 2020-11-02 12:09:40] daphne.cli:287  Starting server at tcp:port=8000:interface=0.0.0.0
[    INFO 2020-11-02 12:09:40] daphne.server:108  HTTP/2 support enabled
[    INFO 2020-11-02 12:09:40] daphne.server:119  Configuring endpoint tcp:port=8000:interface=0.0.0.0
[    INFO 2020-11-02 12:09:40] daphne.server:150  Listening on TCP address 0.0.0.0:8000

However, it does not seem to actually operate on HTTP2, even using things like:

❯ curl -vI HEAD --http2 http://www.myapp.localhost:8000/
* Could not resolve host: HEAD
* Closing connection 0
curl: (6) Could not resolve host: HEAD
*   Trying ::1:8000...
* Connected to www.myapp.localhost (::1) port 8000 (#1)
> HEAD / HTTP/1.1
> Host: www.myapp.localhost:8000
> User-Agent: curl/7.73.0
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAAQCAAAAAAIAAAAA
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< X-Frame-Options: DENY
X-Frame-Options: DENY
< Vary: Cookie, Accept-Language
Vary: Cookie, Accept-Language
< Content-Length: 433909
Content-Length: 433909
< Content-Language: es
Content-Language: es

Am I testing this wrong, or would additional changes be required for daphne to support this?

[adamchainz]

HTTP/2 technically can work without encryption but all browsers only support HTTP/2 with TLS (see wikipedia). So it may be the case that the HTTP2 support in Daphne is limited to TLS connections too.

[carltongibson]

What's the twisted story here, I want to ask? That would be the starting point.

[WhyNotHugo]

all browsers only support HTTP/2 with TLS

Yup, I'm quite aware of this.

This is not my case though; I have daphne behind a load balancer. Using a LB or another proxy that does the TLS termination is not unusual.

[ZuSe]

I totally agree with that. Same setup here. Our ingress is basically terminating the ssl connection. As far as I can see nginx (based ingress) is able to forward http2 messages without requiring tls on the other end.

[tunecrew]

Interested in this too. We have a load balancer that handles the ssl connection also, and all the communication between the load balancer and our worker instances happens in a private network. Would love to not have to configure tls, certificates, etc. for the workers.

[carltongibson]

Anyone had a look at Twisted for this yet? (If it's supported there, very likely we can do it here...)

[moritz89]

@carltongibson It seems that this can be implemented by simply using a different Twisted endpoint class. Instead of using SSL4ServerEndpoint replace it with TCP4ServerEndpoint. I'm referring to this Twisted example.

[carltongibson]

So we call serverFromString https://twistedmatrix.com/documents/15.1.0/api/twisted.internet.endpoints.serverFromString.html

--endpoint let's us pass that string directly.

So, perhaps the right incantation there would already work? 🤔

[thclark]

all browsers only support HTTP/2 with TLS

Yup, I'm quite aware of this.

This is not my case though; I have daphne behind a load balancer. Using a LB or another proxy that does the TLS termination is not unusual.

Yes, it's a common pattern. My expertise is far too limited to make a really coherent contribution here, but just as a little extra context and possibly a useful check, the Cloud Run docs highlight this, noting that:

"Your Cloud Run service must handle requests in HTTP/2 cleartext (h2c) format, because TLS is still terminated automatically by Cloud Run. To confirm that your service supports h2c requests, test the service locally using this cURL command:

curl -i --http2-prior-knowledge http://localhost:PORT

[justdan6]

So we call serverFromString https://twistedmatrix.com/documents/15.1.0/api/twisted.internet.endpoints.serverFromString.html

--endpoint let's us pass that string directly.

So, perhaps the right incantation there would already work? 🤔

I did some testing on my own and don't believe using --endpoint would support this.

I installed Twisted[tls,http2] and started the server using the following command daphne -v 3 -e tcp:8080:interface=0.0.0.0 app.asgi:application which should automatically support http2 with Twisted[tls,http2] installed and uses the TCP4ServerEndpoint as mentioned above

When I run curl --http2-prior-knowledge -v 127.0.0.1:8080 I get the following response.

* processing: 127.0.0.1:8080
*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080
* h2 [:method: GET]
* h2 [:scheme: http]
* h2 [:authority: 127.0.0.1:8080]
* h2 [:path: /]
* h2 [user-agent: curl/8.2.0-DEV]
* h2 [accept: */*]
* Using Stream ID: 1
> GET / HTTP/2
> Host: 127.0.0.1:8080
> User-Agent: curl/8.2.0-DEV
> Accept: */*
> 
* Closing connection
curl: (56) Failure when receiving data from the peer

And the following is in the daphne logs (Note the HTTP/2 support enabled)

2023-07-18 19:23:33,480 INFO     Starting server at tcp:8080:interface=0.0.0.0
2023-07-18 19:23:33,480 INFO     HTTP/2 support enabled
2023-07-18 19:23:33,480 INFO     Configuring endpoint tcp:8080:interface=0.0.0.0
2023-07-18 19:23:33,481 INFO     HTTPFactory starting on 8080
2023-07-18 19:23:33,481 INFO     Starting factory <daphne.http_protocol.HTTPFactory object at 0xffff9293dc00>
2023-07-18 19:23:33,481 INFO     Listening on TCP address 0.0.0.0:8080
2023-07-18 19:28:07,118 DEBUG    HTTP b'PRI' request for ['172.18.0.1', 55712]
Not Found: *
2023-07-18 19:28:07,143 WARNING  Not Found: *
2023-07-18 19:28:07,143 DEBUG    HTTP 404 response started for ['172.18.0.1', 55712]
2023-07-18 19:28:07,143 DEBUG    HTTP close for ['172.18.0.1', 55712]
2023-07-18 19:28:07,143 INFO     "172.18.0.1" - - [18/Jul/2023:19:28:06 +0000] "PRI * HTTP/2.0" 404 1733 "-" "-"
2023-07-18 19:28:07,143 DEBUG    HTTP response complete for ['172.18.0.1', 55712]

I also can't seem to get twisted to work with http2 without tls using the Web Server example on https://twisted.org/ -

from twisted.web import server, resource
from twisted.internet import reactor, endpoints

class Counter(resource.Resource):
    isLeaf = True
    numberRequests = 0

    def render_GET(self, request):
        self.numberRequests += 1
        request.setHeader(b"content-type", b"text/plain")
        content = u"I am request #{}\n".format(self.numberRequests)
        return content.encode("ascii")

endpoints.serverFromString(reactor, "tcp:8080").listen(server.Site(Counter()))
reactor.run()

I am able to get http2 without tls working if I follow this - https://stackoverflow.com/a/64433012 but that calls the h2 library directly.

As far as using http2 behind a load balancer/proxy NGINX doesn't support this and gives an explanation as to why they think it wouldn't make much sense - https://trac.nginx.org/nginx/ticket/923

[Manouchehri]

This would be a really nice feature for Google Cloud Run; we tried switching to hypercorn, but the startup time increased by 400%. 😅

[carltongibson]

This SO thread seems to imply you can get twisted to serve HTTP2 without TLS (see the "Update" in the answer).

https://stackoverflow.com/questions/64378351/twisted-webserver-with-plain-http2-h2c

Can anyone get that working with Daphne? Please post here if so.