Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: reduce memory usage when hijacking #70756

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

das7pad
Copy link

@das7pad das7pad commented Dec 10, 2024

Previously, Hijack allocated a new write buffer and the existing
connection write buffer used an extra 4kB of memory until the handler
finished and the "conn" was garbage collected. Now, hijack re-uses the
existing write buffer and re-attaches it to the raw connection to avoid
referencing the net/http "conn" after returning.

Hijack will also swap the underlying reader of the read buffer to avoid
holding on to the "connReader", which in turn holds on to the "conn".
This allows all of the "conn", "response" and "Request" to get garbage
collected after the handler has exited.

The new underlying reader of the read buffer uses a weak-ref on the
"response" to allow CloseNotify to work before the handler has exited
(i.e. while the "response" is still referenced in "conn.serve"). This
aligns with the documentation of CloseNotifier:

After the Handler has returned, there is no guarantee that the channel
receives a value.

It is possible to preserve the extended behavior of letting CloseNotify
signal after the handler has exited by keeping around the relevant state
for closeNotify in a struct that is used in both "connReader" and the
new "connCloseNotify". It does not seem worth the extra code complexity
for the deprecated CloseNotifier interface.

To achieve a net-positive effect on allocations per request cycle, the
TODO for lazily allocating the CloseNotifier channel has been addressed.
This change also simplifies the read error handling in the new
"connCloseNotify" wrapper.

goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ before │ writer │ lazy-notify │ reader │
│ sec/op │ sec/op vs base │ sec/op vs base │ sec/op vs base │
Server-8 506.6µ ± 1% 508.4µ ± 2% ~ (p=0.315 n=10) 507.0µ ± 1% ~ (p=0.796 n=10) 499.5µ ± 2% -1.40% (p=0.004 n=10)
ServerHijack-8 50.64µ ± 5% 46.19µ ± 5% -8.79% (p=0.000 n=10) 44.09µ ± 29% -12.94% (p=0.000 n=10) 44.56µ ± 27% -12.01% (p=0.002 n=10)
ServerHijackMemoryUsage-8 1.078m ± 3% 1.112m ± 5% ~ (p=0.436 n=10) 1.181m ± 15% ~ (p=0.190 n=10) 1.104m ± 9% ~ (p=0.190 n=10)
geomean 302.4µ 296.7µ -1.89% 297.7µ -1.54% 290.7µ -3.86%

                      │    before    │                writer                │             lazy-notify              │                reader                │
                      │     B/op     │     B/op      vs base                │     B/op      vs base                │     B/op      vs base                │

Server-8 2.298Ki ± 0% 2.297Ki ± 0% ~ (p=0.781 n=10) 2.185Ki ± 0% -4.91% (p=0.000 n=10) 2.188Ki ± 0% -4.78% (p=0.000 n=10)
ServerHijack-8 16.21Ki ± 0% 12.14Ki ± 0% -25.11% (p=0.000 n=10) 12.03Ki ± 0% -25.78% (p=0.000 n=10) 12.09Ki ± 0% -25.45% (p=0.000 n=10)
ServerHijackMemoryUsage-8 15.68Ki ± 0% 11.61Ki ± 0% -25.91% (p=0.000 n=10) 11.51Ki ± 0% -26.58% (p=0.000 n=10) 11.57Ki ± 0% -26.21% (p=0.000 n=10)
geomean 8.358Ki 6.867Ki -17.84% 6.714Ki -19.68% 6.738Ki -19.39%

                      │   before   │               writer                │            lazy-notify            │              reader               │
                      │ allocs/op  │ allocs/op   vs base                 │ allocs/op   vs base               │ allocs/op   vs base               │

Server-8 21.00 ± 0% 21.00 ± 0% ~ (p=1.000 n=10) ¹ 20.00 ± 0% -4.76% (p=0.000 n=10) 20.00 ± 0% -4.76% (p=0.000 n=10)
ServerHijack-8 52.00 ± 0% 50.00 ± 0% -3.85% (p=0.000 n=10) 49.00 ± 0% -5.77% (p=0.000 n=10) 51.00 ± 0% -1.92% (p=0.000 n=10)
ServerHijackMemoryUsage-8 48.00 ± 0% 46.00 ± 0% -4.17% (p=0.000 n=10) 45.00 ± 0% -6.25% (p=0.000 n=10) 47.00 ± 0% -2.08% (p=0.000 n=10)
geomean 37.42 36.42 -2.69% 35.33 -5.60% 36.33 -2.93%
¹ all samples are equal

                      │    before     │                writer                 │              lazy-notify              │                reader                 │
                      │ retained_B/op │ retained_B/op  vs base                │ retained_B/op  vs base                │ retained_B/op  vs base                │

ServerHijackMemoryUsage-8 15.625k ± 0% 11.464k ± 0% -26.63% (p=0.000 n=10) 11.339k ± 0% -27.43% (p=0.000 n=10) 8.714k ± 0% -44.23% (p=0.000 n=10)

Fixes #59567

Previously, Hijack allocated a new write buffer and the existing
connection write buffer used an extra 4kB of memory until the handler
finished and the "conn" was garbage collected. Now, hijack re-uses the
existing write buffer and re-attaches it to the raw connection to avoid
referencing the net/http "conn" after returning.

Why the extra reset of the write buffer? The previous io.Writer of the
write buffer ("checkConnErrorWriter") was recording write errors back on
the "conn" and also cancelling the request context on write errors.
Neither of these properties are desired nor existing in the previous
implementation of Hijack().

Note: The read buffer still holds on to "conn" via "connReader". It is
handling cancelling of the Request context and broadcast on the
CloseNotifier channel. A follow-up patch will simplify this scenario and
allow the GC to reclaim the "conn" and "connReader" as well.

goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
                          │   before    │               after                │
                          │   sec/op    │   sec/op     vs base               │
Server-8                    506.6µ ± 1%   508.4µ ± 2%       ~ (p=0.315 n=10)
ServerHijack-8              50.64µ ± 5%   46.19µ ± 5%  -8.79% (p=0.000 n=10)
ServerHijackMemoryUsage-8   1.078m ± 3%   1.112m ± 5%       ~ (p=0.436 n=10)
geomean                     302.4µ        296.7µ       -1.89%

                          │    before    │                after                 │
                          │     B/op     │     B/op      vs base                │
Server-8                    2.298Ki ± 0%   2.297Ki ± 0%        ~ (p=0.781 n=10)
ServerHijack-8              16.21Ki ± 0%   12.14Ki ± 0%  -25.11% (p=0.000 n=10)
ServerHijackMemoryUsage-8   15.68Ki ± 0%   11.61Ki ± 0%  -25.91% (p=0.000 n=10)
geomean                     8.358Ki        6.867Ki       -17.84%

                          │   before   │                after                │
                          │ allocs/op  │ allocs/op   vs base                 │
Server-8                    21.00 ± 0%   21.00 ± 0%       ~ (p=1.000 n=10) ¹
ServerHijack-8              52.00 ± 0%   50.00 ± 0%  -3.85% (p=0.000 n=10)
ServerHijackMemoryUsage-8   48.00 ± 0%   46.00 ± 0%  -4.17% (p=0.000 n=10)
geomean                     37.42        36.42       -2.69%
¹ all samples are equal

                          │    before     │                 after                 │
                          │ retained_B/op │ retained_B/op  vs base                │
ServerHijackMemoryUsage-8     15.62k ± 0%     11.46k ± 0%  -26.63% (p=0.000 n=10)
goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
                          │   writer    │             lazy-notify             │
                          │   sec/op    │    sec/op     vs base               │
Server-8                    508.4µ ± 2%   507.0µ ±  1%       ~ (p=0.481 n=10)
ServerHijack-8              46.19µ ± 5%   44.09µ ± 29%  -4.55% (p=0.035 n=10)
ServerHijackMemoryUsage-8   1.112m ± 5%   1.181m ± 15%       ~ (p=0.190 n=10)
geomean                     296.7µ        297.7µ        +0.36%

                          │    writer    │             lazy-notify             │
                          │     B/op     │     B/op      vs base               │
Server-8                    2.297Ki ± 0%   2.185Ki ± 0%  -4.87% (p=0.000 n=10)
ServerHijack-8              12.14Ki ± 0%   12.03Ki ± 0%  -0.89% (p=0.000 n=10)
ServerHijackMemoryUsage-8   11.61Ki ± 0%   11.51Ki ± 0%  -0.90% (p=0.000 n=10)
geomean                     6.867Ki        6.714Ki       -2.24%

                          │   writer   │            lazy-notify            │
                          │ allocs/op  │ allocs/op   vs base               │
Server-8                    21.00 ± 0%   20.00 ± 0%  -4.76% (p=0.000 n=10)
ServerHijack-8              50.00 ± 0%   49.00 ± 0%  -2.00% (p=0.000 n=10)
ServerHijackMemoryUsage-8   46.00 ± 0%   45.00 ± 0%  -2.17% (p=0.000 n=10)
geomean                     36.42        35.33       -2.99%

                          │    writer     │             lazy-notify              │
                          │ retained_B/op │ retained_B/op  vs base               │
ServerHijackMemoryUsage-8     11.46k ± 0%     11.34k ± 0%  -1.09% (p=0.000 n=10)
Avoid holding on to the "connReader" after hijacking. This allows all of
the "conn", "response" and "Request" to be garbage collected after the
ServeHTTP handler exits.

Use a weak-ref on the "response" to allow CloseNotify to work _before_
the handler has exited (i.e. while the "response" is still referenced in
"conn.serve"). This aligns with the documentation of CloseNotifier:
> After the Handler has returned, there is no guarantee that the channel
> receives a value.

goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
                          │ lazy-notify  │               reader                │
                          │    sec/op    │    sec/op     vs base               │
Server-8                    507.0µ ±  1%   499.5µ ±  2%  -1.48% (p=0.002 n=10)
ServerHijack-8              44.09µ ± 29%   44.56µ ± 27%       ~ (p=0.529 n=10)
ServerHijackMemoryUsage-8   1.181m ± 15%   1.104m ±  9%       ~ (p=0.143 n=10)
geomean                     297.7µ         290.7µ        -2.36%

                          │ lazy-notify  │               reader                │
                          │     B/op     │     B/op      vs base               │
Server-8                    2.185Ki ± 0%   2.188Ki ± 0%  +0.13% (p=0.014 n=10)
ServerHijack-8              12.03Ki ± 0%   12.09Ki ± 0%  +0.45% (p=0.000 n=10)
ServerHijackMemoryUsage-8   11.51Ki ± 0%   11.57Ki ± 0%  +0.50% (p=0.000 n=10)
geomean                     6.714Ki        6.738Ki       +0.36%

                          │ lazy-notify │               reader                │
                          │  allocs/op  │ allocs/op   vs base                 │
Server-8                     20.00 ± 0%   20.00 ± 0%       ~ (p=1.000 n=10) ¹
ServerHijack-8               49.00 ± 0%   51.00 ± 0%  +4.08% (p=0.000 n=10)
ServerHijackMemoryUsage-8    45.00 ± 0%   47.00 ± 0%  +4.44% (p=0.000 n=10)
geomean                      35.33        36.33       +2.82%
¹ all samples are equal

                          │  lazy-notify  │                reader                 │
                          │ retained_B/op │ retained_B/op  vs base                │
ServerHijackMemoryUsage-8    11.339k ± 0%     8.714k ± 0%  -23.15% (p=0.000 n=10)
@das7pad
Copy link
Author

das7pad commented Dec 10, 2024

It is possible to preserve the extended behavior of letting CloseNotify
signal after the handler has exited by keeping around the relevant state
for closeNotify in a struct that is used in both "connReader" and the
new "connCloseNotify". It does not seem worth the extra code complexity
for the deprecated CloseNotifier interface.

2bc45c1 is a WIP implementation for this -- in case this behavior is deemed useful to keep/worth the complexity.

@gopherbot
Copy link
Contributor

This PR (HEAD: 9f49d26) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/634855.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://go.dev/doc/contribute#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.

During May-July and Nov-Jan the Go project is in a code freeze, during which
little code gets reviewed or merged. If a reviewer responds with a comment like
R=go1.11 or adds a tag like "wait-release", it means that this CL will be
reviewed as part of the next development cycle. See https://go.dev/s/release
for more details.


Please don’t reply on this GitHub thread. Visit golang.org/cl/634855.
After addressing review feedback, remember to publish your drafts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

net/http: bufio.Writer aren't garbage collected after Hijack
2 participants