Skip to content

nghttp2: a response given after the peer reset its stream attaches nothing, so its body no longer leaks and a streamed writer hears the peer gone - #501

Open
MDA2AV wants to merge 2 commits into
mainfrom
fix/nghttp2-submit-closed-stream
Open

MDA2AV wants to merge 2 commits into
mainfrom
fix/nghttp2-submit-closed-stream

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

What was wrong

  • A peer resets a stream after its request has ended and before the handler answers, as a browser does when it cancels a slow request. nghttp2 closes and frees the stream, and the managed side has nothing to mark: the request was already dispatched, and a streamed writer is created only after the read that carried the reset.
  • ih2_submit_response then copied the response's body into an ih2_stream and called nghttp2_submit_response, which accepts any positive stream id. The drain dropped the HEADERS as STREAM_CLOSED with no callback, so on_stream_close never freed that ih2_stream: the body stayed until the connection closed.
  • A streamed writer got the same treatment from ih2_submit_response_stream, and every ih2_stream_write then succeeded into it. Past 64 KiB of unframed bytes its flush parked, and nothing woke it until the connection was disposed.

Details are in #496.

The fix

Both submit functions attach nothing when nghttp2_session_find_stream finds no stream.

  • Buffered: ih2_submit_response returns 0, with nothing queued and nothing kept. An error here would fail the connection, for one stream's end.
  • Streamed: ih2_submit_response_stream returns NGHTTP2_ERR_STREAM_CLOSED. SendStreamedHeaders reads that as "this stream is gone" and the writer marks itself gone, so its flush returns IsCompleted, with or without bytes staged.

Native libraries

runtimes/linux-x64 and runtimes/linux-arm64 are rebuilt with scripts/build-nghttp2-native.sh at its pinned nghttp2 ref: x64 on the host, arm64 with the aarch64-linux-gnu cross toolchain in an Ubuntu 24.04 container. Both still need nothing past GLIBC 2.17.

#495 rebuilds the same libraries. Whichever of the two merges second has to merge main and rerun the build script, rather than take either side's binaries.

Behaviour

  • A response to a stream its peer already reset is dropped at submit. Before, its body was copied and kept until the connection closed.
  • A streamed handler whose stream was reset before it was dispatched sees IsCompleted at its first flush. Before, a flush with nothing staged reported success, and a body past 64 KiB made its flush wait until the connection closed.

API

None.

Tests

New in the Unit suite (Nghttp2ResetResponseTests.cs), over the strict test transport:

  • nghttp2: a buffered response to a stream its peer reset while the handler ran is not kept
    • 220 times: a GET, its RST_STREAM(CANCEL) while the handler is parked, then the handler's 64 KiB answer. After 20 warm-up rounds, malloc's in-use bytes (mallinfo2) may grow under 6 MiB across the other 200. They are the whole process's, so the bound sits well clear of the JIT's; the leak is 12.8 MiB.
  • nghttp2: a streamed response to a stream reset in the same read as its end hears the peer gone at a flush of 131072 B, and of 0 B
    • A GET and its RST_STREAM in one read, so the writer is created after the reset. Its flush has to return IsCompleted, whether it staged 128 KiB, past the 64 KiB a flush may run ahead of nghttp2, or nothing.
    • The connection must not be broken by it, and has to serve a next GET.

Against main's library (the tests only):

FAIL  nghttp2: a buffered response to a stream its peer reset while the handler ran is not kept: 200 responses of 64 KiB to reset streams left 12815 KiB on the native heap
FAIL  nghttp2: a streamed response to a stream reset in the same read as its end hears the peer gone at a flush of 131072 B: the flush of a stream nghttp2 had already closed never returned
FAIL  nghttp2: a streamed response to a stream reset in the same read as its end hears the peer gone at a flush of 0 B: expected [peer gone], got [flushed]
0 passed, 3 failed, 0 skipped

Each streamed test was also run against a mutation of the code it guards and failed there: the writer ignoring the result of its headers (the 0 B flush), and a closed stream failing the connection (both).

With the fix: Unit: 148 passed, E2E: 453, Chaos: 83, Http: 95. The shim compiles with no warnings under -Wall -Wextra -Werror=implicit-function-declaration; the lookup is nghttp2's public nghttp2_session_find_stream, which finds nothing once nghttp2_session_close_stream has destroyed the stream.

Bench

Each response gains one stream lookup in C (nghttp2_session_find_stream). Measured with the same setup as #486:

  • One reactor pinned to CPU 6, and h2load -c 32 -m 16 -t 1 pinned to CPU 2.
  • Main, this branch, and main again from a second worktree as a control.
  • Six interleaved rounds; each build is compared with main inside its own round.
Sample Server CPU per request, branch vs main Control vs main
Http2/Nghttp2Buffered +1.0% -0.5%
Http2/Nghttp2Response (8 chunks of 1 KiB) +1.2% +0.2%

Both are inside the noise, and the per-round ranges overlap: buffered 918-948 ns per request for the branch, 897-954 for main and 881-947 for the control. If the lookup costs anything, it is about 1%.

Measured before the streamed function's result changed for a stream already gone. That changes only that case, not the path of a live stream, so it was not measured again.

Fixes #496.

…thing, so its body no longer leaks and a streamed writer hears the peer gone

A peer that resets a stream after its request ended and before the handler answers (a browser
cancelling a slow request) leaves nothing on the managed side to mark. ih2_submit_response then
copied the response's body into an ih2_stream for a stream nghttp2 had already freed; the HEADERS
were dropped at send with no callback, so the copy stayed until the connection closed. A streamed
writer created after such a reset wrote into its own orphaned ih2_stream, never heard the reset, and
past 64 KiB its flush waited until the connection was disposed (#496).

Both submit functions now return 0 without attaching anything when nghttp2_session_find_stream finds
no stream. The streamed writer's first write then finds none, which SendStreamedData already reads
as that stream's end. Both runtimes' libraries are rebuilt from this source.
…ders, so a flush with nothing staged reports the peer gone too

ih2_submit_response_stream returns NGHTTP2_ERR_STREAM_CLOSED for a gone stream, which only that case can return: a response past the shim's 64 headers returns INVALID_ARGUMENT and still fails the connection.
The test proves the connection serves on, and the leak's bound is well clear of what the rest of the process mallocs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nghttp2: a response submitted after the peer reset its stream leaks its body in the shim, and a streamed writer can hang

1 participant