Skip to content

nghttp2: what a request's header blocks decode to is bounded by a new MaxHeaderListSize, so one 16 KB frame can no longer make 62 MB - #511

Open
MDA2AV wants to merge 1 commit into
mainfrom
fix/508-nghttp2-header-list
Open

MDA2AV wants to merge 1 commit into
mainfrom
fix/508-nghttp2-header-list

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

What was wrong

  • nghttp2 calls back once per decoded field, and CallbackHeader copied each into the request's arena. MaxRequestBytes was only checked against the body.
  • nghttp2 bounds the blocks it reads, not what they decode to. So once a field of about 4 KiB is in the HPACK dynamic table, each one-byte reference to it decodes to the whole field.
  • One HEADERS frame of 16,014 B gave a handler 16,000 fields and 62 MB. A request that never ends holds that in _pending until the connection closes.

Details are in #508.

The fix

  • CallbackHeader counts what a request decodes to, as RFC 9113 6.5.2 measures it: each field's name and value plus 32 bytes, across all of the request's blocks.
  • Past the new Nghttp2Options.MaxHeaderListSize (64 KiB) the request is marked as an oversized body is: nothing more is copied, and its stream is reset with ENHANCE_YOUR_CALM once ih2_read unwinds. No handler sees it.

This is not the MaxRequestBytes check #508 suggested, for the reason in #507: at 8 MiB a stream, 1000 streams would pin 8 GB.

No native change, so the libraries stay as they are.

Behaviour

  • A request whose headers decode past 64 KiB is now reset with ENHANCE_YOUR_CALM. Raise MaxHeaderListSize if a client really sends that much.
  • MaxRequestBytes's doc no longer says "headers plus body", or that nghttp2 limits the header list.

What is left

As in #507: a connection's headers hold at most 1000 times 64 KiB at the defaults, and compression makes that cheap to reach. Lowering MaxHeaderListSize tightens it; the shim fixes the concurrent streams at 1000.

API

New: Nghttp2Options.MaxHeaderListSize, default 64 KiB, as on Http2Options.

Tests

New in the Unit suite (Nghttp2HeaderListTests.cs), over the strict test transport:

  • nghttp2 header list: a block that decodes past the limit resets its stream, keeps nothing and leaves HPACK in step
    • 3 open streams of one 16 KB frame each. All 3 are reset, with no GOAWAY and no handler run. Under 2 MiB is retained, and under 4 MiB is allocated while decoding. A later request referring to the same table entry still decodes it. The request joins the ready list once, not once per field.
  • nghttp2 header list: a request measuring 309 B against a limit of 309 is served, and against 308 is reset
    • The exact measure at its boundary, including that one request's count does not carry over to the next.
  • nghttp2 header list: a handler taking a body reader never sees a request past the limit
  • nghttp2 header list: a streamed response's handler never sees a request past the limit
  • nghttp2 header list: trailers count with the block that opened the request

With the limit check removed, as on main:

FAIL  nghttp2 header list: a block that decodes past the limit resets its stream, keeps nothing and leaves HPACK in step: expected [3 0xb, 5 0xb, 7 0xb], got []
PASS  nghttp2 header list: a request measuring 309 B against a limit of 309 is served
FAIL  nghttp2 header list: a request measuring 309 B against a limit of 308 is reset: expected [0], got [2]
FAIL  nghttp2 header list: a handler taking a body reader never sees a request past the limit: expected [1 0xb], got []
FAIL  nghttp2 header list: a streamed response's handler never sees a request past the limit: expected [1 0xb], got []
FAIL  nghttp2 header list: trailers count with the block that opened the request: expected [1 0xb], got []
1 passed, 5 failed, 0 skipped

Main itself cannot run these, since the option is new. They also failed with no 32 bytes per field, and with a request rejoining the ready list for every field past the limit (16,384 entries).

With the fix:

  • Unit: 151 passed.
  • E2E: 453 passed.
  • Chaos: 83 passed, including the nghttp2 twins.
  • Http: 95 passed.

Merging

Bench

Measured with the same setup as #486:

  • One reactor pinned to CPU 6, and h2load -c 32 -m 16 -t 1 pinned to CPU 2.
  • Main, this branch, and main again from a second worktree as a control.
  • Six interleaved rounds of 10 s; each build is compared with main inside its own round.
Sample Server CPU per request, branch vs main Control vs main
Http2/Nghttp2Buffered -1.1% -0.6%

Inside the noise, and the per-round ranges overlap: 908-971 ns for the branch, 916-968 for main.

Fixes #508.

… MaxHeaderListSize, so one 16 KB frame can no longer make 62 MB

nghttp2 bounds the blocks it reads, not what they decode to, and one byte of HPACK can stand for a whole table entry.
Past the limit a request is marked as an oversized body is, so nothing more is copied and its stream is reset with ENHANCE_YOUR_CALM once ih2_read unwinds.

Fixes #508.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nghttp2: nothing bounds what a request's headers decode to, so one 16 KB frame makes 62 MB (HPACK bomb)

1 participant