Repository navigation
nghttp2: what a request's header blocks decode to is bounded by a new MaxHeaderListSize, so one 16 KB frame can no longer make 62 MB - #511
Open
MDA2AV wants to merge 1 commit into
Conversation
… MaxHeaderListSize, so one 16 KB frame can no longer make 62 MB nghttp2 bounds the blocks it reads, not what they decode to, and one byte of HPACK can stand for a whole table entry. Past the limit a request is marked as an oversized body is, so nothing more is copied and its stream is reset with ENHANCE_YOUR_CALM once ih2_read unwinds. Fixes #508.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
CallbackHeadercopied each into the request's arena.MaxRequestByteswas only checked against the body._pendinguntil the connection closes.Details are in #508.
The fix
CallbackHeadercounts what a request decodes to, as RFC 9113 6.5.2 measures it: each field's name and value plus 32 bytes, across all of the request's blocks.Nghttp2Options.MaxHeaderListSize(64 KiB) the request is marked as an oversized body is: nothing more is copied, and its stream is reset with ENHANCE_YOUR_CALM onceih2_readunwinds. No handler sees it.This is not the
MaxRequestBytescheck #508 suggested, for the reason in #507: at 8 MiB a stream, 1000 streams would pin 8 GB.No native change, so the libraries stay as they are.
Behaviour
MaxHeaderListSizeif a client really sends that much.MaxRequestBytes's doc no longer says "headers plus body", or that nghttp2 limits the header list.What is left
As in #507: a connection's headers hold at most 1000 times 64 KiB at the defaults, and compression makes that cheap to reach. Lowering
MaxHeaderListSizetightens it; the shim fixes the concurrent streams at 1000.API
New:
Nghttp2Options.MaxHeaderListSize, default 64 KiB, as onHttp2Options.Tests
New in the Unit suite (
Nghttp2HeaderListTests.cs), over the strict test transport:nghttp2 header list: a block that decodes past the limit resets its stream, keeps nothing and leaves HPACK in stepnghttp2 header list: a request measuring 309 B against a limit of 309 is served, and against 308 is resetnghttp2 header list: a handler taking a body reader never sees a request past the limitnghttp2 header list: a streamed response's handler never sees a request past the limitnghttp2 header list: trailers count with the block that opened the requestWith the limit check removed, as on main:
Main itself cannot run these, since the option is new. They also failed with no 32 bytes per field, and with a request rejoining the ready list for every field past the limit (16,384 entries).
With the fix:
Merging
CallbackData, where this moves the ready-list line intoOverflow. KeepOverflow, and have it callEnqueue, since nghttp2: a connection reuses a finished request's objects for its next stream, so a warm request allocates none of the parser's #491 makes the list aQueue. The helper is there so that this shows as a conflict: a newAddwould have merged silently and not compiled.PendingRequest, soDisposeresets the count here already.Bench
Measured with the same setup as #486:
h2load -c 32 -m 16 -t 1pinned to CPU 2.Http2/Nghttp2BufferedInside the noise, and the per-round ranges overlap: 908-971 ns for the branch, 916-968 for main.
Fixes #508.