[MDA2AV]

What was wrong

Every request stream got a new set of objects: about 1.66 KB per request.

• ReqStream, with a 16-byte carry buffer
• Http3Request, with its two lists and their arrays
• a 1 KiB arena, made by Array.Resize from empty on the first field

The fix

The same approach #486 takes for ioxide.http2:

• A request whose body arrived whole left _requests at dispatch. Once its handler is done and its response submitted, Recycle resets the ReqStream and keeps it on the connection (_reqPool), with its Http3Request, arena, lists and carry. Feed takes one from there for the next request stream.
  • It covers all three dispatch paths: synchronous, asynchronous (CompleteBufferedAsync) and streamed response (ServeAsync).
  • The pool is bounded:
    • it keeps at most 32;
    • never one whose header lists grew past 64 entries (a peer's thousands of header lines);
    • and only on the reactor, the one thread that rents.
• A request whose body is streamed stays in _requests until its stream ends, so it is left alone, as before.
• An arena past 4 KiB is not kept, so a peer sending large headers leaves no large array on the connection. That also covers a response body pointing into its request's arena while it waits for send capacity: only bodies past 4 KiB wait in _pendingBodies.

Behaviour

• Http3Request is reused for a later stream on the same connection once its handler is done, when its body arrived whole.
  • It was already documented as valid only until the handler returns.
  • The doc comment says so now.
• A response kept past its handler has to own its memories. One whose body or headers point into its request, sent again later, now carries a later request's bytes. On main each request had its own arena, so this happened to work, against the request's documented lifetime. Http3Response's doc says so now. The two HTTP/2 stacks pool their arenas on main already, and have no check for it.
• Unchanged for a streamed body's request.
• A request with a body still allocates its MemoryStream, so "none of its objects" holds for the bodyless case.

API

None public.

Tests

New in the E2E suite (Http3RequestReuseTests.cs), through the bare HTTP/3 test client:

• h3 request (ioxide.http3, synchronous | completed | parked | streamed handler): the next stream's request holds nothing of the one before
  • One test per dispatch path. The completed one returns an already-completed ValueTask from a source that throws on a second read, the path where the first version read it twice.
  • Each sends a POST with two headers, a content-length and a body, then a GET that carries host and no :authority (RFC 9114 4.3.1 allows either), then a second POST with a content-length, on one connection.
  • Each request gets the same object, and it carries none of the one before: a stale :authority would show on the GET, and a stale ContentLengthLeft would make the second content-length read as a repeated one.
  • The test client sends no :authority on a request that carries host; nothing else used host with it.
• h3 request (ioxide.http3): headers past 4 KiB leave no large arena for the next stream
• h3 request (ioxide.http3): a request with 64 header lines is kept for the next stream, and with 65 is not
• h3 request (ioxide.http3): a connection keeps at most 32 requests for later streams
  • These three run the connection over the recording transport from H3ErrorCodeTests, with hand-framed requests: they take no time, and the test client sends at most 64 fields.
• h3 request (ioxide.http3): a warm connection's request allocates none of the request objects
  • It makes 16 requests one after another. The test client spends about 1.5 s on each; the first version made 48, which took 62-83 s in CI.
  • The handler records what the reactor's thread allocated between request 4 and request 16, and checks that both marks come from that thread.
  • It allows under 400 B per request. With the fix it measures 184 B, which is the QUIC send queue's per-stream state. PR ngtcp2: a connection keeps a finished stream's send state for the next stream, so a warm HTTP/3 response allocates none of it #490 removes that too; once it lands, this measures 0, and the bound can drop to 64 B.

Against main's ioxide.http3 (the tests only):

FAIL  h3 request (ioxide.http3, synchronous handler): the next stream's request holds nothing of the one before: each stream reuses the request of the one before
FAIL  h3 request (ioxide.http3, completed handler): the next stream's request holds nothing of the one before: each stream reuses the request of the one before
FAIL  h3 request (ioxide.http3, parked handler): the next stream's request holds nothing of the one before: each stream reuses the request of the one before
FAIL  h3 request (ioxide.http3, streamed handler): the next stream's request holds nothing of the one before: each stream reuses the request of the one before
FAIL  h3 request (ioxide.http3): a request with 64 header lines is kept for the next stream: expected [True], got [False]
FAIL  h3 request (ioxide.http3): a connection keeps at most 32 requests for later streams: Object reference not set to an instance of an object.
FAIL  h3 request (ioxide.http3): a warm connection's request allocates none of the request objects: a request on a warm connection allocated 1840 B on the reactor
2 passed, 7 failed, 0 skipped

The two that pass check what is not kept, and main keeps nothing.

Each test added after the first review was also run against a mutation of the code it guards and failed there: the ValueTask read twice, the pseudo-header ranges not reset, the recycle on the completed path removed, a large arena kept, the 64-line limit removed or moved, and the cap removed.

With the fix:

• Unit: 145 passed.
• E2E: 462 passed.
• Chaos: 83 passed.
• Http: 95 passed.

Review

An 8-agent review of this PR and its siblings found the reuse sound. It found the following, fixed in e2c7129:

• A bug: pending.Result was read twice. Submit(sid, pending.Result); Recycle(rs, pending.Result) consumed a ValueTask twice. A handler whose ValueTask comes from a pooled source threw on the second read and took the connection down. It is read once now.
• Pool bounds: the pool was unbounded and could keep header lists a peer grew to megabytes.
• The arena check's comment named the wrong case.
• Tests: they covered only the synchronous path, could not catch a stale ContentLengthLeft, and took 62-83 s in CI.

A later round removed a check that kept an arena out of the pool when the response body pointed into it (c3e440c). It covered the body and not the header values, for a handler outside the documented lifetime, and the HTTP/2 stacks have none.

Bench

Re-measured after the review changes: allocation per request is unchanged. ManagedBuffered allocates 186 B (base 1,882 B, control 1,885 B), and ManagedStreamedBoth is unchanged, as before. The tables below are the first run.

Setup: the same as #486.

• One reactor pinned to CPU 6, and h2load -c 32 -m 16 -t 1 (with --alpn-list=h3 for HTTP/3) pinned to CPU 2, on separate physical cores.
• Three builds: main (base), this branch, and main again from a second worktree as a control.
• Six rounds of 10 s, each after a 3 s warm-up, with the order rotating every round.
• Each build is compared with base inside its own round; the tables show the median across rounds.
• Streamed samples send 8 chunks of 1 KiB per response.

Sample	Metric	base	branch	control vs base	branch vs base
Http3/ManagedBuffered	server CPU per request	1.44 µs	1.39 µs	+0.3%	-5.0%
	req/s	652K	652K	+0.7%	+1.0%
Http3/ManagedStreamedBoth	server CPU per request	6.69 µs	6.58 µs	-1.0%	-0.9%

• Buffered:
  • The server ran at 90-94% CPU, so its req/s is bounded by h2load. Server CPU per request is the number that holds.
  • The per-round ranges: 1,414-1,501 ns on base, 1,358-1,414 ns on the branch.
• ManagedStreamedBoth streams its request body, which this PR leaves alone, so it is unchanged, as expected.

Allocations, from dotnet-counters over 20 s under the same load:

	base	branch	control
ManagedBuffered, B per request	1,887	185	1,883
ManagedStreamedBoth, B per request	2,814	2,812	2,776
gen0 GCs/s, buffered	63	6	64

• What is left on ManagedBuffered is the QUIC send queue's per-stream state, 185 B, which PR ngtcp2: a connection keeps a finished stream's send state for the next stream, so a warm HTTP/3 response allocates none of it #490 removes.
• Not done here: pooling a streamed body's request. It needs a request stream returned by whichever comes last, its handler ending or its stream closing.

Fixes #488.