[MDA2AV]

What was wrong

A streamed flush allocated about 385 B whenever it waited for the write pump (#484)

• With several streams on a connection, one writer's flush is the pump: it awaits the transport, then wakes the others inline.
• The woken writers resume outside a dispatch pass, so their next flush is a real one. The pump is still flushing, so their bytes are queued and they wait for its next turn.
• That wait went through two async methods (Http2ResponseWriter.FlushCore and Http2Connection.FlushAsync), so both state machines were boxed, plus a new TaskCompletionSource per wait and a TaskCompletionSource[] per turn.

A buffered request allocated about 700 B of parser objects (#485)

• Every stream got a new PendingRequest, Http2Request, request KeyValueList and header array, and a field-range List with its array.
• Decoding a header block made a closure, and every dispatch pass copied the ready list into an array.

Found on the way: a faulted flush stranded the writers whose bytes it carried

• The pump takes the turn's waiters before each transport flush. When that flush threw, the finally woke only the waiters that parked during it, not the ones taken for it.
• Those writers stayed parked forever, and so did their handlers.

The fix

Turn waits

• TurnWaiter (new, internal): an IValueTaskSource<FlushResult> that a streamed writer keeps for its whole life.
• Http2ResponseWriter.FlushCore is no longer an async method on the reactor. It sends what credit allows and returns the flush as it is: done, the pump, or a wait on the writer's own TurnWaiter.
  • Off the reactor, before the headers, or out of credit, it still goes the async way, as before.
• Http2Connection.FlushAsync is no longer async either; the pump moved into PumpAsync.
• A turn is a chain linked through its waiters, so parking allocates nothing and taking a turn copies nothing. Other callers (the read loop, a parked buffered handler's tail) park a new TurnWaiter, one object instead of a TaskCompletionSource and its Task.
• EndTurn also wakes the turn whose own flush faulted. Writers woken by a faulted turn, or by a disposed connection, get IsCompleted, as a TCP pipe writer reports a closed peer.
• A faulted flush also releases writers waiting for credit. The fault reaches the writer that pumps, not the read loop, so no teardown is on its way to them.
• Out of credit, SendAfterCreditAsync loops in one frame until the bytes are out, however many times the peer opens the window.

Request objects

• A retired PendingRequest goes back to its connection (_pendingPool, like _writerPool) with its Http2Request, header list and field ranges, and the next stream reuses them.
• The pool is bounded:
  • it keeps at most 32;
  • never one whose field list grew past 64 (a peer's thousands of header lines);
  • and only on the reactor, the one thread that rents.
• A retired request lets go of the memories it held into the arena that went back to ArrayPool, so the pool pins nothing.
• A stream reset before its request completed goes back to the pool too.
• The HPACK sink takes the request as state, so it is a static lambda.
• The ready list is a Queue that a dispatch drains, instead of a List copied to an array each pass. It only ever holds requests not yet dispatched, so Dispose can free what it finds there.

Behaviour

• Http2Request is reused for a later stream on the same connection once its handler is done. It was already only valid during the handler: its memories point into an arena that goes back to the pool at retire. The doc comment says so now.
• Exceptions still come through the returned task, as before. The non-async path catches what staging throws, or a stopped reactor's ObjectDisposedException, and returns it as a faulted ValueTask.
• Calling FlushAsync again before the last flush completed is now an error in practice. The writer's contract already rules it out, and FlushAsync's doc now says each call has to complete before the next.
  • The earlier ValueTask is no longer valid.
  • The later one may complete with the turn the earlier one joined, one turn early.
  • Main allocated a new waiter each time, so it tolerated this.

API

None public. Internal signatures changed: FlushAsync, MaybeFlushAsync, FlushOutboundAsync and ResetStreamedAsync return ValueTask<FlushResult>, and HpackDecoder.Decode takes a state argument.

Tests

New, in the Unit suite:

• h2 queue: a streamed flush that waits for the pump's turn allocates nothing (Http2OutputQueueTests.cs)
  • It measures writer.FlushAsync() on the handler's thread while the flush waits for its turn, and checks that the flush did wait.
  • It skips the first wait, which grows the connection's queue.
• h2 queue: a faulted flush wakes the writer whose bytes it carried (Http2OutputQueueTests.cs)
  • The writer's flush has to return IsCompleted, not merely wake.
• h2 queue: a flush waiting for credit hears the peer gone when the turn it waits on faults (Http2OutputQueueTests.cs)
  • Stream 3 pumps, and stream 1 queues its one byte of credit and waits for more on the pump's next turn, which faults. The fault reaches stream 3, not the read loop, so no teardown releases stream 1's wait: its flush has to return IsCompleted from the turn itself.
• h2 queue: a flush parked on credit hears the peer gone when another writer's flush faults (Http2OutputQueueTests.cs)
  • Stream 1's own flush is done and it waits for credit when stream 3's flush faults. Main leaves it waiting until the read loop ends too.
• h2 queue: a flush that waits for credit again and again keeps one frame, and resumes inline (Http2OutputQueueTests.cs)
  • The stream window is 0 while the peer opens the connection a byte at a time, 20,000 times inside one flush.
  • It fails on the first version of this PR (4.1 MB retained, resumed on a pool thread) and passes on main.
• h2 request: the next stream's request holds nothing of the one before (Http2RequestReuseTests.cs)
  • A POST over http with an authority, two headers and a body, then a bare GET over https, then a POST with a body, on the same connection.
  • The GET and the second POST split their header block over HEADERS and CONTINUATION, so a HeadersDone or RequestEnded left set by the request before would dispatch them early.
  • Each request gets the same object, and it carries none of the one before.
• h2 request: a parked handler's request goes back once it answers (Http2RequestReuseTests.cs)
  • The CompleteBufferedAsync path. Once back, the request's memories are empty, so the pool pins no arena.
• h2 request: a request reset before it ends comes back for the next stream holding nothing (Http2RequestReuseTests.cs)
  • Headers and part of a body, then RST_STREAM: Discard takes the request back (the pool, read by reflection, holds it), and the next GET carries nothing of it.
• h2 request: a handler that disposes the connection leaves a parked handler's request alone (Http2RequestReuseTests.cs)
  • Streams 1 and 3 in one read: stream 1 parks holding its request, and stream 3 disposes the connection. Stream 1's path stays readable.
• h2 request: a burst leaves at most 32 requests on the connection (Http2RequestReuseTests.cs)
• h2 request: a request with 64 header lines is kept for the next stream, and with 65 is not (Http2RequestReuseTests.cs)
• h2 request: a warm connection's request allocates none of the parser's objects (Http2RequestReuseTests.cs)
  • It sends 16 requests with one header each, after two warm-up passes.
  • It counts that all 16 handlers ran inside the measured window.
  • It allows under 100 B per request. With the fix it measures 65 B: the pass's one flush, shared by the 16 requests, which is the test transport's own allocations plus 136 B of pump state machine. Main's HPACK closure alone brings it to 145 B.

Against main's ioxide.http2 (the tests only):

FAIL  h2 queue: a faulted flush wakes the writer whose bytes it carried: the writer whose chunk the faulted flush carried never woke
FAIL  h2 queue: a flush waiting for credit hears the peer gone when the turn it waits on faults: stream 1's flush still waited for credit after its turn faulted
FAIL  h2 queue: a flush parked on credit hears the peer gone when another writer's flush faults: stream 1's flush still waited for credit after the write side died
FAIL  h2 queue: a streamed flush that waits for the pump's turn allocates nothing: flush 1 allocated 376 B while it waited for its turn
FAIL  h2 request: the next stream's request holds nothing of the one before: each stream reuses the request of the one before
FAIL  h2 request: a parked handler's request goes back once it answers: a retired request still points into the arena that went back
FAIL  h2 request: a warm connection's request allocates none of the parser's objects: a request on a warm connection allocated 818 B

Those are the two test classes only: 6 passed, 11 failed. The other 4 failures read _pendingPool, which main does not have.

Each test added or tightened in the review rounds was also run against a mutation of the code it guards and failed there: the faulted turn woken with default, the credit wait ignoring its turn, credit waits not released on a fault, HeadersDone or RequestEnded not reset, Discard disposing instead of recycling, the HPACK sink capturing a closure, the parked path disposing instead of retiring, a dispatch that leaves dispatched requests in the ready list, the cap removed, the 64-line limit removed or moved, and a retired request keeping its memories.

With the fix (after the review changes below):

• Unit: 158 passed.
• Chaos: 83 passed.
• E2E: 453 passed.
• Tls: 176 passed, 15 skipped (the kTLS cases; no tls module loaded here).
• Http: 95 passed.

The solution builds with 0 errors.

Review

An 8-agent review of this PR and its siblings (#490, #491, #492) found the following, fixed in 7dea7fa:

• A bug in the first version. SendAfterCreditAsync called back into Send, so every credit stall inside one flush nested another async frame, kept until the flush ended.
  • A peer holding a stream window at 0 while it opened the connection a byte at a time grew memory without bound: 4.1 MB per 20,000 one-byte WINDOW_UPDATEs.
  • The deep unwind then moved the writer onto the thread pool. Main's single loop had neither problem; the method loops again now.
• Wrong wake result. Writers woken by a faulted turn were told their flush succeeded.
• The pool was unbounded, kept the memories of its last stream, and was filled off the reactor in one path. All covered above.
• The tests could pass vacuously: nothing checked that a flush waited or that the handlers ran inside the window.

Two more review rounds followed. The second found a credit wait ignoring its turn's PeerGone, fixed in bcc0bb7. The third found a regression that commit brought in, fixed in 7cdc212:

• A dispatch walked the ready list in place, leaving dispatched requests in it until the walk ended. A handler calling connection.Dispose() during the walk disposed a parked handler's request under it. Nothing in this repo does that. The Queue removes the case, and the walk's diff against main is one line.
• A writer already parked on credit stayed parked when another writer's flush faulted, as it does on main.

Bench

Re-measured on the final code, with the same setup, 6 more rounds:

• ManagedBuffered: -16.6% server CPU per request (control -0.3%), +7.3% req/s.
• ManagedStreamedBoth: -34.9% server CPU per flush (control -1.0%), +53.8% flushes/s.
• Allocations: 92 B per request and 0 B per flush.
• One round of the branch ran slow (717 and 467 ns, against 486-494 and 310-314 in the others); the medians hold.
• The Queue has no visible cost: before it, the same rig measured -14.8% and -34.9%.

The tables below are the first run, before those changes.

Setup:

• One reactor pinned to CPU 6, and h2load -c 32 -m 16 -t 1 pinned to CPU 2: different physical cores, with their siblings left idle.
• Three builds: main (base), this branch, and main again from a second worktree as a control.
• Six rounds of 10 s, each after a 3 s warm-up, with the order rotating every round.
• Each build is compared with base inside its own round; the table shows the median across rounds.

Sample	Metric	base	branch	control vs base	branch vs base
Http2/ManagedBuffered /hello	server CPU per request	612 ns	507 ns	-1.6%	-17.0%
	req/s	1.27M	1.38M	+0.5%	+8.3%
Http2/ManagedStreamedBoth /feed, 1 KiB per flush	server CPU per flush	481 ns	316 ns	-1.1%	-34.5%
	flushes/s	2.07M	3.16M	+1.1%	+52.4%

• The buffered server ran at 70-78% CPU, so its req/s is bounded by h2load. Server CPU per request is the number that holds.
• The streamed server ran at 100% CPU.
• The per-round ranges do not overlap:
  • buffered: 606-617 ns on base, 500-518 ns on the branch;
  • streamed: 477-531 ns on base, 313-318 ns on the branch.

Allocations, from dotnet-counters over 20 s under the same load:

	base	branch	control
/hello, B per request	793	89	794
/feed, B per flush	376	0.0	390
gen0 GCs/s, /hello / /feed	67 / 45	7 / 0	67 / 49

• What is left on /hello, from a gc-verbose trace of the branch:
  • the sample handler's own Http2Response and its KeyValueList, 80 B, which is app code;
  • the pump's state machine, once per pass, 8.5 B per request.
• Backpressure: the working set stayed flat over the streamed run on both builds: base 111-118 MiB, branch 91-94 MiB.

Not in this PR: the other stacks named in #484 (ioxide.nghttp2, ioxide.nghttp3, ioxide.http3) still allocate when their flushes wait.

Fixes #484.
Fixes #485.