[MDA2AV]

What was wrong

• Every stream QuicEngineConnection sent on got a new OutStream (Connection/QuicEngineConnection.SendStream.cs), with its own Queue<(nint Ptr, int Len)> whose array grew as chunks queued.
• That cost both HTTP/3 stacks about 185 B per response. For ioxide.nghttp3, it was all of its per-request garbage.

The fix

• PurgeOutStream drops a stream's send state once the stream is closed. It now resets the OutStream and keeps it on the connection (_outStreamPool).
• SendStream takes one from there before it makes a new one.
• Why this is safe: everything reaches an OutStream through _outStreams by its id, so the old id finds nothing after the purge.
  • PumpOut can purge the stream it is pumping, and returns right after.
  • SendStream and ReplayOut then still hold it, and see a reset stream with nothing left to send.
  • When the pool is full the state is dropped unreset. Its id can stay in _outPending until the next ReplayOut, which finds no stream for it and drops it, as on main.
• Bounded:
  • The pool keeps at most 32 OutStreams, so a burst of up to 1024 concurrent streams does not stay on an idle connection.
  • Destroy clears the pool.
  • A queue that grew past 64 slots is trimmed to 64 before it is kept, so a stream of many small chunks leaves no large array behind.

Behaviour

• Unchanged on the wire.
• A connection's pool keeps at most 32 OutStreams; the live ones are bounded by the peer's stream limit, as before.

API

None.

Tests

New in the E2E suite: quic: a warm connection's responses allocate no send-stream state (nghttp3) (QuicSendStreamTests.cs).

• It makes 8 GETs one after another on one connection, which takes about 13 s: the test client spends about 1.5 s on each.
• The handler records what the reactor's thread allocated between request 4 and request 8, and checks that both marks come from that thread.
• ioxide.nghttp3 makes no other garbage per request, so what it measures is the send state alone.
• It allows under 184 B in total, one stream's send state, so a single response that misses the pool fails it.

Also new: quic: a burst leaves the send state of at most 32 streams, with no queue past 64 slots.

• 40 streams open at once, each flushing 100 chunks before any ack can land, so each queue grows to 128 slots. A later request reads the pool by reflection on the reactor's thread: 32 states, and the largest queue is 64.
• It fails with the cap removed (40 states) and with the trim removed (128 slots).

Against main's ioxide.ngtcp2 (the test only):

FAIL  quic: a warm connection's responses allocate no send-stream state (nghttp3): 4 responses on a warm connection allocated 736 B on the reactor, a stream's send state or more

With the fix:

• The new test measured 0 B per response, and passed three runs out of three at its tighter bound.
• E2E: 455 passed.
• Chaos: 83 passed.
• Http: 95 passed.

Review

An 8-agent review of this PR and its siblings found no correctness problem in the reuse itself. It found three things, fixed in 79bb5ee:

• Pool size: the pool's size was set by the peer.
• Trim: the trim went to 0 slots instead of 64.
• Test: it was slow and could pass with marks from two threads.

Bench

Re-measured after the review changes: allocation per request is unchanged. Nghttp3Buffered allocates 0.1 B (base 182 B), and ManagedBuffered 1,695 B (base 1,882 B). The tables below are the first run.

Setup: the same as #486.

• One reactor pinned to CPU 6, and h2load -c 32 -m 16 -t 1 (with --alpn-list=h3 for HTTP/3) pinned to CPU 2, on separate physical cores.
• Three builds: main (base), this branch, and main again from a second worktree as a control.
• Six rounds of 10 s, each after a 3 s warm-up, with the order rotating every round.
• Each build is compared with base inside its own round; the tables show the median across rounds.
• Streamed samples send 8 chunks of 1 KiB per response.

Sample	Metric	base	branch	control vs base	branch vs base
Http3/Nghttp3Buffered	server CPU per request	2.43 µs	2.39 µs	-0.1%	-1.5%
	req/s (server at 100%)	412K	418K	+0.1%	+1.6%
Http3/ManagedBuffered	server CPU per request	1.44 µs	1.41 µs	-1.6%	-2.4%
	req/s (server at 94%)	651K	662K	+2.0%	+1.8%

Allocations, from dotnet-counters over 20 s under the same load:

	base	branch	control
Nghttp3Buffered, B per request	186	0.1	186
ManagedBuffered, B per request	1,887	1,688	1,883
gen0 GCs/s, nghttp3 / managed	4 / 63	0 / 59	4 / 64

• ioxide.nghttp3 now allocates nothing per request.
• The CPU effect is small. An HTTP/3 request costs about 2.4 µs here, mostly QUIC and crypto, so 185 B less garbage is a small share of it.
  • On nghttp3 it is -1.5%, against a control that held within 0.1%.
  • On the managed stack it is inside the noise; the control moved 1.6%.
• The rest of the managed stack's garbage is its own request objects (http3: a request allocates about 1.7 KB of objects that could be reused per connection, 1 KiB of it a new arena #488).

Fixes #487.