nghttp2: a request allocates about 650 B of parser objects that could be reused per connection #489

Each buffered request in ioxide.nghttp2 allocates about 655 B in objects made per request. It is the same set that #486 removes from ioxide.http2 for #485. Async machinery is small here: about 7 B per request, one FlushEgressAsync box per pass.

Measured on main eb5f515 (paths under src/protocols/ioxide.nghttp2/ unless named):

• App: Playground/Http2/Nghttp2Buffered, path /.
• Setup: 1 reactor pinned to one core, and h2load -c 32 -m 16 -t 1 pinned to another physical core: 1.16M req/s.
• Method: a 10 s dotnet-trace --profile gc-verbose window, allocations grouped by type and first ioxide frame.

The streamed sample (Playground/Http2/Nghttp2Response, 8 chunks of 1 KiB):

• It allocates the same request objects, about 660 B per request.
• Its flushes add about 19 B per request, or 2.4 B per flush. A flush that finds a drain already running returns at once instead of waiting, so the flush half of http2: a streamed flush that waits for the write pump allocates nothing, and a connection reuses its request objects #486 has nothing to fix here.

Possible fix: the one #486 applies to ioxide.http2.

• Return a retired PendingRequest to its connection with its Nghttp2Request, header KeyValueList and _fields, and reuse them for the next stream.
  • It is retired at the pending.Dispose() calls in Connection/Nghttp2Connection.RunBuffered.cs, and in ServeStreamedAsync (Connection/Nghttp2Connection.Streamed.cs:95).
  • KeyValueList.Clear already keeps its array for this.
• The request object is then reused once its handler is done, as in http2: a streamed flush that waits for the write pump allocates nothing, and a connection reuses its request objects #486. It was already only valid during the handler.
• Swap two lists instead of _readyThisPass.ToArray().

In ioxide.http2, the same change took a request from 793 B to 89 B, and server CPU per request down by 17%.