feat(supervisor): add streaming HTTP request middleware evaluation #2431

Problem Statement

Parent: #1733

The v1 HTTP request hook, SupervisorMiddleware.EvaluateHttpRequest, is unary. The supervisor buffers the whole request body, builds one HttpRequestEvaluation, and sends it to middleware. Request and replacement bodies are capped at 4 MiB.

That works for small, full-body inspection. It does not work for:

• Middleware that decides from headers alone and should not force the body into a buffer.
• Large uploads that should pass through unchanged, such as headers-only SigV4 signing.
• Incremental inspection or transformation with backpressure.
• Long-lived or unknown-length request bodies.
• HTTP/2 and gRPC streams, which don't fit in one bounded unary payload.

HTTP/2 support (#2426) also needs middleware to run per logical HTTP stream, not on raw connection bytes. The streaming gRPC part of #2166 needs method, metadata, body, duration, rate, and cancellation controls without making middleware implement HTTP/2 framing.

Status

PR #4359 delivers the middleware contract half of this issue: HTTP session hooks for requests and responses over HTTP/1.x. The HTTP/2 half is still open. Traffic the supervisor can't parse, including h2c, never reaches middleware.

Design (as implemented in #4359)

Contract

A new service, HttpSessionMiddleware, is served on the same endpoint as SupervisorMiddleware:

• EvaluateRequest serves HTTP_REQUEST bindings at PRE_CREDENTIALS.
• EvaluateResponse serves HTTP_RESPONSE bindings at PRE_RETURN.

Both are bidirectional streams. Each RPC has its own event and result messages with the same lifecycle. The supervisor opens one stream per middleware stage and HTTP message, so a request and its response use separate streams.

Each stream starts with a preflight. It carries the head, phase, request context, the policy config, the permitted body modes, the modes OpenShell removed and why, the limits, and the declared body length when known. A response preflight also carries the request headers as the request chain forwarded them, before credential injection. The stage answers with one of:

• continue_without_body, with optional header mutations. The stage is done, and later stages still see the body.
• inspect with BUFFERED or STREAM.
• deny for a request, or block_delivery for a response. Both are also valid later in the exchange.

A stage gets its preflight once the previous stage settles its output headers: at buffered_result for BUFFERED, at output_start for STREAM, and immediately for a stage that continues without the body. Later stages therefore choose a mode for the representation they will actually receive.

Body modes

• BUFFERED. One buffered_body with the full body and trailers, up to the stage's payload limit (max 4 MiB). The stage returns unchanged or a replacement, plus late header and trailer mutations. That stage's output waits for its result. Later stages can still stream.
• STREAM. Ordered input_chunk events (64 KiB max each) and one input_end with trailers. The stage writes its own output: output_start, any number of output_chunk, then finish. Input and output chunk counts are independent. There is no total size or time limit. A stage that stalls for 30 seconds fails.

Until every stage finishes, the supervisor withholds the final byte that would complete the message, so a late denial never leaves the recipient with a message that looks complete.

The supervisor removes modes it can't honour and reports why. Encoded bodies get no modes. Bodyless, partial, and no-transform responses get no modes. A declared length over the stage limit removes BUFFERED. Event streams remove BUFFERED for responses. A response that isn't HTTP/1.1 loses STREAM. When a response stage might inspect the body, the supervisor sends Accept-Encoding: identity upstream, so the sandbox can't make responses uninspectable by asking for compression.

Headers whose value carries a credential placeholder are credential headers: session hooks never see them and can't change them. Stages own integrity headers such as ETag and Digest when they change a body.

Request delivery

Request output streams to the upstream while the sandbox is still uploading. The supervisor buffers output instead when a stage selects BUFFERED, the route needs SigV4 signing or request body credential rewriting, or the request uses the forward proxy, HTTP/1.0, or Upgrade. JSON-RPC, MCP, and GraphQL endpoints also buffer, so replaced bodies are re-checked against policy before the next stage or the upstream sees them. The buffer limit is 10 MiB for SigV4, 256 KiB for credential rewriting, and 4 MiB otherwise; the preflight reports it to STREAM stages as limits.max_output_body_bytes. Credentials are injected after all middleware mutations.

If a stage fails after the head reached the upstream, the supervisor closes that upstream connection. It never replays a partly forwarded request. A sandbox that stops uploading for 30 seconds gets 408 request_timeout.

Failure behaviour

Session hooks always fail closed. The gateway rejects on_error: fail_open on entries whose service uses them. Requests get 403 with middleware_denied or middleware_failed. Responses get 403 or 502 before the head is committed, and an aborted delivery after. Failures carry a platform-owned failure_code, such as middleware_timeout, stream_idle_timeout, or body_over_limit. No free-form service text reaches the sandbox.

Version selection and compatibility

A service selects session hooks for all of its HTTP bindings by listing openshell.supervisor-middleware.http-session in both supported_capabilities and required_capabilities. The bindings stay HTTP_REQUEST and HTTP_RESPONSE, and no RPC, message, or enum value carries a version. Gateways and supervisors have rejected unmet required capabilities at Describe since v0.1.0, so older peers refuse the service instead of calling it through v1 RPCs. Registration rejects a manifest that lists the capability as supported but not required, because older peers would accept that service.

• v1 hooks keep their own engines and still work. They are deprecated for removal in 0.2.0, and the docs carry a migration guide.
• One request or response chain can't mix hook versions. The gateway rejects policies whose overlapping selectors would do that, and the supervisor fails such a chain closed with middleware_hook_versions_mixed.

Traffic the supervisor can't inspect

Unchanged from main. Middleware never sees tls: skip tunnels, h2c, unsupported tunnels, raw TCP, or SQL passthrough. If a selected entry is fail_closed, the supervisor refuses the connection. If every selected entry is fail_open, it relays the connection and records a finding. A fail_closed entry still can't select a tls: skip endpoint. An earlier iteration of #4359 added a hook for these connections; it was dropped before merge.

#4359 also rejects middleware selector patterns that contain :, except IPv6 literals. A host:port pattern used to parse as a hostname that never matched.

Remaining work: HTTP/2

#2426 tracks the transport gap: the L7 proxy only negotiates HTTP/1.1, so HTTP/2 needs tls: skip and skips L7 inspection and middleware.

The session hook contract carries no HTTP/1-specific framing, so an HTTP/2 request stream should map onto the same exchange: one stream per stage and logical HTTP request. What's left:

• Negotiate HTTP/1.1 or HTTP/2 for inspected traffic and relay HTTP/2 through the L7 path without tls: skip.
• Run each HTTP/2 request stream through its own session hook exchanges.
• Keep multiplexing, flow control, trailers, and cancellation working, and keep a denial or timeout on one stream from killing the others.
• Review the response mode rules for HTTP/2. STREAM is currently removed for any response that isn't HTTP/1.1.

The middleware API must not expose HTTP/2 frames, HPACK state, or stream IDs.

#2166

Session hooks give the gRPC part of #2166 a base at the HTTP stream level: host, service, and method admission through authority and path, metadata limits, request byte, rate, duration, and concurrency limits, and cancellation and audit. gRPC message boundaries and protobuf parsing would need their own typed operation. Generic body chunks are not gRPC messages.

WebSocket middleware is tracked by #2428 and works the same under both hook versions.

Alternatives Considered

New versioned operations (HTTP_REQUEST_V2, HTTP_RESPONSE_V2). An earlier iteration of #4359. Older peers rejected the unknown operations, but every version would add enum values. A required capability gets the same refusal from older peers without versioning the contract.

Run v1 through an adapter on one shared runner. More moving parts in the supervisor to keep alive an API we plan to remove. Separate engines plus a no-mixing rule are simpler and easier to delete in 0.2.0.

Replace v1 right away. v1 was already public in 0.1.x. Deprecation with a migration guide gives services a release to move.

Add an HTTP/2-specific middleware API. Same request concepts as HTTP/1.x. Two APIs would duplicate chaining, transformation, and failure rules and leak transport details.

Keep buffering every request. Can't handle large uploads, long-lived streams, or headers-only processing.

Definition of Done

Covered by #4359. Check these off when it merges:

• Reviewed design for the streaming hooks, body modes, lifecycle, and compatibility. RFC 0009 and the Supported Operations docs page describe it.
• Middleware can decide from headers without forcing the body into a buffer.
• BUFFERED mode holds the body under a 4 MiB bound and sends no bytes before the stage approves.
• STREAM mode has defined backpressure, transformation, rejection, and partial-disclosure behaviour.
• Mode selection can't silently weaken enforcement. Removed modes come with a reason, and routes that need the full body buffer output.
• v1 hooks keep working, with a deprecation date (0.2.0) and a migration guide.
• Ordering, mutation validation, policy re-checks, findings, metadata, and timeouts stay consistent. Session hooks are always fail-closed.
• Unit tests and the e2e:middleware-http-session suite cover session hooks over HTTP/1.x. The v1 e2e still passes.

Still open:

• The supervisor negotiates HTTP/1.1 or HTTP/2 for inspected traffic and relays HTTP/2 through the L7 path without tls: skip.
• HTTP/1.x requests and HTTP/2 request streams use the same session hook exchanges.
• The HTTP/2 relay keeps multiplexing, flow control, trailers, cancellation, and isolation between streams.
• Tests cover session hooks over HTTP/2.
• Docs describe HTTP/2 behaviour.

Non-Goals

• Parsing raw HTTP/2 frames in middleware.
• Treating HTTP body chunks as gRPC messages.
• Protobuf or app-specific media inspection.
• WebSocket message middleware (feat(supervisor): add WebSocket message introspection middleware #2428).
• HTTP/3.