feat(middleware): introduce improved HTTP request/response hooks for the middleware by pimlock · Pull Request #4359 · NVIDIA/OpenShell

pimlock · 2026-10-09T01:23:18Z

Summary

Introduce HTTP session hooks: streaming HTTP request and response hooks that replace the v1 HTTP hooks.

Why

The v1 hooks were built for small, buffered checks. The use cases we want next, such as signing a Git push or a large AWS request, need the whole body regardless of size and need to change the body and its headers together.

Large bodies. EvaluateHttpRequest is unary: the whole body in one message, at most 4 MiB, under one RPC deadline. A larger request fails with request_body_over_capacity. Session hooks can stream a body with no total size or time cap.
Headers that depend on the body. Signing reads the whole body, then rewrites it along with headers such as Content-Length or a SigV4 payload hash. A session hook stage can send header mutations after it has read the complete body.
Free-form response processing. HttpResponsePreReturn is lockstep: each body unit gets exactly one result, and a service can't hold bytes for a later unit. In STREAM mode, input and output chunks are independent, so a service can redact across chunk boundaries or re-frame a stream.
One model for both directions. Requests and responses share one lifecycle (preflight, body, finish or deny) with the same body modes and limits.

Why not extend v1. A unary RPC can't become a stream without breaking every deployed service. Lockstep is a semantic guarantee, not a field, so relaxing it on the same RPC would change behavior that buf breaking can't detect. Older peers would also ignore new fields and call an extended service with v1 semantics. A separate service gated by a required capability makes older peers refuse it, and v1 keeps working unchanged until it is removed.

Validation. Two prototypes on an earlier head of this PR exercise the API: Git commit signing streams a push larger than 4 MiB through an external signer that keeps the key outside the sandbox, and SigV4 signing signs the final request after session stages transform its body.

Tip

The interactive walkthrough covers the motivation, the new API, version selection, and migration in more detail.

What this PR adds

New streaming API. HttpSessionMiddleware.EvaluateRequest and HttpSessionMiddleware.EvaluateResponse, a new service served on the same endpoint as SupervisorMiddleware, with one stream per HTTP message. A preflight decides whether to continue, inspect the body, reject a request, or block response delivery with block_delivery. Body inspection can be buffered or streamed.
Opt in with a capability, no new HTTP operations. A service keeps its HTTP_REQUEST and HTTP_RESPONSE bindings and lists openshell.supervisor-middleware.http-session in its extension required_capabilities. Gateways and supervisors have rejected unmet required capabilities at Describe since v0.1.0, so older peers refuse the service instead of calling it through the v1 RPCs. No RPC, message, or enum value carries a version.
No fail_open. The session hooks are always fail-closed, which simplifies the UX, the API, and the code.
v1 hooks are deprecated. EvaluateHttpRequest, HttpResponsePreReturn, and the types only they use are marked deprecated in the protobuf contract and are up for removal in the next version bump (0.2.0). Migration notes are in the new docs page. Both hook versions work side by side until then.

Related Issue

Part of #2431 and #3307.

Changes

Gateways and supervisors advertise the openshell.supervisor-middleware.http-session capability. Registration rejects a manifest that lists it as supported but not required, because older peers would accept that service and call v1 HTTP hooks. extension_protocol::http_session_middleware_metadata builds the metadata.
The gateway rejects on_error: fail_open for session hook services, and rejects policies whose overlapping selectors would mix hook versions in one chain. As on main, a fail_closed entry cannot select a tls: skip endpoint, and gateways and supervisors both enforce that rule.
Middleware selector patterns can no longer contain :, except in IPv6 literals. A host:port pattern used to parse as a hostname that never matched, so the entry was silently never selected. Upgrade note: a stored policy with such a pattern now fails validation on the gateway and on supervisors. Remove the port from the pattern; the entry never matched it.
The content guard example moves to HTTP session hooks.
RFC 0009: contract versioning now describes capability-gated contracts and the removal of deprecated elements in a minor release, as RFC 0014 allows.
Docs: session hooks are documented under Supported Operations, with shared body modes and limits, a short legacy migration note, and updated links.

Testing

mise run pre-commit passes
Unit tests added/updated
E2E tests added/updated: new mise run e2e:middleware-http-session suite
buf breaking against v0.1.2 reports no breaking change

Checklist

Follows Conventional Commits
Commits are signed off (DCO)

github-actions · 2026-10-09T01:23:56Z

🌿 Preview your docs: https://nvidia-preview-pr-4359.docs.buildwithfern.com/openshell

copy-pr-bot · 2026-10-09T01:31:17Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

pimlock · 2026-10-10T01:17:25Z

/ok to test 7f09efe

Add EvaluateHttp, a streaming HTTP middleware protocol with BUFFERED and STREAM body modes, next to the released HTTP protocol 1, which keeps its engines unchanged and is deprecated for removal in 0.2.0. - One EvaluateHttp stream per HTTP message and stage, or per uninspectable connection. A preflight selects continue, inspect, or reject; late header mutations apply before the head commits. - A service implements one HTTP protocol, chosen per binding with http_protocol_version, and a protocol 2 service requires the openshell.supervisor-middleware.http-v2 capability. The gateway delivers each service's protocol to supervisors. - Protocol 2 is always fail-closed. The gateway rejects fail_open on protocol 2 services and policies whose overlapping selectors mix protocols; a mixed chain fails closed at runtime. - Protocol 2 middleware decides about uninspectable traffic (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) at a preflight, and is exempt from the static tls: skip conflict rule. - Request bodies stream to the upstream unless a later step needs the whole body; responses stream with chunked framing or buffer before commit. - Migrate the content guard example to protocol 2, add a Docker e2e suite, and document the protocol. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Replace the http-v2 capability and per-binding protocol fields with new binding operations, and stop delivering the protocol to supervisors. - HTTP_REQUEST_V2 and HTTP_RESPONSE_V2 select HTTP protocol 2. Released gateways and supervisors already reject unknown operation/phase pairs at Describe, so the capability, extension_metadata_with_requirements, and MiddlewareBinding.http_protocol_version are removed. - Drop supported_http_body_modes. The preflight offers every mode the message is eligible for, and the stage picks one or continues or rejects. A protocol 2 binding with a zero payload limit is offered no body modes. - Move the tls: skip conflict rule out of the shared policy validator into the gateway's safety validation, which knows each service's protocol. This removes the delivered SupervisorMiddlewareService protocol field, the process-wide protocol 2 set in openshell-policy, and the supervisor's undescribed-service tracking. An undescribed entry follows on_error, which the gateway requires to be fail_closed for protocol 2 services. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Replace EvaluateHttp with EvaluateHttpRequestV2 and EvaluateHttpResponseV2, one RPC per binding operation, as for HTTP protocol 1 and WebSocket. Both keep the shared HttpEvent and HttpResult messages, so the stage pipeline is unchanged; OpenShell picks the RPC from the preflight subject. Traffic OpenShell cannot inspect stays on the request RPC. Only services that bind HTTP_REQUEST_V2 decide about it, so only they are exempt from the gateway's tls: skip conflict rule; a response-only protocol 2 service keeps the rule and its fail-closed entry denies such traffic at runtime. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Rename "HTTP protocol 1/2" to v1 and v2 HTTP hooks across code, proto comments, docs, skills, and the example. The docs page moves to extensibility/supervisor-middleware/http-hooks-v2, the protocol2 modules become http_v2, HttpProtocol becomes HttpHookVersion, and the mixed-chain reason becomes middleware_hook_versions_mixed. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Rename the streaming HTTP hook RPCs to EvaluateHttpRequestSession and EvaluateHttpResponseSession, and drop the unreleased HTTP_REQUEST_V2 and HTTP_RESPONSE_V2 operations. A service keeps its HTTP_REQUEST and HTTP_RESPONSE bindings and selects HTTP session hooks for all of them by listing openshell.supervisor-middleware.http-session in its extension required_capabilities. Released gateways and supervisors already reject unmet required capabilities at Describe, so they never call such a service through the v1 RPCs. Registration rejects a manifest that supports the capability without requiring it. Mark the v1 HTTP hook RPCs, the HttpResponsePreReturn service, and the types only they use deprecated in the protobuf contract. Rename the docs page, module, and e2e suite to HTTP session hooks, and update RFC 0009 so capability-gated contracts and removing deprecated elements in a minor release match the release stability policy. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Since #4016, host e2e runs omit the Docker driver's grpc_endpoint and let the driver discover the gateway. The HTTP session hook test required the key and panicked. Fall back to loopback, which the host-networked supervisor shares with the gateway, and read the key only from the Docker driver table. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

The cannot_inspect result replaced a FAILED_PRECONDITION status convention after review, which asked only for an explicit, typed signal whose effect is the reason. A separate result went further: it added a third terminal result to every phase and a failure response (502 on responses, platform-owned failure_code) for what is a decision the stage makes. Remove HttpCannotInspect and the cannot_inspect result field from both session result messages. They are unreleased, so they are deleted, not reserved. A stage that cannot inspect a message rejects it, or blocks delivery, with a reason_code that says so. The content guard example uses reason code cannot_inspect. The platform failure_code middleware_cannot_inspect remains for messages OpenShell cannot show to a stage. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Resolve conflicts: - e2e-docker-test.yml: keep both new suites in the default matrix, rust-docker-push from main and middleware-http-session. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock · 2026-10-20T20:59:35Z

Moved the new HTTP session RPCs into HttpSessionMiddleware, with EvaluateRequest and EvaluateResponse as the method names. The service shares the existing endpoint, registration, discovery, configuration, and authentication.

This keeps HTTP session methods out of the generated SupervisorMiddleware interface, so middleware implementing other hooks no longer needs HTTP session stubs. It also gives future hook families room to evolve separately. Each preflight still identifies the phase, so the same RPC method can support additional phases later without phase-specific names. This change adds no phases and preserves the legacy RPC paths.

…ed hook Uninspectable connections (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) are not HTTP, so they no longer use an HTTP session preflight. A service binds UNINSPECTABLE_NETWORK/PRE_FORWARD and serves the unary UninspectableNetworkMiddleware.Evaluate on the same endpoint. The hook carries only connection metadata, can be a service's only binding, and always fails closed. Entries without the binding keep their on_error behavior: fail_closed denies, and fail_open relays with a bypass finding. That fail_open exception is deprecated for removal in 0.2.0. The gateway tls: skip conflict rule now exempts middleware that binds the new hook. The tls: skip decision runs before 200 Connection Established, so a denial is a plain 403. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

The L7 parser rejects request heads that are not valid UTF-8 before any middleware runs. A response head that is not valid UTF-8 fails an HTTP session hook chain closed with middleware_cannot_inspect, while v1 response hooks follow on_error. Session hooks also fail a message with trailers that are not valid UTF-8. State this on HttpHeader so service authors know the string type matches what OpenShell forwards. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

HttpRequestSessionPreflight skipped field 2 after the uninspectable subject moved to its own hook. These fields were never released, so renumber them to match HttpResponseSessionPreflight instead of leaving a gap. Rename UninspectableTrafficReason to UninspectableNetworkReason to match UninspectableNetworkMiddleware and the other new UninspectableNetwork names. Document behavior the contract left implicit: a trailer write cannot add a name, Content-Length framing drops trailers, and Process.pid is 0 when unknown. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Routes that hold request output until the chain finishes capped it at 4 MiB, while the contract said STREAM has no total size limit. A pass-through STREAM stage on a SigV4 route therefore failed requests between 4 and 10 MiB that the route accepts without middleware, and the stage had no way to know. Withhold output up to the route's own buffering limit: 10 MiB for SigV4 body signing, 256 KiB for body credential rewriting, and 4 MiB otherwise. Request preflights report that limit to STREAM stages as the new HttpBodyLimits.max_output_body_bytes, so a stage can decide at preflight. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

The trailer mutation rules added in 9f7a04a landed on HttpInputChunk.data and inside the HttpDenial comment. Move them onto HttpBufferedResult.trailer_mutations and HttpFinish.trailer_mutations, and restore the HttpDenial comment. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

…e can inspect bodies HTTP session hooks offer no body mode for a response with a content coding other than identity. The sandbox controls Accept-Encoding, and common clients (requests, httpx, fetch) send gzip by default, so most compressed responses reached response stages uninspectable: a guard either blocked them with cannot_inspect or let them through unread. When the response chain has an HTTP session hook stage that may select a body mode, the relay now replaces every Accept-Encoding field with identity. It does so as the last header step on both the buffered and the streamed request path, after request-stage mutations and before credential injection, so a request stage cannot undo it. Request middleware still sees the sandbox's value. v1 hooks and bindings with max_payload_bytes 0 are unchanged. An encoded response now means the upstream ignored identity. The e2e upstream gains a /compressible route that gzips when asked, and the workload asks for gzip as common clients do. Without the change the guard answers 403 cannot_inspect; with it, the response is redacted. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

A response stage that needs the request's headers, such as a guard that judges a response against what was asked, had to keep per-request state keyed by request_id and route both streams to the same replica. HttpResponsePreflightHead.request_headers now carries the request headers as the request middleware chain forwarded them: after every request stage's mutations, including a streamed body's late mutations, and before OpenShell injects credentials or replaces Accept-Encoding. They have the omissions and limits of the request preflight's headers, and a value that contains a credential placeholder reads [REDACTED], as the response target's query already does. The relay hands response middleware a request that may already carry token grants, under header names the request-header sanitizer does not know, so the head is recorded before injection: by HttpMiddlewareExchange when request middleware allows the request, and before prepare_inspected_request on the inspected relay path. A live body applies its late mutations to that record when its head commits, before the upstream can answer. A response chain with no recorded head fails closed with middleware_cannot_inspect, as do request headers over the head limits. The response preflight's context comment now says request_id matches the request's preflights. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

HTTP session hook heads already omit Authorization, Cookie, and other credential headers, but a credential placeholder in a custom header, such as X-Api-Key: openshell:resolve:env:API_KEY, reached request stages as-is and response stages as [REDACTED]. Whether a stage saw a placeholder depended on which header carried it. A request header whose value carries a credential placeholder, including one inside a Basic credential, is now a credential header for HTTP session hooks. The request preflight, begin, and the response preflight's request_headers omit every value of that name, and the pipeline rejects a stage mutation that writes or removes it with header_mutation_protected_header, so a stage cannot replace or drop a credential reference it cannot see. The [REDACTED] sentinel is gone. Like Envoy's ext_proc forwarding rules, OpenShell drops headers rather than masking their values; a typed list of withheld names can be added later if a stage needs to know they were present. v1 HTTP hooks keep their released behavior and still receive these headers. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Resolve conflicts: - e2e/rust/Cargo.lock: take main's lockfile and add the flate2 dev-dependency entries at the versions this branch already pinned. Adapt to main: - Main's #4397 and #4684 report the inspected transport's scheme to middleware and OCSF. apply_middleware_chain_with_request_id_and_delivery, which the relays use for HTTP session hooks, still passed "https", so a plaintext request reached request stages as https. It now passes ctx.request_scheme(), and the scheme test covers that entry point too. - The request-body client timeout event uses ctx.ocsf_url() instead of a fixed http URL. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

The hook let middleware allow connections OpenShell cannot inspect (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) by metadata alone. No published middleware uses it, and the mixed-port cases it served are better handled by selector changes, so remove it before it ships. Uninspectable traffic goes back to the chain-level on_error rule: a fail_closed entry denies the connection, and an all-fail_open chain relays it with a bypass finding. That fail_open exception is no longer deprecated. The tls: skip overlap rule returns to shared policy validation, so supervisors enforce it again alongside the gateway. The tls: skip gate still runs before 200 Connection Established, so a denial is a plain 403. The proto service, operation, phase, and enums are deleted rather than reserved because they were never released. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Middleware selectors match hostnames on every port, but a pattern such as api.example.com:443 parsed as a hostname that never matches, so the entry was silently never selected. Reject ':' in include and exclude patterns at gateway and supervisor validation instead. IPv6 literals are still accepted. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Rejecting every ':' outside a complete IPv6 literal also rejected IPv6 globs such as 2001:4860:*, which selectors have matched since they shipped and network endpoints still accept. Treat a pattern containing ':' as IPv6 text instead, and reject only the shapes of a host:port authority: bracketed literals, characters IPv6 text cannot contain, dots before the last ':', a single ':' followed by digits, and invalid literals without wildcards. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock · 2026-10-23T17:10:22Z

Re: compatibility review, finding 1 (IPv6 selector globs)

Fixed in 97f049c. Middleware selector patterns that contain : are read as IPv6 text again, so globs such as 2001:4860:*, 2001:*, and fe80::* validate and match as they did before.

The check still rejects port-shaped patterns, which never matched any host:

bracketed literals: [2001:db8::1], [2001:db8::1]:443
characters that can't appear in IPv6 text: api.example.com:443, api.example.com:*
a dot before the last :: 10.0.0.1:*
a single : followed by digits: *:443, 10.0.0.1:443
wildcard-free patterns that aren't valid IPv6 addresses: 2001:db8::1::2

Gateway validation and the supervisor's local-file policy path share this check. Tests cover the accepted and rejected forms, and the schema and middleware configuration docs now say to write IPv6 addresses without brackets.

pimlock requested review from a team, derekwaynecarr, mrunalp and sjenning as code owners October 9, 2026 01:23

pimlock marked this pull request as draft October 9, 2026 01:31

pimlock changed the title ~~feat(middleware): add HTTP middleware protocol 2~~ feat(middleware): introduce improved HTTP request/response hook for the middleware Oct 9, 2026

pimlock changed the title ~~feat(middleware): introduce improved HTTP request/response hook for the middleware~~ feat(middleware): add v2 HTTP request and response hooks Oct 9, 2026

This was referenced Oct 9, 2026

feat(supervisor): add streaming HTTP request middleware evaluation #2431

Open

feat(middleware): add streaming v2 HTTP hooks and deprecate the unary request hook #3307

Open

pimlock changed the title ~~feat(middleware): add v2 HTTP request and response hooks~~ feat(middleware): introduce improved HTTP request/response hooks for the middleware Oct 10, 2026

pimlock marked this pull request as ready for review October 10, 2026 01:45

This comment was marked as outdated.

Sign in to view

pimlock added the gator:in-review Gator is reviewing or awaiting PR review feedback label Oct 10, 2026

This comment was marked as outdated.

Sign in to view

pimlock added the test:e2e Requires end-to-end coverage label Oct 10, 2026

This comment was marked as outdated.

Sign in to view

pimlock added gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback labels Oct 10, 2026

bettyc925 mentioned this pull request Oct 10, 2026

test(openshell): security regressions for NVIDIA/OpenShell#4359 and #4397 AtlaSent/atlasent-mcp-server#247

Merged

10 of 12 tasks

OrenDor mentioned this pull request Oct 11, 2026

feat(supervisor): let request middleware mark a single request as pending a human decision #4406

Open

7 tasks

pimlock requested a review from matthewgrossman October 12, 2026 02:44

pimlock added 5 commits October 13, 2026 08:23

pimlock requested a review from matthewgrossman October 16, 2026 21:06

pimlock added 2 commits October 16, 2026 14:25

This comment was marked as outdated.

Sign in to view

chore(middleware): merge main into HTTP session hooks

440de1a

Resolve conflicts: - e2e-docker-test.yml: keep both new suites in the default matrix, rust-docker-push from main and middleware-http-session. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock mentioned this pull request Oct 20, 2026

refactor(middleware)!: adopt two-mode HTTP body protocol #3450

Closed

13 tasks done

refactor(middleware): split HTTP session RPCs into dedicated service

4096ea0

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock mentioned this pull request Oct 21, 2026

refactor(middleware)!: remove legacy HTTP hooks in 0.2.0 #4781

Open

7 tasks

pimlock added 2 commits October 20, 2026 18:16

refactor(middleware): align HTTP denial naming

6e0de60

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

This comment was marked as outdated.

Sign in to view

pimlock added 9 commits October 21, 2026 11:47

fix(middleware): defer inspection decisions until stage input is ready

e66bfd3

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock added 2 commits October 22, 2026 14:44

This comment was marked as outdated.

Sign in to view

pimlock added 2 commits October 22, 2026 16:20

chore(middleware): merge main into HTTP session hooks

0c673ea

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Navigation Menu

Repository navigation

feat(middleware): introduce improved HTTP request/response hooks for the middleware - #4359

feat(middleware): introduce improved HTTP request/response hooks for the middleware#4359
pimlock wants to merge 41 commits into
mainfrom
feat/2431-http-middleware-protocol-2/pimlock

pimlock commented Oct 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 9, 2026

Uh oh!

copy-pr-bot Bot commented Oct 9, 2026

Uh oh!

pimlock commented Oct 10, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

pimlock commented Oct 20, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

pimlock commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pimlock commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

What this PR adds

Related Issue

Changes

Testing

Checklist

Uh oh!

github-actions Bot commented Oct 9, 2026

Uh oh!

copy-pr-bot Bot commented Oct 9, 2026

Uh oh!

pimlock commented Oct 10, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

pimlock commented Oct 20, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

pimlock commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pimlock commented Oct 9, 2026 •

edited

Loading