[antoinetoussaint-byte]

What

The generated Connect face (templates/connect/connect_gen.py) opened its channel to the service with grpc.max_receive_message_length = max_message_bytes. That channel receives the service's answers, so every transcoded answer was capped at the request bound. Found by review r38 of obin-ai/module-model-gateway#210: the gateway bounds requests well below 4 MiB, and a 393 KB embedding answer came back 429 resource_exhausted on Connect while direct gRPC (relayed as it came) returned it.

Change

• serve(...) / start(...) take max_response_bytes (default DEFAULT_MAX_RESPONSE_BYTES = 64 MiB), the channel's receive length. Past it the call ends resource_exhausted naming grpcio's reason.
• max_message_bytes is unchanged: it bounds a request body before it is read (HTTP 413 naming the bound) and is the channel's send length.
• The relayed gRPC path is untouched (it never applied either bound).

A bounded large default rather than unbounded: the face buffers a unary answer whole and transcodes it (JSON grows it), so an unbounded default would let one handler's answer size the process's memory. A service that needs more sets max_response_bytes the same way it sets max_message_bytes.

Tests (testdata/connect/test_connect_gen.py, run by TestConnectFaceServesEveryProtocol)

• An answer of 393 KiB and of 5 MiB (above grpcio's 4 MiB default) through a face whose request bound is 1024 bytes comes back on Connect unary proto, Connect unary JSON, gRPC-Web and a Connect server stream.
• An oversized request is still refused, 413, with the body content length (N) exceeds the message bound; the existing refusal test now asserts that text too.
• With max_response_bytes=2048 set, an answer within it passes and one past it ends resource_exhausted ("larger than max") on all five media types.
• Diagnosis: with the old max_receive_message_length = max_message_bytes restored, the 7 new cases fail and the other 23 pass. With the fix, 30/30 pass.

Local: go build ./..., go vet ./..., go test -run TestConnect ./..., and the Python suite. I did not run the full go test ./... or the race job locally (Docker-backed tests); CI covers them.

Classification

Fix, at the owner: the face template is what every connect-endpoint service is synced from. The consumer (module-model-gateway) picks it up on its next codefly sync with this agent's release; no change is needed in its call (serve(..., max_message_bytes=bound) now bounds requests only).

Relation to #110

Cut from main (2c266fd). #110 (concurrency, under independent review) was not touched. Its hunks are in the HTTP/2 relay and the constants after line 63; this change sits in the docstring, one new constant and serve(), so the two should merge without conflict in either order.

Reused: grpcio's own channel receive bound (no new mechanism). Not done: no bound on the raw gRPC relay, which keeps the service's and caller's own limits.

Review r45 fixes (commit a7eefee)

• f1 (blocker): the fixed 64 MiB default regressed services with a larger message bound. Fixed. max_response_bytes now defaults to the larger of DEFAULT_MAX_RESPONSE_BYTES and max_message_bytes, so a service that raised max_message_bytes keeps answers that large. New test: a 100 MiB request bound with a 70 MiB answer returns 200.
• f2 (major): the memory argument did not hold. Fixed.
  • The default answer bound drops from 64 MiB to 16 MiB.
  • The bound now counts what the face sends: the answer bytes for proto, the converted document for JSON. So the JSON the face holds per call stays within the bound.
  • What the docstring now says it costs:
    • Per call in flight, the face holds the answer and its encoding, each within the bound, until the client reads.
    • JSON conversion has a transient peak of about 13x the proto for a float list. I measured it with protobuf 6.33.6: a 3.8 MiB proto peaks at 51 MiB, so about 210 MiB at the default bound.
    • Conversions run synchronously on the face's event loop, one at a time, so that peak does not add up across calls.
    • The face does not limit how many calls are in flight, and neither does the gRPC server.
  • Within the bound, Connect returns what gRPC returns. A float-heavy answer can fit the bound as proto but not as JSON; it is then refused on JSON and served on proto.
• f3: the refusal did not name the knob and used a retryable 429. Fixed.
  • The message now reads "the answer (N bytes) exceeds the Connect face's max_response_bytes (M)". It replaces grpcio's "Stream removed (CLIENT: …)" text and is also used for the JSON size check.
  • The code is now failed_precondition: HTTP 400 for Connect unary, grpc-status 9 for gRPC-Web, and the error frame for streams.
  • This departs from the review's suggestion to keep 429, on the coordinator's direction: a deterministic size refusal must not invite blind retries.
• f4: the docstring paragraph is reflowed.

The new tests fail on 434f403. The face suite passes (34 cases); merged with #110 (82a7ade) it merges cleanly and passes 41. The tests sit beside their siblings rather than at the end of the file, which is where they conflicted with #110. go vet passes, and so do the Connect Go tests.

Review r79 fixes (rebased onto main c43d3c3, commit 1cadf23)

The branch is rebased onto main, which now includes #110, #112 and #113. The original commit is kept, and the r45 fixups are squashed into one commit.

This round supersedes two things in the r45 section above:

• The post-conversion JSON size check is gone.

• Answers now have two bounds, one per codec: max_response_bytes for proto, max_json_response_bytes for JSON.

• M1 (major): a service's own refusal was rewritten as the face's. Fixed.

  • Cause: the face rewrote any "Received message larger than max" text into its own failed_precondition, including a refusal the service passes on from its own gRPC client. Both the code and the reason were then false.
  • Now: the face checks its bounds itself, comparing the answer's length with the bound for the call's codec. It treats grpcio's text as its own refusal only when it is grpcio's complete wording and names exactly the receive bound the face set on its channel; that only happens for answers past both bounds.
  • Remaining edge: a service passing on a refusal from a client whose bound happens to equal the face's channel bound exactly would still be misread.
  • Test: test_a_refusal_the_service_passes_on_keeps_its_code_and_words.

• M2 (major): the memory claim was wrong and the JSON check came too late. Fixed.

  • JSON answers now have their own bound on the proto the service returns: max_json_response_bytes, by default the larger of 4 MiB (the bound every answer had before fix(connect): bound answers apart from requests #111) and max_message_bytes. It is checked before converting.

  • The docstring now gives measured process memory with protobuf 6.33 for one 4 MiB answer (probe111/amp3.py):

    Answer shape	Peak memory
    packed floats	17.6x the proto
    nested messages	61.5x
    list of enums	129.6x (its JSON alone is 25x)

  • That is up to about 520 MiB at the default, the same worst case as before fix(connect): bound answers apart from requests #111.

  • Proto answers keep max_response_bytes (16 MiB by default). The face holds about twice the answer beyond the service's own copy: 33 MiB measured for a 16 MiB answer.

  • Test: test_the_json_bound_is_checked_before_converting shows an answer past the bound is never converted.

• m1: the docstring said the bound counts bytes sent. Fixed. It now says what is bounded: the proto the service returns, against the bound for the call's codec.

• m2: max_response_bytes=-1 refused every JSON answer. Fixed. Both bounds must be positive numbers of bytes; 0 or -1 is refused at serve. Test: test_an_answer_bound_is_a_positive_size.

• Behaviour change: a JSON answer above 4 MiB is now refused by default, as it was on main before fix(connect): bound answers apart from requests #111. Proto answers up to 16 MiB still come back.

• Checks:

  • The face suite passes (49 cases). The new tests fail on the previous head.
  • go vet passes.
  • Locally, go test ./... fails only TestConnectOnlyServiceSyncs (from fix: a service whose faces are gRPC and Connect alone loads, syncs and starts #113): Buf's BSR rate-limits this machine ("resource_exhausted: too many requests"), so it is unrelated to this diff.

🤖 Generated with Claude Code