Skip to content

http: forward upstream responses verbatim instead of re-encoding - #5

Open
wille wants to merge 1 commit into
masterfrom
response-passthrough
Open

http: forward upstream responses verbatim instead of re-encoding#5
wille wants to merge 1 commit into
masterfrom
response-passthrough

Conversation

@wille

@wille wille commented Jul 25, 2026

Copy link
Copy Markdown
Owner

What

Stop re-encoding upstream responses. haprovider now writes the raw upstream body back to the client verbatim, keeping the decode only to inspect the response (failover, metrics, batch-size checks).

Why

Every response was fully round-tripped through JSON: buffer the body (N bytes) → DecodeBatchResponse tokenizes all N and copies result into a fresh json.RawMessage (~N) → SerializeBatchResponse re-encodes it to send to the client (~N). That's ~3× the response size in allocations plus tokenize + marshal CPU on every request. For big eth_getLogs / eth_getBlockReceipts responses (often multiple MB) this is real CPU and GC pressure.

The re-encode is pure waste: we forward the client's request unchanged to one chosen provider, so the upstream response body is already exactly what the client should get. We keep the decode (still needed for in-band error detection, per-method metrics, and batch-size validation) and simply write the raw bytes.

How

  • httpx.SendRPCBatchRequest also returns the raw upstream body.
  • forwardResult carries raw; the direct path writes it via a new writeRaw helper (X-Provider + JSON content type, no serialize).
  • Coalesce path: when the caller's id matches the shared response's id (the leader and every non-shared call — the common case) it forwards verbatim; only a genuinely-shared follower with a different id rebuilds the envelope with its own id. The id match also guards against a provider mangling the echoed id.
  • Empty provider replies are refused with a 500 (errEmptyResponse), restoring the prior contract that SerializeBatchResponse used to enforce and removing a latent Responses[0] access on the coalesce path.

Behavior

Unchanged: failover, HandleError, metrics, batch-size mismatch, and auth all still run against the decoded response.

Benchmarks

BenchmarkIncomingHttpRpcHandler/large (200-log response):

before after
B/op 1,328,041 1,041,385 (~-21%)
ns/op 6,952,473 5,604,212 (~-19%)

Savings scale with response size.

Testing

  • New TestPassThrough_Verbatim: the client receives the upstream body byte-for-byte (a pretty-printed mock body survives, which a compact re-encode would not) on both the direct and coalesce paths.
  • New TestPassThrough_EmptyResponseIs500: an empty provider reply → 500 on both paths.
  • Existing coalescing (id echo for shared followers) and in-band-error failover tests still pass.
  • go test -race ./..., go vet, golangci-lint all clean.

Not included

The symmetric request-side pass-through (forwarding the raw request body instead of re-serializing) — smaller/situational payoff since requests are usually tiny; can follow up.

Every response was fully round-tripped through JSON: buffer the upstream
body, decode it (tokenize + copy result into a fresh RawMessage), then
re-encode it to send to the client. That's ~3x the response size in
allocations plus tokenize + marshal CPU on every request. For large
eth_getLogs / eth_getBlockReceipts responses this is real CPU and GC
pressure.

The re-encode is redundant: we forward the client's request unchanged to
one chosen provider, so the upstream response body is already exactly what
the client should receive. Keep the decode (still needed for in-band error
detection, per-method metrics, and batch-size validation) and write the raw
bytes back verbatim.

- SendRPCBatchRequest also returns the raw upstream body.
- forwardResult carries raw; the direct path writes it via writeRaw.
- Coalesce path forwards verbatim when the caller's id matches the shared
  response's id (leader and non-shared calls); only a shared follower with a
  different id rebuilds the envelope. The id match also guards against a
  provider mangling the echoed id.
- Empty provider replies are refused with a 500 (errEmptyResponse),
  restoring the contract SerializeBatchResponse used to enforce and removing
  a latent Responses[0] access on the coalesce path.

BenchmarkIncomingHttpRpcHandler/large (200 logs): ~-21% B/op, ~-19% ns/op.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant