http: forward upstream responses verbatim instead of re-encoding - #5
Open
wille wants to merge 1 commit into
Open
http: forward upstream responses verbatim instead of re-encoding#5wille wants to merge 1 commit into
wille wants to merge 1 commit into
Conversation
Every response was fully round-tripped through JSON: buffer the upstream body, decode it (tokenize + copy result into a fresh RawMessage), then re-encode it to send to the client. That's ~3x the response size in allocations plus tokenize + marshal CPU on every request. For large eth_getLogs / eth_getBlockReceipts responses this is real CPU and GC pressure. The re-encode is redundant: we forward the client's request unchanged to one chosen provider, so the upstream response body is already exactly what the client should receive. Keep the decode (still needed for in-band error detection, per-method metrics, and batch-size validation) and write the raw bytes back verbatim. - SendRPCBatchRequest also returns the raw upstream body. - forwardResult carries raw; the direct path writes it via writeRaw. - Coalesce path forwards verbatim when the caller's id matches the shared response's id (leader and non-shared calls); only a shared follower with a different id rebuilds the envelope. The id match also guards against a provider mangling the echoed id. - Empty provider replies are refused with a 500 (errEmptyResponse), restoring the contract SerializeBatchResponse used to enforce and removing a latent Responses[0] access on the coalesce path. BenchmarkIncomingHttpRpcHandler/large (200 logs): ~-21% B/op, ~-19% ns/op.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stop re-encoding upstream responses. haprovider now writes the raw upstream body back to the client verbatim, keeping the decode only to inspect the response (failover, metrics, batch-size checks).
Why
Every response was fully round-tripped through JSON: buffer the body (N bytes) →
DecodeBatchResponsetokenizes all N and copiesresultinto a freshjson.RawMessage(~N) →SerializeBatchResponsere-encodes it to send to the client (~N). That's ~3× the response size in allocations plus tokenize + marshal CPU on every request. For bigeth_getLogs/eth_getBlockReceiptsresponses (often multiple MB) this is real CPU and GC pressure.The re-encode is pure waste: we forward the client's request unchanged to one chosen provider, so the upstream response body is already exactly what the client should get. We keep the decode (still needed for in-band error detection, per-method metrics, and batch-size validation) and simply write the raw bytes.
How
httpx.SendRPCBatchRequestalso returns the raw upstream body.forwardResultcarriesraw; the direct path writes it via a newwriteRawhelper (X-Provider + JSON content type, no serialize).errEmptyResponse), restoring the prior contract thatSerializeBatchResponseused to enforce and removing a latentResponses[0]access on the coalesce path.Behavior
Unchanged: failover,
HandleError, metrics, batch-size mismatch, and auth all still run against the decoded response.Benchmarks
BenchmarkIncomingHttpRpcHandler/large(200-log response):Savings scale with response size.
Testing
TestPassThrough_Verbatim: the client receives the upstream body byte-for-byte (a pretty-printed mock body survives, which a compact re-encode would not) on both the direct and coalesce paths.TestPassThrough_EmptyResponseIs500: an empty provider reply → 500 on both paths.go test -race ./...,go vet,golangci-lintall clean.Not included
The symmetric request-side pass-through (forwarding the raw request body instead of re-serializing) — smaller/situational payoff since requests are usually tiny; can follow up.