Skip to content

Fix response body decoding and header framing desync in CDP capture - #49

Closed
m4dni5 wants to merge 1 commit into
mandatoryprogrammer:mainfrom
m4dni5:pr-framing
Closed

m4dni5 wants to merge 1 commit into
mandatoryprogrammer:mainfrom
m4dni5:pr-framing

Conversation

@m4dni5

@m4dni5 m4dni5 commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Responses captured through Fetch interception reach the client with a broken body/length/encoding triple. Clients that reuse connections — browsers, chained proxies — see truncated scripts, cancelled stylesheets, and pages that hang. Single-request clients like curl hide the damage, so these bugs are easy to miss.

Root causes

Three independent bugs in the capture handlers in cdp.js:

1. Bodies decoded with the wrong encoding. Non-base64 Fetch.getResponseBody bodies are latin1-encoded text, but the code decodes them as utf8. Bytes >= 0x80 get mangled, changing the body length.

2. Headers describe a different body than the one sent. Fetch.getResponseBody returns the decompressed body, but the captured headers still carry the upstream compressed-size content-length and content-encoding. The body is therefore longer than the declared length, and the surplus bytes get parsed as the start of the next response on that connection, corrupting everything after it.

3. Optional CORS headers crash the preflight mock. The mocked preflight copies Access-Control-Request-Method/Headers from the request. Both are optional per the CORS spec; when absent, the emitted headers have value: undefined, and Chrome's CDP bindings reject the entire Fetch.fulfillRequest call, killing the fetch.

Fix

  • Decode non-base64 bodies as latin1 (both capture handlers).
  • Recompute content-length from the actual body and strip content-encoding.
  • Default the preflight values; only emit Access-Control-Allow-Headers when the request asked for it.

Verification

  • A large JS asset fetched direct vs. through the fixed proxy: byte-identical.
  • content-length matches the delivered body on responses checked.
  • A brotli-heavy, non-ASCII-heavy page behind an aggressive WAF renders fully through a chained intercepting proxy with no truncation or CDP errors in the logs.

Three related fixes in the Fetch interception response handlers:

1. Decode non-base64 getResponseBody bodies as latin1 (CDP convention)
   instead of utf8 — utf8 mangles bytes >= 0x80 (CJK content etc.) and
   breaks content-length parity.

2. Default CORS preflight Origin/Method/Headers values before building
   the fulfillRequest response. These request headers are optional per
   the CORS spec; when absent, undefined header values make Chrome's
   CDP bindings reject the whole fulfillRequest call.

3. Recompute content-length from the decoded buffer and strip
   content-encoding on captured responses. CDP returns the decompressed
   body while the captured headers carry the upstream compressed-size
   content-length; passing both through verbatim desyncs HTTP framing —
   the surplus body bytes get parsed as the start of the next response
   on reused keep-alive connections, corrupting browsers and chained
   proxies downstream.
@m4dni5
m4dni5 marked this pull request as ready for review September 9, 2026 23:53
@mandatoryprogrammer

Copy link
Copy Markdown
Owner

Thank you for the report! Merged a fix for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants