Skip to content

Bound JSON nesting wherever untrusted JSON is parsed (#2144) - #2146

Open
profinch wants to merge 2 commits into
BeamMW:masterfrom
profinch:fix/2144-json-depth-all
Open

profinch wants to merge 2 commits into
BeamMW:masterfrom
profinch:fix/2144-json-depth-all

Conversation

@profinch

@profinch profinch commented Oct 6, 2026 •

Copy link
Copy Markdown

Fixes #2144. Fixes #2147. An alternative to #2145, which covers only the wallet API. The comparison is below.

The bug is wider than the HTTP API

nlohmann::json 3.1.2 parses recursively with no depth limit, at ~190 bytes of stack per level in a release build. The parser recurses on every opening bracket before it ever sees a closing one, so [[[[... without the closing brackets is enough. A 64 KB line holds ~65 000 levels, while the 8 MB main thread fits ~43 000. Before vs after, each with one unauthenticated request:

Entry point master this PR
wallet-api --use_http=1, 80 000 × [ + ] dies (exit 132) -32600, keeps serving
wallet-api TCP (the default mode), 65 000 × [ in one line dies (exit 132) -32600
beam-node --stratum_port, 65 000 × [ in one line, no auth needed dies (exit 139) alive
SBBS instant message from anyone who knows your address, then read_messages dies (exit 139) message returned as a string

Fix

utility/json_depth.h:

  • FindJsonDepthExcess() is one O(n) pass with no allocations. It counts brackets outside of strings the way the lexer sees them, which bounds the depth json::parse can reach.
  • ParseUntrustedJson() rejects nesting deeper than MaxJsonDepth = 128 with a regular json::parse_error. Every existing catch already handles that: the API maps it to InvalidJsonRpc, the bridges to an invalid reply, stratum to message_corrupted. 128 is far beyond real documents and safe even on 512 KB secondary-thread stacks (measured: 2 790 levels fit).

Used for everything that parses JSON from outside:

  • wallet/api/base/api_base.cpp: requests over HTTP, TCP, websockets and the apps API, plus fromError.
  • wallet/api/v7_4, read_messages: instant messages are stored exactly as received from any sender. A message that isn't acceptable JSON (too deep, or not JSON at all) is now returned as a string. Before, a non-JSON message made read_messages fail, and since readIMs marks new messages as read before the response is built, the whole batch was lost. With all: true the call failed every time.
  • pow/stratum.cpp: miner connections to beam-node.
  • The swap bridges (electrum.cpp, bitcoin_core_016.cpp, ethereum_bridge.cpp): replies from Electrum, bitcoind and Ethereum servers.
  • explorer/adapter.cpp: the contract rich parser output.

Local files (config, wallet DB import, wasm key keeper) are left as is.

Compared to #2145

Both use the same pre-scan idea and the same limit of 128. #2145 guards ApiBase only, which covers HTTP and TCP requests. It leaves the remote crash through SBBS messages, the beam-node stratum port and the bridges open.

Tests

  • wallet_api_test:
    • depth boundaries (128 passes, 129 is rejected at the right byte), brackets in strings, escaped quotes and backslashes, stray closing brackets;
    • an 80 000-level request and depth 129 inside params get InvalidJsonRpc;
    • read_messages through the real v7.4 API with messages stored via storeIM, like the wallet does on receipt: an 80 000-level message and a non-JSON one come back as strings, a normal one as an object. Against master's v7_4_api_parse.cpp, the same test crashes (exit 139), and with only the non-JSON message it fails with Invalid JSON-RPC.
  • The table above: binaries built from master and from this branch.
  • stratum_test, bitcoin_rpc_test, electrum_test, swap_board_test, adapter_test pass.

nlohmann::json 3.1.2 parses recursively with no depth limit, and the parser
recurses on every opening bracket, so a single line of [[[[... crashes the
process. utility/json_depth.h pre-scans the input (O(n), no allocations) and
rejects nesting deeper than 128 with a regular json::parse_error.

Used for: wallet-api requests (HTTP and TCP), SBBS instant messages returned
by read_messages, the stratum server of beam-node, replies of the swap
bridges, the explorer's contract parser output. read_messages also returns a
message that isn't acceptable JSON as a string instead of failing.
- explorer: include json_depth.h unconditionally, it was under
  BEAM_ASSET_SWAP_SUPPORT while ParseSafe uses it in every build
- tests: 1 000 000 levels, 80 000 fit the 8 MB main thread and didn't crash
  without the fix

@profinch profinch left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: one build break, fixed in 09025a6. Everything else checks out.

Fixed

  • explorer/adapter.cpp included utility/json_depth.h inside #ifdef BEAM_ASSET_SWAP_SUPPORT, but ParseSafe calls FindJsonDepthExcess in every build. With -DBEAM_ASSET_SWAP_SUPPORT=OFF the file didn't compile (use of undeclared identifier 'FindJsonDepthExcess'). The include is now unconditional, and the file compiles with the option off.
  • The regression tests used 80 000 levels. That doesn't overflow the 8 MB main thread in wallet_api_test, so without the fix the tests failed on a wrong error code instead of crashing. They now use 1 000 000 levels, which crashes the unfixed parser for sure.

Checked

  • FindJsonDepthExcess matches the lexer on every prefix the parser accepts: strings, escapes and \\ are handled, and stray closing brackets can't make the depth negative. So it bounds the real recursion depth. One pass, no allocations.
  • Errors are regular parse_errors, so callGuarded maps them to -32600 as before.
  • read_messages: returning a message that isn't valid JSON as a string is a reasonable change. Before, one bad message failed the whole call after the messages were already marked as read.
  • Tests on this branch (RelWithDebInfo, macOS arm64): wallet_api_test, stratum_test, adapter_test, swap_board_test, wallet_test all pass.
  • End to end:
    • wallet-api HTTP mode, 80 000 and 400 000 levels: rejected with -32600, the API keeps serving.
    • wallet-api TCP mode, one 65 000-[ line: rejected with -32600, the API keeps serving.
    • beam-node stratum, 65 000 [ and 32 000 balanced levels: rejected, the node stays up and login still succeeds (#2147).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant