Skip to content

fix(core): walk the loader-chain AST iteratively so deep input keeps full dataflow analysis - #92

Merged
tonibergholm-codento merged 6 commits into
mainfrom
fix/88-iterative-walk
Nov 20, 2026
Merged

tonibergholm-codento merged 6 commits into
mainfrom
fix/88-iterative-walk

Conversation

@tonibergholm-codento

Copy link
Copy Markdown
Contributor

A loader file padded with a very deep expression (for example 60,000 chained member accesses inside a try {} catch {}) overflowed acorn-walk's recursive walk. The rule then fell back to name matching, which a renamed or helper-routed chain could slip past (#88). The loader-chain analysis now walks the AST iteratively, so deep input costs time, not a stack overflow, and the full dataflow analysis runs.

  • walkAncestor replaces walk.ancestor. It uses acorn-walk's own base visitors with an explicit stack and fires callbacks in the same order with the same ancestor arrays. A parity test compares full visit traces against walk.ancestor on acorn's and acorn-walk's own sources and on a sample covering 31 node types.
  • Wide child lists (huge program bodies and arrays) are walked by index, so peak memory stays close to acorn-walk's.
  • Past 1,000 nested function scopes the walk keeps going. Deeper scopes feed a union-only overlay that can only add taint, never hide or kill a binding, so a correlation is never lost.
  • A 4,194,304-node cap remains as a safety bound. It is unreachable for retained files (at most 2 MiB), and hitting it takes the strict fallback.
  • Acorn's own parser can still run out of stack on extreme nesting. That now counts as an overflow and uses the loose fallback, which also recognises atob; genuine syntax errors keep the strict fallback.

AUDIT_REVISION goes to 6. The existing overflow-fallback tests now reach the fallback through parser overflow (100,000 nested parens).

Declined in review: the overlay can over-report in contrived code nested more than 1,000 functions deep (a mark made in a merged scope outlives a later reassignment). That depth is itself an evasion signal, and over-reporting there is the intended trade.

Proof: 18 new tests in packages/core/test/security-iterative-walk.test.ts, failing before their fixes (parity, scope-bound and overlay cases included). Peak RSS on a flat 1.9M-statement file is 287 MB, against 275 MB for the recursive walk. npm test passes (1532 tests, 0 failures, 4 Linux-only skips). Reviewed with sol-loop: 5 rounds with gpt-6.2-sol; every finding fixed except the declined over-reporting one above.

Closes #88.

Created with Claude Opus 5.5 in Claude Code (T3 Code). A Sonnet 5.5 subagent wrote most of the code.

A file padded with a very deep expression (tens of thousands of chained
members, inside `try { ... } catch {}`) overflowed acorn-walk's recursive
walk. That dropped the dataflow analysis to a regex fallback that cannot
tell a correlated chain from four unrelated mentions: a chain in a form
the fallback does not recognise fell below critical, and a benign
overflowing file that mentioned all four names became critical.

`walkAncestor` replaces `walk.ancestor`. It runs acorn-walk's own base
visitors, collects each node's children, and walks them from an explicit
stack, so it visits the same nodes in the same order (a node's visitor
runs after its children) with the same ancestor list. The scope chain is
kept incrementally and handed to the visitor, so a deep expression does
not make every callback O(depth). The walk is bounded by 2,000,000 nodes
and 1,000 nested scopes; exceeding either is a RangeError, which keeps
mapping to the existing `walkOverflow` path. A huge path argument is no
longer re-sliced whole by every write (capped like snippets).

A deep expression now costs time, not a stack overflow, and the full
analysis runs. The regex fallback remains for genuine parse failures
(including acorn's own "Not enough stack space" SyntaxError on deeply
nested parens, arrays and binary chains) and for the node bound.

Existing overflow-fallback tests padded with `x.b.b...` now pad with
2,000,001 statements, which still reaches the node bound. Bump
AUDIT_REVISION to 6: rule semantics changed.

Closes #88
… error

acorn's parser raises SyntaxError("Not enough stack space to parse
input") on deep nesting (about 700 parens). That took the strict
parse-failure fallback, so a correlated chain padded with nested parens
capped at high. That error, like a RangeError, now maps to `walkOverflow`
so the loose fallback applies. Genuine syntax errors keep the strict
fallback.

Refs #88
…e node cap

- If the module parse exhausts the stack and the script retry then fails
  on something else (`export`), the result still counts as a stack
  overflow: both parse errors are checked.
- Hitting the node or scope bound is a distinct `walkTruncated` outcome
  that uses the strict fallback like a parse failure, never the loose
  overflow one. The node bound is 4,194,304: a retained file is at most
  2 MiB and yields about one node per byte, so real input cannot reach it.
- The walk enumerates children incrementally (an index into the node's
  child list per frame, and the `body` list itself for Program and block
  bodies) instead of copying every child up front. A flat 1.9M-statement
  file walks with about 21 MB of extra peak RSS instead of about 275 MB.
  The visit order is unchanged (the parity test still passes).
- `atob(` is a decode signal in the loose overflow fallback, so a padded
  `atob` chain stays critical.
- The overflow-fallback tests pad with 100,000 nested parens, which
  overflows acorn's parser on any supported Node, instead of a node-bound
  pad.

Refs #88
- Past MAX_SCOPE_DEPTH, deeper functions join the deepest tracked scope
  instead of throwing, so the full dataflow analysis still runs on a file
  nested 1,001 functions deep (over-approximation: taint is never lost).
  Only the node bound throws a WalkBoundError.
- Array elements (ArrayExpression and ArrayPattern) are iterated by index
  from the node's own list, skipping holes, in acorn-walk's order, so a
  wide array no longer buffers every child in two temporary lists. On a
  2 MB benign array the walk's peak RSS is 228 MB against acorn-walk's
  205 MB and 286 MB before.
- The node-bound test now also asserts the exact strict-fallback result
  ("high" for strict read and write signals), and the walk-order parity
  sample covers array literals, holes and array patterns.

Refs #88
Functions nested past MAX_SCOPE_DEPTH join the deepest tracked scope. A
same-named inner local declared there replaced the enclosing binding's
taint (kill-on-reassign), so the enclosing launch lost its
`written-path` correlation and a critical chain dropped to high.

The walker now tells the visitor when a node sits inside a merged
function, and there a declaration, initialization or reassignment only
adds taint to the existing entry. Merged scopes are "may" scopes: the
analysis over-approximates and cannot lose a correlation a correct
analysis would report. Scopes within the bound keep exact kill-on-
reassign semantics, and the scope chain stays bounded.

Refs #88
A declaration inside a function nested past the scope bound wrote an
empty taint entry into the deepest tracked scope. That entry then hid a
binding of the same name in a farther-out tracked scope and stayed
visible after the merged function exited, so a launch of the outer
`written-path` variable lost its correlation.

Merged declarations, assignments and written-path marks now go to a
separate union-only `mergedTaint` overlay (name to tags) and never touch
the tracked scopes. A lookup returns the lexical resolution unioned with
the overlay's tags for that name, so the overlay only adds taint, never
hides a binding, and may outlive the merged function. Exact scopes keep
their kill-on-reassign semantics unchanged.

Refs #88
@coderabbitai

coderabbitai Bot commented Nov 20, 2026

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 65ad775a-eea5-4ad4-a25c-f96ebe41d790
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-20T13:36:51.653770Z e3187e0 PR opened
🔒 Security Review ✅ Completed 2026-11-20T13:43:37.222001Z e3187e0 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

Sentinel dependency audit — WARN · ✓ ok

145 allow · 5 warn · 0 block · 0 error
provenance: 46 verified · 0 invalid · 104 absent · 0 unknown

package verdict score finding how to fix
esbuild@​0.28.1 warn 62/100 `install.js` combines launch + write + read primitives — partial materialization pattern. Review the finding details; approve with a recorded rationale only if you understand and accept the risk.
fast-uri@​3.1.9 warn 75/100 Network egress: connects to a hardcoded IP address. Confirm the egress is expected for this package's purpose; if not, remove it or pin to a version without it.
isexe@​2.0.0 warn 77/100 `test/basic.js` combines read + launch + write primitives — partial materialization pattern. Review the finding details; approve with a recorded rationale only if you understand and accept the risk.
tsx@​4.22.4 warn 76/100 `dist/index-D9F1FXzN.cjs` combines read + write primitives — partial materialization pattern. Review the finding details; approve with a recorded rationale only if you understand and accept the risk.
undici-types@​7.18.2 warn 64/100 Network egress: imports a raw networking module. Confirm the egress is expected for this package's purpose; if not, remove it or pin to a version without it.

▶ Run sentinel explain <package> <version> for a suggested safe version and a ready waiver.

Sentinel · 150 packages audited · 2026-11-20T13:35:58.429Z · SBOM uploaded as a build artifact

@tonibergholm-codento
tonibergholm-codento merged commit 10412f3 into main Nov 20, 2026
7 checks passed
@tonibergholm-codento
tonibergholm-codento deleted the fix/88-iterative-walk branch November 20, 2026 13:46
tonibergholm-codento added a commit that referenced this pull request Nov 20, 2026
Bump all seven workspaces, the exact internal pins, ENGINE_VERSION and the
CLI/MCP versions to 0.1.0-alpha.3, and add the release notes. The release
carries the security audit hardening (#84) and the install-script
detection-gap fixes (#89, #90, #91, #92).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Detection gap: loader-chain walker overflow drops dataflow correlation to a regex fallback

1 participant