Repository navigation
fix(core): walk the loader-chain AST iteratively so deep input keeps full dataflow analysis - #92
Conversation
A file padded with a very deep expression (tens of thousands of chained
members, inside `try { ... } catch {}`) overflowed acorn-walk's recursive
walk. That dropped the dataflow analysis to a regex fallback that cannot
tell a correlated chain from four unrelated mentions: a chain in a form
the fallback does not recognise fell below critical, and a benign
overflowing file that mentioned all four names became critical.
`walkAncestor` replaces `walk.ancestor`. It runs acorn-walk's own base
visitors, collects each node's children, and walks them from an explicit
stack, so it visits the same nodes in the same order (a node's visitor
runs after its children) with the same ancestor list. The scope chain is
kept incrementally and handed to the visitor, so a deep expression does
not make every callback O(depth). The walk is bounded by 2,000,000 nodes
and 1,000 nested scopes; exceeding either is a RangeError, which keeps
mapping to the existing `walkOverflow` path. A huge path argument is no
longer re-sliced whole by every write (capped like snippets).
A deep expression now costs time, not a stack overflow, and the full
analysis runs. The regex fallback remains for genuine parse failures
(including acorn's own "Not enough stack space" SyntaxError on deeply
nested parens, arrays and binary chains) and for the node bound.
Existing overflow-fallback tests padded with `x.b.b...` now pad with
2,000,001 statements, which still reaches the node bound. Bump
AUDIT_REVISION to 6: rule semantics changed.
Closes #88
… error
acorn's parser raises SyntaxError("Not enough stack space to parse
input") on deep nesting (about 700 parens). That took the strict
parse-failure fallback, so a correlated chain padded with nested parens
capped at high. That error, like a RangeError, now maps to `walkOverflow`
so the loose fallback applies. Genuine syntax errors keep the strict
fallback.
Refs #88
…e node cap - If the module parse exhausts the stack and the script retry then fails on something else (`export`), the result still counts as a stack overflow: both parse errors are checked. - Hitting the node or scope bound is a distinct `walkTruncated` outcome that uses the strict fallback like a parse failure, never the loose overflow one. The node bound is 4,194,304: a retained file is at most 2 MiB and yields about one node per byte, so real input cannot reach it. - The walk enumerates children incrementally (an index into the node's child list per frame, and the `body` list itself for Program and block bodies) instead of copying every child up front. A flat 1.9M-statement file walks with about 21 MB of extra peak RSS instead of about 275 MB. The visit order is unchanged (the parity test still passes). - `atob(` is a decode signal in the loose overflow fallback, so a padded `atob` chain stays critical. - The overflow-fallback tests pad with 100,000 nested parens, which overflows acorn's parser on any supported Node, instead of a node-bound pad. Refs #88
- Past MAX_SCOPE_DEPTH, deeper functions join the deepest tracked scope
instead of throwing, so the full dataflow analysis still runs on a file
nested 1,001 functions deep (over-approximation: taint is never lost).
Only the node bound throws a WalkBoundError.
- Array elements (ArrayExpression and ArrayPattern) are iterated by index
from the node's own list, skipping holes, in acorn-walk's order, so a
wide array no longer buffers every child in two temporary lists. On a
2 MB benign array the walk's peak RSS is 228 MB against acorn-walk's
205 MB and 286 MB before.
- The node-bound test now also asserts the exact strict-fallback result
("high" for strict read and write signals), and the walk-order parity
sample covers array literals, holes and array patterns.
Refs #88
Functions nested past MAX_SCOPE_DEPTH join the deepest tracked scope. A same-named inner local declared there replaced the enclosing binding's taint (kill-on-reassign), so the enclosing launch lost its `written-path` correlation and a critical chain dropped to high. The walker now tells the visitor when a node sits inside a merged function, and there a declaration, initialization or reassignment only adds taint to the existing entry. Merged scopes are "may" scopes: the analysis over-approximates and cannot lose a correlation a correct analysis would report. Scopes within the bound keep exact kill-on- reassign semantics, and the scope chain stays bounded. Refs #88
A declaration inside a function nested past the scope bound wrote an empty taint entry into the deepest tracked scope. That entry then hid a binding of the same name in a farther-out tracked scope and stayed visible after the merged function exited, so a launch of the outer `written-path` variable lost its correlation. Merged declarations, assignments and written-path marks now go to a separate union-only `mergedTaint` overlay (name to tags) and never touch the tracked scopes. A lookup returns the lexical resolution unioned with the overlay's tags for that name, so the overlay only adds taint, never hides a binding, and may outlive the merged function. Exact scopes keep their kill-on-reassign semantics unchanged. Refs #88
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Sentinel dependency audit — WARN · ✓ ok145 allow · 5 warn · 0 block · 0 error
▶ Run Sentinel · 150 packages audited · 2026-11-20T13:35:58.429Z · SBOM uploaded as a build artifact |
A loader file padded with a very deep expression (for example 60,000 chained member accesses inside a
try {} catch {}) overflowed acorn-walk's recursive walk. The rule then fell back to name matching, which a renamed or helper-routed chain could slip past (#88). The loader-chain analysis now walks the AST iteratively, so deep input costs time, not a stack overflow, and the full dataflow analysis runs.walkAncestorreplaceswalk.ancestor. It uses acorn-walk's ownbasevisitors with an explicit stack and fires callbacks in the same order with the same ancestor arrays. A parity test compares full visit traces againstwalk.ancestoron acorn's and acorn-walk's own sources and on a sample covering 31 node types.atob; genuine syntax errors keep the strict fallback.AUDIT_REVISIONgoes to 6. The existing overflow-fallback tests now reach the fallback through parser overflow (100,000 nested parens).Declined in review: the overlay can over-report in contrived code nested more than 1,000 functions deep (a mark made in a merged scope outlives a later reassignment). That depth is itself an evasion signal, and over-reporting there is the intended trade.
Proof: 18 new tests in
packages/core/test/security-iterative-walk.test.ts, failing before their fixes (parity, scope-bound and overlay cases included). Peak RSS on a flat 1.9M-statement file is 287 MB, against 275 MB for the recursive walk.npm testpasses (1532 tests, 0 failures, 4 Linux-only skips). Reviewed with sol-loop: 5 rounds with gpt-6.2-sol; every finding fixed except the declined over-reporting one above.Closes #88.
Created with Claude Opus 5.5 in Claude Code (T3 Code). A Sonnet 5.5 subagent wrote most of the code.