Skip to content

perf: incremental parent-map + esprima2 6.0.0 optional-chain support - #14

Merged
itamarga merged 7 commits into
mainfrom
perf/incremental-parent-map
Jul 23, 2026
Merged

perf: incremental parent-map + esprima2 6.0.0 optional-chain support#14
itamarga merged 7 commits into
mainfrom
perf/incremental-parent-map

Conversation

@itamarga

@itamarga itamarga commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

TLDR

Two changes: (1) a quadratic parent-map rebuild that made pyjsclear.deobfuscate() run for 15+ minutes on large obfuscator.io-style samples is now an O(1) incremental update; (2) full support for esprima2 6.0.0, which wraps optional chains in an ESTree ChainExpression node.

Changes

  • base.py: new record_replacement(...) — an O(1) patch of the cached parent map instead of a full invalidation. ObjectSimplifier, ClassStaticResolver, StringRevealer now call it instead of invalidate_parent_map().
  • generator.py: handle ChainExpression (emit the wrapped optional chain) and parenthesize ?? when adjacent to ||/&&.
  • ast_helpers.py: register ChainExpression's child key so traversal descends into it and identifiers inside optional chains still resolve.
  • Depend on esprima2==6.0.0 (was capped below 5.0.2) and bump to 0.1.6.
  • New/updated tests for the incremental parent map, ChainExpression traversal, and ??/||/&& parenthesization; snapshot regenerated (the old one encoded the pre-fix nullish-mixing bug).

Full description

Performance: incremental parent map

Root cause. Every node replacement in ObjectSimplifier, ClassStaticResolver, and StringRevealer called invalidate_parent_map(), so the next find_parent rebuilt the entire parent map with a full-AST walk. With thousands of proxy-object references that is O(n²). Profiling 0.1.5 on a 606KB sample shows build_parent_map called 12,023 times, consuming 980.8s of the 1011.9s total (97%) — 1.76 billion dict.get calls.

Evidence (same machine, both versions under cProfile):

Sample 0.1.5 this branch (cProfile) this branch (no profiler)
606KB obfuscator.io-style JS 1011.9s 10.4s 3.1s
753KB obfuscator.io-style JS 1134.9s 27.5s 8.3s

Fix. record_replacement(replacement, parent, key, index) patches the cached parent map in O(1). This is safe because the replacement swaps a node in place: in-place swaps don't shift list indices, and replaced subtrees are detached and never looked up again, so only the replacement's own entry changes. The incremental map is behaviourally identical to a full rebuild. The parent-map test asserts build_parent_map is called exactly once across many in-place replacements — a deterministic check rather than a timing threshold.

esprima2 6.0.0 support

esprima2 5.0.2+ changed optional-chaining parsing to the ESTree ChainExpression wrapper (the ?. flag lives on the inner MemberExpression/CallExpression), and now correctly rejects ?? combined with ||/&& without parentheses (older versions leniently accepted it). Three fixes make the latest version work:

  • Generator ChainExpression handler — emits the wrapped expression, so optional chains round-trip instead of rendering as /* unknown: ChainExpression */.
  • Traversal child keyChainExpression was not in the static child-key map, and the generic fallback deliberately skips the expression key for non-ExpressionStatement nodes, so transforms never descended into a chain and left identifiers there obfuscated. Registering its child key fixes this.
  • Nullish-mixing parentheses?? shares a precedence level with ||, so the precedence check alone never parenthesized (a ?? b) || c, producing invalid JS. The generator now adds parentheses whenever ?? is adjacent to ||/&&.

The snapshot changed on exactly two lines: the old output mis-grouped a nullish/logical expression (a ?? (true || …)) — the regenerated snapshot renders the true AST correctly ((a ?? true) || …).

Tests. Full unit + regression suite green: 1759 passed, 3 skipped (fuzz runs separately in CI).

🤖 Generated with Claude Code

itamarga and others added 6 commits July 12, 2026 23:31
…KT-16478]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prepare-pr-next polish: condense the repeated coarse-budget clauses in the
deobfuscate/deobfuscate_file docstrings. No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
esprima2 5.0.2 changed optional-chaining (a?.b) parsing to the ESTree
ChainExpression wrapper node. The code generator has no ChainExpression
handler (emits "/* unknown: ChainExpression */") and the deobfuscation
transforms don't see through the wrapper, so on a fresh install (which
floated the unpinned dependency up to 5.0.2/6.0.0) the suite fails on
optional-chaining round-trips, invalid ?? / || output, and leftover _0x
identifiers. The last green build (March) resolved to 5.0.1.

Cap the dependency at the tested range to restore a green build; full
esprima2 >=5.0.2 support (generator + transforms + snapshot) is a
separate follow-up.

Ref: TKT-16478

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tighten the docstrings added for the incremental parent-map maintenance and
the optional deobfuscate() time budget, removing redundancy while keeping the
behavioural caveats. Replace the wall-clock parent-map regression test with
deterministic build_parent_map call-count assertions (immune to machine
variance) and drop a budget test that duplicated existing hex-escape coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the time_budget_seconds parameter from deobfuscate(), deobfuscate_file(),
and Deobfuscator, along with its test. Time limiting is enforced externally by
the caller, so an in-library budget is redundant. The incremental parent-map
maintenance (the main performance fix) is unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@itamarga itamarga changed the title perf: incremental parent-map maintenance perf: incremental parent-map + esprima2 6.0.0 optional-chain support Jul 23, 2026
@itamarga
itamarga force-pushed the perf/incremental-parent-map branch 2 times, most recently from 5c20722 to ed60995 Compare July 23, 2026 08:28
esprima2 5.0.2+ wraps optional chains (a?.b) in an ESTree ChainExpression
node and rejects ?? mixed with ||/&& without parentheses. Add a generator
handler and traverser child-key entry so chains round-trip and identifiers
inside them still resolve, and parenthesize ?? next to ||/&&. Pin
esprima2==6.0.0 and bump version to 0.1.6.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@itamarga
itamarga force-pushed the perf/incremental-parent-map branch from 5fa52a7 to 046b284 Compare July 23, 2026 09:07
@itamarga
itamarga merged commit b8f4328 into main Jul 23, 2026
11 checks passed
@itamarga
itamarga deleted the perf/incremental-parent-map branch July 23, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants