perf: incremental parent-map + esprima2 6.0.0 optional-chain support - #14
Merged
Conversation
…KT-16478] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prepare-pr-next polish: condense the repeated coarse-budget clauses in the deobfuscate/deobfuscate_file docstrings. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
esprima2 5.0.2 changed optional-chaining (a?.b) parsing to the ESTree ChainExpression wrapper node. The code generator has no ChainExpression handler (emits "/* unknown: ChainExpression */") and the deobfuscation transforms don't see through the wrapper, so on a fresh install (which floated the unpinned dependency up to 5.0.2/6.0.0) the suite fails on optional-chaining round-trips, invalid ?? / || output, and leftover _0x identifiers. The last green build (March) resolved to 5.0.1. Cap the dependency at the tested range to restore a green build; full esprima2 >=5.0.2 support (generator + transforms + snapshot) is a separate follow-up. Ref: TKT-16478 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tighten the docstrings added for the incremental parent-map maintenance and the optional deobfuscate() time budget, removing redundancy while keeping the behavioural caveats. Replace the wall-clock parent-map regression test with deterministic build_parent_map call-count assertions (immune to machine variance) and drop a budget test that duplicated existing hex-escape coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the time_budget_seconds parameter from deobfuscate(), deobfuscate_file(), and Deobfuscator, along with its test. Time limiting is enforced externally by the caller, so an in-library budget is redundant. The incremental parent-map maintenance (the main performance fix) is unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
itamarga
force-pushed
the
perf/incremental-parent-map
branch
2 times, most recently
from
July 23, 2026 08:28
5c20722 to
ed60995
Compare
esprima2 5.0.2+ wraps optional chains (a?.b) in an ESTree ChainExpression node and rejects ?? mixed with ||/&& without parentheses. Add a generator handler and traverser child-key entry so chains round-trip and identifiers inside them still resolve, and parenthesize ?? next to ||/&&. Pin esprima2==6.0.0 and bump version to 0.1.6. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
itamarga
force-pushed
the
perf/incremental-parent-map
branch
from
July 23, 2026 09:07
5fa52a7 to
046b284
Compare
davidt99
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Two changes: (1) a quadratic parent-map rebuild that made
pyjsclear.deobfuscate()run for 15+ minutes on large obfuscator.io-style samples is now an O(1) incremental update; (2) full support for esprima2 6.0.0, which wraps optional chains in an ESTreeChainExpressionnode.Changes
base.py: newrecord_replacement(...)— an O(1) patch of the cached parent map instead of a full invalidation.ObjectSimplifier,ClassStaticResolver,StringRevealernow call it instead ofinvalidate_parent_map().generator.py: handleChainExpression(emit the wrapped optional chain) and parenthesize??when adjacent to||/&&.ast_helpers.py: registerChainExpression's child key so traversal descends into it and identifiers inside optional chains still resolve.esprima2==6.0.0(was capped below 5.0.2) and bump to0.1.6.ChainExpressiontraversal, and??/||/&&parenthesization; snapshot regenerated (the old one encoded the pre-fix nullish-mixing bug).Full description
Performance: incremental parent map
Root cause. Every node replacement in
ObjectSimplifier,ClassStaticResolver, andStringRevealercalledinvalidate_parent_map(), so the nextfind_parentrebuilt the entire parent map with a full-AST walk. With thousands of proxy-object references that is O(n²). Profiling 0.1.5 on a 606KB sample showsbuild_parent_mapcalled 12,023 times, consuming 980.8s of the 1011.9s total (97%) — 1.76 billiondict.getcalls.Evidence (same machine, both versions under cProfile):
Fix.
record_replacement(replacement, parent, key, index)patches the cached parent map in O(1). This is safe because the replacement swaps a node in place: in-place swaps don't shift list indices, and replaced subtrees are detached and never looked up again, so only the replacement's own entry changes. The incremental map is behaviourally identical to a full rebuild. The parent-map test assertsbuild_parent_mapis called exactly once across many in-place replacements — a deterministic check rather than a timing threshold.esprima2 6.0.0 support
esprima2 5.0.2+ changed optional-chaining parsing to the ESTree
ChainExpressionwrapper (the?.flag lives on the innerMemberExpression/CallExpression), and now correctly rejects??combined with||/&&without parentheses (older versions leniently accepted it). Three fixes make the latest version work:ChainExpressionhandler — emits the wrapped expression, so optional chains round-trip instead of rendering as/* unknown: ChainExpression */.ChainExpressionwas not in the static child-key map, and the generic fallback deliberately skips theexpressionkey for non-ExpressionStatementnodes, so transforms never descended into a chain and left identifiers there obfuscated. Registering its child key fixes this.??shares a precedence level with||, so the precedence check alone never parenthesized(a ?? b) || c, producing invalid JS. The generator now adds parentheses whenever??is adjacent to||/&&.The snapshot changed on exactly two lines: the old output mis-grouped a nullish/logical expression (
a ?? (true || …)) — the regenerated snapshot renders the true AST correctly ((a ?? true) || …).Tests. Full unit + regression suite green: 1759 passed, 3 skipped (fuzz runs separately in CI).
🤖 Generated with Claude Code