Skip to content

perf: incremental parent-map maintenance - #13

Closed
itamarga wants to merge 5 commits into
mainfrom
fix/TKT-16478-incremental-parent-map
Closed

perf: incremental parent-map maintenance#13
itamarga wants to merge 5 commits into
mainfrom
fix/TKT-16478-incremental-parent-map

Conversation

@itamarga

@itamarga itamarga commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

TLDR

Fixes a quadratic parent-map rebuild that made pyjsclear.deobfuscate() run for 15+ minutes on large obfuscator.io-style samples. On two representative samples, 0.1.5 takes 1011.9s / 1134.9s; this branch takes 10.4s / 27.5s under the same profiler (3.1s / 8.3s without).

Changes

  • base.py: new record_replacement(replacement, parent, key, index) — an O(1) patch of the cached parent map instead of a full invalidation.
  • ObjectSimplifier, ClassStaticResolver, StringRevealer: node replacement now calls record_replacement(...) instead of invalidate_parent_map(), eliminating the full O(N) parent-map rebuild on every subsequent find_parent.
  • Pin esprima2>=…,<5.0.2: 5.0.2 switched optional-chaining (a?.b) parsing to the ESTree ChainExpression wrapper, which the generator and transforms don't yet support (full support tracked separately).
  • Version bumped to 0.1.6.
  • New/updated tests for the incremental parent map.

Full description

Root cause. Every node replacement in ObjectSimplifier, ClassStaticResolver, and StringRevealer called invalidate_parent_map(), so the next find_parent rebuilt the entire parent map with a full-AST walk. With thousands of proxy-object references that is O(n²). Profiling 0.1.5 on the 606KB sample shows build_parent_map called 12,023 times, consuming 980.8s of the 1011.9s total (97%) — 1.76 billion dict.get calls — all triggered by node replacement invalidating the parent map on every swap.

Evidence (same machine, both versions under cProfile):

Sample pyjsclear 0.1.5 this branch (cProfile) this branch (no profiler)
606KB obfuscator.io-style JS 1011.9s 10.4s 3.1s
753KB obfuscator.io-style JS 1134.9s 27.5s 8.3s

Fix. record_replacement(replacement, parent, key, index) patches the cached parent map in O(1). This is safe because the replacement swaps a node in place: in-place swaps don't shift list indices, and replaced subtrees are detached and never looked up again, so only the replacement's own entry changes. The incremental map is behaviourally identical to a full rebuild.

esprima2 pin. esprima2 5.0.2 changed optional-chaining parsing from a bare MemberExpression (with optional: true) to the ESTree-spec ChainExpression wrapper. The code generator has no ChainExpression handler and the transforms don't see through the wrapper, so a fresh install that floats the dependency up to 5.0.2+ breaks optional-chaining round-trips and leaves obfuscated identifiers unresolved. Capping below 5.0.2 restores a green build; full ChainExpression support (generator + transforms) is tracked as a separate follow-up.

Tests. Full suite green: 1754 passed, 3 skipped (unit + regression, fuzz excluded). The parent-map test asserts build_parent_map is called exactly once across many in-place replacements — a deterministic check of the fix rather than a timing threshold.

🤖 Generated with Claude Code

itamarga and others added 5 commits July 12, 2026 23:31
…KT-16478]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prepare-pr-next polish: condense the repeated coarse-budget clauses in the
deobfuscate/deobfuscate_file docstrings. No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
esprima2 5.0.2 changed optional-chaining (a?.b) parsing to the ESTree
ChainExpression wrapper node. The code generator has no ChainExpression
handler (emits "/* unknown: ChainExpression */") and the deobfuscation
transforms don't see through the wrapper, so on a fresh install (which
floated the unpinned dependency up to 5.0.2/6.0.0) the suite fails on
optional-chaining round-trips, invalid ?? / || output, and leftover _0x
identifiers. The last green build (March) resolved to 5.0.1.

Cap the dependency at the tested range to restore a green build; full
esprima2 >=5.0.2 support (generator + transforms + snapshot) is a
separate follow-up.

Ref: TKT-16478

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tighten the docstrings added for the incremental parent-map maintenance and
the optional deobfuscate() time budget, removing redundancy while keeping the
behavioural caveats. Replace the wall-clock parent-map regression test with
deterministic build_parent_map call-count assertions (immune to machine
variance) and drop a budget test that duplicated existing hex-escape coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@itamarga itamarga changed the title perf: incremental parent-map + optional wall-clock budget [TKT-16478] perf: incremental parent-map maintenance + optional wall-clock budget Jul 23, 2026
@itamarga itamarga closed this Jul 23, 2026
@itamarga itamarga changed the title perf: incremental parent-map maintenance + optional wall-clock budget perf: incremental parent-map maintenance Jul 23, 2026
@itamarga

Copy link
Copy Markdown
Collaborator Author

Superseded by #14 — reopened from a renamed branch (perf/incremental-parent-map). Same commits, no content change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant