Skip to content

RAR5: avoid copying decoder state before independent streams - #99

Merged
bitplane merged 1 commit into
masterfrom
perf/rar5-fresh-streaming-state
Oct 25, 2026
Merged

bitplane merged 1 commit into
masterfrom
perf/rar5-fresh-streaming-state

Conversation

@bitplane

Copy link
Copy Markdown
Owner

Streaming a non-solid RAR5 member used to copy the previous dictionary and Huffman tables, then immediately reset them. Prepare an independent decoder instead, for both ordinary and split members. Keep the previous decoder until output and integrity checks succeed, preserving rollback and cancellation.

Readers with a workspace limit now prepare without allocating or charging a duplicate history; some operations that previously exceeded the limit can therefore succeed. Unlimited readers reserve an empty buffer capped by the previous capacity and the next member's output size, avoiding repeated growth without copying discarded bytes. Solid members retain their existing checkpoint behavior.

The implementation adds no public API or persistent decoder fields. Smaller dictionaries still use the existing streaming capacity adjustment; this change does not claim to remove that temporary reservation.

Validation:

  • A regression failed on the original code (20 allocation operations after seeded history versus 18 from an empty decoder) and passes with this change. It covers ordinary and real multi-volume input.
  • Tests cover exhausted budgets, shared accounting, cancellation, populated Huffman tables, zero/small/oversized capacity hints, and CRC/hash failure rollback followed by a history-dependent solid member.
  • Full release package tests: 2,596 passed, 17 existing ignored tests. Minimal-feature library tests: 344 passed. Formatting and Clippy (all targets, warnings denied) passed for both feature sets.
  • claude -p reviewed the plan, capacity refinement, and final implementation; no outstanding findings.

A temporary same-binary codec probe compared old clone/reset with the final preparation code, alternating order and taking seven-sample medians. Across five runs, 16 independent 2 MiB members took 20–35% less time; with a 16 MiB predecessor, they took 45–52% less time. These are forced-streaming microbenchmarks, not end-to-end or default-settings performance claims. The probe was removed from the branch.

@bitplane
bitplane merged commit 22b9eca into master Oct 25, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant