Skip to content

Batch RAR 2.0 history compaction - #103

Merged
bitplane merged 1 commit into
masterfrom
perf/rar20-history-compaction
Oct 25, 2026
Merged

bitplane merged 1 commit into
masterfrom
perf/rar20-history-compaction

Conversation

@bitplane

Copy link
Copy Markdown
Owner

RAR 2.0 moved its retained 1 MiB window after every solid member, even when the member added only a few bytes. Batch those moves across small members for unlimited readers, trimming once the obsolete prefix reaches 64 KiB. Compact earlier if the next member would otherwise exhaust existing buffer capacity.

Unlimited checkpoints copy only the required window and adjust their absolute base position, so deferred bytes do not increase checkpoint allocations or affect restored buffer growth. Quota-limited readers retain the original eager trimming, clone allocation order, and failure prefixes. No ring buffer or additional decoder state is introduced.

Tests compare against the original eager policy across full-distance and overlapping matches, pending matches, irregular sizes, exact-fit capacity boundaries, checkpoint restoration, allocation refusals, sink failures, and cancellation. A 512-member test reduces window moves from 512 to eight while preserving bytes and backing capacity.

The benchmark now includes fresh-seeded solid sequences, owned and streaming output, and checkpoint/restore controls. Run it with cargo bench -p rars --bench rar20_decode -- rar20_solid_history.

Three alternating baseline/final measurement runs on synthetic repeated-byte data showed:

  • 1 KiB members: roughly 16–23× faster; 4093-byte members: roughly 7–8× faster; 16 KiB members: roughly 2.6× faster.
  • Checkpoint creation/restoration workloads: roughly 1.5×/1.2× faster.
  • 64 KiB-plus and 1 MiB controls, and separate single-member controls, remain near baseline.
  • Recorded live, peak, and checkpoint requested heap allocation sizes match baseline in every measured history case. The decoder may retain less than 64 KiB of extra initialized bytes inside existing spare capacity after successful calls; allocation accounting is not an RSS measurement. These are codec microbenchmarks, not general archive-extraction speedups.

Validation:

  • Release package tests, including integration and doc tests: 2,617 passed; 17 existing ignored. Minimal-feature library tests: 357 passed.
  • Formatting and all-target Clippy with warnings denied, with default and minimal features.
  • The committed solid-history Criterion benchmark completed successfully.
  • Independent claude -p plan, checkpoint-refinement, and final reviews: approved, no blocking findings.

@bitplane
bitplane merged commit 0b680f2 into master Oct 25, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant