Skip to content

Reuse legacy streaming output buffers - #100

Merged
bitplane merged 1 commit into
masterfrom
perf/legacy-streaming-buffer-reuse
Oct 25, 2026
Merged

bitplane merged 1 commit into
masterfrom
perf/legacy-streaming-buffer-reuse

Conversation

@bitplane

Copy link
Copy Markdown
Owner

The RAR 1.3/1.5 reader allocated a new output buffer for every roughly 64 KiB streaming chunk. Keep one buffer local to the member and clear it after each successful sink write, reusing its capacity for subsequent chunks.

Matches can cross a chunk boundary and grow that buffer. Unlimited readers retain the grown capacity. Bounded readers reuse only an exact capacity match and release the preceding buffer before allocating a differently sized one, preserving workspace requirements for overshoot and smaller final chunks.

Validation:

  • The allocation regression fails on the original loop and passes with reuse.
  • The old loop remains a test oracle for output boundaries and solid continuations under both policies. The match-heavy fixture needs exactly 262,328 workspace bytes before and after the change.
  • A callback fills the quota before the final chunk to verify that replacement releases the previous buffer first. Additional tests cover every allocation refusal, sink failure, cancellation, and input failure with an empty target.
  • Full release package tests: 2,601 passed, 17 existing ignored tests. Minimal-feature library tests: 344 passed. Formatting and Clippy (all targets, warnings denied) passed for both feature sets.
  • claude -p approved the plan and final implementation; its optional clarity suggestions were applied.

A temporary same-binary probe measured 23 → 8 allocation attempts for a 1 MiB literal-only fixture, including packed-input buffering and excluding decoder construction. The bounded match-heavy fixture remains at 32 attempts, since overshoot requires replacement to preserve the existing quota behavior. Timing differences were small (roughly 0–3%); this is an allocation reduction, not a demonstrated throughput improvement. The temporary probe was removed from the branch.

@bitplane
bitplane merged commit f0a7363 into master Oct 25, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant