Skip to content

Add streaming archive readers with configurable limits - #51

Open
andrew wants to merge 1 commit into
mainfrom
streaming-reader
Open

andrew wants to merge 1 commit into
mainfrom
streaming-reader

Conversation

@andrew

@andrew andrew commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Opening a TAR archive currently retains every expanded file body before callers can inspect entries or enforce their own limits. Add OpenStream and OpenStreamBytes with sequential Next, Read, and Close operations across all supported formats.

Apply input, per-entry, expanded-byte, and entry-count limits before exposing bodies. TAR and gem consume forward-only input; ZIP and conda buffer compressed input or reuse the slice passed to OpenStreamBytes. Expanded bodies are processed one at a time.

Preserve paths, modes, duplicate entries, and TAR link metadata so consumers such as swhid-go can hash streamed content and build directory identities. Document input ownership, decoder memory overhead, and TAR trailer validation limits.

Copilot AI lite review requested due to automatic review settings September 27, 2026 05:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants