Skip to content

fix(sdk): handle short reads when loading TDFs from custom readers - #4220

Closed
sujankota wants to merge 1 commit into
mainfrom
fix/zipstream-short-reads
Closed

sujankota wants to merge 1 commit into
mainfrom
fix/zipstream-short-reads

Conversation

@sujankota

@sujankota sujankota commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Proposed Changes

  • zipstream.readBytes issued a single Read and returned the full buffer even when fewer bytes came back. *os.File and bytes.Reader always fill the buffer, but other io.ReadSeekers passed to LoadTDF (network/blob-backed readers) may legally return short reads. The tail of the manifest or segment was left zeroed, surfacing as a confusing segment signature validation error.
  • Switch to io.ReadFull; running out of input early still returns io.EOF, so callers are unchanged.
  • Prerequisite for loading TDFs over HTTP range requests (DSPX-5112, part of DSPX-4886).

Checklist

  • I have added or updated unit tests
  • I have added or updated integration tests (if appropriate)
  • I have added or updated documentation

Testing Instructions

  • cd sdk && go test ./internal/zipstream -run TestReaderShortReads — reads a zip through a reader that returns one byte per Read. Fails before this change, passes after.
  • cd sdk && go test -race ./... passes.
  • golangci-lint run --new-from-rev=main in sdk: 0 new issues.

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability when reading archived files from sources that provide data in small chunks. Full file contents and requested slices are now read correctly, reducing the risk of incomplete or missing data.

zipstream.readBytes issued a single Read and returned the full buffer
even when fewer bytes came back. Files and bytes.Reader always fill the
buffer, but network-backed io.ReadSeekers passed to LoadTDF may return
short reads, leaving the tail of the manifest or segment zeroed and
failing with a segment signature validation error.

Use io.ReadFull and keep the existing io.EOF contract for callers.

Signed-off-by: sujan kota <sujankota@gmail.com>
@sujankota
sujankota requested review from a team as code owners October 14, 2026 16:46
@github-actions github-actions Bot added comp:sdk A software development kit, including library, for client applications and inter-service communicati size/xs labels Oct 14, 2026
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 0cfb05d8-e0fe-48d7-b6f2-95e8ad9a742b
📥 Commits

Reviewing files that changed from the base of the PR and between 5679545 and 44d9a29.

📒 Files selected for processing: 2
  • sdk/internal/zipstream/reader.go
  • sdk/internal/zipstream/reader_test.go

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

readBytes now retries short reads until it fills the requested buffer or reaches an error. A new test checks full-file and ranged ZIP data reads with a reader that returns one byte per call.

Changes: ZIP stream short-read handling

Layer / File(s) Summary
Short-read handling and coverage
sdk/internal/zipstream/reader.go, sdk/internal/zipstream/reader_test.go
readBytes uses io.ReadFull and returns io.EOF for EOF and unexpected EOF. A new test checks full-file and ranged reads with a one-byte-at-a-time reader.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: dmihalcik-virtru

Merge Risk: ⚪ Minimal · up to 44d9a

No merge-blocking issue is identified; the change is ready for normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: fixing short reads when loading TDFs from custom readers.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 2
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
⚔️ Resolve merge conflicts: 💡
  • Resolve merge conflict in branch fix/zipstream-short-reads
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit reads each byte in flight
Then checks the stream from left to right
Full-file digits, range in view
One-byte hops still carry through
The ZIP path reads its payload true

Comment @coderabbitai help to get the list of available commands.

@sujankota

Copy link
Copy Markdown
Contributor Author

Closing: superseded by #4087, which already landed the same io.ReadFull fix in zipstream.readBytes (with stricter io.ErrUnexpectedEOF semantics). #4221 is being rebased onto main without this commit.

@sujankota sujankota closed this Oct 14, 2026
@sujankota
sujankota deleted the fix/zipstream-short-reads branch October 14, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp:sdk A software development kit, including library, for client applications and inter-service communicati size/xs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants