Repository navigation
fix(sdk): fail fast when a TDF manifest would exceed the read-side size cap (DSPX-4651) - #1019
Draft
dmihalcik-virtru wants to merge 1 commit into
Draft
dmihalcik-virtru wants to merge 1 commit into
dmihalcik-virtru wants to merge 1 commit into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
dmihalcik-virtru
added this pull request to stack #1033
September 10, 2026 20:52
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
from
September 22, 2026 14:23
d3028cf to
b671a6b
Compare
dmihalcik-virtru
removed this pull request from stack #1033
September 22, 2026 14:24
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
2 times, most recently
from
September 23, 2026 18:15
b671a6b to
5ebc5fd
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
September 23, 2026 18:15
319d60c to
79524d7
Compare
X-Test Failure Report |
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
September 24, 2026 14:34
79524d7 to
89388d2
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
from
September 24, 2026 14:34
5ebc5fd to
71aa201
Compare
X-Test Failure Report |
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
September 25, 2026 19:15
89388d2 to
fa233f1
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
from
September 25, 2026 19:15
71aa201 to
683c746
Compare
X-Test Failure Report |
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
September 28, 2026 14:17
fa233f1 to
b12007f
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
from
September 28, 2026 15:04
683c746 to
0cab1f8
Compare
dmihalcik-virtru
marked this pull request as draft
September 28, 2026 15:08
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
September 29, 2026 13:38
b12007f to
d729a0e
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
3 times, most recently
from
October 6, 2026 18:33
783a75c to
901245b
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
October 7, 2026 13:30
901245b to
e9cbf9d
Compare
…ze cap Extracts the 10MB manifest size check shared by write and read into assertManifestWithinSizeLimit (lib/tdf3/src/utils/zip-reader.ts) and calls it from writeStream before the manifest is emitted, so an oversized manifest (e.g. from a very large file at a small segment size) fails at encrypt time with an actionable ConfigurationError instead of producing a TDF that is permanently unreadable. Also fixes a >> 10 bitwise truncation bug in the existing read-side error message for manifests >= 4GiB. This is the first piece of DSPX-4651. The check still runs after the whole payload has been encrypted; making it an up-front estimate, and raising the cap so a 50 TiB file can fit under it at all, is the rest of that sub-task. DSPX-4651
dmihalcik-virtru
force-pushed
the
DSPX-4651-manifest-fail-fast
branch
from
October 15, 2026 13:11
0cab1f8 to
27bbf4a
Compare
dmihalcik-virtru
force-pushed
the
DSPX-4496-count-iv
branch
from
October 15, 2026 13:11
e9cbf9d to
03cac78
Compare
X-Test Failure Reportopentdf-sdk-lib |
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
MANIFEST_MAX_SIZE(10 MB) was enforced only on the read path. This extracts the check into a sharedassertManifestWithinSizeLimit()inlib/tdf3/src/utils/zip-reader.tsand calls it fromwriteStream()before the manifest is emitted.Also fixes a
>> 10bitwise truncation in the read-side error message: for manifests ≥ 4 GiB the shift wraps and reports a nonsense size. NowMath.floor(n / 1024).Why
Today it is possible to encrypt a large file successfully and produce a TDF that no reader can ever open. The write path had no opinion about manifest size; the read path rejects at 10 MB. A 5 GB file at the default 1 MiB segment size produces ~5,000 segments and is fine, but the failure mode scales in silently — and the whole point of the 50 TiB work (DSPX-4648) is that the manifest is the binding constraint, not the payload.
Failing at encrypt time turns a permanent data-loss-shaped bug into an actionable
ConfigurationErrorthat names the segment count and tells the caller to raisesegmentSize.Scope / what this does not do
This is the first of two pieces of DSPX-4651. The check still runs after the entire payload has been encrypted, so it wastes the whole encrypt before rejecting. Making it an up-front estimate from the source size, and raising the cap so 50 TiB can fit under it at all, is the rest of the sub-task and lands separately.
How to test
New cases in
lib/tests/mocha/unit/zip.spec.tscover the boundary (at the limit passes, one byte over throws) and the ≥ 4 GiB message formatting.Risk
Low, but it is a new rejection on the write path: a caller who was previously writing an unreadable TDF now gets an error instead. That is the intent. No existing test produces a manifest anywhere near 10 MB.