Repository navigation
feat(sdk): add opt-in unprefixed manifest entry name - #4049
pflynn-virtru wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
cd0e8a2 to
e35ee47
Compare
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
e35ee47 to
acb8f8d
Compare
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
acb8f8d to
997a398
Compare
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
997a398 to
ad2e466
Compare
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
ad2e466 to
c497bbd
Compare
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
c497bbd to
051b4c0
Compare
X-Test Failure Reportgovulncheck-failure-1 |
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
The OpenTDF spec puts the manifest at the archive root under `manifest.json`. This SDK writes and reads `0.manifest.json`. The `0.` prefix is a holdover from an early design that anticipated several payload/manifest pairs per archive; that design never shipped. Write: `writeStream` emits `manifest.json`. Read: `loadTDFStream` and `getPolicyId` resolve the entry through `manifestEntryName()`, which prefers the spec name and falls back to the legacy name, so archives written by earlier releases keep opening. The dual-name handling lives at the TDF layer rather than in `ZipReader`, whose signature is unchanged — its other lookup is a payload path that must stay exact. Resolution runs against the central directory, not against a failed read, so an oversized manifest under the spec name stays a size error instead of silently yielding the superseded legacy entry. `0.payload` is unchanged: it is recorded in the manifest's `payload.url`, so renaming it would alter manifest contents rather than just archive layout. BREAKING CHANGE: archives written by this SDK name the manifest entry `manifest.json`. Readers that look it up by exact name — web-sdk <= 0.20, java-sdk, client-cpp — cannot read them until they accept the spec name. Refs opentdf/platform#3513, opentdf/platform#4049, opentdf/tests#600. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn-virtru@users.noreply.github.com>
The OpenTDF spec puts the manifest at the archive root under `manifest.json`. This SDK writes and reads `0.manifest.json`. The `0.` prefix is a holdover from an early design that anticipated several payload/manifest pairs per archive; that design never shipped. Write: `writeStream` emits `manifest.json`. Read: `loadTDFStream` and `getPolicyId` resolve the entry through `manifestEntryName()`, which prefers the spec name and falls back to the off-spec name, so archives written by earlier releases keep opening. The dual-name handling lives at the TDF layer rather than in `ZipReader`, whose signature is unchanged — its other lookup is a payload path that must stay exact. Resolution runs against the central directory, not against a failed read, so an oversized manifest under the spec name stays a size error instead of silently yielding the superseded off-spec entry. `0.payload` is unchanged: it is recorded in the manifest's `payload.url`, so renaming it would alter manifest contents rather than just archive layout. BREAKING CHANGE: archives written by this SDK name the manifest entry `manifest.json`. Readers that look it up by exact name — web-sdk <= 0.20, java-sdk, client-cpp — cannot read them until they accept the spec name. Refs opentdf/platform#3513, opentdf/platform#4049, opentdf/tests#600. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn-virtru@users.noreply.github.com>
The OpenTDF spec puts the manifest at the archive root under `manifest.json`. This SDK wrote and read `0.manifest.json`, an exact lookup with no fallback, so a TDF produced by any implementation written against the published spec was rejected outright. Write: `TDFWriter` emits `manifest.json`. Read: `TDFReader` accepts either name, preferring `manifest.json` when an archive carries both, so archives from earlier releases keep working. `SDK.isTDF` accepts either name too, and no longer requires the archive to hold exactly two entries -- the spec fixes where the manifest lives, not what else the archive may hold, and the count check made isTDF stricter than the reader it screens for. `0.payload` is unchanged: it is recorded in the manifest's payload URL field, so renaming it would alter manifest contents rather than just archive layout. Refs opentdf/platform#3513 Follows opentdf/platform#4049 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn-virtru@users.noreply.github.com>
Benchmark results, click to expandBenchmark authorization.GetDecisions Results:
Benchmark authorization.v2.GetMultiResourceDecision Results:
Benchmark Statistics
Bulk Benchmark Results
TDF3 Benchmark Results:
|
Refs opentdf/platform#3513. The read half of #405, split out so it can land on its own. ## Problem The [OpenTDF spec](https://opentdf.io/spec#tdf-structure) names the manifest entry in a `.tdf` archive `manifest.json`: > The `manifest.json` file MUST be in JSON format and reside within the root of the OpenTDF Zip archive. This SDK writes and reads `0.manifest.json`. On the read side `TDFReader` does an exact `containsKey` with no fallback, so a TDF produced by any implementation written against the published spec is rejected with `tdf doesn't contain a manifest` before any schema check runs. `SDK.isTDF` carries its own copy of the literal, so such an archive is screened out one step earlier still. The `0.` prefix is a holdover from an early design that anticipated several payload/manifest pairs per archive. That design never shipped. ## Change **Read** — `TDFReader` prefers `manifest.json` and falls back to `0.manifest.json`. Preference matters rather than first-match: an archive carrying both must not have its conformant entry passed over for the superseded one. **Sniffing** — `SDK.isTDF` accepts either name, so it does not reject an archive the reader can now read. It also no longer requires the archive to hold *exactly* two entries. That check was scoped out at first as unrelated to the entry name, but review pointed out it is not: an archive carrying both manifest names holds three entries, and `prefersTheSpecNameWhenAnArchiveCarriesBoth` is precisely the case the reader now handles — so the sniffer would reject what the reader it screens for accepts. The count was never part of the structure `isTDF` describes; the spec fixes where the manifest lives, not what else the archive may hold. **Duplicate entry names** — a zip may legally list one name twice, and `Collectors.toMap` turned that into an `IllegalStateException` out of `TDFReader`'s constructor: outside its declared `throws SDKException, IOException`, outside what a caller screening untrusted input catches, and outside the fuzz targets' catch list. Dropping the count check above made it newly reachable through `isTDF`, so it is rejected here as an `IllegalArgumentException` alongside the constructor's other malformed-input paths. An archive carrying *both* manifest names is still read, not rejected. Those are two distinct entries and the spec settles which one wins; a single name listed twice offers no principled choice. Worth being explicit about the tradeoff, since the root signature covers only `integrityInformation.segments` — not `policy`, not `keyAccess[].url` — and no hash covers the entry name: a DEK holder can file two internally-valid manifests over one ciphertext and have this SDK enforce one while a `0.`-keyed peer enforces the other. That ambiguity predates this PR; preferring the spec name relocates which side Java lands on rather than creating it. Failing closed would be a cross-SDK decision, not a Java-only one. **Write is untouched.** `TDFWriter` still emits `0.manifest.json`, so this release produces byte-identical archives and every existing reader keeps working. The new `TDF_MANIFEST_FILE_NAME_SPEC` constant is read-side only. Flipping the writer is the breaking half and stays in #405. `0.payload` is untouched. It is recorded in the manifest's `payload.url`, so renaming it would alter manifest contents rather than just archive layout, and neither the spec nor opentdf/platform#4049 fixes it. ## Testing Ported from #405, minus the write-side test: | Test | Failure without the change | |---|---| | `TDFReaderTest.readsManifestUnderTheSpecName` | `IllegalArgumentException: tdf doesn't contain a manifest` | | `TDFReaderTest.readsThePayloadAlongsideASpecNamedManifest` | same | | `TDFReaderTest.prefersTheSpecNameWhenAnArchiveCarriesBoth` | returned the non-aligned manifest | | `SDKTest.testExaminingTDFWithSpecManifestName` | `false` | | `SDKTest.testExaminingTDFWithBothManifestNames` | `false` -- three entries | | `SDKTest.testExaminingTDFWithAnExtraEntry` | `false` -- three entries | | `TDFReaderTest.rejectsAnArchiveThatListsOneNameTwice` | `IllegalStateException: Duplicate key 0.payload` | | `TDFReaderTest.rejectsAnArchiveThatListsTheManifestNameTwice` | same, for `manifest.json` | `prefersTheSpecNameWhenAnArchiveCarriesBoth` files a *different* manifest under each name, so it cannot pass by reading whichever entry the reader happened to pick. Guarding behavior that must not change: `TDFReaderTest.readsManifestUnderTheOffspecName`, `TDFReaderTest.rejectsAnArchiveWithNoManifestUnderEitherName`, `SDKTest.testExaminingZipWithNoManifest`, `SDKTest.testExaminingZipWithNoPayload`, and the pre-existing `SDKTest.testExaminingValidZTDF` / `testExaminingManifest`, which run against the checked-in non-aligned-named `sample.txt.tdf` fixture. The two negative `isTDF` cases hold two entries each, so they failed for the name they were missing rather than for their entry count even before the count check came out. Review of this branch added four tests that pin rules nothing else held down: `rejectsNearMissManifestEntryNames` (a basename or case-insensitive lookup passed the suite before, so `sub/manifest.json` would have been read as the manifest despite the spec's root-only rule), `SDKTest.testExaminingLargerZipWithNoManifest` (both negatives held two entries, so "any zip over two entries is a TDF" passed), and the two duplicate-name cases above. **Verification:** Local Java execution was unavailable because this machine has no JDK or Maven. GitHub CI passed `mavenverify`, Platform Integration (including FIPS), and the full cross-SDK xtest matrix on the PR head; the previously failing matrix job passed on rerun. The new tests were also watched failing against unmodified production code on #405's branch, where the shared code is identical. ## Downstream impact None. Readers gain a name they accept and `isTDF` gains archive shapes it recognizes; nothing either previously accepted is taken away, and no archive this SDK writes changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Compatibility** * Improved support for TDF archives using the standard `manifest.json` entry name. * Continued support for archives using the SDK’s existing `0.manifest.json` entry name. * When both manifest names are present, the standard `manifest.json` entry is used. * Archives may include additional entries beyond the manifest and payload. * **Bug Fixes** * TDF detection now correctly recognizes valid archives and rejects those missing a manifest or payload. * Archives containing duplicate entry names are now rejected with a clear error. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Paul Flynn <pflynn-virtru@users.noreply.github.com> Co-authored-by: Paul Flynn <pflynn-virtru@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Add WithUnprefixedManifestName option that writes the manifest under the zip entry name manifest.json (the name the TDF spec defines) instead of the legacy 0.manifest.json. Pre-4.3.0 WithTargetMode overrides this option and keeps the legacy name. Readers that predate manifest.json support, such as Go SDK releases before v0.35.0, cannot open TDFs created with this option. Restore deleted assertion in testGetManifestBeforeAndAfterFinalize that verifies GetManifest returns a clone. Update godoc for chunked and experimental options to reference compatibility caveats. Fix gocritic appendAssign lint issue by using bytes.Join instead of append. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Paul Flynn <pflynn-virtru@users.noreply.github.com>
094d60e to
4ec7aa7
Compare
|
Proposed Changes
Adds an opt-in that writes the TDF manifest under the zip entry name
manifest.json, the name the TDF spec defines, instead of the legacy0.manifest.json. Default output is unchanged, byte for byte.WithUnprefixedManifestName()(CreateTDF),WithChunkedUnprefixedManifestName()(chunked writer), andWithUnprefixedManifestName()insdk/experimental/tdf. The names describe the format they produce, matchingWithHexlessPolicyBinding(feat(sdk): add opt-in hexless policy binding #4232).WithTargetMode/WithChunkedTargetModeoverrides the opt-in and keeps0.manifest.json, whatever the option order.CreateTDF, the chunked writer and the experimental writer.This replaces the earlier version of this PR, which changed the default and was a breaking change. Making
manifest.jsonthe default remains a separate, later change.Refs #3513.
Checklist
Testing Instructions
cd sdk && go test ./... -racegolangci-lint run --new-from-rev=origin/main ./...insdk: 0 new issues.TestSegmentWriter_ManifestEntryName,TDFSuite.Test_ManifestEntryName,TestChunkedManifestEntryName, and the experimentalManifestEntryNamecase. Each covers the default, the opt-in, and a legacy target mode in both option orders, and round-trips throughLoadTDF.🤖 Generated with Claude Code
Summary by CodeRabbit
manifest.jsoninstead of the default0.manifest.json.0.manifest.json, regardless of option order.