ARSN-617: clean-read filter hiding non-localized versions - #2688
Draft
DarkIsDude wants to merge 3 commits into
Draft
ARSN-617: clean-read filter hiding non-localized versions#2688DarkIsDude wants to merge 3 commits into
DarkIsDude wants to merge 3 commits into
Conversation
Issue: ARSN-617 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a per-call `cleanRead` flag on the metadata read and listing APIs, hiding the object versions whose data still lives on a remote source site (`locations[objMD.dataStoreName].isCRR`), as needed by the clean-room D/R deployments. MongoClientInterface implements it in the MongoDB query itself, so that listing limits and truncation stay correct, and MetadataWrapper sets the flag on every read and listing call when the deployment enables it. Issue: ARSN-617 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Hello darkisdude,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Contributor
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## development/8.5 #2688 +/- ##
===================================================
+ Coverage 74.66% 74.73% +0.06%
===================================================
Files 227 227
Lines 18606 18640 +34
Branches 3880 3857 -23
===================================================
+ Hits 13892 13930 +38
+ Misses 4709 4705 -4
Partials 5 5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Issue: ARSN-617 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Part of the clean-room D/R feature (epic ZENKO-5238, design Artesca Full D/R).
In a clean room, object metadata is replicated before the object data is copied locally: until then the version's
location/dataStoreNamestill refers to the remote production site. Such a version is non-localized, the condition beinglocations[objMD.dataStoreName].isCRR. Clients of the clean room must never observe those versions.Changes
cleanReadflag on the metadata read and listing APIs (getObject,getObjects,getBucketAndObject,listObject,listMultipartUploads), a backend-agnostic contract implemented byMongoClientInterfacefor now;getLocationConstraintsparameter, evaluated per call so configuration updates are picked up (same channel as the existingisLocationTransientpredicate);value.dataStoreName: { $nin: [...] }) rather than after the fetch: listing cursors are limited bymaxKeys, so filtering documents after the limit would under-fill pages and report a complete listing while entries remain. In a metadata search the condition is added as an$andelement after the search query is merged, so a search ondataStoreNameneither overwrites it nor is overwritten by it;getLatestVersionapplies the same filter, so a hidden master (and a PHD key) resolves to the newest localized version;DelimiterMasterin v0 skips the version keys of an object whose master key is hidden, instead of exposing them as master entries (v0 streams master and version keys in the same range); gated on the flag, no change for other deployments;MetadataWrappersets the flag on every read and listing call when clean read is enabled, and refuses to start when the backend does not implement it — a silently dropped filter would expose the very versions it is meant to hide. Backbeat-internal listings (listLifecycleObject) and all write paths are untouched;dataStoreName, are matched by$nin, and are therefore never hidden.Activation is out of scope here: CLDSRV-957 surfaces the environment variable through Cloudserver's configuration, ZKOP-564 sets it on the user-facing Cloudserver only. Write-time master-key handling is ARSN-618.
Tests
tests/unit/storage/metadata/mongoclient/cleanRead.spec.js— generated queries: flag off, flag on, no CRR location, no location configuration, filter propagation togetLatestVersion, andMongoReadStream$andmerging against a colliding search query;tests/unit/storage/metadata/MetadataWrapperCleanRead.spec.js— flag injection on read/listing calls, absence on internal and write calls, refusal on an unsupported backend;tests/functional/metadata/mongodb/cleanRead.spec.js— end-to-end onmongodb-memory-server, both v0 and v1, over a mix of localized versions, non-localized versions, a delete marker and a mixed object: master and version listings, paged version listing withmaxKeys: 1,getObjectby version and by master,getObjectsbatch, plus every case with the flag off.yarn build,yarn lint,yarn lint_md,yarn prettier:diff,yarn test(3941 passing) and thetests/functional/metadata/mongodbsuite (282 passing) are green.The first commit is a prettier reformat of the files touched, kept separate from the functional change.
🤖 Generated with Claude Code