Repository navigation
Retry once when the batch signer returns a partial map - #992
Merged
Merged
Conversation
ndi.cloud.download.internal.batchSignedUrlLookup collapsed two distinct causes of "this uid is not in the scope's map" into one silent per-uid fallback: 1. Data drift: the file was never in this scope. The right answer is to fall back and warn. 2. A lagged index: the file IS in the scope on the server, but the batch endpoint's map has not caught up with a just-completed bulk upload. Falling back per uid here defeats the batch design -- a 28,000-member series that took the fallback would issue 28,000 getFileDetails calls where the batch would have taken one. Waltham-Data-Science/NDI-python#309 (the counterpart on the Python side) observed the second case on TEST_USER_1/prod after Waltham-Data-Science/NDI-python#308 first let a series manifest reach the server: the batch answered signerCalls=1 with a 2-entry map for a 4-member series. The two missing uids fell back per uid. Retrying the same scope 37 minutes later on run_attempt=2 returned the full 4-entry map -- so the endpoint does eventually catch up, just not by the time waitForAllBulkUploads reported done. Fix (#991): when a cached scope answers with a URL but a subsequent uid in that scope misses, drop the entry, pause for partialMapRetrySeconds (default 0.5), and re-fetch once. If the second map covers the uid, take it and every subsequent uid in the scope resolves from the refreshed cache; if not, warn and fall back as before. Bounded to one retry per scope, so a genuine data-drift miss still costs one extra signer call for the whole scope, not one per uid. Tracked in a new STATS.partialMapRetries counter so tests can distinguish the retry path from a plain data-drift miss. Refactor: the fetch/nargout/try dance moves into a nested localFetchScope so the retry can call it a second time without duplicating it. Adds an injectable sleepFcn (defaults to @Pause) so tests exercising the retry path do not pay real wall-clock latency, and a partialMapRetrySeconds=0 mode that keeps the single retry but skips the sleep. The live-cloud FileSeriesRoundTripTest.signerCalls assertion is relaxed to signerCalls == 1 + partialMapRetries with partialMapRetries <= 1, keeping the "one batch call per series' worth of members" contract while allowing the retry. uidMisses == 0 stays firm. Four new unit tests in BatchSignedUrlLookupTest mirror the Python counterpart's TestBatchLookupPartialMapRetry: the retry succeeds; the retry pauses before re-fetching; the retry is bounded to once per scope; a retry that raises still falls back cleanly without double-counting the miss. Existing miss-path tests are updated to account for the added signer call and to skip the sleep via partialMapRetrySeconds=0. Closes #991. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016rEFEXaZ9t3BftShqc9Y4B
Contributor
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #992 +/- ##
==========================================
+ Coverage 29.79% 29.85% +0.05%
==========================================
Files 857 857
Lines 43723 43750 +27
==========================================
+ Hits 13029 13062 +33
+ Misses 30694 30688 -6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
stevewds
approved these changes
Sep 15, 2026
stevevanhooser
pushed a commit
to Waltham-Data-Science/NDI-python
that referenced
this pull request
Sep 15, 2026
NDI-matlab moved past 5804cd80 (VH-Lab/NDI-matlab#992 -- retry once when the batch signer returns a partial map). Purely a hash bump to satisfy test_matlab_bridge_hashes; the entry stays ported and no Python-side behavior needs to follow yet. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
This was referenced Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ndi.cloud.download.internal.batchSignedUrlLookupfor the case where the presign endpoint answers with a populated map that does not name a requested uid (a "partial map"). On the miss, the helper drops the entry, pauses briefly, and re-fetches once. If the second answer covers the uid, every subsequent uid in the same scope resolves from the refreshed cache; if not, the caller falls back per uid as before and the scope's warning fires. Bounded to one retry per scope, so a genuine data-drift miss still costs one extra signer call for the whole scope, not one per uid.STATS.partialMapRetriescounter so tests can distinguish the retry path from a plain data-drift miss.nargout/trydance moves into a nestedlocalFetchScopeso the retry can call it a second time without duplicating it. Adds an injectablesleepFcn(default@pause) and apartialMapRetrySecondsoption so tests exercising the retry do not pay real wall-clock latency.FileSeriesRoundTripTest.signerCallsto1 + batchStats.partialMapRetrieswithpartialMapRetries <= 1, keeping the "one batch call per series' worth of members" contract while allowing the retry.uidMisses == 0stays firm.Motivation
Waltham-Data-Science/NDI-python#309 saw the second case on TEST_USER_1/prod after Waltham-Data-Science/NDI-python#308 first let a series manifest reach the server: the batch answered
signer_calls=1with a 2-entry map for a 4-member series. Re-running the same job 37 minutes later returned the full 4-entry map — so the batch endpoint does eventually catch up, just not by the timewaitForAllBulkUploadsreports done. The MATLAB helper has the same shape and the same risk. This PR is the MATLAB counterpart of Waltham-Data-Science/NDI-python#310, which merged the same fix on the Python side.Closes #991.
Test plan
runtests('ndi.unittest.BatchSignedUrlLookupTest')— the four newtestPartialMapRetry*tests cover retry-succeeds, retry-pauses, retry-once-per-scope, and retry-raises-and-falls-back; existing miss-path tests updated for the added signer call.runtests('ndi.unittest.cloud.FileSeriesRoundTripTest')on live cloud —signerCalls == 1 + partialMapRetriesanduidMisses == 0hold across the previously flaky TEST_USER_1/prod matrix cell.🤖 Generated with Claude Code
https://claude.ai/code/session_016rEFEXaZ9t3BftShqc9Y4B
Generated by Claude Code