Repository navigation
Lightsheet napari viewer: cloud tile streaming, view-aware fetching, problem reporting, and the cloud fixture - #320
Merged
Conversation
Run 2 of build-lightsheet-fixture succeeded at the roundtrip and printed: CLOUD_DATASET_ID=6ab84b549852a120dbcb22bc but crashed on the next line with: Unrecognized field name "remoteDatasetName". The info struct that ndi.test.cloud.lightsheet_blob_cloud_roundtrip returns spells the field ``cloudDatasetName``, not ``remoteDatasetName`` -- I made the wrong name up when I wrote the workflow. Because the crash happened BEFORE the fopen to GITHUB_OUTPUT, neither job output landed and the step summary + lightsheet-fixture-id artifact steps never ran. The cloud upload itself is fine (the fixture is live under test user 1, dataset id 6ab84b549852a120dbcb22bc); this fixes the workflow so the next dispatch surfaces the id through every channel (log, outputs, summary, artifact) instead of only in the raw log. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Fixture: 6ab84b549852a120dbcb22bc (test user 1, prod), uploaded by
the MATLAB roundtrip via build-lightsheet-fixture.yml. Its ids
live in tests/fixtures/lightsheet_cloud_fixture.json so the test
finds them and a regeneration is one edit in one place.
Test file: tests/test_pyramid_loader_integration.py
* Skips automatically when NDI_CLOUD_USERNAME / NDI_CLOUD_PASSWORD
are not set (same gate as test_cloud_live.py), so a laptop
without secrets stays quiet.
* Module-scoped download: pulls the fixture ONCE with sync_files=false
and hands the path to every test. sync_files=false is deliberate
-- we want to exercise the on-demand cloud-fetch path the viewer
uses, not a fully-hydrated local copy.
* Five checks:
- loader.numLevels + loader.docs enumerate finest-first
- loader.specs returns one 3D array per channel
- the coarsest level computes and is not all-zero (guards the
silent-fill_value failure the DID series switch once hid)
- every level in the ladder computes; per-level timings are
printed for the "home vs. office vs. CI" comparison
- loader.stats() reports a non-empty fetcher summary so we know
real cloud fetches ran
Registration:
* new "cloud" pytest marker in pyproject.toml
* test-cloud-api.yml now includes test_pyramid_loader_integration.py
in the same account-and-env matrix as the rest of test_cloud_*.py
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
test-cloud-api.yml was schedule + dispatch only, so a PR that adds or edits the pyramid loader had no PR-time signal it was still reaching the cloud correctly. The new integration test in this PR (tests/test_pyramid_loader_integration.py) needs to run against the real fixture (6ab84b549852a120dbcb22bc) before merging, so we know the loader hasn't regressed while the diff was in flight. Add a path-filtered pull_request trigger covering: the pyramid package (src/ndi/pyramid/**), the fixture registry, the integration test file, the existing test_cloud_*.py suite, and this workflow file. Anything else on a PR does not fire the cloud tests, so unrelated PRs still cost nothing in cloud API calls. Runs under the same matrix (User 1/2 x prod/dev) as the schedule. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
First PR run on run 36281586795 failed with: ModuleNotFoundError: No module named 'dask' on four of the five pyramid loader integration tests (the fifth one only touched loader.docs / .numLevels which do not build any dask array, so it passed). The cloud CI installs with `ndi_install.py --dev --no-validate`, which does not pull dask -- dask lives under the napari extra (``[napari]``) alongside napari itself and its Qt / OpenGL dependencies. Installing the whole napari extra here is overkill: we do not need Qt on a headless cloud runner, only the dask array machinery the loader builds its multiscale ladder with. Add a one-line ``pip install 'dask[array]>=2023.1'`` to the cloud CI's install step so the loader can compute in that environment without dragging in Qt / OpenGL. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
The pyramid loader in ndi.pyramid.ImagePyramidLoader is napari-independent -- that was the point of splitting it out of gui/app/lightsheetZarr. Callers use it from matplotlib, Jupyter, headless batch analyses, and (once the sibling namespace lands) a MATLAB comparison harness. Every one of those callers needs dask[array], because that is what the loader builds its lazy multiscale arrays with. Keeping dask under the napari extra was defensible when the loader lived inside the napari viewer module; it is not now. A headless pip install of ndi should be able to run `from ndi.pyramid.loader import ImagePyramidLoader` and get to work without discovering, deep inside the traceback, that a viewer-side extra was needed. * Move `dask[array]>=2023.1` into project.dependencies alongside numpy / scipy / networkx. * Leave it pinned in the napari extra too, so an existing lock file that resolved dask via napari does not suddenly declare a version conflict. * Drop the workaround from test-cloud-api.yml -- dask now installs via the base package, so the cloud CI job needs no extra step. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
The fixture (currently 6ab84b549852a120dbcb22bc, see lightsheet_cloud_fixture.json) lives on one account + env pair: user 1, prod. The cloud CI matrix runs every account x env combination, so User 1 dev + User 2 (any env) tried to download that id and crashed with HTTP 404 + CloudNotFoundError. Turn the 404 into a skip: catch CloudNotFoundError in the downloaded_dataset fixture and pytest.skip with a message that names the current NDI_CLOUD_USERNAME + CLOUD_API_ENVIRONMENT and points at NDI_LIGHTSHEET_TEST_FIXTURE_ID for redirection. All five tests in the module inherit the skip through the fixture chain, so the three unrelated cloud CI cells no longer fail the build. Behaviour on User 1 prod is unchanged -- the fixture resolves, the download runs, all five tests execute. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
One transient TLS or connection error at start of run used to mark a scope failed and drop every uid in it to per-member getFileDetails -- O(N) API calls on a series that should have cost one. For a 156k-member lightsheet series that hung downloadDataset for hours on a residential network. _default_signer now retries getSignedURLSetAll with exponential backoff (1s / 4s / 16s, 4 total attempts) before surfacing the exception as a scope failure. Injected fake signers keep their old semantics -- tests are unaffected. See #322 and VH-Lab/NDI-matlab#1010 for the parallel MATLAB fix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
A hung getSignedURLSetAll call today emits no output for minutes: the CloudClient retries transient errors up to MAX_ATTEMPTS silently, each retry after a 120s socket timeout, and the whole walk goes silent between HTTP calls too. On a residential connection where the batch endpoint stalls this looks indistinguishable from a genuine deadlock. Two log lines break that silence: * getSignedURLSetAll: "fetching page N ..." and "page N returned M uids in T.TTs" bracket every HTTP call. Silence for more than a page-time now names which page and which document is stuck. * CloudClient._request: each retry after a transient error or 5xx logs the exception name and the retry delay. The underlying retry loop is no longer invisible. Diagnostic only -- no behavior changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
_fetch_manifest asks for ONE known uid (the series manifest file). Going through fetch_cloud_file with the document id set makes the batch cache try to answer, and to answer it fetches the WHOLE document's uid->URL map through getSignedURLSetAll. A lightsheet OME-Zarr level with 121k chunks means walking 243 pages at 25s each -- 100+ minutes to fetch one manifest. Pass ndi_document_id="" so the batch is skipped and fetch_cloud_file drops straight to getFileDetails: one API call for the one uid we want. Downloading a real lightsheet dataset with SyncFiles=false is unblocked by this; before this change the download hung for hours reconstructing manifests. The batch is still the right choice for chunk fetches at viewer time, where hundreds of uids in the same document are requested back-to-back and the O(1) batch amortizes across them. The viewer-time scale problem is separate and needs the async signed-URL-set job family (porting_deferred in the bridge). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…through it Adds the four MATLAB entry points that stabilized on NDI-matlab main at 0a2cdec (VH-Lab/NDI-matlab#1009): createSignedURLSetJob, getSignedURLSetJob, waitForSignedURLSetJob and getSignedURLSetResult. Rewires batch_signed_url._default_signer from the paged getSignedURLSetAll walk (~25 s per 500-uid page, so 100+ min for a 121k-member document) to the async job path that builds the whole uid -> URL map server-side and returns it in one gzipped blob. Retry semantics (NDI-python#322) are preserved around the whole three-step exchange. Bridge YAML flipped for all four functions (porting_deferred -> ported) with matlab_last_sync_hash bumped to 0a2cdec, and batchSignedUrlLookup's decision_log updated to record the signer swap. Filed as #206. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SK8sCN6BWj7VYzt2pieTpd
…id fallback Silent fallback to per-uid getFileDetails hides real bugs in the async signed-URL-set-job path behind slow-but-working chunk fetches. A napari viewport with 100k+ chunks per document is unusable at one API call per chunk (thousands per zoom), and the underlying async-job failure -- endpoint 404, schema drift, timeout, missing resultUrl -- never surfaces. _default_signer now raises BatchScopeUnreachable when every retry attempt of create/wait/read has failed. _fetch_scope propagates it up through BatchSignedUrlLookup.lookup and out to callers (fetch_cloud_file, viewer chunk fetches). The partial-map case (batch returned a map that didn't name a specific uid) stays as a legitimate per-uid fallback -- that's a data-drift path, not a broken endpoint. Injected test signers that raise other exceptions still fall back per the old behavior, so existing failure-mode tests keep working. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Without this, every logger.info() in ndi.cloud.* is silently dropped: async signed-URL-set-job submit/wait/read progress, batch retry decisions, per-page walk timings all vanish. A viewer stuck inside _default_signer looks identical to a healthy one -- no way to tell whether the async job is running server-side or dead on arrival. Set up stderr logging at INFO when the root logger has no handlers so an embedding host (notebook, downstream CLI) that has already configured logging keeps control. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
The retry log printed only the exception type name; a bare 'RuntimeError' tells us the async job path failed but not why. Widen to include the (dataset, document, file_series) scope and the exception message, so a running viewer surfaces the actual reason each attempt failed rather than four identical opaque lines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…lookups Strict-mode BatchScopeUnreachable was raising all the way up, but did not update _failed_scopes -- so every next uid in the viewport triggered another full 4-attempt retry cycle against the broken endpoint. A viewer whose async job path is dead spent 20 s per failed cycle in an infinite loop instead of surfacing the actual cause once and staying quiet. _fetch_scope now records the failure with unreachable=True and the cause string before re-raising. lookup() consults the cache and, when it finds an unreachable entry within failure_ttl_seconds, re-raises a fresh BatchScopeUnreachable naming the original cause and elapsed time. Also widened the retry log to include scope and exception message instead of just the exception type name, so 'RuntimeError' becomes something legible like 'RuntimeError: signed-URL-set job ... failed'. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…er-chunk catch _resolve caught EVERY exception from database_openbinarydoc and returned None, which _read_chunk_from_fetcher then turned into a fill_value block. That catch is right for a per-chunk transient blip (one chunk drops out, canvas keeps rendering) but wrong for a systemic scope failure: every chunk in the scope raises identically, every block becomes fill_value, and the whole canvas silently reads as zeros while the fetcher summary line reports "N failures" that no test asserts on. It is the exact silent-fetch-failure pattern the integration test's non-zero assertion was written to catch (see test_the_coarsest_level_computes_and_is_not_all_zero), now firing on the tiny fixture too. Let BatchScopeUnreachable propagate past the per-chunk catch so a broken batch surfaces through .compute() as an actual error rather than an all-zero result. Ordinary exceptions still degrade to fill_value. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…ync prefetch The upsample fallback never calls chunkPath synchronously -- it uses chunkPathIfCached (memoization only, no fetch) plus prefetchAsync (fire-and-forget). So BatchScopeUnreachable raised inside _resolve fires in a background pool thread and is caught by prefetchAsync's generic 'except Exception' handler, which turns it into path=None. The main thread then goes down the "coarse not cached either" branch and returns a zero_block for every chunk, producing a canvas of silent zeros -- the very pattern the loader's integration test was designed to catch, now firing on the tiny fixture. _ChunkFetcher now tracks systemic scope failures on the instance (_unreachable_exc). When prefetchAsync's background _run catches BatchScopeUnreachable it stores the exception rather than dropping it, and chunkPath / chunkPathIfCached re-raise on subsequent calls. The very first chunk still races: its prefetch may not have failed before the fallback returns zeros -- but any subsequent chunk in the same compute surfaces the exception, which propagates through dask to .compute(). In practice that is the second chunk, which means the test fails loudly instead of silently. Ordinary per-chunk transient failures still degrade to fill_value (the existing 'except Exception' branch below the new one keeps that behavior). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…gible _default_signer and waitForSignedURLSetJob were completely silent during a successful long-running job. A viewer waiting minutes on a 121k-member scope looked identical to a hung viewer, and the only signal was a raw TCP connection to the API host visible in netstat. Add INFO log lines at each step: * _default_signer: submit -> jobId, terminal state + elapsed, result blob delivered with uid count. * waitForSignedURLSetJob: every poll logs the state and signedCount / totalCount, so a long wait shows visible progress instead of silence. Diagnostic only -- no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…ive token expiry Observed today: a signed-URL-set-job poll loop that had to run for ~1 hour (server signs ~30 URLs/sec, and a 121k-member scope means tens of minutes) died at ~9 min with HTTP 403 "Token is invalid or expired". The bearer token's TTL is shorter than the wall-clock time some legitimate long operations need, and the client kept sending the stale token forever until the outer wait timed out at 60 minutes, having produced nothing useful. CloudClient._request now catches a 401/403 once per request, calls self._reauthenticate() (a fresh CloudConfig.from_env() + authenticate() that mutates config.token in place), updates the Authorization header, and retries. If the reauth itself fails, or if the retried request still comes back 401/403, the original error surfaces as before -- a second 401 after a successful reauth is a real credential problem, not something worth looping over. Also updates the "a 401 is not retried" test to acknowledge the new one-shot reauth semantics, and adds four tests pinning: happy-path reauth-and-retry, 403 same as 401, no loop after a persistent 401 past reauth, and the retry using the fresh token in its Authorization header. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Port of VH-Lab/NDI-matlab commit 27661a3 (issue #322). A working scientist reopening the same lightsheet-scale dataset over a day should pay the ~85 min async signed-URL-set-job cost once, not every open. New signed_url_disk_cache module writes each scope (dataset_id, document_id, series_name) to ~/.ndi/signed-url-cache/<ds>/<doc>[_<escaped_series>].json.gz as gzipped JSON, TTL-checked against the server's own filesExpireAt with a 30 min safety buffer, atomically renamed on write. MATLAB and Python write byte-for-byte identical files: schemaVersion, path convention, filename escaping (percent-encode outside [A-Za-z0-9._-], SHA-1 hash above 96 chars), gzip framing, JSON keys, env var (NDI_SIGNED_URL_CACHE_DIR / NDI_SIGNED_URL_CACHE_SAFETY_SECONDS) all match. A cache dir written by MATLAB reads clean from Python. BatchSignedUrlLookup gains disk_cache=False; get_default() flips it on -- production path. Existing test-suite instances stay off by default. On S3 403 during the actual getFile the scope is forgotten from disk so a rotated token doesn't cascade to N per-uid 403s; getFile takes an opt-in error_out sink so the status code drives the hook. Session-scoped conftest fixture isolates the cache dir so no test can pollute a developer's ~/.ndi/. Bridge YAML: adds signedUrlDiskCache entry and bumps batchSignedUrlLookup's matlab_last_sync_hash to 27661a3.
This was referenced Sep 28, 2026
The signed-URL disk cache commit (18ec747) did not run black over the files it touched, so CI's lint check flagged four files as needing reformatting. This applies black without altering any logic. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Three unrelated failures on head 18ec747, all fixed here: 1. cloud.client: gate mid-request reauth on _can_reauth flag test_bad_auth_raises hand-built a CloudClient(CloudConfig(token=...)) with a deliberately invalid token and expected CloudAuthError. On 401 the reauth path added in d7b14b7 called CloudConfig.from_env() and authenticate() against the ambient CI env credentials, silently swapping the hard-coded bad token for a valid one, retrying, and succeeding -- exactly the "used a different token than the caller asked for" behavior the test was supposed to catch. CloudClient now defaults _can_reauth = False. from_env() (and any future login helper) sets it True after authenticate(). A test that stands up a hand-built client to verify auth failure is used as passed. Long-running clients built by from_env() still get the mid-run token refresh needed to survive TTL expiry. Adds test_a_hand_built_client_does_not_reauth_against_env as a regression pin, and updates make_client(*, can_reauth=True) so existing retry tests continue to behave as though the client had been obtained via from_env(). The client fixture in test_cloud_live.py wraps CloudClient(cloud_config) with _can_reauth = True since its config comes from an env-driven login(). 2. tests/test_cloud_filehandler: reset ambient CloudClient around TestGetOrCreateCloudClient test_env_vars_present passed in isolation and in the No-Cloud suite but failed in the live Cloud API suites: an earlier live test populated _ambient_cloud_client via a real cloud fetch, and the "should call from_env once" assertion then saw zero calls (the cached client was returned instead). An autouse fixture resets both slots before and after each test in the class. 3. sync bridge: record ndi.cloud.sync.internal.fetchManifest Added on the MATLAB side in commit 5541a00; the Python analog (ndi.cloud.filehandler._fetch_manifest, commit 5ab9c04) is ported_differently -- inline with its caller rather than a standalone module. Records the port so test_matlab_bridge_completeness stops flagging it as unrecorded. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Fixes the MATLAB/Python bridge completeness job on Waltham-Data-Science/ NDI-python#320 head 8eb7927. - src/ndi/cloud/sync/ndi_matlab_python_bridge.yaml: fetchManifest 0ae6bfad → a84130a7. 0ae6bfad was the blob hash from a raw file fetch, not a commit; a84130a7 is the commit that introduced the standalone helper. No Python change: the port was always inline in ndi.cloud.filehandler._fetch_manifest. - src/ndi/cloud/ndi_matlab_python_bridge.yaml: updateFileInfoForRemoteFiles 9467a0a7 → a84130a7. Same commit extracted the local fetchManifest helper out of this file into its own module. Nothing to port here: the extraction is a pure refactor and Python calls its own inline helper. - src/ndi/gui/component/ndi_matlab_python_bridge.yaml: ndi_gui_component_ProgressBarWindow b367283b → 2b23dd30. Commit 2b23dd30 dropped the (1,:) size constraint from the addBar Timeout argument-block declaration (a MATLAB argument-validation adjustment). No behavioural change; note added to the decision_log. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Contributor
Author
|
CI note — What is failing on
Why it is not this PR's diff
Standing down on this failure for this PR No fix exists to port yet — the trigger is server-side and specific to User 1 prod. Not re-running: the failure is deterministic on this matrix cell, not a flake, and a re-run would reproduce identically. Everything the disk-cache/CI-fix push ( Generated by Claude Code |
Was: one 0.5 s retry per scope when the first signed-URL-set response did not name a requested uid. Now: a bounded schedule of 1 s / 3 s / 9 s. #320's Cloud API (User 1, prod) run shows the signed-URL-set index lagging further than 0.5 s after ``waitForAllBulkUploads`` returns -- the wait covers the bulk-upload extraction subsystem reporting done, but not the seam between that and the signed-URL-set subsystem catching up. One retry isn't enough to bridge it, so the round-trip test's manifest reconstruction sees a partial map for the demoNDISeries members, the strict-mode ingest_locations rebuild produces no entries, and the local add rejects the document (main sees the same server-side race as ``uid_misses > 0`` further along in the same test). Three waves cover the observed tail, still bounded (each wave costs one scope refetch, not one per uid), and never fire in the common case where the first map is already complete. API: - New ``partial_map_retry_delays`` kwarg on BatchSignedUrlLookup (tuple of waits). Defaults to DEFAULT_PARTIAL_MAP_RETRY_DELAYS = (1.0, 3.0, 9.0). - ``partial_map_retry_seconds`` kept as a deprecated scalar (None means "use partial_map_retry_delays"; any real number replaces the schedule with a one-element tuple containing that wait, so ``partial_map_retry_seconds=0`` still fires ONE retry with no wait, matching the pre-schedule contract every existing test pins). - ``_retried_scopes`` set becomes ``_scope_retry_attempts`` dict of attempts spent per scope; the loop walks the schedule until a fresh map names the uid or the schedule is exhausted. Two new unit tests pin the multi-wave behaviour (``test_a_multi_wave_schedule_retries_until_a_hit_or_exhaustion`` and ``test_a_schedule_that_never_settles_bounds_retries_to_len``); the round-trip test's cap on ``partial_map_retries`` relaxes from 1 to ``len(DEFAULT_PARTIAL_MAP_RETRY_DELAYS)``. MATLAB mirror lands in VH-Lab/NDI-matlab commit 02880062d on the same branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
MATLAB's batchSignedUrlLookup.m moved to 02880062 (partial-map retry schedule went from a single 0.5 s wait to a bounded 1 s / 3 s / 9 s schedule). Python's counterpart on this branch shipped the mirror change in the previous commit (4dff8bf), so this hash bump records a same-commit port, not a drift. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
stevevanhooser
pushed a commit
to VH-Lab/NDI-matlab
that referenced
this pull request
Sep 28, 2026
When DID reads a series member, it fetches the manifest first via the same download handler with seriesName="". That call previously went through ``batchSignedUrlLookup`` under the whole-document scope, and for a lightsheet-scale pyramid document (15k+ files) the batch endpoint spent 60-90 s signing URLs the caller had no use for -- just to answer the ONE manifest URL. The subsequent seriesName="chunk.bin" call then fired its own scope, and the two answers overlapped almost completely. Now: when seriesName is empty, the batch is skipped and the fetch takes the direct ``getFileDetails`` path (O(1)), matching what ``ndi.cloud.sync.internal.fetchManifest`` already does for the internal manifest fetch. When DID follows up with a member fetch (seriesName="chunk.bin"), the batch fires there and amortizes across every member exactly as it did before. Python mirror lands in Waltham-Data-Science/NDI-python on the same branch (filehandler.download_file_from_cloud gained the same guard). See #1010 and Waltham-Data-Science/NDI-python#320. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
When DID reads a series member, it fetches the manifest first via the same download handler with seriesName="". That call previously threaded documentId through to fetch_cloud_file, which used the whole-document batch scope, and for a lightsheet-scale pyramid document (15k+ files) the batch endpoint spent 60-90 s signing URLs the caller had no use for -- just to answer the ONE manifest URL. The subsequent seriesName="chunk.bin" call then fired its own scope, and the two answers overlapped almost completely. Now: when seriesName is empty, download_file_from_cloud clears ndi_document_id and the fetch takes the direct getFileDetails path (O(1)), matching what _fetch_manifest already does for the internal manifest fetch. When DID follows up with a member fetch (seriesName="chunk.bin"), the batch fires there and amortizes across every member exactly as it did before. test_ordinary_file_still_carries_document_scope is repurposed as test_a_single_file_fetch_bypasses_batch, pinning the new invariant. MATLAB mirror lands in VH-Lab/NDI-matlab commit 20391306 on the same branch (didsqlite.download_file_from_cloud gained the same guard). Real-world impact: on User 1 prod, a level swap that previously fired a redundant 76 s whole-doc signed-URL-set job now skips it entirely, and only the chunks-series scope needed for the actual member fetches runs -- 76 s of dark screen goes away. See #320. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Kick off the async signed-URL-set job for every level's chunk.bin series as soon as the viewer knows the pyramid's level documents, in a single background thread. The first zoom into a new level no longer waits 20-80 s for the batch endpoint to sign that scope -- the URLs are already cached from work done behind the initial level 0 render. - BatchSignedUrlLookup.prefetch_scope: seed one scope without asking about a specific uid. Skips the partial-map retry on purpose (that's a lookup-time concern; a later lookup drives it if needed). - BatchSignedUrlLookup.start_prefetch: daemon thread that iterates scopes and calls prefetch_scope for each, logging per-scope timing, skipping empty document ids, and continuing past BatchScopeUnreachable so one bad level does not stop the rest. - ImagePyramidLoader.startSignedUrlPrefetch: enumerate the pyramid's level docs, build (cloud_dataset_id, doc.id, "chunk.bin") scopes for every level that has a chunk.bin series, and hand them to the module-level lookup. Off with NDI_LIGHTSHEET_PREFETCH_SIGNED_URLS=0. - lightsheetZarr.viewer.openPyramid: discover the cloud dataset id via session.is_in_cloud() and fire the prefetch after the loader builds, so the jobs run while napari is still opening. Sequential rather than parallel across scopes on purpose: 5 levels at ~1 min each hide inside the initial-render wall time, and a parallel refactor would need to unwind the shared lock that serialises signer calls today. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
… data)
Live test on the user's Maddie lightsheet pyramid showed the fallback
actively blocking the swap-to-fine refresh:
with fallback ON (previous default):
first paint -> upsampled coarse (Path 2) -> napari marks slice
"loaded". Prefetches land. Our refresh hint fires. Napari re-slices,
Path 1 returns real fine data (hit_fine climbs). But napari does
NOT re-render -- the on-screen pixels stay coarse until the user
zooms in and out by hand to generate real mouse-wheel camera
events. The user described this as "the low-res version replaces
the correct draw".
with fallback OFF (NDI_LIGHTSHEET_UPSAMPLE_FALLBACK=0):
first paint -> black for missing chunks. As prefetches land and
napari re-slices, the view refreshes to show real fine data
without needing the user to zoom. Correct behaviour.
Flip the default. Keep the knob (NDI_LIGHTSHEET_UPSAMPLE_FALLBACK=1
now opts IN) so cloud users with multi-minute fetch stretches can
still get the "blurry but present" first paint if they prefer it to
black -- that was the original motivation and it may still be the
right trade on very slow cloud connections. But locally and on fast
cloud links, OFF is the correct default: "black briefly then correct
fine pixels" beats "coarse pixels forever that never refine".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
Follow-up to 9c5134f which flipped env_on() default to False. Two tests still asserted the old ON-by-default behaviour and failed on 3.12 (and would have failed on 3.10 and 3.11 once they finished running): * test_pyramid_upsample_fallback.TestEnvGate.test_absent_env_is_on -> renamed to test_absent_env_is_off and inverted the assertion. * test_pyramid_loader.TestStats.test_fallback_line_only_appears_when_the_env_is_on -> inverted the two branches: unset env now means no 'fallback' key in loader.stats(), and NDI_LIGHTSHEET_UPSAMPLE_FALLBACK=1 is the opt-in that makes it appear. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N67xH9BejMzZp4GNAq8m7w
…dow closed Launch window: closeLaunchWindow() cleared _window, and the very next progress.note() went through _mode() -> _launchWindow(), found no window and built a fresh "starting ..." one that nothing ever closed. A _window_closed latch now stops the rebuild (same fix in genepyramid, where the bug is latent). View tile tally: the fetch counter only saw fetches as napari issued them, so it could say "12 in flight" but not how many the view needed. New viewtiles module works that out from napari's data_level, corner_pixels and dims point plus each level's chunk grid and stored chunk list (now exposed as ImagePyramidLoader.levelMeta), skipping sparse chunks. A main-thread QTimer polls it and feeds the counter, so the status bar reads "view 37/120 tiles ready | ...", the canvas overlay reads "Loading view: 37 / 120 tiles (12 in flight)" and hides when the view is complete, and stderr logs "view complete: N tile(s) in Xs". Skipped in single-level and viewport-clip modes, where the level index does not line up with the level documents. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…k silently When the tally could not read napari's state it quietly reverted to the in-flight-only status, which looks identical to running without the feature. The poll now logs "view tile tally: on (...)" at start and "view tile tally: cannot count this view -- <reason>" whenever the reason changes (level/doc count mismatch, unreadable layer state, axis mapping, view too large); openPyramid logs when single-level or viewport-clip mode turns it off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…d past napari slices on one thread, so zooming through an intermediate level started a slice that then held up the slice for where the user stopped until every one of its tiles had arrived, and the fetch pool was FIFO on top of that. _ChunkFetcher now runs its own priority queue instead of a ThreadPoolExecutor: newest requests first, and chunks in the current view (handed over by the view-tally poll via setView) ahead of everything -- queued immediately even if napari has not asked yet. A blocking read for a chunk that has left the view is released with StaleChunkRead, so the stale slice ends and napari moves on; the fetch itself stays queued at low priority and lands in the cache for when the user zooms back. Reads are only judged against a view set AFTER they began (the poll lags napari), the coarsest level is never released (auto-contrast samples it), and an unknown view releases nothing. StaleChunkRead is raised rather than answered with fill_value so napari's dask cache never keeps a blank block; the viewer filters it out of napari's error notifications. Measured under real napari with async slicing (1 s simulated fetches, 2 workers, zoom through a 20-tile level to a 4-tile view): final view complete 11.9 s after the last zoom before, 2.7 s after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
Brings in ndi.cloud.api.feedback (submitFeedback, deleteFeedback) and its tests (#323) plus the bridge checks for MATLAB files that live on an NDI-matlab branch, ahead of the viewer's Report-a-problem button. Conflict: src/ndi/fun/doc_lightsheet/ndi_matlab_python_bridge.yaml. Both sides recorded the NDI-matlab 5c9e8a8 tile-budget bump; took main's entries, which add the matlab_branch field its new bridge checks use. Semantic fix: this branch's CloudClient re-authenticates on 401/403 when _can_reauth is set (initialised in __init__). The feedback tests build a client with CloudClient.__new__, so their helper now sets _can_reauth = False, as test_cloud_client_retry's helper does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
A beta viewer's problems mostly go unreported: a user launched from MATLAB has no terminal to copy a log from, and writing an email with versions and context is too much. The NDI Cloud panel now has a "Report a problem..." button under the BETA badge. The dialog asks what kind of problem and what the user saw, shows exactly what will be sent, and sends it through ndi.cloud.api.feedback.submitFeedback; Save to file and Copy cover a user who is not signed in or offline. New shared module ndi.gui.feedback (the gene-pyramid viewer can reuse it): * LogBuffer / installLogBuffer: tees stderr and the logging root into a ring of the last 5,000 lines from launch on. * scrub: removes signed-URL credentials (X-Amz-Signature/Credential/ Security-Token), bearer tokens, JWTs and secret-named values -- viewer logs do carry signed URLs. * buildReport: the submitFeedback kwargs, scrubbed, with versions and NDI_* flags as environment, cut to the server's size limits (log keeps its END). * sendReport / saveReport / describeSendError and the Qt dialog (send runs off the UI thread; one report per dialog; errors say what to do). The viewer's collector attaches the pyramid and level geometry, the view tally, each layer's level and region, fetch counts, loader stats and the log, each probe guarded so a report about a broken viewer still builds. _FetchCounter now resolves sys.stderr at print time so its lines reach the log buffer. Checked in real napari: the button docks, the dialog's report carries the view tally, layer level, document id and log, and a signed-URL signature in the log is scrubbed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
… viewer The gene expression viewer gets the same report button as the lightsheet viewer. On a cloud session it sits on the NDI Cloud panel, where the reader signs in (which sending needs); a local session has no cloud panel, so it gets a small "Feedback" dock with just the button -- Save to file and Copy work without signing in. The button itself moves into ndi.gui.feedback.reportButton so both viewers share one definition (text, tooltip, dialog); the lightsheet panel now uses it. ndi.gui.feedback.napariState gives every viewer the same picture of the canvas (dims, camera, each layer's type, visibility, level and region); the gene viewer's collector adds the pyramid id, density mode, the gene selection size, the cloud dataset id and the recent log, and openPyramid starts the log buffer at launch. addCloudPanel and addAllPanels take an optional report_collect; without it nothing changes. Checked in real napari: one button in the cloud and in the local layout, and the report carries app, document id, layers and section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
NDI-matlab 2f9b0eaf routes both MATLAB launches of the napari viewer (ndi.fun.doc.lightsheet.view and LightsheetZarrManager's View dialog) through the new ndi.fun.doc.lightsheet.launchViewer, which sets NDI_LIGHTSHEET_DEBUG=1 and keeps the viewer's output in lightsheet_YYYYMMDDHHMMSS.txt in NDI's temp folder. * view and LightsheetZarrManager: bumped to 2f9b0eaf with a dated note; nothing on the Python side follows (it is the launched process and already honours NDI_LIGHTSHEET_DEBUG). * launchViewer, viewerLogFile, viewerShellCommand: new matlab_only entries -- Python calls the viewer directly, with its output on the caller's own stderr and in the in-viewer log buffer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…id levelSources) * makePyramid: bumped to 04ce18f9 with a dated note -- levelSources and channelNames added for fromSPIM; default OME-Zarr behaviour unchanged. * fromSPIM, reduceBlocks: new porting_deferred entries. The Python side is a viewer today; both port after the MATLAB version is validated, on top of a makePyramid port. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
CI's "MATLAB/Python bridge completeness" checks the bridge against NDI-matlab main, where +ndi/+cloud/+download/+internal/signedUrlDiskCache.m and +ndi/+cloud/+sync/+internal/fetchManifest.m do not exist yet: both live on claude/lightsheet-zarr-ndi-viewer-djp5vk (VH-Lab/NDI-matlab#979). Add matlab_branch to the two entries, as the bridge reference prescribes for files on an unmerged branch. Passes against NDI-matlab main and against the branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
668beb13 preallocates fromSPIM's per-ladder arrays (no behaviour change) alongside a TestFromSPIM expectation fix; nothing to port. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
The previous fixture dataset (6ab84b54...) lost a file blob in S3 (NoSuchKey), failing the production pyramid integration test. Rebuilt it with the Build lightsheet blob fixture workflow (run 38265299422). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…a-gybqx6 # Conflicts: # src/ndi/cloud/api/files.py # src/ndi/cloud/api/ndi_matlab_python_bridge.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…e dask pool A tile takes seconds on the cloud, almost all of it waiting on the network, but only ~CPU-count tiles were ever in flight, and every fetcher worker opened a whole NDI session at launch just so it could call database_openbinarydoc on its own thread. Resolving a chunk needs no database. The level document is in memory, its series manifest names each member's uid, and DID's cached_path_for_uid (documented as safe from any thread) finds a member on disk with no SQL. A member not on disk is now fetched through download_file_from_cloud with the same context DID's open_doc passes (manifest location as source, seriesName and member uid in the context), into DID's file cache under its uid, so DID and later lookups find it exactly where open_doc would have put it. The session path stays as the fallback (manifest not on disk yet, no ndic:// location), and a worker opens a session only on first need. NDI_LIGHTSHEET_DIRECT=0 turns the session-free path off. - Default worker count 8 -> 32 (NDI_LIGHTSHEET_WORKERS still overrides). - configureDaskPool: napari reads through dask's threaded scheduler, whose pool defaults to one thread per CPU; the viewer now sizes it to the worker count so napari can keep every worker busy. - fetch_cloud_file records per-thread stage timing (signing, batch or per-file, transfer, bytes); under NDI_LIGHTSHEET_DEBUG each slow resolve logs "[lightsheet] fetched chunk.bin_N in Xs: sign ..s, transfer ..s, .. MB, .. MB/s", so the log says where a tile's time goes. - stats_summary reports how many resolves went via a session. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
32 tiles at once on a ~100 Mb/s line took ~30 s per tile against ~5 s with 8 on a faster one: past the line's capacity every tile in flight shares the bandwidth, so each arrives later, the ones in view included. The right number depends on the connection, so the reader picks it. - _ChunkFetcher starts max_workers threads (MAX_WORKERS = 64 from levelArrays) and lets at most `workers` fetch at once. setWorkers(n) changes that live (clamped to 1..maxWorkers): threads past the limit wait, running fetches finish undisturbed. Workers are plain threads since the session-free change, so 64 idle ones cost nothing. - Default limit 16 (NDI_LIGHTSHEET_WORKERS still overrides). - The NDI Cloud panel gets a "Tiles in flight" drop-down (8/16/32/64, plus the launch value if different) wired to setWorkers. dask's pool is sized to maxWorkers so any choice can actually be used. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…tra-gybqx6 Add in-app problem reporting and improve tile fetch prioritization
VH-Lab/NDI-matlab#1027 added +ndi/+cloud/+api/+files/waitForFileUpload.m on main, and bridge completeness (which reads NDI-matlab main) failed for want of an entry. Recorded as not_yet_ported with the reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…inst NDI-matlab 9c21eb61 VH-Lab/NDI-matlab#1027 made both wait for NDI Cloud to record uploads (through the new waitForFileUpload). Bumped both hashes with a note that the wait is not ported yet, and gave the waitForFileUpload entry its sync hash (eb583289). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…t_ported Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
…4973c1 (lightsheet branch) CI checks the bridge against the matching NDI-matlab branch, where uploadFilesForDatasetDocuments last changed in the main merge d94973c1 (main's #1027 wait plus that branch's verbose upload count). Pinned the entry there with matlab_branch, to be dropped once NDI-matlab#979 merges. Bridge tests pass against both that branch and NDI-matlab main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
stevevanhooser
changed the title
Debugging scripts from the napari async-slicer investigation; not part of the package and not meant for main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
Every lightsheet entry's MATLAB counterpart is on NDI-matlab main (d292984f), and the claude/lightsheet-zarr-ndi-viewer-djp5vk branch is gone. Hashes unchanged: they are reachable from main and the files have not moved since. Bridge tests pass against NDI-matlab main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
- tests/test_ndi_common_matlab_sync.py: MATLAB_BRANCH_PENDING listed the lightsheet document and schema files as pending on claude/lightsheet-zarr-ndi-viewer-djp5vk. That branch merged into NDI-matlab main (#979, d292984f) and was deleted, so the check failed with 'branch not found'. The files now match main byte-for-byte; the map is empty again. - build-lightsheet-fixture.yml: ndi_matlab_ref defaults to main, where lightsheet_blob_cloud_roundtrip now lives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the lightsheet pyramid viewer work to
main.napariViewLightsheetopens an NDI lightsheet pyramid (thelightsheetZarrPyramid/lightsheetZarrLeveldocuments written by NDI-matlab) and streams its chunks from NDI Cloud on demand. On a bandwidth-limited line, the tiles you are looking at arrive first and the viewer says how far along the view is.The branch began as the cloud fixture and loader integration test. It now also carries the viewer work merged into it, notably #324.
What's in it
Fetching chunks:
ndi.pyramid.multiscale,ndi.pyramid.loadercached_path_for_uid, with no SQL, safe from any thread.download_file_from_cloudwith the same context DID'sopen_docpasses. It lands in DID's file cache under its uid.NDI_LIGHTSHEET_DIRECT=0turns this path off.NDI_LIGHTSHEET_WORKERS.setWorkers()changes N live.configureDaskPoolwidens dask's threaded-scheduler pool, which defaults to one thread per CPU, so napari's I/O-bound reads can keep the workers busy.StaleChunkRead). Its fetch carries on in the background, so zooming back is free.stats_summaryreports cache hits, cloud fetches, failures, released stale reads and fallbacks.Viewer:
ndi.gui.app.lightsheetZarrviewtiles): how many stored chunks the current view needs against how many are on disk. The overlay and status bar show "37 / 120" rather than only an in-flight count.NDI_LIGHTSHEET_DEBUG. Each slow fetch logs its signing time (batch or per-file), transfer time, bytes and MB/s, usingndi.cloud.filehandler.lastFetchTiming._window_closedlatch); the refresh hint and viewport fixes are included.Problem reporting:
ndi.gui.feedback/v1/feedback(ndi.cloud.api.feedback). Save to file and Copy work without signing in.Cloud: signed URLs and downloads
batch_signed_url): through the async signed-URL-set job path, with retry and backoff.signed_url_disk_cachekeeps signed URLs on disk across reopening a dataset.filesAPI additions the above use.Tests and CI
test_gui_lightsheetZarr_viewtilestest_pyramid_fetch_prioritytest_pyramid_fetch_directtest_gui_feedbacktest_cloud_api_signed_url_set_job,test_cloud_batch_signed_url,test_cloud_signed_url_disk_cachetest_cloud_client_retrytests/fixtures/lightsheet_cloud_fixture.jsonpins6accf3a0044c77d93a6ebdb7, rebuilt 2026-10-12 bybuild-lightsheet-fixture.ymlrun 38265299422 after the previous fixture lost a blob.NDI_LIGHTSHEET_TEST_FIXTURE_IDoverrides it.pull_requesttrigger for pyramid-loader changes.Bridge
claude/lightsheet-zarr-ndi-viewer-djp5vkcarrymatlab_branch. They are dropped once Lightsheet pyramids: ingest from OME-Zarr or SmartSPIM, chunk materialization, cloud round trip, and napari viewing VH-Lab/NDI-matlab#979 merges.mainmoved with #1025 and #1027.waitForFileUploadis recorded asporting_deferred, anduploadSingleFile/uploadFilesForDatasetDocumentswere re-examined, with the upload wait noted as not yet ported.Before merging
napari_slicer_repro_v1.py...v6.pydebugging scripts from the repository root (c3cc811).matlab_branchmarkers in the bridge YAMLs. Only then delete NDI-matlab'sclaude/lightsheet-zarr-ndi-viewer-djp5vk.Test plan
7171218: lint, tests on 3.10/3.11/3.12, Windows smoke, CED integration, MATLAB↔Python symmetry, bridge completeness, and Cloud API (users 1 and 2 × prod and dev, including the pyramid-loader integration test against the rebuilt fixture).c3cc811, which only removes the repro scripts.Related
🤖 Generated with Claude Code
https://claude.ai/code/session_016wj6cuvAwgAaYN5dNM1o6L