Skip to content

smartspim: prefer SmartSPIM's stitched All_Channels volume - #142

Merged
stevevanhooser merged 3 commits into
mainfrom
claude/spim-all-channels
Oct 12, 2026
Merged

stevevanhooser merged 3 commits into
mainfrom
claude/spim-all-channels

Conversation

@stevevanhooser

@stevevanhooser stevevanhooser commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

ferret-lightsheet (formats/SmartSPIM.md, spim2zarr --source all_channels) established that All_Channels/ contains SmartSPIM AutoStitch's TeraStitcher merge of the destriped tiles, with flat-field and left/right illumination correction and blended overlaps. This is the better-stitched, harmonized data that Imaris is built from. With this PR, ndr.reader.smartspim reads it first. The PR also fixes two TIFF data-type bugs found along the way (see below).

All_Channels layout:

  • A flat folder of 2-D LZW-compressed uint16 TIFFs, one per (z, channel), named <Y>_<X>_<z*10>_ch<k>.tiff, plus an ImageList.txt.
  • Each slice covers the whole stitched canvas, so there are no tiles.
  • ch<k> follows the filter-channel number, not the acquisition order. It comes from <channel>_MIP_stitched/, or from the filter-channel rank when that folder is missing.

New format functions (+ndr/+format/+smartspim)

  • readAllChannelsInfo: indexes the slices (sorted by numeric z) and maps each channel to its ch<k>. Also returns the canvas size and class from the first TIFF header, and the voxel size from metadata.json.
  • hasAllChannels: a non-throwing probe that checks file names only. It returns true only when every channel has slices.
  • readAllChannelsVolume: reads one channel as (z, y, x), with an optional ZRange.

Reader epochs

Epochstream Reads
{root} stitched volume, all channels on the C axis (ch<k> order)
{root, channel} stitched volume, one channel
{root, channel, tile} destriped Ex_* tile (unchanged)

Fallback: the Ex_* tiles are not obsolete; ferret-lightsheet treats them as the source of truth. When All_Channels is missing or incomplete, {root} and {root, channel} raise the same ChannelNotPinned / TileNotPinned errors as before.

New resolveepoch fields: source, channelNames, numSlices, height, width, dtype and allChannels. info.tile is unchanged for tile epochs.

Fix 1: readTileInfo reported int16 for some uint16 TIFFs (commit 3e300e5)

  • Cause: tiffDtype treated any SampleFormat containing "signed integer" as signed. imfinfo's 'Unsigned integer' contains that text.
  • Effect: a TIFF that stores SampleFormat = 1 explicitly came back as int16. readframes then allocated an int16 array, so every pixel above 32767 was clipped.
  • Why it hadn't shown up: tifffile (which writes the destriped tiles) and MATLAB's imwrite don't store the tag. libtiff-based writers may.
  • Fix: rule out 'unsigned' before testing for 'signed'.
  • Tests: new tests write TIFFs with an explicit UInt / Int SampleFormat tag and check readTileInfo and readAllChannelsInfo. readAllChannelsInfo already used the corrected logic.

Fix 2: tiffstack.tiffclass read float TIFFs as uint32 (commit 8f870c1)

  • Cause: tiffclass matched SampleFormat by exact string, and its float case accepted only 'ieeefloat' / 'float'. imfinfo reports float TIFFs as 'IEEE floating point', so they fell through to the unsigned branch.
  • Effect: a float32 stack came back as uint32. readframes then cast every value to uint32, dropping fractions and clamping negative values to 0.
  • Fix: match by substring, with the same unsigned-before-signed order as Fix 1.
  • Scope: this also covers ndr.reader.prairieview, which reuses tiffclass.
  • Tests: TestTiffstack now round-trips a float32 stack and an int16 stack. writeMultipageTiff takes an optional SampleFormat; existing callers still get uint16.

Tests

  • makeExampleFixture(..., 'AllChannels', true) adds an LZW All_Channels folder to the test fixture.
  • New test classes: TestSmartSPIMAllChannels and TestSmartSPIMReaderAllChannels.
  • The existing tests (fixture without All_Channels) still cover the fallback path.
  • CI at 3e300e5: 396 tests, 0 failures. The tiffstack tests were added afterwards in 8f870c1.

The Python port is VH-Lab/NDR-python#31. Its bridge YAML records b8f0e4b (All_Channels), 3e300e5 (readTileInfo) and 8f870c1 (tiffstack). The Python readers get the data type directly from tifffile and never had either bug.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U33UPxYJWPpcQimNu5t5Cb

ferret-lightsheet (formats/SmartSPIM.md, spim2zarr --source all_channels)
established that All_Channels/ holds SmartSPIM AutoStitch's TeraStitcher
merge of the destriped tiles, with flat-field / left-right illumination
correction and blended overlaps -- the better-stitched, harmonized data
Imaris is built from. The reader now prefers it.

Format layer (new):
  * readAllChannelsInfo - index All_Channels/<Y>_<X>_<z*10>_ch<k>.tiff,
    map each Ex_* channel to its ch<k> (from <channel>_MIP_stitched,
    else filter-channel rank; not acquisition order), slices sorted by
    numeric z, canvas size / class from the first header, voxel size
    from metadata.json.
  * hasAllChannels - non-throwing probe (names only): usable only when
    every channel has slices.
  * readAllChannelsVolume - one channel as (z, y, x), optional ZRange.

Reader epochs:
  {root}                 stitched volume, all channels on C (ch<k> order)
  {root, channel}        stitched volume, one channel
  {root, channel, tile}  destriped Ex_* tile, unchanged
Without a usable All_Channels (AutoStitch not run, partial restore),
{root} and {root, channel} keep the ChannelNotPinned / TileNotPinned
errors: the Ex_* tiles are not obsolete, so the per-tile layout stays.
resolveepoch gains source / channelNames / numSlices / height / width /
dtype / allChannels fields; info.tile is kept for tile epochs.

The test fixture takes 'AllChannels', true to add an LZW All_Channels
folder; new TestSmartSPIMAllChannels and TestSmartSPIMReaderAllChannels.
Not run: no MATLAB in the authoring environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U33UPxYJWPpcQimNu5t5Cb
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

398 tests  +32   395 ✅ +32   36s ⏱️ ±0s
 51 suites + 2     3 💤 ± 0 
  1 files   ± 0     0 ❌ ± 0 

Results for commit 8f870c1. ± Comparison against base commit 95b7e2d.

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.15385% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.26%. Comparing base (95b7e2d) to head (8f870c1).

Files with missing lines Patch % Lines
+ndr/+format/+smartspim/readAllChannelsInfo.m 87.27% 14 Missing ⚠️
+ndr/+format/+smartspim/readAllChannelsVolume.m 78.04% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #142      +/-   ##
==========================================
+ Coverage   53.26%   54.26%   +1.00%     
==========================================
  Files         160      163       +3     
  Lines        9190     9420     +230     
==========================================
+ Hits         4895     5112     +217     
- Misses       4295     4308      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tiffDtype treated any SampleFormat containing "signed integer" as
signed, and imfinfo's 'Unsigned integer' contains that substring. A TIFF
that stores SampleFormat = 1 explicitly (libtiff-based writers may)
therefore came back 'int16', and readframes then preallocated int16,
clipping every pixel above 32767. Files from tifffile and imwrite omit
the tag, which is why the fixtures and real tiles never hit it.

Rule out 'unsigned' before testing for 'signed'. New tests write TIFFs
with an explicit SampleFormat tag (UInt and Int) for readTileInfo and
for readAllChannelsInfo, which already had the corrected logic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U33UPxYJWPpcQimNu5t5Cb
tiffclass matched SampleFormat by exact string, and its float case only
accepted 'ieeefloat' / 'float'. imfinfo reports float TIFFs as 'IEEE
floating point', which fell through to the unsigned branch: a float32
stack came back as 'uint32' and readframes cast every value to uint32,
dropping fractions and clamping negatives to 0. Match by substring
instead, ruling out 'unsigned' before testing for 'signed' (the same
fix as readTileInfo in 3e300e5). Affects ndr.reader.prairieview too,
which reuses tiffclass.

New tests round-trip a float32 and an int16 stack written with an
explicit SampleFormat; writeMultipageTiff takes an optional format and
derives BitsPerSample from the data class (default unchanged: uint16).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U33UPxYJWPpcQimNu5t5Cb
@stevevanhooser
stevevanhooser merged commit 966fe3c into main Oct 12, 2026
8 checks passed
@stevevanhooser
stevevanhooser deleted the claude/spim-all-channels branch October 12, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants