No sharding configuration exposed for tables

[LucaMarconato]

As highlighted here M4i-Imaging-Mass-Spectrometry/thyra#136, we don't expose a way to configure sharding parameters for tables (the case for images and labels is covered by #1106).

We could explore adding this.

[Tomatokeftes]

Hi, I would like to take this one if nobody is on it. Here is what I was thinking, happy to change
it if you prefer something else.

I would add a single keyword argument, table_shard_size_bytes: int | None, on write and
write_element, passed down to write_table.

It is not symmetric with #1106, on purpose. A table is a tree of zarr arrays with different ranks and
dtypes and they all get the same dataset_kwargs from anndata, so no single chunks/shards tuple
works for all of them. I checked on anndata 0.12.16 and zarr 3.2.1 with a normal table:

• a 2-D chunks fails on the sparse X parts and on obs/_index
• a 1-D chunks fails on 2-D obsm
• any shards tuple fails on the uns scalars, since _remove_scalar_compression_args drops
  chunks for scalars but not shards

chunks=<int> broadcasts fine, but shards=<int> gives TypeError: 'int' object is not iterable.
So a table_write_kwargs mirroring raster_write_kwargs would end up with a documented happy path
that cannot actually run.

A byte budget avoids this. zarr works out shard = chunk * n per array, so divisibility and
shard <= array hold whatever the rank, length or dtype.

How it works: nothing goes into dataset_kwargs. spatialdata only scopes
array.target_shard_size_bytes and anndata.settings.override(zarr_write_format=3, auto_shard_zarr_v3=True) around the anndata call for one table's write, and anndata adds
shards="auto" itself where that is safe. The existing write_zarr and write_elem calls stay
exactly as they are, so the #1183 re-fetch is not touched.

To be clear about what this does not do: it narrows the global, it does not remove it. At the moment
a downstream writer has to hold zarr.config open for a whole sdata.write, which is what we do in
Thyra. After this it is scoped to one element and restored on exit, including on errors. Still a
process global underneath.

On naming, happy to add raster_shard_size_bytes as well, so this does not invert the raster_ and
table_ split reserved in #1106.

On sequencing, I was planning to base it on main rather than stack on #1106, so it does not get
blocked. The overlap is about nine one line forwarding additions in _core/spatialdata.py, easy to
sort out whichever lands first. #1055 does not overlap, its io_table.py change is only in
_read_table.

Small heads up from looking at #1106: on the current head import spatialdata raises
KeyError: 'RASTER_WRITE_KWARGS_DOCS' from @docstring_parameter, because the placeholder is still
in the docstrings being formatted.

Also I filed zarr-developers/zarr-python#4304 while looking at this: shards="auto" on a 0-d array
never finishes when array.target_shard_size_bytes is set. Not reachable with the design above
since nothing goes into dataset_kwargs, but it is the reason it must not.

[Tomatokeftes]

Opened #1199 for this.

It is the design from my comment above, unchanged: one keyword-only table_shard_size_bytes: int | None
on write and write_element, reaching write_table as shard_size_bytes. Nothing goes into
dataset_kwargs. spatialdata scopes zarr.config["array.target_shard_size_bytes"] and
anndata.settings.override(zarr_write_format=3, auto_shard_zarr_v3=True) around the existing anndata
call for one table's write, and restores both on exit, including on exceptions. The write_zarr and
write_elem calls are untouched, so the #1183 re-fetch is not affected.

5 source files and 1 test file. Based on main, so it does not depend on #1106 and does not collide
with #1055. All checks are green, including the prerelease leg that installs anndata from git main.

Two notes for whoever picks it up:

• I could not add the release-added label, so the PR body carries a # Release notes section
  instead. Someone with write access may want to add the label.
• The offer above still stands: happy to add raster_shard_size_bytes in the same PR if you would
  rather the raster_ and table_ split land together.

[Tomatokeftes]

Correction on the raster_shard_size_bytes offer I made above. I am withdrawing it. Having read the
raster write path, the symmetry is only in the name and the mechanism does not carry over.

The table budget works because anndata injects shards="auto" itself once auto_shard_zarr_v3 is on.
array.target_shard_size_bytes is read only when zarr is asked to compute an automatic shard shape.
On zarr 3.3.0:

with zarr.config.set({"array.target_shard_size_bytes": 512 * 1024}):
    a = g.create_array("x", shape=(4_000_000,), dtype="f8")                 # chunks (125000,) shards None
    b = g.create_array("y", shape=(4_000_000,), dtype="f8", shards="auto")  # chunks (125000,) shards (125000,)

Nothing injects shards="auto" on the raster side. shards does not appear in _io/io_raster.py at
all; rasters go through ome_zarr.writer.write_image with storage_options. So a
raster_shard_size_bytes built the way the table one is built would be a silently inert argument.
Making it real means spatialdata passing shards="auto" into storage_options for every raster array
and every pyramid level, which is a larger change than the table one rather than a mirror of it.

Separately, a byte budget is the wrong shape for raster. The scalar was forced on tables because a
table is a heterogeneous tree of mixed-rank arrays that all receive one shared dataset_kwargs, so no
tuple satisfies all of them. Raster has uniform known rank, and storage_options already carries an
explicit per-level chunks. Where an explicit tuple works it is the better knob, since shards usually
want to align to tile boundaries and a byte budget cannot express that. #1106's raster_write_kwargs
looks like the right form for that side.

So #1199 stays table only. If the raster_ and table_ prefix asymmetry is a concern, landing #1106
resolves it better than adding a weaker second raster API here.

One open question for whoever takes the raster side: I did not check whether ome-zarr forwards a
shards key from storage_options through to zarr.create_array. That may be a separate blocker.

[Tomatokeftes]

Answering the open question I left above: ome-zarr does forward shards from storage_options, but
only as an explicit tuple.

Measured on ome-zarr 0.18.0, dask 2026.8.0, zarr 3.3.0, writing a (1, 2048, 2048) uint16 image
through spatialdata._io.io_raster.write_image:

no storage_options  : chunks=(1, 2048, 2048)  shards=None
chunks only         : chunks=(1, 512, 512)    shards=None
chunks + shards     : chunks=(1, 256, 256)    shards=(1, 1024, 1024)
shards only         : ValueError: Chunk edge length 1024 in dimension 1 is not divisible by the shard's inner chunk size 2048
shards="auto"       : ValueError: Could not interpret 'a' as a byte unit

Three things that matter for the raster side:

1. It works, and _write_pyramid_to_zarr has deliberate logic behind it. When shards is given it
   rechunks the dask array so that one dask chunk equals one shard, to avoid concurrent writes into
   the same shard, and the user's chunks become the zarr inner chunks.
2. shards="auto" is not merely un-injected, it is unusable. The string reaches dask's rechunk,
   which tries to parse it as a byte size. So the mechanism behind the table byte budget has no
   counterpart on this path. That settles my previous point more firmly than I put it.
3. shards needs an explicit chunks alongside it. On its own it falls back to the level chunksize
   and fails the divisibility check. So the raster API really does want the pair, which is the
   raster_write_kwargs shape in Support sharding through config and raster_write_kwargs #1106.

One gotcha for #1106: the forwarding is gated on USE_DASK_ARRAY_KWARGS = dask >= 2026.3.0, and below
that ome-zarr runs zarr_array_kwargs_copy.pop("shards", None), dropping shards with no error.
spatialdata already pins dask>=2026.3.0, so that branch is unreachable today, but it is worth a note
or an assertion if that floor ever moves, since the failure mode is silence rather than an error.

The storage_options seam itself already exists on both write_image and write_labels. What is
missing is only the plumbing from SpatialData.write down to it.