From a3700794466410fb4352be17c55935d7a54e6045 Mon Sep 17 00:00:00 2001 From: Connor Tsui Date: Fri, 23 Oct 2026 15:56:48 -0400 Subject: [PATCH 1/2] docs: describe scheme variants in the versioning docs The compression section described per-edition scheme configuration as planned work. The decimal, bit-packing and frame-of-reference schemes now pick their v1 or v2 variant from the allowed wire IDs (#10076), so the section and the compatibility footnote describe that behavior instead. The section also names the frame-of-reference exception for child compression. Signed-off-by: Connor Tsui --- docs/specs/versioning/compatibility.md | 9 ++++---- docs/specs/versioning/design.md | 29 +++++++++++++------------- 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/specs/versioning/compatibility.md b/docs/specs/versioning/compatibility.md index e3a6f67a796..a6272bd8a8e 100644 --- a/docs/specs/versioning/compatibility.md +++ b/docs/specs/versioning/compatibility.md @@ -76,8 +76,7 @@ For the compatibility guarantee and minimum reader versions, see [Versioning](.. support for Format B, so Library 1 still writes only Format A. [^compression]: - The default compressor filters schemes by their declared output wire IDs, but general per-writer - scheme configuration is not implemented. The planned configuration would select compatible - behavior before estimation, sampling, and full compression, avoiding recompression solely to - meet the edition. These cells still require the writer to construct a permitted representation. - See [Compression](design.md#compression) for the current behavior and planned work. + The default compressor picks its schemes and their variants from the permitted wire IDs before + it compresses, so it does not need to recompress only to meet the edition. These cells still + require the writer to construct a permitted representation. See + [Compression](design.md#compression) for the current behavior. diff --git a/docs/specs/versioning/design.md b/docs/specs/versioning/design.md index 0efb20b6f32..6f36efa3099 100644 --- a/docs/specs/versioning/design.md +++ b/docs/specs/versioning/design.md @@ -258,20 +258,21 @@ recording the minimum version. ## Compression -The default BtrBlocks compressor excludes a scheme if any of its declared output wire IDs is -forbidden. This filtering also applies to schemes used for child compression. - -The current decimal scheme produces only single-child arrays and declares the v1 wire ID, so values -too wide for it remain in the standard uncompressed decimal representation. The multi-child -serializer exists, but the default scheme does not construct those arrays and no declared edition -permits their v2 ID. - -### Planned scheme configuration - -The planned improvement is to configure a scheme's behavior for the selected editions, allowing it -to retain an older mode when its newer mode requires a forbidden format. That configuration needs to -apply consistently to estimation, sampling, full compression, children, and fallbacks. General -per-writer scheme configuration is not implemented, and its API is unsettled. +The default BtrBlocks compressor uses only the schemes whose output it can write. A wire ID is +allowed when the session registers it, the enabled editions permit it, and the compressor's preset +does not exclude it. For example, the CUDA preset excludes the v2 IDs described below. The +compressor excludes a scheme if any wire ID that the scheme declares is not allowed. Child arrays go +through the same compressor, except that the frame-of-reference scheme always bit-packs its encoded +child with the v1 bit-packing variant. The writer still checks every serialized ID, so a forbidden +ID in any child fails the write. + +The decimal, bit-packing, and frame-of-reference schemes each have a v1 and a v2 variant. The +compressor uses the v2 variant when the v2 wire ID is allowed and the v1 variant otherwise, so +callers do not choose the variant directly. The v2 variant also writes the v1 ID for arrays that fit +the v1 format, so allowing only the v2 ID excludes the scheme. The v1 decimal variant constructs +only single-child arrays, so values too wide for one 64-bit child remain in the standard +uncompressed decimal representation. No declared edition permits a v2 ID yet, so a writer that +enforces editions writes only v1 for these encodings. [^decimal-availability]: No declared edition currently permits the multi-child format. The From 98119a57dac28a94ed2aa0c53984b21a75ff6c53 Mon Sep 17 00:00:00 2001 From: Connor Tsui Date: Fri, 23 Oct 2026 16:48:38 -0400 Subject: [PATCH 2/2] docs: use the decimal scheme as the example of scheme variants Drops the list of schemes with variants, which will change, and the frame-of-reference exception for child compression, which #10751 tracks as a bug. Signed-off-by: Connor Tsui --- docs/specs/versioning/design.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/specs/versioning/design.md b/docs/specs/versioning/design.md index 6f36efa3099..699048ec6bd 100644 --- a/docs/specs/versioning/design.md +++ b/docs/specs/versioning/design.md @@ -260,19 +260,19 @@ recording the minimum version. The default BtrBlocks compressor uses only the schemes whose output it can write. A wire ID is allowed when the session registers it, the enabled editions permit it, and the compressor's preset -does not exclude it. For example, the CUDA preset excludes the v2 IDs described below. The -compressor excludes a scheme if any wire ID that the scheme declares is not allowed. Child arrays go -through the same compressor, except that the frame-of-reference scheme always bit-packs its encoded -child with the v1 bit-packing variant. The writer still checks every serialized ID, so a forbidden -ID in any child fails the write. - -The decimal, bit-packing, and frame-of-reference schemes each have a v1 and a v2 variant. The -compressor uses the v2 variant when the v2 wire ID is allowed and the v1 variant otherwise, so -callers do not choose the variant directly. The v2 variant also writes the v1 ID for arrays that fit -the v1 format, so allowing only the v2 ID excludes the scheme. The v1 decimal variant constructs -only single-child arrays, so values too wide for one 64-bit child remain in the standard -uncompressed decimal representation. No declared edition permits a v2 ID yet, so a writer that -enforces editions writes only v1 for these encodings. +does not exclude it. For example, the CUDA preset excludes some v2 IDs. The compressor excludes a +scheme if any wire ID that the scheme declares is not allowed. Child arrays go through the same +compressor. The writer still checks every serialized ID, so a forbidden ID in any child fails the +write. + +A scheme can have variants that write different wire IDs. The compressor picks the variant from the +allowed wire IDs, so callers do not choose it directly. For example, the decimal scheme uses its v2 +variant, which splits values too wide for one 64-bit child into several children, only when +`vortex.decimal_byte_parts.v2` is allowed. Otherwise its v1 variant constructs only single-child +arrays, and values too wide for one child remain in the standard uncompressed decimal +representation. The v2 variant also writes the v1 ID for arrays that fit the v1 format, so allowing +only the v2 ID excludes the scheme. No declared edition permits `vortex.decimal_byte_parts.v2` yet, +so a writer that enforces editions writes only single-child decimal arrays. [^decimal-availability]: No declared edition currently permits the multi-child format. The