You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
FoR scheme bit-packs its child outside the allowed schemes #10751
FoRScheme::compress bit-packs its encoded child by calling BITPACKING_V1.compress directly (vortex-btrblocks/src/schemes/integer/for_.rs:220). The child skips the compressor's scheme selection, and FoRScheme::produced_encodings declares only fastlanes.for (and fastlanes.for.v2), not the bit-packing IDs that the child uses. The compressor keeps a scheme only when every ID that it declares is allowed (vortex-btrblocks/src/builder.rs, allowed_schemes), so this check misses the bit-packed child.
From reading the code, this has two effects:
If fastlanes.bitpacked is not allowed, or the bit-packing scheme is excluded with exclude_schemes, the FoR scheme still writes a bit-packed child. The writer then rejects the forbidden ID, and the write fails, although the compressor could have picked another scheme. With experimental patches enabled, the same applies to vortex.patched.
When fastlanes.bitpacked.v2 is allowed, the FoR child still uses the v1 bit-packing variant.
Expected: FoR declares the IDs that its child can write, or compresses the child through the selected schemes.
Environment
develop at 3974ee069c. Found by reading the code, not reproduced by a run.
Additional context
Found while updating the compression section of the versioning docs (#10747).
@connortsui20 correction to this -- we should instantiate our schemes rather than having them be static for API design reasons rather than correctness. We can still refine schemes correctly within the status quo
Activity
connortsui20 commentedon Oct 23, 2026
Talked offline with @mhk197, we likely need stateful schemes that have options, otherwise we cannot refine schemes correctly.
So it seems like there is more work to be done here.
docs: use the decimal scheme as the example of scheme variants
mhk197 commentedon Oct 23, 2026
@connortsui20 correction to this -- we should instantiate our schemes rather than having them be static for API design reasons rather than correctness. We can still refine schemes correctly within the status quo