Skip to content

Commit 09702a4

Browse files
committed
Match Java's key bound when serializing deletion vector bitmaps
Java's RoaringPositionBitmap.readKey rejects keys above Integer.MAX_VALUE - 1, while the serializer allowed one more key, so PyIceberg could write a deletion vector Java cannot read.
1 parent ba4d889 commit 09702a4

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

‎pyiceberg/table/deletion_vector.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,8 +112,11 @@ def _serialize_bitmap(bitmaps: list[BitMap]) -> bytes:
112112
with io.BytesIO() as out:
113113
out.write(len(non_empty).to_bytes(8, "little"))
114114
for key, bitmap in non_empty:
115-
if key > MAX_JAVA_SIGNED:
116-
raise ValueError(f"Key {key} is too large, max {MAX_JAVA_SIGNED} to maintain compatibility with Java impl")
115+
# Java's RoaringPositionBitmap.readKey rejects keys above Integer.MAX_VALUE - 1
116+
if key > MAX_JAVA_SIGNED - 1:
117+
raise ValueError(
118+
f"Key {key} is too large, max {MAX_JAVA_SIGNED - 1} to maintain compatibility with Java impl"
119+
)
117120
out.write(key.to_bytes(4, "little"))
118121
# Run-length encode before serializing so run-heavy vectors (e.g. contiguous
119122
# deletes) stay compact, matching Java's BitmapPositionDeleteIndex.

0 commit comments

Comments
 (0)