[core] Add manifest sidecars for partition, row-id and bucket pruning - #9743

Draft

leaves12138 wants to merge 38 commits into

apache:masterfrom

leaves12138:codex/manifest-row-id-block-index

Commits on Sep 11, 2026

[core] Prune manifest blocks with row-id sidecar indexes
leaves12138
committed
34a12bd
[core] Coalesce manifest row-id index and block reads
leaves12138
committed
e9796ae

Commits on Sep 12, 2026

[test] Stabilize manifest sidecar regression tests
leaves12138
committed
d71c531

Commits on Sep 14, 2026

[core] Store manifest row-id sidecars in extra files
leaves12138
committed
3803f7d
[core][python] Preserve cancellation through manifest index fallback
leaves12138
committed
914897f
[core] Share manifest block indexes for partitions and row IDs
leaves12138
committed
00b99d7
[core] Add nullable bucket payloads to manifest block indexes
leaves12138
committed
7e28f90
[core] Remove compatibility with draft manifest index formats
leaves12138
committed
5dd40f3
[core] Rename manifest block indexes to sidecars
leaves12138
committed
bc8f465
[core] Unify encoding for manifest sidecar payloads
leaves12138
committed
06c4402
[core] Preserve manifest cache and explain statistics with sidecars
leaves12138
committed
95620cb
[core] Avoid redundant manifest sidecar interval decoding
leaves12138
committed
533ebe0
Fix minus
leaves12138
committed
56e362b
[core] Use a byte budget for manifest sidecar metadata
leaves12138
committed
98b516b
[core] Derive sidecar memory budget from manifest target size
leaves12138
committed
4215e41
[core] Short-circuit manifest sidecar predicate matching
leaves12138
committed
e7bbc3e
[core] Omit length fields for unavailable sidecar payloads
leaves12138
committed
853c45b
[core] Skip unused manifest sidecar payload decoding
leaves12138
committed
4e8776a
Fix minus
leaves12138
committed
d99bb14
[core] Share the manifest cache with sidecar reads
leaves12138
committed
a0e153b
[core] Reuse SingleSegments for cached sidecar bytes
leaves12138
committed
1306d7c
[core] Remove fixed manifest sidecar header and block limits
leaves12138
committed
cd6ce62
[core] Use 4 MiB manifest sidecar read buffers
leaves12138
committed
6383bc6
[python] Reuse local file cache for manifest sidecars
leaves12138
committed
c300ff5
[core] Reduce manifest sidecar read allocations
leaves12138
committed
fc77243
[core] Select manifest sidecar payloads from table metadata
leaves12138
committed
b2e63d3
[core] Require explicit manifest sidecar payload settings
leaves12138
committed
030fd58
[core] Default manifest sidecar switches to manifest sort
leaves12138
committed
a7841b0
Fix minus
leaves12138
committed
675393a
[core] Pass sidecar settings to the manifest factory constructor
leaves12138
committed
c00ac0c
[core] Enable sidecar row-id payloads only for data evolution tables
leaves12138
committed
404895e
[core] Derive manifest sidecar settings from table configuration
leaves12138
committed
75bdce9
[core] Handle sidecar budget boundaries without excess allocation
leaves12138
committed
c5cee0a
[core] Merge master and preserve manifest sidecar references
leaves12138
committed
af7158f
[test] Remove manifest large-header test with shaded Avro conflict
leaves12138
committed
510bcb3
[core] Fall back on manifest sidecar connection timeouts
leaves12138
committed
3193bf7
[core] Use a dedicated manifest sidecar cache segment
leaves12138
committed
321e33c
[core] Simplify manifest sidecar read failure handling
leaves12138
committed
82b2fa9