feat(compaction): compact fragments over an overlay-count limit

[wjones127]

A minimal, first slice of overlay-aware compaction (OSS-1326), stacked on #7536 (OSS-1324, will/oss-1324-take-can-read-overlays) — review against that base.

What it does

Adds a max_overlays_per_fragment: Option<usize> option to CompactionOptions. When a fragment carries more than this many data overlay files, the planner marks it CompactItself, so compact_files fully rewrites it into a fresh fragment with its overlays and deletions materialized into the base data. A fragment at or below the limit is not a candidate on this basis.

The read side needs no new machinery: compaction already scans each fragment through the normal read path, which (per #7536) resolves overlays and applies deletions, so the merged post-image is what gets written to the new base file.

Index correctness

A full rewrite removes the overlays, and with them the staleness signal the query path uses to mask an index older than an overlay. So the Rewrite commit now drops the rewritten fragment from the coverage of any index left stale by those overlays — field-aware (only indices covering a field an overlay actually supplied) and version-gated (overlay.committed_version > index.dataset_version). Those rows fall back to a flat scan until reindex; an index is never left serving stale values. Non-stale indices (built at/after the overlay, or on un-overlaid fields) are remapped normally.

This reuses the existing Operation::Rewrite path — no new persisted operation or conflict-matrix change.

Scope / follow-ups

Only the "fully compact the fragment" strategy is implemented. Overlay→overlay merge, in-place overlay→base folds (column rewrite preserving row addresses), and rebuild-in-place index reconciliation are left as follow-ups.

Tests

• Planner/e2e (optimize.rs): overlay count over the limit triggers a full compaction (fresh single-file fragment, overlays cleared, values materialized); at-or-below limit is a no-op; deletions are materialized alongside overlays; a stale scalar index is dropped from the compacted fragment's coverage and the indexed query stays correct.
• Unit (transaction.rs): prune_overlay_stale_fields_from_indices is field-aware and version-gated (stale index dropped; index at/after the overlay kept; index on an un-overlaid field untouched).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added an optional setting to compact fragments when they accumulate too many data overlays.
  • Overlay-triggered compaction now materializes updates and deletions into fresh, streamlined data.

• Bug Fixes

  • Improved query accuracy after compaction by preventing stale indexes from returning outdated values.
  • Ensured compacted fragments are correctly removed from outdated index coverage while preserving valid indexes.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b6b50c7c-9e63-4ed4-8f3b-6a910d3338b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch will/oss-1326-min-overlay-compaction

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/lance/src/dataset/optimize.rs`:
- Around line 8074-8106: Update the id_val_map helper to use
scanner.try_into_batch() instead of try_into_stream().try_collect(), then
process the returned single RecordBatch directly while preserving the existing
id-to-value mapping behavior.
- Around line 7946-7969: Update the test helper create_base_dataset to construct
the batch with arrow_array’s record_batch! macro, replacing the manual
ArrowSchema, Arc, and RecordBatch::try_new setup. Preserve the existing id and
val columns and values, then continue using the resulting batch with
RecordBatchIterator and Dataset::write unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4b224d4d-3ae2-4788-a9e7-86c9335b97f8

📥 Commits

Reviewing files that changed from the base of the PR and between a1bce9a and e78fadf.

📒 Files selected for processing (2)

• rust/lance/src/dataset/optimize.rs
• rust/lance/src/dataset/transaction.rs