[RLC] Case 3 — exclude relocation (dataChange=false) files from the append-conflict check #6

Part of the row-level-concurrency umbrella (#3, Case 3). Implemented in #10; opened upstream as
delta-io#7331.

Problem — a long-standing gap (unchanged since 2021)

WinningCommitSummary classifies the winner's added files by the isBlindAppend commitInfo flag
only — never per-file dataChange:

val changedDataAddedFiles = if (isBlindAppendOption.getOrElse(false)) Seq() else addedFiles

OPTIMIZE's commitInfo.isBlindAppend = false (it reads files), so its dataChange=false compacted
outputs land in changedDataAddedFiles and are treated as "changed data the loser should have
read." A concurrent non-blind writer (one that read the table → has read predicates, commits at
WriteSerializable) then hits ConcurrentAppendException against OPTIMIZE's files — even though
OPTIMIZE changed no data. (A pure blind INSERT escapes via the blind-append early-return.)

git blame confirms this classification is unchanged since 2021-07-07 (3fa6bcaa0); the only later
touches were a mechanical rename (2023) and a dataChange-consistency validation (2026,
delta-io#6937 / delta-io#6969) — not a fix.

Fix

Exclude dataChange=false added files from changedDataAddedFiles (they carry the same logical rows
the loser already saw). The per-commit "all FileActions share one dataChange" invariant
(delta-io#6937 / delta-io#6969) means an OPTIMIZE commit is uniformly dataChange=false, so the fix
gates cleanly on the commit-level flag. Gated by
spark.databricks.delta.conflictDetection.excludeNoDataChangeAddedFiles.enabled (internal, default
off).

Correctness & tests

• Excluding dataChange=false files is safe: they introduce no net-new logical rows for the loser to
  have read.
• Tests: OPTIMIZE (winner) vs a non-blind append/writer (loser) → no conflict (today:
  ConcurrentAppendException); OPTIMIZE vs a genuine data-changing conflict on the same file → still
  conflicts.

Notes

Same idea as suppressing an OPTIMIZE-style relocation from the append check, but keyed on the per-file
dataChange flag rather than an operation name — the principled, general version, independent of
row-level concurrency (Case 2 dropped its op-type suppression precisely because dataChange / stats
are the sound key). Opened upstream as delta-io#7331.