[plusplusjiajia]

Purpose

A Format Table has no partition mark done today: PartitionMarkDoneAction.createActions and the mark_partition_done procedures only accept a FileStoreTable. Users moving Hive-style tables to Format Tables often have downstream jobs that wait for a _SUCCESS file in the partition directory.

This PR supports the success-file action for Format Tables:

• FormatTableCommit#committedPartitions() returns the partitions the last successful commit wrote, including the full static partition of an INSERT OVERWRITE that wrote no rows.
• With partition.end-input-to-done, FormatTableCommit deletes the _SUCCESS of every partition before an append, overwrite or truncate changes it, including the partitions a static-prefix or table overwrite clears without writing to, so a stale marker is never seen next to changed data. A marker that cannot be deleted fails a write commit before any of its data changes; TRUNCATE handles partitions one at a time, as it already does, so it fails before the partition whose marker could not be deleted. A partition an overwrite only clears is not marked again. Without partition.end-input-to-done, writes leave markers alone, so procedure users should enable it too; this is documented.
• FormatTableSuccessFileMarkDoneAction writes an empty _SUCCESS in the partition directory, as Hadoop and Spark jobs do (Paimon's own tables keep their JSON content), following format-table.partition-path-only-value. With catalog-managed partitions, a partition registered at a custom location, or not registered, is skipped with a warning, so the file is only written inside the table directory. The registrations of all partitions are looked up in one listPartitionsByNames call (batched by the manager), through a new PartitionMarkDoneAction#markDone(List<String>) that defaults to marking one by one.
• PartitionMarkDoneAction#createActions gets a FormatTable overload. Only success-file is supported; other actions fail with a clear message.
• Spark: with partition.end-input-to-done = true, a Format Table write marks the partitions it committed. The actions are created before committing, so an unsupported action fails the write before any data is published. Once the commit succeeds, a later abort (for example after marking fails) no longer deletes the committed data.
• Spark and Flink mark_partition_done (procedure and Flink action) support Format Tables.
• Only the default format-table.implementation = paimon marks partitions. With engine, Spark writes through its own file source and, as before this PR, ignores partition.end-input-to-done; this is documented.

A partition is assumed to have one writer at a time: a write marks the partition done when that write finishes, and writers are not coordinated, so with concurrent writes to one partition the first to finish marks it while another may still be publishing. This is documented rather than enforced, like the existing requirement that overwrites be externally serialized; no lock can tell a single writer when every writer of a partition is done.

Flink writes to a Format Table do not mark partitions done in this PR (they only delete the markers of the partitions they change): each sink subtask commits on its own in close(), so there is no point where every subtask has finished. That needs a global committer and is left for a follow-up.

Tests

• FormatTableSuccessFileMarkDoneActionTest: writes an empty file over an existing one, value-only and escaped directories, one catalog lookup for many partitions with custom-location and unregistered partitions skipped, invalid arguments.
• FormatTableCommitMarkDoneTest: committedPartitions for append / empty static overwrite / prefix overwrite / failed commit; prefix, table and dynamic overwrites, appends, partition and table truncation delete the _SUCCESS of the partitions they change; nothing is deleted when disabled; a refused delete fails before data is deleted.
• FormatTablePartitionMarkDoneTest (Spark, with and without catalog-managed partitions): writes mark their partitions, a prefix overwrite drops the marker of a partition it clears and recreates the one it writes, an empty static overwrite is marked; the procedure; an unsupported action fails the write before any data and the procedure.
• FormatTableITCase#testMarkPartitionDoneProcedure (Flink).
• Existing FormatTableCommit*Test, FormatTableScanTest, MarkPartitionDoneProcedureTest, PaimonFormatTableTest, MarkPartitionDoneActionITCase pass.

[plusplusjiajia]

@JingsongLi @sundapeng Could you help review this PR? Thanks!

[JingsongLi]

Could you provide a detailed description of the complete end-to-end (E2E) workflow? What happens after the files are generated—how are they used? Moreover, the access permissions for this file are issued by the Catalog; what do users employ to access the file—PVFS?