[Mandukhai-Alimaa]

Rationale for this change

• Closes [Format] Define Canonical Extension Type for FILE type #51143

As storage formats standardize file-level references, Arrow needs a canonical in-memory representation to preserve these semantics across IPC streams, the C Data Interface, Arrow Flight, and language bindings. Without a canonical extension type, file fields lose their semantic metadata during serialization.

[DISCUSS] thread in the mailing list

What changes are included in this PR?

Proposal and documentation for the arrow.parquet.file canonical extension type.

Are these changes tested?

N/A

Are there any user-facing changes?

Yes, this is an extension to the arrow format.

• GitHub Issue: [Format] Define Canonical Extension Type for FILE type #51143

[github-actions]

⚠️ GitHub issue #51143 has been automatically assigned in GitHub to PR creator.

[Mandukhai-Alimaa]

CC: @rok

[wgtmac]

I think we need to move forward with #50892, which is blocked by this proposal to some extent because we want interoperability.

WDYT? @alamb @emkornfield @mapleFU @pitrou @zeroshade

[wgtmac]

For this section, should we quote the Parquet spec verbatim with explicit attribution at the start, or keep the current wording and label it as “adapted from the Parquet FILE logical type specification”? Either way, it would help to make the relationship explicit so readers know whether this is a direct quotation or a restatement intended to preserve the same semantics.

[alamb]

unless there is a good reason to restate all the content, I think it should simply point at the Parquet spec

[alamb]

Thank you for working on this @Mandukhai-Alimaa

[Mandukhai-Alimaa]

Thanks for the feedback. I’ve removed the duplicated text and linked to the Parquet FILE spec instead.

[rok]

Thanks for working on this @Mandukhai-Alimaa! (and appologies for my slowness).

No form comments, only one on semantics.

[rok]

For FileType in Parquet offset will point to a region of a uri referenced file. However Parquet allows for the uri to be relative. How should Arrow users interpret that?

How about we specify that offset with relative uri is - unlinke in Parquet - meaningless.
We can stop there or also recommend that Parquet reader can attempt and record absolute uri, by checking for the file relative to the source parquet file's location.