[CurtHagenlocher]

What's Changed

For a dictionary-encoded extension type such as dictionary<int32, arrow.uuid>, Arrow C++ puts the value type's ARROW:extension:name on the dictionary child ArrowSchema. On the field itself, the same keys mean the extension's storage is the whole dictionary. CArrowSchemaExporter put a value type's extension metadata on the field. As a result, pyarrow rejected such fields (Invalid storage type for UuidType: dictionary<...>), and the importer didn't resolve them in either direction (#473).

Export

• When a dictionary's value type is an ExtensionType, its extension metadata is written on the dictionary child. If the field metadata names that same extension, which is how the IPC reader leaves it, those keys move to the child as well. Other field metadata stays on the field.
• Unresolved extension metadata on a dictionary field, where the value type isn't an ExtensionType, stays on the field. There, it describes an extension over the whole dictionary, which is how Arrow C++ exports one.
• ExportType now writes extension metadata when given an ExtensionType, as Arrow C++ does. The dictionary child is exported through it. Behavior change: ExportType(GuidType.Default, ptr) previously produced a bare w:16 with no extension name.

Import

• The dictionary child's extension metadata resolves the value type against the registry. It isn't copied to the field, where it would mean something else. Like Arrow C++, an unresolved value type's extension metadata is dropped.
• Extension metadata on a dictionary field is resolved against the whole dictionary type only, as in Arrow C++. Behavior change: schemas exported by earlier versions of this library put a value type's extension metadata there, so importing one gives a dictionary of the storage type. C Data schemas aren't persisted, so this only affects exchanges between different Apache.Arrow versions in the same process.

Tests

• .NET: where export puts the metadata, including field-level keys from IPC. Unresolved field-level metadata stays on the field across an import and re-export without a registry. Also schema round trips with and without a registry, field-level metadata on a dictionary not being applied to the values, and a DictionaryArray of GuidArray round trip.
• pyarrow: .NET → pyarrow gives dictionary<values=extension<arrow.uuid>, indices=int32> with equal values, and pyarrow → .NET gives a DictionaryArray whose Dictionary is a GuidArray.

IPC has the same ambiguity, and Apache.Arrow's IPC reader resolves it the opposite way from Arrow C++. That's filed separately as #475.

Closes #473.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[Copilot]

🟡 Changes recommended

Re-exporting an unresolved dictionary-storage extension moves its metadata to the values, changing its meaning.

1 open issue

• Preserve unresolved dictionary extension metadata ownership · New

What changed in this PR

Aligns dictionary value extension metadata with the C Data Interface placement used by Arrow C++, addressing #473.

Changes:

• Exports value-extension metadata on dictionary children.
• Resolves child extensions during import and supports legacy field-level placement.
• Adds .NET round-trip and bidirectional pyarrow interoperability tests.

File	Description
test/​Apache.Arrow.Tests/​ExtensionTypeTests.cs	Tests metadata placement and dictionary extension round trips.
test/​Apache.Arrow.Tests/​CDataInterfacePythonTests.cs	Tests dictionary UUID exchange with pyarrow.
src/​Apache.Arrow/​C/​CArrowSchemaImporter.cs	Resolves dictionary child extensions and preserves metadata.
src/​Apache.Arrow/​C/​CArrowSchemaExporter.cs	Routes extension metadata to the appropriate schema node.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

A test leaks the unmanaged CArrowSchema allocation and should free it after import.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Test leaks unmanaged CArrowSchema allocation

test/​Apache.Arrow.Tests/​CDataInterfacePythonTests.cs:1319

ImportField invokes the foreign schema's release callback but does not free the CArrowSchema allocation itself (the API docs require callers to call CArrowSchema.Free afterward). This test frees cArray but never frees cSchema, leaking the unmanaged schema allocation on every successful run; add CArrowSchema.Free(cSchema) after importing (ideally in a finally).

1 resolved since last review

• Preserve unresolved dictionary extension metadata ownership

_{Review effort: Lite}

[Copilot]

🟢 Approval recommended

The changes are covered by interoperability and round-trip tests; only a minor documentation nit remains.

No open issues 🎉

_{Review effort: Lite}