Skip to content

Read the Arrow layouts the engine now converts, and check run ends and list views - #730

Merged
evertlammerts merged 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:lift-arrow-importer-refusals
Oct 29, 2026
Merged

evertlammerts merged 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:lift-arrow-importer-refusals

Conversation

@evertlammerts

Copy link
Copy Markdown
Member

The Arrow importer refused three layouts the pinned engine converted wrong: a struct or union below an offset, only empty lists over a dictionary child, and a run end encoded child inside a fixed size list. The engine at fbd180f3a40 (duckdb/duckdb#27965, with the backport of #27626 and the struct offset fix) reads all three, and the first refusal hit ordinary routes: a sliced pyarrow Table with a struct of struct column, Table.from_batches over it, a Dataset of it, differently chunked columns.

@evertlammerts
evertlammerts added this pull request to the merge queue Oct 29, 2026
Merged via the queue into duckdb:v2.0-cyanoptera with commit 791d75b Oct 29, 2026
30 checks passed
@evertlammerts
evertlammerts deleted the lift-arrow-importer-refusals branch October 29, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant