Variant support for vortex-datafusion #8117

This is a tracking issue for work adding support for the Variant type to the vortex-datafusion crate.

Steps

• Apache DataFusion support is currently developed in the datafusion-variant repo, while it stabilizes and develops we can integrate it with our own implementation.

Unresolved questions

At this point in time, its unclear to me when or how will this effort be merged into the upstream datafusion repo (see issue below).
There are also ongoing efforts to improve DataFusion's support for all canonical Arrow extensions types, which will make this feature easier.

Relevant issues

Some relevant issues from other repos:

• variant_get_array: Initial implementation datafusion-contrib/datafusion-variant#39
• epic: Initial implementation of this crate datafusion-contrib/datafusion-variant#2
• variant_get_array: Initial implementation datafusion-contrib/datafusion-variant#39
• Support Mapping SQL Types to the Canonical Extension Types apache/datafusion#21626