Repository navigation
Move scalar functions out of vortex-array into vortex-functions - #10762
robert3005 wants to merge 1 commit into
Conversation
vortex-array keeps the scalar function machinery (the `ScalarFnVTable` trait,
binding, reduction, execution, the session registry and kernel registry) and
the few functions its own arrays, validity and expressions are built on:
literal, get_item and pack, is_null, is_not_null, not and the binary
operators, and cast, mask and fill_null, which canonical arrays apply to their
own children. That module is renamed from `scalar_fn::fns` to
`scalar_fn::builtin`, matching `ArrayBuiltins`. Every other scalar function
moves to the new `vortex-functions` crate:
- byte_length, case_when, ext_storage, list_length, list_sum, merge,
replace_time_zone, select, dynamic, stat, between, like, list_contains,
variant_get, is_nan and zip, with their expression constructors in
`vortex_functions::expr` (and `expr::bound`)
- the kernels and parent-reduce rules for vortex-array's encodings, which
`vortex_functions::initialize` now registers with a session's kernel
registry instead of the encodings' static rule sets
- the `ArrayBuiltins`/`ExprBuiltins` methods for the moved functions, which
become the `ArrayFunctions`/`ExprFunctions` extension traits
`apply_reduce_parent_rule` lets a crate register a typed parent-reduce rule for
an encoding it does not own. `ArrayHaystack`, `LazyBitBufferBuilder`,
`DeduplicatedBuffers::{extend_from_iter, finish}`, `ListScalar::element_values`,
`is_not_null_node`, `build_row_comparator` and `merge_typed_scalar_as_variant`
become public for the moved kernels.
The umbrella crate re-exports the moved constructors under `vortex::expr` and
the crate as `vortex::functions`.
The CodSpeed workflow builds `vortex-functions` in the main-library shard, so
the moved benchmarks stay measured, and picks up `vortex-onpair` and
`vortex-arrow`, whose benchmarks no shard built.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j
Signed-off-by: Claude <noreply@anthropic.com>
714ed9d to
fd2bbf1
Compare
Merging this PR will regress 7 benchmarks4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
vortex-array keeps the scalar function machinery (the
ScalarFnVTabletrait, binding, reduction, execution, the session registry and kernel registry) and the few functions its own arrays, validity and expressions are built on: literal, get_item and pack, is_null, is_not_null, not and the binary operators, and cast, mask and fill_null, which canonical arrays apply to their own children. That module is renamed fromscalar_fn::fnstoscalar_fn::builtin, matchingArrayBuiltins. Every other scalar function moves to the newvortex-functionscrate:vortex_functions::expr(andexpr::bound)vortex_functions::initializenow registers with a session's kernel registry instead of the encodings' static rule setsArrayBuiltins/ExprBuiltinsmethods for the moved functions, which become theArrayFunctions/ExprFunctionsextension traitsapply_reduce_parent_rulelets a crate register a typed parent-reduce rule for an encoding it does not own.ArrayHaystack,LazyBitBufferBuilder,DeduplicatedBuffers::{extend_from_iter, finish},ListScalar::element_values,is_not_null_node,build_row_comparatorandmerge_typed_scalar_as_variantbecome public for the moved kernels.