Repository navigation
Call scalar functions by name through the session - #10763
robert3005 wants to merge 3 commits into
Conversation
`ScalarFnPlugin` gains `bind_any`, which builds the function from options passed as `&dyn Any`, downcasting them to the function's own `ScalarFnVTable::Options` type and failing with an error naming that type otherwise. `ScalarFnSessionExt` uses it to look a function up in the session's scalar function registry by name and construct the call: - `scalar_fn(name, options)` builds the registered function - `call(name, options, args)` applies it to `args`, which are arrays (giving the lazy `ScalarFnArray`) or expressions (giving the expression), through the new `ScalarFnArg` trait A function without options takes `&EmptyOptions`. This lets code call functions that another crate registers with the session without depending on that crate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j Signed-off-by: Claude <noreply@anthropic.com>
Merging this PR will degrade performance by 20.31%4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing 
Footnotes
|
`lazy(name, options, children)` builds an expression node that holds only the function id and its options, so it needs neither a session nor the crate that defines the function. `Expression::resolve(session)` later replaces each such node with the function registered under that id, built from the options through `ScalarFnPlugin::bind_any`. A lazy node is a scalar function vtable, `LazyScalarFn`, like the foreign placeholder: it reports the target function's id so it displays and hashes like that function, accepts any arity, and fails to bind, execute, or serialize with an error asking for resolution. `LazyScalarFnOptions` type-erases the options behind a trait dispatching equality and hashing by concrete type, so any `ScalarFnVTable::Options` type fits. Unknown names, options of the wrong type, and arity mismatches are all reported by `resolve`. `ScalarFnSessionExt::scalar_fn_by_id` looks a function up by an already interned id; `scalar_fn(name, ...)` delegates to it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EAanvjKHbMGyQyN2nQAwU Signed-off-by: Claude <noreply@anthropic.com>
An `Expression::Scalar` node no longer holds a `ScalarFnRef`. It holds the id of the scalar function it calls, the function's options with their type erased (`ErasedOptions`), and its children. Building an expression therefore needs neither a session nor the crate that defines the function. `Expression::new` builds a node from an id and erased options; the vtable constructors (`new_expr`, `try_new_expr`) still check the arity up front. Binding resolves the function: `Expression::bind(dtype, session)` and `Scope`, which now carries the session, look each node's id up in the session's scalar function registry, build the function from the options through `ScalarFnPlugin::bind_any`, and check the options type and the number of children, producing a `BoundExpression` as before. Options read for an id the session did not register bind to the foreign placeholder. Consequences across the API: - `ScalarFnVTable::bind_options` takes `Option<&VortexSession>`: `Some` when an expression binds, so options that hold an expression, such as a lambda, can bind it; `None` when a bound node is rebuilt or a `ScalarFnArray` is built. `Lambda::bind` takes the session, and `Lambda::return_dtype` and `Lambda::is_infallible` are gone, since neither can be answered unbound. - `Expression::return_dtype` is gone; bind and read the dtype instead. - `ExprSerializeProtoExt::serialize_proto` takes the session, which serializes each function's options through the new `ScalarFnPlugin::serialize_any`. - `ArrayRef::apply` takes the session and binds first. `vx_array_apply` and the C++ `Array::apply` take a session too. - `Expression::is`, `as_opt`, `as_` and `contains` need `V: Default` to know the vtable's id, so every unit vtable derives `Default`. - Unbound expressions display generically, as `id(child, ..., opts=options)`, with numbered children in `display_tree`; binding gives each function its own formatting. Tests that assert on display bind first. - `DynamicComparison` and `RowCount` are registered by default, and the new `vortex_layout::initialize` registers `RowIdx`, since binding now needs every function an expression calls to be registered. `ScalarFnSessionExt` gains `scalar_fn_by_id`, `scalar_fn_plugin` and `resolve_scalar_fn`. The lazy expression modules from the previous commit are removed, since every expression is now built this way. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EAanvjKHbMGyQyN2nQAwU Signed-off-by: Claude <noreply@anthropic.com>
ScalarFnPlugingainsbind_any, which builds the function from options passed as&dyn Any, downcasting them to the function's ownScalarFnVTable::Optionstype and failing with an error naming that type otherwise.ScalarFnSessionExtuses it to look a function up in the session's scalar function registry by name and construct the call:scalar_fn(name, options)builds the registered functioncall(name, options, args)applies it toargs, which are arrays (giving the lazyScalarFnArray) or expressions (giving the expression), through the newScalarFnArgtrait