Skip to content

Implement RFC 018 common scalar catalog#54

Draft
dannymeijer wants to merge 4 commits into
mainfrom
feature/35-rfc018-common-scalars
Draft

Implement RFC 018 common scalar catalog#54
dannymeijer wants to merge 4 commits into
mainfrom
feature/35-rfc018-common-scalars

Conversation

@dannymeijer
Copy link
Copy Markdown
Collaborator

@dannymeijer dannymeijer commented May 30, 2026

Summary

Completes the remaining RFC 018 common scalar catalog work after the earlier math slice. This adds registry-backed math, string, text encoding, regex, date/time, and compatibility alias helpers, keeps Substrait mappings registry-owned, and adds DataFusion adapter UDFs only where the first backend does not expose a direct API-level equivalent for the InQL contract.

Follow-up review polish in the latest commit keeps concat(...) and concat_ws(...) as variadic helper surfaces, removes redundant DataFusion adapter UDF wrapper code, and tightens regex docs so the portable contract is Rust-regex-compatible rather than DataFusion-owned.

Type of change

  • Bug fix
  • New feature
  • Refactor / maintenance
  • Documentation
  • CI / tooling
  • RFC (adds/updates docs/rfcs/*)

Area(s)

  • Package & tests
  • Specification (RFCs)
  • Documentation
  • Automation & repo config
  • Other

Key details

  • User-facing behavior: authors can now use the full RFC 018 scalar catalog: remaining math helpers, common string helpers, text encoding helpers, regex helpers, date/time helpers, and compatibility aliases such as substr, ucase, lcase, dateadd, datediff, and safe_cast.
  • Internals: helpers register through the existing decorator-driven function registry, declare Substrait extension mappings, and execute through DataFusion APIs/UDFs without making DataFusion the semantic owner. RFC 013 is moved to In Progress and RFC 014/015/018 metadata is cleaned up.
  • Risks: date/time and regex behavior is intentionally documented to the portable contract; broader semi-structured/binary logical values remain owned by RFC 026, not hidden in RFC 018.

Testing / verification

  • make fmt
  • make fmt-check
  • make test-style
  • make registry-metadata
  • make build
  • make test
  • make smoke-consumer
  • make pre-commit
  • Manual verification described below

Manual verification notes:

  • Ran all successful commands with /Users/danny/Development/encero/incan/target/debug/incan from Incan release/v0.3 / 0.3.0-rc30.
  • Focused checks passed: incan test tests/test_common_scalar_functions.incn, incan test tests/test_function_registry.incn, incan test tests/test_substrait_plan.incn, and incan test tests/test_session_projection.incn.
  • make test passed: 194 tests.
  • make smoke-consumer INCAN=/Users/danny/Development/encero/incan/target/debug/incan passed with network access for crates.io resolution.
  • make pre-commit passed, including format, test-style, registry metadata, build, and the 194-test suite.
  • git diff --check passed.

Docs impact

  • No docs changes needed
  • Docs updated
  • Docs follow Divio intent (tutorial/how-to/reference/explanation) where applicable

If docs updated:

  • Link(s): docs/rfcs/018_common_scalar_function_catalog.md, docs/rfcs/013_function_catalog_program.md, docs/language/reference/functions/index.md, docs/release_notes/v0_1.md

Checklist

  • I kept public docs user-focused and moved internals to contributing docs when appropriate
  • I avoided duplicating canonical install/run instructions in multiple places
  • I added/updated tests where it materially reduces regressions

Closes #35.

@incan-triage-bot incan-triage-bot Bot added documentation Improvements or additions to documentation package Library source, tests, incan.toml specification docs/rfcs/ normative RFCs labels May 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation package Library source, tests, incan.toml specification docs/rfcs/ normative RFCs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC 018: Common scalar function catalog

1 participant