Skip to content

Upgrade bundled arrow to ≥59.0.0 to fix panic on decimal128 from FFI #15

Description

@ianmcook

databow panics when a query returns a decimal128 column whose buffer is
8-byte but not 16-byte aligned:

arrow-buffer/src/buffer/scalar.rs: Memory pointer from external source (e.g, FFI)
is not aligned with the specified scalar type.

Repro

  1. Start Spark in Docker as described here.
  2. Run:
databow --driver spark \
  --uri "spark://admin@localhost:15002?auth_type=none&api=connect" \
  --query "SELECT CAST(19.99 AS DECIMAL(5,2)) AS p"

The ADBC Spark driver emits such buffers (spec-legal). The C++ ADBC manager
reads them fine; databow's Rust stack panics.

Cause / fix

databow bundles arrow-buffer 58.1.0. arrow-rs 59.0.0 fixes this by calling
align_buffers() in from_ffi (apache/arrow-rs#10030, closes #10028).
Upgrade to arrow ≥59.0.0.

Version

databow 0.1.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions