[C++][Python] SEGFAULT when casting FixedSizeTensorArray to storage type then back to FixedSizeTensorArray

Describe the bug, including details regarding any error messages, version, and platform.

Minimum reproducible example:

import pyarrow

tensor_type = pyarrow.fixed_shape_tensor(pyarrow.int32(), [4])
storage_type = pyarrow.list_(pyarrow.int32(), 4)

py_list = [[1, 2, 3, 4], [10, 20, 30, 40], [100, 200, 300, 400]]
storage_arr = pyarrow.array(py_list, storage_type)
arr = pyarrow.ExtensionArray.from_storage(tensor_type, storage_arr)
arr.cast(
    storage_type,
).cast(
    tensor_type,
)

Component(s)

C++, Python

@judahrand

Thanks or reporting @judahrand. Do you get any debug information printed besides the segfault?
Issue seems similar to #37004, but not necessary the same.

The gdb traceback I get:

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
arrow::FixedSizeListArray::SetData (this=0x55555673d150, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/array_nested.cc:922
922	  ARROW_CHECK_EQ(list_type()->value_type()->id(), data->child_data[0]->type->id());
(gdb) l
917	
918	void FixedSizeListArray::SetData(const std::shared_ptr<ArrayData>& data) {
919	  ARROW_CHECK_EQ(data->type->id(), Type::FIXED_SIZE_LIST);
920	  this->Array::SetData(data);
921	
922	  ARROW_CHECK_EQ(list_type()->value_type()->id(), data->child_data[0]->type->id());
923	  DCHECK(list_type()->value_type()->Equals(data->child_data[0]->type));
924	  list_size_ = list_type()->list_size();
925	
926	  ARROW_CHECK_EQ(data_->child_data.size(), 1);
(gdb) bt
#0  arrow::FixedSizeListArray::SetData (this=0x55555673d150, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/array_nested.cc:922
#1  0x00007ffff55e876c in arrow::FixedSizeListArray::FixedSizeListArray (this=0x55555673d150, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/array_nested.cc:905
#2  0x00007ffff567e1d0 in std::_Construct<arrow::FixedSizeListArray, std::shared_ptr<arrow::ArrayData> const&> (__p=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_construct.h:109
#3  std::allocator_traits<std::allocator<void> >::construct<arrow::FixedSizeListArray, std::shared_ptr<arrow::ArrayData> const&> (__p=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/alloc_traits.h:635
#4  std::_Sp_counted_ptr_inplace<arrow::FixedSizeListArray, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::shared_ptr<arrow::ArrayData> const&> (__a=..., 
    this=<optimized out>) at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:604
#5  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<arrow::FixedSizeListArray, std::allocator<void>, std::shared_ptr<arrow::ArrayData> const&> (__a=..., 
    __p=<optimized out>, this=<optimized out>) at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:971
#6  std::__shared_ptr<arrow::FixedSizeListArray, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, std::shared_ptr<arrow::ArrayData> const&> (__tag=..., this=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:1712
#7  std::shared_ptr<arrow::FixedSizeListArray>::shared_ptr<std::allocator<void>, std::shared_ptr<arrow::ArrayData> const&> (__tag=..., this=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr.h:464
#8  std::make_shared<arrow::FixedSizeListArray, std::shared_ptr<arrow::ArrayData> const&> ()
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr.h:1010
#9  arrow::(anonymous namespace)::ArrayDataWrapper::Visit<arrow::FixedSizeListType> (this=<synthetic pointer>) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/util.cc:68
#10 arrow::VisitTypeInline<arrow::(anonymous namespace)::ArrayDataWrapper> (visitor=<synthetic pointer>, type=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/visit_type_inline.h:54
#11 arrow::MakeArray (data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/util.cc:338
#12 0x00007ffff5460f65 in arrow::ExtensionArray::SetData (this=0x555556450870, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/extension_type.cc:94
#13 0x00007ffff54612e4 in arrow::ExtensionArray::ExtensionArray (this=this@entry=0x555556450870, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/extension_type.cc:75
#14 0x00007ffff63f8d62 in std::_Construct<arrow::ExtensionArray, std::shared_ptr<arrow::ArrayData>&> (__p=0x555556450870)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_construct.h:109
#15 std::allocator_traits<std::allocator<void> >::construct<arrow::ExtensionArray, std::shared_ptr<arrow::ArrayData>&> (__p=0x555556450870)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/alloc_traits.h:635
#16 std::_Sp_counted_ptr_inplace<arrow::ExtensionArray, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::shared_ptr<arrow::ArrayData>&> (__a=..., 
    this=0x555556450860) at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:604
#17 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<arrow::ExtensionArray, std::allocator<void>, std::shared_ptr<arrow::ArrayData>&> (__a=..., __p=<optimized out>, 
    this=<optimized out>) at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:971
#18 std::__shared_ptr<arrow::ExtensionArray, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, std::shared_ptr<arrow::ArrayData>&> (__tag=..., this=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr_base.h:1712
#19 std::shared_ptr<arrow::ExtensionArray>::shared_ptr<std::allocator<void>, std::shared_ptr<arrow::ArrayData>&> (__tag=..., this=<optimized out>)
    at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr.h:464
#20 std::make_shared<arrow::ExtensionArray, std::shared_ptr<arrow::ArrayData>&> () at /home/joris/conda/envs/arrow-dev/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/shared_ptr.h:1010
#21 arrow::extension::FixedShapeTensorType::MakeArray (this=<optimized out>, data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/extension/fixed_shape_tensor.cc:205
#22 0x00007ffff567e36f in arrow::(anonymous namespace)::ArrayDataWrapper::Visit (type=..., this=<synthetic pointer>) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/util.cc:73
#23 arrow::VisitTypeInline<arrow::(anonymous namespace)::ArrayDataWrapper> (visitor=<synthetic pointer>, type=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/visit_type_inline.h:54
#24 arrow::MakeArray (data=...) at /home/joris/scipy/repos/arrow/cpp/src/arrow/array/util.cc:338
#25 0x00007ffff6fad240 in __pyx_f_7pyarrow_3lib_wrap_datum (__pyx_v_datum=...) at /home/joris/scipy/repos/arrow/python/build/temp.linux-x86_64-cpython-311/lib.cpp:121730
#26 0x00007fff48e7a21f in __pyx_pf_7pyarrow_8_compute_8Function_6call (__pyx_v_length=None, __pyx_v_memory_pool=<optimized out>, __pyx_v_options=<optimized out>, 
    __pyx_v_args=[<pyarrow.lib.FixedSizeListArray at remote 0x7fff4791a2c0>], __pyx_v_self=0x7fff47925f30)

I propose to close this issue as a duplicate of #37004 and track the progress there.

I asked Copilot to compare backtraces from both issues and went through the result. It does seem that, in both cases, when casting an extension type with list-based storage, the cast kernel produces ArrayData with the correct type but drops the child data. When the extension type then tries to wrap this into an ExtensionArray and calls MakeArray, it tries to construct a list array from that incomplete ArrayData and crashes.

Navigation Menu

Repository navigation

[C++][Python] SEGFAULT when casting FixedSizeTensorArray to storage type then back to FixedSizeTensorArray #41756

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[C++][Python] SEGFAULT when casting FixedSizeTensorArray to storage type then back to FixedSizeTensorArray #41756

Description

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Activity

rok commented on May 21, 2024

jorisvandenbossche commented on May 21, 2024

AlenkaF commented on May 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions