Repository navigation
[Python] Several APIs segfault when required Arrow object arguments are None #51043
Description
Summary
Several PyArrow APIs whose signatures require Arrow extension objects reject invalid types with TypeError, but accept None and then terminate the interpreter.
I would expect None to be rejected with TypeError, like the other invalid types, rather than causing a process crash.
I found these cases while fuzzing Python C extension modules.
Versions
PyArrow 25.0.1, CPython 3.12.3, Debian 12 x86_64, glibc 2.36
Reproducer
Run each command independently in a fresh process.
python -c 'import pyarrow.parquet as pq; pq.SortingColumn.from_ordering(None, [])'
python -c 'import pyarrow as pa; import pyarrow.dataset as ds; ds.FileSystemDataset([None], pa.schema([]), ds.ParquetFileFormat())'
python -c 'import pyarrow as pa; dictionary=pa.array([], type=pa.string()); buffers=[None, pa.py_buffer(b"")]; pa.DictionaryArray.from_buffers(None, 0, buffers, dictionary)'SortingColumn.from_ordering SIGSEGV (signal 11)
FileSystemDataset SIGSEGV (signal 11)
DictionaryArray.from_buffers SIGSEGV (signal 11)
ASan/UBSan result
I built PyArrow 25.0.0 from source with Clang 18 using ASan and UBSan instrumentation.
The first sanitizer failures were:
| API | First native failure |
|---|---|
SortingColumn.from_ordering() |
member call on a null arrow::Schema in parquet::arrow::ToParquetSchema() at cpp/src/parquet/arrow/schema.cc:1233 |
FileSystemDataset() |
member access through a null FileFragment extension pointer at build/_dataset.cpp:23011 |
DictionaryArray.from_buffers() |
member call on a null arrow::DataType in arrow::ArrayData::Make() at cpp/src/arrow/array/data.cc:118 |
ASan subsequently reports zero-page reads on the corresponding native paths and aborts each process with exit code 134.
Component(s)
Python
Activity
apacheGH-51043: Reject null dictionary arrays
marinelay commentedon Sep 8, 2026
I found three additional APIs that appear to have the same issue:
Each command terminates independently with SIGSEGV (signal 11) on PyArrow 25.0.0, CPython 3.12.3, Ubuntu 24.04 x86_64, and glibc 2.39.
I think that explicit None checks similar to those added in PR #51161 may be needed for these arguments as well.
apacheGH-51043: [Python] Validate required array buffer arguments
marinelay commentedon Sep 19, 2026
@1fanwang Thanks for working on these issues.
I found three more cases that appear to be outside the current changes:
Each command exits with SIGSEGV on PyArrow 25.0.1
Would you consider updating the PR to validate these
Noneinputs and add regression tests?apacheGH-51043: Reject null required PyArrow objects
apacheGH-51043: Reject null dictionary arrays
apacheGH-51043: [Python] Validate required array buffer arguments
apacheGH-51043: [Python] Reject None in more required object construc…
1fanwang commentedon Sep 24, 2026
Done in bb913b6.
apacheGH-51043: Reject null required PyArrow objects
apacheGH-51043: Reject null dictionary arrays
apacheGH-51043: [Python] Validate required array buffer arguments
apacheGH-51043: [Python] Reject None in more required object construc…