Skip to content

fix: Validate run ends, list view offsets and dictionary indices in unaligned buffers safely - #990

Open
pedrobtz wants to merge 1 commit into
apache:mainfrom
pedrobtz:fix-validate-full-unaligned
Open

pedrobtz wants to merge 1 commit into
apache:mainfrom
pedrobtz:fix-validate-full-unaligned

Conversation

@pedrobtz

@pedrobtz pedrobtz commented Oct 21, 2026 •

Copy link
Copy Markdown

#946 fixed ArrowArrayViewValidate() reading offsets buffers through aligned pointers (#945). Validation still does the same in three other places:

  • run ends, through ArrowArrayViewGetIntUnsafe(), in default validation (first and last run end) and full validation (the increasing check);
  • list view and large list view offsets and sizes, through data.as_int32[i] and data.as_int64[i], in full validation;
  • dictionary indices, through ArrowArrayViewGetIntUnsafe(), in full validation's range check.

Each is a misaligned load when the buffer isn't aligned to the element width. That is undefined behaviour, and UBSan reports it, for input the C Data interface allows.

This adds a static ArrowArrayViewGetIntUnaligned() to array.c. It reads the 16-, 32- and 64-bit integer types with memcpy and falls back to ArrowArrayViewGetIntUnsafe() for the others. The run-end and dictionary-index sites use it. The list view sites use ArrowBufferViewGetInt32Unsafe() and ArrowBufferViewGetInt64Unsafe() from #946.

The new ArrayViewTestValidateFullUnalignedIntegers test validates, at the full level, a dictionary-encoded int32 view with indices at an odd address. It does the same for a run-end-encoded view with odd-addressed run ends, and for list view and large list view with odd-addressed offsets and sizes. Without the fix, UBSan reports the misaligned load (inline_array.h:1160). With it, ctest is clean under UBSAN_OPTIONS=halt_on_error=1.

Found by fuzzing the IPC reader in the zuarrow R package, which vendors nanoarrow. A pre-0.15 message whose body buffer starts at an odd offset reached the dictionary index check through ArrowIpcArrayStreamReader.

ArrowArrayAppendStorageFromArrayViewElement() and ArrowArrayAppendStorageFromArrayViewRange() read integers, list view sizes and run ends the same way. They are not validation, so they are left for a separate change.

ctest passes in a plain Debug build and with ASan and UBSan with halt_on_error=1 (410 tests each). The changed files are formatted with the pinned clang-format 16.0.6.

🤖 Generated with Claude Code

…naligned buffers safely

Default and full validation read these through aligned pointers, which
is undefined behaviour for the unaligned buffers the C Data interface
allows. Follows the fix for offsets buffers in apache#946.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant