Skip to content

Corrupt ALP-RD, RLE and RunEnd metadata can panic a file read #10756

Open

Description

@connortsui20

What happened?

Several deserialize implementations trust their stored metadata, so a corrupt or crafted file can make a read panic instead of returning an error. The clearest case is ALP-RD: ALPRD::deserialize slices the stored dictionary with the stored dict_len without a bounds check (encodings/alp/src/alp_rd/array.rs:184).

Steps to reproduce

Call ALPRD.deserialize for an f32 array of length 1 with two children (buffer![0u16] and buffer![0u32]) and this metadata:

ALPRDMetadata {
    right_bit_width: 32,
    dict_len: 8,
    dict: vec![0, 1],
    left_parts_ptype: PType::U16 as i32,
    patches: None,
}
panicked at encodings/alp/src/alp_rd/array.rs:185:13:
range end index 8 out of range for slice of length 2

Expected: a Serde error.

Environment

develop at 3974ee069c, run as a unit test in vortex-alp.

Additional context

Other read paths can panic for the same reason. These come from reading the code and were not run:

  • ALP-RD accepts a dictionary with more than MAX_DICT_SIZE (8) entries, and the alp crate panics on such a dictionary when it decodes (alp-0.0.4/src/alp_rd/mod.rs:373).
  • ALP-RD does not check the left-part codes against the dictionary length. scalar_at (encodings/alp/src/alp_rd/ops.rs:42) and the patched decode path index the dictionary with each code.
  • ALP-RD checks only that right_bit_width fits a u8, not that it is below the bit width of the float, so the decode shift can overflow (ops.rs:54, :63).
  • ALP-RD accepts any unsigned left_parts_ptype, but execute reads the left parts with into_buffer_mut::<u16>() (array.rs:285, :297).
  • RLE calls dtype.as_ptype() in deserialize (encodings/fastlanes/src/rle/vtable/mod.rs:172), which panics for a non-primitive dtype.
  • RunEnd converts num_runs and offset with vortex_expect (encodings/runend/src/array.rs:156, :160), which panics on 32-bit targets for values above usize::MAX.

Most of these are cheap metadata checks in deserialize or validate. The code bound for ALP-RD needs either a scan of the left parts or a checked lookup when decoding. Found while reviewing #10755.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA bug issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions