Skip to content

Fix RUSTSEC-2026-0197: swap_columns/swap_elements trigger UB when indices alias#566

Open
haraldreingruber-dedalus wants to merge 1 commit into
rustgd:masterfrom
haraldreingruber-dedalus:UCC-fix-swap-columns-ub
Open

Fix RUSTSEC-2026-0197: swap_columns/swap_elements trigger UB when indices alias#566
haraldreingruber-dedalus wants to merge 1 commit into
rustgd:masterfrom
haraldreingruber-dedalus:UCC-fix-swap-columns-ub

Conversation

@haraldreingruber-dedalus

@haraldreingruber-dedalus haraldreingruber-dedalus commented Jul 8, 2026

Copy link
Copy Markdown

Fixes #565.

Problem

Matrix2::swap_columns, Matrix3::swap_columns, Matrix4::swap_columns, and the corresponding swap_elements implementations all used:

unsafe { ptr::swap(&mut self[a], &mut self[b]) };

When a == b, this creates two mutable references (&mut self[a] and &mut self[b]) to the same memory location. This violates Rust's aliasing guarantees and constitutes undefined behaviour under Miri (Stacked Borrows), even though the call is made from safe Rust.

Additionally, even for distinct indices, ptr::swap(&mut self[a], &mut self[b]) is flagged by Miri: both calls go through IndexMut::index_mut(&mut self, …), so the second call creates a new unique reborrow of all of self, invalidating the tag from the first call — a Stacked Borrows violation even though the pointed-to bytes don't overlap.

The same class of bug existed in:

  • Array::swap_elements in structure.rs (UB when i == j)
  • Matrix{2,3,4}::swap_elements (UB when both indices share the same column, i.e. ac == bc), because two exclusive borrows of the same column vector are created simultaneously

Fix (this branch)

Since S: BaseFloat implies S: Copy, column vectors and scalar elements are Copy. All unsafe is removed and the swap is done via a temporary variable:

fn swap_columns(&mut self, a: usize, b: usize) {
    if a == b {
        return;
    }
    let tmp = self[a];
    self[a] = self[b];
    self[b] = tmp;
}

This is fully clean under Miri. An alternative, minimal fix using early-returns only (keeping ptr::swap for distinct indices) is available on branch fix-swap-columns-early-return — that branch resolves the originally reported aliasing UB but is still flagged by Miri (Stacked Borrows) for the distinct-index path.

Tests

Added tests in tests/matrix.rs covering all three matrix sizes for:

  • swap_columns with equal indices (was UB, now no-op)
  • swap_columns with distinct indices (correct swap still works)
  • swap_elements with the same position (was UB, now no-op)
  • swap_elements with the same column and different rows (was potentially UB, now correct)

Note

These tests verify correctness (the operations produce the right values) but they do not catch the soundness bug in a regular cargo test run. Because ptr::swap of a location with itself happens to leave the value unchanged, the value-equality assertions pass with or without the fix. The bug can only be reliably detected by running the tests under Miri (cargo +nightly miri test).

Feel free to remove the new tests if they don't seem valuable enough without Miri coverage in CI.


Note

This fix was developed with the assistance of GitHub Copilot. While investigating the reported swap_columns UB, similar aliasing issues were also found and fixed in Array::swap_elements and Matrix*::swap_elements (same-column case), which were not explicitly called out in the original issue.

@haraldreingruber-dedalus haraldreingruber-dedalus force-pushed the UCC-fix-swap-columns-ub branch 2 times, most recently from 718d899 to dcf3da7 Compare July 8, 2026 12:08
…ces alias

When swap_columns is called with a == b, the previous implementation created
two mutable references (&mut self[a] and &mut self[b]) to the same memory
location (aliased &mut UB). Additionally, even for distinct indices a != b,
using ptr::swap(&mut self[a], &mut self[b]) is UB under Miri (Stacked
Borrows): the second IndexMut call creates a new unique reborrow of all of
self, invalidating the tag from the first call.

The same class of bug existed in:
- Array::swap_elements (structure.rs)
- Matrix{2,3,4}::swap_columns
- Matrix{2,3,4}::swap_elements

Fix: since S: BaseFloat implies S: Copy, column vectors and scalar elements
are Copy. We can swap entirely without unsafe by using a temporary variable:

    let tmp = self[a];
    self[a] = self[b];
    self[b] = tmp;

This avoids creating multiple mutable references simultaneously and requires
no unsafe code at all. Verified clean under Miri.

Tests added in tests/matrix.rs covering all three matrix sizes for:
- swap_columns with equal indices (no-op, was UB)
- swap_columns with distinct indices (correct swap)
- swap_elements with same position (no-op, was UB)
- swap_elements with same column, different rows

Note: the tests verify correctness but do not catch the soundness bug in a
regular cargo test run. Use cargo +nightly miri test to detect aliasing UB.

Fixes: rustgd#565

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@haraldreingruber-dedalus haraldreingruber-dedalus changed the title Fix soundness issue: swap_columns/swap_elements trigger UB when indices alias Fix RUSTSEC-2026-0197: swap_columns/swap_elements trigger UB when indices alias Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Soundness issue: Matrix2/Matrix3/Matrix4 swap_columns can trigger UB in Safe Rust when a == b

1 participant