Skip to content

Fix 64-bit index corruption when L and U use a user-supplied work array - #182

Merged
xiaoyeli merged 1 commit into
masterfrom
claude/epic-rubin-j04ke9
Oct 20, 2026
Merged

xiaoyeli merged 1 commit into
masterfrom
claude/epic-rubin-j04ke9

Conversation

@xiaoyeli

Copy link
Copy Markdown
Owner

Summary

With -DXSDK_INDEX_SIZE=64, any call with lwork > 0 (L and U stored in a user-supplied work[] array) corrupted the factors. Reproducer:

cmake -B build -DXSDK_INDEX_SIZE=64 -Denable_internal_blaslib=ON
cmake --build build
cd build/TESTING && ./s_test -t "LA" -l 1000000 -n 9

This segfaulted in sPivotGrowth (called from sgssvx) on the first factorization; d_test aborted in free() instead. lwork == 0 and the 32-bit index build were unaffected.

Root cause

In xLUMemInit (SRC/{s,d,c,z}memory.c), the user-memory branch carves the column pointer arrays xlsub, xlusup and xusub out of work[] with (n+1) * iword bytes each, where iword = sizeof(int). Those arrays are int_t, so with 64-bit indices each gets half the bytes it needs. xusub overruns into the lusup/ucol block placed immediately after it, and the numeric values in turn overwrite the U column pointers. The heap path uses intMalloc(n+1), which is sized correctly, which is why only lwork > 0 failed.

Changes

In all four xmemory.c files:

  • Allocate xlsub, xlusup and xusub with sizeof(int_t) (new local liword, matching the naming already used in xmemory_usage). xsup and supno remain int.
  • Use sizeof(int_t) for the lsub/usub share in the matching xuser_free of the shrink loop.
  • Make the lwork == -1 estimate count the two int arrays and the three int_t arrays separately, and the lsub/usub blocks with sizeof(int_t), so the reported requirement is sufficient on 64-bit builds.
  • xStackCompress (currently not called anywhere) computed the byte offsets of the int_t lsub/usub blocks with sizeof(int) and the end of ucol with the integer size instead of the value size. Corrected for consistency.

Testing

Build Before After
64-bit index, s_test -t LA -l 1000000 -n 9 segfault passes, 1158 tests
64-bit index, s/d/c/z testers, lwork 0 / 1000000 / 10000000, n 9 / 19, and the g20.rua SP case segfault or abort for lwork > 0 all pass
64-bit index, ctest (24 tests) – 100% pass
32-bit index, s/d/c/z testers and ctest (24 tests) pass pass

The corresponding change still needs to be made in SRC/xmemory.base of the base repository; I can open that PR separately.


Generated by Claude Code

With XSDK_INDEX_SIZE=64 and lwork > 0, xLUMemInit carved the int_t column
pointer arrays xlsub, xlusup and xusub out of work[] with (n+1)*sizeof(int)
bytes each, half of what they need. xusub overran into the lusup/ucol
storage placed right after it and the numeric values overwrote the U
column pointers, so xgssvx crashed in xPivotGrowth (or free() aborted)
on the first factorization. The heap path (lwork == 0) was unaffected
because it uses intMalloc.

Size the three int_t arrays, the matching xuser_free in the shrink loop,
and the lwork == -1 estimate with sizeof(int_t); xsup and supno remain
int. Also fix the byte offsets in the (currently unused) xStackCompress,
which computed the positions of the int_t lsub/usub blocks with
sizeof(int) and the end of ucol with the integer size instead of the
value size.

Reproducer: build with -DXSDK_INDEX_SIZE=64 and run
  TESTING/s_test -t "LA" -l 1000000 -n 9
which segfaulted and now passes; the full ctest suite passes with both
32- and 64-bit indices.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjnVBdsXEM3c41ssQ3hFAM
@xiaoyeli
xiaoyeli merged commit 85edda2 into master Oct 20, 2026
10 checks passed
xiaoyeli pushed a commit to xiaoyeli/superlu_base that referenced this pull request Oct 20, 2026
Template counterpart of xiaoyeli/superlu#182.

With XSDK_INDEX_SIZE=64 and lwork > 0, xLUMemInit carved the int_t column
pointer arrays xlsub, xlusup and xusub out of work[] with (n+1)*sizeof(int)
bytes each, half of what they need. xusub overran into the lusup/ucol
storage placed right after it and the numeric values overwrote the U
column pointers, so xgssvx crashed in xPivotGrowth on the first
factorization. The heap path (lwork == 0) was unaffected because it uses
intMalloc.

Size the three int_t arrays, the matching xuser_free in the shrink loop,
and the lwork == -1 estimate with sizeof(int_t); xsup and supno remain
int. Also fix the byte offsets in the (currently unused) xStackCompress,
which computed the positions of the int_t lsub/usub blocks with
sizeof(int) and the end of ucol with the integer size instead of the
value size.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjnVBdsXEM3c41ssQ3hFAM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants