Repository navigation
Fix 64-bit index corruption when L and U use a user-supplied work array - #182
Merged
Merged
Conversation
With XSDK_INDEX_SIZE=64 and lwork > 0, xLUMemInit carved the int_t column pointer arrays xlsub, xlusup and xusub out of work[] with (n+1)*sizeof(int) bytes each, half of what they need. xusub overran into the lusup/ucol storage placed right after it and the numeric values overwrote the U column pointers, so xgssvx crashed in xPivotGrowth (or free() aborted) on the first factorization. The heap path (lwork == 0) was unaffected because it uses intMalloc. Size the three int_t arrays, the matching xuser_free in the shrink loop, and the lwork == -1 estimate with sizeof(int_t); xsup and supno remain int. Also fix the byte offsets in the (currently unused) xStackCompress, which computed the positions of the int_t lsub/usub blocks with sizeof(int) and the end of ucol with the integer size instead of the value size. Reproducer: build with -DXSDK_INDEX_SIZE=64 and run TESTING/s_test -t "LA" -l 1000000 -n 9 which segfaulted and now passes; the full ctest suite passes with both 32- and 64-bit indices. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjnVBdsXEM3c41ssQ3hFAM
xiaoyeli
pushed a commit
to xiaoyeli/superlu_base
that referenced
this pull request
Oct 20, 2026
Template counterpart of xiaoyeli/superlu#182. With XSDK_INDEX_SIZE=64 and lwork > 0, xLUMemInit carved the int_t column pointer arrays xlsub, xlusup and xusub out of work[] with (n+1)*sizeof(int) bytes each, half of what they need. xusub overran into the lusup/ucol storage placed right after it and the numeric values overwrote the U column pointers, so xgssvx crashed in xPivotGrowth on the first factorization. The heap path (lwork == 0) was unaffected because it uses intMalloc. Size the three int_t arrays, the matching xuser_free in the shrink loop, and the lwork == -1 estimate with sizeof(int_t); xsup and supno remain int. Also fix the byte offsets in the (currently unused) xStackCompress, which computed the positions of the int_t lsub/usub blocks with sizeof(int) and the end of ucol with the integer size instead of the value size. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjnVBdsXEM3c41ssQ3hFAM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With
-DXSDK_INDEX_SIZE=64, any call withlwork > 0(L and U stored in a user-suppliedwork[]array) corrupted the factors. Reproducer:This segfaulted in
sPivotGrowth(called fromsgssvx) on the first factorization;d_testaborted infree()instead.lwork == 0and the 32-bit index build were unaffected.Root cause
In
xLUMemInit(SRC/{s,d,c,z}memory.c), the user-memory branch carves the column pointer arraysxlsub,xlusupandxusubout ofwork[]with(n+1) * iwordbytes each, whereiword = sizeof(int). Those arrays areint_t, so with 64-bit indices each gets half the bytes it needs.xusuboverruns into thelusup/ucolblock placed immediately after it, and the numeric values in turn overwrite the U column pointers. The heap path usesintMalloc(n+1), which is sized correctly, which is why onlylwork > 0failed.Changes
In all four
xmemory.cfiles:xlsub,xlusupandxusubwithsizeof(int_t)(new localliword, matching the naming already used inxmemory_usage).xsupandsupnoremainint.sizeof(int_t)for thelsub/usubshare in the matchingxuser_freeof the shrink loop.lwork == -1estimate count the twointarrays and the threeint_tarrays separately, and thelsub/usubblocks withsizeof(int_t), so the reported requirement is sufficient on 64-bit builds.xStackCompress(currently not called anywhere) computed the byte offsets of theint_tlsub/usubblocks withsizeof(int)and the end ofucolwith the integer size instead of the value size. Corrected for consistency.Testing
s_test -t LA -l 1000000 -n 9lwork0 / 1000000 / 10000000, n 9 / 19, and theg20.ruaSP caselwork > 0ctest(24 tests)ctest(24 tests)The corresponding change still needs to be made in
SRC/xmemory.baseof the base repository; I can open that PR separately.Generated by Claude Code