Repository navigation
Fix 64-bit index corruption when L and U use a user-supplied work array - #4
Merged
Merged
Conversation
Template counterpart of xiaoyeli/superlu#182. With XSDK_INDEX_SIZE=64 and lwork > 0, xLUMemInit carved the int_t column pointer arrays xlsub, xlusup and xusub out of work[] with (n+1)*sizeof(int) bytes each, half of what they need. xusub overran into the lusup/ucol storage placed right after it and the numeric values overwrote the U column pointers, so xgssvx crashed in xPivotGrowth on the first factorization. The heap path (lwork == 0) was unaffected because it uses intMalloc. Size the three int_t arrays, the matching xuser_free in the shrink loop, and the lwork == -1 estimate with sizeof(int_t); xsup and supno remain int. Also fix the byte offsets in the (currently unused) xStackCompress, which computed the positions of the int_t lsub/usub blocks with sizeof(int) and the end of ucol with the integer size instead of the value size. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjnVBdsXEM3c41ssQ3hFAM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Template counterpart of xiaoyeli/superlu#182, applied to
SRC/xmemory.baseso that regenerated{s,d,c,z}memory.ccarry the fix.With
-DXSDK_INDEX_SIZE=64, any call withlwork > 0(L and U stored in a user-suppliedwork[]array) corrupted the factors. In the extracted sources this reproduces with:which segfaulted in
sPivotGrowthon the first factorization.lwork == 0and the 32-bit index build were unaffected.Root cause
In
@(pre)LUMemInit, the user-memory branch carves the column pointer arraysxlsub,xlusupandxusubout ofwork[]with(n+1) * iwordbytes each, whereiword = sizeof(int). Those arrays areint_t, so with 64-bit indices each gets half the bytes it needs.xusuboverruns into thelusup/ucolblock placed immediately after it, and the numeric values in turn overwrite the U column pointers. The heap path usesintMalloc(n+1), which is sized correctly.Changes
xlsub,xlusupandxusubwithsizeof(int_t)(new localliword, matching the naming already used in@(pre)memory_usage).xsupandsupnoremainint.sizeof(int_t)for thelsub/usubshare in the matching@(pre)user_freeof the shrink loop.lwork == -1estimate count the twointarrays and the threeint_tarrays separately, and thelsub/usubblocks withsizeof(int_t).@(pre)StackCompress(currently not called anywhere) computed the byte offsets of theint_tlsub/usubblocks withsizeof(int)and the end ofucolwith the integer size instead of the value size. Corrected for consistency.Verification
The changed lines of this template, with
@(pre)and@(type)substituted for the double precision case, are identical to theSRC/dmemory.cdiff in xiaoyeli/superlu#182 (31 changed lines). The extracted sources in that PR pass the fullctestsuite and every tester with both 32- and 64-bit indices.Generated by Claude Code