Repository navigation
Handle sparse matrices of 2^63 or more entries in reductions, sprand and linear indexing - #1128
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1128 +/- ##
=======================================
Coverage 94.70% 94.70%
=======================================
Files 16 16
Lines 10385 10423 +38
=======================================
+ Hits 9835 9871 +36
- Misses 550 552 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…ries `widelength` multiplied the dimensions as `Int64`, so for an array of 2^63 entries or more it wrapped, and the reductions, linear `setindex!` and `sprand` that rely on it returned wrong results or threw. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d fix the remaining widelength checks Linear getindex and setindex! on a sparse matrix of 2^63 or more entries throw OverflowError up front instead of converting indices with divrem. The two-array mapreduce and _inorder use widelength, prod splits an Int128 count into Int powers so it stays type-stable, and the trim app covers linear indexing and the two-array and ordered mapreduce. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ViralBShah
force-pushed
the
vs/widelength-overflow
branch
from
October 26, 2026 22:41
22fdbb3 to
0b50513
Compare
widelength of a sparse array without overflow past 2^63 entriessprand and linear indexing
ViralBShah
enabled auto-merge (squash)
October 26, 2026 22:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
widelength, the length that the sparse kernels compare withnnz, multiplied the dimensions asInt64. For a matrix of 2^63 entries or more it wrapped, so the reductions andsprandgave wrong results or threw, and linear indexing failed with a misleadingBoundsError.Before
After
widelengthis nowprod(widen, size(x)), anInt128on a 64-bit build andInt64, as before, on a 32-bit one.count, the two-arraymapreduceand_inorderuse it in place oflengthorprod(size(A)), and the counters of unstored runs in the ordered reductions are widened the same way. AnInt128count would widenz * nforComplex{Int}and reach a different^method for an eltype that defines^for anIntpower only, so_addzerosand_powzerossplit it intoIntfactors and powers. For a matrix that fits anIntthey make the same calls as before, andprodstays type-stable. Whenm*noverflows,spranddraws the rows column by column, which keeps the distribution; every size that fits draws the same entries as before.Linear indexing into such a matrix is not supported.
getindexandsetindex!with a linear index, range or array, including the fixed-patternsetindex!, throw anOverflowErrorbefore writing anything; indexing by row and column works. This touches dispatch: scalarA[i] = vgets a method onAbstractSparseMatrixCSC, as Base's fallback checks the index against the wrappinglength.On a 10⁴×10⁴ matrix of density 10⁻³,
mainagainst this branch:mainsum/prod/count(Float64)mapreduce(Float64 / Int)mapreduce(*, -, A, A)(Float64 / Int)findmax(Float64)A[I] = x,A[I](10⁴ indices)A[i] = v,A[i]sprandThe result types and values of 18 reductions over 10 eltypes, and the entries
spranddraws in each of its forms, matchmain.Left out: Base's own
length(A)still wraps, and so do the checks that use it orm*n, such asisempty,Base.checkboundsand the dims reductions'nnz(A) == m*n.🤖 Generated with Claude Code