Repository navigation
Conversation
meson found no OpenBLAS, so numpy fell back to its unoptimized
internal routines and the build still succeeded:
Run-time dependency openblas found: NO (tried pkg-config, pkg-config,
pkg-config, system and cmake)
pkg-config is disabled, and numpy's "cblas" module makes meson's CMake
lookup fail ("invalid module cblas for OpenBLAS"), which leaves the
system lookup. It links a test program, but the paths were in LIB and
INCLUDE, which are MSVC's; clang reads LIBRARY_PATH and CPATH. The
LAPACK probe also needs gcc-rt's gfortran runtime, since OpenBLAS's
LAPACK is gfortran-compiled.
Set those, pass -Dallow-noblas=false so a failed lookup stops the
build, and check in test.py that BLAS and LAPACK were found.
Co-authored-by: wtfyd-agent[bot] <328989958+wtfyd-agent[bot]@users.noreply.github.com>
The pin was commit 68ff451e, labelled 0.3.28 but 376 commits after v0.3.30, built in Debug "to workaround bug". With numpy now using OpenBLAS, Debug makes LAPACK slower than numpy's own fallback. In Release, that commit has OpenMathLib/OpenBLAS#5818: Clang if-converts the pivot reciprocal in lapack/getf2/zgetf2_k.c, so both temp2/temp1 and temp1/temp2 execute, and a real or purely imaginary pivot raises divide-by-zero and invalid. Results are right, but numpy warns, and code with FP traps enabled crashes. 0.3.34 fixes it (#5820). Build the v0.3.34 tag in Release on arm64. x86_64 stays in Debug, as it is untested here. test.py checks that complex det raises no FP exceptions; it fails against the old pin built in Release. Co-authored-by: wtfyd-agent[bot] <328989958+wtfyd-agent[bot]@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
numpy on macOS is built without BLAS or LAPACK. meson never finds OpenBLAS, numpy falls back to its unoptimized internal routines, and the build succeeds anyway. This PR makes numpy find OpenBLAS, makes a missing BLAS fail the build, and moves OpenBLAS to a release that can be built in Release on arm64.
Found by WTFYD-Agent while testing a MonolithPy 3.14 fork.
numpy doesn't find OpenBLAS
From a numpy 2.5.3 build with the current recipe:
All three lookups fail:
PKG_CONFIG=/disabled. (The shippedopenblas.pchas the CI machine's absolute paths anyway.)LIBandINCLUDE, which are MSVC's variables; clang readsLIBRARY_PATHandCPATH, so the link fails withld: library 'openblas' not found.modules: ['cblas'], which meson's generic CMake lookup takes as a CMake component:CMake: invalid module cblas for OpenBLAS. This lookup can't succeed for numpy whatever the paths are.Once the system lookup finds the library, the LAPACK lookup still fails: OpenBLAS's LAPACK is gfortran-compiled, and the test link stops at
__gfortran_compare_string. It needsgcc-rt's-lgfortran -lquadmath -lgcc, which numpy already depends on and the interpreter relink already links.The macOS numpy recipes now set
LIBRARY_PATH,CPATHandLDFLAGSaccordingly, and pass-Dallow-noblas=falseso that a lookup that fails again stops the build instead of producing a slow numpy.test.pychecks that BLAS and LAPACK were found.OpenBLAS: Release on arm64 needs 0.3.34
The OpenBLAS recipe builds in
Debug, with the comment "Must build in Debug to workaround bug." Once numpy actually uses OpenBLAS, Debug makes LAPACK slower than numpy's fallback. In Release, the pinned commit (68ff451e, labelled 0.3.28 but 376 commits after v0.3.30) has the bug fixed upstream in OpenMathLib/OpenBLAS#5820 for #5818:lapack/getf2/zgetf2_k.c:120-130computes a complex pivot's reciprocal with one of two divisions,temp2 / temp1ortemp1 / temp2. Clang if-converts the branch, so the Releasezgetf2_kexecutes both and selects one:For a real or purely imaginary pivot the unused division divides by zero, and the unused denominator becomes
0 * inf. The results are right, but the divide-by-zero and invalid flags are raised: numpy warns (divide by zero encountered in det), and code with FP traps enabled crashes (#5818). The Debug build branches first, which hides it.Checked formally:
fcselcondition selects the same branch as the source, so the selected results are the source's.release(0.0, -1.0).OpenBLAS now builds the v0.3.34 release, labelled 0.3.34, in Release on arm64. x86_64 stays in Debug: I couldn't test it, and the comment there doesn't say which bug it works around.
openblas-intel.patchstill applies to v0.3.34.test.pychecks that complexdetraises no FP exceptions; it fails against the old pin built in Release.Results
mp314 macOS arm64, numpy 2.5.3 and OpenBLAS built from source with these recipes:
meson:
Run-time dependency openblas found: YES 0.3.34for both BLAS and LAPACK, withallow-noblas : false.test.pypasses, including the two new tests.numpy's own test suite: 48946 passed, 5 failed, 42 errors, the same as with the current numpy. The failures and errors are tests that a static interpreter can't run: tests that compile extensions with meson, that read a compiled module's
__file__, or that runnumpy-configfromPATH. The fourdet/slogdetfloating-point warnings that the old pin raises in Release are gone.Time, with
OPENBLAS_NUM_THREADS=3(best of 5):a @ b, 2000×2000solve, 2000×2000svd, 500×500OpenBLAS's default thread count is the build machine's core count. On the 8-core M-series machine I built on (4 performance + 4 efficiency), the default of 8 threads makes
svdtake 145 ms. That isn't changed here; a CI build takes the runner's core count.Not tested
test.pychanges are the same as mp314's, but I had no 3.13 MonolithPy. The mp313 OpenBLAS recipe is unchanged.