Skip to content

numpy (mac): use OpenBLAS; OpenBLAS 0.3.34 in Release on arm64 - #10

Open
KRRT7 wants to merge 2 commits into
Nuitka:masterfrom
warpforgedotco:numpy-macos-openblas
Open

KRRT7 wants to merge 2 commits into
Nuitka:masterfrom
warpforgedotco:numpy-macos-openblas

Conversation

@KRRT7

@KRRT7 KRRT7 commented Oct 16, 2026

Copy link
Copy Markdown

numpy on macOS is built without BLAS or LAPACK. meson never finds OpenBLAS, numpy falls back to its unoptimized internal routines, and the build succeeds anyway. This PR makes numpy find OpenBLAS, makes a missing BLAS fail the build, and moves OpenBLAS to a release that can be built in Release on arm64.

Found by WTFYD-Agent while testing a MonolithPy 3.14 fork.

numpy doesn't find OpenBLAS

From a numpy 2.5.3 build with the current recipe:

Run-time dependency openblas found: NO  (tried pkg-config, pkg-config, pkg-config, system and cmake)

All three lookups fail:

  • pkg-config: the recipe sets PKG_CONFIG=/disabled. (The shipped openblas.pc has the CI machine's absolute paths anyway.)
  • system: numpy's meson finds OpenBLAS by linking a test program. The recipe passes the paths in LIB and INCLUDE, which are MSVC's variables; clang reads LIBRARY_PATH and CPATH, so the link fails with ld: library 'openblas' not found.
  • cmake: numpy asks for modules: ['cblas'], which meson's generic CMake lookup takes as a CMake component: CMake: invalid module cblas for OpenBLAS. This lookup can't succeed for numpy whatever the paths are.

Once the system lookup finds the library, the LAPACK lookup still fails: OpenBLAS's LAPACK is gfortran-compiled, and the test link stops at __gfortran_compare_string. It needs gcc-rt's -lgfortran -lquadmath -lgcc, which numpy already depends on and the interpreter relink already links.

The macOS numpy recipes now set LIBRARY_PATH, CPATH and LDFLAGS accordingly, and pass -Dallow-noblas=false so that a lookup that fails again stops the build instead of producing a slow numpy. test.py checks that BLAS and LAPACK were found.

OpenBLAS: Release on arm64 needs 0.3.34

The OpenBLAS recipe builds in Debug, with the comment "Must build in Debug to workaround bug." Once numpy actually uses OpenBLAS, Debug makes LAPACK slower than numpy's fallback. In Release, the pinned commit (68ff451e, labelled 0.3.28 but 376 commits after v0.3.30) has the bug fixed upstream in OpenMathLib/OpenBLAS#5820 for #5818:

lapack/getf2/zgetf2_k.c:120-130 computes a complex pivot's reciprocal with one of two divisions, temp2 / temp1 or temp1 / temp2. Clang if-converts the branch, so the Release zgetf2_k executes both and selects one:

218: fcmp  d11, d12        ; |temp1| vs |temp2|
21c: fdiv  d0, d9, d10     ; temp1 / temp2  (runs when temp2 == 0)
234: fdiv  d2, d10, d9     ; temp2 / temp1
248: fcsel d0, d0, d3, lt

For a real or purely imaginary pivot the unused division divides by zero, and the unused denominator becomes 0 * inf. The results are right, but the divide-by-zero and invalid flags are raised: numpy warns (divide by zero encountered in det), and code with FP traps enabled crashes (#5818). The Debug build branches first, which hides it.

Checked formally:

  • Z3 (QF_FP, double precision): for every finite nonzero pivot, the fcsel condition selects the same branch as the source, so the selected results are the source's.
  • CrossHair, on a Python model of both forms: no division by zero is found in the source form; the Release form divides by zero at release(0.0, -1.0).

OpenBLAS now builds the v0.3.34 release, labelled 0.3.34, in Release on arm64. x86_64 stays in Debug: I couldn't test it, and the comment there doesn't say which bug it works around. openblas-intel.patch still applies to v0.3.34. test.py checks that complex det raises no FP exceptions; it fails against the old pin built in Release.

Results

mp314 macOS arm64, numpy 2.5.3 and OpenBLAS built from source with these recipes:

  • meson: Run-time dependency openblas found: YES 0.3.34 for both BLAS and LAPACK, with allow-noblas : false.

  • test.py passes, including the two new tests.

  • numpy's own test suite: 48946 passed, 5 failed, 42 errors, the same as with the current numpy. The failures and errors are tests that a static interpreter can't run: tests that compile extensions with meson, that read a compiled module's __file__, or that run numpy-config from PATH. The four det/slogdet floating-point warnings that the old pin raises in Release are gone.

  • Time, with OPENBLAS_NUM_THREADS=3 (best of 5):

    numpy 2.5.3 from the mirror today this PR
    a @ b, 2000×2000 8719 ms 107 ms
    solve, 2000×2000 183 ms 168 ms
    svd, 500×500 53 ms 41 ms

    OpenBLAS's default thread count is the build machine's core count. On the 8-core M-series machine I built on (4 performance + 4 efficiency), the default of 8 threads makes svd take 145 ms. That isn't changed here; a CI build takes the runner's core count.

Not tested

  • mp313: the numpy recipe and test.py changes are the same as mp314's, but I had no 3.13 MonolithPy. The mp313 OpenBLAS recipe is unchanged.
  • macOS x86_64.
  • Windows recipes are unchanged.

KRRT7 and others added 2 commits October 16, 2026 17:59
meson found no OpenBLAS, so numpy fell back to its unoptimized
internal routines and the build still succeeded:

  Run-time dependency openblas found: NO  (tried pkg-config, pkg-config,
  pkg-config, system and cmake)

pkg-config is disabled, and numpy's "cblas" module makes meson's CMake
lookup fail ("invalid module cblas for OpenBLAS"), which leaves the
system lookup. It links a test program, but the paths were in LIB and
INCLUDE, which are MSVC's; clang reads LIBRARY_PATH and CPATH. The
LAPACK probe also needs gcc-rt's gfortran runtime, since OpenBLAS's
LAPACK is gfortran-compiled.

Set those, pass -Dallow-noblas=false so a failed lookup stops the
build, and check in test.py that BLAS and LAPACK were found.

Co-authored-by: wtfyd-agent[bot] <328989958+wtfyd-agent[bot]@users.noreply.github.com>
The pin was commit 68ff451e, labelled 0.3.28 but 376 commits after
v0.3.30, built in Debug "to workaround bug". With numpy now using
OpenBLAS, Debug makes LAPACK slower than numpy's own fallback.

In Release, that commit has OpenMathLib/OpenBLAS#5818: Clang if-converts
the pivot reciprocal in lapack/getf2/zgetf2_k.c, so both temp2/temp1
and temp1/temp2 execute, and a real or purely imaginary pivot raises
divide-by-zero and invalid. Results are right, but numpy warns, and
code with FP traps enabled crashes. 0.3.34 fixes it (#5820).

Build the v0.3.34 tag in Release on arm64. x86_64 stays in Debug, as
it is untested here. test.py checks that complex det raises no FP
exceptions; it fails against the old pin built in Release.

Co-authored-by: wtfyd-agent[bot] <328989958+wtfyd-agent[bot]@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant