From 1b0e793d43f8b4858447ccd4d8c52452f9826b9c Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Sun, 11 Oct 2026 21:09:07 +0200 Subject: [PATCH] A shipped diagnostic entry point, and the GPU jars' real requirements Round 1 of the modular-natives follow-ups: the three items that need no CI run. They touch different files and go in one branch. net.ladenthin.llama.Diagnose plus loader.DiagnosticReport print what the library sees of its own installation: the pinned tag, the JVM, os.name/os.arch and what OSInfo derives from them, every set net.ladenthin.llama.* property, each natives directory of the platform with the llama.cpp tag of its jllama-build.txt, and on a failed load the operating system's own reason. It turns the support thread a failed load produces into one paste, which is the one item on these lists that pays off per user rather than per release. The split across two classes is not tidiness. Every platform and classpath fact is a package-private member of LlamaLoader, and Java access is per package rather than per module, so that half has to live in the loader's package; the main class cannot, because that package is deliberately not exported and a consumer on the module path could not run a main there. The architecture rule allows exactly this direction and not the other, which is also why the pinned-version constant and the load itself are read on the Diagnose side. Two defects of its own, both found by running it rather than reading it, and both the same mistake. It composed the property name from LlamaSystemProperties.PREFIX, which carries no trailing dot, and so read net.ladenthin.llamabackend -- a name nothing sets, so a filter that was in effect was reported as unset. And namedBy compared case-insensitively while LlamaLoader.selectModules compares with equals and fails loud on an unknown name, so OpenVino would have been reported as an enabled module while the load throws on it. Both now go through the loader's own accessor and the loader's own comparison, each with a test proven red against the defect. The rule they broke: a diagnostic may not be kinder, or differently spelled, than the code it describes. No System.exit: the architecture rule forbids it in every class here and says to throw instead, which is the better shape anyway -- main throws after printing the whole report, so the process still exits non-zero while a caller that invokes main programmatically keeps its JVM. README: the two cuda13 rows name the driver requirement, the two openvino rows say they are opt-in and name the OpenCL ICD loader the OpenVINO archive does not ship, and an IMPORTANT block carries the measured detail -- the CUDA abort (ggml_abort, not catchable, reached even with -ngl 0), the 17x/172x OpenVINO regression that is the reason for the opt-in, and Vulkan as the one backend that behaves as designed. The note's sentence that a module without a device simply registers none now names its two exceptions rather than stating the rule as if it held. The README did not mention the OpenVINO opt-in at all, so a consumer adding that jar would have seen nothing happen and no reason why. llama-langchain4j: one test. The premise of that item was mostly outdated -- its resolver test had grown and already covered nine of the core's ten cases, including the reactor-root branch non-vacuously. The genuine gap was fromProperty, tested only in the unset case, so a set property bypassing the resolver would have been invisible: exactly the failure the resolver exists for. 88 Java tests green (DiagnoseTest, DiagnosticReportTest, LlamaLoaderTest, TestConstantsTest, LlamaArchitectureTest), spotbugs:check 0 findings with one documented suppression proven load-bearing (1 without it), spotless applied, check-natives.py 0 disagreements. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01E2h8gXyyE5UeimkL9vQv9G --- README.md | 37 ++- TODO.md | 47 ++- .../llama/langchain4j/TestModelPathsTest.java | 33 ++ llama/spotbugs-exclude.xml | 15 + .../java/net/ladenthin/llama/Diagnose.java | 124 ++++++++ .../llama/loader/DiagnosticReport.java | 301 ++++++++++++++++++ .../ladenthin/llama/loader/LlamaLoader.java | 2 +- .../net/ladenthin/llama/DiagnoseTest.java | 96 ++++++ .../llama/loader/DiagnosticReportTest.java | 170 ++++++++++ 9 files changed, 810 insertions(+), 15 deletions(-) create mode 100644 llama/src/main/java/net/ladenthin/llama/Diagnose.java create mode 100644 llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java create mode 100644 llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java create mode 100644 llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java diff --git a/README.md b/README.md index b5ffe92a9..6b9b7ccce 100644 --- a/README.md +++ b/README.md @@ -257,8 +257,8 @@ platforms you target, e.g. `cpu-linux-x86-64`. | `cpu-windows-aarch64` | CPU | Windows on ARM (Snapdragon X / Surface) | A JDK 8+ JVM; **Windows 10 or newer**. Built natively on `windows-11-arm` with `clang-cl`, same hybrid CRT as the x86-64 jar; one CPU backend module (ggml builds no ARM variant set for Windows), so the Vulkan and OpenCL module jars can join it. | | `metal-macos-aarch64` | Metal + CPU | macOS aarch64 (Apple silicon) | A JDK 8+ JVM. | | `cpu-android-aarch64` / `cpu-android-x86-64` | CPU | Android | For Android use the [`llama-android` AAR](#importing-in-android); these jars are the same libraries (with the CPU modules, aarch64 one per ARM feature level) for other Android JVM setups. | -| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`). | -| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` on `PATH`). | +| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`). **The driver must be new enough for CUDA 13.4** -- see the note below. | +| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` on `PATH`). **The driver must be new enough for CUDA 13.4** -- see the note below. | | `vulkan-linux-x86-64` | Vulkan | Linux x86-64 with a Vulkan 1.2+ GPU (NVIDIA / AMD / Intel) | A Vulkan runtime (`libvulkan.so.1`), which current GPU drivers install. The most portable Linux GPU option. glibc ≈ 2.39 (built on `ubuntu-latest`). | | `vulkan-linux-aarch64` | Vulkan | Linux aarch64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`libvulkan.so.1`). glibc ≥ 2.39. | | `vulkan-windows-x86-64` | Vulkan | Windows x86-64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`vulkan-1.dll`), which current GPU drivers install. The most portable Windows GPU option. | @@ -270,8 +270,8 @@ platforms you target, e.g. `cpu-linux-x86-64`. | `rocm-windows-x86-64` | ROCm / HIP | Windows x86-64 with AMD GPU | The AMD ROCm **10** runtime DLLs (`amdhip64.dll`, `rocblas.dll`, `hipblas.dll`) on `PATH`; every Radeon target TheRock builds for Windows, gfx900 through RDNA4 (gfx900/gfx906/gfx90c/gfx1153 best effort). | | `sycl-linux-x86-64` | SYCL (Intel oneAPI, fp16) | Linux x86-64 with Intel GPU (Arc / iGPU) | An Intel oneAPI / Level-Zero runtime. fp16 accumulation, as upstream's SYCL release builds. | | `sycl-windows-x86-64` | SYCL (Intel oneAPI, fp16) | Windows x86-64 with Intel GPU (Arc / iGPU) | The Intel oneAPI / Level-Zero runtime DLLs on `PATH`. | -| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 (Intel GPU / NPU / CPU) | An Intel OpenVINO runtime. | -| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 (Intel GPU / NPU / CPU) | The Intel OpenVINO runtime DLLs on `PATH`. | +| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 with an Intel GPU or NPU | **Opt-in** (see the note below): the jar does nothing until `-Dnet.ladenthin.llama.backend=openvino` names it. An Intel OpenVINO runtime **and an OpenCL ICD loader** (`libOpenCL.so.1`, package `ocl-icd-libopencl1` -- the OpenVINO archive does not ship it; without it the module silently does not open). | +| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 with an Intel GPU or NPU | **Opt-in** (see the note below): the jar does nothing until `-Dnet.ladenthin.llama.backend=openvino` names it. The Intel OpenVINO runtime DLLs on `PATH` (the OS supplies `OpenCL.dll`). | > [!NOTE] > No vendor runtime is bundled; it comes from the GPU driver or toolkit on the host. The GPU @@ -279,10 +279,37 @@ platforms you target, e.g. `cpu-linux-x86-64`. > every GPU module of the platform next to the library and see ggml skip each one for lack of a > runtime, so end-to-end GPU inference is verified locally / on self-hosted hardware. A module > whose runtime is installed but finds no device (e.g. the CUDA toolkit without an NVIDIA GPU) -> simply registers none. `-Dnet.ladenthin.llama.backend=,` restricts the modules put in +> simply registers none -- **with two measured exceptions, both described below.** `-Dnet.ladenthin.llama.backend=,` restricts the modules put in > place to the named ones (`cpu` for none) and fails loud when a named jar is not on the > classpath; unset, every GPU module jar on the classpath is used. +> [!IMPORTANT] +> **Two GPU jars need more than their runtime, and both were found by measurement rather than by +> reading the code** ([`docs/benchmarks/gpu-module-jars.md`](docs/benchmarks/gpu-module-jars.md)). +> +> **CUDA: the driver must be new enough for the toolkit the module was built with (13.4).** The +> module ships native code for the GPU generations listed in `CMAKE_CUDA_ARCHITECTURES` +> (Turing through Blackwell); a GPU outside that set runs only if the driver can compile the +> module's PTX, and **a driver too old to do so does not fall back to the CPU -- it ends the +> process.** ggml calls `abort()`, which no Java `catch` can intercept, and it happens even with +> `-ngl 0`: having the jar on the classpath is enough, because the backend registers and the device +> is used during setup whatever the layer count says. Measured on a Tesla T4 with driver +> 595.91.07, where that generation still shipped as PTX only. If you cannot control the driver, +> keep the jar off the classpath or pass `-Dnet.ladenthin.llama.backend=cpu`. +> +> **OpenVINO is opt-in, and that is why.** On a host *without* an Intel accelerator its CPU plugin +> opens perfectly well and contributes a device, so llama.cpp prefers it -- measured at +> **17x slower prompt processing and 172x slower generation** than the CPU modules (490.4/34.5 +> down to 29.4/**0.2** tokens per second). It therefore breaks the premise every other module jar +> keeps ("a module that cannot help is skipped") in both directions: it also fails to open at all +> without an OpenCL ICD loader, which its own runtime archive does not include. So its jar on the +> classpath does nothing until `net.ladenthin.llama.backend` names it, and the loader says so on +> stderr. +> +> **Vulkan is the one that behaves exactly as designed** and is the recommended portable choice: +> without an ICD it is skipped, with throughput identical to the CPU baseline (201.8/24.1 against +> 202.2/24.1 on the same machine). + > [!NOTE] > On the **module path** each natives jar is an automatic module > (`net.ladenthin.llama.natives.`, with `_` for `-`) that nothing requires, so diff --git a/TODO.md b/TODO.md index 7fdb6d201..6aa655067 100644 --- a/TODO.md +++ b/TODO.md @@ -143,15 +143,44 @@ host-CPU line says so). **Round 1 -- things that need no run, done while round 0 is in flight.** None of these touch the same files as each other, so they can go in one branch: -1. **The shipped diagnostic entry point** (item 5 of the modular-natives follow-ups). The only item on - any list here that pays off *per user* rather than per release: it turns "it does not work" into a - paste. Testable without natives. -2. **README: the GPU jars' real requirements.** `cuda13` needs a driver new enough for the toolkit - (and what happens when it is not: the JVM aborts); `openvino` is opt-in now and needs an Intel - GPU/NPU *plus* an OpenCL ICD loader; `vulkan` needs only a driver. None of that is documented, and - all three are measured. -3. **`TestConstantsTest` / `TestModelPaths` parity.** `llama-langchain4j` carries its own copy of the - resolver; the vacuity guard added to the core's test has no counterpart there. +1. ~~**The shipped diagnostic entry point**~~ **done 2026-10-11**: `net.ladenthin.llama.Diagnose` + (the main class) plus `loader.DiagnosticReport` (the half only the loader can answer). The split + is not tidiness: every platform and classpath fact is a package-private member of `LlamaLoader`, + and **Java access is per package, not per module** -- so that half has to live in the loader's + package, while the main class cannot, because that package is deliberately not exported and a + consumer on the module path could not run a main there. The architecture rule allows exactly this + direction (`Api -> Loader`, never back), which is also why the pinned-version constant (`value`) + and the load itself (`LlamaModel`) are read on the `Diagnose` side. + It prints the pinned tag, the JVM, `os.name`/`os.arch` and what `OSInfo` derives from them, every + set `net.ladenthin.llama.*` property, each natives directory of the platform **with the llama.cpp + tag of its `jllama-build.txt`** (which is how a mismatched set becomes visible at all) and, on a + failed load, the operating system's own reason. 9 tests, no natives and no model needed. + **Two defects of its own, both found by running it rather than by reading it, and both the same + mistake:** it composed the property name from `LlamaSystemProperties.PREFIX`, which carries **no** + trailing dot, and so read `net.ladenthin.llamabackend` -- a name nothing sets, so a filter that + was in effect was reported as unset; and `namedBy` compared case-insensitively while + `LlamaLoader.selectModules` compares with `equals` and **fails loud** on an unknown name, so + `OpenVino` would have been reported as an enabled module while the load throws on it. Both are now + read through the loader's own accessor and matched by the loader's own comparison, each with a + red/green test. The rule they both broke: **a diagnostic may not be kinder, or differently spelled, + than the code it describes.** +2. ~~**README: the GPU jars' real requirements**~~ **done 2026-10-11**: the two `cuda13` rows name the + driver requirement, the two `openvino` rows say they are opt-in and name the OpenCL ICD loader the + OpenVINO archive does not ship, and an `[!IMPORTANT]` block carries the measured detail -- the + CUDA abort (`ggml_abort`, not catchable, and reached even with `-ngl 0`), the 17x/172x OpenVINO + regression that is the reason for the opt-in, and Vulkan as the one backend that behaves as + designed. The existing note's sentence "simply registers none" now names its two exceptions + instead of stating the rule as if it held. **The README did not mention the OpenVINO opt-in at + all**, so a consumer adding that jar would have seen nothing happen and no reason why. +3. ~~**`TestConstantsTest` / `TestModelPaths` parity**~~ **done 2026-10-11, and the premise was + mostly outdated**: `llama-langchain4j`'s test file had grown since that line was written and + already covered nine of the core's ten cases, including the reactor-root branch non-vacuously + (it resolves `llama-langchain4j/pom.xml`, which exists only when looked up from the parent, so a + pass proves the second branch ran) and the "no test may read a model property without the + resolver" source rule. The one genuine gap was `fromProperty`: only the **unset** case was + tested, so a set property that bypassed the resolver would have been invisible -- which is + exactly the failure the resolver exists for, and the one that stayed hidden in the core for + hundreds of builds because a class-level assumption records no test entries. One test added. **Round 2 -- needs round 0 to have produced artifacts.** 4. **A `g4dn` run proving a T4 works** with the new arch list (round 0 builds the module; the probe diff --git a/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java b/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java index 89f4397dc..5f6b6ce7a 100644 --- a/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java +++ b/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java @@ -6,6 +6,7 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.not; import static org.hamcrest.Matchers.notNullValue; import static org.hamcrest.Matchers.nullValue; import static org.junit.jupiter.api.Assertions.assertFalse; @@ -82,6 +83,38 @@ void fromPropertyIsNullWhenThePropertyIsUnset() { assertThat(TestModelPaths.fromProperty("net.ladenthin.llama.langchain4j.definitely.unset"), is(nullValue())); } + /** + * The half the unset case cannot see: a property that IS set has to go through the resolver, not + * through a bare {@code System.getProperty}. Without this, {@code fromProperty} could return the + * literal and every model-gated class in this module would abort in its {@code @BeforeAll} + * assumption and report as nothing at all -- which is the exact failure the resolver was written + * for, and the one that stayed invisible in the core module for hundreds of builds because a + * class-level assumption records no test entries. + * + *

The fixture is this module's own pom, which exists only when looked up from the reactor root + * -- so a pass proves the second branch ran rather than that the path happened to be right. + */ + @Test + void fromPropertyResolvesAndDoesNotReturnTheLiteral() { + String property = "net.ladenthin.llama.langchain4j.resolver.probe"; + String literal = "llama-langchain4j/pom.xml"; + String previous = System.getProperty(property); + try { + System.setProperty(property, literal); + Path resolved = TestModelPaths.fromProperty(property); + assertThat("a set property must be resolved", resolved, is(notNullValue())); + assertThat("the resolved path must exist: " + resolved, Files.exists(resolved), is(true)); + assertThat("the resolver's second branch must have run, not a bare getProperty", + resolved, is(not(Paths.get(literal)))); + } finally { + if (previous == null) { + System.clearProperty(property); + } else { + System.setProperty(property, previous); + } + } + } + /** * The rule, not a snapshot: no test in this module may read a model path with a bare * {@code System.getProperty}, because that bypasses the resolver and silently re-mutes that class diff --git a/llama/spotbugs-exclude.xml b/llama/spotbugs-exclude.xml index d5e13bb8b..0514d12f9 100644 --- a/llama/spotbugs-exclude.xml +++ b/llama/spotbugs-exclude.xml @@ -46,6 +46,21 @@ SPDX-License-Identifier: MIT leaves no compile-time hint that an I/O failure happened; we accept that tradeoff at these three boundary points. --> + + + + + + + diff --git a/llama/src/main/java/net/ladenthin/llama/Diagnose.java b/llama/src/main/java/net/ladenthin/llama/Diagnose.java new file mode 100644 index 000000000..1a229ecde --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/Diagnose.java @@ -0,0 +1,124 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama; + +import java.io.PrintStream; +import net.ladenthin.llama.loader.DiagnosticReport; +import net.ladenthin.llama.loader.LlamaLoader; +import net.ladenthin.llama.value.LlamaCppVersion; + +/** + * Prints what this library sees of its own installation, so that "it does not load" becomes one + * paste instead of a conversation. + * + *

Every question a failed load actually raises is answerable from inside the process: which + * platform the JVM reports, which natives jars are on the classpath and which llama.cpp build each + * one carries, which library backend and GPU modules the loader selects, and -- when the load fails + * -- the operating system's own reason for refusing the library. This class prints exactly that. + * + * 

{@code
+ * java -cp 'libs/*' net.ladenthin.llama.Diagnose
+ * }
+ * + *

The report is split across two classes for a reason worth knowing before moving anything: + * every platform and classpath fact comes from a package-private member of + * {@code LlamaLoader}, and Java access is per package rather than per module -- so that half lives + * in {@code net.ladenthin.llama.loader.DiagnosticReport}. The main class has to be here instead, + * because the loader's package is deliberately not exported and a consumer on the module path could + * not run a main class in it. The architecture rule allows this direction and not the other, which + * is also why the pinned-version constant and the load itself are read here rather than there. + * + *

No probe throws. A diagnostic that fails while reporting a failure is worse than none, so + * each one reports its own problem as a line of the output instead. + * + *

The one exception is deliberate and is at the very end: when the library did not load, + * {@link #main(String[])} throws after printing the whole report, so the process exits non-zero and a + * script can tell. It is not {@code System.exit} -- the project's architecture rule forbids that in + * every class here and says to throw instead, which happens to be the better shape anyway: a caller + * that invokes {@code main} programmatically keeps its JVM, and {@link #report(PrintStream)} returns + * the outcome as a value for anyone who wants neither. + */ +public final class Diagnose { + + /** The section separator, narrow enough to survive a paste into an issue comment. */ + private static final String RULE = "--------------------------------------------------------"; + + private Diagnose() { + // Not instantiable: a main class, not an object. + } + + /** + * Prints the report to {@code System.out}, and throws afterwards when the native library did not + * load so that the process exits non-zero. + * + * @param args ignored, so that this is an ordinary main class + * @throws UnsatisfiedLinkError when the native library did not load -- after the report, never + * instead of it + */ + public static void main(String[] args) { + boolean loaded = report(System.out); + System.out.flush(); + if (!loaded) { + throw new UnsatisfiedLinkError("the native library did not load -- see the report above"); + } + } + + /** + * Writes the whole report, including an attempt to load the native library. This is the seam a + * test uses, and the entry point for an embedding caller that wants the report without the + * throw {@link #main(String[])} ends in. + * + * @param out where to write + * @return {@code true} when the native library loaded + */ + public static boolean report(PrintStream out) { + out.println("jllama diagnose"); + out.println(RULE); + out.println("llama.cpp pinned: " + LlamaCppVersion.LLAMA_CPP_VERSION); + DiagnosticReport.environment(out); + out.println(RULE); + DiagnosticReport.natives(out); + out.println(RULE); + return load(out); + } + + /** + * Loads the library and reports what came back -- or, on failure, the reason the operating system + * gave, which is the single most useful line of the report. + * + * @param out where to write + * @return {@code true} when the library loaded + */ + private static boolean load(PrintStream out) { + try { + LlamaLoader.initialize(); + } catch (UnsatisfiedLinkError | RuntimeException e) { + out.println("load: FAILED"); + out.println(" " + e); + String reason = DiagnosticReport.lastLoadFailure(); + if (reason != null) { + out.println(" the operating system's reason: " + reason); + } + return false; + } + out.println("load: ok"); + try { + String linked = LlamaModel.getLlamaCppBuildInfo(); + out.println("linked llama.cpp: " + linked); + if (!linked.startsWith(LlamaCppVersion.LLAMA_CPP_VERSION)) { + out.println(" WARNING: the loaded library is not the pinned " + + LlamaCppVersion.LLAMA_CPP_VERSION + + " -- a stale extraction directory, or a natives jar of another release on" + + " the classpath"); + } + } catch (UnsatisfiedLinkError | RuntimeException e) { + out.println("linked llama.cpp: could not be read: " + e); + } + out.println("The loader printed the directory it used, and ggml its backend registry and"); + out.println("devices, on stderr above -- include those lines when reporting a problem."); + return true; + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java b/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java new file mode 100644 index 000000000..6cc9e80e2 --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java @@ -0,0 +1,301 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama.loader; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.PrintStream; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.jspecify.annotations.Nullable; + +/** + * The half of the diagnostic report that only the loader can answer: which platform this JVM + * resolves to, which natives directories are on the classpath and which llama.cpp build each of them + * carries, and -- after a failed load -- the reason the operating system gave. + * + *

It lives here, next to {@link LlamaLoader}, because every one of those facts comes from a + * package-private member of it. Java access is per package, not per module, so a class in + * {@code net.ladenthin.llama} cannot read them; and this package is deliberately not exported, so a + * public method here is still invisible to a consumer on the module path. + * + *

The counterpart is {@code net.ladenthin.llama.Diagnose}, which owns the parts this class must + * not touch: the pinned-version constant lives in {@code value} and the load itself needs + * {@code LlamaModel}, and the architecture rule allows neither direction out of this package. + * + *

Nothing here throws. Each probe reports its own failure as a line of the report, because + * a diagnostic that dies while describing a failure is worse than none. + */ +public final class DiagnosticReport { + + /** + * The prefix of every system property this library reads, with the separating dot that + * {@link LlamaSystemProperties#PREFIX} does not carry -- its own accessors supply it per suffix + * ({@code ".backend"}). Getting that wrong is not cosmetic: the first version of this class + * composed {@code PREFIX + "backend"} and read {@code net.ladenthin.llamabackend}, a name nothing + * ever sets, so the report would have claimed no filter was in effect while one was. + */ + private static final String PROPERTY_PREFIX = LlamaSystemProperties.PREFIX + "."; + + /** + * Read through the same accessors the loader uses, so the report cannot disagree with it about + * which property name carries which setting. + */ + private static final LlamaSystemProperties PROPERTIES = new LlamaSystemProperties(); + + /** The longest backend directory name, for aligning the columns of a pasted report. */ + private static final int NAME_WIDTH = "openvino".length(); + + private DiagnosticReport() { + // Not instantiable: a report builder, not an object. + } + + /** + * The JVM, the platform as {@link OSInfo} derives it, and the {@code net.ladenthin.llama.*} + * properties that are set. Needs no native library. + * + * @param out where to write + */ + public static void environment(PrintStream out) { + out.println("java: " + property("java.version") + " (" + property("java.vendor") + ")"); + out.println("os.name / os.arch: " + property("os.name") + " / " + property("os.arch")); + out.println("resolved platform: " + call(OSInfo::getNativeLibFolderPathForCurrentOS)); + out.println(" OSInfo os/arch: " + call(OSInfo::getOSName) + " / " + call(OSInfo::getArchName)); + out.println(" android / musl: " + + call(() -> String.valueOf(OSInfo.isAndroid())) + + " / " + + call(() -> String.valueOf(OSInfo.isMusl()))); + Map properties = llamaProperties(); + if (properties.isEmpty()) { + out.println("properties: none of " + PROPERTY_PREFIX + "* is set"); + } else { + out.println("properties:"); + for (Map.Entry entry : properties.entrySet()) { + out.println(" " + entry.getKey() + " = " + entry.getValue()); + } + } + } + + /** + * What the classpath offers: the library backend directory of this platform and every GPU module + * directory, each with the jar it comes from and the llama.cpp tag it was built from -- which is + * what a mismatched set looks like, and the one thing a consumer cannot otherwise see. + * + * @param out where to write + */ + public static void natives(PrintStream out) { + String base = LlamaLoader.NATIVE_RESOURCE_BASE + "/" + call(OSInfo::getNativeLibFolderPathForCurrentOS); + out.println("natives on the classpath, below " + base); + int found = 0; + for (String backend : LlamaLoader.LIBRARY_BACKENDS) { + if (describe(out, base, backend, "library")) { + found++; + } + } + List modules = new ArrayList<>(); + for (String backend : LlamaLoader.MODULE_BACKENDS) { + if (describe(out, base, backend, "module ")) { + found++; + modules.add(backend); + } + } + if (found == 0) { + out.println(" (none -- no natives jar for this platform is on the classpath)"); + } + String filter = PROPERTIES.getBackend(); + out.println("backend property: " + (filter == null ? "unset (every module is used)" : filter)); + for (String backend : modules) { + if (LlamaLoader.OPT_IN_BACKENDS.contains(backend) && !namedBy(filter, backend)) { + out.println(" note: module '" + + backend + + "' is opt-in and is NOT used -- name it in " + + PROPERTY_PREFIX + + "backend to enable it"); + } + } + } + + /** + * The reason a failed {@code System.load} gave, which is the single most useful line of the + * report: the library is usually present and one of its dependencies is not. + * + * @return the reason, or {@code null} when no load has failed + */ + public static @Nullable String lastLoadFailure() { + return LlamaLoader.lastLoadFailure; + } + + /** + * Whether a comma-separated backend filter names a backend, matched exactly the way + * {@code LlamaLoader.selectModules} matches it: split on commas, each entry trimmed, then + * compared with {@code equals}. + * + *

That mirroring is the point, and the first version of this method got it wrong in a way + * worth keeping on record: it compared case-insensitively, which is more lenient than the loader. + * The loader fails loud on a name it does not know, so {@code OpenVino} makes the load throw -- + * while a lenient report would have called that module enabled. A diagnostic may not be kinder + * than the code it describes; being a line more cautious costs nothing, being a line more + * generous is a wrong answer. + * + * @param filter the property value, or {@code null} when unset + * @param backend the backend name + * @return {@code true} when the filter names it, by the loader's own comparison + */ + public static boolean namedBy(@Nullable String filter, String backend) { + if (filter == null) { + return false; + } + // The same split the loader uses, with the same suppression and for the same reason: the + // "trailing empties dropped" quirk is benign because an empty entry names no backend. + @SuppressWarnings("StringSplitter") + final String[] entries = filter.split(","); + for (String entry : entries) { + if (entry.trim().equals(backend)) { + return true; + } + } + return false; + } + + /** + * Every {@code net.ladenthin.llama.*} system property that is set, sorted by name. + * + * @return the properties, possibly empty + */ + public static Map llamaProperties() { + List names = new ArrayList<>(); + try { + for (String name : System.getProperties().stringPropertyNames()) { + if (name.startsWith(PROPERTY_PREFIX)) { + names.add(name); + } + } + } catch (SecurityException e) { + Map refused = new LinkedHashMap<>(1); + refused.put("(properties unreadable)", e.toString()); + return refused; + } + Collections.sort(names); + Map found = new LinkedHashMap<>(names.size()); + for (String name : names) { + found.put(name, String.valueOf(System.getProperty(name))); + } + return found; + } + + /** + * One natives directory, if it is on the classpath at all. + * + * @param out where to write + * @param base the platform's resource prefix + * @param backend the backend directory name + * @param kind {@code library} or {@code module}, for the output + * @return {@code true} when the directory is present + */ + private static boolean describe(PrintStream out, String base, String backend, String kind) { + String name = base + "/" + backend + "/" + LlamaLoader.BACKEND_BUILD_FILE; + URL url = resourceOrNull(name); + if (url == null) { + return false; + } + out.println(" " + kind + " " + pad(backend) + " llama.cpp=" + tagOf(name) + " " + url); + return true; + } + + /** + * The llama.cpp tag a {@code jllama-build.txt} names, read through the loader's own parser so the + * two cannot disagree on the format. + * + * @param name the build file's resource name + * @return the tag, or a short description of why it could not be read + */ + private static String tagOf(String name) { + try (InputStream in = LlamaLoader.resourceAsStream(name)) { + if (in == null) { + return "(vanished)"; + } + BufferedReader reader = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8)); + String tag = LlamaLoader.parseBuild(reader).get(LlamaLoader.BUILD_KEY_LLAMA_CPP); + return tag != null ? tag : "(not named)"; + } catch (IOException | RuntimeException e) { + return "(unreadable: " + e + ")"; + } + } + + /** + * Looks a resource up the way the loader does, never throwing. + * + * @param name the resource name + * @return the URL, or {@code null} when absent or unreadable + */ + private static @Nullable URL resourceOrNull(String name) { + try { + return LlamaLoader.resource(name); + } catch (RuntimeException | LinkageError e) { + return null; + } + } + + /** + * Pads a backend name so the columns line up for whoever reads a pasted report. + * + * @param backend the name + * @return the name, padded to the width of the longest backend name + */ + private static String pad(String backend) { + StringBuilder padded = new StringBuilder(backend); + while (padded.length() < NAME_WIDTH) { + padded.append(' '); + } + return padded.toString(); + } + + /** + * A system property, or a placeholder -- never an exception. + * + * @param name the property name + * @return the value, or {@code (unset)} + */ + private static String property(String name) { + String value = System.getProperty(name); + return value != null ? value : "(unset)"; + } + + /** + * Calls a probe that may throw, so one unhappy line cannot take the report down. + * + * @param probe the probe + * @return its value, or a short description of the failure + */ + private static String call(Probe probe) { + try { + String value = probe.get(); + return value != null ? value : "(null)"; + } catch (RuntimeException | LinkageError e) { + return "(failed: " + e + ")"; + } + } + + /** A probe that returns one value of the report and is allowed to fail. */ + @FunctionalInterface + interface Probe { + + /** + * Reads one value. + * + * @return the value, or {@code null} + */ + @Nullable + String get(); + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java index 3a7bf649c..b211a60a1 100644 --- a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java +++ b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java @@ -1059,7 +1059,7 @@ static String getNativeResourcePath() { return classLoader().getResource(name); } - private static @Nullable InputStream resourceAsStream(String name) { + static @Nullable InputStream resourceAsStream(String name) { return classLoader().getResourceAsStream(name); } diff --git a/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java b/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java new file mode 100644 index 000000000..08f98c43a --- /dev/null +++ b/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java @@ -0,0 +1,96 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama; + +import static org.junit.jupiter.api.Assertions.*; + +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; +import java.io.UnsupportedEncodingException; +import java.nio.charset.StandardCharsets; +import net.ladenthin.llama.value.LlamaCppVersion; +import org.junit.jupiter.api.Test; + +/** + * The shipped diagnostic entry point. + * + *

These tests need no native library and no model, which is the point: the report exists for the + * case where the library does not load, so it has to be exercised in exactly that state. + * The suite therefore runs both ways -- with natives on the classpath (CI, or after a local cmake + * build) and without (a plain checkout) -- and every assertion here holds for both. + * + *

{@code Diagnose.main} is never called here: on a checkout without natives it ends by throwing, + * which is the whole point of it (the process has to exit non-zero), but says nothing a test of the + * report wants. {@link Diagnose#report(PrintStream)} is the seam that exists for that. + * + *

The pure helpers live in {@code loader.DiagnosticReport} and are covered by its own test. + */ +@ClaudeGenerated( + purpose = "Verify the shipped diagnostic entry point without a native library: report() never " + + "throws and always states exactly one of the two load outcomes (the property " + + "the whole class exists for -- a diagnostic that fails while reporting a " + + "failure is worse than none); it names the pinned llama.cpp version, the " + + "JVM's os.name/os.arch and the resolved platform directory; and it either " + + "lists the natives directories of this platform with the llama.cpp tag of " + + "each jllama-build.txt or states that there is none, never staying silent.") +public class DiagnoseTest { + + /** + * Runs the full report into a buffer. + * + * @return the report text + */ + private static String report() { + ByteArrayOutputStream buffer = new ByteArrayOutputStream(); + try (PrintStream out = new PrintStream(buffer, true, StandardCharsets.UTF_8.name())) { + Diagnose.report(out); + } catch (UnsupportedEncodingException e) { + throw new AssertionError("every JVM is required to support UTF-8", e); + } + return new String(buffer.toByteArray(), StandardCharsets.UTF_8); + } + + /** + * The property this class exists for: the report has to survive the state it describes. A + * diagnostic that throws while reporting a failure is worse than no diagnostic at all, so this + * is the assertion that matters most -- the others only check that the content is useful. + */ + @Test + public void theReportNeverThrowsWhetherTheNativeLibraryLoadsOrNot() { + String text = report(); + assertFalse(text.isEmpty(), "the report must not be empty"); + // Exactly one of the two outcomes, and never neither: the load section always says what + // happened, so a reader is not left guessing whether it even got that far. + boolean ok = text.contains("load: ok"); + boolean failed = text.contains("load: FAILED"); + assertTrue(ok || failed, "the report must state the load outcome, got:\n" + text); + assertNotEquals(ok, failed, "the load outcome must be exactly one of ok and FAILED"); + } + + /** The platform half is what whoever reports a problem is asked for first. */ + @Test + public void theReportNamesThePinnedVersionTheJvmAndTheResolvedPlatform() { + String text = report(); + assertTrue(text.contains(LlamaCppVersion.LLAMA_CPP_VERSION), "the pinned version must appear"); + assertTrue(text.contains(System.getProperty("os.name")), "os.name must appear verbatim"); + assertTrue(text.contains(System.getProperty("os.arch")), "os.arch must appear verbatim"); + assertTrue(text.contains("resolved platform:"), "the derived platform directory must appear"); + } + + /** + * Either there are natives for this platform and each is named with the llama.cpp tag of its + * {@code jllama-build.txt} -- which is how a mismatched set becomes visible -- or the report says + * there are none. Both are verdicts; saying nothing would not be. + */ + @Test + public void theReportEitherListsTheNativesWithTheirBuildTagOrSaysThereAreNone() { + String text = report(); + boolean none = text.contains("(none -- no natives jar for this platform is on the classpath)"); + boolean listed = text.contains("llama.cpp="); + assertTrue(none || listed, "the natives section must reach a verdict, got:\n" + text); + assertNotEquals(none, listed, "it cannot both list a natives directory and report none"); + } +} diff --git a/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java b/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java new file mode 100644 index 000000000..4cd67b73a --- /dev/null +++ b/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java @@ -0,0 +1,170 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama.loader; + +import static org.junit.jupiter.api.Assertions.*; + +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; +import java.io.UnsupportedEncodingException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import net.ladenthin.llama.ClaudeGenerated; +import org.junit.jupiter.api.Test; + +/** + * The loader half of the diagnostic report: the pure helpers, and that each section writes something + * rather than throwing. + * + *

No native library and no model: these tests run in a plain checkout. + */ +@ClaudeGenerated( + purpose = "Verify the loader half of the diagnostic report without a native library: namedBy " + + "matches the comma-separated net.ladenthin.llama.backend filter entry by " + + "entry, trimmed and ignoring case, so the spacing a human writes works while " + + "a prefix is not a member (a substring test would answer that wrongly in one " + + "direction or the other); llamaProperties collects only this library's " + + "properties and returns them sorted; and environment() and natives() each " + + "write a non-empty section without throwing, naming the resolved platform " + + "and reaching a verdict on the classpath.") +public class DiagnosticReportTest { + + /** + * Captures one section. + * + * @param section the section to run + * @return what it wrote + */ + private static String capture(java.util.function.Consumer section) { + ByteArrayOutputStream buffer = new ByteArrayOutputStream(); + try (PrintStream out = new PrintStream(buffer, true, StandardCharsets.UTF_8.name())) { + section.accept(out); + } catch (UnsupportedEncodingException e) { + throw new AssertionError("every JVM is required to support UTF-8", e); + } + return new String(buffer.toByteArray(), StandardCharsets.UTF_8); + } + + /** + * The backend filter is a comma-separated list, so membership is a comparison of whole entries + * rather than a substring test -- and it tolerates the spacing and the casing a human writes. + */ + @Test + public void theBackendFilterIsMatchedEntryByEntryNotAsASubstring() { + assertTrue(DiagnosticReport.namedBy("openvino", "openvino"), "the only entry names it"); + assertTrue(DiagnosticReport.namedBy("cpu,openvino", "openvino"), "one of several entries names it"); + assertTrue(DiagnosticReport.namedBy("cpu, openvino", "openvino"), "the spacing a human writes works"); + // NOT case-insensitive, deliberately: LlamaLoader.selectModules compares with equals and + // fails loud on a name it does not know, so "OpenVino" makes the load throw. A report that + // called that module enabled would be kinder than the code it describes, which is a wrong + // answer rather than a lenient one. (The first version of namedBy did exactly that.) + assertFalse(DiagnosticReport.namedBy(" OpenVino ", "openvino"), "the loader is case-sensitive"); + assertTrue(DiagnosticReport.namedBy(" openvino ", "openvino"), "but surrounding spaces are trimmed"); + assertFalse(DiagnosticReport.namedBy("cuda13,vulkan", "openvino"), "other entries do not name it"); + // A prefix is not a member. A substring test would answer this wrongly in one direction or + // the other, depending on which of the two strings it searched. + assertFalse(DiagnosticReport.namedBy("openvino", "openv"), "a prefix of an entry is not that entry"); + assertFalse(DiagnosticReport.namedBy("", "openvino"), "an empty filter names nothing"); + assertFalse(DiagnosticReport.namedBy(null, "openvino"), "an unset filter names nothing"); + } + + /** Only this library's properties are collected, and in a stable order. */ + @Test + public void onlyTheLibrariesOwnPropertiesAreCollectedAndTheyAreSorted() { + String mine = LlamaSystemProperties.PREFIX + ".diagnose.test.zzz"; + String alsoMine = LlamaSystemProperties.PREFIX + ".diagnose.test.aaa"; + String foreign = "some.other.product.diagnose.test"; + try { + System.setProperty(mine, "1"); + System.setProperty(alsoMine, "2"); + System.setProperty(foreign, "3"); + + Map properties = DiagnosticReport.llamaProperties(); + assertEquals("1", properties.get(mine), "a property of this library is collected"); + assertEquals("2", properties.get(alsoMine), "and so is the other one"); + assertFalse(properties.containsKey(foreign), "a foreign property is not"); + + List keys = new ArrayList<>(properties.keySet()); + List sorted = new ArrayList<>(keys); + Collections.sort(sorted); + assertEquals(sorted, keys, "the properties are reported in sorted order"); + } finally { + System.clearProperty(mine); + System.clearProperty(alsoMine); + System.clearProperty(foreign); + } + } + + /** A set property has to be visible in the paste, because a wrong one is a common cause. */ + @Test + public void theEnvironmentSectionNamesThePlatformAndEverySetProperty() { + String key = LlamaSystemProperties.PREFIX + ".diagnose.test.marker"; + String previous = System.getProperty(key); + try { + System.setProperty(key, "a-value-only-this-test-sets"); + String text = capture(DiagnosticReport::environment); + assertTrue(text.contains("resolved platform:"), "the derived platform directory appears"); + assertTrue(text.contains(OSInfo.getNativeLibFolderPathForCurrentOS()), "and its value"); + assertTrue(text.contains(key + " = a-value-only-this-test-sets"), "a set property appears"); + } finally { + if (previous == null) { + System.clearProperty(key); + } else { + System.setProperty(key, previous); + } + } + } + + /** + * The natives section reaches a verdict either way: it lists each directory with the llama.cpp + * tag of its {@code jllama-build.txt}, or states that there is none. Silence would be the one + * useless outcome. + */ + @Test + public void theNativesSectionReachesAVerdictWithOrWithoutNativesOnTheClasspath() { + String text = capture(DiagnosticReport::natives); + boolean none = text.contains("(none -- no natives jar for this platform is on the classpath)"); + boolean listed = text.contains("llama.cpp="); + assertTrue(none || listed, "the section must reach a verdict, got:\n" + text); + assertNotEquals(none, listed, "it cannot both list a directory and report none"); + assertTrue(text.contains("backend property:"), "the backend filter is always reported"); + } + + /** + * The report has to read the same property the loader reads. This is the test that would + * have caught the first version of this class: it composed the name from a prefix that carries no + * separating dot and read {@code net.ladenthin.llamabackend}, so a filter that was in effect was + * reported as unset -- a diagnostic stating the opposite of the truth about the one setting that + * decides which modules load. + */ + @Test + public void theBackendFilterIsReadUnderTheNameTheLoaderActuallyUses() { + String real = LlamaSystemProperties.PREFIX + ".backend"; + String previous = System.getProperty(real); + try { + System.setProperty(real, "cpu"); + String text = capture(DiagnosticReport::natives); + assertTrue(text.contains("backend property: cpu"), "the filter must be shown: " + text); + } finally { + if (previous == null) { + System.clearProperty(real); + } else { + System.setProperty(real, previous); + } + } + } + + /** No load has failed in this JVM unless one was attempted, and the accessor must not throw. */ + @Test + public void theLastLoadFailureAccessorIsReadableWithoutALoad() { + // Deliberately no assertion on the value: another test in the same JVM may have attempted a + // load. What is pinned is that reading it is safe, which is what the report relies on. + DiagnosticReport.lastLoadFailure(); + } +}