diff --git a/README.md b/README.md index b5ffe92a9..6b9b7ccce 100644 --- a/README.md +++ b/README.md @@ -257,8 +257,8 @@ platforms you target, e.g. `cpu-linux-x86-64`. | `cpu-windows-aarch64` | CPU | Windows on ARM (Snapdragon X / Surface) | A JDK 8+ JVM; **Windows 10 or newer**. Built natively on `windows-11-arm` with `clang-cl`, same hybrid CRT as the x86-64 jar; one CPU backend module (ggml builds no ARM variant set for Windows), so the Vulkan and OpenCL module jars can join it. | | `metal-macos-aarch64` | Metal + CPU | macOS aarch64 (Apple silicon) | A JDK 8+ JVM. | | `cpu-android-aarch64` / `cpu-android-x86-64` | CPU | Android | For Android use the [`llama-android` AAR](#importing-in-android); these jars are the same libraries (with the CPU modules, aarch64 one per ARM feature level) for other Android JVM setups. | -| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`). | -| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` on `PATH`). | +| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`). **The driver must be new enough for CUDA 13.4** -- see the note below. | +| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` on `PATH`). **The driver must be new enough for CUDA 13.4** -- see the note below. | | `vulkan-linux-x86-64` | Vulkan | Linux x86-64 with a Vulkan 1.2+ GPU (NVIDIA / AMD / Intel) | A Vulkan runtime (`libvulkan.so.1`), which current GPU drivers install. The most portable Linux GPU option. glibc ≈ 2.39 (built on `ubuntu-latest`). | | `vulkan-linux-aarch64` | Vulkan | Linux aarch64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`libvulkan.so.1`). glibc ≥ 2.39. | | `vulkan-windows-x86-64` | Vulkan | Windows x86-64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`vulkan-1.dll`), which current GPU drivers install. The most portable Windows GPU option. | @@ -270,8 +270,8 @@ platforms you target, e.g. `cpu-linux-x86-64`. | `rocm-windows-x86-64` | ROCm / HIP | Windows x86-64 with AMD GPU | The AMD ROCm **10** runtime DLLs (`amdhip64.dll`, `rocblas.dll`, `hipblas.dll`) on `PATH`; every Radeon target TheRock builds for Windows, gfx900 through RDNA4 (gfx900/gfx906/gfx90c/gfx1153 best effort). | | `sycl-linux-x86-64` | SYCL (Intel oneAPI, fp16) | Linux x86-64 with Intel GPU (Arc / iGPU) | An Intel oneAPI / Level-Zero runtime. fp16 accumulation, as upstream's SYCL release builds. | | `sycl-windows-x86-64` | SYCL (Intel oneAPI, fp16) | Windows x86-64 with Intel GPU (Arc / iGPU) | The Intel oneAPI / Level-Zero runtime DLLs on `PATH`. | -| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 (Intel GPU / NPU / CPU) | An Intel OpenVINO runtime. | -| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 (Intel GPU / NPU / CPU) | The Intel OpenVINO runtime DLLs on `PATH`. | +| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 with an Intel GPU or NPU | **Opt-in** (see the note below): the jar does nothing until `-Dnet.ladenthin.llama.backend=openvino` names it. An Intel OpenVINO runtime **and an OpenCL ICD loader** (`libOpenCL.so.1`, package `ocl-icd-libopencl1` -- the OpenVINO archive does not ship it; without it the module silently does not open). | +| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 with an Intel GPU or NPU | **Opt-in** (see the note below): the jar does nothing until `-Dnet.ladenthin.llama.backend=openvino` names it. The Intel OpenVINO runtime DLLs on `PATH` (the OS supplies `OpenCL.dll`). | > [!NOTE] > No vendor runtime is bundled; it comes from the GPU driver or toolkit on the host. The GPU @@ -279,10 +279,37 @@ platforms you target, e.g. `cpu-linux-x86-64`. > every GPU module of the platform next to the library and see ggml skip each one for lack of a > runtime, so end-to-end GPU inference is verified locally / on self-hosted hardware. A module > whose runtime is installed but finds no device (e.g. the CUDA toolkit without an NVIDIA GPU) -> simply registers none. `-Dnet.ladenthin.llama.backend=,` restricts the modules put in +> simply registers none -- **with two measured exceptions, both described below.** `-Dnet.ladenthin.llama.backend=,` restricts the modules put in > place to the named ones (`cpu` for none) and fails loud when a named jar is not on the > classpath; unset, every GPU module jar on the classpath is used. +> [!IMPORTANT] +> **Two GPU jars need more than their runtime, and both were found by measurement rather than by +> reading the code** ([`docs/benchmarks/gpu-module-jars.md`](docs/benchmarks/gpu-module-jars.md)). +> +> **CUDA: the driver must be new enough for the toolkit the module was built with (13.4).** The +> module ships native code for the GPU generations listed in `CMAKE_CUDA_ARCHITECTURES` +> (Turing through Blackwell); a GPU outside that set runs only if the driver can compile the +> module's PTX, and **a driver too old to do so does not fall back to the CPU -- it ends the +> process.** ggml calls `abort()`, which no Java `catch` can intercept, and it happens even with +> `-ngl 0`: having the jar on the classpath is enough, because the backend registers and the device +> is used during setup whatever the layer count says. Measured on a Tesla T4 with driver +> 595.91.07, where that generation still shipped as PTX only. If you cannot control the driver, +> keep the jar off the classpath or pass `-Dnet.ladenthin.llama.backend=cpu`. +> +> **OpenVINO is opt-in, and that is why.** On a host *without* an Intel accelerator its CPU plugin +> opens perfectly well and contributes a device, so llama.cpp prefers it -- measured at +> **17x slower prompt processing and 172x slower generation** than the CPU modules (490.4/34.5 +> down to 29.4/**0.2** tokens per second). It therefore breaks the premise every other module jar +> keeps ("a module that cannot help is skipped") in both directions: it also fails to open at all +> without an OpenCL ICD loader, which its own runtime archive does not include. So its jar on the +> classpath does nothing until `net.ladenthin.llama.backend` names it, and the loader says so on +> stderr. +> +> **Vulkan is the one that behaves exactly as designed** and is the recommended portable choice: +> without an ICD it is skipped, with throughput identical to the CPU baseline (201.8/24.1 against +> 202.2/24.1 on the same machine). + > [!NOTE] > On the **module path** each natives jar is an automatic module > (`net.ladenthin.llama.natives.`, with `_` for `-`) that nothing requires, so diff --git a/TODO.md b/TODO.md index 7fdb6d201..6aa655067 100644 --- a/TODO.md +++ b/TODO.md @@ -143,15 +143,44 @@ host-CPU line says so). **Round 1 -- things that need no run, done while round 0 is in flight.** None of these touch the same files as each other, so they can go in one branch: -1. **The shipped diagnostic entry point** (item 5 of the modular-natives follow-ups). The only item on - any list here that pays off *per user* rather than per release: it turns "it does not work" into a - paste. Testable without natives. -2. **README: the GPU jars' real requirements.** `cuda13` needs a driver new enough for the toolkit - (and what happens when it is not: the JVM aborts); `openvino` is opt-in now and needs an Intel - GPU/NPU *plus* an OpenCL ICD loader; `vulkan` needs only a driver. None of that is documented, and - all three are measured. -3. **`TestConstantsTest` / `TestModelPaths` parity.** `llama-langchain4j` carries its own copy of the - resolver; the vacuity guard added to the core's test has no counterpart there. +1. ~~**The shipped diagnostic entry point**~~ **done 2026-10-11**: `net.ladenthin.llama.Diagnose` + (the main class) plus `loader.DiagnosticReport` (the half only the loader can answer). The split + is not tidiness: every platform and classpath fact is a package-private member of `LlamaLoader`, + and **Java access is per package, not per module** -- so that half has to live in the loader's + package, while the main class cannot, because that package is deliberately not exported and a + consumer on the module path could not run a main there. The architecture rule allows exactly this + direction (`Api -> Loader`, never back), which is also why the pinned-version constant (`value`) + and the load itself (`LlamaModel`) are read on the `Diagnose` side. + It prints the pinned tag, the JVM, `os.name`/`os.arch` and what `OSInfo` derives from them, every + set `net.ladenthin.llama.*` property, each natives directory of the platform **with the llama.cpp + tag of its `jllama-build.txt`** (which is how a mismatched set becomes visible at all) and, on a + failed load, the operating system's own reason. 9 tests, no natives and no model needed. + **Two defects of its own, both found by running it rather than by reading it, and both the same + mistake:** it composed the property name from `LlamaSystemProperties.PREFIX`, which carries **no** + trailing dot, and so read `net.ladenthin.llamabackend` -- a name nothing sets, so a filter that + was in effect was reported as unset; and `namedBy` compared case-insensitively while + `LlamaLoader.selectModules` compares with `equals` and **fails loud** on an unknown name, so + `OpenVino` would have been reported as an enabled module while the load throws on it. Both are now + read through the loader's own accessor and matched by the loader's own comparison, each with a + red/green test. The rule they both broke: **a diagnostic may not be kinder, or differently spelled, + than the code it describes.** +2. ~~**README: the GPU jars' real requirements**~~ **done 2026-10-11**: the two `cuda13` rows name the + driver requirement, the two `openvino` rows say they are opt-in and name the OpenCL ICD loader the + OpenVINO archive does not ship, and an `[!IMPORTANT]` block carries the measured detail -- the + CUDA abort (`ggml_abort`, not catchable, and reached even with `-ngl 0`), the 17x/172x OpenVINO + regression that is the reason for the opt-in, and Vulkan as the one backend that behaves as + designed. The existing note's sentence "simply registers none" now names its two exceptions + instead of stating the rule as if it held. **The README did not mention the OpenVINO opt-in at + all**, so a consumer adding that jar would have seen nothing happen and no reason why. +3. ~~**`TestConstantsTest` / `TestModelPaths` parity**~~ **done 2026-10-11, and the premise was + mostly outdated**: `llama-langchain4j`'s test file had grown since that line was written and + already covered nine of the core's ten cases, including the reactor-root branch non-vacuously + (it resolves `llama-langchain4j/pom.xml`, which exists only when looked up from the parent, so a + pass proves the second branch ran) and the "no test may read a model property without the + resolver" source rule. The one genuine gap was `fromProperty`: only the **unset** case was + tested, so a set property that bypassed the resolver would have been invisible -- which is + exactly the failure the resolver exists for, and the one that stayed hidden in the core for + hundreds of builds because a class-level assumption records no test entries. One test added. **Round 2 -- needs round 0 to have produced artifacts.** 4. **A `g4dn` run proving a T4 works** with the new arch list (round 0 builds the module; the probe diff --git a/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java b/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java index 89f4397dc..5f6b6ce7a 100644 --- a/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java +++ b/llama-langchain4j/src/test/java/net/ladenthin/llama/langchain4j/TestModelPathsTest.java @@ -6,6 +6,7 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.not; import static org.hamcrest.Matchers.notNullValue; import static org.hamcrest.Matchers.nullValue; import static org.junit.jupiter.api.Assertions.assertFalse; @@ -82,6 +83,38 @@ void fromPropertyIsNullWhenThePropertyIsUnset() { assertThat(TestModelPaths.fromProperty("net.ladenthin.llama.langchain4j.definitely.unset"), is(nullValue())); } + /** + * The half the unset case cannot see: a property that IS set has to go through the resolver, not + * through a bare {@code System.getProperty}. Without this, {@code fromProperty} could return the + * literal and every model-gated class in this module would abort in its {@code @BeforeAll} + * assumption and report as nothing at all -- which is the exact failure the resolver was written + * for, and the one that stayed invisible in the core module for hundreds of builds because a + * class-level assumption records no test entries. + * + *

The fixture is this module's own pom, which exists only when looked up from the reactor root + * -- so a pass proves the second branch ran rather than that the path happened to be right. + */ + @Test + void fromPropertyResolvesAndDoesNotReturnTheLiteral() { + String property = "net.ladenthin.llama.langchain4j.resolver.probe"; + String literal = "llama-langchain4j/pom.xml"; + String previous = System.getProperty(property); + try { + System.setProperty(property, literal); + Path resolved = TestModelPaths.fromProperty(property); + assertThat("a set property must be resolved", resolved, is(notNullValue())); + assertThat("the resolved path must exist: " + resolved, Files.exists(resolved), is(true)); + assertThat("the resolver's second branch must have run, not a bare getProperty", + resolved, is(not(Paths.get(literal)))); + } finally { + if (previous == null) { + System.clearProperty(property); + } else { + System.setProperty(property, previous); + } + } + } + /** * The rule, not a snapshot: no test in this module may read a model path with a bare * {@code System.getProperty}, because that bypasses the resolver and silently re-mutes that class diff --git a/llama/spotbugs-exclude.xml b/llama/spotbugs-exclude.xml index d5e13bb8b..0514d12f9 100644 --- a/llama/spotbugs-exclude.xml +++ b/llama/spotbugs-exclude.xml @@ -46,6 +46,21 @@ SPDX-License-Identifier: MIT leaves no compile-time hint that an I/O failure happened; we accept that tradeoff at these three boundary points. --> + + + + + + + diff --git a/llama/src/main/java/net/ladenthin/llama/Diagnose.java b/llama/src/main/java/net/ladenthin/llama/Diagnose.java new file mode 100644 index 000000000..1a229ecde --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/Diagnose.java @@ -0,0 +1,124 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama; + +import java.io.PrintStream; +import net.ladenthin.llama.loader.DiagnosticReport; +import net.ladenthin.llama.loader.LlamaLoader; +import net.ladenthin.llama.value.LlamaCppVersion; + +/** + * Prints what this library sees of its own installation, so that "it does not load" becomes one + * paste instead of a conversation. + * + *

Every question a failed load actually raises is answerable from inside the process: which + * platform the JVM reports, which natives jars are on the classpath and which llama.cpp build each + * one carries, which library backend and GPU modules the loader selects, and -- when the load fails + * -- the operating system's own reason for refusing the library. This class prints exactly that. + * + * 

{@code
+ * java -cp 'libs/*' net.ladenthin.llama.Diagnose
+ * }
+ * + *

The report is split across two classes for a reason worth knowing before moving anything: + * every platform and classpath fact comes from a package-private member of + * {@code LlamaLoader}, and Java access is per package rather than per module -- so that half lives + * in {@code net.ladenthin.llama.loader.DiagnosticReport}. The main class has to be here instead, + * because the loader's package is deliberately not exported and a consumer on the module path could + * not run a main class in it. The architecture rule allows this direction and not the other, which + * is also why the pinned-version constant and the load itself are read here rather than there. + * + *

No probe throws. A diagnostic that fails while reporting a failure is worse than none, so + * each one reports its own problem as a line of the output instead. + * + *

The one exception is deliberate and is at the very end: when the library did not load, + * {@link #main(String[])} throws after printing the whole report, so the process exits non-zero and a + * script can tell. It is not {@code System.exit} -- the project's architecture rule forbids that in + * every class here and says to throw instead, which happens to be the better shape anyway: a caller + * that invokes {@code main} programmatically keeps its JVM, and {@link #report(PrintStream)} returns + * the outcome as a value for anyone who wants neither. + */ +public final class Diagnose { + + /** The section separator, narrow enough to survive a paste into an issue comment. */ + private static final String RULE = "--------------------------------------------------------"; + + private Diagnose() { + // Not instantiable: a main class, not an object. + } + + /** + * Prints the report to {@code System.out}, and throws afterwards when the native library did not + * load so that the process exits non-zero. + * + * @param args ignored, so that this is an ordinary main class + * @throws UnsatisfiedLinkError when the native library did not load -- after the report, never + * instead of it + */ + public static void main(String[] args) { + boolean loaded = report(System.out); + System.out.flush(); + if (!loaded) { + throw new UnsatisfiedLinkError("the native library did not load -- see the report above"); + } + } + + /** + * Writes the whole report, including an attempt to load the native library. This is the seam a + * test uses, and the entry point for an embedding caller that wants the report without the + * throw {@link #main(String[])} ends in. + * + * @param out where to write + * @return {@code true} when the native library loaded + */ + public static boolean report(PrintStream out) { + out.println("jllama diagnose"); + out.println(RULE); + out.println("llama.cpp pinned: " + LlamaCppVersion.LLAMA_CPP_VERSION); + DiagnosticReport.environment(out); + out.println(RULE); + DiagnosticReport.natives(out); + out.println(RULE); + return load(out); + } + + /** + * Loads the library and reports what came back -- or, on failure, the reason the operating system + * gave, which is the single most useful line of the report. + * + * @param out where to write + * @return {@code true} when the library loaded + */ + private static boolean load(PrintStream out) { + try { + LlamaLoader.initialize(); + } catch (UnsatisfiedLinkError | RuntimeException e) { + out.println("load: FAILED"); + out.println(" " + e); + String reason = DiagnosticReport.lastLoadFailure(); + if (reason != null) { + out.println(" the operating system's reason: " + reason); + } + return false; + } + out.println("load: ok"); + try { + String linked = LlamaModel.getLlamaCppBuildInfo(); + out.println("linked llama.cpp: " + linked); + if (!linked.startsWith(LlamaCppVersion.LLAMA_CPP_VERSION)) { + out.println(" WARNING: the loaded library is not the pinned " + + LlamaCppVersion.LLAMA_CPP_VERSION + + " -- a stale extraction directory, or a natives jar of another release on" + + " the classpath"); + } + } catch (UnsatisfiedLinkError | RuntimeException e) { + out.println("linked llama.cpp: could not be read: " + e); + } + out.println("The loader printed the directory it used, and ggml its backend registry and"); + out.println("devices, on stderr above -- include those lines when reporting a problem."); + return true; + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java b/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java new file mode 100644 index 000000000..6cc9e80e2 --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/loader/DiagnosticReport.java @@ -0,0 +1,301 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama.loader; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.PrintStream; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.jspecify.annotations.Nullable; + +/** + * The half of the diagnostic report that only the loader can answer: which platform this JVM + * resolves to, which natives directories are on the classpath and which llama.cpp build each of them + * carries, and -- after a failed load -- the reason the operating system gave. + * + *

It lives here, next to {@link LlamaLoader}, because every one of those facts comes from a + * package-private member of it. Java access is per package, not per module, so a class in + * {@code net.ladenthin.llama} cannot read them; and this package is deliberately not exported, so a + * public method here is still invisible to a consumer on the module path. + * + *

The counterpart is {@code net.ladenthin.llama.Diagnose}, which owns the parts this class must + * not touch: the pinned-version constant lives in {@code value} and the load itself needs + * {@code LlamaModel}, and the architecture rule allows neither direction out of this package. + * + *

Nothing here throws. Each probe reports its own failure as a line of the report, because + * a diagnostic that dies while describing a failure is worse than none. + */ +public final class DiagnosticReport { + + /** + * The prefix of every system property this library reads, with the separating dot that + * {@link LlamaSystemProperties#PREFIX} does not carry -- its own accessors supply it per suffix + * ({@code ".backend"}). Getting that wrong is not cosmetic: the first version of this class + * composed {@code PREFIX + "backend"} and read {@code net.ladenthin.llamabackend}, a name nothing + * ever sets, so the report would have claimed no filter was in effect while one was. + */ + private static final String PROPERTY_PREFIX = LlamaSystemProperties.PREFIX + "."; + + /** + * Read through the same accessors the loader uses, so the report cannot disagree with it about + * which property name carries which setting. + */ + private static final LlamaSystemProperties PROPERTIES = new LlamaSystemProperties(); + + /** The longest backend directory name, for aligning the columns of a pasted report. */ + private static final int NAME_WIDTH = "openvino".length(); + + private DiagnosticReport() { + // Not instantiable: a report builder, not an object. + } + + /** + * The JVM, the platform as {@link OSInfo} derives it, and the {@code net.ladenthin.llama.*} + * properties that are set. Needs no native library. + * + * @param out where to write + */ + public static void environment(PrintStream out) { + out.println("java: " + property("java.version") + " (" + property("java.vendor") + ")"); + out.println("os.name / os.arch: " + property("os.name") + " / " + property("os.arch")); + out.println("resolved platform: " + call(OSInfo::getNativeLibFolderPathForCurrentOS)); + out.println(" OSInfo os/arch: " + call(OSInfo::getOSName) + " / " + call(OSInfo::getArchName)); + out.println(" android / musl: " + + call(() -> String.valueOf(OSInfo.isAndroid())) + + " / " + + call(() -> String.valueOf(OSInfo.isMusl()))); + Map properties = llamaProperties(); + if (properties.isEmpty()) { + out.println("properties: none of " + PROPERTY_PREFIX + "* is set"); + } else { + out.println("properties:"); + for (Map.Entry entry : properties.entrySet()) { + out.println(" " + entry.getKey() + " = " + entry.getValue()); + } + } + } + + /** + * What the classpath offers: the library backend directory of this platform and every GPU module + * directory, each with the jar it comes from and the llama.cpp tag it was built from -- which is + * what a mismatched set looks like, and the one thing a consumer cannot otherwise see. + * + * @param out where to write + */ + public static void natives(PrintStream out) { + String base = LlamaLoader.NATIVE_RESOURCE_BASE + "/" + call(OSInfo::getNativeLibFolderPathForCurrentOS); + out.println("natives on the classpath, below " + base); + int found = 0; + for (String backend : LlamaLoader.LIBRARY_BACKENDS) { + if (describe(out, base, backend, "library")) { + found++; + } + } + List modules = new ArrayList<>(); + for (String backend : LlamaLoader.MODULE_BACKENDS) { + if (describe(out, base, backend, "module ")) { + found++; + modules.add(backend); + } + } + if (found == 0) { + out.println(" (none -- no natives jar for this platform is on the classpath)"); + } + String filter = PROPERTIES.getBackend(); + out.println("backend property: " + (filter == null ? "unset (every module is used)" : filter)); + for (String backend : modules) { + if (LlamaLoader.OPT_IN_BACKENDS.contains(backend) && !namedBy(filter, backend)) { + out.println(" note: module '" + + backend + + "' is opt-in and is NOT used -- name it in " + + PROPERTY_PREFIX + + "backend to enable it"); + } + } + } + + /** + * The reason a failed {@code System.load} gave, which is the single most useful line of the + * report: the library is usually present and one of its dependencies is not. + * + * @return the reason, or {@code null} when no load has failed + */ + public static @Nullable String lastLoadFailure() { + return LlamaLoader.lastLoadFailure; + } + + /** + * Whether a comma-separated backend filter names a backend, matched exactly the way + * {@code LlamaLoader.selectModules} matches it: split on commas, each entry trimmed, then + * compared with {@code equals}. + * + *

That mirroring is the point, and the first version of this method got it wrong in a way + * worth keeping on record: it compared case-insensitively, which is more lenient than the loader. + * The loader fails loud on a name it does not know, so {@code OpenVino} makes the load throw -- + * while a lenient report would have called that module enabled. A diagnostic may not be kinder + * than the code it describes; being a line more cautious costs nothing, being a line more + * generous is a wrong answer. + * + * @param filter the property value, or {@code null} when unset + * @param backend the backend name + * @return {@code true} when the filter names it, by the loader's own comparison + */ + public static boolean namedBy(@Nullable String filter, String backend) { + if (filter == null) { + return false; + } + // The same split the loader uses, with the same suppression and for the same reason: the + // "trailing empties dropped" quirk is benign because an empty entry names no backend. + @SuppressWarnings("StringSplitter") + final String[] entries = filter.split(","); + for (String entry : entries) { + if (entry.trim().equals(backend)) { + return true; + } + } + return false; + } + + /** + * Every {@code net.ladenthin.llama.*} system property that is set, sorted by name. + * + * @return the properties, possibly empty + */ + public static Map llamaProperties() { + List names = new ArrayList<>(); + try { + for (String name : System.getProperties().stringPropertyNames()) { + if (name.startsWith(PROPERTY_PREFIX)) { + names.add(name); + } + } + } catch (SecurityException e) { + Map refused = new LinkedHashMap<>(1); + refused.put("(properties unreadable)", e.toString()); + return refused; + } + Collections.sort(names); + Map found = new LinkedHashMap<>(names.size()); + for (String name : names) { + found.put(name, String.valueOf(System.getProperty(name))); + } + return found; + } + + /** + * One natives directory, if it is on the classpath at all. + * + * @param out where to write + * @param base the platform's resource prefix + * @param backend the backend directory name + * @param kind {@code library} or {@code module}, for the output + * @return {@code true} when the directory is present + */ + private static boolean describe(PrintStream out, String base, String backend, String kind) { + String name = base + "/" + backend + "/" + LlamaLoader.BACKEND_BUILD_FILE; + URL url = resourceOrNull(name); + if (url == null) { + return false; + } + out.println(" " + kind + " " + pad(backend) + " llama.cpp=" + tagOf(name) + " " + url); + return true; + } + + /** + * The llama.cpp tag a {@code jllama-build.txt} names, read through the loader's own parser so the + * two cannot disagree on the format. + * + * @param name the build file's resource name + * @return the tag, or a short description of why it could not be read + */ + private static String tagOf(String name) { + try (InputStream in = LlamaLoader.resourceAsStream(name)) { + if (in == null) { + return "(vanished)"; + } + BufferedReader reader = new BufferedReader(new InputStreamReader(in, StandardCharsets.UTF_8)); + String tag = LlamaLoader.parseBuild(reader).get(LlamaLoader.BUILD_KEY_LLAMA_CPP); + return tag != null ? tag : "(not named)"; + } catch (IOException | RuntimeException e) { + return "(unreadable: " + e + ")"; + } + } + + /** + * Looks a resource up the way the loader does, never throwing. + * + * @param name the resource name + * @return the URL, or {@code null} when absent or unreadable + */ + private static @Nullable URL resourceOrNull(String name) { + try { + return LlamaLoader.resource(name); + } catch (RuntimeException | LinkageError e) { + return null; + } + } + + /** + * Pads a backend name so the columns line up for whoever reads a pasted report. + * + * @param backend the name + * @return the name, padded to the width of the longest backend name + */ + private static String pad(String backend) { + StringBuilder padded = new StringBuilder(backend); + while (padded.length() < NAME_WIDTH) { + padded.append(' '); + } + return padded.toString(); + } + + /** + * A system property, or a placeholder -- never an exception. + * + * @param name the property name + * @return the value, or {@code (unset)} + */ + private static String property(String name) { + String value = System.getProperty(name); + return value != null ? value : "(unset)"; + } + + /** + * Calls a probe that may throw, so one unhappy line cannot take the report down. + * + * @param probe the probe + * @return its value, or a short description of the failure + */ + private static String call(Probe probe) { + try { + String value = probe.get(); + return value != null ? value : "(null)"; + } catch (RuntimeException | LinkageError e) { + return "(failed: " + e + ")"; + } + } + + /** A probe that returns one value of the report and is allowed to fail. */ + @FunctionalInterface + interface Probe { + + /** + * Reads one value. + * + * @return the value, or {@code null} + */ + @Nullable + String get(); + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java index 3a7bf649c..b211a60a1 100644 --- a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java +++ b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java @@ -1059,7 +1059,7 @@ static String getNativeResourcePath() { return classLoader().getResource(name); } - private static @Nullable InputStream resourceAsStream(String name) { + static @Nullable InputStream resourceAsStream(String name) { return classLoader().getResourceAsStream(name); } diff --git a/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java b/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java new file mode 100644 index 000000000..08f98c43a --- /dev/null +++ b/llama/src/test/java/net/ladenthin/llama/DiagnoseTest.java @@ -0,0 +1,96 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama; + +import static org.junit.jupiter.api.Assertions.*; + +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; +import java.io.UnsupportedEncodingException; +import java.nio.charset.StandardCharsets; +import net.ladenthin.llama.value.LlamaCppVersion; +import org.junit.jupiter.api.Test; + +/** + * The shipped diagnostic entry point. + * + *

These tests need no native library and no model, which is the point: the report exists for the + * case where the library does not load, so it has to be exercised in exactly that state. + * The suite therefore runs both ways -- with natives on the classpath (CI, or after a local cmake + * build) and without (a plain checkout) -- and every assertion here holds for both. + * + *

{@code Diagnose.main} is never called here: on a checkout without natives it ends by throwing, + * which is the whole point of it (the process has to exit non-zero), but says nothing a test of the + * report wants. {@link Diagnose#report(PrintStream)} is the seam that exists for that. + * + *

The pure helpers live in {@code loader.DiagnosticReport} and are covered by its own test. + */ +@ClaudeGenerated( + purpose = "Verify the shipped diagnostic entry point without a native library: report() never " + + "throws and always states exactly one of the two load outcomes (the property " + + "the whole class exists for -- a diagnostic that fails while reporting a " + + "failure is worse than none); it names the pinned llama.cpp version, the " + + "JVM's os.name/os.arch and the resolved platform directory; and it either " + + "lists the natives directories of this platform with the llama.cpp tag of " + + "each jllama-build.txt or states that there is none, never staying silent.") +public class DiagnoseTest { + + /** + * Runs the full report into a buffer. + * + * @return the report text + */ + private static String report() { + ByteArrayOutputStream buffer = new ByteArrayOutputStream(); + try (PrintStream out = new PrintStream(buffer, true, StandardCharsets.UTF_8.name())) { + Diagnose.report(out); + } catch (UnsupportedEncodingException e) { + throw new AssertionError("every JVM is required to support UTF-8", e); + } + return new String(buffer.toByteArray(), StandardCharsets.UTF_8); + } + + /** + * The property this class exists for: the report has to survive the state it describes. A + * diagnostic that throws while reporting a failure is worse than no diagnostic at all, so this + * is the assertion that matters most -- the others only check that the content is useful. + */ + @Test + public void theReportNeverThrowsWhetherTheNativeLibraryLoadsOrNot() { + String text = report(); + assertFalse(text.isEmpty(), "the report must not be empty"); + // Exactly one of the two outcomes, and never neither: the load section always says what + // happened, so a reader is not left guessing whether it even got that far. + boolean ok = text.contains("load: ok"); + boolean failed = text.contains("load: FAILED"); + assertTrue(ok || failed, "the report must state the load outcome, got:\n" + text); + assertNotEquals(ok, failed, "the load outcome must be exactly one of ok and FAILED"); + } + + /** The platform half is what whoever reports a problem is asked for first. */ + @Test + public void theReportNamesThePinnedVersionTheJvmAndTheResolvedPlatform() { + String text = report(); + assertTrue(text.contains(LlamaCppVersion.LLAMA_CPP_VERSION), "the pinned version must appear"); + assertTrue(text.contains(System.getProperty("os.name")), "os.name must appear verbatim"); + assertTrue(text.contains(System.getProperty("os.arch")), "os.arch must appear verbatim"); + assertTrue(text.contains("resolved platform:"), "the derived platform directory must appear"); + } + + /** + * Either there are natives for this platform and each is named with the llama.cpp tag of its + * {@code jllama-build.txt} -- which is how a mismatched set becomes visible -- or the report says + * there are none. Both are verdicts; saying nothing would not be. + */ + @Test + public void theReportEitherListsTheNativesWithTheirBuildTagOrSaysThereAreNone() { + String text = report(); + boolean none = text.contains("(none -- no natives jar for this platform is on the classpath)"); + boolean listed = text.contains("llama.cpp="); + assertTrue(none || listed, "the natives section must reach a verdict, got:\n" + text); + assertNotEquals(none, listed, "it cannot both list a natives directory and report none"); + } +} diff --git a/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java b/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java new file mode 100644 index 000000000..4cd67b73a --- /dev/null +++ b/llama/src/test/java/net/ladenthin/llama/loader/DiagnosticReportTest.java @@ -0,0 +1,170 @@ +/* + * SPDX-FileCopyrightText: 2026 Bernard Ladenthin + * + * SPDX-License-Identifier: MIT OR Apache-2.0 + */ +package net.ladenthin.llama.loader; + +import static org.junit.jupiter.api.Assertions.*; + +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; +import java.io.UnsupportedEncodingException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import net.ladenthin.llama.ClaudeGenerated; +import org.junit.jupiter.api.Test; + +/** + * The loader half of the diagnostic report: the pure helpers, and that each section writes something + * rather than throwing. + * + *

No native library and no model: these tests run in a plain checkout. + */ +@ClaudeGenerated( + purpose = "Verify the loader half of the diagnostic report without a native library: namedBy " + + "matches the comma-separated net.ladenthin.llama.backend filter entry by " + + "entry, trimmed and ignoring case, so the spacing a human writes works while " + + "a prefix is not a member (a substring test would answer that wrongly in one " + + "direction or the other); llamaProperties collects only this library's " + + "properties and returns them sorted; and environment() and natives() each " + + "write a non-empty section without throwing, naming the resolved platform " + + "and reaching a verdict on the classpath.") +public class DiagnosticReportTest { + + /** + * Captures one section. + * + * @param section the section to run + * @return what it wrote + */ + private static String capture(java.util.function.Consumer section) { + ByteArrayOutputStream buffer = new ByteArrayOutputStream(); + try (PrintStream out = new PrintStream(buffer, true, StandardCharsets.UTF_8.name())) { + section.accept(out); + } catch (UnsupportedEncodingException e) { + throw new AssertionError("every JVM is required to support UTF-8", e); + } + return new String(buffer.toByteArray(), StandardCharsets.UTF_8); + } + + /** + * The backend filter is a comma-separated list, so membership is a comparison of whole entries + * rather than a substring test -- and it tolerates the spacing and the casing a human writes. + */ + @Test + public void theBackendFilterIsMatchedEntryByEntryNotAsASubstring() { + assertTrue(DiagnosticReport.namedBy("openvino", "openvino"), "the only entry names it"); + assertTrue(DiagnosticReport.namedBy("cpu,openvino", "openvino"), "one of several entries names it"); + assertTrue(DiagnosticReport.namedBy("cpu, openvino", "openvino"), "the spacing a human writes works"); + // NOT case-insensitive, deliberately: LlamaLoader.selectModules compares with equals and + // fails loud on a name it does not know, so "OpenVino" makes the load throw. A report that + // called that module enabled would be kinder than the code it describes, which is a wrong + // answer rather than a lenient one. (The first version of namedBy did exactly that.) + assertFalse(DiagnosticReport.namedBy(" OpenVino ", "openvino"), "the loader is case-sensitive"); + assertTrue(DiagnosticReport.namedBy(" openvino ", "openvino"), "but surrounding spaces are trimmed"); + assertFalse(DiagnosticReport.namedBy("cuda13,vulkan", "openvino"), "other entries do not name it"); + // A prefix is not a member. A substring test would answer this wrongly in one direction or + // the other, depending on which of the two strings it searched. + assertFalse(DiagnosticReport.namedBy("openvino", "openv"), "a prefix of an entry is not that entry"); + assertFalse(DiagnosticReport.namedBy("", "openvino"), "an empty filter names nothing"); + assertFalse(DiagnosticReport.namedBy(null, "openvino"), "an unset filter names nothing"); + } + + /** Only this library's properties are collected, and in a stable order. */ + @Test + public void onlyTheLibrariesOwnPropertiesAreCollectedAndTheyAreSorted() { + String mine = LlamaSystemProperties.PREFIX + ".diagnose.test.zzz"; + String alsoMine = LlamaSystemProperties.PREFIX + ".diagnose.test.aaa"; + String foreign = "some.other.product.diagnose.test"; + try { + System.setProperty(mine, "1"); + System.setProperty(alsoMine, "2"); + System.setProperty(foreign, "3"); + + Map properties = DiagnosticReport.llamaProperties(); + assertEquals("1", properties.get(mine), "a property of this library is collected"); + assertEquals("2", properties.get(alsoMine), "and so is the other one"); + assertFalse(properties.containsKey(foreign), "a foreign property is not"); + + List keys = new ArrayList<>(properties.keySet()); + List sorted = new ArrayList<>(keys); + Collections.sort(sorted); + assertEquals(sorted, keys, "the properties are reported in sorted order"); + } finally { + System.clearProperty(mine); + System.clearProperty(alsoMine); + System.clearProperty(foreign); + } + } + + /** A set property has to be visible in the paste, because a wrong one is a common cause. */ + @Test + public void theEnvironmentSectionNamesThePlatformAndEverySetProperty() { + String key = LlamaSystemProperties.PREFIX + ".diagnose.test.marker"; + String previous = System.getProperty(key); + try { + System.setProperty(key, "a-value-only-this-test-sets"); + String text = capture(DiagnosticReport::environment); + assertTrue(text.contains("resolved platform:"), "the derived platform directory appears"); + assertTrue(text.contains(OSInfo.getNativeLibFolderPathForCurrentOS()), "and its value"); + assertTrue(text.contains(key + " = a-value-only-this-test-sets"), "a set property appears"); + } finally { + if (previous == null) { + System.clearProperty(key); + } else { + System.setProperty(key, previous); + } + } + } + + /** + * The natives section reaches a verdict either way: it lists each directory with the llama.cpp + * tag of its {@code jllama-build.txt}, or states that there is none. Silence would be the one + * useless outcome. + */ + @Test + public void theNativesSectionReachesAVerdictWithOrWithoutNativesOnTheClasspath() { + String text = capture(DiagnosticReport::natives); + boolean none = text.contains("(none -- no natives jar for this platform is on the classpath)"); + boolean listed = text.contains("llama.cpp="); + assertTrue(none || listed, "the section must reach a verdict, got:\n" + text); + assertNotEquals(none, listed, "it cannot both list a directory and report none"); + assertTrue(text.contains("backend property:"), "the backend filter is always reported"); + } + + /** + * The report has to read the same property the loader reads. This is the test that would + * have caught the first version of this class: it composed the name from a prefix that carries no + * separating dot and read {@code net.ladenthin.llamabackend}, so a filter that was in effect was + * reported as unset -- a diagnostic stating the opposite of the truth about the one setting that + * decides which modules load. + */ + @Test + public void theBackendFilterIsReadUnderTheNameTheLoaderActuallyUses() { + String real = LlamaSystemProperties.PREFIX + ".backend"; + String previous = System.getProperty(real); + try { + System.setProperty(real, "cpu"); + String text = capture(DiagnosticReport::natives); + assertTrue(text.contains("backend property: cpu"), "the filter must be shown: " + text); + } finally { + if (previous == null) { + System.clearProperty(real); + } else { + System.setProperty(real, previous); + } + } + } + + /** No load has failed in this JVM unless one was attempted, and the accessor must not throw. */ + @Test + public void theLastLoadFailureAccessorIsReadableWithoutALoad() { + // Deliberately no assertion on the value: another test in the same JVM may have attempted a + // load. What is pinned is that reading it is safe, which is what the report relies on. + DiagnosticReport.lastLoadFailure(); + } +}