From 5cb3eddd51eb8c3b960a74be6bfd1bdf873886c0 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 30 Sep 2026 21:46:16 +0000 Subject: [PATCH 01/22] Ship natives as one jar per backend, declared once in natives.csv net.ladenthin:llama is now the Java classes only. Every native build ships as its own jar of the same artifact, classifier --, holding exactly one directory net/ladenthin/llama////. The directories never overlap, so any combination of natives jars shares one classpath: LlamaLoader tries the backends it finds in a fixed order (GPU backends, metal, msvc, cpu) through its ClassLoader and falls through to the CPU library when a GPU runtime is missing. A GPU jar next to the CPU jar now has a CPU fallback; before, each GPU classifier was a complete replacement jar with its own compile pass and none. - llama-platform: new pom-packaging module (nothing to upload besides the pom) naming the classes jar plus the CPU/Metal jars of every desktop. - .github/natives.csv is the single list of natives jars. The merge and fat-jar scripts read it; check-natives.py (code-style job) fails when the generated pom executions, llama-platform, the workflow artifact names or BACKEND_PRIORITY disagree with it. The 16 classifier profiles (17 compile passes) became one generated `natives` profile: pom 2278 -> ~1570 lines. - CMake writes to src/main/natives////; build jobs upload natives-, and the package/publish jobs fetch them with one glob instead of ~20 download steps each. The merge checks every listed artifact arrived holding only its own directory. - The all-backends fat jars are plain merges carrying only their own OS/arch (all-windows-x86-64 was ~71.5 MB of other platforms' CPU natives); the jllama-backends.txt manifest is gone. Sibling files a backend needs first are listed in a per-directory jllama-extras.txt. - Module path: each natives jar declares a unique Automatic-Module-Name. Without it every natives jar derives the module name `llama` and the JVM silently keeps only the first (measured); package-fatjars.sh checks the manifest of every built jar. module-info now requires Jackson and SLF4J, without which the classes jar never worked on the module path. - smoke-natives-jars.sh loads the classes jar with all 26 natives jars at once, on the classpath and on the module path, in the package job. - Three test classes checked for the library at a hard-coded path and would have skipped silently on the new layout; one shared NativeLibraryPresence helper replaces them. - The Atmosphere agent depends on llama-platform (opt out with -Dllama.natives=none, as CI does); the langchain4j and agent integration jobs point the loader at the downloaded library via lib.path. Verified locally with a real Linux x86-64 build plus placeholder libraries for the other 25 directories: 1823 tests green, all 26 natives jars and 4 all-backends jars built and checked, the fat jar and the natives jars load on classpath and module path. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/check-natives.py | 135 ++ .github/merge-native-artifacts.sh | 91 +- .github/natives.csv | 41 + .github/package-fatjars.sh | 256 ++- .github/smoke-native-macos.sh | 4 +- .github/smoke-natives-jars.sh | 47 + .github/smoke-test-fatjar.sh | 18 +- .github/verify-native-deps.py | 79 +- .github/workflows/publish.yml | 628 +++---- .gitignore | 22 +- CLAUDE.md | 474 ++--- README.md | 194 +- REUSE.toml | 6 +- TODO.md | 19 +- llama-android/build.gradle.kts | 21 +- llama-atmosphere-agent/README.md | 16 +- llama-atmosphere-agent/pom.xml | 51 +- .../src/assembly/agent-jar.xml | 4 +- llama-langchain4j/README.md | 5 +- llama-platform/pom.xml | 104 ++ llama/CMakeLists.txt | 100 +- llama/pom.xml | 1565 +++++------------ llama/spotbugs-exclude.xml | 12 - llama/src/assembly/fat-jar.xml | 14 +- llama/src/main/java/module-info.java | 27 +- .../ladenthin/llama/loader/LlamaLoader.java | 274 ++- .../llama/loader/LlamaSystemProperties.java | 9 +- .../net/ladenthin/llama/LlamaLoggerTest.java | 22 +- .../ladenthin/llama/RpcIntegrationTest.java | 3 +- .../net/ladenthin/llama/RpcServerTest.java | 10 +- ...festLoadTest.java => BackendLoadTest.java} | 99 +- .../llama/loader/LlamaLoaderTest.java | 147 +- .../loader/NativeLibraryLoadSmokeTest.java | 23 +- .../llama/loader/NativeLibraryPresence.java | 34 + .../backendtest-ok/cuda13/jllama-extras.txt | 6 + .../{clash => cuda13}/libextra.so | Bin .../{extrafail => cuda13}/libjllama.so | 0 .../Linux/backendtest-ok/jllama-backends.txt | 11 - .../backendtest-ok/rocm/jllama-extras.txt | 6 + .../{extrafail => rocm}/libextra.so | Bin .../{clash => rocm}/libjllama.so | 0 .../{workinggpu => vulkan}/libjllama.so | Bin .../backendtest/cuda13/jllama-extras.txt | 6 + .../{fakegpu => cuda13}/libextra.so | 0 .../{fakegpu => cuda13}/libjllama.so | 0 .../Linux/backendtest/jllama-backends.txt | 10 - .../Linux/backendtest/rocm/jllama-extras.txt | 6 + .../{missingextra => rocm}/libjllama.so | 0 .../{fallbackgpu => vulkan}/libjllama.so | 0 pom.xml | 1 + 50 files changed, 1938 insertions(+), 2662 deletions(-) create mode 100644 .github/check-natives.py create mode 100644 .github/natives.csv create mode 100755 .github/smoke-natives-jars.sh create mode 100644 llama-platform/pom.xml rename llama/src/test/java/net/ladenthin/llama/loader/{BackendManifestLoadTest.java => BackendLoadTest.java} (55%) create mode 100644 llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryPresence.java create mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/jllama-extras.txt rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/{clash => cuda13}/libextra.so (100%) rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/{extrafail => cuda13}/libjllama.so (100%) delete mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/jllama-backends.txt create mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/jllama-extras.txt rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/{extrafail => rocm}/libextra.so (100%) rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/{clash => rocm}/libjllama.so (100%) rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/{workinggpu => vulkan}/libjllama.so (100%) create mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/jllama-extras.txt rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/{fakegpu => cuda13}/libextra.so (100%) rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/{fakegpu => cuda13}/libjllama.so (100%) delete mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/jllama-backends.txt create mode 100644 llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/jllama-extras.txt rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/{missingextra => rocm}/libjllama.so (100%) rename llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/{fallbackgpu => vulkan}/libjllama.so (100%) diff --git a/.github/check-natives.py b/.github/check-natives.py new file mode 100644 index 000000000..758e917ca --- /dev/null +++ b/.github/check-natives.py @@ -0,0 +1,135 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +"""Fail when anything that names the natives jars disagrees with .github/natives.csv. + +The list is the one place a natives jar is declared. Everything else either reads it (the merge, +the fat-jar assembly) or must repeat it, and is checked here: + * llama/pom.xml one jar execution per row: classifier, directory, Automatic-Module-Name + * llama-platform depends on exactly the rows marked platform=yes + * publish.yml a build job uploads natives- for every row, and no other + * LlamaLoader BACKEND_PRIORITY tries every backend (else a jar ships and never loads) + +Usage: + check-natives.py check, exit 1 on any disagreement + check-natives.py pom print the natives jar executions for llama/pom.xml +""" + +import csv +import os +import re +import sys +import xml.etree.ElementTree as ET + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +NS = {"m": "http://maven.apache.org/POM/4.0.0"} + + +def rows(): + with open(os.path.join(ROOT, ".github", "natives.csv"), encoding="utf-8") as f: + lines = [line for line in f if line.strip() and not line.startswith("#")] + return list(csv.DictReader(lines)) + + +def module_name(classifier): + """Each natives jar needs its own: without one, every jar derives the module name `llama` + from its file name, and the module path silently keeps only the first.""" + return "net.ladenthin.llama.natives." + classifier.replace("-", "_") + + +def pom_execution(row): + return f"""\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\tnatives-{row['classifier']} +\t\t\t\t\t\t\t\tpackage +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\tjar +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t{row['classifier']} +\t\t\t\t\t\t\t\t\t${{project.basedir}}/src/main/natives +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\tnet/ladenthin/llama/{row['directory']}/** +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\tfalse +\t\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\t\t{module_name(row['classifier'])} +\t\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t""" + + +def pom_natives(path): + """classifier -> (include, module name) for every jar execution of the natives profile.""" + tree = ET.parse(path) + found = {} + for profile in tree.getroot().iterfind("m:profiles/m:profile", NS): + if profile.findtext("m:id", namespaces=NS) != "natives": + continue + for ex in profile.iterfind(".//m:plugin/m:executions/m:execution", NS): + conf = ex.find("m:configuration", NS) + classifier = conf.findtext("m:classifier", namespaces=NS) if conf is not None else None + if classifier: + found[classifier] = (conf.findtext("m:includes/m:include", namespaces=NS), + conf.findtext("m:archive/m:manifestEntries/m:Automatic-Module-Name", + namespaces=NS)) + return found + + +def compare(what, expected, actual, failures): + for name in sorted(expected - actual): + failures.append(f"{what}: missing {name}") + for name in sorted(actual - expected): + failures.append(f"{what}: {name} is not in .github/natives.csv") + + +def main(argv): + natives = rows() + if len(argv) > 1 and argv[1] == "pom": + print("\n".join(pom_execution(r) for r in natives)) + return 0 + failures = [] + by_classifier = {r["classifier"]: r for r in natives} + if len(by_classifier) != len(natives): + failures.append("natives.csv lists a classifier twice") + for r in natives: + backend, rest = r["directory"].rsplit("/", 1)[1], r["classifier"] + if not rest.startswith(backend + "-") or r["platform"] not in ("yes", "no"): + failures.append(f"natives.csv: row {r['classifier']} -- classifier must start with its " + f"directory's backend '{backend}', platform must be yes or no") + + pom = pom_natives(os.path.join(ROOT, "llama", "pom.xml")) + compare("llama/pom.xml natives profile", set(by_classifier), set(pom), failures) + for classifier in sorted(set(pom) & set(by_classifier)): + want = (f"net/ladenthin/llama/{by_classifier[classifier]['directory']}/**", module_name(classifier)) + if pom[classifier] != want: + failures.append(f"llama/pom.xml: {classifier} has {pom[classifier]}, expected {want} " + f"(check-natives.py pom prints the executions)") + + platform = ET.parse(os.path.join(ROOT, "llama-platform", "pom.xml")) + deps = {d.findtext("m:classifier", namespaces=NS) + for d in platform.getroot().iterfind("m:dependencies/m:dependency", NS)} - {None} + compare("llama-platform/pom.xml", {r["classifier"] for r in natives if r["platform"] == "yes"}, deps, failures) + + with open(os.path.join(ROOT, ".github", "workflows", "publish.yml"), encoding="utf-8") as f: + uploads = set(re.findall(r"upload-artifact@\S+\s+with:\s+name:\s*natives-(\S+)", f.read())) + compare("publish.yml natives-* uploads", set(by_classifier), uploads, failures) + + loader = os.path.join(ROOT, "llama", "src", "main", "java", "net", "ladenthin", "llama", "loader", + "LlamaLoader.java") + with open(loader, encoding="utf-8") as f: + block = re.search(r"BACKEND_PRIORITY\s*=(.*?);", f.read(), re.S) + priority = set(re.findall(r'"([^"]+)"', block.group(1))) if block else set() + for backend in sorted({r["directory"].rsplit("/", 1)[1] for r in natives} - priority): + failures.append(f"LlamaLoader.BACKEND_PRIORITY does not try '{backend}' -- its jars would never load") + + for f in failures: + print(f"::error::{f}", file=sys.stderr) + print(f"{len(natives)} natives jars, {len(failures)} disagreements") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/.github/merge-native-artifacts.sh b/.github/merge-native-artifacts.sh index eb071dc5e..095895791 100755 --- a/.github/merge-native-artifacts.sh +++ b/.github/merge-native-artifacts.sh @@ -4,43 +4,41 @@ # # SPDX-License-Identifier: MIT OR Apache-2.0 -# Merges the per-artifact native-library trees downloaded by the `*-libraries` glob into the -# single default-JAR resource tree — and FAILS LOUD if two artifacts claim the same file path. +# Merges the per-build native-library artifacts downloaded by the `natives-*` glob into the one +# natives tree (llama/src/main/natives/net/ladenthin/llama/) that the `natives` Maven profile +# packages — and FAILS LOUD when an artifact does not hold exactly what its name promises. # -# Why this exists: the `package` / `publish-snapshot` / `publish-release` jobs pull every build -# job whose artifact name ends in `-libraries` with one globbed `actions/download-artifact`. -# That is convenient (a new CPU platform ships in the default JAR by naming its artifact -# `-libraries`, no packaging change) but it is silently unsafe: an artifact name says -# nothing about which `{OS}/{ARCH}` subdirectory the job's CMake run actually wrote. When two -# artifacts carry the same relative path, `merge-multiple: true` extracts both onto that one -# path and the survivor can be a byte-level hybrid of the two, not either input. +# Every shipped build uploads its tree as `natives-`, one per row of +# .github/natives.csv, which names the one directory it may write. Checked before anything is +# merged: +# * every listed artifact arrived, with its library, and no unlisted one did; +# * every file lies below the artifact's own directory. A build whose CMake picked another +# backend name or platform (the routing lives in llama/CMakeLists.txt) would otherwise land in +# some other natives jar, or in none; +# * no relative path is claimed by two artifacts. This follows from the second check, and is +# kept anyway because it is the failure that actually shipped: all three macOS build jobs +# wrote Mac/aarch64 and used to share one glob, the merge produced a byte-level hybrid of two +# dylibs, and macOS SIGKILLed every process that loaded it (5.0.6 and several 5.0.7 +# snapshots). A check on the merged tree cannot see it — the collision leaves exactly one +# file on the path, a corrupt one — so it has to run before the merge. # -# That is exactly what happened to macOS arm64: all three macOS build jobs write -# `Mac/aarch64/libjllama.dylib` (none of them passes -DOS_NAME/-DOS_ARCH, so CMakeLists -# auto-detects the same subdir) and all three used to upload under a `*-libraries` name. The -# published dylib became a hybrid whose ad-hoc linker signature no longer matched its own -# __TEXT pages, so macOS SIGKILLed every process that loaded it — shipped broken in 5.0.6 and -# several 5.0.7 snapshots. The immediate fix renamed those artifacts out of the glob; this -# script is the backstop that stops the same hole from being reopened by a future job. -# -# NOTE ON WHAT *CANNOT* WORK AS A GUARD: asserting "exactly one library per {OS}/{ARCH}" on the -# merged tree does not detect this. The collision overwrites one path, so the merged tree still -# holds exactly one file there — a corrupt one. The collision is only observable BEFORE the -# merge, which is why this script does the merge itself instead of checking afterwards. +# After the merge, a backend directory holding files beside its library gets a +# jllama-extras.txt listing them; LlamaLoader loads those first (e.g. the OpenCL ICD loader that +# OpenVINO ships on Windows). # # Usage: merge-native-artifacts.sh -# output of `actions/download-artifact` with `pattern: "*-libraries"` and +# output of `actions/download-artifact` with `pattern: "natives-*"` and # `merge-multiple: false`, i.e. one subdirectory per artifact name. -# the tree the artifacts are merged into, e.g. -# llama/src/main/resources/net/ladenthin/llama/ +# the tree the artifacts are merged into, +# llama/src/main/natives/net/ladenthin/llama/ # -# Fail-loud: aborts when the staging directory holds no artifacts (a silently empty default JAR -# is worse than a red job) and when any relative path is claimed by more than one artifact. +# Fail-loud: also aborts when the staging directory holds no artifacts. set -euo pipefail STAGING="${1:?usage: merge-native-artifacts.sh }" DEST="${2:?usage: merge-native-artifacts.sh }" +LIST="$(dirname "$0")/natives.csv" if [ ! -d "$STAGING" ]; then echo "::error::staging directory '$STAGING' does not exist — the globbed download did not run." >&2 @@ -52,12 +50,31 @@ artifacts=() while IFS= read -r d; do artifacts+=("$(basename "$d")"); done < <(find "$STAGING" -mindepth 1 -maxdepth 1 -type d | sort) if [ "${#artifacts[@]}" -eq 0 ]; then - echo "::error::no '*-libraries' artifacts found in '$STAGING' — the default JAR would ship without native libraries." >&2 + echo "::error::no 'natives-*' artifacts found in '$STAGING' — there would be no natives jars." >&2 exit 1 fi echo "Merging ${#artifacts[@]} native-library artifact(s) into $DEST" -for a in "${artifacts[@]}"; do echo " - $a"; done +listed=0 +while IFS=, read -r classifier dir lib _; do + listed=$((listed + 1)) + a="natives-$classifier" + if [ ! -d "$STAGING/$a" ]; then + echo "::error::no artifact '$a' -- the build job for this row of $LIST did not upload it" >&2 + exit 1 + fi + stray="$(cd "$STAGING/$a" && find . -type f | sed 's|^\./||' | grep -v "^$dir/" || true)" + if [ -n "$stray" ] || [ ! -f "$STAGING/$a/$dir/$lib" ]; then + echo "::error::artifact '$a' must hold $dir/$lib and nothing outside $dir/; outside it:" >&2 + printf '%s\n' "$stray" | sed 's|^|::error:: |' >&2 + exit 1 + fi + echo " - $a -> $dir/" +done < <(grep -v -e '^#' -e '^classifier,' -e '^$' "$LIST") +if [ "$listed" -ne "${#artifacts[@]}" ]; then + echo "::error::$STAGING holds ${#artifacts[@]} natives-* artifacts but $LIST lists $listed: $(printf '%s ' "${artifacts[@]}")" >&2 + exit 1 +fi # relpath -> space-separated list of artifacts that carry it. Bash 3.2 (macOS) has no # associative arrays, so this stays a sorted "\t" stream processed by awk. @@ -71,16 +88,11 @@ collisions="$( )" if [ -n "$collisions" ]; then - echo "::error::two or more '*-libraries' artifacts write the same path — merging them would produce a hybrid, corrupt native library." >&2 + echo "::error::two or more 'natives-*' artifacts write the same path — merging them would produce a hybrid, corrupt native library." >&2 while IFS=$'\t' read -r path owners; do echo "::error:: $path <- claimed by:$owners" >&2 done <<< "$collisions" - cat >&2 <<'EOF' -::error::Fix: only ONE artifact per {OS}/{ARCH} may be named `*-libraries`. Rename the extra -::error::build jobs' artifacts outside the glob (as the macOS jobs do: macos-15-metal / -::error::macos-14-metal / macos-15-no-metal) and download the variant that ships explicitly by -::error::name. See CLAUDE.md, "macOS arm64: three build jobs, one shipped dylib". -EOF + echo "::error::Fix: only one shipped build per --; name a test-only variant outside the glob (see CLAUDE.md, \"macOS arm64\")." >&2 exit 1 fi @@ -89,5 +101,14 @@ for a in "${artifacts[@]}"; do cp -R "$STAGING/$a/." "$DEST/" done +# Sibling files of a backend's library are loaded before it, in name order. +find "$DEST" -mindepth 3 -maxdepth 3 -type d | sort | while IFS= read -r dir; do + extras="$(cd "$dir" && find . -maxdepth 1 -type f ! -name 'libjllama.*' ! -name 'jllama.dll' ! -name '*.metal' ! -name jllama-extras.txt | sed 's|^\./||' | sort)" + if [ -n "$extras" ]; then + printf '%s\n' "$extras" > "$dir/jllama-extras.txt" + echo "extras for ${dir#"$DEST"}: $(echo "$extras" | tr '\n' ' ')" + fi +done + echo "Merged native tree:" find "$DEST" -type f | sort | sed 's|^| |' diff --git a/.github/natives.csv b/.github/natives.csv new file mode 100644 index 000000000..e11898923 --- /dev/null +++ b/.github/natives.csv @@ -0,0 +1,41 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# The natives jars: the single list every packaging step reads (see CLAUDE.md, "Natives jars"). +# One row per jar, net.ladenthin:llama:, holding exactly one directory, +# net/ladenthin/llama//, whose name ends in the backend LlamaLoader tries. +# classifier --; the build job uploads its tree as artifact natives- +# directory //, as llama/CMakeLists.txt writes it and OSInfo resolves it +# library the file that must be in it +# platform yes = a dependency of net.ladenthin:llama-platform (the CPU natives of a desktop) +# Adding a row: add the natives jar execution to llama/pom.xml (`check-natives.py pom` prints it), +# a build job uploading natives-, and, for a new backend, its name in CMakeLists.txt +# and LlamaLoader.BACKEND_PRIORITY. .github/check-natives.py fails until all of them agree. +classifier,directory,library,platform +cpu-linux-x86-64,Linux/x86_64/cpu,libjllama.so,yes +cpu-linux-aarch64,Linux/aarch64/cpu,libjllama.so,yes +cpu-linux-s390x,Linux/s390x/cpu,libjllama.so,yes +cpu-android-aarch64,Linux-Android/aarch64/cpu,libjllama.so,no +cpu-android-x86-64,Linux-Android/x86_64/cpu,libjllama.so,no +cpu-windows-x86-64,Windows/x86_64/cpu,jllama.dll,yes +cpu-windows-x86,Windows/x86/cpu,jllama.dll,yes +cpu-windows-aarch64,Windows/aarch64/cpu,jllama.dll,yes +metal-macos-aarch64,Mac/aarch64/metal,libjllama.dylib,yes +msvc-windows-x86-64,Windows/x86_64/msvc,jllama.dll,no +msvc-windows-x86,Windows/x86/msvc,jllama.dll,no +cuda13-linux-x86-64,Linux/x86_64/cuda13,libjllama.so,no +cuda13-windows-x86-64,Windows/x86_64/cuda13,jllama.dll,no +vulkan-linux-x86-64,Linux/x86_64/vulkan,libjllama.so,no +vulkan-linux-aarch64,Linux/aarch64/vulkan,libjllama.so,no +vulkan-windows-x86-64,Windows/x86_64/vulkan,jllama.dll,no +opencl-android-aarch64,Linux-Android/aarch64/opencl,libjllama.so,no +opencl-windows-x86-64,Windows/x86_64/opencl,jllama.dll,no +opencl-windows-aarch64,Windows/aarch64/opencl,jllama.dll,no +rocm-linux-x86-64,Linux/x86_64/rocm,libjllama.so,no +rocm-windows-x86-64,Windows/x86_64/rocm,jllama.dll,no +sycl-fp16-linux-x86-64,Linux/x86_64/sycl-fp16,libjllama.so,no +sycl-fp32-linux-x86-64,Linux/x86_64/sycl-fp32,libjllama.so,no +sycl-windows-x86-64,Windows/x86_64/sycl,jllama.dll,no +openvino-linux-x86-64,Linux/x86_64/openvino,libjllama.so,no +openvino-windows-x86-64,Windows/x86_64/openvino,jllama.dll,no diff --git a/.github/package-fatjars.sh b/.github/package-fatjars.sh index 4d84ed19d..80a556ac6 100755 --- a/.github/package-fatjars.sh +++ b/.github/package-fatjars.sh @@ -4,79 +4,80 @@ # # SPDX-License-Identifier: MIT -# Assembles the per-OS "all backends" server fat jars distributed as GitHub Release -# assets (never deployed to Maven Central). +# Checks every natives jar and assembles the per-OS "all backends" server fat jars distributed +# as GitHub Release assets (never deployed to Maven Central). # -# For every OS/arch that has GPU classifier jars, the default jar-with-dependencies -# uber jar (library classes + Java runtime deps + default CPU natives for every -# platform) is copied and each backend's native tree is added under a backend -# subdirectory (net/ladenthin/llama////), together with a -# jllama-backends.txt manifest listing the backends in priority order. LlamaLoader -# reads that manifest at runtime and loads the first backend whose library loads, -# falling back to the default CPU library (see LlamaLoader.BACKEND_MANIFEST_FILE). +# A natives jar holds exactly one directory, net/ladenthin/llama////, named +# by its classifier --. Because the directories never overlap, an all-backends +# jar is a plain merge: the default fat jar (classes + Java deps + the CPU natives of every +# platform), minus every native tree of another OS/arch, plus every natives jar of its own OS/arch. +# LlamaLoader tries the backend directories in its fixed priority order (BACKEND_PRIORITY) and +# falls back to the CPU one. +# +# Only the exact OS+arch of the jar name is kept, so all-windows-x86-64 also loses Windows/x86 +# and Windows/aarch64: a 32-bit JVM finds no natives there. The default fat jar (step 5) still +# carries every platform. +# +# The natives jars are the rows of .github/natives.csv (classifier, directory, library); that +# the pom, llama-platform, the workflow and LlamaLoader agree with it is checked by +# check-natives.py. This script checks the built jars. # # Fail-loud invariants (a broken invariant must red the pipeline, never skip): -# * every in llama/pom.xml must be parseable/explicitly excluded, -# * the pom classifier set and the on-disk classifier-jar set must match exactly, -# * every expected backend library must end up inside the combined jar, -# * the Main-Class of the combined jar must survive the zip update. +# * the listed natives jars and the natives jars on disk match exactly, +# * every natives jar holds its own directory, with its library, and nothing else, and names +# the Automatic-Module-Name the module path needs (without it the JVM silently drops the jar), +# * a combined jar keeps its own CPU library (byte-identical), carries no other OS/arch, +# had at least one tree removed, keeps every .class of the default fat jar, holds every +# added backend byte-identical to its natives jar, and keeps its Main-Class. # -# Usage: package-fatjars.sh [pom] +# Usage: package-fatjars.sh # jars-dir directory holding the `llama-jars` artifact (llama/target/*.jar) # out-dir output directory for the combined fat jars + sha256 files -# pom path to llama/pom.xml (default: llama/pom.xml) set -euo pipefail -JARS_DIR="${1:?usage: package-fatjars.sh [pom]}" -OUT_DIR="${2:?usage: package-fatjars.sh [pom]}" -POM="${3:-llama/pom.xml}" +JARS_DIR="${1:?usage: package-fatjars.sh }" +OUT_DIR="${2:?usage: package-fatjars.sh }" +LIST="$(dirname "$0")/natives.csv" fail() { echo "::error::$*" >&2 exit 1 } -# Resolve to absolute paths: the zip update below runs from the staging directory. [ -d "$JARS_DIR" ] || fail "jars dir not found: $JARS_DIR" JARS_DIR="$(cd "$JARS_DIR" && pwd)" mkdir -p "$OUT_DIR" OUT_DIR="$(cd "$OUT_DIR" && pwd)" -# Classifiers that intentionally get no combined-jar treatment: -# msvc-windows CPU-only build variant, redundant with the default Windows natives -# opencl-android-aarch64 `java -jar` is not applicable on Android (AAR is the delivery path) -EXCLUDED_CLASSIFIERS=("msvc-windows" "opencl-android-aarch64") - -# Backend priority order used for the manifest (first loadable backend wins at runtime). -# A classifier whose backend token is not in this list fails the build: a new backend -# must be consciously ranked here, not silently appended. -BACKEND_PRIORITY=("cuda13" "rocm" "sycl-fp16" "sycl-fp32" "sycl" "vulkan" "opencl" "openvino") +# Backends that get no place in an all-backends jar: msvc is the same CPU build from another +# generator, so it would only be a second CPU library the loader tries before the first. +EXCLUDED_BACKENDS=" msvc " +# Platforms that get no all-backends jar: `java -jar` does not apply on Android (the AAR does). +EXCLUDED_OSES=" Linux-Android " MAIN_CLASS="net.ladenthin.llama.server.ServerLauncher" -is_excluded() { - local classifier="$1" excluded - for excluded in "${EXCLUDED_CLASSIFIERS[@]}"; do - [ "$classifier" = "$excluded" ] && return 0 - done - return 1 +# Native trees are the upper-case DIRECTORIES below net/ladenthin/llama/ (OSInfo folder names: +# Linux, Linux-Android, Mac, Windows); Java packages there are lower-case, and top-level class +# files such as LlamaModel.class are upper-case too, so a match must require the directory's +# trailing slash. Prints /, sorted. +native_trees() { + unzip -Z1 "$1" | sed -n -E 's#^net/ladenthin/llama/([A-Z][^/]*/[^/]+)/.*#\1#p' | sort -u } -backend_priority_index() { - local backend="$1" i - for i in "${!BACKEND_PRIORITY[@]}"; do - [ "${BACKEND_PRIORITY[$i]}" = "$backend" ] && { echo "$i"; return 0; } - done - return 1 +class_count() { + unzip -Z1 "$1" | grep -c '\.class$' } -# --- 1. Source of truth #1: the set declared in the pom ------------------- -[ -f "$POM" ] || fail "pom not found: $POM" -mapfile -t POM_CLASSIFIERS < <(grep -oP '\K[^<]+(?=)' "$POM" | sort -u) -[ "${#POM_CLASSIFIERS[@]}" -gt 0 ] || fail "no entries parsed from $POM" -echo "pom classifiers (${#POM_CLASSIFIERS[@]}): ${POM_CLASSIFIERS[*]}" +# --- 1. The listed natives jars, the default fat jar, and the natives jars on disk --------- +declare -A DIR LIB +LISTED=() +while IFS=, read -r classifier dir lib _; do + LISTED+=("$classifier") DIR[$classifier]="$dir" LIB[$classifier]="$lib" +done < <(grep -v -e '^#' -e '^classifier,' -e '^$' "$LIST") +[ "${#LISTED[@]}" -gt 0 ] || fail "no rows in $LIST" +mapfile -t LISTED < <(printf '%s\n' "${LISTED[@]}" | sort) -# --- 2. The base fat jar (exactly one) + the version derived from its file name -------- mapfile -t BASE_FAT_JARS < <(find "$JARS_DIR" -maxdepth 1 -name 'llama-*-jar-with-dependencies.jar' | sort) [ "${#BASE_FAT_JARS[@]}" -eq 1 ] \ || fail "expected exactly 1 default jar-with-dependencies in $JARS_DIR, got ${#BASE_FAT_JARS[@]}: ${BASE_FAT_JARS[*]:-none}" @@ -86,7 +87,6 @@ VERSION="${VERSION#llama-}" VERSION="${VERSION%-jar-with-dependencies.jar}" echo "base fat jar: $BASE_FAT_JAR (version $VERSION)" -# --- 3. Source of truth #2: the classifier jars actually built by the package job ------ DISK_CLASSIFIERS=() for jar in "$JARS_DIR"/llama-"$VERSION"-*.jar; do [ -e "$jar" ] || continue @@ -98,124 +98,94 @@ for jar in "$JARS_DIR"/llama-"$VERSION"-*.jar; do esac DISK_CLASSIFIERS+=("$classifier") done -[ "${#DISK_CLASSIFIERS[@]}" -gt 0 ] || fail "no classifier jars found in $JARS_DIR for version $VERSION" +[ "${#DISK_CLASSIFIERS[@]}" -gt 0 ] || fail "no natives jars found in $JARS_DIR for version $VERSION" mapfile -t DISK_CLASSIFIERS < <(printf '%s\n' "${DISK_CLASSIFIERS[@]}" | sort -u) - -# --- 4. Exact set equality in BOTH directions (no silent gaps, no unknown jars) -------- -if ! diff <(printf '%s\n' "${POM_CLASSIFIERS[@]}") <(printf '%s\n' "${DISK_CLASSIFIERS[@]}"); then - fail "classifier set mismatch between $POM and the jars in $JARS_DIR (see diff above)" +if ! diff <(printf '%s\n' "${LISTED[@]}") <(printf '%s\n' "${DISK_CLASSIFIERS[@]}"); then + fail "natives jars in $JARS_DIR differ from $LIST (see diff above)" fi -echo "classifier sets match (${#POM_CLASSIFIERS[@]} classifiers)" - -# --- 5. Parse every non-excluded classifier as -- ------------------- -# Associative arrays keyed by "-" in classifier notation (e.g. linux-x86-64). -declare -A TARGET_BACKENDS # target -> space-separated backend names -declare -A BACKEND_CLASSIFIER # "/" -> classifier string -for classifier in "${POM_CLASSIFIERS[@]}"; do - if is_excluded "$classifier"; then - echo "excluded from combined jars: $classifier" - continue - fi - case "$classifier" in - *-linux-x86-64) os="linux" arch="x86-64" backend="${classifier%-linux-x86-64}" ;; - *-linux-aarch64) os="linux" arch="aarch64" backend="${classifier%-linux-aarch64}" ;; - *-windows-x86-64) os="windows" arch="x86-64" backend="${classifier%-windows-x86-64}" ;; - *-windows-aarch64) os="windows" arch="aarch64" backend="${classifier%-windows-aarch64}" ;; - *) fail "unparseable classifier '$classifier': expected -- — add a parse rule or an explicit exclusion" ;; - esac - backend_priority_index "$backend" > /dev/null \ - || fail "backend '$backend' (classifier '$classifier') is not in BACKEND_PRIORITY — rank the new backend consciously" - target="$os-$arch" - TARGET_BACKENDS[$target]="${TARGET_BACKENDS[$target]:-} $backend" - BACKEND_CLASSIFIER["$target/$backend"]="$classifier" +echo "natives jars match $LIST (${#LISTED[@]})" + +# --- 2. Every natives jar holds its own directory and nothing else ------------------------- +declare -A TARGET_CLASSIFIERS # "-" -> space-separated classifiers to merge +declare -A TARGET_FOLDER # "-" -> "/ " +for classifier in "${LISTED[@]}"; do + dir="net/ladenthin/llama/${DIR[$classifier]}/" lib="${LIB[$classifier]}" + jar="$JARS_DIR/llama-$VERSION-$classifier.jar" + entries="$(unzip -Z1 "$jar" | grep -v -e '^META-INF/' -e '/$' || true)" + grep -qxF "$dir$lib" <<< "$entries" || fail "$(basename "$jar") lacks $dir$lib" + stray="$(grep -v "^$dir" <<< "$entries" || true)" + [ -z "$stray" ] || fail "$(basename "$jar") holds files outside $dir: $(echo "$stray" | tr '\n' ' ')" + # Manifest lines are folded at 72 bytes; a continuation line starts with one space. + module="$(unzip -p "$jar" META-INF/MANIFEST.MF | tr -d '\r' | sed ':a;N;$!ba;s/\n //g' \ + | sed -n 's/^Automatic-Module-Name: //p')" + [ "$module" = "net.ladenthin.llama.natives.${classifier//-/_}" ] \ + || fail "$(basename "$jar") declares Automatic-Module-Name '$module', expected net.ladenthin.llama.natives.${classifier//-/_}" + echo "natives jar OK: $classifier -> $dir ($module)" + backend="${DIR[$classifier]##*/}" tree="${DIR[$classifier]%/*}" + case "$EXCLUDED_BACKENDS" in *" $backend "*) continue ;; esac + case "$EXCLUDED_OSES" in *" ${tree%%/*} "*) continue ;; esac + target="${classifier#"$backend"-}" + TARGET_CLASSIFIERS[$target]="${TARGET_CLASSIFIERS[$target]:-} $classifier" + TARGET_FOLDER[$target]="$tree $lib" done -[ "${#TARGET_BACKENDS[@]}" -gt 0 ] || fail "no combined-jar targets derived from the classifier set" -# --- 6. Assemble one combined jar per target -------------------------------------------- +# --- 3. One combined jar per OS/arch that has a backend besides the CPU one ----------------- WORK_DIR="$(mktemp -d)" trap 'rm -rf "$WORK_DIR"' EXIT -for target in $(printf '%s\n' "${!TARGET_BACKENDS[@]}" | sort); do - case "$target" in - *-x86-64) os="${target%-x86-64}" arch="x86-64" ;; - *-aarch64) os="${target%-aarch64}" arch="aarch64" ;; - *) fail "internal error: unexpected target '$target'" ;; - esac - # Classifier notation -> resource-folder notation (OSInfo folder names). - case "$os" in - linux) os_folder="Linux" main_lib="libjllama.so" ;; - windows) os_folder="Windows" main_lib="jllama.dll" ;; - *) fail "internal error: unexpected os '$os'" ;; - esac - case "$arch" in - x86-64) arch_folder="x86_64" ;; - aarch64) arch_folder="aarch64" ;; - *) fail "internal error: unexpected arch '$arch'" ;; - esac - resource_dir="net/ladenthin/llama/$os_folder/$arch_folder" - - # Order this target's backends by BACKEND_PRIORITY. - ordered_backends=() - for backend in "${BACKEND_PRIORITY[@]}"; do - case " ${TARGET_BACKENDS[$target]} " in - *" $backend "*) ordered_backends+=("$backend") ;; - esac - done +for target in $(printf '%s\n' "${!TARGET_CLASSIFIERS[@]}" | sort); do + classifiers="${TARGET_CLASSIFIERS[$target]}" + [ "$(wc -w <<< "$classifiers")" -gt 1 ] || continue + case "$classifiers " in *" cpu-$target "*) ;; *) fail "$target has GPU backends but no cpu-$target natives jar" ;; esac + read -r tree lib <<< "${TARGET_FOLDER[$target]}" + own="net/ladenthin/llama/$tree/" staging="$WORK_DIR/$target" - mkdir -p "$staging/$resource_dir" - manifest="$staging/$resource_dir/jllama-backends.txt" - { - echo "# Native backends in priority order; first loadable backend wins." - echo "# Extra tokens after a backend name are sibling files loaded before its library." - } > "$manifest" - - for backend in "${ordered_backends[@]}"; do - classifier="${BACKEND_CLASSIFIER["$target/$backend"]}" - classifier_jar="$JARS_DIR/llama-$VERSION-$classifier.jar" - extract_dir="$WORK_DIR/extract-$target-$backend" - mkdir -p "$extract_dir" - unzip -q "$classifier_jar" "$resource_dir/*" -d "$extract_dir" \ - || fail "$classifier_jar contains no native tree at $resource_dir" - [ -f "$extract_dir/$resource_dir/$main_lib" ] \ - || fail "$classifier_jar: expected $resource_dir/$main_lib not found" - mkdir -p "$staging/$resource_dir/$backend" - mv "$extract_dir/$resource_dir/"* "$staging/$resource_dir/$backend/" - # Manifest line: backend name + every sibling file (sorted, deterministic) except - # the main library — the loader extracts and loads the siblings first. - extras="$(cd "$staging/$resource_dir/$backend" && find . -type f ! -name "$main_lib" -printf '%P\n' | sort | tr '\n' ' ')" - extras="${extras% }" - if [ -n "$extras" ]; then - echo "$backend $extras" >> "$manifest" - else - echo "$backend" >> "$manifest" - fi + mkdir -p "$staging" + for classifier in $classifiers; do + unzip -q -o "$JARS_DIR/llama-$VERSION-$classifier.jar" 'net/*' -d "$staging" done out_jar="$OUT_DIR/llama-$VERSION-all-$target-jar-with-dependencies.jar" cp "$BASE_FAT_JAR" "$out_jar" + # Drop every native tree but this target's. Matched by exact path components, never by + # prefix: a Linux* glob would also hit Linux-Android. + foreign_entries="$WORK_DIR/foreign-$target.txt" + unzip -Z1 "$out_jar" \ + | awk -v keep="$own" -v own_os="net/ladenthin/llama/${tree%%/*}/" \ + '/^net\/ladenthin\/llama\/[A-Z][^\/]*\// && index($0, keep) != 1 && $0 != own_os' \ + > "$foreign_entries" + [ -s "$foreign_entries" ] \ + || fail "$out_jar: no foreign native tree to remove — did the OSInfo folder names change?" + zip -q -d -nw "$out_jar" -@ < "$foreign_entries" || fail "$out_jar: removing foreign native trees failed" (cd "$staging" && zip -q -ur "$out_jar" net) - # --- Verify the combined jar ----------------------------------------------------- - listing="$(unzip -l "$out_jar")" - for backend in "${ordered_backends[@]}"; do - echo "$listing" | grep -qF "$resource_dir/$backend/$main_lib" \ - || fail "$out_jar: missing $resource_dir/$backend/$main_lib after zip update" + # --- Verify the combined jar -------------------------------------------------------- + for classifier in $classifiers; do + backend="${classifier%-"$target"}" + unzip -p "$out_jar" "$own$backend/$lib" | cmp -s - "$staging/$own$backend/$lib" \ + || fail "$out_jar: $own$backend/$lib is missing or differs from its natives jar" done - echo "$listing" | grep -qF "$resource_dir/jllama-backends.txt" \ - || fail "$out_jar: missing $resource_dir/jllama-backends.txt" + unzip -p "$out_jar" "${own}cpu/$lib" | cmp -s - <(unzip -p "$BASE_FAT_JAR" "${own}cpu/$lib") \ + || fail "$out_jar: CPU fallback ${own}cpu/$lib is missing or differs from the default fat jar's" + out_trees="$(native_trees "$out_jar")" + [ "$out_trees" = "$tree" ] \ + || fail "$out_jar: expected only the native tree $tree, found: $(echo "$out_trees" | tr '\n' ' ')" + [ "$(class_count "$out_jar")" -eq "$(class_count "$BASE_FAT_JAR")" ] \ + || fail "$out_jar: .class count differs from the default fat jar — the tree filter removed classes" unzip -p "$out_jar" META-INF/MANIFEST.MF | grep -qF "Main-Class: $MAIN_CLASS" \ || fail "$out_jar: Main-Class $MAIN_CLASS did not survive the zip update" - sample_backend="${ordered_backends[0]}" - unzip -p "$out_jar" "$resource_dir/$sample_backend/$main_lib" \ - | cmp -s - "$staging/$resource_dir/$sample_backend/$main_lib" \ - || fail "$out_jar: $resource_dir/$sample_backend/$main_lib differs from its source" (cd "$OUT_DIR" && sha256sum "$(basename "$out_jar")" > "$(basename "$out_jar").sha256") - echo "OK: $(basename "$out_jar") ($(du -h "$out_jar" | cut -f1); backends: ${ordered_backends[*]})" + echo "OK: $(basename "$out_jar") ($(du -h "$out_jar" | cut -f1);$classifiers)" +done + +# --- 4. At least the four release targets were produced ------------------------------------ +for target in linux-x86-64 linux-aarch64 windows-x86-64 windows-aarch64; do + [ -f "$OUT_DIR/llama-$VERSION-all-$target-jar-with-dependencies.jar" ] || fail "no all-$target fat jar produced" done -# --- 7. The default (all-platform CPU) fat jar is a release asset too ------------------ +# --- 5. The default (all-platform CPU) fat jar is a release asset too ------------------------- cp "$BASE_FAT_JAR" "$OUT_DIR/" (cd "$OUT_DIR" && sha256sum "$(basename "$BASE_FAT_JAR")" > "$(basename "$BASE_FAT_JAR").sha256") echo "OK: $(basename "$BASE_FAT_JAR") (default CPU fat jar, copied as-is)" diff --git a/.github/smoke-native-macos.sh b/.github/smoke-native-macos.sh index a8e2902b3..ace6f7241 100755 --- a/.github/smoke-native-macos.sh +++ b/.github/smoke-native-macos.sh @@ -9,7 +9,7 @@ # # Why this exists (the gap it closes): the three macOS Java test jobs each run against the dylib # THEIR OWN build job produced. Nothing in the pipeline ever loaded the one that goes into the -# published jar. So when the `*-libraries` artifact glob merged three different macOS dylibs onto +# published jar. So when an artifact glob merged three different macOS dylibs onto # one path and produced a byte-level hybrid, the result — a library whose ad-hoc linker signature no # longer matched its own __TEXT pages, which macOS SIGKILLs on load — shipped in 5.0.6 and several # 5.0.7 snapshots with an all-green pipeline. Linux and Windows already had the equivalent gate @@ -50,7 +50,7 @@ while IFS= read -r j; do jars+=("$j"); done < <(find "$JAR_DIR" -type f -name "$ JAR="$(cd "$(dirname "${jars[0]}")" && pwd)/$(basename "${jars[0]}")" echo "smoke jar: $JAR" -DYLIB_ENTRY="net/ladenthin/llama/Mac/aarch64/libjllama.dylib" +DYLIB_ENTRY="net/ladenthin/llama/Mac/aarch64/metal/libjllama.dylib" WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT diff --git a/.github/smoke-natives-jars.sh b/.github/smoke-natives-jars.sh new file mode 100755 index 000000000..5879eb19b --- /dev/null +++ b/.github/smoke-natives-jars.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +# Loads the library from the published jars exactly as a consumer does: the classes jar, its +# runtime dependencies and EVERY natives jar at once -- first on the classpath, then on the module +# path. Each natives jar holds its own /// directory, so all of them fit on one +# classpath; on the module path each is an automatic module and needs a unique +# Automatic-Module-Name, or the JVM refuses to start ("Two versions of module ... found"). +# +# On a GPU-less runner the GPU backends normally fail their load (no vendor runtime) and the loader +# falls through to the CPU library. A GPU library whose runtime happens to be installed may load and +# find no device, which is benign, so any selected backend passes; the log names the one chosen. +# +# Usage: smoke-natives-jars.sh +# directory holding llama-.jar and the llama--.jar natives jars +# the classes jar's runtime dependencies (mvn dependency:build-classpath) +set -euo pipefail + +TARGET="${1:?usage: smoke-natives-jars.sh }" +DEPS="${2:?usage: smoke-natives-jars.sh }" +SMOKE="$(dirname "$0")/smoke/NativeLoadSmoke.java" + +# The classes jar is the one without a classifier; every other jar but sources/javadoc/the fat jar +# is a natives jar. +classes="$(find "$TARGET" -maxdepth 1 -name 'llama-*.jar' | grep -E '/llama-[0-9][0-9.]*(-SNAPSHOT)?\.jar$' || true)" +[ "$(wc -l <<< "$classes")" -eq 1 ] && [ -n "$classes" ] || { echo "::error::expected one classes jar in $TARGET: $classes" >&2; exit 2; } +mapfile -t natives < <(find "$TARGET" -maxdepth 1 -name 'llama-*.jar' ! -path "$classes" ! -name '*-sources.jar' \ + ! -name '*-javadoc.jar' ! -name '*-jar-with-dependencies.jar' | sort) +[ "${#natives[@]}" -gt 0 ] || { echo "::error::no natives jars in $TARGET" >&2; exit 2; } +path="$classes:$(IFS=:; echo "${natives[*]}"):$DEPS" +echo "classes jar: $classes; ${#natives[@]} natives jars" + +run() { + local label="$1" + shift + local out + out="$(java "$@" "$SMOKE" 2>&1)" || { echo "$out" >&2; echo "::error::$label: the library did not load" >&2; exit 1; } + backend="$(grep -o "\[jllama\] using native backend '[^']*'" <<< "$out" || true)" + [ -n "$backend" ] || { echo "$out" >&2; echo "::error::$label: the loader reported no backend" >&2; exit 1; } + echo "OK ($label): $backend; $(grep 'native load smoke OK' <<< "$out")" +} + +run "classpath" -cp "$path" +run "module path" --module-path "$path" --add-modules ALL-MODULE-PATH diff --git a/.github/smoke-test-fatjar.sh b/.github/smoke-test-fatjar.sh index 30a3eb34c..6c37bbf6f 100755 --- a/.github/smoke-test-fatjar.sh +++ b/.github/smoke-test-fatjar.sh @@ -5,10 +5,10 @@ # SPDX-License-Identifier: MIT # Smoke test for an all-backends server fat jar on a GPU-less runner: -# `java -jar` must start the embedded server — with every manifest backend failing -# its load cleanly and the loader falling back to the default CPU natives — then -# answer GET /health with 200 and a POST /v1/chat/completions with a valid choice. -# This exercises manifest parsing, per-backend extraction, and the fallback chain +# `java -jar` must start the embedded server — with every GPU backend failing its +# load cleanly and the loader falling back to the CPU backend — then answer GET +# /health with 200 and a POST /v1/chat/completions with a valid choice. This +# exercises backend probing, per-backend extraction, and the fallback chain # end-to-end through a real fat-jar launch. # # Usage: smoke-test-fatjar.sh [port] @@ -70,10 +70,10 @@ assert message is not None, "choices[0].message missing" print("chat completion OK:", json.dumps(message)[:200]) ' || fail "malformed chat completion response: $RESPONSE" -# The loader must have reported its backend decision (a chosen backend on a GPU -# machine, the CPU fallback on a GPU-less runner) — this pins that the manifest was -# actually read, i.e. the smoke really ran the multi-backend code path. -grep -hE '\[jllama\] (using native backend|no manifest backend loadable)' server-out.log server-err.log \ - || fail "no backend-selection log line found — the backend manifest was not processed" +# The loader must have reported its backend decision (normally the CPU fallback on a +# GPU-less runner; a GPU backend whose runtime happens to be installed may load and +# find no device, which is benign) — this pins that the smoke ran the backend probing. +grep -hE '\[jllama\] using native backend' server-out.log server-err.log \ + || fail "no backend-selection log line found — the loader did not report a backend" echo "smoke test PASSED" diff --git a/.github/verify-native-deps.py b/.github/verify-native-deps.py index ab26f3812..d8edd5750 100755 --- a/.github/verify-native-deps.py +++ b/.github/verify-native-deps.py @@ -14,13 +14,13 @@ Mach-O). Usage: - verify-native-deps.py --default # exact allowlist per / - verify-native-deps.py --deny ... # classifier trees: only the denylist + verify-native-deps.py ---default checks net/ladenthin/llama/// under the root against ALLOWED below: a -dependency outside the list fails, and so does an / without a list (a new platform -must be listed consciously). --deny scans every native library under the directories for DENIED -names only, because GPU classifiers legitimately need their vendor runtime. +Checks every library under /net/ladenthin/llama////. The CPU +builds (backend cpu, metal, msvc) are held to the exact allowlist in ALLOWED: a dependency outside +it fails, and so does a CPU build without a list (a new platform must be listed consciously). The +GPU backends are checked against DENIED only, because they legitimately need their vendor runtime. +That every listed build arrived is merge-native-artifacts.sh's check. Exit codes: 0 clean, 1 violation, 2 nothing found to check. """ @@ -29,20 +29,20 @@ import struct import sys -# What each default-JAR library needed when this check was introduced (5.1.0 plus the RPC backend, +# What each CPU library needed when this check was introduced (5.1.0 plus the RPC backend, # which adds nothing: its sockets are libc/libSystem/WS2_32, all already present). ALLOWED = { - "Linux/x86_64": {"libdl.so.2", "libgomp.so.1", "libpthread.so.0", "librt.so.1", "libstdc++.so.6", + "Linux/x86_64/cpu": {"libdl.so.2", "libgomp.so.1", "libpthread.so.0", "librt.so.1", "libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld-linux-x86-64.so.2"}, - "Linux/aarch64": {"libgomp.so.1", "libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", + "Linux/aarch64/cpu": {"libgomp.so.1", "libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld-linux-aarch64.so.1"}, - "Linux/s390x": {"libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld64.so.1"}, - "Linux-Android/aarch64": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, - "Linux-Android/x86_64": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, - "Windows/x86_64": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, - "Windows/x86": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, - "Windows/aarch64": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll"}, - "Mac/aarch64": {"/usr/lib/libc++.1.dylib", "/usr/lib/libSystem.B.dylib", + "Linux/s390x/cpu": {"libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld64.so.1"}, + "Linux-Android/aarch64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, + "Linux-Android/x86_64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, + "Windows/x86_64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, + "Windows/x86/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, + "Windows/aarch64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll"}, + "Mac/aarch64/metal": {"/usr/lib/libc++.1.dylib", "/usr/lib/libSystem.B.dylib", "/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation", "/System/Library/Frameworks/Metal.framework/Versions/A/Metal", "/System/Library/Frameworks/MetalKit.framework/Versions/A/MetalKit", @@ -57,6 +57,11 @@ "/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib", "/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib"}, } +# The Visual Studio generator build of the same compiler and runtime. +ALLOWED["Windows/x86_64/msvc"] = ALLOWED["Windows/x86_64/cpu"] +ALLOWED["Windows/x86/msvc"] = ALLOWED["Windows/x86/cpu"] +CPU_BACKENDS = ("cpu", "metal", "msvc") +LIBRARY_NAMES = ("libjllama.so", "jllama.dll", "libjllama.dylib") # Never acceptable in any artifact: libraries a consumer cannot be expected to have. DENIED = ("libibverbs", "librdma", "rdma.dylib", "libmlx") @@ -174,32 +179,32 @@ def denied(deps): def main(argv): - if len(argv) < 3 or argv[1] not in ("--default", "--deny"): + if len(argv) != 2: print(__doc__, file=sys.stderr) return 2 - mode, roots = argv[1], argv[2:] + root = argv[1] checked = 0 failures = [] - for root in roots: - for path in sorted(find_libraries(root)): - deps = dependencies(path) - checked += 1 - rel = os.path.relpath(path, root).replace(os.sep, "/") - print(f"{rel}: {' '.join(deps)}") - for d in denied(deps): - failures.append(f"{rel} needs {d}, which no consumer can be expected to have") - if mode == "--default": - parts = rel.split("/") - key = "/".join(parts[-3:-1]) if len(parts) >= 3 else "" - allowed = ALLOWED.get(key) - if allowed is None: - failures.append(f"{rel}: no dependency allowlist for '{key}' -- add one to ALLOWED") - continue - for d in deps: - if d.lower() not in {a.lower() for a in allowed}: - failures.append(f"{rel} needs {d}, which it did not need before (allowed: {sorted(allowed)})") + for path in sorted(find_libraries(root)): + deps = dependencies(path) + checked += 1 + rel = os.path.relpath(path, root).replace(os.sep, "/") + print(f"{rel}: {' '.join(deps)}") + for d in denied(deps): + failures.append(f"{rel} needs {d}, which no consumer can be expected to have") + parts = rel.split("/") + key = "/".join(parts[-4:-1]) if len(parts) >= 4 else "" + if key.rsplit("/", 1)[-1] not in CPU_BACKENDS or os.path.basename(path) not in LIBRARY_NAMES: + continue + allowed = ALLOWED.get(key) + if allowed is None: + failures.append(f"{rel}: no dependency allowlist for '{key}' -- add one to ALLOWED") + continue + for d in deps: + if d.lower() not in {a.lower() for a in allowed}: + failures.append(f"{rel} needs {d}, which it did not need before (allowed: {sorted(allowed)})") if checked == 0: - print(f"no native library found under {roots}", file=sys.stderr) + print(f"no native library found under {root}", file=sys.stderr) return 2 for f in failures: print(f"::error::{f}", file=sys.stderr) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index f5ddb12a9..8ee750c9f 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -365,6 +365,8 @@ jobs: with: java-version: '21' distribution: temurin + - name: Natives list check (pom, llama-platform, workflow and loader agree with natives.csv) + run: python3 .github/check-natives.py - name: Spotless check (fail fast on format violations) run: mvn -B --no-transfer-progress -f llama/pom.xml spotless:check - name: SpotBugs check (fail fast on static-analysis findings) @@ -449,8 +451,8 @@ jobs: - name: Download Linux x86_64 native library (reused, not rebuilt) uses: actions/download-artifact@v8 with: - name: Linux-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - name: Restore shared GGUF model cache (populated by download-models; no re-download) # Restore-only: consumer jobs can NEVER write the cache, so a job running on a # cache miss cannot re-save an empty/partial entry under the immutable key. @@ -465,7 +467,7 @@ jobs: with: distribution: 'temurin' java-version: ${{ env.JAVA_VERSION }} - - name: Install parent + core net.ladenthin:llama (bundles the downloaded native library) + - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) run: > mvn -B --no-transfer-progress -pl llama -am -DskipTests -Denforcer.skip=true -Dspotless.check.skip=true -Dspotbugs.skip=true @@ -477,6 +479,7 @@ jobs: -Dnet.ladenthin.llama.langchain4j.embedding.model=models/${NOMIC_EMBED_MODEL_NAME} -Dnet.ladenthin.llama.langchain4j.rerank.model=models/${RERANKING_MODEL_NAME} -Dnet.ladenthin.llama.langchain4j.tool.model=models/${TOOL_MODEL_NAME} + -Dnet.ladenthin.llama.lib.path=${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu # This job is model-backed and crosses JNI, so a forked test JVM here can abort exactly the # way the six test-java-* jobs can -- but it had neither of their diagnostics. Same step and @@ -568,16 +571,16 @@ jobs: - name: Resolve the reactor version run: echo "VERSION=$(mvn -q -DforceStdout help:evaluate -Dexpression=project.version | tail -n1)" >> "$GITHUB_ENV" - name: Spotless check - run: mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" spotless:check + run: mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" -Dllama.natives=none spotless:check - name: Build and test (unit + model-free wire contract against the real OpenAiCompatServer) - run: mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" verify + run: mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" -Dllama.natives=none verify # The GitHub Release asset llama-atmosphere-agent--jar-with-dependencies.jar: # the agent plus Atmosphere/JLine, WITHOUT the core (src/assembly/agent-jar.xml), so it is a # few MB and the natives are not in the release twice. Never deployed to Maven Central. # smoke-agent-linux launches it next to the real core fat jar; the attach jobs sign it. - name: Build the agent release jar (without the core) run: > - mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" + mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" -Dllama.natives=none -P assembly -DskipTests package - name: Collect the agent release jar + sha256 run: | @@ -603,8 +606,8 @@ jobs: - name: Download Linux x86_64 native library (reused, not rebuilt) uses: actions/download-artifact@v8 with: - name: Linux-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - name: Restore shared GGUF model cache (populated by download-models; no re-download) uses: actions/cache/restore@v6 with: @@ -615,7 +618,7 @@ jobs: with: distribution: 'temurin' java-version: ${{ env.JAVA_VERSION }} - - name: Install parent + core net.ladenthin:llama (bundles the downloaded native library) + - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) run: > mvn -B --no-transfer-progress -pl llama -am -DskipTests -Denforcer.skip=true -Dspotless.check.skip=true -Dspotbugs.skip=true @@ -624,10 +627,11 @@ jobs: run: echo "VERSION=$(mvn -q -DforceStdout help:evaluate -Dexpression=project.version | tail -n1)" >> "$GITHUB_ENV" - name: Run the Atmosphere tool-loop integration test (cached Qwen2.5-1.5B tool model, CPU) run: > - mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" test + mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" -Dllama.natives=none test -Dtest=AtmosphereToolLoopIntegrationTest -Dsurefire.failIfNoSpecifiedTests=false -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} -Dnet.ladenthin.llama.test.ngl=0 + -Dnet.ladenthin.llama.lib.path=${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu # Model-backed and crossing JNI: same crash diagnostics as the langchain4j integration job. - name: Print crash logs (on failure) if: failure() @@ -793,8 +797,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: linux-libraries-cuda - path: ${{ github.workspace }}/llama/src/main/resources_linux_cuda/net/ladenthin/llama/ + name: natives-cuda13-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ crosscompile-linux-x86_64: name: Cross-Compile manylinux2014 x86_64 @@ -833,8 +837,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ crosscompile-linux-aarch64: name: Build and Test Linux aarch64 @@ -888,8 +892,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-aarch64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-linux-s390x: name: Build and Test Linux s390x (big-endian, qemu) @@ -897,11 +901,11 @@ jobs: # Cross-compile for IBM Z (s390x, BIG-ENDIAN) with the GCC cross toolchain, then run the full # C++ unit suite under qemu-user — a real big-endian correctness gate for our helpers and # serializers (esp. the little-endian WAV writer, JSON/token/embedding transforms). The BUILD - # is native speed (x86 cross-gcc); only the tiny test binary is emulated. s390x is a DEFAULT-jar - # CPU platform (like aarch64), so the artifact merges via the `*-libraries` glob (no classifier / - # pom profile). Model-backed Java tests are NOT run under emulation (a JVM + GGUF inference under - # qemu-user is slow/flaky); the C++ gate covers the actual byte-order risk since the Java<->JNI - # boundary uses host-native array copies. GGML_OPENMP=OFF avoids cross-libgomp issues (ggml uses + # is native speed (x86 cross-gcc); only the tiny test binary is emulated. s390x is a CPU + # platform like aarch64 and ships as the cpu-linux-s390x natives jar. Model-backed Java tests + # are NOT run under emulation (a JVM + GGUF inference under qemu-user is slow/flaky); the + # C++ gate covers the actual byte-order risk since the Java<->JNI boundary uses host-native + # array copies. GGML_OPENMP=OFF avoids cross-libgomp issues (ggml uses # its own std::thread pool). CMAKE_CROSSCOMPILING_EMULATOR makes ctest run the s390x exe via qemu; # QEMU_LD_PREFIX lets the emulated binary find the s390x sysroot libs. runs-on: ubuntu-latest @@ -945,8 +949,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-s390x-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-s390x + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-linux-x86_64-vulkan: name: Build Linux x86_64 Vulkan @@ -987,8 +991,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ + name: natives-vulkan-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-linux-aarch64-vulkan: @@ -996,8 +1000,7 @@ jobs: needs: [startgate, build-webui] # Native ARM64 Vulkan build on GitHub's free arm64 runner (same runner as the aarch64 CPU job). # Build-only (GPU-less runner); GGML_NATIVE=OFF for portability across ARMv8 generations; GCC 14 - # to match the aarch64 CPU job. Reuses the resources_linux_vulkan tree (arch subdir Linux/aarch64); - # the vulkan-linux-aarch64 Maven profile packages only that subtree. + # to match the aarch64 CPU job. Ships as the vulkan-linux-aarch64 natives jar. runs-on: ubuntu-24.04-arm env: USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} @@ -1028,8 +1031,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-aarch64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ + name: natives-vulkan-linux-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error crosscompile-android-aarch64: @@ -1066,8 +1069,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-Android-aarch64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-android-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ crosscompile-android-x86_64: name: Cross-Compile Android x86_64 @@ -1075,8 +1078,7 @@ jobs: runs-on: ubuntu-latest # Android x86_64 CPU ABI: consumed by the llama-android AAR as jni/x86_64 (so the # Android emulator — and x86_64 Android devices/Chromebooks — can run the binding) - # and merged into the default JAR's Linux-Android/x86_64 tree via the *-libraries - # glob. Same dockcross + sccache steady-state env as the arm64 job; the wrapper + # and shipped as the cpu-android-x86-64 natives jar. Same dockcross + sccache steady-state env as the arm64 job; the wrapper # pins the same image tag. Fail-loud and in the package/publish needs graphs. env: USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} @@ -1097,8 +1099,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-Android-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-android-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error crosscompile-android-aarch64-opencl: @@ -1128,8 +1130,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: android-libraries-opencl - path: ${{ github.workspace }}/llama/src/main/resources_android_opencl/net/ladenthin/llama/ + name: natives-opencl-android-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # --------------------------------------------------------------------------- # Android AAR packaging (net.ladenthin:llama-android + llama-android-opencl). @@ -1167,24 +1169,24 @@ jobs: - name: Download Android CPU natives (arm64) uses: actions/download-artifact@v8 with: - name: Linux-Android-aarch64-libraries + name: natives-cpu-android-aarch64 path: stage/cpu/ - name: Download Android CPU natives (x86_64) uses: actions/download-artifact@v8 with: - name: Linux-Android-x86_64-libraries + name: natives-cpu-android-x86-64 path: stage/cpu-x86_64/ - name: Download Android OpenCL natives uses: actions/download-artifact@v8 with: - name: android-libraries-opencl + name: natives-opencl-android-aarch64 path: stage/opencl/ - name: Stage natives for the AAR build run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 llama-android/natives/opencl/arm64-v8a - cp stage/cpu/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp stage/cpu-x86_64/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ - cp stage/opencl/Linux-Android/aarch64/libjllama.so llama-android/natives/opencl/arm64-v8a/ + cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ + cp stage/opencl/Linux-Android/aarch64/opencl/libjllama.so llama-android/natives/opencl/arm64-v8a/ - name: Assemble AARs + publish to mavenLocal run: gradle -p llama-android aarCpu aarOpencl publishToMavenLocal - name: Validate AAR structure + 16 KB page-size alignment @@ -1313,18 +1315,18 @@ jobs: - name: Download Android CPU natives (arm64) uses: actions/download-artifact@v8 with: - name: Linux-Android-aarch64-libraries + name: natives-cpu-android-aarch64 path: stage/cpu/ - name: Download Android CPU natives (x86_64) uses: actions/download-artifact@v8 with: - name: Linux-Android-x86_64-libraries + name: natives-cpu-android-x86-64 path: stage/cpu-x86_64/ - name: Stage natives + publish the CPU AAR to mavenLocal run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 - cp stage/cpu/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp stage/cpu-x86_64/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ + cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ gradle -p llama-android publishLlamaAndroidPublicationToMavenLocal - name: Restore shared GGUF model cache (populated by download-models; no re-download) # Restore-only: consumer jobs can NEVER write the cache, so a job running on a @@ -1407,18 +1409,18 @@ jobs: - name: Download Android CPU natives (arm64) uses: actions/download-artifact@v8 with: - name: Linux-Android-aarch64-libraries + name: natives-cpu-android-aarch64 path: stage/cpu/ - name: Download Android CPU natives (x86_64) uses: actions/download-artifact@v8 with: - name: Linux-Android-x86_64-libraries + name: natives-cpu-android-x86-64 path: stage/cpu-x86_64/ - name: Stage natives + publish the CPU AAR to mavenLocal run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 - cp stage/cpu/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp stage/cpu-x86_64/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ + cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ gradle -p llama-android publishLlamaAndroidPublicationToMavenLocal - name: Decode upload keystore (if configured) # Optional Play upload key. Set the ANDROID_UPLOAD_KEYSTORE_BASE64 secret @@ -1503,18 +1505,18 @@ jobs: - name: Download Android CPU natives (arm64) uses: actions/download-artifact@v8 with: - name: Linux-Android-aarch64-libraries + name: natives-cpu-android-aarch64 path: stage/cpu/ - name: Download Android CPU natives (x86_64) uses: actions/download-artifact@v8 with: - name: Linux-Android-x86_64-libraries + name: natives-cpu-android-x86-64 path: stage/cpu-x86_64/ - name: Stage natives + publish the CPU AAR to mavenLocal run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 - cp stage/cpu/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp stage/cpu-x86_64/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ + cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ gradle -p llama-android publishLlamaAndroidPublicationToMavenLocal - name: Restore shared GGUF model cache (populated by download-models; no re-download) uses: actions/cache/restore@v6 @@ -1597,7 +1599,7 @@ jobs: uses: actions/upload-artifact@v7 with: name: macos-15-no-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-macos-arm64-metal: name: Build and Test macOS 14 arm64 (Metal) @@ -1642,7 +1644,7 @@ jobs: uses: actions/upload-artifact@v7 with: name: macos-14-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-windows-x86_64-msvc: name: Build and Test Windows 2025 x86_64 (MSVC / VS 2026, classifier) @@ -1675,8 +1677,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-msvc - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-msvc-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-windows-x86-msvc: name: Build and Test Windows 2025 x86 (MSVC / VS 2026, classifier) @@ -1709,8 +1711,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86-msvc - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-msvc-windows-x86 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # --------------------------------------------------------------------------- # Windows Ninja Multi-Config + sccache — the DEFAULT Windows CPU natives. @@ -1719,10 +1721,9 @@ jobs: # (build.bat probe-guards it). Both generators use the same MSVC toolchain (cl.exe, # static /MT CRT) on the same runner, so the produced jllama.dll/llama.dll/ggml.dll # are functionally equivalent with identical runtime dependencies — the only delta - # is build-system plumbing + caching. The Ninja build is therefore the default JAR - # (artifacts `Windows-*-libraries`, picked up by the package job's `pattern: - # "*-libraries"`); the MSVC build above is shipped as the `msvc-windows` classifier - # for anyone who wants the Visual-Studio-generator natives. Upstream llama.cpp also + # is build-system plumbing + caching. The Ninja build therefore ships as the + # cpu-windows-* natives jars; the MSVC build above as msvc-windows-*, for anyone who + # wants the Visual-Studio-generator natives. Upstream llama.cpp also # builds its Windows artifacts with Ninja Multi-Config + MSVC. # --------------------------------------------------------------------------- @@ -1777,8 +1778,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-windows-x86: name: Build and Test Windows 2025 x86 (Ninja Multi-Config + sccache, default) @@ -1831,16 +1832,15 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-windows-x86 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ build-windows-arm64: name: Build and Test Windows 11 arm64 (Ninja Multi-Config, default) needs: [startgate, build-webui] - # Native arm64 build on GitHub's free windows-11-arm runner. Goes into the DEFAULT JAR (no - # classifier): OSInfo maps a Windows-on-ARM JVM (os.arch=aarch64) to Windows/aarch64, the same - # path CMake emits here, and the `*-libraries` glob in the package/publish jobs merges it into - # src/main/resources. sccache: the native aarch64-pc-windows-msvc release, wrapping clang-cl + # Native arm64 build on GitHub's free windows-11-arm runner; ships as the cpu-windows-aarch64 + # natives jar: OSInfo maps a Windows-on-ARM JVM (os.arch=aarch64) to Windows/aarch64, the same + # path CMake emits here. sccache: the native aarch64-pc-windows-msvc release, wrapping clang-cl # (guarded by build.bat's probe + uncached retry like every other Windows job). # # Compiler: clang-cl, NOT MSVC cl.exe. ggml's ggml-cpu/CMakeLists.txt aborts with "MSVC is not @@ -1900,18 +1900,17 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-aarch64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-windows-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # --------------------------------------------------------------------------- # Windows GPU classifiers (x86_64 only) — CUDA, Vulkan, OpenCL. # All three use the same Ninja Multi-Config + MSVC + sccache toolchain as the # default CPU build; they differ only by the GGML backend flag (and the build-time # SDK each needs). CMakeLists.txt routes each backend's output to its own - # src/main/resources_windows_{cuda,vulkan,opencl}/ tree, which the matching Maven - # profile (cuda-windows / vulkan-windows / opencl-windows) turns into a classifier - # JAR. GPU runtime libraries are NOT bundled — the consumer's GPU driver / toolkit - # provides them (CUDA: cudart64_13/cublas64_13 from the CUDA Toolkit; Vulkan: + # src/main/natives/.../Windows/x86_64// directory, which the natives Maven + # profile turns into a natives jar. GPU runtime libraries are NOT bundled — the + # consumer's GPU driver / toolkit provides them (CUDA: cudart64_13/cublas64_13 from the CUDA Toolkit; Vulkan: # vulkan-1.dll from the driver; OpenCL: System32\OpenCL.dll from the driver). # NOTE: GitHub-hosted Windows runners have NO GPU, so these jobs build + run the # C++ unit suite (ctest, CPU-only) but cannot run model-backed GPU inference; @@ -1998,8 +1997,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-cuda - path: ${{ github.workspace }}/llama/src/main/resources_windows_cuda/net/ladenthin/llama/ + name: natives-cuda13-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-x86_64-vulkan: @@ -2047,8 +2046,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_windows_vulkan/net/ladenthin/llama/ + name: natives-vulkan-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-x86_64-opencl: @@ -2091,8 +2090,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ + name: natives-opencl-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error # --------------------------------------------------------------------------- @@ -2101,8 +2100,8 @@ jobs: # Intel OpenVINO. All BUILD-ONLY (GitHub runners have no AMD/Intel/Adreno GPU, and # no ctest — a GPU-linked jllama_test can't enumerate a device). GPU runtime libs # are NOT bundled — the consumer's driver/toolkit supplies them. CMakeLists.txt - # routes each backend to its own src/main/resources_* tree; the matching Maven - # profile turns it into a classifier JAR. Toolchain install steps are first-pass — + # routes each backend to its own src/main/natives//// directory; the + # natives Maven profile turns it into a natives jar. Toolchain install steps are first-pass — # if a vendor URL/version 404s in CI, adjust it (the failure is intentional signal). # --------------------------------------------------------------------------- @@ -2167,12 +2166,12 @@ jobs: - name: Verify the GPU code is compressed # ggml-hip is compiled with --offload-compress (llama/CMakeLists.txt); without it the # library carries every target's kernels uncompressed. Fails on an uncompressed bundle. - run: python3 .github/verify-hip-offload-compressed.py llama/src/main/resources_linux_rocm + run: python3 .github/verify-hip-offload-compressed.py llama/src/main/natives - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_linux_rocm/net/ladenthin/llama/ + name: natives-rocm-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-x86_64-rocm: @@ -2253,12 +2252,12 @@ jobs: - name: Verify the GPU code is compressed # Same check as the Linux ROCm job: an uncompressed bundle means ~1 GB jllama.dll again. shell: pwsh - run: python .github/verify-hip-offload-compressed.py llama/src/main/resources_windows_rocm + run: python .github/verify-hip-offload-compressed.py llama/src/main/natives - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_windows_rocm/net/ladenthin/llama/ + name: natives-rocm-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-linux-x86_64-sycl-fp16: @@ -2300,8 +2299,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-sycl-fp16 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp16/net/ladenthin/llama/ + name: natives-sycl-fp16-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-linux-x86_64-sycl-fp32: @@ -2343,8 +2342,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-sycl-fp32 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp32/net/ladenthin/llama/ + name: natives-sycl-fp32-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-x86_64-sycl: @@ -2397,17 +2396,16 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-sycl - path: ${{ github.workspace }}/llama/src/main/resources_windows_sycl/net/ladenthin/llama/ + name: natives-sycl-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-arm64-opencl: name: Build Windows 11 arm64 OpenCL (Adreno) needs: [startgate, build-webui] # Windows-on-ARM OpenCL (Snapdragon X / Adreno). Same clang-cl + GGML_OPENMP=OFF - # toolchain as the arm64 CPU job (ggml refuses MSVC cl.exe on ARM). Reuses the - # resources_windows_opencl tree under Windows/aarch64; the opencl-windows-aarch64 - # Maven profile packages only that subtree. build_opencl_windows.bat stages the + # toolchain as the arm64 CPU job (ggml refuses MSVC cl.exe on ARM). Ships as the + # opencl-windows-aarch64 natives jar. build_opencl_windows.bat stages the # OpenCL headers + ICD loader before delegating to build.bat. runs-on: windows-11-arm env: @@ -2447,8 +2445,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-aarch64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ + name: natives-opencl-windows-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-linux-x86_64-openvino: @@ -2498,8 +2496,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Linux-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_linux_openvino/net/ladenthin/llama/ + name: natives-openvino-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error build-windows-x86_64-openvino: @@ -2559,8 +2557,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: Windows-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_windows_openvino/net/ladenthin/llama/ + name: natives-openvino-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ if-no-files-found: error # --------------------------------------------------------------------------- @@ -2639,8 +2637,8 @@ jobs: - name: Upload artifacts uses: actions/upload-artifact@v7 with: - name: macos-15-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-metal-macos-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # --------------------------------------------------------------------------- # Java test jobs — download release artifact, run mvn test @@ -2662,8 +2660,8 @@ jobs: cat /proc/cpuinfo - uses: actions/download-artifact@v8 with: - name: Linux-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-linux-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share @@ -2828,7 +2826,7 @@ jobs: - uses: actions/download-artifact@v8 with: name: macos-14-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share @@ -2939,7 +2937,7 @@ jobs: - uses: actions/download-artifact@v8 with: name: macos-15-no-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share @@ -3049,8 +3047,8 @@ jobs: system_profiler SPHardwareDataType - uses: actions/download-artifact@v8 with: - name: macos-15-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-metal-macos-aarch64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share @@ -3163,8 +3161,8 @@ jobs: Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" - uses: actions/download-artifact@v8 with: - name: Windows-x86_64-libraries - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-cpu-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. validate-models is kept as an integrity guard so a partial/absent @@ -3274,7 +3272,7 @@ jobs: ${{ github.workspace }}\llama\target\surefire-reports\*.dumpstream ${{ github.workspace }}\llama\target\surefire-reports\*.txt ${{ github.workspace }}\llama\target\surefire-reports\TEST-*.xml - ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/**/* + ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/**/* if-no-files-found: warn # Java/inference validation of the MSVC-built x86_64 DLL (the analogue of @@ -3300,8 +3298,8 @@ jobs: Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" - uses: actions/download-artifact@v8 with: - name: Windows-x86_64-msvc - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ + name: natives-msvc-windows-x86-64 + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ # GGUF models are downloaded + cached ONCE by the upstream `download-models` job # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job # download logic. validate-models is kept as an integrity guard so a partial/absent @@ -3411,7 +3409,7 @@ jobs: ${{ github.workspace }}\llama\target\surefire-reports\*.dumpstream ${{ github.workspace }}\llama\target\surefire-reports\*.txt ${{ github.workspace }}\llama\target\surefire-reports\TEST-*.xml - ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/**/* + ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/**/* if-no-files-found: warn # --------------------------------------------------------------------------- @@ -3456,114 +3454,32 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - # Downloaded UNMERGED (one subdirectory per artifact name) on purpose: `merge-multiple: true` - # would silently overwrite — and can byte-level interleave — two artifacts that carry the same - # relative path, which is how the corrupt macOS dylib shipped. merge-native-artifacts.sh does - # the merge instead, and fails the job if any {OS}/{ARCH} path is claimed by more than one - # `*-libraries` artifact. A post-merge check cannot catch this (the collision leaves exactly - # one file on the path — a corrupt one), so the check has to happen before the merge. + # Every shipped native build uploads `natives-`. Downloaded UNMERGED (one + # subdirectory per artifact) on purpose: merge-native-artifacts.sh checks that each artifact + # holds exactly the /// directory its name promises and that no path is + # claimed twice, then merges. `merge-multiple: true` would silently overwrite -- and can + # byte-level interleave -- two artifacts with the same path, which is how a corrupt macOS + # dylib shipped; the test-only macOS builds (macos-14-metal, macos-15-no-metal) are named + # outside the glob for that reason. - uses: actions/download-artifact@v8 with: - pattern: "*-libraries" + pattern: "natives-*" path: ${{ github.workspace }}/native-artifacts/ - - name: Merge native libraries into the default resource tree (collision-checked) + - name: Merge native libraries into the natives tree (checked per artifact) run: | bash .github/merge-native-artifacts.sh \ "${{ github.workspace }}/native-artifacts" \ - "${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/" - # All three macOS arm64 build jobs emit the same path Mac/aarch64/libjllama.dylib, so their - # artifact names are kept outside the "*-libraries" glob above — merging them would drop - # three different dylibs onto one file. The variant that ships is selected explicitly: - # macos-15-metal is the only one built with both Metal and GGML_NATIVE=OFF (portable across - # CPU generations); macos-14-metal and macos-15-no-metal remain test-only. - - uses: actions/download-artifact@v8 - with: - name: macos-15-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: linux-libraries-cuda - path: ${{ github.workspace }}/llama/src/main/resources_linux_cuda/net/ladenthin/llama/ - # Linux Vulkan classifiers (x86_64 + aarch64) share one tree; the two Maven profiles - # split it by arch subdir into one single-arch classifier JAR each. - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-aarch64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: android-libraries-opencl - path: ${{ github.workspace }}/llama/src/main/resources_android_opencl/net/ladenthin/llama/ - # MSVC-built Windows natives -> `msvc-windows` classifier tree. The default JAR - # now ships the Ninja `*-libraries` natives merged above (default flip). - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - # Windows GPU classifiers (x86_64 only) -> one tree each. - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-cuda - path: ${{ github.workspace }}/llama/src/main/resources_windows_cuda/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_windows_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_linux_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_windows_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp16 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp16/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp32 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp32/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-sycl - path: ${{ github.workspace }}/llama/src/main/resources_windows_sycl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-aarch64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_linux_openvino/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_windows_openvino/net/ladenthin/llama/ + "${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/" # Runtime dependencies of every shipped native library, read from the file itself (ELF - # DT_NEEDED, PE imports incl. Windows arm64, Mach-O load commands). The default tree must - # match an exact per-{OS}/{ARCH} allowlist -- a new dependency is a load failure on every - # machine that lacks it. The classifier trees legitimately need their vendor runtime, so - # there only the denylist applies. The case this was written for: ggml-rpc's RDMA + # DT_NEEDED, PE imports incl. Windows arm64, Mach-O load commands). The CPU builds must + # match an exact per-// allowlist -- a new dependency is a load failure + # on every machine that lacks it. The GPU backends legitimately need their vendor runtime, + # so there only the denylist applies. The case this was written for: ggml-rpc's RDMA # transport, which upstream enables whenever the build host has libibverbs/librdma # (llama/CMakeLists.txt forces it off). - name: Verify native runtime dependencies run: | - python3 .github/verify-native-deps.py --default llama/src/main/resources - python3 .github/verify-native-deps.py --deny llama/src/main/resources_* + python3 .github/verify-native-deps.py llama/src/main/natives - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3574,13 +3490,12 @@ jobs: # default-platform native libs in one drop-on-classpath JAR, runnable via its # ServerLauncher Main-Class). It lands in target/ and is uploaded in the `llama-jars` # artifact below - never a Maven Central asset; the `package-fatjars` job downstream - # combines it with the classifier jars into the GitHub-Release fat-jar assets. - # Windows classifier JARs: `windows-msvc` (MSVC-built CPU natives) plus the GPU - # backends `cuda-windows` / `vulkan-windows` / `opencl-windows`. The default JAR's - # Windows natives are the Ninja `*-libraries` merged into src/main/resources/ above. - run: mvn --batch-mode --no-transfer-progress -P release,cuda,vulkan-linux,vulkan-linux-aarch64,opencl-android,windows-msvc,cuda-windows,vulkan-windows,opencl-windows,rocm-linux,rocm-windows,sycl-fp16-linux,sycl-fp32-linux,sycl-windows,opencl-windows-aarch64,openvino-linux,openvino-windows,assembly -Dmaven.test.skip=true -Dgpg.skip=true package - # Class-file floor, checked on every jar this job just built (all 16 classifier - # jars plus the default fat jar). Production code targets Java 8, so anything a + # combines it with the natives jars into the GitHub-Release fat-jar assets. + # `natives` builds one jar per -- from the tree merged above; its + # enforcer rule fails the build when any of them is missing. + run: mvn --batch-mode --no-transfer-progress -P release,natives,assembly -Dmaven.test.skip=true -Dgpg.skip=true package + # Class-file floor, checked on every jar this job just built (the classes jar, the + # natives jars and the default fat jar). Production code targets Java 8, so anything a # consumer's JVM can load must be major 52 or lower: a single Java 11 class kills # the process with UnsupportedClassVersionError before any of our code runs, which # is exactly what shipped when logback's LogbackServiceProvider was the binding. @@ -3588,6 +3503,15 @@ jobs: # classpath JVM never loads them. Kept byte-identical across all four sibling repos. - name: Verify Java 8 bytecode (no class newer than major 52) run: .github/verify-bytecode-version.sh --max-major 52 llama/target + # The jars as a consumer uses them: the classes jar, its runtime dependencies and all + # natives jars at once, on the classpath and on the module path. Proves the directories + # never collide and each natives jar is its own automatic module; on this GPU-less runner the + # loader normally falls through the GPU backends to the CPU library. + - name: Load the natives jars together (classpath and module path) + run: | + mvn -B -q --no-transfer-progress -f llama/pom.xml dependency:build-classpath \ + -DincludeScope=runtime -Dmdep.outputFile=runtime-classpath.txt + bash .github/smoke-natives-jars.sh llama/target "$(cat llama/runtime-classpath.txt)" - name: Upload JARs uses: actions/upload-artifact@v7 with: @@ -3606,16 +3530,16 @@ jobs: path: jars/ # One self-contained multi-backend server fat jar per OS/arch # (llama--all---jar-with-dependencies.jar): the default - # jar-with-dependencies plus every GPU backend of that OS/arch in backend - # subdirectories, described by the jllama-backends.txt priority manifest that - # LlamaLoader reads at runtime (first loadable backend wins, CPU fallback). + # jar-with-dependencies reduced to that OS/arch plus every natives jar of it; each + # backend keeps its own directory, which LlamaLoader tries in its fixed priority + # order (first loadable backend wins, CPU fallback). # These are GitHub-Release download assets ONLY - never deployed to Maven # Central (the deploy jobs run without the `assembly` profile and are untouched). - # The script enumerates the classifiers from llama/pom.xml and fails loud on any - # mismatch with the built classifier jars, so a new classifier cannot be silently - # skipped. + # The script takes the natives jars from .github/natives.csv and fails loud on any + # mismatch with the built jars or on a natives jar holding anything but its own + # directory, so a new natives jar cannot be silently skipped. - name: Assemble all-backends fat jars - run: bash .github/package-fatjars.sh jars fatjars llama/pom.xml + run: bash .github/package-fatjars.sh jars fatjars - name: Upload fat jars uses: actions/upload-artifact@v7 with: @@ -3662,9 +3586,9 @@ jobs: retention-days: 7 if-no-files-found: error - # GPU-less runners: every manifest backend must fail its load cleanly (missing vendor - # runtimes) and the server must come up on the default CPU natives — this exercises - # manifest parsing, per-backend extraction, and the fallback chain end-to-end through + # GPU-less runners: every GPU backend must fail its load cleanly (missing vendor + # runtimes) and the server must come up on the CPU backend — this exercises backend + # probing, per-backend extraction, and the fallback chain end-to-end through # a real `java -jar`, then proves the server serves /health + /v1/chat/completions. smoke-fatjar-linux: name: Smoke test all-backends fat jar (Linux) @@ -3693,8 +3617,8 @@ jobs: distribution: 'temurin' java-version: ${{ env.JAVA_VERSION }} # Same floor, re-checked on the ASSEMBLED release asset rather than the module - # jars the `package` job verified: package-fatjars rewrites the zip (backend native - # trees + the jllama-backends.txt manifest), and this is the artifact users download. + # jars the `package` job verified: package-fatjars rewrites the zip (drops the other + # platforms, adds the backend trees), and this is the artifact users download. - name: Verify Java 8 bytecode (no class newer than major 52) run: .github/verify-bytecode-version.sh --max-major 52 fatjars - name: Run fat-jar server smoke test @@ -3994,98 +3918,22 @@ jobs: contents: write steps: - uses: actions/checkout@v7 - # Downloaded UNMERGED (one subdirectory per artifact name) on purpose: `merge-multiple: true` - # would silently overwrite — and can byte-level interleave — two artifacts that carry the same - # relative path, which is how the corrupt macOS dylib shipped. merge-native-artifacts.sh does - # the merge instead, and fails the job if any {OS}/{ARCH} path is claimed by more than one - # `*-libraries` artifact. A post-merge check cannot catch this (the collision leaves exactly - # one file on the path — a corrupt one), so the check has to happen before the merge. + # Every shipped native build uploads `natives-`. Downloaded UNMERGED (one + # subdirectory per artifact) on purpose: merge-native-artifacts.sh checks that each artifact + # holds exactly the /// directory its name promises and that no path is + # claimed twice, then merges. `merge-multiple: true` would silently overwrite -- and can + # byte-level interleave -- two artifacts with the same path, which is how a corrupt macOS + # dylib shipped; the test-only macOS builds (macos-14-metal, macos-15-no-metal) are named + # outside the glob for that reason. - uses: actions/download-artifact@v8 with: - pattern: "*-libraries" + pattern: "natives-*" path: ${{ github.workspace }}/native-artifacts/ - - name: Merge native libraries into the default resource tree (collision-checked) + - name: Merge native libraries into the natives tree (checked per artifact) run: | bash .github/merge-native-artifacts.sh \ "${{ github.workspace }}/native-artifacts" \ - "${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/" - # All three macOS arm64 build jobs emit the same path Mac/aarch64/libjllama.dylib, so their - # artifact names are kept outside the "*-libraries" glob above — merging them would drop - # three different dylibs onto one file. The variant that ships is selected explicitly: - # macos-15-metal is the only one built with both Metal and GGML_NATIVE=OFF (portable across - # CPU generations); macos-14-metal and macos-15-no-metal remain test-only. - - uses: actions/download-artifact@v8 - with: - name: macos-15-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: linux-libraries-cuda - path: ${{ github.workspace }}/llama/src/main/resources_linux_cuda/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-aarch64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: android-libraries-opencl - path: ${{ github.workspace }}/llama/src/main/resources_android_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-cuda - path: ${{ github.workspace }}/llama/src/main/resources_windows_cuda/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_windows_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_linux_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_windows_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp16 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp16/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp32 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp32/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-sycl - path: ${{ github.workspace }}/llama/src/main/resources_windows_sycl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-aarch64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_linux_openvino/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_windows_openvino/net/ladenthin/llama/ + "${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/" - name: Set up Maven Central Repository uses: actions/setup-java@v6 with: @@ -4105,9 +3953,9 @@ jobs: *-SNAPSHOT) echo "OK: -SNAPSHOT version, continuing snapshot deploy." ;; *) echo "::error::Refusing to publish non-SNAPSHOT version '$VERSION' from the snapshot job. Snapshot publishing requires a -SNAPSHOT version; releases go through the v* tag path."; exit 1 ;; esac - # One reactor deploy publishes all four Maven artifacts at the same version: - # net.ladenthin:llama-parent (the pom), :llama (the core jar + classifiers), - # :llama-langchain4j, and :llama-kotlin. The `release` profile (GPG + Central + # One reactor deploy publishes all five Maven artifacts at the same version: + # net.ladenthin:llama-parent (the pom), :llama (the classes jar + natives jars), + # :llama-langchain4j, :llama-kotlin and :llama-platform (a pom). The `release` profile (GPG + Central # Publishing) is inherited from the parent, so every module — including the # parent pom — is signed. The Android AARs are published by the separate # Gradle step below (Maven cannot deploy aar). @@ -4115,9 +3963,9 @@ jobs: # profile set as the deploy below, so the resolved central-publishing configuration # (waitUntil/waitMaxTime etc.) is visible for debugging. - name: Show effective POM (debug) - run: mvn --batch-mode --no-transfer-progress -P release,cuda,vulkan-linux,vulkan-linux-aarch64,opencl-android,windows-msvc,cuda-windows,vulkan-windows,opencl-windows,rocm-linux,rocm-windows,sycl-fp16-linux,sycl-fp32-linux,sycl-windows,opencl-windows-aarch64,openvino-linux,openvino-windows help:effective-pom - - name: Publish snapshot (reactor - parent + llama + llama-langchain4j + llama-kotlin) - run: mvn --batch-mode --no-transfer-progress -P release,cuda,vulkan-linux,vulkan-linux-aarch64,opencl-android,windows-msvc,cuda-windows,vulkan-windows,opencl-windows,rocm-linux,rocm-windows,sycl-fp16-linux,sycl-fp32-linux,sycl-windows,opencl-windows-aarch64,openvino-linux,openvino-windows -Dmaven.test.skip=true deploy + run: mvn --batch-mode --no-transfer-progress -P release,natives help:effective-pom + - name: Publish snapshot (reactor - parent + llama + llama-langchain4j + llama-kotlin + llama-platform) + run: mvn --batch-mode --no-transfer-progress -P release,natives -Dmaven.test.skip=true deploy env: MAVEN_USERNAME: ${{ secrets.CENTRAL_USERNAME }} MAVEN_PASSWORD: ${{ secrets.CENTRAL_TOKEN }} @@ -4132,9 +3980,9 @@ jobs: - name: Publish Android AAR snapshots (llama-android + llama-android-opencl) run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 llama-android/natives/opencl/arm64-v8a - cp llama/src/main/resources/net/ladenthin/llama/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp llama/src/main/resources/net/ladenthin/llama/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ - cp llama/src/main/resources_android_opencl/net/ladenthin/llama/Linux-Android/aarch64/libjllama.so llama-android/natives/opencl/arm64-v8a/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/aarch64/opencl/libjllama.so llama-android/natives/opencl/arm64-v8a/ gradle -p llama-android publishAllPublicationsToCentralSnapshotsRepository env: CENTRAL_USERNAME: ${{ secrets.CENTRAL_USERNAME }} @@ -4266,98 +4114,22 @@ jobs: contents: write steps: - uses: actions/checkout@v7 - # Downloaded UNMERGED (one subdirectory per artifact name) on purpose: `merge-multiple: true` - # would silently overwrite — and can byte-level interleave — two artifacts that carry the same - # relative path, which is how the corrupt macOS dylib shipped. merge-native-artifacts.sh does - # the merge instead, and fails the job if any {OS}/{ARCH} path is claimed by more than one - # `*-libraries` artifact. A post-merge check cannot catch this (the collision leaves exactly - # one file on the path — a corrupt one), so the check has to happen before the merge. + # Every shipped native build uploads `natives-`. Downloaded UNMERGED (one + # subdirectory per artifact) on purpose: merge-native-artifacts.sh checks that each artifact + # holds exactly the /// directory its name promises and that no path is + # claimed twice, then merges. `merge-multiple: true` would silently overwrite -- and can + # byte-level interleave -- two artifacts with the same path, which is how a corrupt macOS + # dylib shipped; the test-only macOS builds (macos-14-metal, macos-15-no-metal) are named + # outside the glob for that reason. - uses: actions/download-artifact@v8 with: - pattern: "*-libraries" + pattern: "natives-*" path: ${{ github.workspace }}/native-artifacts/ - - name: Merge native libraries into the default resource tree (collision-checked) + - name: Merge native libraries into the natives tree (checked per artifact) run: | bash .github/merge-native-artifacts.sh \ "${{ github.workspace }}/native-artifacts" \ - "${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/" - # All three macOS arm64 build jobs emit the same path Mac/aarch64/libjllama.dylib, so their - # artifact names are kept outside the "*-libraries" glob above — merging them would drop - # three different dylibs onto one file. The variant that ships is selected explicitly: - # macos-15-metal is the only one built with both Metal and GGML_NATIVE=OFF (portable across - # CPU generations); macos-14-metal and macos-15-no-metal remain test-only. - - uses: actions/download-artifact@v8 - with: - name: macos-15-metal - path: ${{ github.workspace }}/llama/src/main/resources/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: linux-libraries-cuda - path: ${{ github.workspace }}/llama/src/main/resources_linux_cuda/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-aarch64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_linux_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: android-libraries-opencl - path: ${{ github.workspace }}/llama/src/main/resources_android_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86-msvc - path: ${{ github.workspace }}/llama/src/main/resources_windows_msvc/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-cuda - path: ${{ github.workspace }}/llama/src/main/resources_windows_cuda/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-vulkan - path: ${{ github.workspace }}/llama/src/main/resources_windows_vulkan/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_linux_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-rocm - path: ${{ github.workspace }}/llama/src/main/resources_windows_rocm/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp16 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp16/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-sycl-fp32 - path: ${{ github.workspace }}/llama/src/main/resources_linux_sycl_fp32/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-sycl - path: ${{ github.workspace }}/llama/src/main/resources_windows_sycl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-aarch64-opencl - path: ${{ github.workspace }}/llama/src/main/resources_windows_opencl/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Linux-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_linux_openvino/net/ladenthin/llama/ - - uses: actions/download-artifact@v8 - with: - name: Windows-x86_64-openvino - path: ${{ github.workspace }}/llama/src/main/resources_windows_openvino/net/ladenthin/llama/ + "${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/" - name: Set up Maven Central Repository uses: actions/setup-java@v6 with: @@ -4368,17 +4140,17 @@ jobs: server-password: MAVEN_PASSWORD gpg-private-key: ${{ secrets.GPG_PRIVATE_KEY }} gpg-passphrase: MAVEN_GPG_PASSPHRASE - # One reactor deploy publishes all three artifacts at the same version: - # net.ladenthin:llama-parent (the pom), :llama (the core jar + classifiers), and - # :llama-langchain4j. The `release` profile (GPG + Central Publishing) is inherited + # One reactor deploy publishes all five Maven artifacts at the same version: + # net.ladenthin:llama-parent (the pom), :llama (the classes jar + natives jars), + # :llama-langchain4j, :llama-kotlin and :llama-platform (a pom). The `release` profile (GPG + Central Publishing) is inherited # from the parent, so every module — including the parent pom — is signed. # Informational only (nothing depends on it): logs the effective POM with the same # profile set as the deploy below, so the resolved central-publishing configuration # (waitUntil/waitMaxTime etc.) is visible for debugging. - name: Show effective POM (debug) - run: mvn --batch-mode --no-transfer-progress -P release,cuda,vulkan-linux,vulkan-linux-aarch64,opencl-android,windows-msvc,cuda-windows,vulkan-windows,opencl-windows,rocm-linux,rocm-windows,sycl-fp16-linux,sycl-fp32-linux,sycl-windows,opencl-windows-aarch64,openvino-linux,openvino-windows help:effective-pom - - name: Publish release (reactor - parent + llama + llama-langchain4j + llama-kotlin) - run: mvn --batch-mode --no-transfer-progress -P release,cuda,vulkan-linux,vulkan-linux-aarch64,opencl-android,windows-msvc,cuda-windows,vulkan-windows,opencl-windows,rocm-linux,rocm-windows,sycl-fp16-linux,sycl-fp32-linux,sycl-windows,opencl-windows-aarch64,openvino-linux,openvino-windows -Dmaven.test.skip=true deploy + run: mvn --batch-mode --no-transfer-progress -P release,natives help:effective-pom + - name: Publish release (reactor - parent + llama + llama-langchain4j + llama-kotlin + llama-platform) + run: mvn --batch-mode --no-transfer-progress -P release,natives -Dmaven.test.skip=true deploy env: MAVEN_USERNAME: ${{ secrets.CENTRAL_USERNAME }} MAVEN_PASSWORD: ${{ secrets.CENTRAL_TOKEN }} @@ -4398,9 +4170,9 @@ jobs: run: | set -euo pipefail mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 llama-android/natives/opencl/arm64-v8a - cp llama/src/main/resources/net/ladenthin/llama/Linux-Android/aarch64/libjllama.so llama-android/natives/cpu/arm64-v8a/ - cp llama/src/main/resources/net/ladenthin/llama/Linux-Android/x86_64/libjllama.so llama-android/natives/cpu/x86_64/ - cp llama/src/main/resources_android_opencl/net/ladenthin/llama/Linux-Android/aarch64/libjllama.so llama-android/natives/opencl/arm64-v8a/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ + cp llama/src/main/natives/net/ladenthin/llama/Linux-Android/aarch64/opencl/libjllama.so llama-android/natives/opencl/arm64-v8a/ gradle -p llama-android publishAllPublicationsToStagingRepository VERSION=$(mvn -q -DforceStdout help:evaluate -Dexpression=project.version | tail -n1) ( cd llama-android/build/staging-repo && zip -r -q ../central-bundle.zip net -x "*maven-metadata*" ) diff --git a/.gitignore b/.gitignore index e81cba162..840edb395 100644 --- a/.gitignore +++ b/.gitignore @@ -46,25 +46,9 @@ models/*.gguf # the agent README downloads its model next to the agent: llama-atmosphere-agent/models/ llama-atmosphere-agent/models/*.gguf llama/src/main/cpp/net_ladenthin_llama_*.h -llama/src/main/resources_linux_cuda/ -# Per-classifier native trees, staged by CI before the matching Maven profile runs, -# never committed (same policy as the default-tree native libs below). -llama/src/main/resources_linux_vulkan/ -llama/src/main/resources_windows_msvc/ -llama/src/main/resources_windows_cuda/ -llama/src/main/resources_windows_vulkan/ -llama/src/main/resources_windows_opencl/ -llama/src/main/resources_linux_rocm/ -llama/src/main/resources_windows_rocm/ -llama/src/main/resources_linux_sycl_fp16/ -llama/src/main/resources_linux_sycl_fp32/ -llama/src/main/resources_windows_sycl/ -llama/src/main/resources_linux_openvino/ -llama/src/main/resources_windows_openvino/ -llama/src/main/resources/**/*.so -llama/src/main/resources/**/*.dylib -llama/src/main/resources/**/*.dll -llama/src/main/resources/**/*.metal +# Native libraries: one tree, ///, written by CMake and staged by CI, +# never committed. +llama/src/main/natives/ llama/src/test/resources/**/*.gbnf # Generated WebUI assets (ui.cpp/ui.h) produced once by the build-webui CI job and diff --git a/CLAUDE.md b/CLAUDE.md index 3bf6b0f47..3d67a8fd1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,6 +8,69 @@ Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, p Current llama.cpp pinned version: **b11259** +## Natives jars: one directory per backend (`.github/natives.csv`) + +`net.ladenthin:llama` is the **Java classes only**. Every native build ships as its own jar of the +same artifact, classifier `--`, holding exactly one directory +`net/ladenthin/llama////` (26 today: `cpu-*` for 8 platforms, `metal-macos-aarch64`, +`msvc-windows-*`, and the GPU backends). Because the directories never overlap, **any combination of +natives jars can share one classpath**, and `LlamaLoader` tries the backends it finds in a fixed +order (`BACKEND_PRIORITY`: cuda13, rocm, sycl-fp16, sycl-fp32, sycl, vulkan, opencl, openvino, metal, +msvc, cpu), taking the first whose library loads. Every build carries the CPU backend, so a GPU +library that loads can still run a model on the CPU (`-ngl 0`); one that fails to load (no vendor +runtime) falls through to the next directory, ending at `cpu`. `net.ladenthin.llama.backend=` +forces one and fails loud. `net.ladenthin:llama-platform` is a **pom-packaging** module (a +dependency list, no jar, nothing to upload) naming the classes jar plus the CPU/Metal jars of every +desktop platform; consumers use it with `pom`. + +This replaced (in 5.2.0) the old layout, where the default jar carried the CPU natives of every +platform at `//` and each GPU classifier was a complete replacement jar with its own +compile pass: two jars could not share a classpath, a GPU jar had no CPU fallback, and the +all-backends fat jars needed a `jllama-backends.txt` manifest to tell backends apart. + +**The list is `.github/natives.csv`** (classifier, directory, library, platform yes/no) — the one +place a natives jar is declared, in the same spirit as `models.csv`. Everything else either reads +it or is checked against it: + +| Place | Relation to the list | +|---|---| +| `llama/CMakeLists.txt` | names the backend directory (`JLLAMA_BACKEND`: GGML flag → name; `msvc` = Visual Studio generator; `metal` = Apple + `GGML_METAL`) and writes to `src/main/natives/net/ladenthin/llama////` | +| `llama/pom.xml` `natives` profile | one `maven-jar-plugin` execution per row, **generated**: `python3 .github/check-natives.py pom` prints them | +| `llama-platform/pom.xml` | depends on the rows with `platform=yes` | +| `publish.yml` build jobs | each uploads its tree as artifact `natives-` | +| `LlamaLoader.BACKEND_PRIORITY` | must contain every backend directory | +| `.github/check-natives.py` | runs in `code-style` (first minutes of every run); fails when the four rows above disagree with the list | +| `.github/merge-native-artifacts.sh` | reads the list: every listed `natives-*` artifact present, no other, each holding its library and nothing outside its directory, no path claimed twice; writes `jllama-extras.txt` (sibling files loaded before the library, e.g. OpenVINO's `OpenCL.dll` on Windows) | +| `.github/package-fatjars.sh` | reads the list: the built natives jars match it, each holds only its directory and the right `Automatic-Module-Name`; merges one `all--` fat jar per OS/arch with a GPU backend (no `msvc`, no Android) | +| `.github/verify-native-deps.py` | exact dependency allowlist per CPU directory (`cpu`/`metal`/`msvc`), denylist for GPU ones | +| `.github/smoke-natives-jars.sh` (`package` job) | loads the real jars: classes + all 26 natives jars at once, on the classpath **and** the module path (on the GPU-less runner normally ending at `cpu`) | + +**Adding a natives jar:** a row in `natives.csv`, the execution `check-natives.py pom` prints, a build +job uploading `natives-`, the backend name in CMake and `BACKEND_PRIORITY` if new, a +README row. `check-natives.py` fails until they agree; nothing else needs touching. + +**Why the build jobs are not spawned from the list as one matrix** (considered and rejected): the 26 +builds use genuinely different toolchains — dockcross images, the CUDA redist archives, ROCm pip +wheels, oneAPI, OpenVINO, `clang-cl` on arm64, qemu for s390x, three macOS variants — so a single +matrix job would be a web of `if:` conditions; and `needs:` on a matrix waits for every entry, so +each test job would wait for the slowest build (CUDA). The list removes the scattering that matters +(the packaging facts); a build job's only contract is its artifact name, and the merge check +enforces it. + +**Module path.** Resources are looked up through the `ClassLoader`, never `Class.getResource` +(which only sees the own module's resources on the module path). Each natives jar declares +`Automatic-Module-Name: net.ladenthin.llama.natives.`. **Without it every +natives jar derives the module name `llama` from its file name and the JVM does not refuse to start — +it silently keeps the first one and drops the others' natives** (measured; `package-fatjars.sh` +checks the manifest of every built jar for that reason). Nothing `requires` a natives module, so a +module-path launch needs `--add-modules` (or the natives jars on the classpath). `module-info.java` +`requires` Jackson and SLF4J — it did not before, so the classes jar never worked on the module path. + +**Locally**, CMake writes into `llama/src/main/natives/` (git-ignored); surefire adds that directory +to the test classpath (`additionalClasspathElements`), so `mvn test` finds the library. `-P natives` +is CI-only (a local tree holds one directory); a missing directory there is caught by the merge +step, not by Maven. + ## Upgrading CUDA Version Current CUDA version: **13.4** (Linux `cuda-toolkit-13-4` from NVIDIA's rhel8 repo; Windows 13.4 redist archives) @@ -22,8 +85,10 @@ To change the CUDA version, update the following places: `.github/actions/windows-setup-cuda/action.yml` at the pinned `GIT_TAG`** — the component versions differ per component (cuBLAS and CCCL have their own numbering) and cannot be derived from the CUDA version. (This replaced `Jimver/cuda-toolkit`, which never shipped 13.4.) -3. **`llama/pom.xml`** — the ``s `cuda13-linux-x86-64` / `cuda13-windows-x86-64` - (major version only — no change for a minor bump). +3. **Only for a major bump:** the backend name `cuda13` — in `llama/CMakeLists.txt` + (`JLLAMA_BACKEND`), `.github/natives.csv` (the two `cuda13-*` rows), the generated pom executions + (`check-natives.py pom`), the build jobs' artifact names, and `LlamaLoader.BACKEND_PRIORITY`. + `check-natives.py` fails until they agree. No change for a minor bump. 4. **`CLAUDE.md`** — the "Current CUDA version" line above. Available CUDA versions for RHEL8/Manylinux_2_28 can be browsed at: @@ -123,24 +188,12 @@ detects Android via `OS_NAME MATCHES "Android"` (CI passes ## OpenCL / Adreno backend on Android -A second Android arm64 artifact is built with the OpenCL backend enabled and -Adreno-tuned kernels embedded. It ships under the Maven classifier -`opencl-android-aarch64` and is consumed only when callers explicitly request it. -The default Android arm64 JAR remains CPU-only. - -Three places wire it together (mirrors the CUDA classifier pattern): - -1. **`llama/CMakeLists.txt`** — `elseif(GGML_OPENCL)` branch routes artifacts to - `src/main/resources_android_opencl/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}/`. -2. **`.github/workflows/publish.yml`** — `crosscompile-android-aarch64-opencl` - job runs the dockcross-android-arm64 build with - `-DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON -DGGML_OPENCL_USE_ADRENO_KERNELS=ON` - and uploads as artifact `android-libraries-opencl`. The `package`, - `publish-snapshot`, and `publish-release` jobs download it into - `resources_android_opencl/` and activate the `opencl-android` Maven profile. -3. **`llama/pom.xml`** — the `opencl-android` profile produces a second JAR with - `opencl-android-aarch64` from the - `${project.build.outputDirectory}_opencl_android` tree. +A second Android arm64 build has the OpenCL backend enabled and Adreno-tuned kernels embedded. +It ships as the natives jar `opencl-android-aarch64` (directory `Linux-Android/aarch64/opencl/`) +and as the `llama-android-opencl` AAR; the `cpu-android-*` jars and the `llama-android` AAR stay +CPU-only. The `crosscompile-android-aarch64-opencl` job runs the dockcross-android-arm64 build with +`-DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON -DGGML_OPENCL_USE_ADRENO_KERNELS=ON` and uploads +`natives-opencl-android-aarch64`; the rest is the natives-jar wiring (see "Natives jars" above). Local sanity build: ```bash @@ -149,37 +202,36 @@ Local sanity build: -DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON \ -DGGML_OPENCL_USE_ADRENO_KERNELS=ON" ``` -Artifacts land in `src/main/resources_android_opencl/net/ladenthin/llama/Linux-Android/aarch64/`. +Artifacts land in `src/main/natives/net/ladenthin/llama/Linux-Android/aarch64/opencl/`. The dockcross image does not ship OpenCL headers or a stub `libOpenCL.so`, so `build_opencl_android.sh` first stages Khronos `OpenCL-Headers` and cross-builds `OpenCL-ICD-Loader` into `/tmp/opencl-stage/` before invoking the main project cmake with `-DOpenCL_INCLUDE_DIR=...` and `-DOpenCL_LIBRARY=...`. At runtime the device must provide its own OpenCL ICD (`libOpenCL.so`); -Qualcomm Adreno drivers do. Devices without an ICD should use the default -CPU-only Android JAR. +Qualcomm Adreno drivers do. Devices without an ICD should use the CPU-only +`llama-android` AAR. -## Windows native classifiers (default Ninja CPU + MSVC classifier + CUDA/Vulkan/OpenCL GPU) +## Windows natives (Ninja CPU + MSVC + CUDA/Vulkan/OpenCL GPU) -The Windows native libraries ship in **five** forms. The **default JAR's** Windows natives are now -built with the **`Ninja Multi-Config`** generator (the *default flip*); the Visual Studio / MSVC -build is shipped as the **`msvc-windows`** classifier; and three GPU backends ship as -**`cuda13-windows-x86-64`**, **`vulkan-windows-x86-64`**, and **`opencl-windows-x86-64`** (all -**x86_64 only**, all Ninja). +The Windows x86-64 natives ship in **five** forms (natives jars): the CPU build with the +**`Ninja Multi-Config`** generator (`cpu-windows-x86-64` / `cpu-windows-x86`, the ones in +`llama-platform`), the Visual Studio / MSVC build (`msvc-windows-x86-64` / `msvc-windows-x86`), and +three GPU backends, **`cuda13-windows-x86-64`**, **`vulkan-windows-x86-64`** and +**`opencl-windows-x86-64`** (all Ninja). **Why Ninja is the default (the flip).** The Visual Studio generator ignores `CMAKE_{C,CXX}_COMPILER_LAUNCHER`, so only Ninja Multi-Config can front `cl.exe` with sccache over Depot WebDAV. **Both generators use the same MSVC toolchain** (`cl.exe`, static `/MT` CRT via `CMAKE_MSVC_RUNTIME_LIBRARY`, same Release flags, same runner), so the produced `jllama.dll` binaries are **functionally equivalent with identical runtime -dependencies** — the only difference is build-system plumbing + caching. Making Ninja the default -gives the most-pulled JAR the sccache cache; MSVC stays available as a classifier for anyone who -wants the Visual-Studio-generator build. (Upstream llama.cpp also builds its Windows artifacts with +dependencies** — the only difference is build-system plumbing + caching. Making Ninja the `cpu` +build gives the most-pulled natives the sccache cache; MSVC stays available (directory `msvc`, +which the loader tries before `cpu`) for anyone who wants the Visual-Studio-generator build. (Upstream llama.cpp also builds its Windows artifacts with Ninja Multi-Config + MSVC.) Both Windows CPU builds are validated end-to-end with the full -model-backed Java suite (`test-java-windows-x86_64` = default/Ninja, `test-java-windows-x86_64-msvc` -= MSVC classifier). +model-backed Java suite (`test-java-windows-x86_64` = Ninja, `test-java-windows-x86_64-msvc` = MSVC). -**GPU runtime libraries are NOT bundled.** The GPU JARs ship only the single monolithic +**GPU runtime libraries are NOT bundled.** The GPU natives jars ship only the single monolithic `jllama.dll` (llama.cpp + ggml + the backend are statically linked in — `BUILD_SHARED_LIBS OFF`). The consumer's driver/toolkit must supply the runtime: CUDA needs the installed CUDA 13 Toolkit (`cudart64_13.dll`/`cublas64_13.dll`/`cublasLt64_13.dll` on `PATH`); Vulkan needs `vulkan-1.dll` (ships with current GPU drivers); OpenCL needs the vendor ICD @@ -190,45 +242,26 @@ device, so `gtest_discover_tests` registers a failing `*_NOT_BUILT` sentinel). T suite is fully covered by the `C++ Tests` job + the CPU Windows jobs; model-backed GPU inference is local / self-hosted. -Wiring (mirrors the CUDA-Linux / OpenCL-Android classifier pattern): +Windows-specific wiring (the rest is the natives-jar wiring, see "Natives jars" above; CMake names +the Visual Studio generator build `msvc`, the Ninja one `cpu`): -1. **`llama/CMakeLists.txt`** — the `if(GGML_CUDA) … elseif(GGML_VULKAN) … elseif(GGML_OPENCL) … else()` - chain is **OS-aware**: CUDA → `resources_windows_cuda` on Windows (else `resources_linux_cuda`), - Vulkan → `resources_windows_vulkan` on Windows (else `resources_linux_vulkan` — see "Linux Vulkan - classifiers" above), OpenCL → `resources_windows_opencl` on Windows (else - `resources_android_opencl`). The default CPU build (both generators) still emits to the canonical - `src/main/resources/.../Windows/{x86_64,x86}/`, so the Ninja-vs-MSVC split is purely a - CI-artifact-name + pom-profile concern (no CMake change for it). -2. **`.github/build.bat`** — the sccache probe guard (mirrors `build.sh`) wraps the **cl.exe** C/C++ TUs +1. **`.github/build.bat`** — the sccache probe guard (mirrors `build.sh`) wraps the **cl.exe** C/C++ TUs only. Unlike `build.sh` (Linux), it does **not** wrap `nvcc`: sccache on Windows can't parse the nvcc command line (`sccache: error: Could not parse shell line`) and fails every `.cu` compile, so CUDA device code builds with nvcc directly (uncached). `build.bat` also propagates a `cmake --build` failure as a non-zero exit (a prior bug let a failed CUDA build exit 0 → empty artifact → late `package` failure); the GPU upload steps additionally use `if-no-files-found: error` as a backstop. -3. **`.github/build_opencl_windows.bat`** — stages Khronos OpenCL-Headers + builds OpenCL-ICD-Loader +2. **`.github/build_opencl_windows.bat`** — stages Khronos OpenCL-Headers + builds OpenCL-ICD-Loader (`OpenCL.lib`), then delegates to `build.bat` with `-DOpenCL_INCLUDE_DIR`/`-DOpenCL_LIBRARY` (the Windows analogue of `build_opencl_android.sh`). -4. **`.github/workflows/publish.yml`** — build jobs (all `windows-2025-vs2026`, `ilammy/msvc-dev-cmd@v1`, - sccache v0.18.0 zip + Depot WebDAV): - - `build-windows-x86_64` / `build-windows-x86` — **Ninja CPU**, artifacts `Windows-{arch}-libraries` - → picked up by the `package` job's `pattern: "*-libraries"` into the **default** tree. - - `build-windows-x86_64-msvc` / `build-windows-x86-msvc` — **MSVC CPU**, artifacts `Windows-{arch}-msvc`. +3. **`.github/workflows/publish.yml`** — build jobs (all `windows-2025-vs2026`, `ilammy/msvc-dev-cmd@v1`, + sccache v0.18.0 zip + Depot WebDAV), each uploading `natives-`: + - `build-windows-x86_64` / `build-windows-x86` — **Ninja CPU**. + - `build-windows-x86_64-msvc` / `build-windows-x86-msvc` — **MSVC CPU**. - `build-windows-x86_64-cuda` — CUDA `13.4` assembled from NVIDIA's redist archives (upstream's - `windows-setup-cuda` component list; `Jimver/cuda-toolkit` stops at 13.3.1) + `-DGGML_CUDA=ON`, - artifact `Windows-x86_64-cuda`. - - `build-windows-x86_64-vulkan` — `jakoch/install-vulkan-sdk-action` + `-DGGML_VULKAN=ON`, artifact - `Windows-x86_64-vulkan`. - - `build-windows-x86_64-opencl` — `build_opencl_windows.bat -DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON`, - artifact `Windows-x86_64-opencl`. - The `package`, `publish-snapshot`, and `publish-release` jobs download each non-default artifact into - its `src/main/resources_windows_{msvc,cuda,vulkan,opencl}/` tree and activate the - `windows-msvc,cuda-windows,vulkan-windows,opencl-windows` Maven profiles. -5. **`llama/pom.xml`** — profiles `windows-msvc` / `cuda-windows` / `vulkan-windows` / `opencl-windows`, - each a separate compile pass + resource copy + classified jar (classifiers `msvc-windows` / - `cuda13-windows-x86-64` / `vulkan-windows-x86-64` / `opencl-windows-x86-64`). Activated only in CI. -6. **`README.md`** — the classifier table + dependency snippets in "Choosing the right classifier". - -`src/main/resources_windows_{msvc,cuda,vulkan,opencl}/` are git-ignored (staged by CI, never committed). + `windows-setup-cuda` component list; `Jimver/cuda-toolkit` stops at 13.3.1) + `-DGGML_CUDA=ON`. + - `build-windows-x86_64-vulkan` — `jakoch/install-vulkan-sdk-action` + `-DGGML_VULKAN=ON`. + - `build-windows-x86_64-opencl` — `build_opencl_windows.bat -DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON`. **First CI run (PR #276, run 28327740376):** the default Ninja CPU flip, the MSVC classifier, and the **OpenCL** job were green on the first try. Two GPU jobs needed a toolchain fix: **CUDA** failed with @@ -237,8 +270,7 @@ bumped to `@v0.2.35` + `13.2.0` (matches the Linux pin, classifier stays `cuda13 `find_package(Vulkan)` because `humbletim/install-vulkan-sdk` set `VULKAN_SDK` but laid the SDK out in a way CMake's `FindVulkan` couldn't read → switched to `jakoch/install-vulkan-sdk-action` (purpose-built, FindVulkan-compatible). Because all five Windows build jobs are in the `package`/publish `needs:` graph, a -GPU-toolchain failure blocks packaging — the same release-gating policy the Linux-CUDA / Android-OpenCL -jobs already follow. +GPU-toolchain failure blocks packaging — the same release-gating policy every build job follows. **sccache on every Windows Ninja job.** All ten Ninja build jobs install sccache (x86_64 or the native `aarch64` release) with the same `USE_CACHE` / `SCCACHE_WEBDAV_*` env; only the two MSVC-classifier @@ -265,45 +297,24 @@ ctest --test-dir build --output-on-failure .github\build_opencl_windows.bat -G "Ninja Multi-Config" -DGGML_OPENCL=ON -DGGML_OPENCL_EMBED_KERNELS=ON -DOS_NAME=Windows -DOS_ARCH=x86_64 ``` -## Linux Vulkan classifiers + Windows arm64 CPU +## Linux Vulkan natives + Windows arm64 CPU -Three additional artifacts extend the matrix toward upstream llama.cpp's release set. They follow -the same classifier/resource-tree pattern as CUDA-Linux and Vulkan-Windows. +Three natives jars that extend the matrix toward upstream llama.cpp's release set. **Linux Vulkan (`vulkan-linux-x86-64` + `vulkan-linux-aarch64`).** A vendor-neutral GPU jar for -Linux (NVIDIA / AMD / Intel) with no CUDA toolkit — the intersection of the existing Vulkan-Windows -and CUDA-Linux wiring. Four places: - -1. **`llama/CMakeLists.txt`** — the `elseif(GGML_VULKAN)` branch is now **OS-aware** (mirrors - `GGML_CUDA`): Windows → `resources_windows_vulkan`, else → `resources_linux_vulkan` - (`.../Linux/${OS_ARCH}/`). One tree holds both arches under `Linux/{x86_64,aarch64}`. -2. **`.github/workflows/publish.yml`** — `build-linux-x86_64-vulkan` (native `ubuntu-latest`, **not** - dockcross — the Vulkan SDK is a trivial apt install and upstream builds ubuntu-vulkan the same way) - and `build-linux-aarch64-vulkan` (`ubuntu-24.04-arm` + GCC 14). Both `apt-get install libvulkan-dev - glslc glslang-tools`, build `-DGGML_VULKAN=ON -DGGML_NATIVE=OFF`, and are **build-only** (no - `ctest`: a Vulkan-linked `jllama_test` errors enumerating devices on a GPU-less runner — same as the - Windows GPU jobs). Artifacts `Linux-{x86_64,aarch64}-vulkan` → both downloaded into the **one** - `resources_linux_vulkan/` tree by `package`/`publish-*`. Glibc floor rises to the ubuntu baseline - (like the aarch64 CPU jar); acceptable for a GPU artifact. -3. **`llama/pom.xml`** — profiles `vulkan-linux` (classifier `vulkan-linux-x86-64`) and - `vulkan-linux-aarch64` (classifier `vulkan-linux-aarch64`). Both read the shared - `resources_linux_vulkan` tree but the resource-copy `` is **arch-scoped** - (`net/ladenthin/llama/Linux/{x86_64,aarch64}/**`), so each classifier JAR carries only its own - arch (verified: each jar contains exactly one `libjllama.so`). Separate output dirs - `_linux_vulkan` / `_linux_vulkan_aarch64` avoid collision. Activated in CI via - `-P …,vulkan-linux,vulkan-linux-aarch64,…`. -4. **`README.md`** — classifier table + dependency snippets. - -`src/main/resources_linux_vulkan/` is git-ignored (staged by CI, never committed). GPU runtime -`libvulkan.so.1` is supplied by the consumer's driver — nothing is bundled (same policy as every GPU -classifier). - -**Windows arm64 CPU (default JAR, no classifier).** `build-windows-arm64` runs natively on GitHub's -free `windows-11-arm` runner (`ilammy/msvc-dev-cmd` `arch: arm64`, Ninja Multi-Config, `-DOS_ARCH=aarch64`, -build + `ctest`). It emits to the **canonical** `resources/.../Windows/aarch64/` and uploads -`Windows-aarch64-libraries`, which the `package`/`publish-*` `*-libraries` glob merges into the default -tree — so it ships in the **default** JAR alongside Windows x86-64 / x86 (like those, it is not a -classifier). No Java change was needed: `OSInfo` already maps a Windows-on-ARM JVM (`os.arch=aarch64`) +Linux (NVIDIA / AMD / Intel) with no CUDA toolkit. The build jobs are `build-linux-x86_64-vulkan` +(native `ubuntu-latest`, **not** dockcross — the Vulkan SDK is a trivial apt install and upstream +builds ubuntu-vulkan the same way) and `build-linux-aarch64-vulkan` (`ubuntu-24.04-arm` + GCC 14). +Both `apt-get install libvulkan-dev glslc glslang-tools`, build `-DGGML_VULKAN=ON -DGGML_NATIVE=OFF`, +and are **build-only** (no `ctest`: a Vulkan-linked `jllama_test` errors enumerating devices on a +GPU-less runner — same as the Windows GPU jobs). Glibc floor rises to the ubuntu baseline (like the +aarch64 CPU jar); acceptable for a GPU artifact. GPU runtime `libvulkan.so.1` is supplied by the +consumer's driver — nothing is bundled (same policy as every GPU backend). + +**Windows arm64 CPU (`cpu-windows-aarch64`, in `llama-platform`).** `build-windows-arm64` runs +natively on GitHub's free `windows-11-arm` runner (`ilammy/msvc-dev-cmd` `arch: arm64`, Ninja +Multi-Config, `-DOS_ARCH=aarch64`, build + `ctest`) and writes `Windows/aarch64/cpu/`. No Java change +was needed: `OSInfo` already maps a Windows-on-ARM JVM (`os.arch=aarch64`) to `Windows/aarch64` (it isn't in `archMapping`, so it falls through `translateArchNameToFolderName`). sccache runs here too, from its native `aarch64-pc-windows-msvc` release (it wraps `clang-cl`; see "sccache on every Windows Ninja job" above). **Compiler: `clang-cl`, not MSVC @@ -322,25 +333,23 @@ threadpool, leaving the arm64 `jllama.dll` self-contained (the x86_64/x86 jobs k x64 runner with `vcvarsall amd64_arm64` + a `clang`/`clang++` toolchain file and no arm64 tests; the native-runner + `clang-cl` route here keeps the `/MT` CRT and lets `ctest` run on real ARM hardware.) -## Additional GPU-backend classifiers (ROCm/HIP, SYCL, Win-arm64 OpenCL, OpenVINO) +## Additional GPU-backend natives (ROCm/HIP, SYCL, Win-arm64 OpenCL, OpenVINO) -Eight further GPU classifiers extend the matrix toward upstream llama.cpp's full release set. They -follow the **exact same 5-place wiring** as the CUDA/Vulkan classifiers (no special cases — KISS): a -`CMakeLists.txt` backend branch, a `publish.yml` build job (in `package.needs`, **fail-loud** — a -broken build reds the pipeline, same policy as every GPU job), a `pom.xml` classifier profile, a -`README.md` row, and a git-ignored `resources_*` tree. All are **build-only** (GitHub runners have no -matching GPU) and bundle **no** vendor runtime. +Eight further GPU natives jars extend the matrix toward upstream llama.cpp's full release set, with +the same natives-jar wiring as every other one (see "Natives jars"; each build job is in +`package.needs`, **fail-loud**). All are **build-only** (GitHub runners have no matching GPU) and +bundle **no** vendor runtime. -| Classifier | GGML flag(s) | Job runner / toolchain | Tree | +| Classifier | GGML flag(s) | Job runner / toolchain | Directory | |---|---|---|---| -| `rocm-linux-x86-64` | `GGML_HIP=ON -DCMAKE_HIP_COMPILER=… -DGPU_TARGETS=…` | `ubuntu-latest` + ROCm 10 TheRock wheels (pip, `rocm-sdk path`) | `resources_linux_rocm` | -| `rocm-windows-x86-64` | `GGML_HIP=ON` | `windows-2022` + ROCm 10 TheRock wheels (pip) | `resources_windows_rocm` | -| `sycl-fp16-linux-x86-64` | `GGML_SYCL=ON -DGGML_SYCL_F16=ON` (`icx`/`icpx`) | `ubuntu-latest` + Intel oneAPI apt | `resources_linux_sycl_fp16` | -| `sycl-fp32-linux-x86-64` | `GGML_SYCL=ON` (`icx`/`icpx`) | `ubuntu-latest` + Intel oneAPI apt | `resources_linux_sycl_fp32` | -| `sycl-windows-x86-64` | `GGML_SYCL=ON` (`icx`) | `windows-2025-vs2026` + oneAPI installer | `resources_windows_sycl` | -| `opencl-windows-aarch64` | `GGML_OPENCL=ON …ADRENO_KERNELS=ON` (clang-cl, `GGML_OPENMP=OFF`) | `windows-11-arm` (arm64 CPU job's toolchain) | `resources_windows_opencl` (arch subdir `aarch64`) | -| `openvino-linux-x86-64` | `GGML_OPENVINO=ON` | `ubuntu-latest` + OpenVINO apt | `resources_linux_openvino` | -| `openvino-windows-x86-64` | `GGML_OPENVINO=ON` | `windows-2025-vs2026` + OpenVINO archive | `resources_windows_openvino` | +| `rocm-linux-x86-64` | `GGML_HIP=ON -DCMAKE_HIP_COMPILER=… -DGPU_TARGETS=…` | `ubuntu-latest` + ROCm 10 TheRock wheels (pip, `rocm-sdk path`) | `Linux/x86_64/rocm` | +| `rocm-windows-x86-64` | `GGML_HIP=ON` | `windows-2022` + ROCm 10 TheRock wheels (pip) | `Windows/x86_64/rocm` | +| `sycl-fp16-linux-x86-64` | `GGML_SYCL=ON -DGGML_SYCL_F16=ON` (`icx`/`icpx`) | `ubuntu-latest` + Intel oneAPI apt | `Linux/x86_64/sycl-fp16` | +| `sycl-fp32-linux-x86-64` | `GGML_SYCL=ON` (`icx`/`icpx`) | `ubuntu-latest` + Intel oneAPI apt | `Linux/x86_64/sycl-fp32` | +| `sycl-windows-x86-64` | `GGML_SYCL=ON` (`icx`) | `windows-2025-vs2026` + oneAPI installer | `Windows/x86_64/sycl` | +| `opencl-windows-aarch64` | `GGML_OPENCL=ON …ADRENO_KERNELS=ON` (clang-cl, `GGML_OPENMP=OFF`) | `windows-11-arm` (arm64 CPU job's toolchain) | `Windows/aarch64/opencl` | +| `openvino-linux-x86-64` | `GGML_OPENVINO=ON` | `ubuntu-latest` + OpenVINO apt | `Linux/x86_64/openvino` | +| `openvino-windows-x86-64` | `GGML_OPENVINO=ON` | `windows-2025-vs2026` + OpenVINO archive | `Windows/x86_64/openvino` | **ROCm comes from TheRock, and the version and GPU targets follow upstream.** Since ROCm 7.14 AMD builds and releases ROCm through [TheRock](https://github.com/ROCm/TheRock); both ROCm jobs install @@ -373,11 +382,9 @@ any uncompressed bundle (`__CLANG_OFFLOAD_BUNDLE__`) or on none compressed — a change that drops the flag reds the job instead of quietly shipping the 1 GB library again. The jar barely shrinks (zip already compressed the code); what shrinks is the extracted library. -Two routing notes mirror existing precedent: **Linux SYCL** ships two precision variants at the *same* -arch, so `CMakeLists.txt` routes them to two *distinct* trees by `GGML_SYCL_F16` (fp16 vs fp32). -**Windows OpenCL** now holds both `x86_64` (desktop ICD) and `aarch64` (Snapdragon/Adreno) in the one -`resources_windows_opencl` tree, split by the `opencl-windows` / `opencl-windows-aarch64` profiles' -arch-scoped `` — exactly like the `vulkan-linux` / `vulkan-linux-aarch64` split. +One routing note: **Linux SYCL** ships two precision variants at the *same* arch, so +`CMakeLists.txt` names them two *distinct* backends by `GGML_SYCL_F16` (`sycl-fp16` vs `sycl-fp32`); +the loader tries fp16 first. The Linux jobs that install a multi-GB vendor toolchain (CUDA, ROCm, both SYCL) start with `ggml-org/free-disk-space` — the same guard upstream llama.cpp's CUDA/ROCm jobs use (ROCm also clears the @@ -386,53 +393,44 @@ tool cache, as upstream does, which is safe only because the step runs before `s The vendor toolchain install steps in `publish.yml` are **first-pass** (apt repos / vendor installers pinned to a specific version): if a URL/version 404s in CI, the job fails loud and the step is adjusted — the failure is intentional signal, not a regression to hide behind `continue-on-error`. -`src/main/resources_{linux_rocm,windows_rocm,linux_sycl_fp16,linux_sycl_fp32,windows_sycl,linux_openvino,windows_openvino}/` -are all git-ignored (staged by CI, never committed). ## macOS arm64: three build jobs, one shipped dylib -macOS arm64 is the **only** `{OS}/{ARCH}` built by more than one job, and it has **no classifier** — -all three variants ship (or don't) into the same default-JAR path `Mac/aarch64/libjllama.dylib`: +macOS arm64 is the **only** platform built by more than one job; one of them ships, as the natives +jar `metal-macos-aarch64` (directory `Mac/aarch64/metal/`, in `llama-platform`): | Job | Build flags | Artifact | Role | |---|---|---|---| -| `build-macos-arm64-metal-15` (macos-15) | `-DLLAMA_METAL_EMBED_LIBRARY=ON -DGGML_NATIVE=OFF` | `macos-15-metal` | **shipped** in the default JAR | +| `build-macos-arm64-metal-15` (macos-15) | `-DLLAMA_METAL_EMBED_LIBRARY=ON -DGGML_NATIVE=OFF` | `natives-metal-macos-aarch64` | **shipped** | | `build-macos-arm64-metal` (macos-14) | `-DLLAMA_METAL_EMBED_LIBRARY=ON` (host-native) | `macos-14-metal` | test-only | -| `build-macos-arm64-no-metal` (macos-15) | `-DLLAMA_METAL=OFF -DGGML_NATIVE=OFF` | `macos-15-no-metal` | test-only | - -None of the three passes `-DOS_NAME`/`-DOS_ARCH`, so `CMakeLists.txt` auto-detects the **same** -output subdirectory in all three. The shipped variant is `macos-15-metal` because it is the only one -with **both** Metal **and** `GGML_NATIVE=OFF` (portable across Apple-silicon generations); the other -two exist to prove the no-Metal path and the macos-14 SDK still build and pass the Java suite. - -**The invariant: at most one `*-libraries` artifact per `{OS}/{ARCH}`.** The `package`, -`publish-snapshot` and `publish-release` jobs collect the default JAR's natives with one globbed -`actions/download-artifact` (`pattern: "*-libraries"`). An artifact *name* says nothing about which -subdirectory the job actually wrote, so two artifacts sharing a relative path get extracted onto one -file — and the survivor can be a **byte-level hybrid** of both, not either input. All three macOS -jobs used to upload under a `*-libraries` name: the published dylib's ad-hoc linker signature then no -longer matched its own `__TEXT` pages and macOS **SIGKILLed every process that loaded it** (shipped -broken in 5.0.6 and several 5.0.7 snapshots; 66/4078 and 1141/4097 code pages failed their stored -hashes). Windows already avoided this by naming its MSVC variants outside the glob; macOS now does -the same, and the shipped variant is chosen by an **explicit download step by name**, never by which -artifact name happens to match a glob. - -**The guard: `.github/merge-native-artifacts.sh`.** The three consumer jobs download the glob +| `build-macos-arm64-no-metal` (macos-15) | `-DLLAMA_METAL=OFF -DGGML_NATIVE=OFF` | `macos-15-no-metal` | test-only (writes `Mac/aarch64/cpu/`) | + +The shipped variant is the metal-15 build because it is the only one with **both** Metal **and** +`GGML_NATIVE=OFF` (portable across Apple-silicon generations); the other two exist to prove the +no-Metal path and the macos-14 SDK still build and pass the Java suite. The two Metal builds write +the **same** directory, which is why the test-only artifacts are named outside the `natives-*` glob. + +**Why that naming matters (the incident).** Two artifacts sharing a relative path, downloaded with +one glob and merged, get extracted onto one file — and the survivor can be a **byte-level hybrid** of +both, not either input. All three macOS jobs used to upload under one glob: the published dylib's +ad-hoc linker signature then no longer matched its own `__TEXT` pages and macOS **SIGKILLed every +process that loaded it** (shipped broken in 5.0.6 and several 5.0.7 snapshots; 66/4078 and 1141/4097 +code pages failed their stored hashes). + +**The guard: `.github/merge-native-artifacts.sh`.** The three consumer jobs download `natives-*` **unmerged** (`merge-multiple` off → one subdirectory per artifact) and let that script do the merge. -It fails the job when any relative path is claimed by more than one `*-libraries` artifact, and when -the glob matched nothing at all (a silently native-library-free default JAR). Note **why the check -runs before the merge**: a collision still leaves exactly one file on the path, so a post-merge -assertion like "exactly one dylib per `{OS}/{ARCH}`" cannot see it — the collision is only observable -while the artifacts are still separate. A future job that reopens the hole (a second job writing -`Mac/aarch64`, or a new platform whose auto-detected subdir clashes) reds the pipeline instead of -shipping a corrupt binary. +Each artifact must hold exactly the directory its row in `natives.csv` names, and no relative path may +be claimed twice. Note **why the check runs before the merge**: a collision still leaves exactly one +file on the path, so a post-merge assertion cannot see it — it is only observable while the artifacts +are still separate. A future job that reopens the hole reds the pipeline instead of shipping a +corrupt binary. **The end-to-end gate: `smoke-fatjar-macos`.** The three macOS Java test jobs each test the dylib *their own job* built, so until now nothing exercised the **packaged** artifact on macOS — Linux and Windows had `smoke-fatjar-*` downstream of `package`, macOS had none, which is why this bug reached three releases with a fully green pipeline. The job (`needs: [package]`, `macos-15`, gates both publish jobs) downloads `llama-jars` and runs `.github/smoke-native-macos.sh`, which extracts -`Mac/aarch64/libjllama.dylib` from the **default fat jar** and asserts two things: `codesign --verify +`Mac/aarch64/metal/libjllama.dylib` from the **default fat jar** and asserts two things: `codesign --verify --strict` (re-hashes the code pages against the signature's stored hashes — the direct check for a dylib assembled from two builds) and a real JVM load via `.github/smoke/NativeLoadSmoke.java` (`java -cp …`, the JDK single-file source launcher), which forces @@ -440,9 +438,9 @@ dylib assembled from two builds) and a real JVM load via `.github/smoke/NativeLo `getLlamaCppBuildInfo()`, checked against the `LlamaCppVersion` pin. Two macOS specifics: it targets the **default** fat jar because there is no `all-macos-*` fat jar to -target (macOS has no GPU classifier — Metal is in the default jar — so `package-fatjars` builds no -macOS variant), and it asserts native loadability rather than a CLI exit code, so it cannot reuse -`smoke-test-fatjar.sh` (whose backend-manifest grep never matches the manifest-less default jar). +target (macOS has no GPU backend besides Metal, which the default fat jar carries, so +`package-fatjars` builds no macOS variant), and it asserts native loadability plus the code +signature rather than a server round trip, so it does not reuse `smoke-test-fatjar.sh`. Deliberately model-free (~1 min, no GGUF/cache restore/network): a full model-backed macOS server smoke would be strictly more, but this catches the failure class that actually shipped and is cheap enough to always run. It is the macOS member of the cross-repo convention in @@ -461,36 +459,47 @@ and are untouched. (The cross-repo "fat jar → GitHub Release, never Central, s convention shared with BAF and srcmorph is documented in [`../workspace/policies/fat-jar-release-assets.md`](../workspace/policies/fat-jar-release-assets.md).) -Mechanism (three pieces): +Mechanism: 1. **`.github/package-fatjars.sh`** — run by the `package-fatjars` job (`needs: [package]`, - downloads `llama-jars`). Enumerates the `` set from `llama/pom.xml` (source of - truth) and cross-checks it in **both** directions against the built classifier jars; parses - each classifier as `--`; **fails loud** on unparseable classifiers, - backends missing from its priority table, missing native trees, or zip-update corruption - (entry list, `Main-Class`, sample byte-compare). Excluded by design: `msvc-windows` - (redundant CPU variant) and `opencl-android-aarch64` (no `java -jar` on Android). For each - OS/arch it copies the default fat jar and adds every backend's native tree under - `net/ladenthin/llama////` plus a **`jllama-backends.txt`** manifest - (backends in priority order `cuda13 rocm sycl-fp16 sycl-fp32 sycl vulkan opencl openvino`; - extra tokens per line list sibling files such as openvino-windows' bundled `OpenCL.dll`). - **A new classifier fails this script until it is consciously ranked/excluded** — that is the - no-silent-gaps guarantee. -2. **`LlamaLoader` backend selection** — when (and only when) the manifest resource exists, - the loader tries each backend subdirectory in order: extract into a per-backend temp subdir - (`jllama-backend-/`; backends share file names), load manifest extras first, then the - backend's `jllama` library. A load failure (missing vendor runtime → `UnsatisfiedLinkError`) - moves to the next backend; after the list it falls back to the default CPU natives. System - property `net.ladenthin.llama.backend` forces one backend (fail-loud) or `default`/`cpu`. - Jars without a manifest take the unchanged legacy path. A backend whose extra module is - already resident from a previously failed attempt is skipped (by-name import cross-wiring). + downloads `llama-jars`). Takes the natives jars from `.github/natives.csv` and checks them in + **both** directions against the built jars; every natives jar must hold exactly its own + directory, its library, and its `Automatic-Module-Name`. Because the directories never overlap, + an all-backends jar is a **plain merge**: for each OS/arch that has a GPU backend (Android and the + `msvc` backend excluded — no `java -jar` on Android, and `msvc` would only be a second CPU library), + copy the default fat jar, **remove the native trees of every other platform**, and unzip every + natives jar of that OS/arch into it. No manifest: the loader's fixed priority order does the rest. + + **Only the jar's own OS+arch tree is kept.** Originally each combined jar was the default fat jar + plus backends, i.e. it carried the CPU natives of all nine platforms — ~71.5 MB of dead weight in + `all-windows-x86-64` (341.5 MB → ~270 MB, computed from the per-tree sizes of run 36606543343). + That was how the jar was built, not a requirement: the introducing commit (`aff68f5e`) gives no + reason. The rule is deliberately the strict one, so `all-windows-x86-64` also loses `Windows/x86` + and `Windows/aarch64` — a 32-bit JVM on 64-bit Windows needs the default fat jar, which still runs + everywhere. The "natives missing from a jar" incidents in the history are a different layer — the + default jar's collection (`package` once lacked `needs:` on three build jobs, `7ad8066a`), which the + merge step's completeness check against `natives.csv` now covers. + Trees are found as the **upper-case directories** below `net/ladenthin/llama/` (OSInfo folder + names; Java packages are lower-case) and matched by exact path component — a `Linux*` prefix + would also hit `Linux-Android`, and a pattern without the trailing `/` of a directory matches + upper-case class files such as `LlamaModel.class`. Fail-loud checks after the zip update: every + added backend byte-identical to its natives jar, the own CPU library byte-identical to the default + jar's, **no other `/` tree left**, the `.class` count equal to the default jar's (the + check that caught the missing-slash variant during development), `Main-Class` intact, and all + four release targets produced. +2. **`LlamaLoader` backend selection** — the same code path as for any classpath: every backend + directory present is tried in `BACKEND_PRIORITY` order, each extracted into its own temp subdir + (`jllama-backend-/`; backends share file names), its `jllama-extras.txt` files loaded first. + A load failure (missing vendor runtime → `UnsatisfiedLinkError`) moves to the next backend, ending + at `cpu`. `net.ladenthin.llama.backend` forces one backend (fail-loud). A backend whose extra module + is already resident from a previously failed attempt is skipped (by-name import cross-wiring). 3. **`publish.yml` wiring** — **all four** OS/arch fat jars are launched, one smoke job each: `smoke-fatjar-linux` / `smoke-fatjar-windows` (x86-64) plus `smoke-fatjar-linux-aarch64` (`ubuntu-24.04-arm`) / `smoke-fatjar-windows-arm64` (`windows-11-arm`). Each runs its jar via real `java -jar` on a GPU-less runner (cached draft model, `--chat-template chatml`): poll `/health` to 200, assert a `/v1/chat/completions` choice, and require the loader's - backend-selection log line — so every manifest backend failing its load and falling back to the - CPU natives is exercised on the actual release asset. The four jobs consume four small + backend-selection log line — so every GPU backend failing its load and falling back to the + CPU backend is exercised on the actual release asset. The four jobs consume four small single-jar artifacts (`llama-fatjar-smoke-{linux,windows,linux-aarch64,windows-arm64}`) rather than the multi-GB `llama-fatjars` set. **The two aarch64 jobs close a real gap**: those jars were built, GPG-signed and attached to every release while `publish.yml` referenced them zero times, @@ -922,8 +931,7 @@ byte-identical body). No local patch needed — the tree already matches what `0 ## RPC backend: `--rpc` client and the in-JVM `RpcServer` -llama.cpp's RPC backend (`ggml-rpc`) is compiled into **every** artifact — the default JAR and all -GPU classifiers — with `GGML_RPC=ON` forced in `llama/CMakeLists.txt`. It stays **one** `jllama` +llama.cpp's RPC backend (`ggml-rpc`) is compiled into **every** natives jar — CPU and GPU — with `GGML_RPC=ON` forced in `llama/CMakeLists.txt`. It stays **one** `jllama` library: `ggml_add_backend_library` makes `ggml-rpc` a static library linked into `ggml` (only `GGML_BACKEND_DL`, which needs shared libs, would split it out), and its registration is compiled in (`GGML_USE_RPC`). Client and server live in the same file, so the flag brings both. @@ -937,9 +945,10 @@ statically in a useful way either (it `dlopen`s its hardware providers), and ups comment says the Apple weak link does not survive a static `ggml-rpc`. **`.github/verify-native-deps.py`** (the `package` job) reads each shipped library's dependency list straight from the file — ELF `DT_NEEDED`, PE import table including Windows arm64 (which binutils cannot read), Mach-O load -commands — and holds the default tree to an exact per-`{OS}/{ARCH}` allowlist (the classifier trees -only to a denylist, since they need their vendor runtime). A new `{OS}/{ARCH}` without an allowlist -fails too. **It found a pre-existing defect on its first run**: the macOS dylib links Homebrew's +commands — and holds every CPU directory (`cpu`, `metal`, `msvc`) to an exact per-directory +allowlist (the GPU directories only to a denylist, since they need their vendor runtime). A new CPU +directory without an allowlist fails too. (That every build arrived is the merge step's check against +`natives.csv`.) **It found a pre-existing defect on its first run**: the macOS dylib links Homebrew's `openssl@3` (`/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` + `libcrypto`), so it does not load on a Mac without that formula. Those two paths are in the allowlist marked as a known defect so the check reports only *new* dependencies; the fix is on file in `TODO.md`. @@ -1001,7 +1010,7 @@ loading thread — the lock is reentrant, so a second complete load ran while th backend again over the library being loaded. The fat-jar RPC smoke timed out on it (the doubled extraction of the CUDA/ROCm/SYCL libraries); it now waits 300 s like the other smoke and fails if the backend is selected more than once. Only the *nested* call is skipped — later calls still run the body, -which `BackendManifestLoadTest` relies on. +which `BackendLoadTest` relies on. **Single instance per process** (ggml keeps the server state in globals). `startLocal` binds loopback only; `startOnNetwork` is the explicit, warned opt-in, and binding needs an IPv4 literal (the server uses `inet_addr`). Endpoints on the client side are `value.RpcEndpoint` @@ -1234,15 +1243,17 @@ cmake --build build --config Release cmake -B build -DLLAMA_CURL=ON ``` -Built libraries are placed in `src/main/resources/net/ladenthin/llama/{OS}/{ARCH}/`. +Built libraries are placed in `src/main/natives/net/ladenthin/llama/{OS}/{ARCH}/{backend}/` +(`cpu` for a plain build; see "Natives jars"). ### Building the native library for local Java tests `mvn test` does **not** build the native library — Maven only compiles Java and runs surefire. The shared library must already exist on disk under the -platform-specific resource path that `LlamaLoader` resolves at runtime. -Without it the JVM throws `UnsatisfiedLinkError` and every Java test fails -immediately (it does not auto-skip). +platform-specific path that `LlamaLoader` resolves at runtime; surefire puts +`src/main/natives/` on the test classpath (`additionalClasspathElements`). +Without it the native-backed tests skip themselves or fail with +`UnsatisfiedLinkError`. The output path is derived by `CMakeLists.txt` from `OS_NAME` and `OS_ARCH` detected by the helper script `.github/dockcross/dockcross-resolve-host` @@ -1250,20 +1261,20 @@ detected by the helper script `.github/dockcross/dockcross-resolve-host` mirrors `OSInfo.translateOSNameToFolderName` on the Java side, so the same folder name is produced on both ends. -| Host | Library file | Resource path produced by `cmake --build` | +| Host | Library file | Path produced by `cmake --build` | |------|--------------|-------------------------------------------| -| Linux x86_64 | `libjllama.so` | `src/main/resources/net/ladenthin/llama/Linux/x86_64/` | -| Linux aarch64 | `libjllama.so` | `src/main/resources/net/ladenthin/llama/Linux/aarch64/` | -| macOS Apple Silicon | `libjllama.dylib` | `src/main/resources/net/ladenthin/llama/Mac/aarch64/` | -| macOS Intel | `libjllama.dylib` | `src/main/resources/net/ladenthin/llama/Mac/x86_64/` | -| Windows x86_64 | `jllama.dll` | `src/main/resources/net/ladenthin/llama/Windows/x86_64/` | +| Linux x86_64 | `libjllama.so` | `src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu/` | +| Linux aarch64 | `libjllama.so` | `src/main/natives/net/ladenthin/llama/Linux/aarch64/cpu/` | +| macOS Apple Silicon | `libjllama.dylib` | `src/main/natives/net/ladenthin/llama/Mac/aarch64/metal/` | +| Windows x86_64 | `jllama.dll` | `src/main/natives/net/ladenthin/llama/Windows/x86_64/cpu/` (Ninja) or `…/msvc/` (Visual Studio generator) | On every platform exactly **one** `jllama` library is produced: `CMakeLists.txt` forces `BUILD_SHARED_LIBS OFF`, so upstream `llama` and `ggml` are static libraries linked into `jllama` (the `RUNTIME_OUTPUT_DIRECTORY_*` block that also names the `llama`/`ggml` targets is a no-op for them — verified against the published 5.0.5 jars, which contain only `jllama.dll` per Windows arch). `LlamaLoader` accordingly extracts and loads a single file -from the jar (plus `ggml-metal.metal` on macOS). Historical note: upstream kherud once +per backend (plus the files in its `jllama-extras.txt`, and `ggml-metal.metal` from a Metal build +that does not embed it). Historical note: upstream kherud once shipped split `ggml` + `jllama` libraries, which is where stale "three co-located DLLs" claims came from. @@ -1276,8 +1287,8 @@ mvn -q compile # 2. Configure + build the native library for the current host cmake -B build cmake --build build --config Release -j$(nproc) -# The shared lib lands directly in src/main/resources/.../{OS}/{ARCH}/ — -# no separate install step is needed. +# The shared lib lands directly in src/main/natives/.../{OS}/{ARCH}/{backend}/ — +# no separate install step is needed; surefire puts that tree on the test classpath. # 3. Ensure model files referenced by tests are present under models/. # The default test models (downloaded by CI in publish.yml) are: @@ -1363,7 +1374,7 @@ tests cannot do this in a restricted sandbox — they self-skip via # which is reachable even when huggingface.co is not): mvn -q compile cmake -B build -DBUILD_TESTING=ON -cmake --build build --config Release -j$(nproc) # -> src/main/resources/...///libjllama.so +cmake --build build --config Release -j$(nproc) # -> src/main/natives/...///cpu/libjllama.so # 2. Force LlamaModel. (System.load -> JNI_OnLoad) with no model: mvn test -Dtest=NativeLibraryLoadSmokeTest ``` @@ -1388,8 +1399,8 @@ to a local `mvn test` (model tests skipped) or to the pure-Java unit tests. [root], `exception.LlamaException`, `value.LogLevel`, `args.LogFormat`, `callback.LoadProgressCallback`), update the matching `FindClass` / `"L…;"` signature string in `src/main/cpp/jllama.cpp` and keep the native-resource root -anchored at `net/ladenthin/llama/` in `LlamaLoader.NATIVE_RESOURCE_BASE` (it must -not track the loader's own Java package). This is the same +anchored at `net/ladenthin/llama` in `LlamaLoader.NATIVE_RESOURCE_BASE` (it must +not track the loader's own Java package; no leading slash, it is resolved through the ClassLoader). This is the same "FQN/path not updated after a package move" class as the stale `spotbugs-exclude.xml`, PIT `targetClasses`, and `CMakeLists.txt` OSInfo repairs. @@ -1639,8 +1650,9 @@ Java parameters are serialized to JSON strings and passed to native code, which ### Native Library Resolution `LlamaLoader` tries in order: 1. System property `net.ladenthin.llama.lib.path` -2. `java.library.path` -3. Extracts from JAR resources at `net/ladenthin/llama/{os}/{arch}/` +2. `System.loadLibrary` on Android (the AAR's `jni/`), then `java.library.path` +3. The natives jars on the classpath: every `net/ladenthin/llama/{os}/{arch}/{backend}/` present, in + `BACKEND_PRIORITY` order, extracted and loaded until one loads (see "Natives jars") ### Cross-compilation Docker-based cross-compilation scripts are in `.github/dockcross/` for **Android** targets (and the @@ -1675,7 +1687,7 @@ Wiring (mirrors the macOS native jobs, not the dockcross jobs): ### Linux s390x: big-endian cross-build + qemu test gate -`build-linux-s390x` extends the default JAR to **IBM Z (s390x, big-endian)** — the one target whose +`build-linux-s390x` adds **IBM Z (s390x, big-endian)** — the one target whose byte order differs from every other platform. It **cross-compiles** with the GCC s390x toolchain (`g++-s390x-linux-gnu`, native x86 speed — no emulated build) and then runs the **full C++ unit suite under `qemu-user`** (`CMAKE_CROSSCOMPILING_EMULATOR=/usr/bin/qemu-s390x-static`, `QEMU_LD_PREFIX=/usr/s390x-linux-gnu`). @@ -1685,8 +1697,8 @@ which is where an endian bug in *our* code could hide. Model-backed **Java** tes **not** run under emulation (a JVM + GGUF inference under `qemu-user` is slow and flaky); the Java↔JNI boundary uses host-native array copies (endian-transparent), so the C++ gate covers the actual risk. `-DGGML_OPENMP=OFF` sidesteps cross-libgomp issues (ggml uses its own `std::thread` pool). s390x is a -CPU platform like aarch64, so it ships in the **default** JAR (`Linux-s390x-libraries` merges via the -`*-libraries` glob; `OSInfo` maps `os.arch=s390x` → `Linux/s390x`) — no classifier, no pom profile. +CPU platform like aarch64, so it ships as the natives jar `cpu-linux-s390x`, part of `llama-platform` +(`OSInfo` maps `os.arch=s390x` → `Linux/s390x`). **Fail-loud** and in `package.needs` like every other build. (Upstream llama.cpp already supports s390x — it ships `ubuntu-s390x` with GGUF big-endian handling — so the native inference path is upstream's concern; this job validates only *our* layer's endian-safety.) @@ -1983,7 +1995,7 @@ EXPECT_FALSE(j.contains("stop_type")); // filtered out - **Java 8+** runtime required. Built with JDK 21 targeting bytecode 1.8 for broad compatibility. - Native memory allocated by llama.cpp is not GC-managed — always use `LlamaModel` in try-with-resources or call `close()` explicitly. - The `server.hpp` file is adapted from llama.cpp upstream — minimize modifications to ease future upgrades. -- Platform-specific native libraries must be pre-built and placed under `src/main/resources/` before packaging for distribution. +- Platform-specific native libraries must be pre-built and placed under `src/main/natives/` before packaging for distribution (CI does this; see "Natives jars"). ## Javadoc Conventions @@ -2025,14 +2037,14 @@ exceeds `--max-major`: ``` Paths may be jars or directories (searched recursively for `*.jar`), so one invocation covers a whole -artifact set — here all 16 classifier jars plus every `all--` fat jar. `module-info.class` +artifact set — here the classes jar, all 26 natives jars and every `all--` fat jar. `module-info.class` and `META-INF/versions/**` are skipped unconditionally: a classpath JVM never loads either, which is why a `release 9` `module-info` is fine. `--allow` is a repeatable glob matched against `:` for anything else that must be tolerated. Exit codes: 0 clean, 1 violations, **2 nothing to scan** (an empty input is a failure, never a pass — the first version of this check reported a clean pass over a directory a failed build had left empty). -It runs twice: in the `package` job over `llama/target` (every classifier jar plus the default fat +It runs twice: in the `package` job over `llama/target` (every jar plus the default fat jar, as early as they exist), and again in `smoke-fatjar-linux` over the downloaded `fatjars/` — `package-fatjars` rewrites those zips, and they are the artifacts users actually download. @@ -2126,10 +2138,10 @@ keeping it clear of the JPMS module-mode javadoc trap that bit BAF. **Before rai javadoc source level to ≥ 9, read** [`../workspace/policies/jpms-module-descriptor.md`](../workspace/policies/jpms-module-descriptor.md). -## Repository layout — Maven reactor (`llama/` + `llama-langchain4j/` + `llama-kotlin/`) + the `llama-android/` Gradle build +## Repository layout — Maven reactor (`llama/` + `llama-langchain4j/` + `llama-kotlin/` + `llama-platform/`) + the `llama-android/` Gradle build The repo root is a thin **aggregator/parent POM** (`net.ladenthin:llama-parent`, -`packaging=pom`) with three modules: +`packaging=pom`) with four modules: - **`llama/`** — the native JNI core (`net.ladenthin:llama`). *All the core sources and build files live here now:* `llama/src/`, `llama/CMakeLists.txt`, `llama/cmake/`, `llama/patches/`, @@ -2137,12 +2149,14 @@ The repo root is a thin **aggregator/parent POM** (`net.ladenthin:llama-parent`, Its published coordinates are unchanged (`net.ladenthin:llama`), so consumers are unaffected. - **`llama-langchain4j/`** — the LangChain4j adapters (see below). - **`llama-kotlin/`** — the Kotlin coroutines façade (see "Android AAR + Kotlin façade" below). +- **`llama-platform/`** — a pom-packaging dependency list: the classes jar plus the CPU natives jars + of every desktop platform (see "Natives jars" above). All modules inherit the single `` from the parent, so they **ship in lockstep by construction** (no CI guard needed). The parent also holds the shared `release` profile (GPG + -Central Publishing), so one reactor `mvn -P release deploy` signs and publishes all four -Maven artifacts (`llama-parent` pom, `llama`, `llama-langchain4j`, `llama-kotlin`) at the same -version. +Central Publishing), so one reactor `mvn -P release,natives deploy` signs and publishes all five +Maven artifacts (`llama-parent` pom, `llama` with its natives jars, `llama-langchain4j`, +`llama-kotlin`, `llama-platform` pom) at the same version. **`llama-android/` is deliberately NOT a reactor module** but a standalone plain-Gradle build (no AGP, no Android SDK needed to build): Maven cannot produce or deploy an artifact with @@ -2164,12 +2178,13 @@ file shows `cmake -B build` / `src/main/...` / `mvn compile` at the root, read i **Version bump:** the child modules declare **no `` of their own** — their *project* version is inherited from the parent. But each child still hardcodes the parent version inside its `` pointer (Maven requires a literal there — there is **no `${revision}`/CI-friendly -versioning** here), so a version change must be applied to **all four poms in lockstep**: +versioning** here), so a version change must be applied to **all five poms in lockstep**: - `pom.xml` (root) — `` - `llama/pom.xml` — `` - `llama-langchain4j/pom.xml` — `` - `llama-kotlin/pom.xml` — `` +- `llama-platform/pom.xml` — `` (`llama-android/` needs **no** edit — its Gradle build reads the root pom's version at configure time.) @@ -2237,13 +2252,13 @@ Wiring: jar so a release-time javadoc break is caught in PR CI). The `publish-snapshot`/ `publish-release` jobs `needs:` this job; deployment is a **single reactor** `mvn -P release deploy` (no separate module deploy step — the parent's inherited `release` - profile signs and publishes parent + llama + llama-langchain4j together at the same version). + profile signs and publishes every reactor module together at the same version). A separate **`test-java-llama-langchain4j-integration`** job runs the model-backed tests (chat/streaming/embedding/scoring adapters) by **reusing** the shared GGUF cache - (`gguf-models-v1`, restore-only — no extra download) and the `Linux-x86_64-libraries` native + (`gguf-models-v1`, restore-only — no extra download) and the `natives-cpu-linux-x86-64` native artifact: it `needs: [crosscompile-linux-x86_64, download-models]` (so the cache is already - populated and it runs in parallel), installs parent+core with the downloaded native lib - bundled, and passes the already-cached chat (`REASONING_MODEL_NAME`), nomic-embedding and + populated and it runs in parallel), installs parent+core (classes), points the test JVM at the + downloaded library with `-Dnet.ladenthin.llama.lib.path`, and passes the already-cached chat (`REASONING_MODEL_NAME`), nomic-embedding and jina-reranker model paths via the module's `-Dnet.ladenthin.llama.langchain4j.{embedding,rerank}.model` / `net.ladenthin.llama.model.path` properties. It is validation-only (not a release gate); a cold cache degrades to a self-skip. @@ -2269,14 +2284,14 @@ the recommended path (README "Importing in Android", Option 1): - **`net.ladenthin:llama-android`** / **`llama-android-opencl`** — AARs (`aar`) carrying the core classes + the CI-built `libjllama.so` natives under `jni/` — the CPU AAR is **multi-ABI** (`arm64-v8a` devices + `x86_64` emulators/Chromebooks, built by the - `crosscompile-android-x86_64` dockcross job whose artifact also merges into the default JAR's - `Linux-Android/x86_64` tree via the `*-libraries` glob; the OpenCL flavor stays arm64-only — + `crosscompile-android-x86_64` dockcross job, which also ships as the `cpu-android-x86-64` natives + jar; the OpenCL flavor stays arm64-only — Adreno is Qualcomm ARM hardware), a `minSdkVersion 28` manifest (AGP enforces the floor on consumers), and consumer R8/ProGuard rules (`consumer-proguard.txt` → `proguard.txt` in the AAR; keeps `net.ladenthin.llama.**` for the JNI `FindClass`/Jackson reflection surface). The AAR's `classes.jar` is the - **byte-identical Maven-built core jar** minus the desktop/Android native resource trees - (~70 MB APK bloat otherwise) and `module-info.class` (D8 rejects it); on Android `LlamaLoader` + **byte-identical Maven-built classes jar** minus `module-info.class` (D8 rejects it) — it carries no + natives, which ship as separate natives jars; on Android `LlamaLoader` resolves via `System.loadLibrary("jllama")`, which finds the AAR-installed `.so` — no loader change was needed. Built by the **standalone plain-Gradle build** in `llama-android/` (see "Repository layout" for why it is not a Maven module); the POM mirrors the core's @@ -2336,7 +2351,10 @@ needs Java 21 (Atmosphere's floor) while the core stays Java 8. CI builds it aga installed (`-Dllama.version=`); a user copies the folder and runs `mvn compile exec:java -Dexec.args="…"` with no `-D` at all — the pom's `llama.version` names the **released** core the READMEs describe (currently `5.2.0`, written as if released so the docs are -right the moment the release lands). +right the moment the release lands). The natives come from the +`natives` profile (`llama-platform`, active unless `-Dllama.natives=none`) plus the `gpu-natives` +profile (`-Dllama.classifier=`); CI passes `-Dllama.natives=none`, since it installs only +the classes and the model-backed job uses `-Dnet.ladenthin.llama.lib.path`. **Release asset: the agent jar WITHOUT the core.** `mvn -P assembly package` (the pom's `assembly` profile, descriptor `src/assembly/agent-jar.xml`) builds diff --git a/README.md b/README.md index ba1081a16..f37d54a3c 100644 --- a/README.md +++ b/README.md @@ -121,20 +121,30 @@ Inference of Meta's LLaMA model (and others) in pure C/C++. - **Distributed inference over RPC** — offload a model's layers to llama.cpp RPC servers on other machines (`ModelParameters.setRpcServers(...)` / `--rpc host:port`), and serve this machine's devices to them with `RpcServer` (the in-JVM `rpc-server`). See [Distributed inference over RPC](#distributed-inference-over-rpc). - **Local agent** ([`llama-atmosphere-agent`](llama-atmosphere-agent/), release asset, JDK 21+) — a fully offline agent on top of this library that reads and edits files and, if allowed, runs commands. One agent session, three ways to use it: a **terminal** (full console or line-oriented `--plain`, fine over SSH/PuTTY), a **browser** (`--web`, token-protected, loopback by default — reach it from elsewhere through an SSH tunnel), and **IDEs** over the [Agent Client Protocol](https://agentclientprotocol.com) (`--acp`: JetBrains IDEs and Zed natively, VS Code through an ACP extension). See [Local agent](#local-agent-terminal-browser-ide-via-acp). - **Multi-model router mode** (`--models-dir` + per-request model selection, managed via the typed `RouterClient`) and **attach mode** (`NativeServer(LlamaModel, ...)` serves an already-loaded model over the full upstream HTTP frontend — one copy of the weights). -- Pre-built native binaries in the default JAR for Linux (x86-64, aarch64, s390x), macOS (x86-64, arm64 — Metal included), Windows (x86-64, x86, arm64) and Android (arm64, x86-64); GPU backends (CUDA, Vulkan, OpenCL, ROCm/HIP, SYCL, OpenVINO) ship as Maven classifiers — see [Choosing the right classifier](#choosing-the-right-classifier). Android additionally ships as the [`llama-android` AAR](#importing-in-android) with the optional `llama-kotlin` coroutines façade. +- Pre-built native binaries for Linux (x86-64, aarch64, s390x), macOS (arm64, Metal included), Windows (x86-64, x86, arm64) and Android (arm64, x86-64), plus GPU backends (CUDA, Vulkan, OpenCL, ROCm/HIP, SYCL, OpenVINO) — one natives jar each, all loadable side by side with automatic CPU fallback; see [Choosing the natives jars](#choosing-the-natives-jars). Android additionally ships as the [`llama-android` AAR](#importing-in-android) with the optional `llama-kotlin` coroutines façade. ## Quick Start -Access this library via Maven (released versions on Maven Central): +Access this library via Maven (released versions on Maven Central). `llama-platform` brings +the classes plus the CPU natives of every desktop platform: ```xml net.ladenthin - llama - 5.1.0 + llama-platform + 5.2.0 + pom ``` +(`pom` is required in Maven: `llama-platform` is a dependency list, not a jar. +In Gradle it is just `implementation("net.ladenthin:llama-platform:5.2.0")`.) + +> [!NOTE] +> This layout starts with **5.2.0**. Up to 5.1.0, `net.ladenthin:llama` was one jar that +> carried the CPU natives of every platform, and each GPU classifier was a complete +> replacement for it. + There are multiple [examples](llama/src/test/java/examples). ### Snapshot builds @@ -155,8 +165,9 @@ To use the latest snapshot, add the repository and dependency to your `pom.xml`: net.ladenthin - llama + llama-platform 5.2.0-SNAPSHOT + pom ``` @@ -166,93 +177,92 @@ No credentials are required — the repository is publicly readable. We support CPU inference for the following platforms out of the box: -- Linux x86-64, aarch64 -- MacOS x86-64, aarch64 (M-series) -- Windows x86-64, x64 +- Linux x86-64, aarch64, s390x +- macOS aarch64 (Apple silicon, with Metal) +- Windows x86-64, x86, aarch64 +- Android aarch64, x86-64 (see [Importing in Android](#importing-in-android)) If any of these match your platform, you can include the Maven dependency and get started. -### Choosing the right classifier +### Choosing the natives jars -The Maven coordinate `net.ladenthin:llama` publishes one default JAR (CPU-only; -its Windows natives are built with the Ninja Multi-Config + MSVC toolchain) plus -optional JARs selected via a Maven ``: NVIDIA CUDA (Linux / Windows), -Vulkan (Linux x86-64 / aarch64, Windows), AMD ROCm/HIP (Linux / Windows), Intel -SYCL (Linux fp16 / fp32, Windows) and OpenVINO (Linux / Windows) GPU builds, OpenCL -(Android Adreno, Windows x86-64 / Snapdragon-arm64), and an alternate-toolchain MSVC -Windows CPU build. Pick at most one GPU/accelerator classifier — those are mutually -exclusive — and optionally a CPU Windows build. +`net.ladenthin:llama` is the Java classes only. The native libraries ship as separate jars of +the same artifact, one per backend and platform, selected by a Maven `` of the form +`--`. Each holds exactly one directory, +`net/ladenthin/llama////`, so **any combination can share one classpath**. +At startup the loader tries the backends it finds in a fixed order — CUDA → ROCm → SYCL +(fp16, fp32) → Vulkan → OpenCL → OpenVINO → Metal → MSVC → CPU — and uses the first whose +library loads. A GPU jar next to the CPU jar therefore just works on a machine without that GPU +or its runtime: the GPU library fails to load and the CPU one is used. -| Classifier | Backend | Target platform | Runtime requirement | -|---|---|---|---| -| _(none)_ | CPU | Linux x86-64 / aarch64 / s390x, macOS x86-64 / aarch64, Windows x86-64 / x86 / aarch64 (Ninja Multi-Config + MSVC), Android aarch64 + x86-64 (CPU) | A JDK 8+ JVM. **Linux `aarch64` additionally requires glibc ≥ 2.39** (e.g. Ubuntu 24.04+, Debian 13+) — it is built natively on `ubuntu-24.04-arm`, matching upstream llama.cpp's own ARM binaries; older-glibc ARM hosts (Ubuntu 22.04, Debian 12, RHEL 8/9, Amazon Linux 2023) are not supported. Linux x86-64 keeps a glibc 2.17 floor (manylinux2014). **Windows `aarch64`** (Windows on ARM — Snapdragon X / Surface) is built natively on `windows-11-arm` and ships in the default JAR alongside the x86-64 / x86 natives. | -| `msvc-windows` | CPU (MSVC / Visual Studio generator) | Windows x86-64 and x86 | None beyond a JDK 8+ JVM. Same CPU backend as the default JAR's Windows natives, but compiled with the Visual Studio generator instead of `Ninja Multi-Config`. Both use the same MSVC toolchain (static `/MT` CRT), so they are functionally equivalent — provided as an alternate-toolchain option. | -| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit installed on the host (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` resolvable on `PATH`). The runtime libraries are **not bundled** in the JAR; native-library load fails with `UnsatisfiedLinkError` if they are absent. No CPU fallback. | -| `vulkan-windows-x86-64` | Vulkan | Windows x86-64 with a Vulkan 1.2+ GPU (NVIDIA / AMD / Intel) | A Vulkan runtime (`vulkan-1.dll`), which current GPU drivers install. No Vulkan SDK is needed at runtime. The most portable Windows GPU option (vendor-independent). | -| `opencl-windows-x86-64` | OpenCL | Windows x86-64 with an OpenCL 2.0+ GPU | A vendor OpenCL ICD (`OpenCL.dll`, installed by the GPU driver). **Note:** the GGML OpenCL backend is Adreno-tuned; on desktop GPUs CUDA or Vulkan are better supported. | -| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`) installed on the host. The shared library is dynamically linked against them and will fail to `dlopen` if they are absent — there is no automatic fallback to CPU. | -| `vulkan-linux-x86-64` | Vulkan | Linux x86-64 with a Vulkan 1.2+ GPU (NVIDIA / AMD / Intel) | A Vulkan runtime (`libvulkan.so.1`), which current GPU drivers install. No Vulkan SDK is needed at runtime. The most portable Linux GPU option (vendor-independent, no CUDA toolkit). Built natively on `ubuntu-latest`, so it shares the aarch64 build's higher glibc floor (≈ 2.39). | -| `vulkan-linux-aarch64` | Vulkan | Linux aarch64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`libvulkan.so.1`) from the device/driver. glibc ≥ 2.39 (built on `ubuntu-24.04-arm`). | -| `opencl-android-aarch64` | OpenCL (Adreno) | Android aarch64 with Qualcomm Adreno GPU | A device-supplied OpenCL ICD (`libOpenCL.so`). Devices without an ICD (e.g. most non-Snapdragon Android hardware) must use the default CPU JAR. | -| `rocm-linux-x86-64` | ROCm / HIP | Linux x86-64 with AMD GPU | An installed AMD ROCm **10** runtime (`libamdhip64.so`, `librocblas.so`, `libhipblas.so`) on the host — built against ROCm 10.0 (TheRock), the same version upstream llama.cpp ships; targets every GPU TheRock builds for Linux, Instinct included (gfx900/gfx906/gfx90c/gfx1153 best effort — built, but not release-ready in ROCm 10). Not bundled; native load fails without it. No CPU fallback. | -| `rocm-windows-x86-64` | ROCm / HIP | Windows x86-64 with AMD GPU | The AMD ROCm **10** runtime DLLs (`amdhip64.dll`, `rocblas.dll`, `hipblas.dll`) on `PATH` — built against ROCm 10.0 (TheRock); every Radeon target TheRock builds for Windows, gfx900 through RDNA4 (gfx900/gfx906/gfx90c/gfx1153 best effort). Not bundled. No CPU fallback. | -| `sycl-fp16-linux-x86-64` | SYCL (Intel oneAPI, fp16) | Linux x86-64 with Intel GPU (Arc / iGPU) | An installed Intel oneAPI / Level-Zero runtime. fp16 accumulation (faster, slightly lower precision). Not bundled. | -| `sycl-fp32-linux-x86-64` | SYCL (Intel oneAPI, fp32) | Linux x86-64 with Intel GPU (Arc / iGPU) | An installed Intel oneAPI / Level-Zero runtime. fp32 accumulation (higher precision). Not bundled. | -| `sycl-windows-x86-64` | SYCL (Intel oneAPI) | Windows x86-64 with Intel GPU (Arc / iGPU) | The Intel oneAPI / Level-Zero runtime DLLs on `PATH`. Not bundled. | -| `opencl-windows-aarch64` | OpenCL (Adreno) | Windows-on-ARM aarch64 (Snapdragon X) with Adreno GPU | A device-supplied OpenCL ICD (`OpenCL.dll`, from the Adreno driver). Not bundled. | -| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 (Intel GPU / NPU / CPU) | An installed Intel OpenVINO runtime. Not bundled. | -| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 (Intel GPU / NPU / CPU) | The Intel OpenVINO runtime DLLs on `PATH`. Not bundled. | - -> [!NOTE] -> The AMD (`rocm-*`), Intel SYCL (`sycl-*`), Windows-on-ARM OpenCL -> (`opencl-windows-aarch64`) and Intel OpenVINO (`openvino-*`) classifiers are -> newly added GPU backends. Like the other GPU classifiers they are validated -> **build-only** in CI (GitHub runners have no matching GPU), so end-to-end -> inference is verified locally / on self-hosted hardware. As with every GPU JAR, -> the vendor runtime is supplied by the consumer's driver/toolkit and is not bundled. - -For the default CPU JAR, omit the ``. For a GPU/accelerator or -alternate-CPU build, add the `` for your platform from the table -above — the backend, target platform and runtime requirement are all listed -there. Pick **at most one** classifier (they are mutually exclusive): +`llama-platform` is the classes jar plus the CPU jars of every desktop platform. For GPU +acceleration, add the jar for your GPU next to it: ```xml - net.ladenthin - llama - 5.1.0 + llama-platform + 5.2.0 + pom - - + net.ladenthin llama - 5.1.0 + 5.2.0 cuda13-linux-x86-64 ``` -> [!IMPORTANT] -> The GPU JARs are **GPU-only at runtime**. On a host without the matching -> GPU driver/runtime the JVM fails at native-library load time with -> `UnsatisfiedLinkError`: the CUDA JARs are dynamically linked against the -> CUDA runtime (`libcudart.so.13` on Linux, `cudart64_13.dll` / -> `cublas64_13.dll` / `cublasLt64_13.dll` on Windows — the Windows CUDA -> runtime is **not bundled**, install the CUDA 13 Toolkit), the Vulkan JAR -> needs a Vulkan runtime (`vulkan-1.dll`, shipped with current GPU drivers), -> and the OpenCL JARs need a vendor OpenCL ICD. There is no automatic -> fallback to CPU. If you want a single artifact that works on both CPU and -> GPU hosts, depend on the default (CPU) JAR; users who want GPU acceleration -> on an unlisted platform must compile locally with the matching `-DGGML_*=ON` -> flag (see [Setup required](#setup-required)). +To ship less, depend on `net.ladenthin:llama` (the classes) plus only the natives jars of the +platforms you target, e.g. `cpu-linux-x86-64`. + +| Classifier | Backend | Target platform | Runtime requirement | +|---|---|---|---| +| `cpu-linux-x86-64` | CPU | Linux x86-64 | A JDK 8+ JVM; glibc ≥ 2.17 (manylinux2014). | +| `cpu-linux-aarch64` | CPU | Linux aarch64 | glibc ≥ 2.39 (e.g. Ubuntu 24.04+, Debian 13+) — built natively on `ubuntu-24.04-arm`, matching upstream llama.cpp's own ARM binaries; older-glibc ARM hosts (Ubuntu 22.04, Debian 12, RHEL 8/9, Amazon Linux 2023) are not supported. | +| `cpu-linux-s390x` | CPU | Linux s390x (IBM Z, big-endian) | A JDK 8+ JVM. | +| `cpu-windows-x86-64` / `cpu-windows-x86` | CPU | Windows x86-64 / x86 | A JDK 8+ JVM. Built with Ninja Multi-Config + MSVC (static `/MT` CRT). | +| `cpu-windows-aarch64` | CPU | Windows on ARM (Snapdragon X / Surface) | A JDK 8+ JVM. Built natively on `windows-11-arm` with `clang-cl`. | +| `metal-macos-aarch64` | Metal + CPU | macOS aarch64 (Apple silicon) | A JDK 8+ JVM. | +| `cpu-android-aarch64` / `cpu-android-x86-64` | CPU | Android | For Android use the [`llama-android` AAR](#importing-in-android); these jars are the same libraries for other Android JVM setups. | +| `msvc-windows-x86-64` / `msvc-windows-x86` | CPU (Visual Studio generator) | Windows x86-64 / x86 | Same CPU backend and MSVC toolchain as `cpu-windows-*`, built with the Visual Studio generator instead of Ninja — an alternate-toolchain option; tried before the `cpu` jar when both are present. | +| `cuda13-linux-x86-64` | CUDA 13 | Linux x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 runtime libraries (`libcudart.so.13`, `libcublas.so.13`). | +| `cuda13-windows-x86-64` | CUDA 13 | Windows x86-64 with NVIDIA GPU | NVIDIA driver + CUDA 13 Toolkit (`cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll` on `PATH`). | +| `vulkan-linux-x86-64` | Vulkan | Linux x86-64 with a Vulkan 1.2+ GPU (NVIDIA / AMD / Intel) | A Vulkan runtime (`libvulkan.so.1`), which current GPU drivers install. The most portable Linux GPU option. glibc ≈ 2.39 (built on `ubuntu-latest`). | +| `vulkan-linux-aarch64` | Vulkan | Linux aarch64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`libvulkan.so.1`). glibc ≥ 2.39. | +| `vulkan-windows-x86-64` | Vulkan | Windows x86-64 with a Vulkan 1.2+ GPU | A Vulkan runtime (`vulkan-1.dll`), which current GPU drivers install. The most portable Windows GPU option. | +| `opencl-windows-x86-64` | OpenCL | Windows x86-64 with an OpenCL 2.0+ GPU | A vendor OpenCL ICD (`OpenCL.dll`). The GGML OpenCL backend is Adreno-tuned; on desktop GPUs CUDA or Vulkan are better supported. | +| `opencl-windows-aarch64` | OpenCL (Adreno) | Windows on ARM (Snapdragon X) | The Adreno driver's OpenCL ICD (`OpenCL.dll`). | +| `opencl-android-aarch64` | OpenCL (Adreno) | Android aarch64 with Adreno GPU | A device OpenCL ICD (`libOpenCL.so`); see also the `llama-android-opencl` AAR. | +| `rocm-linux-x86-64` | ROCm / HIP | Linux x86-64 with AMD GPU | An AMD ROCm **10** runtime (`libamdhip64.so`, `librocblas.so`, `libhipblas.so`) — built against ROCm 10.0 (TheRock), like upstream llama.cpp; every GPU TheRock builds for Linux, Instinct included (gfx900/gfx906/gfx90c/gfx1153 best effort — built, but not release-ready in ROCm 10). | +| `rocm-windows-x86-64` | ROCm / HIP | Windows x86-64 with AMD GPU | The AMD ROCm **10** runtime DLLs (`amdhip64.dll`, `rocblas.dll`, `hipblas.dll`) on `PATH`; every Radeon target TheRock builds for Windows, gfx900 through RDNA4 (gfx900/gfx906/gfx90c/gfx1153 best effort). | +| `sycl-fp16-linux-x86-64` | SYCL (Intel oneAPI, fp16) | Linux x86-64 with Intel GPU (Arc / iGPU) | An Intel oneAPI / Level-Zero runtime. fp16 accumulation (faster, slightly lower precision). | +| `sycl-fp32-linux-x86-64` | SYCL (Intel oneAPI, fp32) | Linux x86-64 with Intel GPU (Arc / iGPU) | An Intel oneAPI / Level-Zero runtime. fp32 accumulation (higher precision). | +| `sycl-windows-x86-64` | SYCL (Intel oneAPI) | Windows x86-64 with Intel GPU (Arc / iGPU) | The Intel oneAPI / Level-Zero runtime DLLs on `PATH`. | +| `openvino-linux-x86-64` | OpenVINO | Linux x86-64 (Intel GPU / NPU / CPU) | An Intel OpenVINO runtime. | +| `openvino-windows-x86-64` | OpenVINO | Windows x86-64 (Intel GPU / NPU / CPU) | The Intel OpenVINO runtime DLLs on `PATH`. | + +> [!NOTE] +> No vendor runtime is bundled; it comes from the GPU driver or toolkit on the host. The GPU +> jars are validated **build-only** in CI (GitHub runners have no GPU), so end-to-end GPU +> inference is verified locally / on self-hosted hardware. A GPU library that loads but finds +> no usable device (e.g. CUDA installed, no NVIDIA GPU) runs on the CPU inside that library and +> keeps the loader from trying the next backend; force one with +> `-Dnet.ladenthin.llama.backend=` (e.g. `vulkan` or `cpu`), which fails loud instead +> of falling back. + +> [!NOTE] +> On the **module path** each natives jar is an automatic module +> (`net.ladenthin.llama.natives.`, with `_` for `-`) that nothing requires, so +> resolve them with `--add-modules ALL-MODULE-PATH` (or keep the natives jars on the +> classpath). > [!NOTE] > Android `armeabi-v7a` (32-bit ARM) is **not** published. Only 64-bit > Android binaries are shipped: `aarch64` (devices) and `x86_64` -> (emulators, Chromebooks, x86-64 Android hardware) in the CPU-only default -> JAR and the `llama-android` AAR, plus `aarch64` as +> (emulators, Chromebooks, x86-64 Android hardware) as the `cpu-android-*` +> natives jars and in the `llama-android` AAR, plus `aarch64` as > `opencl-android-aarch64`. 32-bit Android devices are unsupported > by the released artifacts; building from source via the > `.github/dockcross/dockcross-android-arm` toolchain is possible but not @@ -280,20 +290,14 @@ java -jar llama--all-linux-x86-64-jar-with-dependencies.jar -m model.gg | `llama--all-windows-aarch64-jar-with-dependencies.jar` | OpenCL (Adreno / Snapdragon X) | yes | | `llama--jar-with-dependencies.jar` | none (CPU only, incl. macOS Metal) | — | -Each all-backends jar contains the library classes, all Java runtime -dependencies, the default CPU natives for **every** platform, and every GPU -backend for its named OS/arch. At startup the loader tries the bundled backends -in priority order (CUDA → ROCm → SYCL → Vulkan → OpenCL → OpenVINO) and uses -the **first one whose native library loads**; if none loads — e.g. no GPU -driver/toolkit installed — it falls back to the CPU natives, so the jar starts -everywhere. The usual GPU policy applies: vendor runtimes are **not** bundled -(see the classifier table above for what each backend needs on the host). -Force a specific backend with `-Dnet.ladenthin.llama.backend=` -(e.g. `vulkan`; fails loud instead of falling back) or force CPU with -`-Dnet.ladenthin.llama.backend=default`. A `.sha256` checksum file accompanies -every jar. These fat jars are GitHub download assets only — they are **not** -published to Maven Central (Maven users combine the thin classifier jars -instead, see above). +Each all-backends jar is the classes, all Java runtime dependencies, and every natives jar +of **its named OS/arch** (except `msvc`) merged into one file — pick the jar that matches +your platform. (A 32-bit JVM on 64-bit Windows, for example, needs the default +`llama--jar-with-dependencies.jar`, which carries the CPU natives of every platform.) +The loader picks the backend exactly as described [above](#choosing-the-natives-jars), so the +jar starts on every host of its OS/arch, with or without a GPU. A `.sha256` checksum file and a +detached GPG `.asc` signature accompany every jar. These fat jars are GitHub download assets +only — they are **not** published to Maven Central. ### Setup required @@ -319,12 +323,15 @@ cmake --build build --config Release > [!TIP] > Use `-DLLAMA_CURL=ON` to download models via Java code using `ModelParameters#setModelUrl(String)`. -All compiled libraries will be put in a resources directory matching your platform, which will appear in the cmake output. For example something like: +The library is put in a directory matching your platform and backend, which appears in the cmake output. For example: ```shell --- Installing files to /java-llama.cpp/llama/src/main/resources/net/ladenthin/llama/Linux/x86_64 +-- Backend 'cpu' - installing files to /java-llama.cpp/llama/src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu ``` +`mvn test` puts that directory on the test classpath; `mvn -P natives package` turns it into a +natives jar (in CI, together with every other platform's). + #### Library Location This project has to load a single shared library `jllama`. @@ -337,8 +344,8 @@ The application will search in the following order in the following locations: - In **java.library.path**: These are predefined locations for each OS, e.g., `/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib` on Linux. You can find out the locations using `System.out.println(System.getProperty("java.library.path"))`. Use this option if you want to install the shared libraries as system libraries. -- From the **JAR**: If any of the libraries weren't found yet, the application will try to use a prebuilt shared library. - This of course only works for the [supported platforms](#no-setup-required) . +- From the **natives jars** on the classpath: every backend directory found for your platform, in the + order described in [Choosing the natives jars](#choosing-the-natives-jars). #### System Properties Reference @@ -349,7 +356,7 @@ Every `net.ladenthin.llama.*` system property recognised by the library, deep-sc | `net.ladenthin.llama.lib.path` | unset (falls back to `java.library.path`) | runtime | `LlamaLoader` | Directory containing the native `jllama` shared library. Checked first, before `java.library.path`. Set with `-Dnet.ladenthin.llama.lib.path=/path/to/dir`. | | `net.ladenthin.llama.tmpdir` | unset (falls back to `java.io.tmpdir`) | runtime | `LlamaLoader` | Custom temporary directory used when extracting the native library from the JAR. | | `net.ladenthin.llama.osinfo.architecture` | unset (uses `os.arch`) | runtime | `OSInfo` | Override for the architecture string used to locate the bundled library inside the JAR. Useful when `os.arch` reports an unexpected value (e.g. inside dockcross / chrooted environments). | -| `net.ladenthin.llama.backend` | unset (auto: first loadable backend, then CPU) | runtime | `LlamaLoader` | Backend override for the [all-backends fat jars](#standalone-server-fat-jars-github-releases) (jars carrying a `jllama-backends.txt` manifest). Names one bundled backend (e.g. `cuda13`, `vulkan`) to load exclusively — failure is then fatal instead of falling back — or `default`/`cpu` to skip all GPU backends. Ignored by jars without a backend manifest. | +| `net.ladenthin.llama.backend` | unset (auto: the first backend on the classpath whose library loads) | runtime | `LlamaLoader` | Names one backend directory (e.g. `cuda13`, `vulkan`, `cpu`) to load exclusively — failure is then fatal instead of trying the next backend. See [Choosing the natives jars](#choosing-the-natives-jars). | | `net.ladenthin.llama.test.ngl` | `43` for the general suite; `0` for `ToolCallingIntegrationTest` | test | Model-backed integration tests | Number of GPU layers used during testing. Pin to `0` on CPU-only hosts: `mvn test -Dnet.ladenthin.llama.test.ngl=0`. The tool test also selects device `none` at zero layers so Metal/CUDA is not initialized. | | `net.ladenthin.llama.tool.model` | `models/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` (test self-skips if missing) | test | `ToolCallingIntegrationTest` | Path to a tool-capable GGUF used to verify required blocking and streaming tool calls. The default matches the Qwen2.5 model in upstream llama.cpp's tool-call test matrix. | | `net.ladenthin.llama.nomic.path` | unset (test self-skips) | test | `LlamaEmbeddingsTest#testNomicEmbedLoads` | Path to a Nomic embedding model (`nomic-embed-text-v1.5.f16.gguf` or a compatible BERT-family encoder). Regression test for upstream issue #98 (BERT-encoder `result_output` assertion). | @@ -987,8 +994,8 @@ RouterClient client = new RouterClient(8080, System.getenv("LLAMA_API_KEY")); llama.cpp's RPC backend spreads one model over the devices of several machines: every machine that contributes runs an **RPC server**, and the machine that loads the model names them with `--rpc`. Layers are then distributed over local and remote devices exactly as over several local -GPUs (`setGpuLayers`, `setTensorSplit`). Both halves are in every artifact — the default JAR and -every GPU classifier — with **no additional runtime dependency** (plain TCP over the system socket +GPUs (`setGpuLayers`, `setTensorSplit`). Both halves are in every natives jar — CPU and GPU — +with **no additional runtime dependency** (plain TCP over the system socket library the library already links). Serve this machine's devices (every GPU this library found, else the CPU): @@ -1066,6 +1073,9 @@ forcing that floor on every core consumer. It ships and versions in lockstep wit ``` +From 5.2.0 on, add the natives next to it — `llama-platform` or the natives jars you need (see +[Choosing the natives jars](#choosing-the-natives-jars)); the core it depends on is classes only. + Each adapter **borrows** a `LlamaModel` you already loaded — it never loads or closes the native model, so you manage its lifecycle (try-with-resources), and one `LlamaModel` can back several adapters at once: diff --git a/REUSE.toml b/REUSE.toml index 31bb8a6be..950bce38d 100644 --- a/REUSE.toml +++ b/REUSE.toml @@ -126,12 +126,12 @@ path = "llama/src/test/resources/audios/sample.wav" SPDX-FileCopyrightText = "2026 Bernard Ladenthin " SPDX-License-Identifier = "MIT" -# Backend-manifest loader test fixtures (backendtest/ + backendtest-ok/): +# Backend loader test fixtures (backendtest/ + backendtest-ok/): # deliberately unloadable fake *.so placeholders (one-line text files) plus two # trivial real x86-64 ELF dummies in backendtest-ok/ (built from a one-line C # TU, no upstream code) — .so files have no inline-comment syntax for SPDX -# headers; the manifest .txt files carry theirs inline. Consumed by -# BackendManifestLoadTest. +# headers; the jllama-extras.txt files carry theirs inline. Consumed by +# BackendLoadTest. [[annotations]] path = "llama/src/test/resources/net/ladenthin/llama/Linux/backendtest*/**" SPDX-FileCopyrightText = "2026 Bernard Ladenthin " diff --git a/TODO.md b/TODO.md index 903c1ff01..2160fce5f 100644 --- a/TODO.md +++ b/TODO.md @@ -19,9 +19,9 @@ so everything below is genuinely still open. ### macOS dylib links Homebrew OpenSSL (found by `verify-native-deps.py`) -- **The shipped `Mac/aarch64/libjllama.dylib` needs `/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` +- **The shipped `Mac/aarch64/metal/libjllama.dylib` needs `/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` and `libcrypto.3.dylib`** (verified on the published 5.1.0 jar and the current snapshot). The macOS - build finds the runner's Homebrew OpenSSL and links it dynamically, so the default JAR does not load + build finds the runner's Homebrew OpenSSL and links it dynamically, so the macOS natives do not load on a Mac without `brew install openssl@3` — and the macOS smoke cannot see it, because the runner has it. Likely fix: build BoringSSL statically on macOS as on Windows (`LLAMA_BUILD_BORINGSSL`, `llama/CMakeLists.txt`), or turn HTTPS off there (`-DLLAMA_OPENSSL=OFF`; @@ -350,7 +350,7 @@ real arm64 hardware and the Adreno/OpenCL flavor. Treat LLaMAndroid as prior art **What to investigate before committing**: - **JNI-loading shape.** Native Image supports JNI but requires `--enable-native-access=ALL-UNNAMED` + reflection/JNI configuration files (`reflect-config.json`, `jni-config.json`, `resource-config.json`) describing every class/method/field reachable across the JNI boundary. The 34 native methods in `jllama.cpp` plus the JNI-side `FindClass` / `GetFieldID` / `GetMethodID` calls at `JNI_OnLoad` need to be mapped. The GraalVM tracing agent (`-agentlib:native-image-agent=config-output-dir=...`) can auto-generate the config during a representative test run, but the `LlamaLoader` JAR-extraction path needs at least one resource-config rule for `net/ladenthin/llama/{OS}/{ARCH}/lib*.so`. - **Native-library packaging.** The current `LlamaLoader` extracts the OS-specific `.so`/`.dll`/`.dylib` from the JAR to a tmp dir at first use. Native Image needs the same file at AOT-execution time, so either (a) ship the native lib alongside the produced binary as a sidecar file and adjust `LlamaLoader` to find it on the same directory, or (b) embed the native lib as a resource and keep the existing extract-to-tmpdir flow (which Native Image supports via `resource-config.json`). - - **CUDA / Metal / OpenCL backend selection.** Today the choice between CPU-only / `cuda13-linux-x86-64` / `opencl-android-aarch64` JARs is at Maven-classifier time. Native Image would need either one binary per backend (multiplying the release matrix) or a runtime selector inside `LlamaLoader` that picks among bundled backend libs. The latter is a bigger refactor. + - **CUDA / Metal / OpenCL backend selection.** `LlamaLoader` already selects at runtime among the natives jars on the classpath (one directory per backend, fixed priority order). Native Image would need those directories as bundled resources (`resource-config.json`) or as sidecar files next to the binary. - **Startup-time benchmark to justify the work.** Measure cold-start of a current java-llama.cpp `LlamaModel(new ModelParameters().setModel("...").setNPredict(1))` invocation: how much is JVM startup + class load vs JNI load + model parse + tokenize + 1 token? If JVM startup is < 10 % of cold-start, Native Image yields little. If JVM startup is > 50 %, it's a clear win for CLI / serverless use cases. - **Maintenance cost.** Native Image adds a second build matrix (per OS × per backend × per JDK) and a new failure surface (Native Image config drift when a llama.cpp version bump adds new JNI-reachable types). Should ship only with a CI job that exercises the Native Image build on at least one OS, otherwise the config files will rot silently. @@ -368,8 +368,7 @@ green pipeline, and is the macOS member of the cross-repo convention in **Optional depth, not scheduled:** a full model-backed macOS server smoke (poll `/health`, assert a `/v1/chat/completions` choice) as Linux and Windows run. It would need `verify-model-cache` + a cache restore, and — since there is no `all-macos-*` fat jar — either a macOS variant from -`package-fatjars` or a `smoke-test-fatjar.sh` flag making the backend-manifest grep optional for the -manifest-less default jar. Worth doing only if a macOS-specific *inference* regression ever appears; +`package-fatjars` or running `smoke-test-fatjar.sh` against the default jar. Worth doing only if a macOS-specific *inference* regression ever appears; the load-time failure class is already covered, and a slow smoke tends to get made non-gating. **Not yet observed green in CI** — the job and the two sibling-repo smokes landed in one change set @@ -454,12 +453,10 @@ introduced by the version bump — they were deferred to keep that PR landable. the Java → JNI → native trainer round trip. Adding a small training model to `models.csv` plus the matching property to the Java test jobs would close it. -- **`LlamaLoader`'s jar-extraction internals need synthetic jar fixtures.** `readBackendManifest`, - `tryLoadBackend`, `extractFile`, `moveIntoPlace`, `cleanPath` and `hasNativeLib` are named in no - test; `BackendManifestLoadTest` and `LlamaLoaderTest` drive the class only from outside via system - properties. Covering the multi-backend fat-jar path (per-backend temp subdir extraction, - manifest-extras-first ordering, `UnsatisfiedLinkError` fallback to the next backend and then to the - default CPU natives) means building jars carrying a `jllama-backends.txt` and dummy payloads. +- **`LlamaLoader`'s jar-extraction internals are tested only through directory fixtures.** + `BackendLoadTest` drives backend probing, extras, fallthrough and forcing over the committed + `Linux/backendtest*/` trees on the test classpath (directories, not jars); extraction out of a real + jar is exercised in CI by `smoke-natives-jars.sh` and the fat-jar smokes, not by a unit test. - **`Java8CompatibilityHelper` is mostly dead code — decide delete vs. test.** Six of its seven public methods have zero call sites repo-wide; the only live one is diff --git a/llama-android/build.gradle.kts b/llama-android/build.gradle.kts index 1e1a4b710..4343fd052 100644 --- a/llama-android/build.gradle.kts +++ b/llama-android/build.gradle.kts @@ -14,9 +14,9 @@ // with plain Gradle means: // 1. classes.jar carries the BYTE-IDENTICAL Maven-built core classes (no // recompilation, no Lombok/AGP coupling, no drift from the tested jar); -// only the desktop/Android native resources and module-info.class are -// stripped (the .so ships under jni/ instead — LlamaLoader already calls -// System.loadLibrary("jllama") first on Android, see LlamaLoader). +// only module-info.class is stripped. The core jar carries no natives (they +// ship as separate natives jars); here the .so ships under jni/ instead — +// LlamaLoader calls System.loadLibrary("jllama") first on Android. // 2. The published POM says aar, which plain Maven // cannot produce — this is exactly how AGP-built libraries on Central // declare themselves, so `implementation("net.ladenthin:llama-android:V")` @@ -28,8 +28,9 @@ // // Inputs expected before running the aar tasks (fail-loud checks below): // ../llama/target/llama-.jar mvn -pl llama -am -DskipTests package -// natives/cpu/arm64-v8a/libjllama.so CI artifact Linux-Android-aarch64-libraries -// natives/opencl/arm64-v8a/libjllama.so CI artifact android-libraries-opencl +// natives/cpu/arm64-v8a/libjllama.so CI artifact natives-cpu-android-aarch64 +// natives/cpu/x86_64/libjllama.so CI artifact natives-cpu-android-x86-64 +// natives/opencl/arm64-v8a/libjllama.so CI artifact natives-opencl-android-aarch64 import org.w3c.dom.Document import org.w3c.dom.Element @@ -112,10 +113,6 @@ val coreClassesJar = tasks.register("coreClassesJar") { } } from(zipTree(coreJarFile)) { - exclude("net/ladenthin/llama/Linux/**") - exclude("net/ladenthin/llama/Linux-Android/**") - exclude("net/ladenthin/llama/Mac/**") - exclude("net/ladenthin/llama/Windows/**") exclude("module-info.class") exclude("META-INF/maven/**") } @@ -173,9 +170,9 @@ fun registerAarTask(taskName: String, artifactBase: String, nativesSubdir: Strin val so = File(nativesDir, "$abi/libjllama.so") require(so.isFile) { "Missing Android native library: $so — stage the CI-built libjllama.so there " + - "(artifacts 'Linux-Android-aarch64-libraries' / 'Linux-Android-x86_64-libraries' " + - "for cpu, 'android-libraries-opencl' for opencl; the artifact tree is " + - "net/ladenthin/llama/Linux-Android//libjllama.so)" + "(artifacts 'natives-cpu-android-aarch64' / 'natives-cpu-android-x86-64' " + + "for cpu, 'natives-opencl-android-aarch64' for opencl; the artifact tree is " + + "///libjllama.so, e.g. Linux-Android/aarch64/cpu/libjllama.so)" } } } diff --git a/llama-atmosphere-agent/README.md b/llama-atmosphere-agent/README.md index 8bcfcda32..4c41487c6 100644 --- a/llama-atmosphere-agent/README.md +++ b/llama-atmosphere-agent/README.md @@ -100,10 +100,11 @@ Windows (PowerShell — quote the whole `-D` argument, no line continuation with mvn -q compile exec:java "-Dexec.args=--model models\Qwen3-4B-Instruct-2507-Q4_K_M.gguf --ctx-size 16384 --workspace C:\path\to\project --allow-shell" ``` -**4. Optional: use the GPU.** Add a core classifier and offload the layers, e.g. +**4. Optional: use the GPU.** Add a natives jar of the core and offload the layers, e.g. `-Dllama.classifier=vulkan-windows-x86-64` (or `vulkan-linux-x86-64`; any current GPU driver) or -`cuda13-linux-x86-64` (needs the CUDA 13 toolkit), plus `--ngl 99` inside `-Dexec.args`. macOS uses -Metal with the default jar already. The root README's classifier table lists every backend. +`cuda13-linux-x86-64` (needs the CUDA 13 toolkit), plus `--ngl 99` inside `-Dexec.args`. It comes on +top of the CPU natives, which stay the fallback. macOS uses Metal already. The root README's +natives table lists every backend. ## Quick start @@ -164,10 +165,11 @@ first compiler error"*. On Windows PowerShell, quote the whole argument instead: `"-Dexec.args=--model C:\models\… --allow-shell --system '…'"`. Inside `--system '…'` avoid the apostrophe (write *the user* rather than *user's*): the value is already single-quoted. -GPU natives: pick the core classifier, e.g. `-Dllama.classifier=cuda13-linux-x86-64` or -`vulkan-windows-x86-64` (the vendor runtime must be installed — see the root README's classifier -table). Without it the default CPU jar (incl. macOS Metal) is used. In mode A the classifier is -irrelevant: inference stays in the running server, the agent's JVM loads no model. +GPU natives: add a natives jar of the core, e.g. `-Dllama.classifier=cuda13-linux-x86-64` or +`vulkan-windows-x86-64` (the vendor runtime must be installed — see the root README's natives +table). The CPU natives of every desktop platform (`llama-platform`, macOS with Metal) are always +there and stay the fallback. In mode A the natives are irrelevant: inference stays in the running +server, the agent's JVM loads no model. ### Options diff --git a/llama-atmosphere-agent/pom.xml b/llama-atmosphere-agent/pom.xml index 126527be5..201443823 100644 --- a/llama-atmosphere-agent/pom.xml +++ b/llama-atmosphere-agent/pom.xml @@ -45,9 +45,6 @@ SPDX-License-Identifier: MIT else can override it too, e.g. -Dllama.version=5.3.0-SNAPSHOT for a pre-release core. Bump it as part of every release. --> 5.2.0 - - 4.0.71 4.4.6 12.1.13 @@ -80,12 +77,12 @@ SPDX-License-Identifier: MIT - + net.ladenthin llama ${llama.version} - ${llama.classifier} + + natives + + + llama.natives + !none + + + + + net.ladenthin + llama-platform + ${llama.version} + pom + + + + + + gpu-natives + + + llama.classifier + + + + + net.ladenthin + llama + ${llama.version} + ${llama.classifier} + + + + + + 4.0.0 + + + net.ladenthin + llama-parent + 5.2.0-SNAPSHOT + ../pom.xml + + + + llama-platform + pom + + ${project.groupId}:${project.artifactId} + net.ladenthin:llama with the CPU natives of every desktop platform + (Linux x86-64/aarch64/s390x, Windows x86-64/x86/aarch64, macOS aarch64 with Metal). + https://github.com/bernardladenthin/java-llama.cpp + + + + MIT License + https://www.opensource.org/licenses/mit-license.php + repo + + + + + + Bernard Ladenthin + https://github.com/bernardladenthin + + + + + scm:git:https://github.com/bernardladenthin/java-llama.cpp.git + scm:git:https://github.com/bernardladenthin/java-llama.cpp.git + https://github.com/bernardladenthin/java-llama.cpp/tree/main + + + + + net.ladenthin + llama + ${project.version} + + + net.ladenthin + llama + ${project.version} + cpu-linux-x86-64 + + + net.ladenthin + llama + ${project.version} + cpu-linux-aarch64 + + + net.ladenthin + llama + ${project.version} + cpu-linux-s390x + + + net.ladenthin + llama + ${project.version} + cpu-windows-x86-64 + + + net.ladenthin + llama + ${project.version} + cpu-windows-x86 + + + net.ladenthin + llama + ${project.version} + cpu-windows-aarch64 + + + net.ladenthin + llama + ${project.version} + metal-macos-aarch64 + + + diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index f5379408f..6fe352628 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -286,77 +286,41 @@ if(NOT OS_ARCH) message(FATAL_ERROR "Could not determine CPU architecture") endif() -# Backend + OS routing for the native-library output tree. Each GPU backend ships -# under its own Maven classifier, so it must land in a backend-specific resource -# root (the default CPU tree stays src/main/resources/). The GPU branches are -# OS-aware because the same GGML flag is used on more than one platform: -# - GGML_CUDA -> Linux (resources_linux_cuda) AND Windows (resources_windows_cuda) -# - GGML_OPENCL -> Android (resources_android_opencl) AND Windows (resources_windows_opencl) -# - GGML_VULKAN -> Windows (resources_windows_vulkan) AND Linux (resources_linux_vulkan) -# - GGML_HIP -> Linux (resources_linux_rocm) AND Windows (resources_windows_rocm) [AMD ROCm/HIP] -# - GGML_SYCL -> Windows (resources_windows_sycl) AND Linux (fp16/fp32 split, see below) [Intel oneAPI] -# - GGML_OPENVINO -> Linux (resources_linux_openvino) AND Windows (resources_windows_openvino) [Intel OpenVINO] -# The classifier->tree mapping is mirrored by the matching Maven profile in pom.xml. The Linux -# Vulkan tree holds both x86_64 and aarch64 under Linux/${OS_ARCH}; two Maven profiles -# (vulkan-linux / vulkan-linux-aarch64) split it into one single-arch classifier JAR each. The -# Windows OpenCL tree likewise holds both x86_64 (desktop ICD) and aarch64 (Snapdragon/Adreno), -# split by the opencl-windows / opencl-windows-aarch64 profiles. Linux SYCL ships two precision -# variants at the SAME arch, so it is routed to two distinct trees by GGML_SYCL_F16 (fp16 vs fp32). -if(GGML_CUDA) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_cuda/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (CUDA Windows) build - Installing files to ${JLLAMA_DIR}") +# Native-library output tree: src/main/natives/net/ladenthin/llama////. +# Every build writes into its own backend directory, so all builds of one pipeline share one +# tree without overwriting each other, and each directory becomes exactly one natives jar +# (classifier --, see llama/pom.xml). LlamaLoader tries the directories in a +# fixed priority order (LlamaLoader.BACKEND_PRIORITY), which must list every name set here. +# `cuda13` names the CUDA major version the classifier promises; change it together with the +# classifiers when CUDA's major version changes. `msvc` is the Visual Studio generator build, +# which ships next to the default Ninja build of the same compiler. +if(NOT DEFINED JLLAMA_BACKEND) + if(GGML_CUDA) + set(JLLAMA_BACKEND cuda13) + elseif(GGML_VULKAN) + set(JLLAMA_BACKEND vulkan) + elseif(GGML_OPENCL) + set(JLLAMA_BACKEND opencl) + elseif(GGML_HIP) + set(JLLAMA_BACKEND rocm) + elseif(GGML_SYCL AND OS_NAME STREQUAL "Windows") + set(JLLAMA_BACKEND sycl) + elseif(GGML_SYCL AND GGML_SYCL_F16) + set(JLLAMA_BACKEND sycl-fp16) + elseif(GGML_SYCL) + set(JLLAMA_BACKEND sycl-fp32) + elseif(GGML_OPENVINO) + set(JLLAMA_BACKEND openvino) + elseif(APPLE AND GGML_METAL) + set(JLLAMA_BACKEND metal) + elseif(CMAKE_GENERATOR MATCHES "^Visual Studio") + set(JLLAMA_BACKEND msvc) else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_cuda/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (CUDA Linux) build - Installing files to ${JLLAMA_DIR}") + set(JLLAMA_BACKEND cpu) endif() -elseif(GGML_VULKAN) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_vulkan/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (Vulkan Windows) build - Installing files to ${JLLAMA_DIR}") - else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_vulkan/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (Vulkan Linux) build - Installing files to ${JLLAMA_DIR}") - endif() -elseif(GGML_OPENCL) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_opencl/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (OpenCL Windows) build - Installing files to ${JLLAMA_DIR}") - else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_android_opencl/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (OpenCL Android) build - Installing files to ${JLLAMA_DIR}") - endif() -elseif(GGML_HIP) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_rocm/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (ROCm/HIP Windows) build - Installing files to ${JLLAMA_DIR}") - else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_rocm/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (ROCm/HIP Linux) build - Installing files to ${JLLAMA_DIR}") - endif() -elseif(GGML_SYCL) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_sycl/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (SYCL Windows) build - Installing files to ${JLLAMA_DIR}") - elseif(GGML_SYCL_F16) - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_sycl_fp16/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (SYCL Linux fp16) build - Installing files to ${JLLAMA_DIR}") - else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_sycl_fp32/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (SYCL Linux fp32) build - Installing files to ${JLLAMA_DIR}") - endif() -elseif(GGML_OPENVINO) - if(OS_NAME STREQUAL "Windows") - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_windows_openvino/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (OpenVINO Windows) build - Installing files to ${JLLAMA_DIR}") - else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources_linux_openvino/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "GPU (OpenVINO Linux) build - Installing files to ${JLLAMA_DIR}") - endif() -else() - set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/resources/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}) - message(STATUS "CPU build - Installing files to ${JLLAMA_DIR}") endif() +set(JLLAMA_DIR ${CMAKE_SOURCE_DIR}/src/main/natives/net/ladenthin/llama/${OS_NAME}/${OS_ARCH}/${JLLAMA_BACKEND}) +message(STATUS "Backend '${JLLAMA_BACKEND}' - installing files to ${JLLAMA_DIR}") # include jni.h and jni_md.h if(NOT DEFINED JNI_INCLUDE_DIRS) diff --git a/llama/pom.xml b/llama/pom.xml index e2afad0f6..185edde1b 100644 --- a/llama/pom.xml +++ b/llama/pom.xml @@ -585,9 +585,9 @@ SPDX-License-Identifier: MIT the Error Prone / NullAway / Checker Framework processors that target ordinary source files only. Java 8 runtimes silently ignore the resulting module-info.class at the JAR root, so existing classpath - consumers continue to work unchanged. LlamaLoader loads native libs - via Class.getResourceAsStream on its own class object, which resolves - to this module and therefore does not require any opens directives. --> + consumers continue to work unchanged. LlamaLoader finds the native libs + (separate natives jars) through its ClassLoader, which needs no opens + directives. --> 9 module-info.java @@ -726,6 +726,15 @@ SPDX-License-Identifier: MIT org.slf4j:slf4j-simple + + + ${project.basedir}/src/main/natives + + - cuda + natives - - org.apache.maven.plugins - maven-compiler-plugin - - - - gpu - compile - - compile - - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_cuda - - - - - - maven-resources-plugin - - - - copy-resources - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_cuda - - - - ${basedir}/src/main/resources_linux_cuda/ - - **/*.* - - - - - - - org.apache.maven.plugins maven-jar-plugin - - cuda + natives-cpu-linux-x86-64 package jar - cuda13-linux-x86-64 - - ${project.build.outputDirectory}_cuda + cpu-linux-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_linux_x86_64 + + - - - - - - - - opencl-android - - - - org.apache.maven.plugins - maven-compiler-plugin - - - - opencl-android - compile - - compile - - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_opencl_android - - - - - - maven-resources-plugin - - - - copy-resources-opencl-android - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_opencl_android - - - - ${basedir}/src/main/resources_android_opencl/ - - **/*.* - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - - opencl-android + natives-cpu-linux-aarch64 package jar - opencl-android-aarch64 - - ${project.build.outputDirectory}_opencl_android + cpu-linux-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/aarch64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_linux_aarch64 + + - - - - - - - - windows-msvc - - - - org.apache.maven.plugins - maven-compiler-plugin - - - - windows-msvc - compile - - compile - - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_msvc - - - - - - maven-resources-plugin - - - - copy-resources-windows-msvc - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_msvc - - - - ${basedir}/src/main/resources_windows_msvc/ - - **/*.* - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - - windows-msvc + natives-cpu-linux-s390x package jar - msvc-windows - - ${project.build.outputDirectory}_windows_msvc + cpu-linux-s390x + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/s390x/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_linux_s390x + + - - - - - - - - - cuda-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - cuda-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_cuda - - - - - - maven-resources-plugin - - - copy-resources-cuda-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_cuda - - - - ${basedir}/src/main/resources_windows_cuda/ - - **/*.* - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - cuda-windows + natives-cpu-android-aarch64 package jar - cuda13-windows-x86-64 - - ${project.build.outputDirectory}_windows_cuda + cpu-android-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux-Android/aarch64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_android_aarch64 + + - - - - - - - - - vulkan-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - vulkan-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_vulkan - - - - - - maven-resources-plugin - - - copy-resources-vulkan-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_vulkan - - - - ${basedir}/src/main/resources_windows_vulkan/ - - **/*.* - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - vulkan-windows + natives-cpu-android-x86-64 package jar - vulkan-windows-x86-64 - - ${project.build.outputDirectory}_windows_vulkan + cpu-android-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux-Android/x86_64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_android_x86_64 + + + + + + natives-cpu-windows-x86-64 + package + + jar + + + cpu-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_windows_x86_64 + + - - - - - - - - vulkan-linux - - - - org.apache.maven.plugins - maven-compiler-plugin - - - vulkan-linux - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_vulkan - - - - - - maven-resources-plugin - - - copy-resources-vulkan-linux - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_vulkan - - - - ${basedir}/src/main/resources_linux_vulkan/ - - net/ladenthin/llama/Linux/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - vulkan-linux + natives-cpu-windows-x86 + package + + jar + + + cpu-windows-x86 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_windows_x86 + + + + + + natives-cpu-windows-aarch64 + package + + jar + + + cpu-windows-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/aarch64/cpu/** + + + false + + net.ladenthin.llama.natives.cpu_windows_aarch64 + + + + + + natives-metal-macos-aarch64 + package + + jar + + + metal-macos-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Mac/aarch64/metal/** + + + false + + net.ladenthin.llama.natives.metal_macos_aarch64 + + + + + + natives-msvc-windows-x86-64 + package + + jar + + + msvc-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/msvc/** + + + false + + net.ladenthin.llama.natives.msvc_windows_x86_64 + + + + + + natives-msvc-windows-x86 + package + + jar + + + msvc-windows-x86 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86/msvc/** + + + false + + net.ladenthin.llama.natives.msvc_windows_x86 + + + + + + natives-cuda13-linux-x86-64 + package + + jar + + + cuda13-linux-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/cuda13/** + + + false + + net.ladenthin.llama.natives.cuda13_linux_x86_64 + + + + + + natives-cuda13-windows-x86-64 + package + + jar + + + cuda13-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/cuda13/** + + + false + + net.ladenthin.llama.natives.cuda13_windows_x86_64 + + + + + + natives-vulkan-linux-x86-64 package jar vulkan-linux-x86-64 - - ${project.build.outputDirectory}_linux_vulkan + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/vulkan/** + + + false + + net.ladenthin.llama.natives.vulkan_linux_x86_64 + + - - - - - - - - - vulkan-linux-aarch64 - - - - org.apache.maven.plugins - maven-compiler-plugin - - - vulkan-linux-aarch64 - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_vulkan_aarch64 - - - - - - maven-resources-plugin - - - copy-resources-vulkan-linux-aarch64 - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_vulkan_aarch64 - - - - ${basedir}/src/main/resources_linux_vulkan/ - - net/ladenthin/llama/Linux/aarch64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - vulkan-linux-aarch64 + natives-vulkan-linux-aarch64 package jar vulkan-linux-aarch64 - - ${project.build.outputDirectory}_linux_vulkan_aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/aarch64/vulkan/** + + + false + + net.ladenthin.llama.natives.vulkan_linux_aarch64 + + + + + + natives-vulkan-windows-x86-64 + package + + jar + + + vulkan-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/vulkan/** + + + false + + net.ladenthin.llama.natives.vulkan_windows_x86_64 + + - - - - - - - - - opencl-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - opencl-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_opencl - - - - - - maven-resources-plugin - - - copy-resources-opencl-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_opencl - - - - ${basedir}/src/main/resources_windows_opencl/ - - net/ladenthin/llama/Windows/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - opencl-windows + natives-opencl-android-aarch64 + package + + jar + + + opencl-android-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux-Android/aarch64/opencl/** + + + false + + net.ladenthin.llama.natives.opencl_android_aarch64 + + + + + + natives-opencl-windows-x86-64 package jar opencl-windows-x86-64 - - ${project.build.outputDirectory}_windows_opencl + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/opencl/** + + + false + + net.ladenthin.llama.natives.opencl_windows_x86_64 + + - - - - - - - - - rocm-linux - - - - org.apache.maven.plugins - maven-compiler-plugin - - - rocm-linux - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_rocm - - - - - - maven-resources-plugin - - - copy-resources-rocm-linux - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_rocm - - - - ${basedir}/src/main/resources_linux_rocm/ - - net/ladenthin/llama/Linux/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - rocm-linux + natives-opencl-windows-aarch64 package jar - rocm-linux-x86-64 - - ${project.build.outputDirectory}_linux_rocm + opencl-windows-aarch64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/aarch64/opencl/** + + + false + + net.ladenthin.llama.natives.opencl_windows_aarch64 + + - - - - - - - - rocm-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - rocm-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_rocm - - - - - - maven-resources-plugin - - - copy-resources-rocm-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_rocm - - - - ${basedir}/src/main/resources_windows_rocm/ - - net/ladenthin/llama/Windows/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - rocm-windows + natives-rocm-linux-x86-64 package jar - rocm-windows-x86-64 - - ${project.build.outputDirectory}_windows_rocm + rocm-linux-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/rocm/** + + + false + + net.ladenthin.llama.natives.rocm_linux_x86_64 + + - - - - - - - - sycl-fp16-linux - - - - org.apache.maven.plugins - maven-compiler-plugin - - - sycl-fp16-linux - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_sycl_fp16 - - - - - - maven-resources-plugin - - - copy-resources-sycl-fp16-linux - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_sycl_fp16 - - - - ${basedir}/src/main/resources_linux_sycl_fp16/ - - net/ladenthin/llama/Linux/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - sycl-fp16-linux + natives-rocm-windows-x86-64 package jar - sycl-fp16-linux-x86-64 - - ${project.build.outputDirectory}_linux_sycl_fp16 + rocm-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/rocm/** + + + false + + net.ladenthin.llama.natives.rocm_windows_x86_64 + + - - - - - - - - sycl-fp32-linux - - - - org.apache.maven.plugins - maven-compiler-plugin - - - sycl-fp32-linux - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_sycl_fp32 - - - - - - maven-resources-plugin - - - copy-resources-sycl-fp32-linux - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_sycl_fp32 - - - - ${basedir}/src/main/resources_linux_sycl_fp32/ - - net/ladenthin/llama/Linux/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - sycl-fp32-linux + natives-sycl-fp16-linux-x86-64 package jar - sycl-fp32-linux-x86-64 - - ${project.build.outputDirectory}_linux_sycl_fp32 + sycl-fp16-linux-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/sycl-fp16/** + + + false + + net.ladenthin.llama.natives.sycl_fp16_linux_x86_64 + + - - - - - - - - sycl-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - sycl-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_sycl - - - - - - maven-resources-plugin - - - copy-resources-sycl-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_sycl - - - - ${basedir}/src/main/resources_windows_sycl/ - - net/ladenthin/llama/Windows/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - sycl-windows + natives-sycl-fp32-linux-x86-64 package jar - sycl-windows-x86-64 - - ${project.build.outputDirectory}_windows_sycl + sycl-fp32-linux-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/sycl-fp32/** + + + false + + net.ladenthin.llama.natives.sycl_fp32_linux_x86_64 + + - - - - - - - - opencl-windows-aarch64 - - - - org.apache.maven.plugins - maven-compiler-plugin - - - opencl-windows-aarch64 - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_opencl_aarch64 - - - - - - maven-resources-plugin - - - copy-resources-opencl-windows-aarch64 - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_opencl_aarch64 - - - - ${basedir}/src/main/resources_windows_opencl/ - - net/ladenthin/llama/Windows/aarch64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - opencl-windows-aarch64 + natives-sycl-windows-x86-64 package jar - opencl-windows-aarch64 - - ${project.build.outputDirectory}_windows_opencl_aarch64 + sycl-windows-x86-64 + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/sycl/** + + + false + + net.ladenthin.llama.natives.sycl_windows_x86_64 + + - - - - - - - - openvino-linux - - - - org.apache.maven.plugins - maven-compiler-plugin - - - openvino-linux - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_linux_openvino - - - - - - maven-resources-plugin - - - copy-resources-openvino-linux - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_linux_openvino - - - - ${basedir}/src/main/resources_linux_openvino/ - - net/ladenthin/llama/Linux/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - openvino-linux + natives-openvino-linux-x86-64 package jar openvino-linux-x86-64 - - ${project.build.outputDirectory}_linux_openvino + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Linux/x86_64/openvino/** + + + false + + net.ladenthin.llama.natives.openvino_linux_x86_64 + + - - - - - - - - openvino-windows - - - - org.apache.maven.plugins - maven-compiler-plugin - - - openvino-windows - compile - - compile - - - - module-info.java - - - -h - ${project.basedir}/src/main/cpp - - - ${project.build.outputDirectory}_windows_openvino - - - - - - maven-resources-plugin - - - copy-resources-openvino-windows - process-classes - - copy-resources - - - - ${project.build.outputDirectory}_windows_openvino - - - - ${basedir}/src/main/resources_windows_openvino/ - - net/ladenthin/llama/Windows/x86_64/** - - - - - - - - - org.apache.maven.plugins - maven-jar-plugin - - openvino-windows + natives-openvino-windows-x86-64 package jar openvino-windows-x86-64 - - ${project.build.outputDirectory}_windows_openvino + ${project.basedir}/src/main/natives + + net/ladenthin/llama/Windows/x86_64/openvino/** + + + false + + net.ladenthin.llama.natives.openvino_windows_x86_64 + + diff --git a/llama/spotbugs-exclude.xml b/llama/spotbugs-exclude.xml index 32809ffc7..8d29e6e24 100644 --- a/llama/spotbugs-exclude.xml +++ b/llama/spotbugs-exclude.xml @@ -799,18 +799,6 @@ SPDX-License-Identifier: MIT - - - - - - - jar-with-dependencies @@ -41,5 +45,13 @@ SPDX-License-Identifier: MIT simplelogger.properties + + ${project.basedir}/src/main/natives + / + + net/ladenthin/llama/*/*/cpu/** + net/ladenthin/llama/Mac/*/metal/** + + diff --git a/llama/src/main/java/module-info.java b/llama/src/main/java/module-info.java index b73f4a0b8..280d5a32c 100644 --- a/llama/src/main/java/module-info.java +++ b/llama/src/main/java/module-info.java @@ -6,13 +6,14 @@ /** * JPMS module descriptor for the java-llama.cpp JNI bindings. * - *

Exports the three hand-written public packages - * ({@code net.ladenthin.llama}, {@code net.ladenthin.llama.args}, - * {@code net.ladenthin.llama.json}). The native libraries shipped under - * {@code /net/ladenthin/llama/{OS}/{ARCH}/} are loaded by - * {@link net.ladenthin.llama.loader.LlamaLoader} via - * {@link Class#getResourceAsStream(String)} on its own class object, so the resources - * are looked up in this module and do not need to be {@code opens}'d.

+ *

Exports the public packages. The native libraries are not in this module: they ship as + * separate natives jars, one directory {@code net/ladenthin/llama////} each, + * which on the module path are automatic modules (each jar declares its own + * {@code Automatic-Module-Name}). {@link net.ladenthin.llama.loader.LlamaLoader} finds them + * through its {@link ClassLoader}, which sees the resources of every module and of the + * classpath, so nothing needs to be {@code opens}'d. Nothing {@code requires} a natives module, + * so a module-path launch resolves them only with {@code --add-modules} (or with the natives + * jars on the classpath).

* *

JSpecify {@code @NullMarked} is declared at the module level here so that no source * file compiled at {@code --release 8} references the JSpecify annotation type directly. @@ -26,12 +27,9 @@ *

{@code requires static org.jspecify} is needed only at compile time of this * descriptor; JSpecify annotations carry {@code RetentionPolicy.CLASS} so module-path * consumers never need jspecify on their runtime path. Checker Framework qualifiers and - * the Codehaus animal-sniffer annotation are likewise compile-time only. Jackson and - * SLF4J are referenced from ordinary sources only; javac in the separate - * {@code module-info-compile} execution compiles {@code module-info.java} in isolation - * and therefore does not need their module names. Consumers that put this jar - * on the module path will load these dependencies through their own {@code requires} - * graph; consumers on the classpath are unaffected.

+ * the Codehaus animal-sniffer annotation are likewise compile-time only. Jackson (tree model + * only; a caller's own type given to {@code completeAsJson} must be open to Jackson) and SLF4J + * are runtime dependencies and therefore required.

* *

This descriptor compiles at {@code --release 9}; the rest of the source compiles * at {@code --release 8}. Java 8 runtimes silently ignore {@code module-info.class} at @@ -50,6 +48,9 @@ // com.sun.net.httpserver, so module-path consumers need to read jdk.httpserver. It is a // platform module (always present in the JDK), not an external dependency. requires jdk.httpserver; + // Runtime dependencies of the classes (JSON tree model, logging facade). + requires com.fasterxml.jackson.databind; + requires org.slf4j; exports net.ladenthin.llama; exports net.ladenthin.llama.args; diff --git a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java index 99e2b5ec3..4ab9ad781 100644 --- a/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java +++ b/llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java @@ -11,6 +11,7 @@ import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; +import java.net.URL; import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.LinkOption; @@ -32,7 +33,9 @@ * so that the library can find {@code *.dll}, {@code *.dylib} and * {@code *.so} files, according to the current OS (Windows, Linux, macOS). * - *

The library files are automatically extracted from this project's package (JAR). + *

The library files are extracted from the natives jars on the classpath: one directory per + * backend, {@code net/ladenthin/llama////}, tried in {@link #BACKEND_PRIORITY} + * order unless {@code net.ladenthin.llama.backend} forces one. * *

Historically the loader also honoured a {@code net.ladenthin.llama.lib.name} * property that overrode the resolved library filename. Upstream removed the @@ -71,54 +74,52 @@ public class LlamaLoader { private static final NativeLibraryPermissionSetter permissionSetter = new NativeLibraryPermissionSetter(System.err); /** - * Canonical classpath root for the bundled native libraries. Fixed by - * {@code CMakeLists.txt} and the publish workflow (both emit to - * {@code resources/net/ladenthin/llama///}); it must NOT be - * derived from this loader's own Java package, which moved to + * Classpath root of the bundled native libraries, without a leading slash because it is + * resolved through the {@link ClassLoader} (see {@link #resource(String)}). Below it every + * backend has its own directory, {@code ///}, and each natives jar + * carries exactly one of them, so any combination of natives jars can share one classpath. + * It must NOT be derived from this loader's own Java package, which moved to * {@code net.ladenthin.llama.loader} during the layered restructure. */ - private static final String NATIVE_RESOURCE_BASE = "/net/ladenthin/llama"; + static final String NATIVE_RESOURCE_BASE = "net/ladenthin/llama"; /** - * File name of the optional multi-backend manifest located next to the native libraries - * ({@code net/ladenthin/llama///jllama-backends.txt}). Only the multi-backend - * ("all") fat jars assembled by the release pipeline carry it; jars without it behave - * exactly as before. Format: one backend per line in priority order — the backend - * subdirectory name optionally followed by whitespace-separated extra files to extract and - * load before the main library; blank lines and {@code #} comments are skipped. + * The backend directories tried, in this order, when no backend is forced: accelerators + * first, then the CPU builds. The first one present on the classpath whose library loads + * wins; a missing vendor runtime fails its load cleanly and the next one is tried. Every + * build carries the CPU backend, so a GPU library that loads still runs a model on the CPU + * with {@code -ngl 0}. {@code msvc} precedes {@code cpu} because nobody adds the MSVC + * natives jar except to use it. */ - static final String BACKEND_MANIFEST_FILE = "jllama-backends.txt"; + static final List BACKEND_PRIORITY = Collections.unmodifiableList(Arrays.asList( + "cuda13", + "rocm", + "sycl-fp16", + "sycl-fp32", + "sycl", + "vulkan", + "opencl", + "openvino", + "metal", + "msvc", + "cpu")); /** - * Prefix of the per-backend extraction subdirectory below the temp dir. Deliberately starts - * with {@code jllama} so {@link #shouldCleanPath(Path)} matches it during cleanup. + * Optional file in a backend directory naming sibling files (one per line, {@code #} + * comments allowed) to extract and load, in order, before that backend's library — + * e.g. the OpenCL ICD loader bundled with OpenVINO on Windows, which Windows would not find + * next to the library on its own. */ - static final String BACKEND_TEMP_DIR_PREFIX = "jllama-backend-"; - - /** Special {@code net.ladenthin.llama.backend} value selecting the default (CPU) library. */ - static final String BACKEND_DEFAULT = "default"; - - /** Alias of {@link #BACKEND_DEFAULT}. */ - static final String BACKEND_CPU = "cpu"; + static final String BACKEND_EXTRAS_FILE = "jllama-extras.txt"; /** - * One entry of the multi-backend manifest: a backend subdirectory below the OS/arch native - * resource folder, plus the extra files (e.g. a bundled ICD loader) to extract and load - * before the main {@code jllama} library, in listed order. + * Prefix of the per-backend extraction subdirectory below the temp dir. Deliberately starts + * with {@code jllama} so {@link #shouldCleanPath(Path)} matches it during cleanup. */ - static final class BackendEntry { - - /** Backend subdirectory name below the OS/arch native resource folder. */ - final String name; - - /** Extra files to extract and load before the main library, in order; may be empty. */ - final List extraFiles; + static final String BACKEND_TEMP_DIR_PREFIX = "jllama-backend-"; - BackendEntry(String name, List extraFiles) { - this.name = name; - this.extraFiles = Collections.unmodifiableList(new ArrayList<>(extraFiles)); - } - } + /** Shader source a non-embedding Metal build ships next to its library. */ + private static final String METAL_SOURCE_FILE = "ggml-metal.metal"; /** Static utility holder; not instantiable. */ private LlamaLoader() {} @@ -163,15 +164,6 @@ private static void load() { if (!extracted) { cleanup(); } - if ("Mac".equals(OSInfo.getOSName())) { - String nativeDirName = getNativeResourcePath(); - String tempFolder = getTempDir().getAbsolutePath(); - System.out.println(nativeDirName); - Path metalFilePath = extractFile(nativeDirName, "ggml-metal.metal", tempFolder); - if (metalFilePath == null) { - System.err.println("'ggml-metal.metal' not found"); - } - } loadNativeLibrary("jllama"); extracted = true; } @@ -260,51 +252,39 @@ private static void loadNativeLibrary(String name) { } } - // Multi-backend ("all") fat jars carry a backend manifest next to their native - // libraries. Try each listed backend subdirectory in priority order; the first one - // whose library loads wins (a missing vendor runtime fails its load cleanly, and the - // next backend is tried). Jars without a manifest skip this block entirely. - String backendOverride = systemProperties.getBackend(); - List backendCandidates = selectBackendCandidates(readBackendManifest(), backendOverride); + // The natives jars: one directory per backend below //. A forced backend is + // the only candidate and fails loud; otherwise every backend present on the classpath + // is tried in priority order. + String forced = systemProperties.getBackend(); + List candidates = forced != null ? Collections.singletonList(forced) : BACKEND_PRIORITY; String nativeResourcePath = getNativeResourcePath(); Set residentExtraFiles = new HashSet<>(); - for (BackendEntry backend : backendCandidates) { - if (tryLoadBackend(nativeResourcePath, backend, residentExtraFiles)) { - System.out.println("[jllama] using native backend '" + backend.name + "'"); + for (String backend : candidates) { + String backendResourcePath = nativeResourcePath + "/" + backend; + if (resource(backendResourcePath + "/" + nativeLibName) == null) { + continue; + } + if (tryLoadBackend(backendResourcePath, backend, residentExtraFiles)) { + System.out.println("[jllama] using native backend '" + backend + "'"); return; } - triedPaths.add(nativeResourcePath + "/" + backend.name); + triedPaths.add(backendResourcePath); } - if (isForcedBackend(backendOverride) && !backendCandidates.isEmpty()) { - // An explicitly requested backend must fail loud instead of silently falling back. + if (forced != null) { throw new UnsatisfiedLinkError(String.format( "Forced native backend '%s' (%s.backend) could not be loaded for os.name=%s, os.arch=%s," + " paths=[%s]", - backendOverride, + forced, LlamaSystemProperties.PREFIX, OSInfo.getOSName(), OSInfo.getArchName(), String.join(File.pathSeparator, triedPaths))); } - if (!backendCandidates.isEmpty()) { - System.out.println("[jllama] no manifest backend loadable, using default (CPU) native library"); - } - - // As a last resort try load the os-dependent library from the jar file - nativeLibPath = nativeResourcePath; - if (hasNativeLib(nativeLibPath, nativeLibName)) { - // temporary library folder - String tempFolder = getTempDir().getAbsolutePath(); - // Try extracting the library from jar - if (extractAndLoadLibraryFile(nativeLibPath, nativeLibName, tempFolder)) { - return; - } else { - triedPaths.add(nativeLibPath); - } - } throw new UnsatisfiedLinkError(String.format( - "No native library found for os.name=%s, os.arch=%s, paths=[%s]", + "No native library found for os.name=%s, os.arch=%s, paths=[%s] -- add a natives jar for" + + " this platform (e.g. classifier cpu--) to the classpath, or on the module path together" + + " with --add-modules", OSInfo.getOSName(), OSInfo.getArchName(), String.join(File.pathSeparator, triedPaths))); } @@ -367,7 +347,7 @@ public static boolean loadNativeLibrary(Path path) { // into place so a concurrent loader never observes a half-written library. Path tempFile = Files.createTempFile(Paths.get(targetDirectory), fileName + ".", ".tmp"); try { - try (InputStream reader = LlamaLoader.class.getResourceAsStream(nativeLibraryFilePath)) { + try (InputStream reader = resourceAsStream(nativeLibraryFilePath)) { if (reader == null) { return null; } @@ -416,7 +396,7 @@ private static void moveIntoPlace(Path source, Path target) throws IOException { /** Whether the classpath resource at {@code resourcePath} is byte-identical to {@code file}. */ static boolean resourceMatchesFile(String resourcePath, Path file) throws IOException { - try (InputStream resource = LlamaLoader.class.getResourceAsStream(resourcePath); + try (InputStream resource = resourceAsStream(resourcePath); InputStream onDisk = Files.newInputStream(file)) { if (resource == null) { return false; @@ -426,110 +406,72 @@ static boolean resourceMatchesFile(String resourcePath, Path file) throws IOExce } /** - * Parses the multi-backend manifest (see {@link #BACKEND_MANIFEST_FILE} for the format). + * Parses a {@link #BACKEND_EXTRAS_FILE}. * - * @param reader the manifest content - * @return the listed backends in manifest (priority) order; empty for an empty manifest + * @param reader the file content + * @return the listed file names in order; blank lines and {@code #} comments are skipped * @throws IOException when reading fails */ - static List parseBackendManifest(BufferedReader reader) throws IOException { - List entries = new ArrayList<>(); + static List parseExtras(BufferedReader reader) throws IOException { + List extras = new ArrayList<>(); String line; while ((line = reader.readLine()) != null) { String trimmed = line.trim(); - if (trimmed.isEmpty() || trimmed.startsWith("#")) { - continue; + if (!trimmed.isEmpty() && !trimmed.startsWith("#")) { + extras.add(trimmed); } - String[] tokens = trimmed.split("\\s+"); - entries.add(new BackendEntry(tokens[0], Arrays.asList(tokens).subList(1, tokens.length))); } - return entries; + return extras; } /** - * Selects which backends to attempt, honoring the {@code net.ladenthin.llama.backend} - * override. + * Reads the {@link #BACKEND_EXTRAS_FILE} of a backend directory, if it has one. * - * @param manifest the parsed manifest entries in priority order - * @param override the override property value, or {@code null} if unset - * @return the manifest as-is without an override; an empty list for - * {@link #BACKEND_DEFAULT}/{@link #BACKEND_CPU}; otherwise exactly the named backend - * (synthesized without extra files when the manifest does not list it) + * @param backendResourcePath the backend's classpath directory + * @return the listed file names, or an empty list when the backend has no extras + * @throws IOException when the file exists but cannot be read */ - static List selectBackendCandidates(List manifest, @Nullable String override) { - if (override == null) { - return manifest; - } - if (BACKEND_DEFAULT.equals(override) || BACKEND_CPU.equals(override)) { - return Collections.emptyList(); - } - for (BackendEntry entry : manifest) { - if (entry.name.equals(override)) { - return Collections.singletonList(entry); - } - } - return Collections.singletonList(new BackendEntry(override, Collections.emptyList())); - } - - /** - * Whether the override names a specific backend (as opposed to being unset or selecting the - * default library), which makes a load failure fatal instead of falling back. - * - * @param override the override property value, or {@code null} if unset - * @return {@code true} when a specific backend is forced - */ - static boolean isForcedBackend(@Nullable String override) { - return override != null && !BACKEND_DEFAULT.equals(override) && !BACKEND_CPU.equals(override); - } - - /** - * Reads the multi-backend manifest from the classpath, if present. - * - * @return the parsed entries, or an empty list when no manifest ships in this jar (the - * normal case for every artifact except the multi-backend fat jars) - */ - private static List readBackendManifest() { - String manifestResource = getNativeResourcePath() + "/" + BACKEND_MANIFEST_FILE; - InputStream stream = LlamaLoader.class.getResourceAsStream(manifestResource); + private static List readExtras(String backendResourcePath) throws IOException { + InputStream stream = resourceAsStream(backendResourcePath + "/" + BACKEND_EXTRAS_FILE); if (stream == null) { return Collections.emptyList(); } try (BufferedReader reader = new BufferedReader(new InputStreamReader(stream, StandardCharsets.UTF_8))) { - return parseBackendManifest(reader); - } catch (IOException e) { - System.err.println("Failed to read backend manifest " + manifestResource + ": " + e.getMessage()); - return Collections.emptyList(); + return parseExtras(reader); } } /** - * Attempts to extract and load one manifest backend from its resource subdirectory into a - * per-backend temp subdirectory (backends share file names, so they must not overwrite each - * other's extractions). + * Attempts to extract and load one backend from its resource directory into a per-backend + * temp subdirectory (backends share file names, so they must not overwrite each other's + * extractions). * - * @param baseResourcePath the OS/arch native resource folder - * @param entry the backend to attempt - * @param residentExtraFiles file names of extra modules already loaded by earlier failed - * attempts; updated with this attempt's loaded extras. A native - * module cannot be unloaded, and imports bind by module name, so a - * backend declaring an extra file that is already resident from - * another backend must be skipped to avoid cross-wiring. + * @param backendResourcePath the backend's classpath directory + * @param backend the backend directory name + * @param residentExtraFiles file names of extra modules already loaded by earlier failed + * attempts; updated with this attempt's loaded extras. A native + * module cannot be unloaded, and imports bind by module name, so a + * backend declaring an extra file that is already resident from + * another backend must be skipped to avoid cross-wiring. * @return whether the backend's main library was successfully loaded */ - private static boolean tryLoadBackend(String baseResourcePath, BackendEntry entry, Set residentExtraFiles) { - String backendResourcePath = baseResourcePath + "/" + entry.name; - String mainLibraryFileName = System.mapLibraryName("jllama"); - if (!hasNativeLib(backendResourcePath, mainLibraryFileName)) { + private static boolean tryLoadBackend(String backendResourcePath, String backend, Set residentExtraFiles) { + List extraFiles; + try { + extraFiles = readExtras(backendResourcePath); + } catch (IOException e) { + System.err.println( + "Failed to read " + backendResourcePath + "/" + BACKEND_EXTRAS_FILE + ": " + e.getMessage()); return false; } - for (String extraFile : entry.extraFiles) { + for (String extraFile : extraFiles) { if (residentExtraFiles.contains(extraFile)) { - System.err.println("[jllama] skipping backend '" + entry.name + "': module '" + extraFile + System.err.println("[jllama] skipping backend '" + backend + "': module '" + extraFile + "' is already resident from a previously failed backend attempt"); return false; } } - Path targetDirPath = getTempDir().toPath().resolve(BACKEND_TEMP_DIR_PREFIX + entry.name); + Path targetDirPath = getTempDir().toPath().resolve(BACKEND_TEMP_DIR_PREFIX + backend); try { Files.createDirectories(targetDirPath); } catch (IOException e) { @@ -540,14 +482,19 @@ private static boolean tryLoadBackend(String baseResourcePath, BackendEntry entr // files registered afterwards by extractFile are deleted first, then this directory. targetDirPath.toFile().deleteOnExit(); String targetFolder = targetDirPath.toAbsolutePath().toString(); - for (String extraFile : entry.extraFiles) { + for (String extraFile : extraFiles) { Path extraPath = extractFile(backendResourcePath, extraFile, targetFolder); if (extraPath == null || !loadNativeLibrary(extraPath)) { return false; } residentExtraFiles.add(extraFile); } - return extractAndLoadLibraryFile(backendResourcePath, mainLibraryFileName, targetFolder); + // Only a Metal build that does not embed its shader source ships ggml-metal.metal + // (every CI build embeds it); ggml looks for it next to the library. + if (resource(backendResourcePath + "/" + METAL_SOURCE_FILE) != null) { + extractFile(backendResourcePath, METAL_SOURCE_FILE, targetFolder); + } + return extractAndLoadLibraryFile(backendResourcePath, System.mapLibraryName("jllama"), targetFolder); } /** @@ -596,7 +543,24 @@ static String getNativeResourcePath() { return String.format("%s/%s", NATIVE_RESOURCE_BASE, OSInfo.getNativeLibFolderPathForCurrentOS()); } - private static boolean hasNativeLib(String path, String libraryName) { - return LlamaLoader.class.getResource(path + "/" + libraryName) != null; + /** + * Looks a resource up through this class's {@link ClassLoader}, never through + * {@link Class#getResource(String)}: on the module path the latter only sees this module's + * own resources, while the natives live in separate jars (automatic modules). + * + * @param name the resource name, relative to the classpath root (no leading slash) + * @return the resource URL, or {@code null} when absent + */ + static @Nullable URL resource(String name) { + return classLoader().getResource(name); + } + + private static @Nullable InputStream resourceAsStream(String name) { + return classLoader().getResourceAsStream(name); + } + + private static ClassLoader classLoader() { + ClassLoader loader = LlamaLoader.class.getClassLoader(); + return loader != null ? loader : ClassLoader.getSystemClassLoader(); } } diff --git a/llama/src/main/java/net/ladenthin/llama/loader/LlamaSystemProperties.java b/llama/src/main/java/net/ladenthin/llama/loader/LlamaSystemProperties.java index a1952125f..262238b05 100644 --- a/llama/src/main/java/net/ladenthin/llama/loader/LlamaSystemProperties.java +++ b/llama/src/main/java/net/ladenthin/llama/loader/LlamaSystemProperties.java @@ -62,12 +62,9 @@ public LlamaSystemProperties() {} } /** - * Native-backend override for multi-backend ("all") fat jars that carry a - * {@code jllama-backends.txt} manifest next to their native libraries. Names one backend - * subdirectory (e.g. {@code cuda13}, {@code vulkan}) to load exclusively — loading then - * fails loud instead of falling back — or the special value {@code default} (alias - * {@code cpu}) to skip all manifest backends and load the default library directly. Ignored - * by jars without a backend manifest. + * Native-backend override. Names one backend directory (e.g. {@code cuda13}, {@code vulkan}, + * {@code cpu}) to load exclusively; loading then fails loud instead of trying the next + * backend. Unset, every backend on the classpath is tried in the loader's priority order. * * @return the configured backend name, or {@code null} if unset */ diff --git a/llama/src/test/java/net/ladenthin/llama/LlamaLoggerTest.java b/llama/src/test/java/net/ladenthin/llama/LlamaLoggerTest.java index ffcb80624..3b33b190d 100644 --- a/llama/src/test/java/net/ladenthin/llama/LlamaLoggerTest.java +++ b/llama/src/test/java/net/ladenthin/llama/LlamaLoggerTest.java @@ -31,7 +31,7 @@ import java.util.stream.Collectors; import net.ladenthin.llama.args.LogFormat; import net.ladenthin.llama.exception.LlamaException; -import net.ladenthin.llama.loader.OSInfo; +import net.ladenthin.llama.loader.NativeLibraryPresence; import net.ladenthin.llama.parameters.ModelParameters; import net.ladenthin.llama.value.LogLevel; import org.junit.jupiter.api.AfterEach; @@ -59,12 +59,6 @@ class LlamaLoggerTest { @TempDir Path tempDir; - private static boolean nativeLibraryOnClasspath() { - String resource = "/net/ladenthin/llama/" + OSInfo.getNativeLibFolderPathForCurrentOS() + "/" - + System.mapLibraryName("jllama"); - return LlamaLoggerTest.class.getResource(resource) != null; - } - private static final class Line { private final LogLevel level; private final String text; @@ -87,7 +81,7 @@ public String toString() { */ @AfterEach void restoreConsoleLogging() { - if (!nativeLibraryOnClasspath()) { + if (!NativeLibraryPresence.onClasspath()) { return; } LlamaModel.setLogger(LogFormat.TEXT, null); @@ -127,7 +121,7 @@ private static boolean sawLoadingModel(List lines) { @Test void loggerSetBeforeTheLoadReceivesTheLoadsOwnLines() throws IOException { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); List lines = linesOfAFailedLoad(LogFormat.TEXT); @@ -148,7 +142,7 @@ void loggerSetBeforeTheLoadReceivesTheLoadsOwnLines() throws IOException { @Test void jsonModeWrapsEveryLineIntoOneObject() throws IOException { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); List lines = linesOfAFailedLoad(LogFormat.JSON); @@ -164,7 +158,7 @@ void jsonModeWrapsEveryLineIntoOneObject() throws IOException { @Test void anEmptyCallbackDiscardsWithoutFailingTheLoad() throws IOException { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); LlamaModel.setLogger(LogFormat.TEXT, (level, text) -> {}); Path file = notAGguf(); @@ -192,7 +186,7 @@ private static Map toMap(JsonNode node) { */ @Test void concurrentSetLoggerCallsDoNotRaceOnTheLogWorker() throws Exception { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); final int threads = 4; final int rounds = 200; java.util.concurrent.ExecutorService pool = java.util.concurrent.Executors.newFixedThreadPool(threads); @@ -224,7 +218,7 @@ void concurrentSetLoggerCallsDoNotRaceOnTheLogWorker() throws Exception { */ @Test void verbosityThresholdIsProcessWideAndEveryLoadSetsIt() throws IOException { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); try { List errorsOnly = linesOfAFailedLoad(LogFormat.TEXT, new ModelParameters().setLogVerbosity(1)); assertThat("-lv 1 must drop the server's INFO line: " + errorsOnly, sawLoadingModel(errorsOnly), is(false)); @@ -255,7 +249,7 @@ void verbosityThresholdIsProcessWideAndEveryLoadSetsIt() throws IOException { */ @Test void deliveryIsAsynchronousOnTheLogWorkerAndRemovingTheLoggerDrains() throws Exception { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping logger guard"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping logger guard"); Thread caller = Thread.currentThread(); Set deliveringThreads = Collections.synchronizedSet(new HashSet<>()); AtomicInteger delivered = new AtomicInteger(); diff --git a/llama/src/test/java/net/ladenthin/llama/RpcIntegrationTest.java b/llama/src/test/java/net/ladenthin/llama/RpcIntegrationTest.java index 02a61c8ff..cb89175dc 100644 --- a/llama/src/test/java/net/ladenthin/llama/RpcIntegrationTest.java +++ b/llama/src/test/java/net/ladenthin/llama/RpcIntegrationTest.java @@ -18,6 +18,7 @@ import java.util.concurrent.CopyOnWriteArrayList; import net.ladenthin.llama.args.LogFormat; import net.ladenthin.llama.exception.LlamaException; +import net.ladenthin.llama.loader.NativeLibraryPresence; import net.ladenthin.llama.parameters.InferenceParameters; import net.ladenthin.llama.parameters.ModelParameters; import org.junit.jupiter.api.Test; @@ -40,7 +41,7 @@ public class RpcIntegrationTest { @Test public void aModelRunsOnAnRpcServerAndALaterLoadWithoutRpcIgnoresIt() throws Exception { - assumeTrue(RpcServerTest.nativeLibraryOnClasspath(), "libjllama not on classpath"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath"); assumeTrue(new File(TestConstants.DRAFT_MODEL_PATH).exists(), "draft model not found"); List log = new CopyOnWriteArrayList<>(); diff --git a/llama/src/test/java/net/ladenthin/llama/RpcServerTest.java b/llama/src/test/java/net/ladenthin/llama/RpcServerTest.java index 5aa0f5e1e..6352a31d6 100644 --- a/llama/src/test/java/net/ladenthin/llama/RpcServerTest.java +++ b/llama/src/test/java/net/ladenthin/llama/RpcServerTest.java @@ -21,7 +21,7 @@ import java.nio.file.Path; import java.util.Arrays; import net.ladenthin.llama.exception.LlamaException; -import net.ladenthin.llama.loader.OSInfo; +import net.ladenthin.llama.loader.NativeLibraryPresence; import net.ladenthin.llama.parameters.ModelParameters; import net.ladenthin.llama.value.RpcEndpoint; import org.junit.jupiter.api.BeforeEach; @@ -45,7 +45,7 @@ public class RpcServerTest { @BeforeEach void requireNativeLibrary() { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping RpcServerTest"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping RpcServerTest"); } @Test @@ -167,10 +167,4 @@ private static boolean accepts(int port) { return false; } } - - static boolean nativeLibraryOnClasspath() { - String resource = "/net/ladenthin/llama/" + OSInfo.getNativeLibFolderPathForCurrentOS() + "/" - + System.mapLibraryName("jllama"); - return RpcServerTest.class.getResource(resource) != null; - } } diff --git a/llama/src/test/java/net/ladenthin/llama/loader/BackendManifestLoadTest.java b/llama/src/test/java/net/ladenthin/llama/loader/BackendLoadTest.java similarity index 55% rename from llama/src/test/java/net/ladenthin/llama/loader/BackendManifestLoadTest.java rename to llama/src/test/java/net/ladenthin/llama/loader/BackendLoadTest.java index 4784cabdb..35c38e2b7 100644 --- a/llama/src/test/java/net/ladenthin/llama/loader/BackendManifestLoadTest.java +++ b/llama/src/test/java/net/ladenthin/llama/loader/BackendLoadTest.java @@ -16,17 +16,17 @@ import org.junit.jupiter.api.io.TempDir; @ClaudeGenerated( - purpose = "Drive LlamaLoader.initialize() end-to-end against the committed backend-manifest " - + "fixture trees (src/test/resources/net/ladenthin/llama/Linux/backendtest*/) by " + purpose = "Drive LlamaLoader.initialize() end-to-end against the committed backend fixture " + + "trees (src/test/resources/net/ladenthin/llama/Linux/backendtest*//) by " + "redirecting the arch component via the osinfo.architecture override. backendtest/ " - + "holds only unloadable fake libraries, exercising every failure branch (manifest " - + "read, per-backend temp-dir extraction, extra-file handling present and missing, " - + "clean load-failure fallback, forced-backend fail-loud, default/cpu manifest skip, " - + "no-manifest legacy path); backendtest-ok/ adds two trivial real x86-64 ELF dummies " - + "so the success path, the resident-extra bookkeeping, and the same-extra clash skip " - + "execute too. Linux-only (the trees are committed under the Linux OS folder); " - + "self-skips elsewhere.") -public class BackendManifestLoadTest { + + "holds only unloadable fake libraries in real backend directories (cuda13, rocm, " + + "vulkan), exercising every failure branch (priority probing, per-backend temp-dir " + + "extraction, jllama-extras.txt present and naming a missing file, clean load-failure " + + "fallthrough, forced-backend fail-loud, no backend at all); backendtest-ok/ adds two " + + "trivial real x86-64 ELF dummies so the success path, the resident-extra bookkeeping, " + + "and the same-extra clash skip execute too. Linux-only (the trees are committed under " + + "the Linux OS folder); self-skips elsewhere.") +public class BackendLoadTest { /** Arch-folder override selecting the committed fixture tree {@code Linux/backendtest/}. */ private static final String FIXTURE_ARCH = "backendtest"; @@ -71,7 +71,7 @@ private static void assumeLinuxFixtureTree() { // The fixture tree is committed under the Linux OS folder; the OS path component // cannot be overridden, so these tests are meaningful only on a Linux JVM (which is // where the CI coverage run executes). - assumeTrue("Linux".equals(OSInfo.getOSName()), "backend-manifest fixtures are committed for Linux only"); + assumeTrue("Linux".equals(OSInfo.getOSName()), "backend fixtures are committed for Linux only"); } private static void assumeX86_64() { @@ -85,31 +85,29 @@ private static void assumeX86_64() { } @Test - public void autoModeTriesEveryManifestBackendThenFailsWithoutDefaultLibrary() { + public void triesEveryBackendOnTheClasspathInPriorityOrderThenFails() { assumeLinuxFixtureTree(); UnsatisfiedLinkError error = assertThrows(UnsatisfiedLinkError.class, LlamaLoader::initialize); - // Every manifest backend must have been attempted (and failed cleanly): the final - // error lists each backend resource path plus the default path as tried. + // Every backend present in the fixture tree was attempted (and failed cleanly); backends + // with no directory there (sycl, opencl, cpu, ...) are not listed as tried. String message = error.getMessage(); - String base = "/net/ladenthin/llama/Linux/" + FIXTURE_ARCH; - assertTrue(message.contains(base + "/fakegpu"), message); - assertTrue(message.contains(base + "/missingextra"), message); - assertTrue(message.contains(base + "/nodir"), message); - assertTrue(message.contains(base + "/fallbackgpu"), message); - assertTrue(message.contains(base), message); - // fakegpu: the extra file is extracted into the per-backend temp subdir before its + String base = "net/ladenthin/llama/Linux/" + FIXTURE_ARCH; + assertTrue(message.indexOf(base + "/cuda13") < message.indexOf(base + "/rocm"), message); + assertTrue(message.indexOf(base + "/rocm") < message.indexOf(base + "/vulkan"), message); + assertFalse(message.contains(base + "/cpu"), message); + // cuda13: the extra file is extracted into the per-backend temp subdir before its // load fails; the main library is never reached for that backend. assertTrue(Files.isRegularFile( - tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "fakegpu").resolve("libextra.so"))); + tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "cuda13").resolve("libextra.so"))); assertFalse(Files.exists( - tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "fakegpu").resolve("libjllama.so"))); - // fallbackgpu (no extras): its main library is extracted, then fails to load. - assertTrue(Files.isRegularFile(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "fallbackgpu") - .resolve("libjllama.so"))); + tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "cuda13").resolve("libjllama.so"))); + // vulkan (no extras): its main library is extracted, then fails to load. + assertTrue(Files.isRegularFile( + tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "vulkan").resolve("libjllama.so"))); } @Test - public void autoModeLoadsFirstWorkingBackendAndSkipsResidentClash() throws java.io.IOException { + public void loadsFirstWorkingBackendAndSkipsResidentClash() throws java.io.IOException { assumeLinuxFixtureTree(); assumeX86_64(); System.setProperty(ARCH_PROP, "backendtest-ok"); @@ -120,56 +118,43 @@ public void autoModeLoadsFirstWorkingBackendAndSkipsResidentClash() throws java. tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "stale").resolve("nested"); Files.createDirectories(stale); Files.write(stale.resolve("libjllama.so"), new byte[] {1}); - // Must succeed: extrafail's real extra library loads but its fake main library - // fails; clash declares the same extra file name and is skipped (already resident, - // so its temp dir is never even created); workinggpu's real dummy library loads. + // Must succeed: cuda13's real extra library loads but its fake main library fails; + // rocm lists the same extra file name and is skipped (already resident, so its temp + // dir is never even created); vulkan's real dummy library loads. LlamaLoader.initialize(); - assertTrue(Files.isRegularFile(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "extrafail") - .resolve("libextra.so"))); - assertFalse(Files.exists(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "clash"))); - assertTrue(Files.isRegularFile(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "workinggpu") - .resolve("libjllama.so"))); + assertTrue(Files.isRegularFile( + tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "cuda13").resolve("libextra.so"))); + assertFalse(Files.exists(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "rocm"))); + assertTrue(Files.isRegularFile( + tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "vulkan").resolve("libjllama.so"))); } @Test - public void withoutManifestNoBackendIsAttempted() { + public void withoutAnyBackendTheErrorNamesTheMissingNativesJar() { assumeLinuxFixtureTree(); System.setProperty(ARCH_PROP, "backendtest-none"); UnsatisfiedLinkError error = assertThrows(UnsatisfiedLinkError.class, LlamaLoader::initialize); - // No manifest resource exists for this arch folder: the loader must take the - // unchanged legacy path with zero backend attempts. String message = error.getMessage(); assertFalse(message.contains("backendtest-none/"), message); assertTrue(message.contains("os.arch=backendtest-none"), message); + assertTrue(message.contains("natives jar"), message); } @Test public void forcedBackendFailsLoudInsteadOfFallingBack() { assumeLinuxFixtureTree(); - System.setProperty(BACKEND_PROP, "fakegpu"); + System.setProperty(BACKEND_PROP, "cuda13"); UnsatisfiedLinkError error = assertThrows(UnsatisfiedLinkError.class, LlamaLoader::initialize); - assertTrue(error.getMessage().contains("Forced native backend 'fakegpu'"), error.getMessage()); + assertTrue(error.getMessage().contains("Forced native backend 'cuda13'"), error.getMessage()); + // Only the forced backend was attempted. + assertFalse(Files.exists(tempDir.resolve(LlamaLoader.BACKEND_TEMP_DIR_PREFIX + "vulkan"))); } @Test - public void forcedUnknownBackendFailsLoud() { + public void forcedAbsentBackendFailsLoud() { assumeLinuxFixtureTree(); - System.setProperty(BACKEND_PROP, "does-not-exist"); + System.setProperty(BACKEND_PROP, "cpu"); UnsatisfiedLinkError error = assertThrows(UnsatisfiedLinkError.class, LlamaLoader::initialize); - assertTrue(error.getMessage().contains("Forced native backend 'does-not-exist'"), error.getMessage()); - } - - @Test - public void forcedDefaultSkipsAllManifestBackends() { - assumeLinuxFixtureTree(); - System.setProperty(BACKEND_PROP, "default"); - UnsatisfiedLinkError error = assertThrows(UnsatisfiedLinkError.class, LlamaLoader::initialize); - // The default library is also missing from the fixture tree (its resource path is - // then not even listed as tried), so loading still fails — but without any backend - // attempt: no backend path may appear in the tried list. - String message = error.getMessage(); - assertFalse(message.contains("/fakegpu"), message); - assertFalse(message.contains("/fallbackgpu"), message); - assertTrue(message.contains("os.arch=" + FIXTURE_ARCH), message); + assertTrue(error.getMessage().contains("Forced native backend 'cpu'"), error.getMessage()); } } diff --git a/llama/src/test/java/net/ladenthin/llama/loader/LlamaLoaderTest.java b/llama/src/test/java/net/ladenthin/llama/loader/LlamaLoaderTest.java index 8e6d29676..9c6d3c75a 100644 --- a/llama/src/test/java/net/ladenthin/llama/loader/LlamaLoaderTest.java +++ b/llama/src/test/java/net/ladenthin/llama/loader/LlamaLoaderTest.java @@ -31,14 +31,15 @@ + "contentsEquals performs a correct byte-level stream comparison " + "including BufferedInputStream wrapping and length mismatches; getTempDir " + "honours the 'net.ladenthin.llama.tmpdir' system-property override; and " - + "getNativeResourcePath produces the expected classpath resource prefix; and " + + "getNativeResourcePath produces the expected classpath resource prefix; parseExtras reads a " + + "backend extras file; BACKEND_PRIORITY ends with msvc then cpu; and " + "resourceMatchesFile compares a classpath resource to an on-disk file byte-for-byte; and extractFile extracts a resource, reuses an already-identical copy without rewriting it, and replaces one whose content differs.") public class LlamaLoaderTest { private static final String TMPDIR_PROP = LlamaSystemProperties.PREFIX + ".tmpdir"; /** A small file present on the test classpath, used as a byte-comparison fixture. */ - private static final String EXISTING_TEST_RESOURCE = "/images/test-image.jpg"; + private static final String EXISTING_TEST_RESOURCE = "images/test-image.jpg"; private String previousTmpDir; @@ -115,114 +116,31 @@ public void testShouldCleanPathGgmlPrefix() { } // ------------------------------------------------------------------------- - // parseBackendManifest + // parseExtras / BACKEND_PRIORITY // ------------------------------------------------------------------------- - private static java.util.List parseManifest(String content) throws IOException { - return LlamaLoader.parseBackendManifest(new java.io.BufferedReader(new java.io.StringReader(content))); + private static java.util.List parseExtras(String content) throws IOException { + return LlamaLoader.parseExtras(new java.io.BufferedReader(new java.io.StringReader(content))); } @Test - public void testParseBackendManifestPreservesPriorityOrder() throws IOException { - java.util.List entries = parseManifest("cuda13\nvulkan\nopencl\n"); - assertEquals(3, entries.size()); - assertEquals("cuda13", entries.get(0).name); - assertEquals("vulkan", entries.get(1).name); - assertEquals("opencl", entries.get(2).name); + public void testParseExtrasKeepsOrderAndSkipsCommentsBlankLinesAndCrLf() throws IOException { + assertEquals( + java.util.Arrays.asList("OpenCL.dll", "second.dll"), + parseExtras("# loaded first\r\n\n OpenCL.dll \r\n\tsecond.dll\n# tail\n")); } @Test - public void testParseBackendManifestTokenizesExtraFiles() throws IOException { - java.util.List entries = parseManifest("openvino OpenCL.dll second.dll\n"); - assertEquals(1, entries.size()); - assertEquals("openvino", entries.get(0).name); - assertEquals(java.util.Arrays.asList("OpenCL.dll", "second.dll"), entries.get(0).extraFiles); + public void testParseExtrasEmptyContent() throws IOException { + assertTrue(parseExtras("").isEmpty()); } @Test - public void testParseBackendManifestNoExtraFiles() throws IOException { - java.util.List entries = parseManifest("vulkan\n"); - assertTrue(entries.get(0).extraFiles.isEmpty()); - } - - @Test - public void testParseBackendManifestSkipsCommentsAndBlankLines() throws IOException { - java.util.List entries = - parseManifest("# priority order\n\n \ncuda13\n# tail comment\n"); - assertEquals(1, entries.size()); - assertEquals("cuda13", entries.get(0).name); - } - - @Test - public void testParseBackendManifestToleratesCrLfAndIndentation() throws IOException { - // BufferedReader.readLine strips \r\n; leading/trailing whitespace is trimmed per line. - java.util.List entries = parseManifest(" cuda13\r\n\tvulkan \r\n"); - assertEquals(2, entries.size()); - assertEquals("cuda13", entries.get(0).name); - assertEquals("vulkan", entries.get(1).name); - } - - @Test - public void testParseBackendManifestEmptyContent() throws IOException { - assertTrue(parseManifest("").isEmpty()); - } - - // ------------------------------------------------------------------------- - // selectBackendCandidates / isForcedBackend - // ------------------------------------------------------------------------- - - @Test - public void testSelectBackendCandidatesWithoutOverrideReturnsManifestOrder() throws IOException { - java.util.List manifest = parseManifest("cuda13\nvulkan\n"); - assertEquals(manifest, LlamaLoader.selectBackendCandidates(manifest, null)); - } - - @Test - public void testSelectBackendCandidatesDefaultSkipsAllBackends() throws IOException { - java.util.List manifest = parseManifest("cuda13\n"); - assertTrue(LlamaLoader.selectBackendCandidates(manifest, "default").isEmpty()); - } - - @Test - public void testSelectBackendCandidatesCpuAliasSkipsAllBackends() throws IOException { - java.util.List manifest = parseManifest("cuda13\n"); - assertTrue(LlamaLoader.selectBackendCandidates(manifest, "cpu").isEmpty()); - } - - @Test - public void testSelectBackendCandidatesForcedKnownBackendKeepsItsExtraFiles() throws IOException { - java.util.List manifest = parseManifest("cuda13\nopenvino OpenCL.dll\n"); - java.util.List selected = LlamaLoader.selectBackendCandidates(manifest, "openvino"); - assertEquals(1, selected.size()); - assertEquals("openvino", selected.get(0).name); - assertEquals(java.util.Collections.singletonList("OpenCL.dll"), selected.get(0).extraFiles); - } - - @Test - public void testSelectBackendCandidatesForcedUnknownBackendIsSynthesized() throws IOException { - // A backend name absent from the manifest is still attempted (stale-manifest override), - // just without extra files; the loader fails loud if its library is missing. - java.util.List manifest = parseManifest("cuda13\n"); - java.util.List selected = LlamaLoader.selectBackendCandidates(manifest, "rocm"); - assertEquals(1, selected.size()); - assertEquals("rocm", selected.get(0).name); - assertTrue(selected.get(0).extraFiles.isEmpty()); - } - - @Test - public void testIsForcedBackendUnsetIsNotForced() { - assertFalse(LlamaLoader.isForcedBackend(null)); - } - - @Test - public void testIsForcedBackendDefaultAndCpuAreNotForced() { - assertFalse(LlamaLoader.isForcedBackend("default")); - assertFalse(LlamaLoader.isForcedBackend("cpu")); - } - - @Test - public void testIsForcedBackendSpecificBackendIsForced() { - assertTrue(LlamaLoader.isForcedBackend("cuda13")); + public void testBackendPriorityTriesAcceleratorsBeforeCpuAndMsvcBeforeCpu() { + java.util.List priority = LlamaLoader.BACKEND_PRIORITY; + assertEquals("cpu", priority.get(priority.size() - 1)); + assertEquals("msvc", priority.get(priority.size() - 2)); + assertEquals("cuda13", priority.get(0)); } @Test @@ -247,7 +165,7 @@ public void resourceMatchesFileFalseWhenResourceAbsent() throws IOException { try { java.nio.file.Files.write(tmp, new byte[] {1, 2, 3}); // A missing classpath resource must compare as "not matching", never throw. - assertFalse(LlamaLoader.resourceMatchesFile("/net/ladenthin/llama/does-not-exist.bin", tmp)); + assertFalse(LlamaLoader.resourceMatchesFile("net/ladenthin/llama/does-not-exist.bin", tmp)); } finally { java.nio.file.Files.deleteIfExists(tmp); } @@ -258,7 +176,8 @@ public void resourceMatchesFileTrueWhenBytesIdentical() throws IOException { // The fast-path reuse predicate: a present resource and a byte-identical on-disk copy match. java.nio.file.Path tmp = java.nio.file.Files.createTempFile("llama-loader-test", ".bin"); try { - try (java.io.InputStream in = LlamaLoader.class.getResourceAsStream(EXISTING_TEST_RESOURCE)) { + try (java.io.InputStream in = + LlamaLoader.class.getClassLoader().getResourceAsStream(EXISTING_TEST_RESOURCE)) { assertNotNull(in, "fixture must be on the test classpath: " + EXISTING_TEST_RESOURCE); java.nio.file.Files.copy(in, tmp, java.nio.file.StandardCopyOption.REPLACE_EXISTING); } @@ -274,7 +193,8 @@ public void resourceMatchesFileFalseWhenContentDiffers() throws IOException { // so a stale/partial file is never mistaken for the shipped library on the reuse fast path. java.nio.file.Path tmp = java.nio.file.Files.createTempFile("llama-loader-test", ".bin"); try { - try (java.io.InputStream in = LlamaLoader.class.getResourceAsStream(EXISTING_TEST_RESOURCE)) { + try (java.io.InputStream in = + LlamaLoader.class.getClassLoader().getResourceAsStream(EXISTING_TEST_RESOURCE)) { assertNotNull(in, "fixture must be on the test classpath: " + EXISTING_TEST_RESOURCE); java.nio.file.Files.copy(in, tmp, java.nio.file.StandardCopyOption.REPLACE_EXISTING); } @@ -354,9 +274,10 @@ public void testGetTempDirUsesOverrideProperty() { // ------------------------------------------------------------------------- @Test - public void testGetNativeResourcePathStartsWithSlash() { + public void testGetNativeResourcePathIsClassLoaderRelative() { + // Resolved through the ClassLoader, which rejects a leading slash (see LlamaLoader.resource). String path = LlamaLoader.getNativeResourcePath(); - assertTrue(path.startsWith("/"), "Resource path should start with '/'"); + assertFalse(path.startsWith("/"), "Resource path must not start with '/': " + path); } @Test @@ -376,7 +297,7 @@ public void testGetNativeResourcePathContainsOsAndArch() { /** * Regression for the layered-restructure bug: the native-library classpath - * root is fixed at {@code /net/ladenthin/llama//} by CMakeLists + + * root is fixed at {@code net/ladenthin/llama//} by CMakeLists + * the publish workflow, so it must NOT track the loader's own Java package * (which moved to {@code net.ladenthin.llama.loader}). Deriving it from * {@code LlamaLoader.class.getPackage()} produced {@code .../llama/loader/...}, @@ -387,11 +308,11 @@ public void testGetNativeResourcePathContainsOsAndArch() { public void testGetNativeResourcePathIsPackageIndependent() { String path = LlamaLoader.getNativeResourcePath(); String osArch = OSInfo.getNativeLibFolderPathForCurrentOS(); - assertEquals("/net/ladenthin/llama/" + osArch, path); + assertEquals("net/ladenthin/llama/" + osArch, path); assertFalse( path.contains("/loader/"), "Resource path must not include the loader subpackage — the native libs live at " - + "/net/ladenthin/llama//, not under the loader package: " + path); + + "net/ladenthin/llama//, not under the loader package: " + path); } // ------------------------------------------------------------------------- @@ -408,7 +329,7 @@ public void testGetNativeResourcePathIsPackageIndependent() { Path extractDir; private static byte[] testResourceBytes() throws IOException { - try (InputStream in = LlamaLoader.class.getResourceAsStream(EXISTING_TEST_RESOURCE)) { + try (InputStream in = LlamaLoader.class.getClassLoader().getResourceAsStream(EXISTING_TEST_RESOURCE)) { assertNotNull(in, "fixture must be on the test classpath: " + EXISTING_TEST_RESOURCE); java.io.ByteArrayOutputStream out = new java.io.ByteArrayOutputStream(); byte[] buffer = new byte[8192]; @@ -422,7 +343,7 @@ private static byte[] testResourceBytes() throws IOException { @Test public void extractFileWritesTheResourceIntoTheTargetDirectory() throws IOException { - Path extracted = LlamaLoader.extractFile("/images", "test-image.jpg", extractDir.toString()); + Path extracted = LlamaLoader.extractFile("images", "test-image.jpg", extractDir.toString()); assertNotNull(extracted); assertEquals(extractDir.resolve("test-image.jpg"), extracted); assertArrayEquals(testResourceBytes(), Files.readAllBytes(extracted)); @@ -430,12 +351,12 @@ public void extractFileWritesTheResourceIntoTheTargetDirectory() throws IOExcept @Test public void extractFileReturnsNullWhenTheResourceIsAbsent() { - assertNull(LlamaLoader.extractFile("/images", "no-such-file.bin", extractDir.toString())); + assertNull(LlamaLoader.extractFile("images", "no-such-file.bin", extractDir.toString())); } @Test public void extractFileLeavesNoTemporaryFileBehind() throws IOException { - LlamaLoader.extractFile("/images", "test-image.jpg", extractDir.toString()); + LlamaLoader.extractFile("images", "test-image.jpg", extractDir.toString()); try (java.util.stream.Stream entries = Files.list(extractDir)) { assertEquals( 1L, @@ -457,7 +378,7 @@ public void extractFileReusesAnIdenticalCopyWithoutRewritingIt() throws IOExcept FileTime seeded = FileTime.fromMillis(1_000_000_000L); Files.setLastModifiedTime(target, seeded); - Path extracted = LlamaLoader.extractFile("/images", "test-image.jpg", extractDir.toString()); + Path extracted = LlamaLoader.extractFile("images", "test-image.jpg", extractDir.toString()); assertEquals(target, extracted); assertEquals(seeded, Files.getLastModifiedTime(target)); @@ -471,7 +392,7 @@ public void extractFileReplacesACopyWhoseContentDiffers() throws IOException { Path target = extractDir.resolve("test-image.jpg"); Files.write(target, "stale content from an older release".getBytes(StandardCharsets.UTF_8)); - Path extracted = LlamaLoader.extractFile("/images", "test-image.jpg", extractDir.toString()); + Path extracted = LlamaLoader.extractFile("images", "test-image.jpg", extractDir.toString()); assertEquals(target, extracted); assertArrayEquals(testResourceBytes(), Files.readAllBytes(target)); diff --git a/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryLoadSmokeTest.java b/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryLoadSmokeTest.java index 3dffac9ed..7fd9ccb6a 100644 --- a/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryLoadSmokeTest.java +++ b/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryLoadSmokeTest.java @@ -42,9 +42,10 @@ *

The test self-skips when {@code libjllama} is not on the classpath (a * pure-Java checkout with no native build), so a plain {@code mvn test} stays * green without a CMake build; CI's {@code test-java-*} jobs and any local build - * have the library and run it for real. The presence check uses the canonical - * resource layout directly (not {@link LlamaLoader#getNativeResourcePath()}) so - * a regression in that method cannot silently skip this guard. + * have the library and run it for real. The presence check + * ({@link NativeLibraryPresence}) spells out the canonical resource layout rather + * than calling {@link LlamaLoader#getNativeResourcePath()}, so a regression in that + * method cannot silently skip this guard. */ @ClaudeGenerated( purpose = "Model-free native-load smoke: force LlamaModel. so System.load + JNI_OnLoad " @@ -53,15 +54,9 @@ + "cleanly when libjllama is not on the classpath.") class NativeLibraryLoadSmokeTest { - private static boolean nativeLibraryOnClasspath() { - String resource = "/net/ladenthin/llama/" + OSInfo.getNativeLibFolderPathForCurrentOS() + "/" - + System.mapLibraryName("jllama"); - return NativeLibraryLoadSmokeTest.class.getResource(resource) != null; - } - @Test void loadingNativeLibraryRunsJniOnLoadWithoutError() { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping native-load smoke"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping native-load smoke"); assertDoesNotThrow( () -> Class.forName("net.ladenthin.llama.LlamaModel"), "LlamaModel. must load the native library and JNI_OnLoad must resolve " @@ -87,7 +82,8 @@ void loadingNativeLibraryRunsJniOnLoadWithoutError() { */ @Test void nativeBuildInfoMatchesPinnedVersionConstant() { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping native build-info check"); + assumeTrue( + NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping native build-info check"); String buildInfo = LlamaModel.getLlamaCppBuildInfo(); assertNotNull(buildInfo, "getLlamaCppBuildInfo() must return the linked llama.cpp build identifier"); assertTrue( @@ -125,7 +121,8 @@ void nativeBuildInfoMatchesPinnedVersionConstant() { */ @Test void quantizerNativeEntryPointResolves() { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping quantizer linkage check"); + assumeTrue( + NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping quantizer linkage check"); assertThrows( LlamaException.class, () -> LlamaQuantizer.quantize( @@ -158,7 +155,7 @@ void quantizerNativeEntryPointResolves() { */ @Test void jsonSchemaToGrammarWorksWithoutAModel() { - assumeTrue(nativeLibraryOnClasspath(), "libjllama not on classpath — skipping grammar check"); + assumeTrue(NativeLibraryPresence.onClasspath(), "libjllama not on classpath — skipping grammar check"); String grammar = LlamaModel.jsonSchemaToGrammar("{\"type\":\"object\",\"properties\":{\"a\":{\"type\":\"string\"}}}"); diff --git a/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryPresence.java b/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryPresence.java new file mode 100644 index 000000000..d0e996a71 --- /dev/null +++ b/llama/src/test/java/net/ladenthin/llama/loader/NativeLibraryPresence.java @@ -0,0 +1,34 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// +// SPDX-License-Identifier: MIT + +package net.ladenthin.llama.loader; + +/** + * Whether a native library for this platform is on the test classpath, i.e. whether the + * native-backed tests can run. The single copy of this check: it spells out the resource layout + * ({@code net/ladenthin/llama////}) instead of asking the loader, so a loader + * regression cannot make these tests skip silently, and a layout change needs one edit here + * instead of one per test class (three had drifted to the old flat layout once). + */ +public final class NativeLibraryPresence { + + private NativeLibraryPresence() {} + + /** + * Checks every backend directory the loader would try. + * + * @return whether any of them holds the {@code jllama} library for this platform + */ + public static boolean onClasspath() { + ClassLoader loader = NativeLibraryPresence.class.getClassLoader(); + for (String backend : LlamaLoader.BACKEND_PRIORITY) { + String resource = "net/ladenthin/llama/" + OSInfo.getNativeLibFolderPathForCurrentOS() + "/" + backend + "/" + + System.mapLibraryName("jllama"); + if (loader.getResource(resource) != null) { + return true; + } + } + return false; + } +} diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/jllama-extras.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/jllama-extras.txt new file mode 100644 index 000000000..ba3d1529b --- /dev/null +++ b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/jllama-extras.txt @@ -0,0 +1,6 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# Test fixture for BackendLoadTest: files loaded, in order, before this backend's library. +libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/clash/libextra.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/libextra.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/clash/libextra.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/extrafail/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/extrafail/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/cuda13/libjllama.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/jllama-backends.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/jllama-backends.txt deleted file mode 100644 index 5cf2840b0..000000000 --- a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/jllama-backends.txt +++ /dev/null @@ -1,11 +0,0 @@ -# SPDX-FileCopyrightText: 2026 Bernard Ladenthin -# -# SPDX-License-Identifier: MIT -# -# Test fixture for BackendManifestLoadTest's success scenario: extrafail's real -# extra library loads but its fake main library fails; clash declares the same -# extra file name and must be skipped (already resident); workinggpu's real -# dummy library loads successfully. -extrafail libextra.so -clash libextra.so -workinggpu diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/jllama-extras.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/jllama-extras.txt new file mode 100644 index 000000000..ba3d1529b --- /dev/null +++ b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/jllama-extras.txt @@ -0,0 +1,6 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# Test fixture for BackendLoadTest: files loaded, in order, before this backend's library. +libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/extrafail/libextra.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/libextra.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/extrafail/libextra.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/clash/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/clash/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/rocm/libjllama.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/workinggpu/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/vulkan/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/workinggpu/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest-ok/vulkan/libjllama.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/jllama-extras.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/jllama-extras.txt new file mode 100644 index 000000000..ba3d1529b --- /dev/null +++ b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/jllama-extras.txt @@ -0,0 +1,6 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# Test fixture for BackendLoadTest: files loaded, in order, before this backend's library. +libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fakegpu/libextra.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/libextra.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fakegpu/libextra.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/libextra.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fakegpu/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fakegpu/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/cuda13/libjllama.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/jllama-backends.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/jllama-backends.txt deleted file mode 100644 index 85863212c..000000000 --- a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/jllama-backends.txt +++ /dev/null @@ -1,10 +0,0 @@ -# SPDX-FileCopyrightText: 2026 Bernard Ladenthin -# -# SPDX-License-Identifier: MIT -# -# Test fixture for BackendManifestLoadTest: a backend manifest whose fake libraries -# can never actually load, exercising every failure branch of the fallback chain. -fakegpu libextra.so -missingextra libmissing.so -nodir -fallbackgpu diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/jllama-extras.txt b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/jllama-extras.txt new file mode 100644 index 000000000..33e3dbcf1 --- /dev/null +++ b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/jllama-extras.txt @@ -0,0 +1,6 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# Test fixture for BackendLoadTest: files loaded, in order, before this backend's library. +libmissing.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/missingextra/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/missingextra/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/rocm/libjllama.so diff --git a/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fallbackgpu/libjllama.so b/llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/vulkan/libjllama.so similarity index 100% rename from llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/fallbackgpu/libjllama.so rename to llama/src/test/resources/net/ladenthin/llama/Linux/backendtest/vulkan/libjllama.so diff --git a/pom.xml b/pom.xml index 0d7196505..584438479 100644 --- a/pom.xml +++ b/pom.xml @@ -59,6 +59,7 @@ SPDX-License-Identifier: MIT llama llama-langchain4j llama-kotlin + llama-platform From 551f527d28bd8f604ebc2ca6183828bf94d8c1a6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 30 Sep 2026 22:38:38 +0000 Subject: [PATCH 02/22] Build checks as a tested library; shared files, release gate, leaner workflow Shared with java-llama.cpp, BitcoinAddressFinder, srcmorph and streambuffer (listed in .github/shared-files.sha256, checked by the new shared-files job, which fails on a copy changed in one repository alone and warns on a sibling whose copy differs): - .github/buildcheck/{workflow,releasegate,sharedfiles}.py + unit tests (stdlib-only Python), check-release-gate.py and check-shared-files.py; - print-crash-logs.sh (replaces the crash-log steps pasted into every test job) and verify-signing-key.sh (the body of the verify-signing-key job); - the files that were already kept identical by hand, now in the manifest. The release gate: every job of publish.yml gates both publish jobs unless .github/release-gate-exemptions.txt names it with a reason. It found vmlens gating nothing; vmlens and shared-files now gate both publish jobs. This repository additionally: - moves check-natives / verify-native-deps / verify-hip-offload-compressed into .github/buildcheck with unit tests (synthesized ELF/PE/Mach-O, workflow fixtures). check-natives now also checks that package waits for every natives build, that the all-backends fat jars derived from natives.csv are each uploaded for and launched by a smoke job and named in the agent jar's Class-Path and the README, CMake's backend names, the dependency allowlists, the README rows, and that every *_MODEL_NAME of the workflow is in models.csv. package-fatjars.sh asks for the targets instead of hard-coding four; - holds the Android libraries to nativedeps (bionic-only + libOpenCL.so for the OpenCL flavour, 16 KB LOAD alignment read from the ELF headers) instead of an inline allowlist in the AAR job; - makes the Java tests default to CI's model set (TestConstants DEFAULT_*, new net.ladenthin.llama.train.model default stories260K.gguf), asserted equal to models.csv in both directions, and drops the 48 -D model lines and 7 env vars from the workflow; - adds composite actions restore-models (16 call sites) and install-sccache-windows (10), print-host-info.sh (18 CPU-info steps), drops validate-models.bat (Windows runs the bash script, now xxd-free), hoists USE_CACHE / SCCACHE_WEBDAV_ENDPOINT, GRADLE_VERSION and JAVA_VERSION, and sets scorecard.yml to permissions: read-all like the siblings. publish.yml: 4293 -> ~3530 lines, job and check names unchanged. Verified: 60+ build-check unit tests, all check scripts, actionlint, reuse lint, package-fatjars.sh on locally built jars, verify-signing-key.sh with a throwaway key, TestConstantsTest (mutation-checked). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .../install-sccache-windows/action.yml | 32 + .github/actions/restore-models/action.yml | 33 + .github/buildcheck/__init__.py | 14 + .github/buildcheck/hipoffload.py | 82 ++ .github/buildcheck/models.py | 24 + .github/buildcheck/nativedeps.py | 252 ++++ .github/buildcheck/natives.py | 237 ++++ .github/buildcheck/releasegate.py | 60 + .github/buildcheck/sharedfiles.py | 137 +++ .github/buildcheck/tests/__init__.py | 3 + .github/buildcheck/tests/helpers.py | 26 + .github/buildcheck/tests/test_hipoffload.py | 41 + .github/buildcheck/tests/test_models.py | 28 + .github/buildcheck/tests/test_nativedeps.py | 159 +++ .github/buildcheck/tests/test_natives.py | 181 +++ .github/buildcheck/tests/test_releasegate.py | 61 + .github/buildcheck/tests/test_sharedfiles.py | 112 ++ .github/buildcheck/tests/test_workflow.py | 70 ++ .github/buildcheck/workflow.py | 120 ++ .github/check-natives.py | 133 +- .github/check-release-gate.py | 33 + .github/check-shared-files.py | 20 + .github/package-fatjars.sh | 14 +- .github/print-crash-logs.sh | 48 + .github/print-host-info.sh | 37 + .github/release-gate-exemptions.txt | 19 + .github/shared-files.sha256 | 31 + .github/sign-fatjars.sh | 5 +- .github/validate-models.bat | 85 -- .github/validate-models.sh | 11 +- .github/verify-bytecode-version.sh | 5 +- .github/verify-hip-offload-compressed.py | 79 +- .github/verify-native-deps.py | 207 +--- .github/verify-signing-key.sh | 100 ++ .github/workflows/publish.yml | 1066 +++-------------- .github/workflows/scorecard.yml | 3 +- .gitignore | 3 + CHANGELOG.md | 19 + CLAUDE.md | 222 ++-- README.md | 13 +- TODO.md | 11 +- .../ladenthin/llama/LlamaEmbeddingsTest.java | 14 +- .../llama/LlamaTrainerIntegrationTest.java | 14 +- .../llama/MultimodalIntegrationTest.java | 6 +- .../net/ladenthin/llama/TestConstants.java | 37 +- .../ladenthin/llama/TestConstantsTest.java | 45 + .../ladenthin/llama/TtsIntegrationTest.java | 12 +- 47 files changed, 2430 insertions(+), 1534 deletions(-) create mode 100644 .github/actions/install-sccache-windows/action.yml create mode 100644 .github/actions/restore-models/action.yml create mode 100644 .github/buildcheck/__init__.py create mode 100644 .github/buildcheck/hipoffload.py create mode 100644 .github/buildcheck/models.py create mode 100755 .github/buildcheck/nativedeps.py create mode 100644 .github/buildcheck/natives.py create mode 100644 .github/buildcheck/releasegate.py create mode 100644 .github/buildcheck/sharedfiles.py create mode 100644 .github/buildcheck/tests/__init__.py create mode 100644 .github/buildcheck/tests/helpers.py create mode 100644 .github/buildcheck/tests/test_hipoffload.py create mode 100644 .github/buildcheck/tests/test_models.py create mode 100644 .github/buildcheck/tests/test_nativedeps.py create mode 100644 .github/buildcheck/tests/test_natives.py create mode 100644 .github/buildcheck/tests/test_releasegate.py create mode 100644 .github/buildcheck/tests/test_sharedfiles.py create mode 100644 .github/buildcheck/tests/test_workflow.py create mode 100644 .github/buildcheck/workflow.py mode change 100644 => 100755 .github/check-natives.py create mode 100755 .github/check-release-gate.py create mode 100755 .github/check-shared-files.py create mode 100755 .github/print-crash-logs.sh create mode 100755 .github/print-host-info.sh create mode 100644 .github/release-gate-exemptions.txt create mode 100644 .github/shared-files.sha256 delete mode 100644 .github/validate-models.bat mode change 100644 => 100755 .github/verify-hip-offload-compressed.py create mode 100755 .github/verify-signing-key.sh diff --git a/.github/actions/install-sccache-windows/action.yml b/.github/actions/install-sccache-windows/action.yml new file mode 100644 index 000000000..35f0da5cc --- /dev/null +++ b/.github/actions/install-sccache-windows/action.yml @@ -0,0 +1,32 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +# Puts sccache on PATH on a Windows runner, from the release zip (x86_64, or the native +# aarch64-pc-windows-msvc build on windows-11-arm, where it wraps clang-cl). The caller keeps the +# `if:` (USE_CACHE and a Depot token present) and `continue-on-error: true`: a failed install is +# only a slower build, because build.bat probes sccache before trusting it and retries a build that +# fails with it as the launcher once uncached (CLAUDE.md, "sccache on every Windows Ninja job"). +# Keep `version` equal to SCCACHE_DL_VERSION in .github/build.sh (the Linux fetch). +name: Install sccache (Windows) +description: Download the sccache release zip matching the runner's architecture and add it to PATH. +inputs: + version: + description: sccache release to install. + required: false + default: '0.18.0' +runs: + using: composite + steps: + - name: Install sccache ${{ inputs.version }} + shell: pwsh + env: + SCCACHE_VERSION: ${{ inputs.version }} + run: | + $arch = if ($env:RUNNER_ARCH -eq 'ARM64') { 'aarch64' } else { 'x86_64' } + $rel = "sccache-v$env:SCCACHE_VERSION-$arch-pc-windows-msvc" + $url = "https://github.com/mozilla/sccache/releases/download/v$env:SCCACHE_VERSION/$rel.zip" + Write-Host "Downloading $url" + Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" + Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force + Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" diff --git a/.github/actions/restore-models/action.yml b/.github/actions/restore-models/action.yml new file mode 100644 index 000000000..c1a59dc4c --- /dev/null +++ b/.github/actions/restore-models/action.yml @@ -0,0 +1,33 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +# Restores the shared GGUF model cache and validates it -- the one place a job gets its models. +# The download-models job is the cache's ONLY writer (actions/cache, same key); every consumer +# uses this restore-only action, so a job running on a cache miss can never re-save an empty or +# partial entry under the immutable key. enableCrossOsArchive matches the writer's entry version, +# so macOS and Windows restore the same ubuntu-built entry (without it the entries are per OS, and +# the unreachable Windows-side one was once found re-saved EMPTY after an eviction). The key is +# the hash of .github/models.csv, so editing the manifest creates a fresh complete entry. +# validate-models.sh then fails a partial or absent restore loudly instead of letting the +# model-backed tests self-skip. Needs the repository checked out. +name: Restore GGUF models +description: Restore the shared GGUF model cache (written by download-models) and validate every model of .github/models.csv. +inputs: + fail-on-cache-miss: + description: Fail at the restore already when the cache entry is missing ('true' in verify-model-cache). + required: false + default: 'false' +runs: + using: composite + steps: + - name: Restore shared GGUF model cache (populated by download-models; no re-download) + uses: actions/cache/restore@v6 + with: + path: models/ + key: gguf-models-${{ hashFiles('.github/models.csv') }} + enableCrossOsArchive: true + fail-on-cache-miss: ${{ inputs.fail-on-cache-miss }} + - name: Validate model files + shell: bash + run: bash .github/validate-models.sh diff --git a/.github/buildcheck/__init__.py b/.github/buildcheck/__init__.py new file mode 100644 index 000000000..e49ad24b7 --- /dev/null +++ b/.github/buildcheck/__init__.py @@ -0,0 +1,14 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Build checks as a library, so buildcheck/tests can test them. Standard library only. + +Two kinds of module live here. workflow.py, releasegate.py and sharedfiles.py (with their tests +and the check-*.py entry points next to this package) are kept BYTE-IDENTICAL in java-llama.cpp, +BitcoinAddressFinder, srcmorph and streambuffer: each repository lists them in +.github/shared-files.sha256, which its `shared-files` job checks (see sharedfiles.py). Every other +module is the repository's own. + +Run the tests from the repository root: + python3 -m unittest discover -s .github/buildcheck/tests -t .github +""" diff --git a/.github/buildcheck/hipoffload.py b/.github/buildcheck/hipoffload.py new file mode 100644 index 000000000..8a70d83a3 --- /dev/null +++ b/.github/buildcheck/hipoffload.py @@ -0,0 +1,82 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +"""Fail when a ROCm/HIP build ships its GPU code uncompressed. + +llama/CMakeLists.txt compiles ggml-hip with clang's --offload-compress, so every embedded +device-code bundle is a compressed "CCOB" bundle. An uncompressed bundle starts with the magic +"__CLANG_OFFLOAD_BUNDLE__"; one of those in the shipped library means the flag was lost and the +library is back to carrying every GPU target's code uncompressed (~1 GB on Windows). + +Usage (the CLI is .github/verify-hip-offload-compressed.py): verify-hip-offload-compressed.py ... + +Scans every jllama.dll / libjllama.so found, prints its size and the bundle counts, and exits +non-zero if a library has an uncompressed bundle, has no compressed bundle at all, or if no +library was found. Only the standard library, so it runs on any runner. +""" + +import os +import sys + +UNCOMPRESSED = b"__CLANG_OFFLOAD_BUNDLE__" +COMPRESSED = b"CCOB" +NAMES = ("jllama.dll", "libjllama.so") +CHUNK = 64 * 1024 * 1024 + + +def count(path, chunk=CHUNK): + """Counts both magics in one streaming pass (the file can be ~1 GB).""" + overlap = len(UNCOMPRESSED) - 1 + uncompressed = compressed = 0 + tail = b"" + with open(path, "rb") as f: + while True: + block = f.read(chunk) + if not block: + break + data = tail + block + # a match lying entirely inside `tail` was counted in the previous round + uncompressed += data.count(UNCOMPRESSED) - tail.count(UNCOMPRESSED) + compressed += data.count(COMPRESSED) - tail.count(COMPRESSED) + tail = data[-overlap:] + return uncompressed, compressed + + +def libraries(paths): + for p in paths: + if os.path.isfile(p): + yield p + for root, _, files in os.walk(p): + for name in files: + if name in NAMES: + yield os.path.join(root, name) + + +def main(argv): + if len(argv) < 2: + print("usage: verify-hip-offload-compressed.py ...", file=sys.stderr) + return 2 + found = failed = 0 + summary = [] + for lib in libraries(argv[1:]): + found += 1 + size = os.path.getsize(lib) + uncompressed, compressed = count(lib) + line = f"{lib}: {size / 1024 / 1024:.0f} MiB, {compressed} compressed / {uncompressed} uncompressed offload bundle(s)" + print(line) + summary.append(line) + if uncompressed: + print(f"::error::{lib} embeds {uncompressed} uncompressed GPU code bundle(s); is --offload-compress still set on ggml-hip?") + failed += 1 + elif not compressed: + print(f"::error::{lib} embeds no compressed GPU code bundle; was it built with GGML_HIP=ON?") + failed += 1 + if not found: + print(f"::error::no {' / '.join(NAMES)} found under {argv[1:]}") + return 2 + step_summary = os.environ.get("GITHUB_STEP_SUMMARY") + if step_summary: + with open(step_summary, "a", encoding="utf-8") as f: + f.write("### ROCm/HIP device code\n\n" + "\n".join(f"- `{s}`" for s in summary) + "\n") + return 1 if failed else 0 diff --git a/.github/buildcheck/models.py b/.github/buildcheck/models.py new file mode 100644 index 000000000..f223a97ff --- /dev/null +++ b/.github/buildcheck/models.py @@ -0,0 +1,24 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +"""The GGUF file names publish.yml's `env:` names (for the smoke scripts, the Android emulator jobs +and the integration jobs outside the llama module), checked against .github/models.csv -- the list +the download-models job fetches and validate-models.sh requires. A name the list lacks is a model +no job downloads: its consumer would fail or self-skip. (The llama module's tests default to the +list itself; TestConstantsTest checks that side.) +""" + +import re + +ENV_MODEL = re.compile(r'^ ([A-Z0-9_]+):\s*"?([^"\s]+\.gguf)"?\s*$', re.M) + + +def filenames(models_csv_text): + return {line.split(",", 1)[0].strip() for line in models_csv_text.splitlines() + if line.strip() and not line.startswith("#")} + + +def check(models_csv_text, workflow_text): + listed = filenames(models_csv_text) + return [f"publish.yml env {name}={value} is not a filename of .github/models.csv -- no job downloads it" + for name, value in ENV_MODEL.findall(workflow_text) if value not in listed] diff --git a/.github/buildcheck/nativedeps.py b/.github/buildcheck/nativedeps.py new file mode 100755 index 000000000..bd79fb07e --- /dev/null +++ b/.github/buildcheck/nativedeps.py @@ -0,0 +1,252 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +"""Fail when a shipped native library needs a runtime library it did not need before. + +Every jllama library is ONE file with llama.cpp and ggml linked in statically, so its dynamic +dependencies are exactly what a consumer's machine must provide. A new one is a silent break on +every machine that lacks it -- the case this guards against is ggml-rpc's RDMA transport, which +upstream switches on whenever the build host has libibverbs/librdma and which would make the +library unloadable without rdma-core. It reads the dependency list straight from the file (ELF +DT_NEEDED, PE import table, Mach-O LC_LOAD_DYLIB) with the standard library only, so it runs on +any runner and checks every architecture, including the ones binutils cannot read (Windows arm64, +Mach-O). + +Usage (the CLI is .github/verify-native-deps.py): + verify-native-deps.py + +Checks every library under /...////. The CPU builds (backend cpu, +metal, msvc) and the Android OpenCL build are held to the exact allowlist in ALLOWED: a dependency +outside it fails, and so does a CPU build without a list (a new platform must be listed +consciously). The other GPU backends are checked against DENIED only, because they legitimately need +their vendor runtime. Android libraries must also have every LOAD segment 16 KB aligned. +That every listed build arrived is merge-native-artifacts.sh's check. + +Exit codes: 0 clean, 1 violation, 2 nothing found to check. +""" + +import os +import struct +import sys + +# What each CPU library needed when this check was introduced (5.1.0 plus the RPC backend, +# which adds nothing: its sockets are libc/libSystem/WS2_32, all already present). +ALLOWED = { + "Linux/x86_64/cpu": {"libdl.so.2", "libgomp.so.1", "libpthread.so.0", "librt.so.1", "libstdc++.so.6", + "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld-linux-x86-64.so.2"}, + "Linux/aarch64/cpu": {"libgomp.so.1", "libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", + "ld-linux-aarch64.so.1"}, + "Linux/s390x/cpu": {"libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld64.so.1"}, + "Linux-Android/aarch64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, + "Linux-Android/x86_64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, + "Windows/x86_64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, + "Windows/x86/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, + "Windows/aarch64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll"}, + "Mac/aarch64/metal": {"/usr/lib/libc++.1.dylib", "/usr/lib/libSystem.B.dylib", + "/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation", + "/System/Library/Frameworks/Metal.framework/Versions/A/Metal", + "/System/Library/Frameworks/MetalKit.framework/Versions/A/MetalKit", + "/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate", + "/usr/lib/libobjc.A.dylib", + "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation", + "/System/Library/Frameworks/Security.framework/Versions/A/Security", + # KNOWN DEFECT, allowed only so this check reports NEW dependencies: the macOS + # build picks up the runner's Homebrew OpenSSL, so the shipped dylib does not load + # on a Mac without `brew install openssl@3`. See TODO.md ("macOS dylib links + # Homebrew OpenSSL"); remove these two lines with the fix. + "/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib", + "/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib"}, +} +# The Visual Studio generator build of the same compiler and runtime. +ALLOWED["Windows/x86_64/msvc"] = ALLOWED["Windows/x86_64/cpu"] +ALLOWED["Windows/x86/msvc"] = ALLOWED["Windows/x86/cpu"] +# The OpenCL AAR flavour: an app bundles no other native library, so the Android GPU build is held +# to an exact list as well -- the bionic system libraries plus the vendor ICD. (libomp.so and +# libc++_shared.so once shipped exactly this way and failed System.loadLibrary on every device.) +ALLOWED["Linux-Android/aarch64/opencl"] = ALLOWED["Linux-Android/aarch64/cpu"] | {"libOpenCL.so"} + +# Google Play's 16 KB page-size requirement (Android 15+ targets): every LOAD segment of an +# Android library must be aligned to a multiple of it. CMake pins -Wl,-z,max-page-size=16384. +ANDROID_PAGE_ALIGNMENT = 16384 +CPU_BACKENDS = ("cpu", "metal", "msvc") +LIBRARY_NAMES = ("libjllama.so", "jllama.dll", "libjllama.dylib") + +# Never acceptable in any artifact: libraries a consumer cannot be expected to have. +DENIED = ("libibverbs", "librdma", "rdma.dylib", "libmlx") + + +def elf_needed(data): + if data[:4] != b"\x7fELF": + raise ValueError("not an ELF file") + is64 = data[4] == 2 + end = "<" if data[5] == 1 else ">" + if is64: + shoff = struct.unpack_from(end + "Q", data, 0x28)[0] + shentsize, shnum = struct.unpack_from(end + "HH", data, 0x3A) + else: + shoff = struct.unpack_from(end + "I", data, 0x20)[0] + shentsize, shnum = struct.unpack_from(end + "HH", data, 0x2E) + sections = [] + for i in range(shnum): + off = shoff + i * shentsize + if is64: + _, sh_type, _, _, sh_offset, sh_size, sh_link = struct.unpack_from(end + "IIQQQQI", data, off) + else: + _, sh_type, _, _, sh_offset, sh_size, sh_link = struct.unpack_from(end + "IIIIIII", data, off) + sections.append((sh_type, sh_offset, sh_size, sh_link)) + out = [] + for sh_type, sh_offset, sh_size, sh_link in sections: + if sh_type != 6: # SHT_DYNAMIC + continue + strtab = sections[sh_link] + entry = 16 if is64 else 8 + for off in range(sh_offset, sh_offset + sh_size, entry): + tag, val = struct.unpack_from(end + ("qQ" if is64 else "iI"), data, off) + if tag == 0: + break + if tag == 1: # DT_NEEDED + start = strtab[1] + val + out.append(data[start:data.index(b"\0", start)].decode()) + return out + + +def elf_load_alignments(data): + """p_align of every PT_LOAD program header.""" + if data[:4] != b"\x7fELF": + raise ValueError("not an ELF file") + is64 = data[4] == 2 + end = "<" if data[5] == 1 else ">" + if is64: + phoff = struct.unpack_from(end + "Q", data, 0x20)[0] + phentsize, phnum = struct.unpack_from(end + "HH", data, 0x36) + else: + phoff = struct.unpack_from(end + "I", data, 0x1C)[0] + phentsize, phnum = struct.unpack_from(end + "HH", data, 0x2A) + out = [] + for i in range(phnum): + off = phoff + i * phentsize + if struct.unpack_from(end + "I", data, off)[0] == 1: # PT_LOAD + out.append(struct.unpack_from(end + ("Q" if is64 else "I"), data, off + (0x30 if is64 else 0x1C))[0]) + return out + + +def pe_imports(data): + if data[:2] != b"MZ": + raise ValueError("not a PE file") + pe = struct.unpack_from("///); `alignments` are its LOAD segment alignments (checked + for Android libraries).""" + failures = [f"{rel} needs {d}, which no consumer can be expected to have" for d in denied(deps)] + parts = rel.split("/") + key = "/".join(parts[-4:-1]) if len(parts) >= 4 else "" + if key.startswith("Linux-Android/"): + failures += [f"{rel}: LOAD alignment {a} is not a multiple of {ANDROID_PAGE_ALIGNMENT} " + f"(Google Play 16 KB page-size requirement)" for a in alignments if a % ANDROID_PAGE_ALIGNMENT] + if parts[-1] not in LIBRARY_NAMES: + return failures + allowed = ALLOWED.get(key) + if allowed is None: + if key.rsplit("/", 1)[-1] not in CPU_BACKENDS: + return failures + return failures + [f"{rel}: no dependency allowlist for '{key}' -- add one to ALLOWED"] + lowered = {a.lower() for a in allowed} + return failures + [f"{rel} needs {d}, which it did not need before (allowed: {sorted(allowed)})" + for d in deps if d.lower() not in lowered] + + +def main(argv): + if len(argv) != 2: + print(__doc__, file=sys.stderr) + return 2 + root = argv[1] + checked = 0 + failures = [] + for path in sorted(find_libraries(root)): + deps = dependencies(path) + checked += 1 + rel = os.path.relpath(path, root).replace(os.sep, "/") + alignments = () + if "/Linux-Android/" in "/" + rel: + with open(path, "rb") as f: + alignments = elf_load_alignments(f.read()) + print(f"{rel}: {' '.join(deps)}") + failures += violations(rel, deps, alignments) + if checked == 0: + print(f"no native library found under {root}", file=sys.stderr) + return 2 + for f in failures: + print(f"::error::{f}", file=sys.stderr) + print(f"{checked} native libraries checked, {len(failures)} violations") + return 1 if failures else 0 diff --git a/.github/buildcheck/natives.py b/.github/buildcheck/natives.py new file mode 100644 index 000000000..731b43f40 --- /dev/null +++ b/.github/buildcheck/natives.py @@ -0,0 +1,237 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +"""Everything that names a natives jar, checked against .github/natives.csv. + +The list is the one place a natives jar is declared. Everything else either reads it (the merge, +the fat-jar assembly) or has to repeat it, and each repetition is checked here: + * llama/pom.xml one jar execution per row: classifier, directory, Automatic-Module-Name + * llama-platform depends on exactly the rows marked platform=yes + * publish.yml a build job uploads natives- for every row and no other, and + `package` waits for each of them (else it packages without that build) + * LlamaLoader BACKEND_PRIORITY tries every backend (else a jar ships and never loads) + * CMakeLists.txt names exactly the backend directories of the list + * nativedeps.ALLOWED holds an allowlist for every CPU directory (cpu, metal, msvc), and for no + directory the list lacks + * README.md documents every classifier +and, for the all-backends fat jars derived from the list (fatjar_targets), that each one is +uploaded for a smoke job, launched by one, named in the agent jar's Class-Path and in the README. +package-fatjars.sh does not repeat the targets at all: it asks this module for them. + +Every check is a function of the texts it compares, so the tests drive them with literals. +""" + +import csv +import io +import os +import re +import xml.etree.ElementTree as ET + +from . import nativedeps, workflow + +NS = {"m": "http://maven.apache.org/POM/4.0.0"} + +# Backends that get no place in an all-backends jar: msvc is the same CPU build from another +# generator, so it would only be a second CPU library the loader tries before the first. +FATJAR_EXCLUDED_BACKENDS = ("msvc",) +# Platforms that get no all-backends jar: `java -jar` does not apply on Android (the AAR does). +FATJAR_EXCLUDED_OSES = ("Linux-Android",) + +FATJAR_NAME = re.compile(r"all-([a-z0-9]+(?:-[a-z0-9]+)*?)-jar-with-dependencies") + + +def rows(text): + """The rows of natives.csv (comment lines and blank lines skipped).""" + lines = [line for line in io.StringIO(text) if line.strip() and not line.startswith("#")] + return list(csv.DictReader(lines)) + + +def backend(row): + return row["directory"].rsplit("/", 1)[1] + + +def tree(row): + """/ of the row.""" + return row["directory"].rsplit("/", 1)[0] + + +def module_name(classifier): + """Each natives jar needs its own: without one, every jar derives the module name `llama` + from its file name, and the module path silently keeps only the first.""" + return "net.ladenthin.llama.natives." + classifier.replace("-", "_") + + +def pom_execution(row): + return f"""\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\tnatives-{row['classifier']} +\t\t\t\t\t\t\t\tpackage +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\tjar +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t{row['classifier']} +\t\t\t\t\t\t\t\t\t${{project.basedir}}/src/main/natives +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\tnet/ladenthin/llama/{row['directory']}/** +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\tfalse +\t\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t\t\t{module_name(row['classifier'])} +\t\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t\t +\t\t\t\t\t\t\t""" + + +def fatjar_targets(natives): + """The all-backends fat jars: one per - (the classifier suffix) with more than one + natives jar, excluded backends and platforms left out. Sorted. package-fatjars.sh builds + exactly these (it asks for them: check-natives.py fatjar-targets).""" + groups = {} + for row in natives: + if backend(row) in FATJAR_EXCLUDED_BACKENDS or tree(row).split("/")[0] in FATJAR_EXCLUDED_OSES: + continue + target = row["classifier"][len(backend(row)) + 1:] + groups.setdefault(target, set()).add(backend(row)) + return sorted(t for t, backends in groups.items() if len(backends) > 1) + + +def fatjar_names(text): + """The fat-jar targets a text names (`...-all--jar-with-dependencies...`).""" + return set(FATJAR_NAME.findall(text)) + + +def compare(what, expected, actual): + return ([f"{what}: missing {name}" for name in sorted(set(expected) - set(actual))] + + [f"{what}: {name} is not in .github/natives.csv" for name in sorted(set(actual) - set(expected))]) + + +def check_rows(natives): + failures = [] + classifiers = [r["classifier"] for r in natives] + if len(set(classifiers)) != len(classifiers): + failures.append("natives.csv lists a classifier twice") + for r in natives: + if not r["classifier"].startswith(backend(r) + "-") or r["platform"] not in ("yes", "no"): + failures.append(f"natives.csv: row {r['classifier']} -- classifier must start with its " + f"directory's backend '{backend(r)}', platform must be yes or no") + return failures + + +def check_pom(natives, pom_text): + """The natives profile of llama/pom.xml: one jar execution per row.""" + found = {} + for profile in ET.fromstring(pom_text).iterfind("m:profiles/m:profile", NS): + if profile.findtext("m:id", namespaces=NS) != "natives": + continue + for ex in profile.iterfind(".//m:plugin/m:executions/m:execution", NS): + conf = ex.find("m:configuration", NS) + classifier = conf.findtext("m:classifier", namespaces=NS) if conf is not None else None + if classifier: + found[classifier] = (conf.findtext("m:includes/m:include", namespaces=NS), + conf.findtext("m:archive/m:manifestEntries/m:Automatic-Module-Name", + namespaces=NS)) + by_classifier = {r["classifier"]: r for r in natives} + failures = compare("llama/pom.xml natives profile", by_classifier, found) + for classifier in sorted(set(found) & set(by_classifier)): + want = (f"net/ladenthin/llama/{by_classifier[classifier]['directory']}/**", module_name(classifier)) + if found[classifier] != want: + failures.append(f"llama/pom.xml: {classifier} has {found[classifier]}, expected {want} " + f"(check-natives.py pom prints the executions)") + return failures + + +def check_platform(natives, platform_pom_text): + deps = {d.findtext("m:classifier", namespaces=NS) + for d in ET.fromstring(platform_pom_text).iterfind("m:dependencies/m:dependency", NS)} - {None} + return compare("llama-platform/pom.xml", [r["classifier"] for r in natives if r["platform"] == "yes"], deps) + + +def check_workflow(natives, jobs): + """The build jobs upload natives- for every row, `package` waits for each of them, + and every fat-jar target is uploaded for a smoke job and launched by one.""" + uploads = {} + for job in jobs.values(): + for name in job.uploads(): + if name.startswith("natives-"): + uploads[name[len("natives-"):]] = job.name + failures = compare("publish.yml natives-* uploads", [r["classifier"] for r in natives], uploads) + if "package" not in jobs: + return failures + ["publish.yml has no `package` job"] + waited_for = workflow.closure(jobs, "package") + for classifier, job in sorted(uploads.items()): + if job not in waited_for: + failures.append(f"publish.yml: package does not wait for {job}, which uploads natives-{classifier} " + f"-- add it to package's needs") + targets = fatjar_targets(natives) + assembler = jobs.get("package-fatjars") + if assembler is None: + return failures + ["publish.yml has no `package-fatjars` job"] + uploaded = fatjar_names("\n".join(line for line in assembler.lines if "path:" in line)) + failures += compare("publish.yml package-fatjars smoke-jar uploads", targets, uploaded) + launched = set() + for job in jobs.values(): + if job.name != "package-fatjars": + launched |= fatjar_names("\n".join(line for line in job.lines if ".github/smoke-" in line)) + failures += compare("publish.yml fat-jar smoke runs", targets, launched) + return failures + + +def check_loader(natives, loader_text): + block = re.search(r"BACKEND_PRIORITY\s*=(.*?);", loader_text, re.S) + priority = set(re.findall(r'"([^"]+)"', block.group(1))) if block else set() + return [f"LlamaLoader.BACKEND_PRIORITY does not try '{b}' -- its jars would never load" + for b in sorted({backend(r) for r in natives} - priority)] + + +def check_cmake(natives, cmake_text): + named = set(re.findall(r"set\(JLLAMA_BACKEND\s+([A-Za-z0-9_-]+)\)", cmake_text)) + listed = {backend(r) for r in natives} + return ([f"llama/CMakeLists.txt never sets JLLAMA_BACKEND {b} -- no build writes that directory" + for b in sorted(listed - named)] + + [f"llama/CMakeLists.txt sets JLLAMA_BACKEND {b}, which no natives jar ships" + for b in sorted(named - listed)]) + + +def check_dependency_allowlist(natives, allowed): + """nativedeps.ALLOWED holds a list for every CPU directory (the package job would otherwise fail + on a new one only after every build finished), and none for a directory no jar ships.""" + directories = {r["directory"] for r in natives} + cpu = {r["directory"] for r in natives if backend(r) in nativedeps.CPU_BACKENDS} + return ([f"buildcheck/nativedeps.py ALLOWED has no allowlist for {d}" for d in sorted(cpu - set(allowed))] + + [f"buildcheck/nativedeps.py ALLOWED lists {d}, which no natives jar of natives.csv ships" + for d in sorted(set(allowed) - directories)]) + + +def check_readme(natives, readme_text): + failures = [f"README.md does not document the natives jar `{r['classifier']}`" + for r in natives if f"`{r['classifier']}`" not in readme_text] + return failures + compare("README.md all-backends fat jars", fatjar_targets(natives), fatjar_names(readme_text)) + + +def check_agent_class_path(natives, agent_pom_text): + """`java -jar` on the agent jar finds the core through its manifest Class-Path, which names + every all-backends fat jar.""" + match = re.search(r"(.*?)", agent_pom_text, re.S) + named = fatjar_names(match.group(1)) if match else set() + return compare("llama-atmosphere-agent/pom.xml Class-Path", fatjar_targets(natives), named) + + +def read(root, path): + with open(os.path.join(root, path), encoding="utf-8") as f: + return f.read() + + +def check(root): + """Every check, over the files of the repository at `root`.""" + natives = rows(read(root, ".github/natives.csv")) + return (check_rows(natives) + + check_pom(natives, read(root, "llama/pom.xml")) + + check_platform(natives, read(root, "llama-platform/pom.xml")) + + check_workflow(natives, workflow.parse(read(root, ".github/workflows/publish.yml"))) + + check_loader(natives, read(root, "llama/src/main/java/net/ladenthin/llama/loader/LlamaLoader.java")) + + check_cmake(natives, read(root, "llama/CMakeLists.txt")) + + check_dependency_allowlist(natives, nativedeps.ALLOWED) + + check_readme(natives, read(root, "README.md")) + + check_agent_class_path(natives, read(root, "llama-atmosphere-agent/pom.xml"))) diff --git a/.github/buildcheck/releasegate.py b/.github/buildcheck/releasegate.py new file mode 100644 index 000000000..729e2150b --- /dev/null +++ b/.github/buildcheck/releasegate.py @@ -0,0 +1,60 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Every job of publish.yml gates both publish jobs, unless the repository lists it in +.github/release-gate-exemptions.txt with a reason. + +A job that nothing waits for can go red and a release still ships -- the natives-build jobs that +`package` once forgot to wait for, and the aarch64 fat jars that were signed and attached for +releases without any job launching them, were both of that shape. So "not gating" has to be a +decision written down here, not the default a new job gets by being forgotten in two `needs:` lists. + +The check runs both ways: a job outside the gates and outside the exemptions fails, and so does an +exemption that names no job or a job that gates both publish jobs after all (a stale +exemption would hide the next job of that name). +""" + +from . import workflow + +GATES = ("publish-snapshot", "publish-release") + +# Each repository lists its own exemptions here, one `: ` per line (# comments). +EXEMPTIONS_FILE = ".github/release-gate-exemptions.txt" + + +def read_exemptions(text): + """The exemptions file: job -> reason. A line without a reason is an error -- the point of the + file is that every job allowed to stay red says why.""" + exemptions = {} + for number, line in enumerate(text.splitlines(), 1): + line = line.strip() + if not line or line.startswith("#"): + continue + job, _, reason = line.partition(":") + if not reason.strip() or not job.strip(): + raise ValueError(f"line {number}: expected `: `, got {line!r}") + if job.strip() in exemptions: + raise ValueError(f"line {number}: {job.strip()} is listed twice") + exemptions[job.strip()] = reason.strip() + return exemptions + + +def check(jobs, non_gating, gates=GATES): + missing = [g for g in gates if g not in jobs] + if missing: + return [f"publish.yml has no job {g}" for g in missing] + closures = [workflow.closure(jobs, g) for g in gates] + failures = [] + for name in jobs: + gated = [g for g, c in zip(gates, closures) if name in c] + if name in non_gating: + if len(gated) == len(gates): + failures.append(f"release-gate-exemptions.txt lists {name}, which gates {', '.join(gates)} " + f"-- remove the stale exemption") + elif len(gated) != len(gates): + ungated = [g for g in gates if g not in gated] + failures.append(f"publish.yml: {', '.join(ungated)} does not wait for {name} -- add it to the " + f"needs, or to release-gate-exemptions.txt with the reason it may stay red") + failures += [f"release-gate-exemptions.txt lists {name}, which is no job of publish.yml" + for name in sorted(set(non_gating) - set(jobs))] + return failures diff --git a/.github/buildcheck/sharedfiles.py b/.github/buildcheck/sharedfiles.py new file mode 100644 index 000000000..1c771ee40 --- /dev/null +++ b/.github/buildcheck/sharedfiles.py @@ -0,0 +1,137 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""The files kept byte-identical across java-llama.cpp, BitcoinAddressFinder, srcmorph and +streambuffer, checked by every repository's own `shared-files` job. + +Each repository lists the shared files it carries, with their SHA-256, in +.github/shared-files.sha256 -- `sha256sum` format, so `sha256sum -c` reads it as well. The job + * fails when a listed file is missing or its content no longer matches its line: a shared file + was edited here alone. Edit it in every repository that lists it, then update each manifest + (`check-shared-files.py --write` rewrites the hashes of this repository's lines); + * warns when another repository's manifest (its default branch) lists the same file with a + different hash: one side of a sync is not merged yet. A warning and not a failure, because a + sync is one change per repository and lands in four steps. +A file is matched across repositories by its path, or, when the other side has no such path, by +its file name if that is unique there (lombok.config lives at llama/lombok.config in the +java-llama.cpp reactor and at the root elsewhere). +""" + +import hashlib +import os +import re +import sys +import urllib.request + +REPOS = ("java-llama.cpp", "BitcoinAddressFinder", "srcmorph", "streambuffer") +OWNER = "bernardladenthin" +MANIFEST = ".github/shared-files.sha256" +RAW_URL = "https://raw.githubusercontent.com/{owner}/{repo}/HEAD/" + MANIFEST +LINE = re.compile(r"^([0-9a-f]{64}) [ *](.+)$") + + +def parse(text): + """path -> sha256 of a manifest. Comment and blank lines are skipped; anything else that is + not a `sha256sum` line is an error, and so is a path listed twice.""" + entries = {} + for number, line in enumerate(text.splitlines(), 1): + if not line.strip() or line.startswith("#"): + continue + match = LINE.match(line) + if not match: + raise ValueError(f"{MANIFEST} line {number}: not ` `: {line!r}") + if match.group(2) in entries: + raise ValueError(f"{MANIFEST} line {number}: {match.group(2)} is listed twice") + entries[match.group(2)] = match.group(1) + return entries + + +def sha256(path): + with open(path, "rb") as f: + return hashlib.sha256(f.read()).hexdigest() + + +def verify(root, entries): + """Failures: every listed file exists and still has its listed hash.""" + failures = [] + for path, digest in entries.items(): + full = os.path.join(root, path) + if not os.path.isfile(full): + failures.append(f"{path} is listed in {MANIFEST} but does not exist") + elif sha256(full) != digest: + failures.append(f"{path} differs from its line in {MANIFEST}: a shared file was changed in " + f"this repository alone -- change every copy, then update every manifest") + return failures + + +def rewrite(text, root): + """The manifest with every hash recomputed from the files; comments and order kept.""" + out = [] + for line in text.splitlines(): + match = LINE.match(line) + out.append(f"{sha256(os.path.join(root, match.group(2)))} {match.group(2)}" if match else line) + return "\n".join(out) + "\n" + + +def counterpart(path, other): + """The path `other` (another manifest) lists for our `path`: the same path, else a unique + file of the same name, else None.""" + if path in other: + return path + same_name = [p for p in other if os.path.basename(p) == os.path.basename(path)] + return same_name[0] if len(same_name) == 1 else None + + +def compare(own, others): + """Warnings for every shared file another repository lists with a different hash.""" + warnings = [] + for repo, entries in sorted(others.items()): + for path, digest in own.items(): + theirs = counterpart(path, entries) + if theirs is not None and entries[theirs] != digest: + warnings.append(f"{path} differs from {repo}'s {theirs}: finish or redo the sync") + return warnings + + +def current_repo(root): + name = os.environ.get("GITHUB_REPOSITORY", "").split("/")[-1] + return name or os.path.basename(os.path.abspath(root)) + + +def fetch(repo): + with urllib.request.urlopen(RAW_URL.format(owner=OWNER, repo=repo), timeout=20) as response: + return response.read().decode("utf-8") + + +def main(argv, root, fetcher=fetch, out=sys.stdout, err=sys.stderr): + """check-shared-files.py [--write | --offline]""" + path = os.path.join(root, MANIFEST) + with open(path, encoding="utf-8") as f: + text = f.read() + if argv[1:] == ["--write"]: + with open(path, "w", encoding="utf-8", newline="\n") as f: + f.write(rewrite(text, root)) + print(f"rewrote the hashes of {MANIFEST}", file=out) + return 0 + if argv[1:] not in ([], ["--offline"]): + print(main.__doc__, file=err) + return 2 + own = parse(text) + failures = verify(root, own) + for failure in failures: + print(f"::error::{failure}", file=err) + print(f"{len(own)} shared files, {len(failures)} changed here alone", file=out) + if argv[1:] == ["--offline"]: + return 1 if failures else 0 + others = {} + for repo in REPOS: + if repo == current_repo(root): + continue + try: + others[repo] = parse(fetcher(repo)) + except Exception as e: # noqa: BLE001 -- a sibling that cannot be read is a warning, never a red + print(f"::warning::could not read {repo}'s {MANIFEST}: {e}", file=err) + for warning in compare(own, others): + print(f"::warning::{warning}", file=err) + print(f"compared with {', '.join(sorted(others)) or 'no other repository'}", file=out) + return 1 if failures else 0 diff --git a/.github/buildcheck/tests/__init__.py b/.github/buildcheck/tests/__init__.py new file mode 100644 index 000000000..4833295f3 --- /dev/null +++ b/.github/buildcheck/tests/__init__.py @@ -0,0 +1,3 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 diff --git a/.github/buildcheck/tests/helpers.py b/.github/buildcheck/tests/helpers.py new file mode 100644 index 000000000..375a81282 --- /dev/null +++ b/.github/buildcheck/tests/helpers.py @@ -0,0 +1,26 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Shared fixtures of the buildcheck tests.""" + +import os + +REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))) + + +def workflow_text(jobs): + """A workflow with the given jobs, each (name, needs, body lines).""" + lines = ["name: t", "on:", " push:", "jobs:"] + for name, needs, body in jobs: + lines.append(f" {name}:") + if needs is not None: + lines.append(f" needs: {needs}") + lines.append(" runs-on: ubuntu-latest") + lines.append(" steps:") + lines += body + return "\n".join(lines) + "\n" + + +def upload(artifact, path="x/"): + return [" - uses: actions/upload-artifact@v7", " with:", f" name: {artifact}", + f" path: {path}"] diff --git a/.github/buildcheck/tests/test_hipoffload.py b/.github/buildcheck/tests/test_hipoffload.py new file mode 100644 index 000000000..f3cc02707 --- /dev/null +++ b/.github/buildcheck/tests/test_hipoffload.py @@ -0,0 +1,41 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +import os +import tempfile +import unittest + +from buildcheck import hipoffload + + +class HipOffloadTest(unittest.TestCase): + + def library(self, folder, content): + path = os.path.join(folder, "libjllama.so") + with open(path, "wb") as f: + f.write(content) + return path + + def test_counts_a_magic_split_across_two_chunks_once(self): + with tempfile.TemporaryDirectory() as d: + magic = hipoffload.UNCOMPRESSED + path = self.library(d, b"x" * 30 + magic + b"y" * 5 + hipoffload.COMPRESSED) + for chunk in (7, 32, 40, 1 << 20): + with self.subTest(chunk=chunk): + self.assertEqual(hipoffload.count(path, chunk), (1, 1)) + + def test_exit_codes(self): + with tempfile.TemporaryDirectory() as d: + self.assertEqual(hipoffload.main(["x", d]), 2) # nothing found + path = self.library(d, b"CCOB..CCOB") + self.assertEqual(hipoffload.main(["x", d]), 0) + self.library(d, b"CCOB" + hipoffload.UNCOMPRESSED) + self.assertEqual(hipoffload.main(["x", path]), 1) # a file argument works as well + self.library(d, b"no bundles") + self.assertEqual(hipoffload.main(["x", d]), 1) + self.assertEqual(hipoffload.main(["x"]), 2) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/tests/test_models.py b/.github/buildcheck/tests/test_models.py new file mode 100644 index 000000000..3e96bfbdd --- /dev/null +++ b/.github/buildcheck/tests/test_models.py @@ -0,0 +1,28 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +import unittest + +from buildcheck import models, natives +from buildcheck.tests.helpers import REPO + +CSV = "# comment\na.gguf,https://x/a\nb.gguf,https://x/b\n" + + +class ModelsTest(unittest.TestCase): + + def test_env_names_must_be_listed(self): + workflow = 'env:\n JAVA_VERSION: \'21\'\n A_MODEL_NAME: "a.gguf"\n B: b.gguf\n' + self.assertEqual(models.check(CSV, workflow), []) + self.assertEqual(models.check(CSV, workflow + ' C_MODEL_NAME: "c.gguf"\n'), + ["publish.yml env C_MODEL_NAME=c.gguf is not a filename of .github/models.csv " + "-- no job downloads it"]) + + def test_the_repository_passes(self): + self.assertEqual(models.check(natives.read(REPO, ".github/models.csv"), + natives.read(REPO, ".github/workflows/publish.yml")), []) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/tests/test_nativedeps.py b/.github/buildcheck/tests/test_nativedeps.py new file mode 100644 index 000000000..0b724bf6f --- /dev/null +++ b/.github/buildcheck/tests/test_nativedeps.py @@ -0,0 +1,159 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +import os +import struct +import tempfile +import unittest + +from buildcheck import nativedeps + + +def elf64(needed): + """A minimal little-endian ELF64 with a .dynstr and a .dynamic naming `needed`.""" + strtab = b"\0" + offsets = [] + for name in needed: + offsets.append(len(strtab)) + strtab += name.encode() + b"\0" + dynamic = b"".join(struct.pack(" +# +# SPDX-License-Identifier: MIT + +import os +import subprocess +import sys +import unittest + +from buildcheck import natives, workflow +from buildcheck.tests.helpers import REPO, upload, workflow_text + +CSV = """# a comment +classifier,directory,library,platform +cpu-linux-x86-64,Linux/x86_64/cpu,libjllama.so,yes +cuda13-linux-x86-64,Linux/x86_64/cuda13,libjllama.so,no +cpu-linux-s390x,Linux/s390x/cpu,libjllama.so,yes +cpu-windows-x86,Windows/x86/cpu,jllama.dll,yes +msvc-windows-x86,Windows/x86/msvc,jllama.dll,no +cpu-android-aarch64,Linux-Android/aarch64/cpu,libjllama.so,no +opencl-android-aarch64,Linux-Android/aarch64/opencl,libjllama.so,no +metal-macos-aarch64,Mac/aarch64/metal,libjllama.dylib,yes +""" + +ROWS = natives.rows(CSV) + + +def pom(rows): + execs = "\n".join(natives.pom_execution(r) for r in rows) + return f"""natives + +{execs} +""" + + +def platform(classifiers): + deps = "".join(f"{c}" for c in classifiers) + return f'{deps}' + + +def jobs(rows=ROWS, package_needs="[build]", smoke_targets=("linux-x86-64",), run_targets=("linux-x86-64",)): + build = [line for r in rows for line in upload("natives-" + r["classifier"])] + fatjars = [line for t in smoke_targets for line in upload("s-" + t, f"f/llama-*-all-{t}-jar-with-dependencies.jar")] + smoke = [f" - run: .github/smoke-test-fatjar.sh f 'llama-*-all-{t}-jar-with-dependencies.jar'" + for t in run_targets] + return workflow.parse(workflow_text([("build", None, build), ("other", None, []), + ("package", package_needs, []), + ("package-fatjars", "[package]", fatjars), + ("smoke", "[package-fatjars]", smoke)])) + + +class ListTest(unittest.TestCase): + + def test_rows_skip_comments(self): + self.assertEqual(ROWS[0], {"classifier": "cpu-linux-x86-64", "directory": "Linux/x86_64/cpu", + "library": "libjllama.so", "platform": "yes"}) + self.assertEqual(len(ROWS), 8) + + def test_module_name_is_unique_per_classifier(self): + self.assertEqual(natives.module_name("sycl-fp16-linux-x86-64"), "net.ladenthin.llama.natives.sycl_fp16_linux_x86_64") + + def test_fatjar_targets_need_a_second_backend_and_skip_msvc_and_android(self): + # linux-x86-64: cpu + cuda13. windows-x86: cpu + msvc (excluded). android: excluded. + # s390x and macos: one backend each. + self.assertEqual(natives.fatjar_targets(ROWS), ["linux-x86-64"]) + + def test_fatjar_names(self): + text = "llama-*-all-linux-x86-64-jar-with-dependencies.jar llama--all---jar-with-dependencies.jar " \ + "llama-${llama.version}-all-windows-aarch64-jar-with-dependencies.jar llama-5.2.0-jar-with-dependencies.jar" + self.assertEqual(natives.fatjar_names(text), {"linux-x86-64", "windows-aarch64"}) + + def test_check_rows(self): + self.assertEqual(natives.check_rows(ROWS), []) + bad = natives.rows("classifier,directory,library,platform\n" + "cuda-linux-x86-64,Linux/x86_64/cuda13,x,no\ncpu-a,A/b/cpu,x,maybe\ncpu-a,A/b/cpu,x,no\n") + failures = natives.check_rows(bad) + self.assertEqual(len(failures), 3, failures) + self.assertIn("twice", failures[0]) + + +class ConsumerTest(unittest.TestCase): + + def test_pom(self): + self.assertEqual(natives.check_pom(ROWS, pom(ROWS)), []) + self.assertIn("missing cuda13-linux-x86-64", natives.check_pom(ROWS, pom(ROWS[:1] + ROWS[2:]))[0]) + wrong = pom(ROWS).replace("Linux/x86_64/cuda13/**", "Linux/x86_64/cuda/**") + self.assertIn("cuda13-linux-x86-64 has", natives.check_pom(ROWS, wrong)[0]) + + def test_platform(self): + yes = [r["classifier"] for r in ROWS if r["platform"] == "yes"] + self.assertEqual(natives.check_platform(ROWS, platform(yes)), []) + self.assertEqual(natives.check_platform(ROWS, platform(yes + ["cuda13-linux-x86-64"])), + ["llama-platform/pom.xml: cuda13-linux-x86-64 is not in .github/natives.csv"]) + + def test_workflow_passes_when_everything_is_wired(self): + self.assertEqual(natives.check_workflow(ROWS, jobs()), []) + + def test_workflow_needs_an_upload_per_row(self): + failures = natives.check_workflow(ROWS, jobs(rows=ROWS[1:])) + self.assertIn("publish.yml natives-* uploads: missing cpu-linux-x86-64", failures) + + def test_package_must_wait_for_every_natives_build(self): + failures = natives.check_workflow(ROWS, jobs(package_needs="[other]")) + self.assertTrue(failures) + self.assertTrue(all("package does not wait for build" in f for f in failures), failures) + + def test_package_may_wait_transitively(self): + wf = workflow_text([("build", None, [line for r in ROWS for line in upload("natives-" + r["classifier"])]), + ("mid", "build", []), ("package", "mid", []), + ("package-fatjars", "package", + upload("s", "f/llama-*-all-linux-x86-64-jar-with-dependencies.jar")), + ("smoke", "package-fatjars", + [" - run: .github/smoke-test-fatjar.sh f 'llama-*-all-linux-x86-64-jar-with-dependencies.jar'"])]) + self.assertEqual(natives.check_workflow(ROWS, workflow.parse(wf)), []) + + def test_every_fatjar_target_is_uploaded_and_launched(self): + self.assertEqual(natives.check_workflow(ROWS, jobs(smoke_targets=())), + ["publish.yml package-fatjars smoke-jar uploads: missing linux-x86-64"]) + self.assertEqual(natives.check_workflow(ROWS, jobs(run_targets=("linux-x86-64", "linux-s390x"))), + ["publish.yml fat-jar smoke runs: linux-s390x is not in .github/natives.csv"]) + + def test_loader(self): + text = 'List BACKEND_PRIORITY = Arrays.asList(\n "cuda13", "msvc",\n "metal", "opencl", "cpu");' + self.assertEqual(natives.check_loader(ROWS, text), []) + self.assertEqual(natives.check_loader(ROWS, text.replace('"cuda13", ', "")), + ["LlamaLoader.BACKEND_PRIORITY does not try 'cuda13' -- its jars would never load"]) + + def test_cmake_both_directions(self): + text = "\n".join(f" set(JLLAMA_BACKEND {b})" for b in ("cuda13", "cpu", "msvc", "opencl", "metal")) + self.assertEqual(natives.check_cmake(ROWS, text), []) + failures = natives.check_cmake(ROWS, text.replace("cuda13", "cuda14")) + self.assertEqual(len(failures), 2) + self.assertIn("never sets JLLAMA_BACKEND cuda13", failures[0]) + self.assertIn("sets JLLAMA_BACKEND cuda14, which no natives jar ships", failures[1]) + + def test_dependency_allowlist_covers_every_cpu_directory_and_nothing_unshipped(self): + cpu = ["Linux/x86_64/cpu", "Linux/s390x/cpu", "Windows/x86/cpu", "Windows/x86/msvc", + "Linux-Android/aarch64/cpu", "Mac/aarch64/metal"] + self.assertEqual(natives.check_dependency_allowlist(ROWS, dict.fromkeys(cpu)), []) + # a GPU directory may carry a list (the Android OpenCL build does), a directory no jar ships may not + self.assertEqual(natives.check_dependency_allowlist(ROWS, dict.fromkeys(cpu + ["Linux/x86_64/cuda13"])), []) + failures = natives.check_dependency_allowlist(ROWS, dict.fromkeys(cpu[1:] + ["Linux/x86_64/cuda12"])) + self.assertEqual(len(failures), 2, failures) + self.assertIn("no allowlist for Linux/x86_64/cpu", failures[0]) + self.assertIn("Linux/x86_64/cuda12, which no natives jar", failures[1]) + + def test_readme(self): + text = " ".join(f"`{r['classifier']}`" for r in ROWS) + " llama--all-linux-x86-64-jar-with-dependencies.jar" + self.assertEqual(natives.check_readme(ROWS, text), []) + failures = natives.check_readme(ROWS, text.replace("`cpu-linux-s390x`", "cpu-linux-s390x")) + self.assertEqual(failures, ["README.md does not document the natives jar `cpu-linux-s390x`"]) + + def test_agent_class_path(self): + good = "llama-${v}-all-linux-x86-64-jar-with-dependencies.jar llama-${v}-jar-with-dependencies.jar" + self.assertEqual(natives.check_agent_class_path(ROWS, good), []) + self.assertEqual(natives.check_agent_class_path(ROWS, ""), + ["llama-atmosphere-agent/pom.xml Class-Path: missing linux-x86-64"]) + + +class RepositoryTest(unittest.TestCase): + """The checks over this repository: what code-style runs, so a red here is a red there.""" + + def test_the_repository_agrees_with_its_list(self): + self.assertEqual(natives.check(REPO), []) + + def test_the_repository_derives_the_four_release_targets(self): + rows = natives.rows(natives.read(REPO, ".github/natives.csv")) + self.assertEqual(natives.fatjar_targets(rows), + ["linux-aarch64", "linux-x86-64", "windows-aarch64", "windows-x86-64"]) + + def test_cli_prints_the_pom_executions_and_the_targets(self): + cli = os.path.join(REPO, ".github", "check-natives.py") + out = subprocess.run([sys.executable, cli, "fatjar-targets"], capture_output=True, text=True, check=True) + self.assertEqual(out.stdout.split(), ["linux-aarch64", "linux-x86-64", "windows-aarch64", "windows-x86-64"]) + out = subprocess.run([sys.executable, cli, "pom"], capture_output=True, text=True, check=True) + self.assertEqual(out.stdout.count(""), 26) + self.assertEqual(subprocess.run([sys.executable, cli, "nonsense"], capture_output=True).returncode, 2) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/tests/test_releasegate.py b/.github/buildcheck/tests/test_releasegate.py new file mode 100644 index 000000000..19fe70490 --- /dev/null +++ b/.github/buildcheck/tests/test_releasegate.py @@ -0,0 +1,61 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +import os +import unittest + +from buildcheck import releasegate, workflow +from buildcheck.tests.helpers import REPO, workflow_text + +GATES = ("pub-a", "pub-b") + + +def jobs(*extra): + return workflow.parse(workflow_text([("test", None, []), ("build", "test", []), *extra, + ("pub-a", "[build]", []), ("pub-b", "[build]", [])])) + + +class ReleaseGateTest(unittest.TestCase): + + def check(self, js, non_gating): + return releasegate.check(js, dict.fromkeys(non_gating, "reason"), GATES) + + def test_transitively_gating_jobs_pass(self): + self.assertEqual(self.check(jobs(), ["pub-a", "pub-b"]), []) + + def test_a_job_nothing_waits_for_fails(self): + self.assertEqual(self.check(jobs(("lint", None, [])), ["pub-a", "pub-b"]), + ["publish.yml: pub-a, pub-b does not wait for lint -- add it to the needs, or to " + "release-gate-exemptions.txt with the reason it may stay red"]) + + def test_gating_only_one_publish_job_fails(self): + js = workflow.parse(workflow_text([("lint", None, []), ("pub-a", "[lint]", []), ("pub-b", None, [])])) + self.assertEqual(len(self.check(js, ["pub-a", "pub-b"])), 1) + + def test_a_listed_job_may_stay_outside(self): + self.assertEqual(self.check(jobs(("lint", None, [])), ["pub-a", "pub-b", "lint"]), []) + + def test_a_stale_exemption_fails(self): + self.assertIn("remove the stale exemption", self.check(jobs(), ["pub-a", "pub-b", "build"])[0]) + self.assertEqual(self.check(jobs(), ["pub-a", "pub-b", "gone"]), + ["release-gate-exemptions.txt lists gone, which is no job of publish.yml"]) + + def test_reads_the_exemptions_file(self): + self.assertEqual(releasegate.read_exemptions("# c\n\na: why: because\n b :x\n"), + {"a": "why: because", "b": "x"}) + + def test_an_exemption_without_a_reason_is_an_error(self): + for text in ("a\n", "a:\n", "a: \n", ": x\n", "a: x\na: y\n"): + with self.subTest(text=text), self.assertRaises(ValueError): + releasegate.read_exemptions(text) + + def test_the_repository_passes(self): + with open(os.path.join(REPO, ".github", "workflows", "publish.yml"), encoding="utf-8") as f: + jobs = workflow.parse(f.read()) + with open(os.path.join(REPO, releasegate.EXEMPTIONS_FILE), encoding="utf-8") as f: + self.assertEqual(releasegate.check(jobs, releasegate.read_exemptions(f.read())), []) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/tests/test_sharedfiles.py b/.github/buildcheck/tests/test_sharedfiles.py new file mode 100644 index 000000000..e78d610a3 --- /dev/null +++ b/.github/buildcheck/tests/test_sharedfiles.py @@ -0,0 +1,112 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +import hashlib +import io +import os +import tempfile +import unittest + +from buildcheck import sharedfiles +from buildcheck.tests.helpers import REPO + +A = hashlib.sha256(b"a").hexdigest() +B = hashlib.sha256(b"b").hexdigest() + + +class ParseTest(unittest.TestCase): + + def test_reads_sha256sum_lines_and_skips_comments(self): + text = f"# header\n\n{A} x/one.sh\n{B} *two.py\n" + self.assertEqual(sharedfiles.parse(text), {"x/one.sh": A, "two.py": B}) + + def test_rejects_malformed_and_duplicate_lines(self): + for text in ("nonsense\n", f"{A[:-1]} x\n", f"{A} x\n{B} x\n"): + with self.subTest(text=text), self.assertRaises(ValueError): + sharedfiles.parse(text) + + +class TreeTest(unittest.TestCase): + + def setUp(self): + self.root = tempfile.mkdtemp() + os.makedirs(os.path.join(self.root, ".github")) + with open(os.path.join(self.root, "one.sh"), "wb") as f: + f.write(b"a") + + def manifest(self, text): + with open(os.path.join(self.root, sharedfiles.MANIFEST), "w", encoding="utf-8") as f: + f.write(text) + + def run_main(self, *args, others=None): + out, err = io.StringIO(), io.StringIO() + others = others or {} + + def fetcher(repo): + if repo not in others: + raise OSError("unreachable") + return others[repo] + code = sharedfiles.main(["x", *args], self.root, fetcher, out, err) + return code, err.getvalue() + + def test_an_unchanged_file_passes(self): + self.manifest(f"# h\n{A} one.sh\n") + self.assertEqual(self.run_main("--offline"), (0, "")) + + def test_a_file_changed_alone_or_missing_fails(self): + self.manifest(f"{B} one.sh\n{A} gone.sh\n") + code, err = self.run_main("--offline") + self.assertEqual(code, 1) + self.assertIn("one.sh differs", err) + self.assertIn("gone.sh is listed", err) + + def test_write_recomputes_the_hashes_and_keeps_the_comments(self): + self.manifest(f"# keep me\n{B} one.sh\n") + self.assertEqual(self.run_main("--write")[0], 0) + with open(os.path.join(self.root, sharedfiles.MANIFEST), encoding="utf-8") as f: + self.assertEqual(f.read(), f"# keep me\n{A} one.sh\n") + + def test_sibling_differences_are_warnings_not_failures(self): + self.manifest(f"{A} one.sh\n") + others = {"srcmorph": f"{B} one.sh\n", "streambuffer": f"{A} one.sh\n"} + code, err = self.run_main(others=others) + self.assertEqual(code, 0) + self.assertIn("::warning::one.sh differs from srcmorph's one.sh", err) + self.assertNotIn("streambuffer's", err) + self.assertIn("could not read BitcoinAddressFinder", err) + + def test_unknown_arguments(self): + self.manifest("") + self.assertEqual(self.run_main("--nope")[0], 2) + + +class CompareTest(unittest.TestCase): + + def test_matches_by_path_then_by_a_unique_file_name(self): + own = {"llama/lombok.config": A, "a/__init__.py": A} + self.assertEqual(sharedfiles.compare(own, {"r": {"lombok.config": B}}), + ["llama/lombok.config differs from r's lombok.config: finish or redo the sync"]) + # two candidates of the same name: no guess + self.assertEqual(sharedfiles.compare(own, {"r": {"b/__init__.py": B, "c/__init__.py": B}}), []) + # a file the other repository does not share is no difference + self.assertEqual(sharedfiles.compare(own, {"r": {}}), []) + + +class RepositoryTest(unittest.TestCase): + + def test_this_repository_matches_its_manifest(self): + with open(os.path.join(REPO, sharedfiles.MANIFEST), encoding="utf-8") as f: + entries = sharedfiles.parse(f.read()) + self.assertEqual(sharedfiles.verify(REPO, entries), []) + + def test_the_shared_build_checks_list_themselves(self): + with open(os.path.join(REPO, sharedfiles.MANIFEST), encoding="utf-8") as f: + entries = sharedfiles.parse(f.read()) + for path in (".github/buildcheck/sharedfiles.py", ".github/check-shared-files.py", + ".github/buildcheck/tests/test_sharedfiles.py"): + self.assertIn(path, entries) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/tests/test_workflow.py b/.github/buildcheck/tests/test_workflow.py new file mode 100644 index 000000000..b9bf48d87 --- /dev/null +++ b/.github/buildcheck/tests/test_workflow.py @@ -0,0 +1,70 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +import os +import unittest + +from buildcheck import workflow +from buildcheck.tests.helpers import REPO, upload, workflow_text + + +class ParseTest(unittest.TestCase): + + def test_reads_every_needs_shape(self): + text = workflow_text([ + ("a", None, []), + ("b", "a", []), + ("c", "[a, b]", []), + ("d", "", []), + ]).replace(" needs: \n", " needs:\n - a\n - c # why\n") + jobs = workflow.parse(text) + self.assertEqual(list(jobs), ["a", "b", "c", "d"]) + self.assertEqual(jobs["a"].needs, []) + self.assertEqual(jobs["b"].needs, ["a"]) + self.assertEqual(jobs["c"].needs, ["a", "b"]) + self.assertEqual(jobs["d"].needs, ["a", "c"]) + + def test_keys_outside_jobs_are_not_jobs(self): + jobs = workflow.parse(workflow_text([("a", None, [])]) + "env:\n X: 1\n") + self.assertEqual(list(jobs), ["a"]) + + def test_unknown_need_is_an_error(self): + with self.assertRaisesRegex(ValueError, "needs nope"): + workflow.parse(workflow_text([("a", "nope", [])])) + + def test_unreadable_needs_is_an_error_not_a_guess(self): + with self.assertRaisesRegex(ValueError, "cannot read needs"): + workflow.parse(workflow_text([("a", None, []), ("b", "${{ fromJSON(x) }}", [])])) + + def test_uploads_reads_the_artifact_name_not_the_step_name(self): + body = [" - name: Upload something", " if: always()", + " uses: actions/upload-artifact@v7"] + upload("natives-x")[1:] \ + + [" - run: echo name: fake"] + upload("second") + job = workflow.parse(workflow_text([("a", None, body)]))["a"] + self.assertEqual(job.uploads(), ["natives-x", "second"]) + + def test_closure_is_transitive_and_excludes_the_job(self): + jobs = workflow.parse(workflow_text([("a", None, []), ("b", "a", []), ("c", "[b]", []), ("x", None, [])])) + self.assertEqual(workflow.closure(jobs, "c"), {"a", "b"}) + self.assertEqual(workflow.closure(jobs, "a"), set()) + + +class RepositoryWorkflowsTest(unittest.TestCase): + + def test_every_workflow_of_the_repository_parses(self): + folder = os.path.join(REPO, ".github", "workflows") + for name in sorted(os.listdir(folder)): + with self.subTest(name): + with open(os.path.join(folder, name), encoding="utf-8") as f: + self.assertTrue(workflow.parse(f.read())) + + def test_publish_has_both_publish_jobs_and_they_wait_for_something(self): + with open(os.path.join(REPO, ".github", "workflows", "publish.yml"), encoding="utf-8") as f: + jobs = workflow.parse(f.read()) + for gate in ("publish-snapshot", "publish-release"): + self.assertGreater(len(workflow.closure(jobs, gate)), 3, gate) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/workflow.py b/.github/buildcheck/workflow.py new file mode 100644 index 000000000..de7055a6b --- /dev/null +++ b/.github/buildcheck/workflow.py @@ -0,0 +1,120 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""The part of a GitHub Actions workflow the checks need: its jobs, their `needs`, the artifacts +they upload and the text of their steps. + +Deliberately not a YAML parser. The runners' Python has no PyYAML guaranteed, and the checks +need only the job graph, which the workflow writes in two fixed shapes: jobs at two spaces of +indentation below `jobs:`, and `needs:` at four, either inline (`needs: a` / `needs: [a, b]`) or +as a block list (`- a` at six). A workflow written any other way is reported by `parse`, never +misread silently: every job must be found, and a `needs:` that matches neither shape fails. +""" + +import re + +JOB = re.compile(r"^ ([A-Za-z0-9_-]+):\s*(#.*)?$") +NEEDS = re.compile(r"^ needs:\s*(.*?)\s*(#.*)?$") +NEEDS_ITEM = re.compile(r"^ - ([A-Za-z0-9_-]+)\s*(#.*)?$") +STEP = re.compile(r"^ - ") + + +class Job: + """One job: its id, the jobs it needs, and its lines (the job header included).""" + + def __init__(self, name): + self.name = name + self.needs = [] + self.lines = [] + + @property + def text(self): + return "\n".join(self.lines) + + def steps(self): + """The job's steps, each as a list of lines (a step starts at ` - `).""" + out = [] + for line in self.lines: + if STEP.match(line): + out.append([line]) + elif out: + out[-1].append(line) + return out + + def uploads(self): + """The names of the artifacts the job uploads (upload-artifact's `with: name:`).""" + names = [] + for step in self.steps(): + if not any("actions/upload-artifact@" in line for line in step): + continue + with_indent = None + for line in step: + stripped = line.lstrip() + indent = len(line) - len(stripped) + if stripped.startswith("with:"): + with_indent = indent + elif with_indent is not None and indent > with_indent and stripped.startswith("name:"): + names.append(stripped[len("name:"):].strip().strip("'\"")) + break + elif with_indent is not None and indent <= with_indent: + with_indent = None + return names + + +def parse(text): + """Jobs by id, in file order. Raises ValueError on a `needs:` in a shape it does not read.""" + jobs = {} + in_jobs = False + current = None + block_needs = False + for number, line in enumerate(text.splitlines(), 1): + if line and not line[0].isspace() and not line.startswith("#"): + in_jobs = line.rstrip() == "jobs:" + current = None + continue + if not in_jobs: + continue + match = JOB.match(line) + if match: + current = jobs[match.group(1)] = Job(match.group(1)) + current.lines.append(line) + block_needs = False + continue + if current is None: + continue + current.lines.append(line) + match = NEEDS.match(line) + if match: + value = match.group(1) + if value == "": + block_needs = True + elif value.startswith("[") and value.endswith("]"): + current.needs += [n.strip() for n in value[1:-1].split(",") if n.strip()] + elif re.fullmatch(r"[A-Za-z0-9_-]+", value): + current.needs.append(value) + else: + raise ValueError(f"line {number}: cannot read needs of job {current.name}: {value!r}") + continue + if block_needs: + item = NEEDS_ITEM.match(line) + if item: + current.needs.append(item.group(1)) + elif line.strip() and not line.lstrip().startswith("#"): + block_needs = False + for job in jobs.values(): + for need in job.needs: + if need not in jobs: + raise ValueError(f"job {job.name} needs {need}, which is no job of this workflow") + return jobs + + +def closure(jobs, name): + """Every job `name` waits for, directly or through another job (itself excluded).""" + seen = set() + todo = list(jobs[name].needs) + while todo: + need = todo.pop() + if need not in seen: + seen.add(need) + todo += jobs[need].needs + return seen diff --git a/.github/check-natives.py b/.github/check-natives.py old mode 100644 new mode 100755 index 758e917ca..6d2653782 --- a/.github/check-natives.py +++ b/.github/check-natives.py @@ -2,132 +2,43 @@ # SPDX-FileCopyrightText: 2026 Bernard Ladenthin # # SPDX-License-Identifier: MIT -"""Fail when anything that names the natives jars disagrees with .github/natives.csv. +"""Fail when anything that names the natives jars disagrees with .github/natives.csv (and +when a model name of publish.yml is not in .github/models.csv, see buildcheck/models.py). -The list is the one place a natives jar is declared. Everything else either reads it (the merge, -the fat-jar assembly) or must repeat it, and is checked here: - * llama/pom.xml one jar execution per row: classifier, directory, Automatic-Module-Name - * llama-platform depends on exactly the rows marked platform=yes - * publish.yml a build job uploads natives- for every row, and no other - * LlamaLoader BACKEND_PRIORITY tries every backend (else a jar ships and never loads) +See buildcheck/natives.py for what is checked. Usage: - check-natives.py check, exit 1 on any disagreement - check-natives.py pom print the natives jar executions for llama/pom.xml + check-natives.py check, exit 1 on any disagreement + check-natives.py pom print the natives jar executions for llama/pom.xml + check-natives.py fatjar-targets print the all-backends fat-jar targets, one per line """ -import csv import os -import re import sys -import xml.etree.ElementTree as ET -ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) -NS = {"m": "http://maven.apache.org/POM/4.0.0"} - - -def rows(): - with open(os.path.join(ROOT, ".github", "natives.csv"), encoding="utf-8") as f: - lines = [line for line in f if line.strip() and not line.startswith("#")] - return list(csv.DictReader(lines)) - - -def module_name(classifier): - """Each natives jar needs its own: without one, every jar derives the module name `llama` - from its file name, and the module path silently keeps only the first.""" - return "net.ladenthin.llama.natives." + classifier.replace("-", "_") - - -def pom_execution(row): - return f"""\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\tnatives-{row['classifier']} -\t\t\t\t\t\t\t\tpackage -\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\tjar -\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t{row['classifier']} -\t\t\t\t\t\t\t\t\t${{project.basedir}}/src/main/natives -\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t\tnet/ladenthin/llama/{row['directory']}/** -\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t\tfalse -\t\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t\t\t{module_name(row['classifier'])} -\t\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t\t -\t\t\t\t\t\t\t""" - - -def pom_natives(path): - """classifier -> (include, module name) for every jar execution of the natives profile.""" - tree = ET.parse(path) - found = {} - for profile in tree.getroot().iterfind("m:profiles/m:profile", NS): - if profile.findtext("m:id", namespaces=NS) != "natives": - continue - for ex in profile.iterfind(".//m:plugin/m:executions/m:execution", NS): - conf = ex.find("m:configuration", NS) - classifier = conf.findtext("m:classifier", namespaces=NS) if conf is not None else None - if classifier: - found[classifier] = (conf.findtext("m:includes/m:include", namespaces=NS), - conf.findtext("m:archive/m:manifestEntries/m:Automatic-Module-Name", - namespaces=NS)) - return found +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import models, natives # noqa: E402 - -def compare(what, expected, actual, failures): - for name in sorted(expected - actual): - failures.append(f"{what}: missing {name}") - for name in sorted(actual - expected): - failures.append(f"{what}: {name} is not in .github/natives.csv") +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) def main(argv): - natives = rows() - if len(argv) > 1 and argv[1] == "pom": - print("\n".join(pom_execution(r) for r in natives)) + rows = natives.rows(natives.read(ROOT, ".github/natives.csv")) + if argv[1:] == ["pom"]: + print("\n".join(natives.pom_execution(r) for r in rows)) return 0 - failures = [] - by_classifier = {r["classifier"]: r for r in natives} - if len(by_classifier) != len(natives): - failures.append("natives.csv lists a classifier twice") - for r in natives: - backend, rest = r["directory"].rsplit("/", 1)[1], r["classifier"] - if not rest.startswith(backend + "-") or r["platform"] not in ("yes", "no"): - failures.append(f"natives.csv: row {r['classifier']} -- classifier must start with its " - f"directory's backend '{backend}', platform must be yes or no") - - pom = pom_natives(os.path.join(ROOT, "llama", "pom.xml")) - compare("llama/pom.xml natives profile", set(by_classifier), set(pom), failures) - for classifier in sorted(set(pom) & set(by_classifier)): - want = (f"net/ladenthin/llama/{by_classifier[classifier]['directory']}/**", module_name(classifier)) - if pom[classifier] != want: - failures.append(f"llama/pom.xml: {classifier} has {pom[classifier]}, expected {want} " - f"(check-natives.py pom prints the executions)") - - platform = ET.parse(os.path.join(ROOT, "llama-platform", "pom.xml")) - deps = {d.findtext("m:classifier", namespaces=NS) - for d in platform.getroot().iterfind("m:dependencies/m:dependency", NS)} - {None} - compare("llama-platform/pom.xml", {r["classifier"] for r in natives if r["platform"] == "yes"}, deps, failures) - - with open(os.path.join(ROOT, ".github", "workflows", "publish.yml"), encoding="utf-8") as f: - uploads = set(re.findall(r"upload-artifact@\S+\s+with:\s+name:\s*natives-(\S+)", f.read())) - compare("publish.yml natives-* uploads", set(by_classifier), uploads, failures) - - loader = os.path.join(ROOT, "llama", "src", "main", "java", "net", "ladenthin", "llama", "loader", - "LlamaLoader.java") - with open(loader, encoding="utf-8") as f: - block = re.search(r"BACKEND_PRIORITY\s*=(.*?);", f.read(), re.S) - priority = set(re.findall(r'"([^"]+)"', block.group(1))) if block else set() - for backend in sorted({r["directory"].rsplit("/", 1)[1] for r in natives} - priority): - failures.append(f"LlamaLoader.BACKEND_PRIORITY does not try '{backend}' -- its jars would never load") - + if argv[1:] == ["fatjar-targets"]: + print("\n".join(natives.fatjar_targets(rows))) + return 0 + if argv[1:]: + print(__doc__, file=sys.stderr) + return 2 + failures = natives.check(ROOT) + models.check(natives.read(ROOT, ".github/models.csv"), + natives.read(ROOT, ".github/workflows/publish.yml")) for f in failures: print(f"::error::{f}", file=sys.stderr) - print(f"{len(natives)} natives jars, {len(failures)} disagreements") + print(f"{len(rows)} natives jars, {len(natives.fatjar_targets(rows))} all-backends fat jars, " + f"{len(failures)} disagreements") return 1 if failures else 0 diff --git a/.github/check-release-gate.py b/.github/check-release-gate.py new file mode 100755 index 000000000..447256fef --- /dev/null +++ b/.github/check-release-gate.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Fail when a job of publish.yml gates neither publish job without a written reason. + +See buildcheck/releasegate.py; the reasons are in .github/release-gate-exemptions.txt. +Usage (from anywhere): check-release-gate.py +""" + +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import releasegate, workflow # noqa: E402 + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +def main(): + with open(os.path.join(ROOT, ".github", "workflows", "publish.yml"), encoding="utf-8") as f: + jobs = workflow.parse(f.read()) + with open(os.path.join(ROOT, releasegate.EXEMPTIONS_FILE), encoding="utf-8") as f: + exemptions = releasegate.read_exemptions(f.read()) + failures = releasegate.check(jobs, exemptions) + for failure in failures: + print(f"::error::{failure}", file=sys.stderr) + print(f"{len(jobs)} jobs, {len(exemptions)} exempt, {len(failures)} violations") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/check-shared-files.py b/.github/check-shared-files.py new file mode 100755 index 000000000..47359ee67 --- /dev/null +++ b/.github/check-shared-files.py @@ -0,0 +1,20 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Check the files this repository shares byte-identically with its sibling repositories. + +See buildcheck/sharedfiles.py; the list is .github/shared-files.sha256. +Usage: check-shared-files.py verify the local copies, compare with the siblings (warnings) + check-shared-files.py --offline verify the local copies only + check-shared-files.py --write recompute the hashes of the listed files after a sync +""" + +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import sharedfiles # noqa: E402 + +if __name__ == "__main__": + sys.exit(sharedfiles.main(sys.argv, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))) diff --git a/.github/package-fatjars.sh b/.github/package-fatjars.sh index 80a556ac6..66c5b4217 100755 --- a/.github/package-fatjars.sh +++ b/.github/package-fatjars.sh @@ -180,10 +180,16 @@ for target in $(printf '%s\n' "${!TARGET_CLASSIFIERS[@]}" | sort); do echo "OK: $(basename "$out_jar") ($(du -h "$out_jar" | cut -f1);$classifiers)" done -# --- 4. At least the four release targets were produced ------------------------------------ -for target in linux-x86-64 linux-aarch64 windows-x86-64 windows-aarch64; do - [ -f "$OUT_DIR/llama-$VERSION-all-$target-jar-with-dependencies.jar" ] || fail "no all-$target fat jar produced" -done +# --- 4. Exactly the targets the list derives were produced --------------------------------- +# check-natives.py derives them from natives.csv as well and checks every consumer against the +# same derivation (smoke jobs, the agent jar's Class-Path, README), so a target this loop drops +# or invents cannot reach a release unlaunched. +produced="$(cd "$OUT_DIR" && ls llama-"$VERSION"-all-*-jar-with-dependencies.jar \ + | sed -e "s/^llama-$VERSION-all-//" -e 's/-jar-with-dependencies\.jar$//' | sort)" +expected="$(python3 "$(dirname "$0")/check-natives.py" fatjar-targets | sort)" +[ -n "$expected" ] || fail "check-natives.py derived no all-backends fat jar from $LIST" +[ "$produced" = "$expected" ] \ + || fail "all-backends fat jars produced: $(echo $produced) -- natives.csv derives: $(echo $expected)" # --- 5. The default (all-platform CPU) fat jar is a release asset too ------------------------- cp "$BASE_FAT_JAR" "$OUT_DIR/" diff --git a/.github/print-crash-logs.sh b/.github/print-crash-logs.sh new file mode 100755 index 000000000..faa409ed9 --- /dev/null +++ b/.github/print-crash-logs.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# Cross-repo shared script -- kept BYTE-IDENTICAL in java-llama.cpp, BitcoinAddressFinder, +# srcmorph and streambuffer (listed in each repo's .github/shared-files.sha256). The "Print crash +# logs" step of workspace/policies/ci-test-diagnostics.md section 3.1: echoes the JVM crash logs +# of a failed test job into the job log, because an uploaded artifact is unreadable from anywhere +# that cannot fetch from Azure Blob Storage (a phone, a restricted network, an agent sandbox). +# +# Usage: print-crash-logs.sh [...] (default: the current directory) +# For each module directory:

/hs_err_pid*.log (first 200 lines -- the diagnostic core is at +# the top, the tail is thread dumps and the memory map) and +# /target/surefire-reports/*.dumpstream|*.dump (whole; they are small). +# +# Always exits 0: it runs under `if: failure()` and must never replace the job's real failure. +shopt -s nullglob +[ "$#" -gt 0 ] || set -- . +found=0 +for dir in "$@"; do + for f in "$dir"/hs_err_pid*.log; do + found=1 + echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" + sed -n '1,200p' "$f" + done + for f in "$dir"/target/surefire-reports/*.dumpstream "$dir"/target/surefire-reports/*.dump; do + found=1 + echo "===== $f =====" + cat "$f" + done +done +if [ "$found" = 0 ]; then + # Worded defensively on purpose (policy section 3): the step fires on EVERY job failure, and + # an ordinary assertion failure writes no crash log. Never assert an abort from a missing file. + echo "No hs_err_pid*.log and no surefire dump/dumpstream was written (looked in: $*)." + echo + echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" + echo "any job failure, and an assertion failure, a timeout or a compile error writes no" + echo "crash log. Read the surefire output above for the real cause." + echo + echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" + echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" + echo "test results. In that case the abort bypassed the JVM error handler (a native" + echo "exit()/terminate() rather than a raised signal), which is why no file was written." +fi +exit 0 diff --git a/.github/print-host-info.sh b/.github/print-host-info.sh new file mode 100755 index 000000000..47fff5d53 --- /dev/null +++ b/.github/print-host-info.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +# Prints the runner's CPU, for reading a native build or test log: which instruction sets the +# compiler saw (GGML_NATIVE, the x86 variants) and how many cores the job had. Informational only +# -- it never fails the step. One script for every OS (the step runs `shell: bash`, which is Git +# Bash on Windows). +case "$(uname -s)" in + Linux) + echo "=== CPU information (lscpu) ===" + lscpu || true + echo + echo "=== CPU details (/proc/cpuinfo) ===" + cat /proc/cpuinfo || true + ;; + Darwin) + echo "=== CPU information (sysctl) ===" + sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu \ + hw.packages hw.memsize hw.ncpu 2>/dev/null || true + echo + echo "=== Processor details (system_profiler) ===" + system_profiler SPHardwareDataType || true + ;; + MINGW* | MSYS* | CYGWIN*) + echo "=== CPU information (Get-CimInstance Win32_Processor) ===" + powershell.exe -NoProfile -Command 'Get-CimInstance Win32_Processor | Select-Object * | Format-List' || true + echo "=== Processor (systeminfo) ===" + systeminfo | grep -i processor || true + ;; + *) + echo "no CPU report for $(uname -s)" + ;; +esac +exit 0 diff --git a/.github/release-gate-exemptions.txt b/.github/release-gate-exemptions.txt new file mode 100644 index 000000000..21e2b590b --- /dev/null +++ b/.github/release-gate-exemptions.txt @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# Jobs of publish.yml that do not gate both publish jobs, each with the reason it may stay red. +# Every other job must gate both; check-release-gate.py (code-style) fails on a job that does not, +# and on a line here that names no job or a job that gates both after all. +# Format: : +publish-snapshot: a publish job itself +publish-release: a publish job itself +check-snapshot: publish-snapshot's own precondition (the version is a SNAPSHOT) +check-tag: publish-release's own precondition (the version matches the tag) +github-snapshot: runs after publishing; attaches the fat jars to the rolling snapshot release +github-release-signed: runs after publishing; attaches the signed fat jars to the tag release +verify-signing-key: red by design where the secret is withheld (forks, other contributors' branches); the publish jobs sign with the same key and fail on it themselves +verify-signing-key-gradle: same as verify-signing-key, for the Gradle/BouncyCastle signing path +test-java-llama-langchain4j-integration: validation only; a cold model cache degrades it to self-skips (CLAUDE.md, LangChain4j integration) +build-android-llmservice: an app, not a library release; a Compose/AGP pin must not block a release (CLAUDE.md, Android app "LLM Service") +test-android-llmservice: the emulator UI test of that app, optional for the same reason diff --git a/.github/shared-files.sha256 b/.github/shared-files.sha256 new file mode 100644 index 000000000..da88e36dd --- /dev/null +++ b/.github/shared-files.sha256 @@ -0,0 +1,31 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT +# +# The files this repository keeps BYTE-IDENTICAL with java-llama.cpp, BitcoinAddressFinder, +# srcmorph and streambuffer (each lists the ones it carries). Checked by the `shared-files` job of +# .github/workflows/publish.yml (.github/check-shared-files.py, see .github/buildcheck/sharedfiles.py): +# a copy changed here alone fails it, a copy another repository lists with a different hash warns. +# To change a shared file: change it in every repository listing it, then run +# `python3 .github/check-shared-files.py --write` in each. `sha256sum -c` reads this file too. +82171de97c6621b4c1f3e9cd66afd49990f6d31331857423b37f2b9a2f35fbc0 .github/buildcheck/__init__.py +3d1a9f2e93709941261a9655bc19bf50d5becefd8f6a6c2f5564096a757996bb .github/buildcheck/workflow.py +5a70eee1bb8af180edfd2a4bae61be61fac15b409f5df6a8bb9869425d7fc381 .github/buildcheck/releasegate.py +86545e924c95b69a16f4b9f759a38779b441aa04d8b635a52eb029a196e39a11 .github/buildcheck/sharedfiles.py +0059120aab539c6ce8055675ae0d3b040097f80320051e40385072f68845ddb0 .github/buildcheck/tests/__init__.py +561efd3f26d7728e475c8ce83c10b4761a6703b585b7b2a7ab052df10f73166c .github/buildcheck/tests/helpers.py +4985f25fa177bd79a79335742adfd0da0431e101124b4e7e214906b56ae903de .github/buildcheck/tests/test_workflow.py +cbe504ad081ba31ec16e11c11f879c64179ca7741bf0b1d1318a7fc969ed67dd .github/buildcheck/tests/test_releasegate.py +7cbef54c09c6612503da7bd3733b4ab51c6f34666ef062885a7586e075a907de .github/buildcheck/tests/test_sharedfiles.py +7fe208dd33184b4543328f97b339c95e23c408b120e6bfa6095de4182aeb9a19 .github/check-release-gate.py +1604ccdea83998814a3c1f98f746272222039867b0c7d692355e61495c1c648b .github/check-shared-files.py +7da53fdb9537ccf7d3f3740be31f295bacb914fe186304b9ed8a65fd3d3d413e .github/print-crash-logs.sh +c8c02dc39aeacc154fad74071443c6513666aa94ca9f9c5fab49c5f39dea7ee6 .github/verify-signing-key.sh +e0b2a4f331eae326b61c71f78d6272d06d99140a85dedb7710f93402de862466 .github/verify-bytecode-version.sh +ab45f5c102b47dd16c325d4d9c283d158ba90c05f484eac45b2767885c4462f9 .github/signing-selftest/build.gradle.kts +9b2ea5b5ff8d48607e26e4e211ad6d496f7660e71c84e42caaa82b84f7001710 .github/signing-selftest/settings.gradle.kts +7232b092d3ba49b97bee7b539aaf6ee4c698e86bd3d4dd256e8ae2f85f653ee9 .github/ISSUE_TEMPLATE/bug_report.md +0f08122e597f93dbbdc9c80e88984b4bf4738951d5902813df3d4640cdb11bac .github/ISSUE_TEMPLATE/feature_request.md +ebfcc0adf59f5858bbe4dc077c906304a197f72a55256f0d5aac669bee5e871f .github/PULL_REQUEST_TEMPLATE.md +d3ca255bda928a293883c7c9c1718dde3f7e69e230af54f9fa9c08c804d7dfd3 .github/sign-fatjars.sh +42f1842270af691bdfe561355bee4eb9ae326383f1852db19763abb888d6b90e llama/lombok.config diff --git a/.github/sign-fatjars.sh b/.github/sign-fatjars.sh index c430c39a6..e354ecdad 100755 --- a/.github/sign-fatjars.sh +++ b/.github/sign-fatjars.sh @@ -4,8 +4,9 @@ # # SPDX-License-Identifier: MIT OR Apache-2.0 -# Cross-repo shared script — kept BYTE-IDENTICAL in java-llama.cpp and srcmorph (sync any -# edit to both). GPG-signs the fat jars (jar-with-dependencies) in a directory with a +# Cross-repo shared script — kept BYTE-IDENTICAL in java-llama.cpp and srcmorph (listed in each +# repo's .github/shared-files.sha256: sync any edit to both, then `check-shared-files.py --write`). +# GPG-signs the fat jars (jar-with-dependencies) in a directory with a # detached, armored .asc signature — the authenticity counterpart to any .sha256 integrity # file. The caller builds/collects the fat jars; this only signs every # *-jar-with-dependencies*.jar it finds in (thin jars are left untouched). diff --git a/.github/validate-models.bat b/.github/validate-models.bat deleted file mode 100644 index 2a21c4b54..000000000 --- a/.github/validate-models.bat +++ /dev/null @@ -1,85 +0,0 @@ -REM SPDX-FileCopyrightText: 2026 Bernard Ladenthin -REM SPDX-FileCopyrightText: 2023-2025 Konstantin Herud -REM -REM SPDX-License-Identifier: MIT - -@echo off -REM Validate that all required model files exist and are valid GGUF files -REM GGUF files start with magic bytes: 0x47 0x47 0x55 0x46 ("GGUF") - -setlocal enabledelayedexpansion - -REM Every CI Java test job (incl. Windows) restores the shared model cache before -REM validating and runs the embedding / vision / TTS integration tests, so all of -REM these are REQUIRED (a missing one is a hard failure, not a silent self-skip). -REM -REM The required set comes from the single source of truth .github\models.csv -REM (filename,url per line; # comments ignored) so this gate can never drift from -REM what download-models actually fetches. The list is built UNQUOTED (no filename -REM contains spaces): an earlier hardcoded form embedded literal quotes into the -REM variable, which made the for-loop see the whole list as ONE token — `if not -REM exist` then parsed it as path-plus-command, so only a fragment was ever checked -REM and the exit /b 1 never fired (the ERROR printed but the step passed; observed -REM in run 28805360584, where an empty model cache sailed through this "gate"). -if not exist "%~dp0models.csv" ( - echo ERROR: model manifest not found: %~dp0models.csv - exit /b 1 -) -set MODELS= -for /f "usebackq eol=# tokens=1 delims=," %%N in ("%~dp0models.csv") do set "MODELS=!MODELS! models\%%N" -if "!MODELS!"=="" ( - echo ERROR: no models parsed from %~dp0models.csv - exit /b 1 -) - -REM No optional models remain (the audio-input model has no CI download and its -REM test self-skips). Left empty so the optional loop below is a no-op. -set "OPTIONAL_MODELS=" - -echo Validating required model files... -for %%M in (%MODELS%) do ( - if not exist "%%M" ( - echo ERROR: Model not found: %%M - exit /b 1 - ) - - REM Check file size using PowerShell - for /f %%S in ('powershell -Command "(Get-Item '%%M').Length"') do set "size=%%S" - - if !size! lss 4 ( - echo ERROR: Model file too small (likely corrupted^): %%M (size: !size! bytes^) - exit /b 1 - ) - - REM Check GGUF magic bytes using PowerShell: 47475546 in hex = GGUF in ASCII - for /f %%H in ('powershell -Command "[System.BitConverter]::ToString((Get-Content '%%M' -Encoding Byte -ReadCount 4)[0]) -replace '-',''"') do set "magic=%%H" - - if not "!magic!"=="47475546" ( - echo ERROR: Invalid GGUF magic bytes in %%M (got: !magic!, expected: 47475546^) - exit /b 1 - ) - - echo OK: %%M ^(!size! bytes^) -) - -echo Validating optional vision model files... -for %%M in (%OPTIONAL_MODELS%) do ( - if not exist "%%M" ( - echo SKIP: %%M not present - ) else ( - for /f %%S in ('powershell -Command "(Get-Item '%%M').Length"') do set "size=%%S" - if !size! lss 4 ( - echo ERROR: Model file too small (likely corrupted^): %%M (size: !size! bytes^) - exit /b 1 - ) - for /f %%H in ('powershell -Command "[System.BitConverter]::ToString((Get-Content '%%M' -Encoding Byte -ReadCount 4)[0]) -replace '-',''"') do set "magic=%%H" - if not "!magic!"=="47475546" ( - echo ERROR: Invalid GGUF magic bytes in %%M (got: !magic!, expected: 47475546^) - exit /b 1 - ) - echo OK: %%M ^(!size! bytes^) - ) -) - -echo All models validated successfully! -exit /b 0 diff --git a/.github/validate-models.sh b/.github/validate-models.sh index bdfbbd38e..2ee415c7f 100755 --- a/.github/validate-models.sh +++ b/.github/validate-models.sh @@ -21,6 +21,7 @@ MODELS_CSV="$(dirname "$0")/models.csv" [ -f "${MODELS_CSV}" ] || { echo "ERROR: model manifest not found: ${MODELS_CSV}"; exit 1; } MODELS=() while IFS=, read -r name _url; do + name="${name%$'\r'}" # a CRLF checkout on Windows case "$name" in ''|\#*) continue ;; esac MODELS+=("models/$name") done < "${MODELS_CSV}" @@ -45,16 +46,18 @@ validate_gguf() { return fi fi + # wc and head, not stat/xxd: the same script runs under Linux, macOS (BSD tools) and Git Bash + # on Windows, which has no xxd and whose stat does not read BSD's -f. local size - size=$(stat -f%z "$model" 2>/dev/null || stat -c%s "$model" 2>/dev/null) + size=$(wc -c < "$model" | tr -d ' ') if [[ $size -lt 4 ]]; then echo "ERROR: Model file too small (likely corrupted): $model (size: $size bytes)" exit 1 fi local magic - magic=$(xxd -p -l 4 "$model") - if [[ "$magic" != "47475546" ]]; then - echo "ERROR: Invalid GGUF magic bytes in $model (got: $magic, expected: 47475546)" + magic=$(head -c 4 "$model") + if [[ "$magic" != "GGUF" ]]; then + echo "ERROR: Invalid GGUF magic bytes in $model (got: $(head -c 4 "$model" | od -An -tx1 | tr -d ' '), expected: 47475546 'GGUF')" exit 1 fi echo "✓ $model ($(numfmt --to=iec-i --suffix=B $size 2>/dev/null || echo $size bytes))" diff --git a/.github/verify-bytecode-version.sh b/.github/verify-bytecode-version.sh index a7a395468..384f3c962 100755 --- a/.github/verify-bytecode-version.sh +++ b/.github/verify-bytecode-version.sh @@ -5,8 +5,9 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 # Cross-repo shared script — kept BYTE-IDENTICAL in java-llama.cpp, srcmorph, -# BitcoinAddressFinder and streambuffer (sync any edit to all four, and to the checksum table in -# workspace/crossrepostatus.md). Fails when a built jar contains a class file newer than the Java +# BitcoinAddressFinder and streambuffer (listed in each repo's .github/shared-files.sha256, which +# the `shared-files` job checks: sync any edit to all four, then `check-shared-files.py --write`). +# Fails when a built jar contains a class file newer than the Java # release the artifact claims to support. # # Why: `maven.compiler.release` governs only the code WE compile. A dependency compiled for a newer diff --git a/.github/verify-hip-offload-compressed.py b/.github/verify-hip-offload-compressed.py old mode 100644 new mode 100755 index 343d04d17..b6059118f --- a/.github/verify-hip-offload-compressed.py +++ b/.github/verify-hip-offload-compressed.py @@ -1,86 +1,17 @@ +#!/usr/bin/env python3 # SPDX-FileCopyrightText: 2026 Bernard Ladenthin # # SPDX-License-Identifier: MIT - """Fail when a ROCm/HIP build ships its GPU code uncompressed. -llama/CMakeLists.txt compiles ggml-hip with clang's --offload-compress, so every embedded -device-code bundle is a compressed "CCOB" bundle. An uncompressed bundle starts with the magic -"__CLANG_OFFLOAD_BUNDLE__"; one of those in the shipped library means the flag was lost and the -library is back to carrying every GPU target's code uncompressed (~1 GB on Windows). - -Usage: verify-hip-offload-compressed.py ... - -Scans every jllama.dll / libjllama.so found, prints its size and the bundle counts, and exits -non-zero if a library has an uncompressed bundle, has no compressed bundle at all, or if no -library was found. Only the standard library, so it runs on any runner. +See buildcheck/hipoffload.py. Usage: verify-hip-offload-compressed.py ... """ import os import sys -UNCOMPRESSED = b"__CLANG_OFFLOAD_BUNDLE__" -COMPRESSED = b"CCOB" -NAMES = ("jllama.dll", "libjllama.so") -CHUNK = 64 * 1024 * 1024 - - -def count(path): - """Counts both magics in one streaming pass (the file can be ~1 GB).""" - overlap = len(UNCOMPRESSED) - 1 - uncompressed = compressed = 0 - tail = b"" - with open(path, "rb") as f: - while True: - block = f.read(CHUNK) - if not block: - break - data = tail + block - # a match lying entirely inside `tail` was counted in the previous round - uncompressed += data.count(UNCOMPRESSED) - tail.count(UNCOMPRESSED) - compressed += data.count(COMPRESSED) - tail.count(COMPRESSED) - tail = data[-overlap:] - return uncompressed, compressed - - -def libraries(paths): - for p in paths: - if os.path.isfile(p): - yield p - for root, _, files in os.walk(p): - for name in files: - if name in NAMES: - yield os.path.join(root, name) - - -def main(): - if len(sys.argv) < 2: - print("usage: verify-hip-offload-compressed.py ...", file=sys.stderr) - return 2 - found = failed = 0 - summary = [] - for lib in libraries(sys.argv[1:]): - found += 1 - size = os.path.getsize(lib) - uncompressed, compressed = count(lib) - line = f"{lib}: {size / 1024 / 1024:.0f} MiB, {compressed} compressed / {uncompressed} uncompressed offload bundle(s)" - print(line) - summary.append(line) - if uncompressed: - print(f"::error::{lib} embeds {uncompressed} uncompressed GPU code bundle(s); is --offload-compress still set on ggml-hip?") - failed += 1 - elif not compressed: - print(f"::error::{lib} embeds no compressed GPU code bundle; was it built with GGML_HIP=ON?") - failed += 1 - if not found: - print(f"::error::no {' / '.join(NAMES)} found under {sys.argv[1:]}") - return 2 - step_summary = os.environ.get("GITHUB_STEP_SUMMARY") - if step_summary: - with open(step_summary, "a", encoding="utf-8") as f: - f.write("### ROCm/HIP device code\n\n" + "\n".join(f"- `{s}`" for s in summary) + "\n") - return 1 if failed else 0 - +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import hipoffload # noqa: E402 if __name__ == "__main__": - sys.exit(main()) + sys.exit(hipoffload.main(sys.argv)) diff --git a/.github/verify-native-deps.py b/.github/verify-native-deps.py index d8edd5750..d21e70937 100755 --- a/.github/verify-native-deps.py +++ b/.github/verify-native-deps.py @@ -4,213 +4,14 @@ # SPDX-License-Identifier: MIT """Fail when a shipped native library needs a runtime library it did not need before. -Every jllama library is ONE file with llama.cpp and ggml linked in statically, so its dynamic -dependencies are exactly what a consumer's machine must provide. A new one is a silent break on -every machine that lacks it -- the case this guards against is ggml-rpc's RDMA transport, which -upstream switches on whenever the build host has libibverbs/librdma and which would make the -library unloadable without rdma-core. It reads the dependency list straight from the file (ELF -DT_NEEDED, PE import table, Mach-O LC_LOAD_DYLIB) with the standard library only, so it runs on -any runner and checks every architecture, including the ones binutils cannot read (Windows arm64, -Mach-O). - -Usage: - verify-native-deps.py - -Checks every library under /net/ladenthin/llama////. The CPU -builds (backend cpu, metal, msvc) are held to the exact allowlist in ALLOWED: a dependency outside -it fails, and so does a CPU build without a list (a new platform must be listed consciously). The -GPU backends are checked against DENIED only, because they legitimately need their vendor runtime. -That every listed build arrived is merge-native-artifacts.sh's check. - -Exit codes: 0 clean, 1 violation, 2 nothing found to check. +See buildcheck/nativedeps.py. Usage: verify-native-deps.py """ import os -import struct import sys -# What each CPU library needed when this check was introduced (5.1.0 plus the RPC backend, -# which adds nothing: its sockets are libc/libSystem/WS2_32, all already present). -ALLOWED = { - "Linux/x86_64/cpu": {"libdl.so.2", "libgomp.so.1", "libpthread.so.0", "librt.so.1", "libstdc++.so.6", - "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld-linux-x86-64.so.2"}, - "Linux/aarch64/cpu": {"libgomp.so.1", "libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", - "ld-linux-aarch64.so.1"}, - "Linux/s390x/cpu": {"libstdc++.so.6", "libm.so.6", "libgcc_s.so.1", "libc.so.6", "ld64.so.1"}, - "Linux-Android/aarch64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, - "Linux-Android/x86_64/cpu": {"liblog.so", "libm.so", "libdl.so", "libc.so", "libandroid.so"}, - "Windows/x86_64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, - "Windows/x86/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll", "vcomp140.dll"}, - "Windows/aarch64/cpu": {"ws2_32.dll", "kernel32.dll", "shell32.dll", "advapi32.dll"}, - "Mac/aarch64/metal": {"/usr/lib/libc++.1.dylib", "/usr/lib/libSystem.B.dylib", - "/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation", - "/System/Library/Frameworks/Metal.framework/Versions/A/Metal", - "/System/Library/Frameworks/MetalKit.framework/Versions/A/MetalKit", - "/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate", - "/usr/lib/libobjc.A.dylib", - "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation", - "/System/Library/Frameworks/Security.framework/Versions/A/Security", - # KNOWN DEFECT, allowed only so this check reports NEW dependencies: the macOS - # build picks up the runner's Homebrew OpenSSL, so the shipped dylib does not load - # on a Mac without `brew install openssl@3`. See TODO.md ("macOS dylib links - # Homebrew OpenSSL"); remove these two lines with the fix. - "/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib", - "/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib"}, -} -# The Visual Studio generator build of the same compiler and runtime. -ALLOWED["Windows/x86_64/msvc"] = ALLOWED["Windows/x86_64/cpu"] -ALLOWED["Windows/x86/msvc"] = ALLOWED["Windows/x86/cpu"] -CPU_BACKENDS = ("cpu", "metal", "msvc") -LIBRARY_NAMES = ("libjllama.so", "jllama.dll", "libjllama.dylib") - -# Never acceptable in any artifact: libraries a consumer cannot be expected to have. -DENIED = ("libibverbs", "librdma", "rdma.dylib", "libmlx") - -LIB_NAMES = ("libjllama.so", "jllama.dll", "libjllama.dylib") - - -def elf_needed(data): - if data[:4] != b"\x7fELF": - raise ValueError("not an ELF file") - is64 = data[4] == 2 - end = "<" if data[5] == 1 else ">" - if is64: - shoff = struct.unpack_from(end + "Q", data, 0x28)[0] - shentsize, shnum = struct.unpack_from(end + "HH", data, 0x3A) - else: - shoff = struct.unpack_from(end + "I", data, 0x20)[0] - shentsize, shnum = struct.unpack_from(end + "HH", data, 0x2E) - sections = [] - for i in range(shnum): - off = shoff + i * shentsize - if is64: - _, sh_type, _, _, sh_offset, sh_size, sh_link = struct.unpack_from(end + "IIQQQQI", data, off) - else: - _, sh_type, _, _, sh_offset, sh_size, sh_link = struct.unpack_from(end + "IIIIIII", data, off) - sections.append((sh_type, sh_offset, sh_size, sh_link)) - out = [] - for sh_type, sh_offset, sh_size, sh_link in sections: - if sh_type != 6: # SHT_DYNAMIC - continue - strtab = sections[sh_link] - entry = 16 if is64 else 8 - for off in range(sh_offset, sh_offset + sh_size, entry): - tag, val = struct.unpack_from(end + ("qQ" if is64 else "iI"), data, off) - if tag == 0: - break - if tag == 1: # DT_NEEDED - start = strtab[1] + val - out.append(data[start:data.index(b"\0", start)].decode()) - return out - - -def pe_imports(data): - if data[:2] != b"MZ": - raise ValueError("not a PE file") - pe = struct.unpack_from("= 4 else "" - if key.rsplit("/", 1)[-1] not in CPU_BACKENDS or os.path.basename(path) not in LIBRARY_NAMES: - continue - allowed = ALLOWED.get(key) - if allowed is None: - failures.append(f"{rel}: no dependency allowlist for '{key}' -- add one to ALLOWED") - continue - for d in deps: - if d.lower() not in {a.lower() for a in allowed}: - failures.append(f"{rel} needs {d}, which it did not need before (allowed: {sorted(allowed)})") - if checked == 0: - print(f"no native library found under {root}", file=sys.stderr) - return 2 - for f in failures: - print(f"::error::{f}", file=sys.stderr) - print(f"{checked} native libraries checked, {len(failures)} violations") - return 1 if failures else 0 - +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import nativedeps # noqa: E402 if __name__ == "__main__": - sys.exit(main(sys.argv)) + sys.exit(nativedeps.main(sys.argv)) diff --git a/.github/verify-signing-key.sh b/.github/verify-signing-key.sh new file mode 100755 index 000000000..8940490b0 --- /dev/null +++ b/.github/verify-signing-key.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# Cross-repo shared script -- kept BYTE-IDENTICAL in java-llama.cpp, BitcoinAddressFinder, +# srcmorph and streambuffer (listed in each repo's .github/shared-files.sha256). The body of the +# `verify-signing-key` job: reproduces what maven-gpg-plugin does at deploy time, so a bad or +# expired key or a wrong passphrase is caught in seconds instead of failing the publish stage. +# +# Input: GPG_PRIVATE_KEY (armored secret key) and GPG_PASSPHRASE in the environment. +# +# SECURITY: prints no secret material. The key is imported into an ephemeral keyring via stdin; +# only PUBLIC key metadata is printed (key id, fingerprint, owner UID, algorithm, created/expiry, +# all of which live on public keyservers); the passphrase is validated by producing and verifying +# a throwaway signature, reaches gpg on fd 3 only (never argv, never a log line), and is +# additionally `::add-mask::`ed. `set -x` must never be enabled here. +set -euo pipefail # NOTE: deliberately NO `set -x` — it would echo the passphrase. + +if [ -z "${GPG_PRIVATE_KEY:-}" ]; then + echo "::error::GPG_PRIVATE_KEY is empty for this run. Either the secret is not set, or it is scoped to a different environment/branch than 'maven-central' on this ref. Nothing to verify." + exit 1 +fi +# Defensive: even though we never print it, register the passphrase as a +# masked value so any accidental echo downstream is redacted. +if [ -n "${GPG_PASSPHRASE:-}" ]; then echo "::add-mask::${GPG_PASSPHRASE}"; fi + +echo "gpg: $(gpg --version | head -n1)" + +# Ephemeral, private keyring; removed on exit. +export GNUPGHOME="$(mktemp -d)" +chmod 700 "$GNUPGHOME" +cleanup() { gpgconf --kill gpg-agent >/dev/null 2>&1 || true; rm -rf "$GNUPGHOME"; } +trap cleanup EXIT + +echo "== Import private key into an ephemeral keyring (key via stdin, never argv) ==" +printf '%s\n' "$GPG_PRIVATE_KEY" | gpg --batch --import + +COLONS="$(gpg --list-secret-keys --with-colons --fixed-list-mode)" +SECCOUNT="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{n++} END{print n+0}')" +echo "Secret keys imported: $SECCOUNT" +if [ "$SECCOUNT" -lt 1 ]; then + echo "::error::No secret key was imported — GPG_PRIVATE_KEY is not a valid armored secret key (check that the secret contains the full -----BEGIN PGP PRIVATE KEY BLOCK----- with intact newlines)." + exit 1 +fi + +KEYID="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{print $5; exit}')" +ALGO="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{print $4; exit}')" +CREATED="$(printf '%s\n' "$COLONS"| awk -F: '$1=="sec"{print $6; exit}')" +EXPIRES="$(printf '%s\n' "$COLONS"| awk -F: '$1=="sec"{print $7; exit}')" +FPR="$(printf '%s\n' "$COLONS" | awk -F: '$1=="fpr"{print $10; exit}')" + +echo "== PUBLIC key metadata ==" +echo " Key ID (long): $KEYID" +echo " Fingerprint: $FPR" +echo " Pubkey algo id: $ALGO" +echo " Created (UTC): $(date -u -d "@$CREATED" 2>/dev/null || echo "$CREATED")" +echo " Owner UID(s):" +printf '%s\n' "$COLONS" | awk -F: '$1=="uid"{print " - " $10}' + +# --- Expiration gate --- +NOW="$(date -u +%s)" +if [ -n "$EXPIRES" ]; then + echo " Expires (UTC): $(date -u -d "@$EXPIRES" 2>/dev/null || echo "$EXPIRES")" + if [ "$EXPIRES" -le "$NOW" ]; then + echo "::error::Signing key is EXPIRED — Maven Central will reject its signatures. Extend the key's expiry and update the GPG_PRIVATE_KEY secret." + exit 1 + fi + echo " Days to expiry: $(( (EXPIRES - NOW) / 86400 ))" + if [ "$(( (EXPIRES - NOW) / 86400 ))" -lt 30 ]; then + echo "::warning::Signing key expires in under 30 days — plan to rotate it." + fi +else + echo " Expires (UTC): never" +fi + +# --- Signing-capability gate --- +if printf '%s\n' "$COLONS" | awk -F: '($1=="sec"||$1=="ssb"){print $12}' | grep -q 's'; then + echo " Signing capability: present" +else + echo "::error::No signing-capable (sub)key found — this key cannot produce release signatures." + exit 1 +fi + +# --- Passphrase unlock + sign + verify roundtrip (the exact failure mode) --- +# Passphrase on fd 3 only. Payload is a throwaway nonce; only the signature's +# validity (exit codes) matters — no secret is ever emitted. +echo "== Passphrase unlock + detached-sign + verify self-test ==" +WORK="$(mktemp -d)" +printf '%s' "ai-index signing-selftest" > "$WORK/payload.txt" +gpg --batch --yes --pinentry-mode loopback --passphrase-fd 3 \ + --local-user "$KEYID" \ + --detach-sign --armor --output "$WORK/payload.txt.asc" "$WORK/payload.txt" \ + 3<<<"${GPG_PASSPHRASE:-}" +echo " Signature produced: $(wc -c < "$WORK/payload.txt.asc") armored bytes" +gpg --batch --verify "$WORK/payload.txt.asc" "$WORK/payload.txt" +rm -rf "$WORK" + +echo "RESULT: OK — key imports, is not expired, is signing-capable, and the passphrase successfully unlocked it to produce a VALID signature. maven-gpg-plugin will be able to sign with this key/passphrase." diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 8ee750c9f..066ad9686 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -21,38 +21,24 @@ on: default: true env: JAVA_VERSION: '21' - # Model DOWNLOAD URLS live in .github/models.csv (the single source of truth for the - # CI model set; the model cache key is derived from that file's hash). The *_NAME - # vars below are consumer-side wiring only (-Dnet.ladenthin.llama.* test properties) - # and must match the filename column of models.csv. - MODEL_NAME: "codellama-7b.Q2_K.gguf" + # The Gradle builds (Android AAR, the consumer fixture, the llmservice app, the signing self-test). + # AGP 9.4.x needs Gradle >= 9.6.0 (CLAUDE.md, Android app). Not seen by Dependabot: bump by hand. + GRADLE_VERSION: '9.8.0' + # The sccache/Depot compiler cache (CLAUDE.md, "CI build cache"). Inert without the token, which + # stays per build job (SCCACHE_WEBDAV_TOKEN): at workflow level it would reach every job, + # including those running third-party actions. + USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} + SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev + # The CI model set is .github/models.csv (URLs, and the model cache key is its hash). The Java + # tests default to exactly that set (TestConstants; TestConstantsTest asserts both directions), + # so the test jobs pass no model properties. The names below are only for the consumers outside + # the llama module's tests -- the smoke scripts, the Android emulator jobs and the langchain4j / + # agent integration jobs; check-natives.py fails when one is not a filename of models.csv. RERANKING_MODEL_NAME: "jina-reranker-v1-tiny-en-Q4_0.gguf" DRAFT_MODEL_NAME: "AMD-Llama-135m-code.Q2_K.gguf" REASONING_MODEL_NAME: "Qwen3-0.6B-Q4_K_M.gguf" TOOL_MODEL_NAME: "Qwen2.5-1.5B-Instruct-Q4_K_M.gguf" NOMIC_EMBED_MODEL_NAME: "nomic-embed-text-v1.5.f16.gguf" - # Vision model + mmproj for MultimodalIntegrationTest. - # SmolVLM-500M is the smallest community vision GGUF that loads reliably - # under the upstream mtmd pipeline. - VISION_MODEL_NAME: "SmolVLM-500M-Instruct-Q8_0.gguf" - VISION_MMPROJ_NAME: "mmproj-SmolVLM-500M-Instruct-Q8_0.gguf" - # Qwen3-TTS backbone + mmproj for TtsIntegrationTest (the OuteTTS+WavTokenizer pair this replaced - # was retired when upstream #26254 replaced the whole TTS pipeline — see - # docs/history/llama-cpp-breaking-changes.md, b10269-b10270). Smallest available quants: - # Q4_K_M backbone (0.96 GiB), Q8_0 mmproj (0.42 GiB; no smaller mmproj quant is published). - TTS_MODEL_NAME: "Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf" - TTS_MMPROJ_NAME: "mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf" - # Fine-tuning fixture for LlamaTrainerIntegrationTest. stories260K is 1.19 MB and, unlike every - # other model here, is **F32** — which is the only reason it works: llama_set_param silently skips - # any tensor that is not GGML_TYPE_F32, so a quantized model would "train" nothing but the norm - # weights and still write an output GGUF that the test's assertions accept. Its small trained - # context also suits the test's short corpus; the test pins nCtx itself so that is belt-and-braces. - TRAIN_MODEL_NAME: "stories260K.gguf" - # Test image used by MultimodalIntegrationTest is committed to the repo - # at src/test/resources/images/test-image.jpg (see the README in that - # directory for licensing). No download step is needed; CI just points - # mvn test at the committed path. - VISION_IMAGE_PATH: "llama/src/test/resources/images/test-image.jpg" # Supersede an in-flight run when a PR branch is pushed again. # # Without this every push starts a full parallel pipeline and the older ones keep @@ -100,6 +86,28 @@ jobs: steps: - run: echo "Start gate elapsed — proceeding with pipeline." + # --------------------------------------------------------------------------- + # Files kept byte-identical with the sibling repositories (java-llama.cpp, + # BitcoinAddressFinder, srcmorph, streambuffer), listed in .github/shared-files.sha256. + # A copy changed here alone fails; one the siblings list with another hash warns. + # Also runs the tests of the shared build checks and the release-gate check. This + # job is itself identical in all four repositories. + # --------------------------------------------------------------------------- + shared-files: + name: Shared files + build checks + needs: startgate + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - name: Shared files match .github/shared-files.sha256 (siblings compared, warnings only) + run: python3 .github/check-shared-files.py + - name: Build-check tests + run: python3 -m unittest discover -s .github/buildcheck/tests -t .github + - name: Release gate (every job gates both publish jobs, or release-gate-exemptions.txt says why) + run: python3 .github/check-release-gate.py + # --------------------------------------------------------------------------- # GPG signing-key preflight (standalone, no `needs:` — runs in parallel at the # very start on every trigger). Reproduces what maven-gpg-plugin does at deploy @@ -124,94 +132,15 @@ jobs: runs-on: ubuntu-latest environment: maven-central steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false - name: Import key + run sign/verify self-test (prints only PUBLIC metadata) - shell: bash env: GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }} GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }} - run: | - set -euo pipefail # NOTE: deliberately NO `set -x` — it would echo the passphrase. - - if [ -z "${GPG_PRIVATE_KEY:-}" ]; then - echo "::error::GPG_PRIVATE_KEY is empty for this run. Either the secret is not set, or it is scoped to a different environment/branch than 'maven-central' on this ref. Nothing to verify." - exit 1 - fi - # Defensive: even though we never print it, register the passphrase as a - # masked value so any accidental echo downstream is redacted. - if [ -n "${GPG_PASSPHRASE:-}" ]; then echo "::add-mask::${GPG_PASSPHRASE}"; fi - - echo "gpg: $(gpg --version | head -n1)" - - # Ephemeral, private keyring; removed on exit. - export GNUPGHOME="$(mktemp -d)" - chmod 700 "$GNUPGHOME" - cleanup() { gpgconf --kill gpg-agent >/dev/null 2>&1 || true; rm -rf "$GNUPGHOME"; } - trap cleanup EXIT - - echo "== Import private key into an ephemeral keyring (key via stdin, never argv) ==" - printf '%s\n' "$GPG_PRIVATE_KEY" | gpg --batch --import - - COLONS="$(gpg --list-secret-keys --with-colons --fixed-list-mode)" - SECCOUNT="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{n++} END{print n+0}')" - echo "Secret keys imported: $SECCOUNT" - if [ "$SECCOUNT" -lt 1 ]; then - echo "::error::No secret key was imported — GPG_PRIVATE_KEY is not a valid armored secret key (check that the secret contains the full -----BEGIN PGP PRIVATE KEY BLOCK----- with intact newlines)." - exit 1 - fi - - KEYID="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{print $5; exit}')" - ALGO="$(printf '%s\n' "$COLONS" | awk -F: '$1=="sec"{print $4; exit}')" - CREATED="$(printf '%s\n' "$COLONS"| awk -F: '$1=="sec"{print $6; exit}')" - EXPIRES="$(printf '%s\n' "$COLONS"| awk -F: '$1=="sec"{print $7; exit}')" - FPR="$(printf '%s\n' "$COLONS" | awk -F: '$1=="fpr"{print $10; exit}')" - - echo "== PUBLIC key metadata ==" - echo " Key ID (long): $KEYID" - echo " Fingerprint: $FPR" - echo " Pubkey algo id: $ALGO" - echo " Created (UTC): $(date -u -d "@$CREATED" 2>/dev/null || echo "$CREATED")" - echo " Owner UID(s):" - printf '%s\n' "$COLONS" | awk -F: '$1=="uid"{print " - " $10}' - - # --- Expiration gate --- - NOW="$(date -u +%s)" - if [ -n "$EXPIRES" ]; then - echo " Expires (UTC): $(date -u -d "@$EXPIRES" 2>/dev/null || echo "$EXPIRES")" - if [ "$EXPIRES" -le "$NOW" ]; then - echo "::error::Signing key is EXPIRED — Maven Central will reject its signatures. Extend the key's expiry and update the GPG_PRIVATE_KEY secret." - exit 1 - fi - echo " Days to expiry: $(( (EXPIRES - NOW) / 86400 ))" - if [ "$(( (EXPIRES - NOW) / 86400 ))" -lt 30 ]; then - echo "::warning::Signing key expires in under 30 days — plan to rotate it." - fi - else - echo " Expires (UTC): never" - fi - - # --- Signing-capability gate --- - if printf '%s\n' "$COLONS" | awk -F: '($1=="sec"||$1=="ssb"){print $12}' | grep -q 's'; then - echo " Signing capability: present" - else - echo "::error::No signing-capable (sub)key found — this key cannot produce release signatures." - exit 1 - fi - - # --- Passphrase unlock + sign + verify roundtrip (the exact failure mode) --- - # Passphrase on fd 3 only. Payload is a throwaway nonce; only the signature's - # validity (exit codes) matters — no secret is ever emitted. - echo "== Passphrase unlock + detached-sign + verify self-test ==" - WORK="$(mktemp -d)" - printf '%s' "ai-index signing-selftest" > "$WORK/payload.txt" - gpg --batch --yes --pinentry-mode loopback --passphrase-fd 3 \ - --local-user "$KEYID" \ - --detach-sign --armor --output "$WORK/payload.txt.asc" "$WORK/payload.txt" \ - 3<<<"${GPG_PASSPHRASE:-}" - echo " Signature produced: $(wc -c < "$WORK/payload.txt.asc") armored bytes" - gpg --batch --verify "$WORK/payload.txt.asc" "$WORK/payload.txt" - rm -rf "$WORK" - - echo "RESULT: OK — key imports, is not expired, is signing-capable, and the passphrase successfully unlocked it to produce a VALID signature. maven-gpg-plugin will be able to sign with this key/passphrase." + # Shared, byte-identical in all four repos; the security notes are in the script. + run: bash .github/verify-signing-key.sh # --------------------------------------------------------------------------- # GPG signing-key preflight — GRADLE / BouncyCastle path. @@ -241,11 +170,11 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: '21' + java-version: ${{ env.JAVA_VERSION }} distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Sign a throwaway artifact via useInMemoryPgpKeys (BouncyCastle) shell: bash env: @@ -331,25 +260,13 @@ jobs: strategy: fail-fast: false matrix: - include: - - os: ubuntu-latest - validate: bash .github/validate-models.sh - - os: macos-15 - validate: bash .github/validate-models.sh - - os: windows-2025-vs2026 - validate: .github\validate-models.bat + os: [ubuntu-latest, macos-15, windows-2025-vs2026] runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v7 - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - uses: actions/cache/restore@v6 + - uses: ./.github/actions/restore-models with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true fail-on-cache-miss: true - - name: Validate model files - run: ${{ matrix.validate }} # --------------------------------------------------------------------------- # Cross-compile jobs (Docker / dockcross) — produce release artifacts, no testing @@ -363,9 +280,10 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: '21' + java-version: ${{ env.JAVA_VERSION }} distribution: temurin - - name: Natives list check (pom, llama-platform, workflow and loader agree with natives.csv) + # .github/buildcheck/natives.py; its unit tests run in the shared-files job. + - name: Natives list check (everything that names a natives jar agrees with natives.csv) run: python3 .github/check-natives.py - name: Spotless check (fail fast on format violations) run: mvn -B --no-transfer-progress -f llama/pom.xml spotless:check @@ -453,16 +371,7 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -481,38 +390,11 @@ jobs: -Dnet.ladenthin.llama.langchain4j.tool.model=models/${TOOL_MODEL_NAME} -Dnet.ladenthin.llama.lib.path=${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu - # This job is model-backed and crosses JNI, so a forked test JVM here can abort exactly the - # way the six test-java-* jobs can -- but it had neither of their diagnostics. Same step and - # same path set as those, scoped to this module. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama-langchain4j/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama-langchain4j/target/surefire-reports/*.dumpstream llama-langchain4j/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama-langchain4j - if: failure() uses: actions/upload-artifact@v7 with: @@ -608,12 +490,7 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -632,20 +509,11 @@ jobs: -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} -Dnet.ladenthin.llama.test.ngl=0 -Dnet.ladenthin.llama.lib.path=${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/Linux/x86_64/cpu - # Model-backed and crossing JNI: same crash diagnostics as the langchain4j integration job. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - for f in llama-atmosphere-agent/hs_err_pid*.log; do - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama-atmosphere-agent/target/surefire-reports/*.dumpstream llama-atmosphere-agent/target/surefire-reports/*.dump; do - echo "===== $f =====" - cat "$f" - done + run: bash .github/print-crash-logs.sh llama-atmosphere-agent - if: failure() uses: actions/upload-artifact@v7 with: @@ -734,7 +602,7 @@ jobs: # all, which is why the naive grep used to be sufficient.) Assert the ACTIVE # #define and a non-zero asset count instead -- verified against both paths. N=$(sed -n 's/.*std::array.*/\1/p' "$OUT/ui.h" | head -1) - if grep -qE '^[[:space:]]*#define[[:space:]]+LLAMA_UI_HAS_ASSETS' "$OUT/ui.h" \ + if grep -qE '^[[:space:]]*#define[[:space:]]+LLAMA_UI_HAS_ASSETS' "$OUT/ui.h" && [ -n "$N" ] && [ "$N" -gt 0 ]; then echo "LLAMA_UI_HAS_ASSETS: present, $N assets embedded" else @@ -766,8 +634,6 @@ jobs: # prints the `sccache --show-stats` hit table at the end of every run. Inert without DEPOT_TOKEN # (fork PRs) or use_cache=false. env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE" steps: @@ -784,12 +650,7 @@ jobs: path: ${{ github.workspace }}/llama/webui-generated/ - name: Display CPU Info shell: bash - run: | - echo "=== Host CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + run: bash .github/print-host-info.sh - name: Build libraries shell: bash run: | @@ -811,8 +672,6 @@ jobs: # DEPOT_TOKEN (fork PRs) or with use_cache=false; a crashing sccache still falls back to a # green uncached build via the build.sh probe. env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE" steps: @@ -824,12 +683,7 @@ jobs: path: ${{ github.workspace }}/llama/webui-generated/ - name: Display CPU Info shell: bash - run: | - echo "=== Host CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + run: bash .github/print-host-info.sh - name: Build libraries shell: bash run: | @@ -854,8 +708,6 @@ jobs: # display name changed, so update any branch-protection required-check that pinned the old name. runs-on: ubuntu-24.04-arm env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -876,12 +728,7 @@ jobs: echo "CXX=g++-14" >> "$GITHUB_ENV" - name: Display CPU Info shell: bash - run: | - echo "=== Host CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + run: bash .github/print-host-info.sh - name: Build libraries shell: bash run: | @@ -911,8 +758,6 @@ jobs: runs-on: ubuntu-latest env: QEMU_LD_PREFIX: /usr/s390x-linux-gnu - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -965,8 +810,6 @@ jobs: # self-fetches sccache; the probe guards it (a miss just builds uncached). runs-on: ubuntu-latest env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1003,8 +846,6 @@ jobs: # to match the aarch64 CPU job. Ships as the vulkan-linux-aarch64 natives jar. runs-on: ubuntu-24.04-arm env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1043,8 +884,6 @@ jobs: # the build.sh probe makes it safe to enable without a separate verification run. Inert # without DEPOT_TOKEN (fork PRs) or use_cache=false. env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE" steps: @@ -1056,12 +895,7 @@ jobs: path: ${{ github.workspace }}/llama/webui-generated/ - name: Display CPU Info shell: bash - run: | - echo "=== Host CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + run: bash .github/print-host-info.sh - name: Build libraries shell: bash run: | @@ -1081,8 +915,6 @@ jobs: # and shipped as the cpu-android-x86-64 natives jar. Same dockcross + sccache steady-state env as the arm64 job; the wrapper # pins the same image tag. Fail-loud and in the package/publish needs graphs. env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE" steps: @@ -1112,8 +944,6 @@ jobs: # sccache probe + launcher). Same steady-state env as the other dockcross jobs. Inert # without DEPOT_TOKEN (fork PRs) or use_cache=false. env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE" steps: @@ -1160,7 +990,7 @@ jobs: - uses: gradle/actions/setup-gradle@v6 with: # AGP 9.4.0 (the .github/android-consumer-test fixture) requires Gradle >= 9.6.0. - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Build core jar (byte-identical classes payload for the AAR) run: > mvn -B --no-transfer-progress -pl llama -am -DskipTests -Denforcer.skip=true @@ -1189,7 +1019,7 @@ jobs: cp stage/opencl/Linux-Android/aarch64/opencl/libjllama.so llama-android/natives/opencl/arm64-v8a/ - name: Assemble AARs + publish to mavenLocal run: gradle -p llama-android aarCpu aarOpencl publishToMavenLocal - - name: Validate AAR structure + 16 KB page-size alignment + - name: Validate AAR structure shell: bash run: | set -euo pipefail @@ -1207,41 +1037,23 @@ jobs: for entry in $ENTRIES; do unzip -l "$AAR" | grep -q "${entry}$" || { echo "::error::$AAR is missing $entry"; exit 1; } done - unzip -p "$AAR" AndroidManifest.xml | grep -q 'android:minSdkVersion="28"' \ + unzip -p "$AAR" AndroidManifest.xml | grep -q 'android:minSdkVersion="28"' || { echo "::error::$AAR manifest lost minSdkVersion 28"; exit 1; } unzip -p "$AAR" classes.jar > /tmp/aar-classes.jar - unzip -l /tmp/aar-classes.jar | grep -q "net/ladenthin/llama/LlamaModel.class" \ + unzip -l /tmp/aar-classes.jar | grep -q "net/ladenthin/llama/LlamaModel.class" || { echo "::error::$AAR classes.jar lost LlamaModel"; exit 1; } if unzip -l /tmp/aar-classes.jar | grep -qE "module-info\.class|net/ladenthin/llama/(Linux|Mac|Windows)/"; then echo "::error::$AAR classes.jar carries module-info or desktop native resources"; exit 1 fi - rm -rf /tmp/aar-natives && unzip -o -q -d /tmp/aar-natives "$AAR" "jni/*/libjllama.so" - # Google Play 16 KB page-size requirement (Android 15+ targets): every - # LOAD segment must be aligned to a multiple of 16384. CMake pins - # -Wl,-z,max-page-size=16384 for every Android ABI; this asserts it held. - for so in /tmp/aar-natives/jni/*/libjllama.so; do - for align in $(readelf -lW "$so" | awk '$1=="LOAD"{print $NF}'); do - if [ $(( align % 16384 )) -ne 0 ]; then - echo "::error::$so: LOAD alignment $align is not a multiple of 16384 (16 KB page-size regression)"; exit 1 - fi - done - # dlopen-ability gate: an app consuming the AAR bundles no other native - # libs, so every DT_NEEDED must be a bionic system library (plus the - # vendor ICD libOpenCL.so for the OpenCL flavor). A stray dependency — - # libomp.so / libc++_shared.so once shipped exactly this way — makes - # System.loadLibrary fail on every device with UnsatisfiedLinkError. - # CMake's Android guard (GGML_OPENMP OFF + -static-libstdc++) keeps the - # list clean; this asserts it held. - ALLOWED="libc.so libm.so libdl.so liblog.so libandroid.so" - if [ "$flavor" = "llama-android-opencl" ]; then ALLOWED="$ALLOWED libOpenCL.so"; fi - for needed in $(readelf -dW "$so" | awk '/\(NEEDED\)/{gsub(/[\[\]]/,"",$NF); print $NF}'); do - case " $ALLOWED " in - *" $needed "*) ;; - *) echo "::error::$so: DT_NEEDED '$needed' is not a bionic system library — dlopen would fail on-device"; exit 1 ;; - esac - done - done + # The libraries themselves (bionic-only DT_NEEDED + libOpenCL.so for the OpenCL + # flavour, every LOAD segment 16 KB aligned for Google Play) are checked on the staged + # tree below, by the same allowlists the package job applies to every natives jar. + cmp -s <(unzip -p "$AAR" jni/arm64-v8a/libjllama.so) \ + "llama-android/natives/$([ "$flavor" = llama-android ] && echo cpu || echo opencl)/arm64-v8a/libjllama.so" \ + || { echo "::error::$AAR jni/arm64-v8a/libjllama.so is not the staged library"; exit 1; } done + - name: Android libraries -- dependency allowlist + 16 KB page alignment (buildcheck/nativedeps.py) + run: python3 .github/verify-native-deps.py stage - name: AGP consumer smoke test (R8 release build from mavenLocal) shell: bash run: | @@ -1251,11 +1063,11 @@ jobs: APK=.github/android-consumer-test/app/build/outputs/apk/release/app-release.apk test -f "$APK" for abi in arm64-v8a x86_64; do - unzip -l "$APK" | grep -q "lib/$abi/libjllama.so" \ + unzip -l "$APK" | grep -q "lib/$abi/libjllama.so" || { echo "::error::consumer APK is missing lib/$abi/libjllama.so"; exit 1; } done # The AAR's consumer proguard.txt must have carried the binding through R8. - unzip -p "$APK" "classes*.dex" | grep -aq "Lnet/ladenthin/llama/LlamaModel;" \ + unzip -p "$APK" "classes*.dex" | grep -aq "Lnet/ladenthin/llama/LlamaModel;" || { echo "::error::R8 stripped net.ladenthin.llama.LlamaModel — consumer proguard rules broken"; exit 1; } - name: Upload AARs uses: actions/upload-artifact@v7 @@ -1301,7 +1113,7 @@ jobs: - uses: gradle/actions/setup-gradle@v6 with: # AGP 9.4.0 (the .github/android-consumer-test fixture) requires Gradle >= 9.6.0. - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Enable KVM group permissions (GitHub-hosted runner) run: | echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules @@ -1328,16 +1140,7 @@ jobs: cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ gradle -p llama-android publishLlamaAndroidPublicationToMavenLocal - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true + - uses: ./.github/actions/restore-models - name: Free disk space for the emulator (only the draft model is needed on-device) # Same guard as test-android-llmservice: the AVD userdata partition needs ~7.4 GB; drop the # rest of the restored ~10 GB GGUF cache (this job adb-pushes only the tiny draft model) plus @@ -1397,7 +1200,7 @@ jobs: # AGP 9.4.0 (android-llmservice) requires Gradle >= 9.6.0; also satisfies Kotlin # 2.4's Gradle-plugin floor. The AAR/lib-only jobs stay on an older Gradle since # they build no AGP project. - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Build core jar + install llama-kotlin facade to mavenLocal # install (not package) so the app's Gradle build resolves llama-kotlin + # the core POM from mavenLocal. llama-kotlin's core dep is provided-scope, so the @@ -1491,7 +1294,7 @@ jobs: - uses: gradle/actions/setup-gradle@v6 with: # AGP 9.4.0 (android-llmservice) requires Gradle >= 9.6.0. - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Enable KVM group permissions (GitHub-hosted runner) run: | echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules @@ -1518,12 +1321,7 @@ jobs: cp stage/cpu/Linux-Android/aarch64/cpu/libjllama.so llama-android/natives/cpu/arm64-v8a/ cp stage/cpu-x86_64/Linux-Android/x86_64/cpu/libjllama.so llama-android/natives/cpu/x86_64/ gradle -p llama-android publishLlamaAndroidPublicationToMavenLocal - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true + - uses: ./.github/actions/restore-models - name: Free disk space for the emulator (only the draft model is needed on-device) # The AVD userdata partition needs ~7.4 GB; the full ~10 GB GGUF cache restore leaves too # little free, so the emulator FATALs ("Not enough space to create userdata partition") and @@ -1562,8 +1360,6 @@ jobs: runs-on: macos-15 env: BUILD_JOBS: 2 - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1578,12 +1374,7 @@ jobs: java-version: ${{ env.JAVA_VERSION }} - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true @@ -1607,8 +1398,6 @@ jobs: runs-on: macos-14 env: BUILD_JOBS: 2 - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1623,12 +1412,7 @@ jobs: java-version: ${{ env.JAVA_VERSION }} - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true @@ -1658,16 +1442,8 @@ jobs: name: webui-generated path: ${{ github.workspace }}/llama/webui-generated/ - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - name: Build libraries shell: cmd run: | @@ -1692,16 +1468,8 @@ jobs: name: webui-generated path: ${{ github.workspace }}/llama/webui-generated/ - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - name: Build libraries shell: cmd run: | @@ -1732,8 +1500,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1749,26 +1515,10 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - name: Build libraries shell: cmd run: | @@ -1786,8 +1536,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1803,26 +1551,10 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - name: Build libraries shell: cmd run: | @@ -1858,8 +1590,6 @@ jobs: # self-contained with no libomp dependency to ship. The x86_64/x86 jobs keep OpenMP (MSVC vcomp). runs-on: windows-11-arm env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1875,18 +1605,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - # sccache ships a native Windows-on-ARM build; it wraps clang-cl here. - # build.bat probes sccache before trusting it and, should a build fail with it as the - # launcher, retries once uncached -- so this can speed the job up but never red it. - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-aarch64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # No mvn compile needed: the JNI header (jllama.h) is committed and the native build @@ -1922,8 +1641,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -1977,15 +1694,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # GPU jobs build the artifact only — no -DBUILD_TESTING / ctest. The C++ unit @@ -2006,8 +1715,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2028,15 +1735,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # Build the artifact only (see the CUDA job's note: GPU-less runner can't run a @@ -2055,8 +1754,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2072,15 +1769,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # Build the artifact only (see the CUDA job's note: GPU-less runner can't run a @@ -2110,8 +1799,6 @@ jobs: needs: [startgate, build-webui] runs-on: ubuntu-latest env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - name: Free up disk space @@ -2184,8 +1871,6 @@ jobs: # reason (it drives ROCm's own clang and relies on the older MSVC STL). runs-on: windows-2022 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2228,18 +1913,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - # Wraps ROCm's clang (HIP is compiled as C++ on Windows, so the C/CXX launcher covers it). - # build.bat probes sccache before trusting it and, should a build fail with it as the - # launcher, retries once uncached -- so this can speed the job up but never red it. - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # Upstream's compiler wiring for TheRock (clang under lib\llvm\bin, not bin\); @@ -2265,8 +1939,6 @@ jobs: needs: [startgate, build-webui] runs-on: ubuntu-latest env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - name: Free up disk space @@ -2308,8 +1980,6 @@ jobs: needs: [startgate, build-webui] runs-on: ubuntu-latest env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - name: Free up disk space @@ -2351,8 +2021,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2376,18 +2044,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - # Wraps cl.exe (C) and icx (C++); if sccache cannot handle icx the retry builds uncached. - # build.bat probes sccache before trusting it and, should a build fail with it as the - # launcher, retries once uncached -- so this can speed the job up but never red it. - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd run: | @@ -2409,8 +2066,6 @@ jobs: # OpenCL headers + ICD loader before delegating to build.bat. runs-on: windows-11-arm env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2426,18 +2081,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - # sccache ships a native Windows-on-ARM build; it wraps clang-cl here. - # build.bat probes sccache before trusting it and, should a build fail with it as the - # launcher, retries once uncached -- so this can speed the job up but never red it. - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-aarch64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd run: | @@ -2454,8 +2098,6 @@ jobs: needs: [startgate, build-webui] runs-on: ubuntu-latest env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2505,8 +2147,6 @@ jobs: needs: [startgate, build-webui] runs-on: windows-2025-vs2026 env: - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2537,18 +2177,7 @@ jobs: - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true - shell: pwsh - # Plain cl.exe build, same as the Vulkan/OpenCL jobs. - # build.bat probes sccache before trusting it and, should a build fail with it as the - # launcher, retries once uncached -- so this can speed the job up but never red it. - run: | - $ver = "0.18.0" - $rel = "sccache-v$ver-x86_64-pc-windows-msvc" - $url = "https://github.com/mozilla/sccache/releases/download/v$ver/$rel.zip" - Write-Host "Downloading $url" - Invoke-WebRequest -Uri $url -OutFile "$env:RUNNER_TEMP\sccache.zip" - Expand-Archive -Path "$env:RUNNER_TEMP\sccache.zip" -DestinationPath "$env:RUNNER_TEMP\sccache" -Force - Add-Content -Path $env:GITHUB_PATH -Value "$env:RUNNER_TEMP\sccache\$rel" + uses: ./.github/actions/install-sccache-windows - name: Build libraries shell: cmd # vcpkg toolchain file wires in the OpenCL (incl. cl2.hpp) that ggml-openvino needs. @@ -2576,12 +2205,8 @@ jobs: distribution: 'temurin' java-version: ${{ env.JAVA_VERSION }} - name: Display CPU Info - run: | - echo "=== CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + shell: bash + run: bash .github/print-host-info.sh - name: Build libraries run: | mvn -q --no-transfer-progress -f llama/pom.xml compile @@ -2601,8 +2226,6 @@ jobs: runs-on: macos-15 env: BUILD_JOBS: 2 - USE_CACHE: ${{ github.event_name != 'workflow_dispatch' || inputs.use_cache }} - SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }} steps: - uses: actions/checkout@v7 @@ -2617,12 +2240,7 @@ jobs: java-version: ${{ env.JAVA_VERSION }} - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true @@ -2652,33 +2270,12 @@ jobs: - uses: actions/checkout@v7 - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - lscpu - echo "" - echo "=== CPU Details from /proc/cpuinfo ===" - cat /proc/cpuinfo + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share - # one entry (key gguf-models-). validate-models is kept as an integrity guard so a - # partial/absent restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: bash .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -2691,15 +2288,7 @@ jobs: echo "${{ github.workspace }}/core.%e.%p" | sudo tee /proc/sys/kernel/core_pattern - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml -P jcstress test \ - -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} \ - -Dnet.ladenthin.llama.nomic.path=models/${NOMIC_EMBED_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.model=models/${VISION_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.mmproj=models/${VISION_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.vision.image=${VISION_IMAGE_PATH} \ - -Dnet.ladenthin.llama.tts.model=models/${TTS_MODEL_NAME} \ - -Dnet.ladenthin.llama.tts.mmproj=models/${TTS_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.train.model=models/${TRAIN_MODEL_NAME} + mvn -e --no-transfer-progress -f llama/pom.xml -P jcstress test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -2732,40 +2321,11 @@ jobs: - name: Memory after tests if: always() run: free -h - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -2817,33 +2377,12 @@ jobs: - uses: actions/checkout@v7 - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: macos-14-metal path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share - # one entry (key gguf-models-). validate-models is kept as an integrity guard so a - # partial/absent restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: bash .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -2854,15 +2393,7 @@ jobs: run: ulimit -c unlimited - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml -Dnet.ladenthin.llama.test.ngl=0 test \ - -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} \ - -Dnet.ladenthin.llama.nomic.path=models/${NOMIC_EMBED_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.model=models/${VISION_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.mmproj=models/${VISION_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.vision.image=${VISION_IMAGE_PATH} \ - -Dnet.ladenthin.llama.tts.model=models/${TTS_MODEL_NAME} \ - -Dnet.ladenthin.llama.tts.mmproj=models/${TTS_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.train.model=models/${TRAIN_MODEL_NAME} + mvn -e --no-transfer-progress -f llama/pom.xml -Dnet.ladenthin.llama.test.ngl=0 test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -2873,40 +2404,11 @@ jobs: - name: Memory after tests if: always() run: vm_stat && sysctl hw.memsize hw.physmem - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -2928,33 +2430,12 @@ jobs: - uses: actions/checkout@v7 - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: macos-15-no-metal path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share - # one entry (key gguf-models-). validate-models is kept as an integrity guard so a - # partial/absent restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: bash .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -2965,15 +2446,7 @@ jobs: run: ulimit -c unlimited - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml test \ - -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} \ - -Dnet.ladenthin.llama.nomic.path=models/${NOMIC_EMBED_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.model=models/${VISION_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.mmproj=models/${VISION_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.vision.image=${VISION_IMAGE_PATH} \ - -Dnet.ladenthin.llama.tts.model=models/${TTS_MODEL_NAME} \ - -Dnet.ladenthin.llama.tts.mmproj=models/${TTS_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.train.model=models/${TRAIN_MODEL_NAME} + mvn -e --no-transfer-progress -f llama/pom.xml test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -2984,40 +2457,11 @@ jobs: - name: Memory after tests if: always() run: vm_stat && sysctl hw.memsize hw.physmem - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -3039,33 +2483,12 @@ jobs: - uses: actions/checkout@v7 - name: Display CPU Info shell: bash - run: | - echo "=== CPU Information ===" - sysctl hw.model hw.cachelinesize hw.cpufrequency hw.cachesize hw.physicalcpu hw.logicalcpu hw.packages hw.memsize hw.ncpu 2>/dev/null || true - echo "" - echo "=== Processor Details ===" - system_profiler SPHardwareDataType + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: natives-metal-macos-aarch64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. GGUF is platform-independent, so ubuntu + macOS + Windows share - # one entry (key gguf-models-). validate-models is kept as an integrity guard so a - # partial/absent restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: bash .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3076,15 +2499,7 @@ jobs: run: ulimit -c unlimited - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml test \ - -Dnet.ladenthin.llama.tool.model=models/${TOOL_MODEL_NAME} \ - -Dnet.ladenthin.llama.nomic.path=models/${NOMIC_EMBED_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.model=models/${VISION_MODEL_NAME} \ - -Dnet.ladenthin.llama.vision.mmproj=models/${VISION_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.vision.image=${VISION_IMAGE_PATH} \ - -Dnet.ladenthin.llama.tts.model=models/${TTS_MODEL_NAME} \ - -Dnet.ladenthin.llama.tts.mmproj=models/${TTS_MMPROJ_NAME} \ - -Dnet.ladenthin.llama.train.model=models/${TRAIN_MODEL_NAME} + mvn -e --no-transfer-progress -f llama/pom.xml test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -3095,40 +2510,11 @@ jobs: - name: Memory after tests if: always() run: vm_stat && sysctl hw.memsize hw.physmem - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -3149,36 +2535,13 @@ jobs: steps: - uses: actions/checkout@v7 - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: natives-cpu-windows-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. validate-models is kept as an integrity guard so a partial/absent - # restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github\validate-models.bat + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3205,15 +2568,7 @@ jobs: Get-ItemProperty -Path $key | Format-List - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml test ` - "-Dnet.ladenthin.llama.tool.model=models/$env:TOOL_MODEL_NAME" ` - "-Dnet.ladenthin.llama.nomic.path=models/$env:NOMIC_EMBED_MODEL_NAME" ` - "-Dnet.ladenthin.llama.vision.model=models/$env:VISION_MODEL_NAME" ` - "-Dnet.ladenthin.llama.vision.mmproj=models/$env:VISION_MMPROJ_NAME" ` - "-Dnet.ladenthin.llama.vision.image=$env:VISION_IMAGE_PATH" ` - "-Dnet.ladenthin.llama.tts.model=models/$env:TTS_MODEL_NAME" ` - "-Dnet.ladenthin.llama.tts.mmproj=models/$env:TTS_MMPROJ_NAME" ` - "-Dnet.ladenthin.llama.train.model=models/$env:TRAIN_MODEL_NAME" + mvn -e --no-transfer-progress -f llama/pom.xml test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -3226,40 +2581,11 @@ jobs: if: always() run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List shell: pwsh - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -3286,36 +2612,13 @@ jobs: steps: - uses: actions/checkout@v7 - name: Display CPU Info - shell: pwsh - run: | - Write-Host "=== CPU Information (Get-CimInstance - All Properties) ===" - Get-CimInstance Win32_Processor | Select-Object * | Format-List - Write-Host "" - Write-Host "=== CPU Information (systeminfo) ===" - systeminfo | Select-String "Processor" - Write-Host "" - Write-Host "=== CPU Information (Get-ComputerInfo) ===" - Get-ComputerInfo -Property "CsProcessors*" 2>$null || Write-Host "Get-ComputerInfo not available" + shell: bash + run: bash .github/print-host-info.sh - uses: actions/download-artifact@v8 with: name: natives-msvc-windows-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - # GGUF models are downloaded + cached ONCE by the upstream `download-models` job - # (this job `needs:` it), so here we only RESTORE the shared cache — no per-job - # download logic. validate-models is kept as an integrity guard so a partial/absent - # restore fails loudly instead of silently self-skipping required tests. - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github\validate-models.bat + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3342,15 +2645,7 @@ jobs: Get-ItemProperty -Path $key | Format-List - name: Run tests run: | - mvn -e --no-transfer-progress -f llama/pom.xml test ` - "-Dnet.ladenthin.llama.tool.model=models/$env:TOOL_MODEL_NAME" ` - "-Dnet.ladenthin.llama.nomic.path=models/$env:NOMIC_EMBED_MODEL_NAME" ` - "-Dnet.ladenthin.llama.vision.model=models/$env:VISION_MODEL_NAME" ` - "-Dnet.ladenthin.llama.vision.mmproj=models/$env:VISION_MMPROJ_NAME" ` - "-Dnet.ladenthin.llama.vision.image=$env:VISION_IMAGE_PATH" ` - "-Dnet.ladenthin.llama.tts.model=models/$env:TTS_MODEL_NAME" ` - "-Dnet.ladenthin.llama.tts.mmproj=models/$env:TTS_MMPROJ_NAME" ` - "-Dnet.ladenthin.llama.train.model=models/$env:TRAIN_MODEL_NAME" + mvn -e --no-transfer-progress -f llama/pom.xml test # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that # fails makes Surefire record tests="0" for that class -- no entries at all, so it is # invisible to a skip check. That is exactly how every model-gated class stayed silently @@ -3363,40 +2658,11 @@ jobs: if: always() run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List shell: pwsh - # A forked test JVM that aborts leaves an hs_err_pid log and a surefire - # dumpstream -- both otherwise ONLY inside the artifact uploaded below, - # which is unreachable from anywhere that cannot fetch from Azure Blob - # (a phone, a restricted network, an agent sandbox). Echo them here so the - # aborting frame is readable from the run page itself. See - # ../workspace/policies/ci-test-diagnostics.md section 3.1. + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - name: Print crash logs (on failure) if: failure() shell: bash - run: | - shopt -s nullglob - found=0 - for f in llama/hs_err_pid*.log; do - found=1 - echo "===== $f (first 200 lines; full file in the uploaded artifact) =====" - sed -n '1,200p' "$f" - done - for f in llama/target/surefire-reports/*.dumpstream llama/target/surefire-reports/*.dump; do - found=1 - echo "===== $f =====" - cat "$f" - done - if [ "$found" = 0 ]; then - echo "No hs_err_pid*.log and no surefire dump/dumpstream was written." - echo - echo "For an ordinary test failure that is EXPECTED, not a finding: this step runs on" - echo "any job failure, and an assertion failure, a timeout or a compile error writes no" - echo "crash log. Read the surefire output above for the real cause." - echo - echo "It points at a JVM-level abort only if the log ALSO shows a fork ending abnormally" - echo "-- 'The forked VM terminated without properly saying goodbye', or an exit with no" - echo "test results. In that case the abort bypassed the JVM error handler (a native" - echo "exit()/terminate() rather than a raised signal), which is why no file was written." - fi + run: bash .github/print-crash-logs.sh llama - if: failure() uses: actions/upload-artifact@v7 with: @@ -3600,18 +2866,7 @@ jobs: with: name: llama-fatjar-smoke-linux path: fatjars/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3662,14 +2917,7 @@ jobs: with: name: llama-fatjar-smoke-linux path: agent-assets/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3705,18 +2953,7 @@ jobs: with: name: llama-fatjar-smoke-windows path: fatjars/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so Windows/macOS restore the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github\validate-models.bat + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3751,18 +2988,7 @@ jobs: with: name: llama-fatjar-smoke-linux-aarch64 path: fatjars/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: consumer jobs can NEVER write the cache, so a job running on a - # cache miss cannot re-save an empty/partial entry under the immutable key. - # download-models is the single writer; enableCrossOsArchive matches its - # cross-OS entry version so every consumer restores the same ubuntu-built entry. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github/validate-models.sh + - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' @@ -3794,15 +3020,7 @@ jobs: with: name: llama-fatjar-smoke-windows-arm64 path: fatjars/ - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - # Restore-only: see the note on the Linux job above. - uses: actions/cache/restore@v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Validate model files - run: .github\validate-models.bat + - uses: ./.github/actions/restore-models # temurin publishes a Windows/AArch64 JDK for this java-version; the build-windows-arm64 # job already resolves it on this same runner. - uses: actions/setup-java@v6 @@ -3910,7 +3128,7 @@ jobs: publish-snapshot: name: Publish Snapshot to Central - needs: [check-snapshot, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux] + needs: [check-snapshot, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] if: needs.check-snapshot.result == 'success' && inputs.publish_to_central runs-on: ubuntu-latest environment: maven-central @@ -3976,7 +3194,7 @@ jobs: # downloads above; the core jar was just built by the reactor deploy. - uses: gradle/actions/setup-gradle@v6 with: - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Publish Android AAR snapshots (llama-android + llama-android-opencl) run: | mkdir -p llama-android/natives/cpu/arm64-v8a llama-android/natives/cpu/x86_64 llama-android/natives/opencl/arm64-v8a @@ -4088,7 +3306,7 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - gh release view snapshot --repo ${{ github.repository }} 2>/dev/null \ + gh release view snapshot --repo ${{ github.repository }} 2>/dev/null || gh release create snapshot \ --repo ${{ github.repository }} \ --prerelease \ @@ -4107,7 +3325,7 @@ jobs: publish-release: name: Publish Release to Central if: needs.check-tag.result == 'success' && inputs.publish_to_central - needs: [check-tag, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux] + needs: [check-tag, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] runs-on: ubuntu-latest environment: maven-central permissions: @@ -4164,7 +3382,7 @@ jobs: # built by the reactor deploy. - uses: gradle/actions/setup-gradle@v6 with: - gradle-version: "9.8.0" + gradle-version: ${{ env.GRADLE_VERSION }} - name: Publish Android AAR release bundle to Central Portal shell: bash run: | diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 892520ca9..9f0b7ebbe 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -12,8 +12,7 @@ on: branches: [ main ] workflow_dispatch: -permissions: - contents: read +permissions: read-all jobs: analysis: diff --git a/.gitignore b/.gitignore index 840edb395..eae320eb5 100644 --- a/.gitignore +++ b/.gitignore @@ -83,3 +83,6 @@ android-llmservice/.gradle/ android-llmservice/local.properties *.jks *.keystore + +# Python bytecode of the .github/buildcheck checks +__pycache__/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 826ab3926..dc3023517 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,25 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] +### Changed +- **The model-backed tests default to CI's model set** (`.github/models.csv`): the vision, TTS, nomic + and trainer tests read `models/` unless a `-Dnet.ladenthin.llama.*` property names another file, + so a model downloaded into `models/` needs no property (new: `net.ladenthin.llama.train.model` + defaults to `models/stories260K.gguf`). `TestConstantsTest` asserts the defaults and the list are the + same set. +- **Build checks are a tested Python library** (`.github/buildcheck/`) and check more: that `package` + waits for every natives build, that the all-backends fat jars derived from `natives.csv` are each + smoke-launched, named in the agent jar's `Class-Path` and the README (`package-fatjars.sh` now asks + for them instead of hard-coding four), CMake's backend names, the dependency allowlists, and that + every job gates both publish jobs unless `.github/release-gate-exemptions.txt` says why (`vmlens` + gated nothing; it now gates). The Android AAR libraries are held to the same allowlist and 16 KB + alignment check as every natives jar. +- **Files shared with the sibling repositories are checksummed** in `.github/shared-files.sha256` and + checked by a `shared-files` job (crash-log printing, the signing-key preflight and the build-check + library are now such shared scripts). The workflow lost ~760 lines to two composite actions + (`restore-models`, `install-sccache-windows`) and `print-host-info.sh`; `validate-models.bat` is gone + (Windows runs the bash script). + ### Fixed - **`ToolCallingIntegrationTest#requiredToolCallIsParsedFromStreamingResponse` failed on both Windows x86-64 jobs after the b11211 bump** (the Ubuntu run and the blocking twin stayed green). The streamed diff --git a/CLAUDE.md b/CLAUDE.md index 3d67a8fd1..4e6ee2f85 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,17 +37,21 @@ it or is checked against it: | `llama/CMakeLists.txt` | names the backend directory (`JLLAMA_BACKEND`: GGML flag → name; `msvc` = Visual Studio generator; `metal` = Apple + `GGML_METAL`) and writes to `src/main/natives/net/ladenthin/llama////` | | `llama/pom.xml` `natives` profile | one `maven-jar-plugin` execution per row, **generated**: `python3 .github/check-natives.py pom` prints them | | `llama-platform/pom.xml` | depends on the rows with `platform=yes` | -| `publish.yml` build jobs | each uploads its tree as artifact `natives-` | +| `publish.yml` build jobs | each uploads its tree as artifact `natives-`; `package` waits for every one of them (transitively) | | `LlamaLoader.BACKEND_PRIORITY` | must contain every backend directory | -| `.github/check-natives.py` | runs in `code-style` (first minutes of every run); fails when the four rows above disagree with the list | +| the all-backends fat jars | **derived**: one `all--` per OS/arch with more than one natives jar (no `msvc`, no Android) — `check-natives.py fatjar-targets` prints them; each must be uploaded for and launched by a smoke job, named in the agent jar's `Class-Path` and in the README | +| `.github/check-natives.py` | runs in `code-style` (first minutes of every run); fails when any row of this table disagrees with the list — including CMake's backend names, the dependency allowlists and the README rows | | `.github/merge-native-artifacts.sh` | reads the list: every listed `natives-*` artifact present, no other, each holding its library and nothing outside its directory, no path claimed twice; writes `jllama-extras.txt` (sibling files loaded before the library, e.g. OpenVINO's `OpenCL.dll` on Windows) | -| `.github/package-fatjars.sh` | reads the list: the built natives jars match it, each holds only its directory and the right `Automatic-Module-Name`; merges one `all--` fat jar per OS/arch with a GPU backend (no `msvc`, no Android) | -| `.github/verify-native-deps.py` | exact dependency allowlist per CPU directory (`cpu`/`metal`/`msvc`), denylist for GPU ones | +| `.github/package-fatjars.sh` | reads the list: the built natives jars match it, each holds only its directory and the right `Automatic-Module-Name`; merges the all-backends fat jars and fails unless it produced exactly the targets `check-natives.py fatjar-targets` derives | +| `.github/verify-native-deps.py` | exact dependency allowlist per CPU directory (`cpu`/`metal`/`msvc`) and for the Android OpenCL build (bionic + `libOpenCL.so`), denylist for the other GPU ones; every Android library 16 KB page-aligned (Google Play). Runs in `package` and on the staged AAR libraries | | `.github/smoke-natives-jars.sh` (`package` job) | loads the real jars: classes + all 26 natives jars at once, on the classpath **and** the module path (on the GPU-less runner normally ending at `cpu`) | **Adding a natives jar:** a row in `natives.csv`, the execution `check-natives.py pom` prints, a build job uploading `natives-`, the backend name in CMake and `BACKEND_PRIORITY` if new, a -README row. `check-natives.py` fails until they agree; nothing else needs touching. +README row. `check-natives.py` fails until they agree; nothing else needs touching — the fat-jar +targets follow by themselves (a new OS/arch with a GPU backend also needs its smoke job, which +`check-natives.py` then demands). The checks are Python in `.github/buildcheck/` with unit tests +(see "Build checks, shared files and the release gate"). **Why the build jobs are not spawned from the list as one matrix** (considered and rejected): the 26 builds use genuinely different toolchains — dockcross images, the CUDA redist archives, ROCm pip @@ -71,6 +75,60 @@ to the test classpath (`additionalClasspathElements`), so `mvn test` finds the l is CI-only (a local tree holds one directory); a missing directory there is caught by the merge step, not by Maven. +## Build checks, shared files and the release gate (`.github/buildcheck/`) + +The pipeline's static checks are a small **stdlib-only Python package**, `.github/buildcheck/`, with +unit tests in `.github/buildcheck/tests/` (`python3 -m unittest discover -s .github/buildcheck/tests +-t .github`, from the repo root). The `.github/*.py` scripts are thin command-line wrappers around it, +so every check can be driven with literal inputs in a test — including the ELF/PE/Mach-O parsers +(synthesized binaries) and the workflow parser. It deliberately does not use PyYAML: the runners do +not guarantee it, and the checks only need the job graph, which `workflow.py` reads from the two +shapes this repo writes and rejects anything else loudly. + +| Module | CLI | What it checks | +|---|---|---| +| `natives.py` | `check-natives.py` (`code-style`) | everything that names a natives jar agrees with `natives.csv` (see "Natives jars") | +| `models.py` | `check-natives.py` | every `*_MODEL_NAME` of publish.yml's `env:` is a filename of `models.csv` | +| `nativedeps.py` | `verify-native-deps.py` (`package`, `package-android-aar`) | dependency allowlists, 16 KB Android alignment | +| `hipoffload.py` | `verify-hip-offload-compressed.py` (the two ROCm jobs) | no uncompressed GPU code bundle | +| `workflow.py` | — | the job graph of a workflow (`needs`, uploads, transitive closure) | +| `releasegate.py` | `check-release-gate.py` (`shared-files`) | every job gates **both** publish jobs, unless `.github/release-gate-exemptions.txt` names it with a reason | +| `sharedfiles.py` | `check-shared-files.py` (`shared-files`) | the files kept byte-identical across the four sibling repos | + +**The release gate.** A job nothing waits for can go red while a release still ships — the natives +builds `package` once forgot to wait for, and the aarch64 fat jars that were signed and attached +without any job launching them, were both that shape. So "not gating" is a written decision: a line +`: ` in `.github/release-gate-exemptions.txt`. The check fails both ways — a job neither +gating nor listed, and a listed job that gates after all (a stale exemption hides the next job of that +name). Introducing it found `vmlens` gating nothing in all four repos; it now gates both publish jobs. + +**Shared files (all four repos).** Files kept byte-identical in java-llama.cpp, BitcoinAddressFinder, +srcmorph and streambuffer are listed with their SHA-256 in each repo's **`.github/shared-files.sha256`** +(`sha256sum` format, so `sha256sum -c` reads it too). The `shared-files` job — itself identical in all +four `publish.yml` files, and gating both publish jobs — **fails** when a listed file changed here +alone, and **warns** when another repo's default branch lists the same file with a different hash (a +sync is one change per repo and lands in four steps, so a warning, not a failure). **The manifest is +the reference for what must stay equal**: changing a shared file means changing every copy, then +`python3 .github/check-shared-files.py --write` in each repo. Forgetting one is not a disaster — the +job says so, and the history shows it — but nobody can edit a shared script without learning that it +is shared. Duplicating the library across repos is therefore deliberate. A separate actions/library +repo was considered and rejected: every consumer would need a pinned SHA per use (Scorecard's +pinned-dependencies rule), a release process of its own, and a cross-repo checkout; copies with a +checksum keep each repo self-contained. Shared today: the `buildcheck` modules above marked shared in +`__init__.py` (+ their tests and CLIs), `print-crash-logs.sh`, `verify-signing-key.sh`, +`verify-bytecode-version.sh`, `sign-fatjars.sh` (jllama + srcmorph), `smoke-fatjar-cli.sh` (BAF + +srcmorph), the signing self-test, `lombok.config` and the issue/PR templates. + +**Composite actions (`.github/actions/`, this repo only).** `restore-models` (restore the shared GGUF +cache + `validate-models.sh`; the one place a job gets its models — 16 call sites) and +`install-sccache-windows` (10 call sites; the caller keeps `if:` and `continue-on-error`). Composite, +not reusable workflows: they replace steps without changing any job or check name. + +**Considered and not done** (measured after the above): a reusable workflow for the Java test jobs +(the three macOS jobs are 53 lines each with 7 differing, the two Windows jobs 77) and a matrix for +the four fat-jar smokes (32–47 lines each). Both would save about a hundred lines and rename every +affected check, which means updating branch protection — not worth it at this size. + ## Upgrading CUDA Version Current CUDA version: **13.4** (Linux `cuda-toolkit-13-4` from NVIDIA's rhel8 repo; Windows 13.4 redist archives) @@ -273,7 +331,9 @@ FindVulkan-compatible). Because all five Windows build jobs are in the `package` GPU-toolchain failure blocks packaging — the same release-gating policy every build job follows. **sccache on every Windows Ninja job.** All ten Ninja build jobs install sccache (x86_64 or the -native `aarch64` release) with the same `USE_CACHE` / `SCCACHE_WEBDAV_*` env; only the two MSVC-classifier +native `aarch64` release) through the composite action `.github/actions/install-sccache-windows` +(arch from `RUNNER_ARCH`; its version must equal `SCCACHE_DL_VERSION` in `build.sh`) with the same +`USE_CACHE` / `SCCACHE_WEBDAV_*` env; only the two MSVC-classifier jobs cannot, because the Visual Studio generator ignores compiler launchers. It cannot red a build, by three guards in `build.bat`: the install step is `continue-on-error`; a probe compiles through sccache before it is trusted; and — because that probe only proves `cl.exe`, while arm64 builds with `clang-cl`, @@ -690,7 +750,10 @@ v0.16.0 + the probe this is no longer a risk.) Job-by-job status: OpenCL headers/loader, then delegates the jllama cmake build to `build.sh` via `exec` (same pattern as `build_cuda_linux.sh`), so it inherits the probe and launcher automatically. -Per-job recipe: add `env:` { `USE_CACHE`, `SCCACHE_WEBDAV_ENDPOINT`, `SCCACHE_WEBDAV_TOKEN` } and +Per-job recipe: `USE_CACHE` and `SCCACHE_WEBDAV_ENDPOINT` are workflow-level `env:` (every job has +them); a cached job adds only `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` to its own `env:` — +kept per job on purpose, since at workflow level the token would reach every job, including those +running third-party actions — and, for dockcross, `DOCKCROSS_ARGS: "-e SCCACHE_WEBDAV_ENDPOINT -e SCCACHE_WEBDAV_TOKEN -e USE_CACHE"` — the dockcross wrapper only forwards host env it is explicitly told to via `-e`. The fetched sccache version is the `SCCACHE_DL_VERSION` knob in `build.sh` (default **0.18.0**; overridable per-job @@ -943,10 +1006,11 @@ the *build host* has `libibverbs` (Linux) or `librdma` (Apple), which would make `libibverbs.so.1` at load time and fail on every machine without rdma-core; it cannot be linked statically in a useful way either (it `dlopen`s its hardware providers), and upstream's own CMake comment says the Apple weak link does not survive a static `ggml-rpc`. **`.github/verify-native-deps.py`** -(the `package` job) reads each shipped library's dependency list straight from the file — ELF +(the `package` job; the code and its tests are `.github/buildcheck/nativedeps.py`) reads each shipped library's dependency list straight from the file — ELF `DT_NEEDED`, PE import table including Windows arm64 (which binutils cannot read), Mach-O load -commands — and holds every CPU directory (`cpu`, `metal`, `msvc`) to an exact per-directory -allowlist (the GPU directories only to a denylist, since they need their vendor runtime). A new CPU +commands — and holds every CPU directory (`cpu`, `metal`, `msvc`) and the Android OpenCL build to an +exact per-directory allowlist (the other GPU directories only to a denylist, since they need their +vendor runtime). A new CPU directory without an allowlist fails too. (That every build arrived is the merge step's check against `natives.csv`.) **It found a pre-existing defect on its first run**: the macOS dylib links Homebrew's `openssl@3` (`/opt/homebrew/opt/openssl@3/lib/libssl.3.dylib` + `libcrypto`), so it does not load on a @@ -1307,50 +1371,41 @@ mvn test -Dtest=MemoryManagementTest mvn test -Dtest=LlamaModelTest#testGenerateAnswer ``` -**Optional models** referenced by individual tests are gated on a system -property so CI can skip them cleanly when the GGUF is not downloaded. -The full property → consumer → default table for every `net.ladenthin.llama.*` -property the library understands (runtime + test) is the user-facing -**[System Properties Reference](README.md#system-properties-reference)** in -the README. The summary below covers only the optional-model bindings: +**Optional models.** Every model a test uses defaults to `models/` from `.github/models.csv` +(`TestConstants.DEFAULT_*`), so downloading a model from that list into `models/` is all it takes — +the tests find it (module-relative or at the reactor root) and self-skip while it is missing. The +properties below only **override** a default. The full property → consumer → default table for every +`net.ladenthin.llama.*` property the library understands (runtime + test) is the user-facing +**[System Properties Reference](README.md#system-properties-reference)** in the README. The summary +below covers the model bindings: -| Property | Default test that uses it | Model | +| Property | Default test that uses it | Default (`models/…` unless noted) | |----------|---------------------------|-------| +| `net.ladenthin.llama.tool.model` | `ToolCallingIntegrationTest`, `OpenAiServerToolCallingIntegrationTest` | `Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` | | `net.ladenthin.llama.nomic.path` | `LlamaEmbeddingsTest#testNomicEmbedLoads` | `nomic-embed-text-v1.5.f16.gguf` (issue #98 regression) | | `net.ladenthin.llama.vision.model` | `MultimodalIntegrationTest` | `SmolVLM-500M-Instruct-Q8_0.gguf` (any vision-capable GGUF works) | -| `net.ladenthin.llama.vision.mmproj` | `MultimodalIntegrationTest` | matching mmproj for the vision model, e.g. `mmproj-SmolVLM-500M-Instruct-Q8_0.gguf` | -| `net.ladenthin.llama.vision.image` | `MultimodalIntegrationTest` | committed default `src/test/resources/images/test-image.jpg`; override to any png/jpeg/webp/gif on disk | -| `net.ladenthin.llama.audio.model` | `AudioInputIntegrationTest` (llama.cpp discussion #13759) | audio-input model GGUF, e.g. `ultravox-v0_5-llama-3_2-1b.gguf` | -| `net.ladenthin.llama.audio.mmproj` | `AudioInputIntegrationTest` | matching audio mmproj/encoder, e.g. `mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf` | -| `net.ladenthin.llama.audio.input` | `AudioInputIntegrationTest` | committed default `src/test/resources/audios/sample.wav`; override to any `.wav`/`.mp3` on disk | -| `net.ladenthin.llama.tts.model` | `TtsIntegrationTest` | Qwen3-TTS backbone GGUF (any Qwen3-TTS-family model works) | -| `net.ladenthin.llama.tts.mmproj` | `TtsIntegrationTest` | matching mmproj GGUF (speaker encoder + code predictor + code2wav) | - -Run those tests by setting the property: +| `net.ladenthin.llama.vision.mmproj` | `MultimodalIntegrationTest` | `mmproj-SmolVLM-500M-Instruct-Q8_0.gguf` | +| `net.ladenthin.llama.vision.image` | `MultimodalIntegrationTest` | committed `src/test/resources/images/test-image.jpg`; any png/jpeg/webp/gif | +| `net.ladenthin.llama.tts.model` | `TtsIntegrationTest` | `Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf` (any Qwen3-TTS-family model works) | +| `net.ladenthin.llama.tts.mmproj` | `TtsIntegrationTest` | `mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf` | +| `net.ladenthin.llama.train.model` | `LlamaTrainerIntegrationTest` | `stories260K.gguf` (must be **F32**) | +| `net.ladenthin.llama.audio.model` | `AudioInputIntegrationTest` (llama.cpp discussion #13759) | none (not in CI's set) — e.g. `ultravox-v0_5-llama-3_2-1b.gguf` | +| `net.ladenthin.llama.audio.mmproj` | `AudioInputIntegrationTest` | none — e.g. `mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf` | +| `net.ladenthin.llama.audio.input` | `AudioInputIntegrationTest` | committed `src/test/resources/audios/sample.wav`; any `.wav`/`.mp3` | + ```bash -mvn test -Dtest=LlamaEmbeddingsTest#testNomicEmbedLoads \ - -Dnet.ladenthin.llama.nomic.path=models/nomic-embed-text-v1.5.f16.gguf -mvn test -Dtest=MultimodalIntegrationTest \ - -Dnet.ladenthin.llama.vision.model=models/SmolVLM-500M-Instruct-Q8_0.gguf \ - -Dnet.ladenthin.llama.vision.mmproj=models/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf -# The vision.image property defaults to src/test/resources/images/test-image.jpg -# (a CC-BY-4.0 / MIT-granted photo of flowers and bees by the project author); -# override only if you want to test a different image. - -# Audio input (Ultravox / Qwen2.5-Omni; the audio clip has no committed default): +# CI's models, e.g. the vision pair (URLs in .github/models.csv) -- no property needed afterwards: +mvn test -Dtest=MultimodalIntegrationTest +# Another model than the default: +mvn test -Dtest=TtsIntegrationTest \ + -Dnet.ladenthin.llama.tts.model=/path/to/qwen3-tts-backbone.gguf \ + -Dnet.ladenthin.llama.tts.mmproj=/path/to/qwen3-tts-mmproj.gguf +# Audio input has no CI model, so it always needs the properties: mvn test -Dtest=AudioInputIntegrationTest \ -Dnet.ladenthin.llama.audio.model=models/ultravox-v0_5-llama-3_2-1b.gguf \ - -Dnet.ladenthin.llama.audio.mmproj=models/mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf \ - -Dnet.ladenthin.llama.audio.input=/path/to/speech.wav # optional: defaults to the committed src/test/resources/audios/sample.wav -mvn test -Dtest=TtsIntegrationTest \ - -Dnet.ladenthin.llama.tts.model=models/qwen3-tts-backbone.gguf \ - -Dnet.ladenthin.llama.tts.mmproj=models/qwen3-tts-mmproj.gguf + -Dnet.ladenthin.llama.audio.mmproj=models/mmproj-ultravox-v0_5-llama-3_2-1b-f16.gguf ``` -`MultimodalIntegrationTest` self-skips when any of the three vision properties -points at a missing path, so a partial setup (just the vision model + the -committed image, no mmproj) lets the test class load without erroring. - **Restricted-network environments.** Some hosts (e.g. ephemeral remote execution sandboxes) block outbound traffic to `huggingface.co`. In that case downloading models for the Java tests is not possible from the host @@ -1740,18 +1795,18 @@ the wiring (a future edit that drops the wrapper from a constant fails that test silently re-muting the suite). `llama-langchain4j` had the identical defect and carries the same resolver as `TestModelPaths` (test classes are not shared between modules). -`validate-models.{sh,bat}` -treats all of these as **required** (a missing model hard-fails the job before tests run, so a -download regression can never silently downgrade to a skip). **Two** classes still self-skip on -every platform, both because their model is outside the manifest: `AudioInputIntegrationTest` — the -prompt clip is committed (`src/test/resources/audios/sample.wav`) but the audio model + mmproj have -no CI download — and `LlamaTrainerIntegrationTest`, whose `net.ladenthin.llama.train.model` property -is set by no job and whose model is in no `models.csv` row. The trainer one matters more than it -looks: `train_engine.cpp` carries the same `postprocess_cpu_params` pair as `tts_params.hpp`, so the -JVM-abort class of bug documented under "Qwen3-TTS" can regress there with no runnable guard. Two -slices of it are now covered without a model — `test_tts_params.cpp` drives `build_train_params` and -`jllama::resolve_cpu_params`, and `test_wire_contracts.cpp` pins the configuration key set against -`TrainingField` — but the Java → JNI → native round trip itself still runs nowhere. +`validate-models.sh` treats all of these as **required** (a missing model hard-fails the job before +tests run, so a download regression can never silently downgrade to a skip). **The tests default to +exactly the CI model set**: every model-gated test reads its path through a `TestConstants` +`DEFAULT_*` constant (`models/`), a `-Dnet.ladenthin.llama.*` property only overrides it, and +`TestConstantsTest.theModelDefaultsAreExactlyTheCiModelSet` asserts that the `.gguf` constants and the +rows of `models.csv` are the same set in both directions (a row no constant names is downloaded for +nothing; a constant naming a model the list lacks self-skips in CI). So the test jobs pass **no** model +properties. That also closed a gap this file used to describe: `LlamaTrainerIntegrationTest` defaults +to `stories260K.gguf` and runs on every Java test job, so the Java → JNI → native trainer round trip +has a runnable guard. **One** class still self-skips everywhere: `AudioInputIntegrationTest` — its +prompt clip is committed (`src/test/resources/audios/sample.wav`), but the audio model + mmproj have no +CI download. The model set has a **single source of truth: `.github/models.csv`** (one `filename,url` row per model; `#` comments). Everything derives from it: the **`download-models`** job (ubuntu, `needs: startgate`) is the only place models are fetched from HuggingFace (one manifest-driven @@ -1759,24 +1814,25 @@ model; `#` comments). Everything derives from it: the **`download-models`** job GGUF cache (path `models/`, key **`gguf-models-`** — so *editing the manifest automatically creates a fresh complete cache entry*; no manual cache deletion on a model-set change). The writer sets **`enableCrossOsArchive: true`**, making the one ubuntu-built entry the -same entry macOS and Windows restore. A **`verify-model-cache` matrix job** (ubuntu / macOS / -Windows, `needs: download-models`) then proves the entry is restorable -(`actions/cache/restore` + `fail-on-cache-miss: true`) **and complete** -(`validate-models.{sh,bat}`, which read their required list from the same manifest) on every OS -**before any model-consuming job starts** — all `test-java-*` jobs, the langchain4j integration -job, the Android emulator job and the fat-jar smoke jobs `need: verify-model-cache` and use the -**restore-only** action themselves (no per-job download, no save — a consumer can never re-save -an empty/partial entry), keeping validate as a per-job integrity guard. (This design hardened -after run 28805360584: without the cross-OS flag, cache entries are versioned per-OS and the -unreachable Windows-side entry had been re-saved **empty** (343 B) after an eviction; and -`validate-models.bat`'s quoted `MODELS` list broke cmd's for-tokenization so its `exit /b 1` -never fired — the empty cache sailed through the "gate" and the Windows jobs silently -self-skipped every model-backed test until the fat-jar smoke's hard check caught it.) The -`*_MODEL_NAME` workflow env vars remain consumer-side wiring for the `-Dnet.ladenthin.llama.*` -test properties and must match the manifest's filename column — locally the model tests still -self-skip when a GGUF is absent (`Assume.assumeTrue`), so a partial local checkout is fine. - -Set the model path via system property or environment variable (see test files for exact property names). +same entry macOS and Windows restore. Every consumer uses the composite action +**`.github/actions/restore-models`** — restore-only (`actions/cache/restore`; a consumer can never +re-save an empty/partial entry) plus `validate-models.sh`, which reads its required list from the +same manifest. A **`verify-model-cache` matrix job** (ubuntu / macOS / Windows, `needs: +download-models`) runs the same action with `fail-on-cache-miss: true` and so proves the entry is +restorable **and complete** on every OS **before any model-consuming job starts** — all `test-java-*` +jobs, the langchain4j integration job, the Android emulator job and the fat-jar smoke jobs `need: +verify-model-cache`. (This design hardened after run 28805360584: without the cross-OS flag, cache +entries are versioned per-OS and the unreachable Windows-side entry had been re-saved **empty** (343 B) +after an eviction; and the then-separate `validate-models.bat`'s quoted `MODELS` list broke cmd's +for-tokenization so its `exit /b 1` never fired — the empty cache sailed through the "gate" and the +Windows jobs silently self-skipped every model-backed test until the fat-jar smoke's hard check caught +it. The `.bat` is gone: Windows runs the one bash script under Git Bash, which is why it uses +`head -c`/`wc -c` rather than `xxd`/`stat`.) The remaining `*_MODEL_NAME` env vars of publish.yml +serve only the consumers outside the llama module's tests (smoke scripts, Android emulator jobs, +langchain4j/agent integration); `check-natives.py` fails when one is not a filename of `models.csv`. +Locally the model tests self-skip when a GGUF is absent (`Assume.assumeTrue`), so a partial local +checkout is fine. + Test files are in `src/test/java/net/ladenthin/llama/` and `src/test/java/examples/`. @@ -2028,7 +2084,7 @@ easy to undo by accident: scope only. Safe because no source imports `org.checkerframework`. **The gate: `.github/verify-bytecode-version.sh`.** Kept **byte-identical** across java-llama.cpp / -BitcoinAddressFinder / streambuffer / srcmorph (checksum table in `workspace/crossrepostatus.md`). +BitcoinAddressFinder / streambuffer / srcmorph (listed in `.github/shared-files.sha256`, checked by the `shared-files` job). It opens every `.class` in every jar it is given and fails on any whose class-file major version exceeds `--max-major`: @@ -2306,9 +2362,11 @@ the recommended path (README "Importing in Android", Option 1): `Iterable & AutoCloseable` seam (`closeableIterableFlow`/`withCancellationToken` internals). **16 KB page-size invariant (Google Play, Android 15+ targets):** `llama/CMakeLists.txt` pins -`-Wl,-z,max-page-size=16384` in the Android guard block, and the `package-android-aar` CI job -asserts every LOAD segment of the shipped `.so` is 16384-aligned via `readelf` — a dockcross -toolchain bump cannot silently regress Play compatibility. +`-Wl,-z,max-page-size=16384` in the Android guard block, and `.github/buildcheck/nativedeps.py` +(`verify-native-deps.py`, run by the `package-android-aar` job on the staged libraries and by +`package` on every natives jar) asserts every LOAD segment of an Android `.so` is 16384-aligned, +reading the ELF program headers itself — a dockcross toolchain bump cannot silently regress Play +compatibility. **dlopen-ability invariant (bionic-only DT_NEEDED):** the same Android guard block sets `GGML_OPENMP OFF` (ggml uses its std::thread pool — Android ships no `libomp.so`; same trade @@ -2317,8 +2375,10 @@ dependency — that runtime only exists when an app packages it itself). Without dockcross cross-clang emitted `DT_NEEDED` on `libomp.so` + `libc++_shared.so`, which made `System.loadLibrary("jllama")` fail with `UnsatisfiedLinkError` on every device (caught by the `test-android-emulator` job; the released 5.0.5 arm64 lib had the same latent defect). The -`package-android-aar` job enforces a per-`.so` `DT_NEEDED` whitelist (`libc.so libm.so libdl.so -liblog.so libandroid.so`, plus `libOpenCL.so` for the OpenCL flavor) via `readelf -dW`, and +`nativedeps.ALLOWED` holds each Android directory to an exact `DT_NEEDED` list (`libc.so libm.so +libdl.so liblog.so libandroid.so`, plus `libOpenCL.so` for `Linux-Android/aarch64/opencl`) — the +same allowlist mechanism as every desktop CPU build, where the AAR job used to carry a copy of its +own; the AAR job additionally asserts the AAR's `jni/` library is the staged one. And `LlamaLoader` now includes the swallowed `System.loadLibrary` message in its "Directly from .apk/lib (…)" tried-path entry so a future dlopen reason is never invisible. diff --git a/README.md b/README.md index f37d54a3c..fa63c2a17 100644 --- a/README.md +++ b/README.md @@ -359,17 +359,18 @@ Every `net.ladenthin.llama.*` system property recognised by the library, deep-sc | `net.ladenthin.llama.backend` | unset (auto: the first backend on the classpath whose library loads) | runtime | `LlamaLoader` | Names one backend directory (e.g. `cuda13`, `vulkan`, `cpu`) to load exclusively — failure is then fatal instead of trying the next backend. See [Choosing the natives jars](#choosing-the-natives-jars). | | `net.ladenthin.llama.test.ngl` | `43` for the general suite; `0` for `ToolCallingIntegrationTest` | test | Model-backed integration tests | Number of GPU layers used during testing. Pin to `0` on CPU-only hosts: `mvn test -Dnet.ladenthin.llama.test.ngl=0`. The tool test also selects device `none` at zero layers so Metal/CUDA is not initialized. | | `net.ladenthin.llama.tool.model` | `models/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` (test self-skips if missing) | test | `ToolCallingIntegrationTest` | Path to a tool-capable GGUF used to verify required blocking and streaming tool calls. The default matches the Qwen2.5 model in upstream llama.cpp's tool-call test matrix. | -| `net.ladenthin.llama.nomic.path` | unset (test self-skips) | test | `LlamaEmbeddingsTest#testNomicEmbedLoads` | Path to a Nomic embedding model (`nomic-embed-text-v1.5.f16.gguf` or a compatible BERT-family encoder). Regression test for upstream issue #98 (BERT-encoder `result_output` assertion). | -| `net.ladenthin.llama.vision.model` | unset (test self-skips) | test | `MultimodalIntegrationTest` | Path to a vision-capable model GGUF. Any vision-capable GGUF works; CI default is `SmolVLM-500M-Instruct-Q8_0.gguf`. | -| `net.ladenthin.llama.vision.mmproj` | unset (test self-skips) | test | `MultimodalIntegrationTest` | Matching mmproj GGUF for the vision model. | +| `net.ladenthin.llama.nomic.path` | `models/nomic-embed-text-v1.5.f16.gguf` (test self-skips if missing) | test | `LlamaEmbeddingsTest#testNomicEmbedLoads` | Path to a Nomic embedding model (`nomic-embed-text-v1.5.f16.gguf` or a compatible BERT-family encoder). Regression test for upstream issue #98 (BERT-encoder `result_output` assertion). | +| `net.ladenthin.llama.vision.model` | `models/SmolVLM-500M-Instruct-Q8_0.gguf` (test self-skips if missing) | test | `MultimodalIntegrationTest` | Path to a vision-capable model GGUF. Any vision-capable GGUF works; CI default is `SmolVLM-500M-Instruct-Q8_0.gguf`. | +| `net.ladenthin.llama.vision.mmproj` | `models/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf` (test self-skips if missing) | test | `MultimodalIntegrationTest` | Matching mmproj GGUF for the vision model. | | `net.ladenthin.llama.vision.image` | `llama/src/test/resources/images/test-image.jpg` (a CC-BY-4.0 / MIT-granted photo committed to the repo) | test | `MultimodalIntegrationTest` | Visual prompt image. Any png/jpeg/webp/gif works; the extension drives MIME detection. | | `net.ladenthin.llama.audio.model` | unset (test self-skips) | test | `AudioInputIntegrationTest` (llama.cpp discussion #13759) | Path to an audio-input model GGUF (e.g. Ultravox, Qwen2.5-Omni). | | `net.ladenthin.llama.audio.mmproj` | unset (test self-skips) | test | `AudioInputIntegrationTest` | Matching audio mmproj (encoder) GGUF. | | `net.ladenthin.llama.audio.input` | `src/test/resources/audios/sample.wav` (committed) | test | `AudioInputIntegrationTest` | `.wav`/`.mp3` audio prompt clip; the extension drives format detection. | -| `net.ladenthin.llama.tts.model` | unset (test self-skips) | test | `TtsIntegrationTest` | Path to the Qwen3-TTS backbone (text) GGUF. Any Qwen3-TTS-family model works; CI default is `Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf`. | -| `net.ladenthin.llama.tts.mmproj` | unset (test self-skips) | test | `TtsIntegrationTest` | Path to the matching Qwen3-TTS mmproj GGUF (speaker encoder + code predictor + code2wav decoder); CI default is `mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf`. | +| `net.ladenthin.llama.tts.model` | `models/Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf` (test self-skips if missing) | test | `TtsIntegrationTest` | Path to the Qwen3-TTS backbone (text) GGUF. Any Qwen3-TTS-family model works; CI default is `Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf`. | +| `net.ladenthin.llama.tts.mmproj` | `models/mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf` (test self-skips if missing) | test | `TtsIntegrationTest` | Path to the matching Qwen3-TTS mmproj GGUF (speaker encoder + code predictor + code2wav decoder); CI default is `mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf`. | +| `net.ladenthin.llama.train.model` | `models/stories260K.gguf` (test self-skips if missing) | test | `LlamaTrainerIntegrationTest` | Path to the model the fine-tuning smoke trains. Must be **F32**: `llama_set_param` skips every other tensor type, so a quantized model would train nothing. | -`MultimodalIntegrationTest` self-skips when any of the three `vision.*` properties points at a missing path, so a partial setup (just the vision model + the committed image, no mmproj) lets the test class load without erroring. `AudioInputIntegrationTest` self-skips the same way over the three `audio.*` properties. `TtsIntegrationTest` likewise self-skips unless both `tts.model` and `tts.mmproj` point at existing files. +The test-model defaults are exactly CI's model set (`.github/models.csv`, URLs included), so a model downloaded into `models/` is found without any property; the properties only point a test at another file. `MultimodalIntegrationTest` self-skips when any of the three `vision.*` paths is missing, so a partial setup (just the vision model + the committed image, no mmproj) lets the test class load without erroring. `AudioInputIntegrationTest` self-skips the same way over the three `audio.*` properties. `TtsIntegrationTest` likewise self-skips unless both `tts.model` and `tts.mmproj` paths exist, and `LlamaTrainerIntegrationTest` unless `train.model` (default `models/stories260K.gguf`, an **F32** model) does. ## Documentation diff --git a/TODO.md b/TODO.md index 2160fce5f..dd33cbbbd 100644 --- a/TODO.md +++ b/TODO.md @@ -26,7 +26,7 @@ so everything below is genuinely still open. has it. Likely fix: build BoringSSL statically on macOS as on Windows (`LLAMA_BUILD_BORINGSSL`, `llama/CMakeLists.txt`), or turn HTTPS off there (`-DLLAMA_OPENSSL=OFF`; the library only needs it for URL model downloads). Then delete the two allowlist lines marked - KNOWN DEFECT in `.github/verify-native-deps.py`. Needs a macOS CI run to verify, which is why it is + KNOWN DEFECT in `.github/buildcheck/nativedeps.py`. Needs a macOS CI run to verify, which is why it is not folded into the RPC PR that surfaced it. ### RPC backend — follow-ups @@ -444,15 +444,6 @@ introduced by the version bump — they were deferred to keep that PR landable. that does not exist. Cheap to close: set `os.arch`, assert the non-identity aliases only (identity entries such as `s390x` are behaviourally redundant with the `\W`-stripping fallback). -- **`LlamaTrainer`'s end-to-end path runs on no CI platform.** `LlamaTrainerIntegrationTest` - self-skips everywhere: `net.ladenthin.llama.train.model` is set by no job and its model is in no - `.github/models.csv` row, so `validate-models.{sh,bat}` does not treat it as required. Two slices - are now mitigated — `test_tts_params.cpp`'s `TrainParams` + `ResolveCpuParams` suites for the - parameter build, and `test_wire_contracts.cpp`'s `JavaTrainingFieldContract` for the configuration - key set (`parameters.TrainingField` against `jllama_train::config_keys()`) — but nothing exercises - the Java → JNI → native trainer round trip. Adding a small training model to `models.csv` plus the - matching property to the Java test jobs would close it. - - **`LlamaLoader`'s jar-extraction internals are tested only through directory fixtures.** `BackendLoadTest` drives backend probing, extras, fallthrough and forcing over the committed `Linux/backendtest*/` trees on the test classpath (directories, not jars); extraction out of a real diff --git a/llama/src/test/java/net/ladenthin/llama/LlamaEmbeddingsTest.java b/llama/src/test/java/net/ladenthin/llama/LlamaEmbeddingsTest.java index 380f35018..2056a2299 100644 --- a/llama/src/test/java/net/ladenthin/llama/LlamaEmbeddingsTest.java +++ b/llama/src/test/java/net/ladenthin/llama/LlamaEmbeddingsTest.java @@ -211,18 +211,14 @@ private static void assertEmbeddingValid(float[] embedding, PoolingType type) { * *

This test reproduces the reporter's batch/ubatch sizing plus the fix * ({@code enableEmbedding()}) and asserts the model loads and produces a 768-dimensional - * vector. Gated on the {@link TestConstants#PROP_NOMIC_MODEL_PATH} system property so - * CI hosts without the ~120 MB GGUF file self-skip cleanly. - * - *

Run with: - * 

-     *   mvn test -Dtest=LlamaEmbeddingsTest#testNomicEmbedLoads \
-     *            -Dnet.ladenthin.llama.nomic.path=models/nomic-embed-text-v1.5.f16.gguf
-     *
+ * vector. Uses {@link TestConstants#DEFAULT_NOMIC_MODEL_PATH} (in CI's model set) unless + * {@link TestConstants#PROP_NOMIC_MODEL_PATH} names another file, and self-skips when the + * ~120 MB GGUF is missing. */ @Test public void testNomicEmbedLoads() { - String nomicPath = TestConstants.resolveModelProperty(TestConstants.PROP_NOMIC_MODEL_PATH); + String nomicPath = TestConstants.resolveModelProperty( + TestConstants.PROP_NOMIC_MODEL_PATH, TestConstants.DEFAULT_NOMIC_MODEL_PATH); Assumptions.assumeTrue( nomicPath != null, "Set -D" + TestConstants.PROP_NOMIC_MODEL_PATH + " to a nomic-embed-text GGUF to run this test"); diff --git a/llama/src/test/java/net/ladenthin/llama/LlamaTrainerIntegrationTest.java b/llama/src/test/java/net/ladenthin/llama/LlamaTrainerIntegrationTest.java index 69213a7d1..7479eccf5 100644 --- a/llama/src/test/java/net/ladenthin/llama/LlamaTrainerIntegrationTest.java +++ b/llama/src/test/java/net/ladenthin/llama/LlamaTrainerIntegrationTest.java @@ -20,10 +20,10 @@ /** * End-to-end fine-tuning smoke over a real model — the only test that exercises the - * Java -> JNI -> native trainer round trip. Self-skips unless a GGUF is provided via - * {@code -Dnet.ladenthin.llama.train.model=/path/to/model.gguf}; CI sets that on every Java test - * job to {@code stories260K.gguf} (1.19 MB, F32), so it runs there rather than self-skipping. - * A local build without that model still skips cleanly. + * Java -> JNI -> native trainer round trip. Uses {@link TestConstants#DEFAULT_TRAIN_MODEL_PATH} + * ({@code models/stories260K.gguf}, 1.19 MB, F32 -- in CI's model set, so it runs on every Java test + * job) unless {@code -Dnet.ladenthin.llama.train.model=/path/to/model.gguf} names another; a local + * build without that model skips cleanly. * *

The fixture must be F32: {@code llama_set_param} silently ignores any tensor * that is not {@code GGML_TYPE_F32}, so a quantized model trains nothing and still produces an @@ -33,10 +33,12 @@ class LlamaTrainerIntegrationTest { @Test void finetuneWritesAnOutputModel(@TempDir Path tmp) throws Exception { - String modelPath = TestConstants.resolveModelProperty("net.ladenthin.llama.train.model"); + String modelPath = TestConstants.resolveModelProperty( + TestConstants.PROP_TRAIN_MODEL_PATH, TestConstants.DEFAULT_TRAIN_MODEL_PATH); Assumptions.assumeTrue( modelPath != null && !modelPath.isEmpty() && Files.exists(Paths.get(modelPath)), - "set -Dnet.ladenthin.llama.train.model=/path/to/small.gguf to run the fine-tune smoke"); + "model missing: " + modelPath + " (or set -D" + TestConstants.PROP_TRAIN_MODEL_PATH + + "=/path/to/f32.gguf)"); StringBuilder corpus = new StringBuilder(); for (int i = 0; i < 64; i++) { diff --git a/llama/src/test/java/net/ladenthin/llama/MultimodalIntegrationTest.java b/llama/src/test/java/net/ladenthin/llama/MultimodalIntegrationTest.java index bebe952fa..656a2b214 100644 --- a/llama/src/test/java/net/ladenthin/llama/MultimodalIntegrationTest.java +++ b/llama/src/test/java/net/ladenthin/llama/MultimodalIntegrationTest.java @@ -91,8 +91,10 @@ public class MultimodalIntegrationTest { @BeforeAll public static void setup() { - modelPath = TestConstants.resolveModelProperty(TestConstants.PROP_VISION_MODEL_PATH); - mmprojPath = TestConstants.resolveModelProperty(TestConstants.PROP_VISION_MMPROJ_PATH); + modelPath = TestConstants.resolveModelProperty( + TestConstants.PROP_VISION_MODEL_PATH, TestConstants.DEFAULT_VISION_MODEL_PATH); + mmprojPath = TestConstants.resolveModelProperty( + TestConstants.PROP_VISION_MMPROJ_PATH, TestConstants.DEFAULT_VISION_MMPROJ_PATH); // Image path falls back to the committed test resource when the // -D property is unset, so the test works on local dev checkouts // without any extra wiring. The model / mmproj remain externally diff --git a/llama/src/test/java/net/ladenthin/llama/TestConstants.java b/llama/src/test/java/net/ladenthin/llama/TestConstants.java index 4c173c368..b66bfa17a 100644 --- a/llama/src/test/java/net/ladenthin/llama/TestConstants.java +++ b/llama/src/test/java/net/ladenthin/llama/TestConstants.java @@ -99,24 +99,35 @@ public static String resolveModelProperty(String key) { * ({@code nomic-embed-text-v1.5.f16.gguf} or a compatible BERT-family encoder). * Used by {@link LlamaEmbeddingsTest#testNomicEmbedLoads} to confirm upstream * issue #98 (BERT-encoder result_output assertion) stays resolved. - * When the property is unset the test self-skips. + * When the property is unset the test uses {@link #DEFAULT_NOMIC_MODEL_PATH}, and self-skips + * when the file is missing. */ public static final String PROP_NOMIC_MODEL_PATH = LlamaSystemProperties.PREFIX + ".nomic.path"; + /** Path used when {@link #PROP_NOMIC_MODEL_PATH} is unset: the CI model (.github/models.csv). */ + public static final String DEFAULT_NOMIC_MODEL_PATH = resolveModelPath("models/nomic-embed-text-v1.5.f16.gguf"); + /** Expected embedding dimension of nomic-embed-text-v1.5 (hidden size = 768). */ public static final int NOMIC_EMBED_DIM = 768; /** * System property holding a path to a vision-capable model GGUF. Consumed by * {@code MultimodalIntegrationTest}. The CI default is the - * SmolVLM-500M Q8_0 GGUF; the test self-skips when the property is unset or - * the file is missing. + * SmolVLM-500M Q8_0 GGUF ({@link #DEFAULT_VISION_MODEL_PATH}, used when the property is + * unset); the test self-skips when the file is missing. */ public static final String PROP_VISION_MODEL_PATH = LlamaSystemProperties.PREFIX + ".vision.model"; /** System property holding a path to the matching mmproj GGUF for the vision model. */ public static final String PROP_VISION_MMPROJ_PATH = LlamaSystemProperties.PREFIX + ".vision.mmproj"; + /** Path used when {@link #PROP_VISION_MODEL_PATH} is unset: the CI model (.github/models.csv). */ + public static final String DEFAULT_VISION_MODEL_PATH = resolveModelPath("models/SmolVLM-500M-Instruct-Q8_0.gguf"); + + /** Path used when {@link #PROP_VISION_MMPROJ_PATH} is unset: the CI mmproj (.github/models.csv). */ + public static final String DEFAULT_VISION_MMPROJ_PATH = + resolveModelPath("models/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf"); + /** * System property holding a path to an image used as the visual prompt in * {@code MultimodalIntegrationTest}. When unset the test falls back to @@ -162,7 +173,8 @@ public static String resolveModelProperty(String key) { /** * System property holding a path to the Qwen3-TTS backbone GGUF used by - * {@code TtsIntegrationTest}. The test self-skips when this or the mmproj is unset/missing. + * {@code TtsIntegrationTest} ({@link #DEFAULT_TTS_MODEL_PATH} when unset). The test self-skips when + * this or the mmproj file is missing. */ public static final String PROP_TTS_MODEL = LlamaSystemProperties.PREFIX + ".tts.model"; @@ -171,4 +183,21 @@ public static String resolveModelProperty(String key) { * predictor + code2wav decoder). */ public static final String PROP_TTS_MMPROJ = LlamaSystemProperties.PREFIX + ".tts.mmproj"; + + /** Path used when {@link #PROP_TTS_MODEL} is unset: the CI backbone (.github/models.csv). */ + public static final String DEFAULT_TTS_MODEL_PATH = resolveModelPath("models/Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf"); + + /** Path used when {@link #PROP_TTS_MMPROJ} is unset: the CI mmproj (.github/models.csv). */ + public static final String DEFAULT_TTS_MMPROJ_PATH = + resolveModelPath("models/mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf"); + + /** + * System property holding a path to the F32 model {@code LlamaTrainerIntegrationTest} fine-tunes + * (F32 because {@code llama_set_param} skips every other tensor type). The test self-skips when + * the file is missing. + */ + public static final String PROP_TRAIN_MODEL_PATH = LlamaSystemProperties.PREFIX + ".train.model"; + + /** Path used when {@link #PROP_TRAIN_MODEL_PATH} is unset: the CI model (.github/models.csv). */ + public static final String DEFAULT_TRAIN_MODEL_PATH = resolveModelPath("models/stories260K.gguf"); } diff --git a/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java b/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java index b33106e6b..b91d919fb 100644 --- a/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java +++ b/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java @@ -78,6 +78,35 @@ public void propertyLookupResolvesAndHonoursTheDefault() { } } + /** + * The models the tests default to and the models CI downloads are one set. + * + *

{@code .github/models.csv} is what download-models fetches and validate-models requires; + * every model-gated test defaults to a {@code models/} constant here. A model in the list + * that no constant names is downloaded for nothing, and a constant naming a model the list lacks + * makes its test self-skip in CI -- green, and silently. Both directions are asserted, which is + * also why CI passes no {@code -Dnet.ladenthin.llama.*} model property: the defaults are the CI set. + */ + @Test + public void theModelDefaultsAreExactlyTheCiModelSet() throws Exception { + java.util.Set defaults = new java.util.TreeSet<>(); + for (java.lang.reflect.Field field : TestConstants.class.getFields()) { + Object value = field.get(null); + if (value instanceof String && ((String) value).endsWith(".gguf")) { + defaults.add(Paths.get((String) value).getFileName().toString()); + } + } + Path manifest = Paths.get(TestConstants.resolveModelPath(".github/models.csv")); + assertTrue(Files.exists(manifest), "model manifest not found: " + manifest); + java.util.Set ci = new java.util.TreeSet<>(); + for (String line : Files.readAllLines(manifest, StandardCharsets.UTF_8)) { + if (!line.trim().isEmpty() && !line.startsWith("#")) { + ci.add(line.split(",", 2)[0].trim()); + } + } + assertEquals(ci, defaults); + } + @Test public void theShippedModelConstantsGoThroughTheResolver() { // Guards the wiring itself: if a future edit drops the resolveModelPath(...) wrapper from a @@ -98,6 +127,22 @@ public void theShippedModelConstantsGoThroughTheResolver() { assertEquals( TestConstants.resolveModelPath("src/test/resources/images/test-image.jpg"), TestConstants.DEFAULT_VISION_IMAGE_PATH); + assertEquals( + TestConstants.resolveModelPath("models/nomic-embed-text-v1.5.f16.gguf"), + TestConstants.DEFAULT_NOMIC_MODEL_PATH); + assertEquals( + TestConstants.resolveModelPath("models/SmolVLM-500M-Instruct-Q8_0.gguf"), + TestConstants.DEFAULT_VISION_MODEL_PATH); + assertEquals( + TestConstants.resolveModelPath("models/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf"), + TestConstants.DEFAULT_VISION_MMPROJ_PATH); + assertEquals( + TestConstants.resolveModelPath("models/Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf"), + TestConstants.DEFAULT_TTS_MODEL_PATH); + assertEquals( + TestConstants.resolveModelPath("models/mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf"), + TestConstants.DEFAULT_TTS_MMPROJ_PATH); + assertEquals(TestConstants.resolveModelPath("models/stories260K.gguf"), TestConstants.DEFAULT_TRAIN_MODEL_PATH); assertEquals( TestConstants.resolveModelPath("src/test/resources/audios/sample.wav"), TestConstants.DEFAULT_AUDIO_INPUT_PATH); diff --git a/llama/src/test/java/net/ladenthin/llama/TtsIntegrationTest.java b/llama/src/test/java/net/ladenthin/llama/TtsIntegrationTest.java index cce35aff6..37c78fea5 100644 --- a/llama/src/test/java/net/ladenthin/llama/TtsIntegrationTest.java +++ b/llama/src/test/java/net/ladenthin/llama/TtsIntegrationTest.java @@ -23,9 +23,9 @@ * {@code mtmd_helper::gen_audio} pipeline). Loads the backbone+mmproj TTS pipeline and synthesizes * a short clip, checking the WAV container is well-formed. * - *

Self-skips when {@link TestConstants#PROP_TTS_MODEL} or {@link TestConstants#PROP_TTS_MMPROJ} - * is unset or its file is missing, so it runs only where the (large) Qwen3-TTS backbone + mmproj - * GGUFs have been staged. + *

Uses CI's Qwen3-TTS backbone + mmproj ({@link TestConstants#DEFAULT_TTS_MODEL_PATH}, + * {@link TestConstants#DEFAULT_TTS_MMPROJ_PATH}) unless {@link TestConstants#PROP_TTS_MODEL} / + * {@link TestConstants#PROP_TTS_MMPROJ} name others, and self-skips when a file is missing. */ public class TtsIntegrationTest { @@ -36,8 +36,10 @@ public class TtsIntegrationTest { @DisplayName("synthesize() returns a well-formed, non-silent 24 kHz mono 16-bit WAV") @Timeout(value = 300_000, unit = TimeUnit.MILLISECONDS) public void synthesizesWellFormedWav() { - String model = TestConstants.resolveModelProperty(TestConstants.PROP_TTS_MODEL); - String mmproj = TestConstants.resolveModelProperty(TestConstants.PROP_TTS_MMPROJ); + String model = + TestConstants.resolveModelProperty(TestConstants.PROP_TTS_MODEL, TestConstants.DEFAULT_TTS_MODEL_PATH); + String mmproj = TestConstants.resolveModelProperty( + TestConstants.PROP_TTS_MMPROJ, TestConstants.DEFAULT_TTS_MMPROJ_PATH); Assumptions.assumeTrue( model != null && !model.isEmpty(), "TTS model not set (-D" + TestConstants.PROP_TTS_MODEL + "=...)"); Assumptions.assumeTrue( From 03f9fb6011d21fa3c7499d84b447853d7a6db491 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 30 Sep 2026 22:54:10 +0000 Subject: [PATCH 03/22] Smoke fat jars as one matrix, macOS/Windows Java tests as a reusable workflow The four fat-jar smoke jobs are one smoke-fatjar matrix (one row per all-- jar, fail-fast off); their single-jar artifacts are named llama-fatjar-smoke-. check-natives.py holds the uploads (name and the jar inside) and the matrix rows to the targets natives.csv derives. The three macOS and two Windows Java test jobs call .github/workflows/java-tests.yml with four inputs and keep their ids, names and needs. The JDK version moved to .java-version, read by every setup-java step, since a called workflow sees none of the caller's env. Check names of these jobs change; no repository has a required status check, so no setting refers to them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/buildcheck/natives.py | 45 +- .github/buildcheck/tests/test_natives.py | 47 +- .github/smoke-native-macos.sh | 2 +- .github/workflows/codeql.yml | 2 +- .github/workflows/java-tests.yml | 122 +++++ .github/workflows/publish.yml | 569 +++++------------------ .github/workflows/sonarqube.yml | 2 +- .java-version | 1 + .java-version.license | 3 + CHANGELOG.md | 6 + CLAUDE.md | 71 +-- TODO.md | 5 +- 12 files changed, 390 insertions(+), 485 deletions(-) create mode 100644 .github/workflows/java-tests.yml create mode 100644 .java-version create mode 100644 .java-version.license diff --git a/.github/buildcheck/natives.py b/.github/buildcheck/natives.py index 731b43f40..b00d4ddf7 100644 --- a/.github/buildcheck/natives.py +++ b/.github/buildcheck/natives.py @@ -15,7 +15,8 @@ directory the list lacks * README.md documents every classifier and, for the all-backends fat jars derived from the list (fatjar_targets), that each one is -uploaded for a smoke job, launched by one, named in the agent jar's Class-Path and in the README. +uploaded as llama-fatjar-smoke-, launched by a smoke job (a script line naming it, or a row +of the smoke-fatjar matrix), named in the agent jar's Class-Path and in the README. package-fatjars.sh does not repeat the targets at all: it asks this module for them. Every check is a function of the texts it compares, so the tests drive them with literals. @@ -39,6 +40,12 @@ FATJAR_NAME = re.compile(r"all-([a-z0-9]+(?:-[a-z0-9]+)*?)-jar-with-dependencies") +# package-fatjars uploads each all-backends fat jar alone, as llama-fatjar-smoke-, for the +# smoke-fatjar matrix; a matrix row is a flow mapping that starts with its target. +SMOKE_ARTIFACT = "llama-fatjar-smoke-" +MATRIX_ROW = re.compile(r"^\s*-\s*\{\s*target:\s*([a-z0-9-]+)\s*[,}]") +MATRIX_FATJAR = "all-${{ matrix.target }}-jar-with-dependencies" + def rows(text): """The rows of natives.csv (comment lines and blank lines skipped).""" @@ -168,16 +175,46 @@ def check_workflow(natives, jobs): assembler = jobs.get("package-fatjars") if assembler is None: return failures + ["publish.yml has no `package-fatjars` job"] - uploaded = fatjar_names("\n".join(line for line in assembler.lines if "path:" in line)) - failures += compare("publish.yml package-fatjars smoke-jar uploads", targets, uploaded) + failures += check_smoke_uploads(targets, assembler) launched = set() for job in jobs.values(): if job.name != "package-fatjars": - launched |= fatjar_names("\n".join(line for line in job.lines if ".github/smoke-" in line)) + launched |= smoke_runs(job, failures) failures += compare("publish.yml fat-jar smoke runs", targets, launched) return failures +def check_smoke_uploads(targets, assembler): + """package-fatjars uploads llama-fatjar-smoke- for every target, each holding that + target's jar (the path names the same target as the artifact).""" + failures, uploaded = [], set() + for step in assembler.steps(): + text = "\n".join(step) + names = [n for n in re.findall(r"name:\s*(\S+)", text) if n.startswith(SMOKE_ARTIFACT)] + if "actions/upload-artifact@" not in text or not names: + continue + target = names[0][len(SMOKE_ARTIFACT):] + uploaded.add(target) + if fatjar_names("\n".join(line for line in step if "path:" in line)) != {target}: + failures.append(f"publish.yml package-fatjars: {names[0]} does not upload the {target} fat jar") + return compare("publish.yml package-fatjars smoke-jar uploads", targets, uploaded) + failures + + +def smoke_runs(job, failures): + """The fat-jar targets a job launches: the ones its smoke-script lines name, and, when those run + `all-${{ matrix.target }}-...`, every row of its matrix -- which must then download each row's + own smoke jar.""" + smoke = "\n".join(line for line in job.lines if ".github/smoke-" in line) + if not smoke: + return set() + launched = fatjar_names(smoke) + if MATRIX_FATJAR in job.text: + launched |= {m.group(1) for m in map(MATRIX_ROW.match, job.lines) if m} + if f"name: {SMOKE_ARTIFACT}${{{{ matrix.target }}}}" not in job.text: + failures.append(f"publish.yml: {job.name} does not download {SMOKE_ARTIFACT}${{{{ matrix.target }}}}") + return launched + + def check_loader(natives, loader_text): block = re.search(r"BACKEND_PRIORITY\s*=(.*?);", loader_text, re.S) priority = set(re.findall(r'"([^"]+)"', block.group(1))) if block else set() diff --git a/.github/buildcheck/tests/test_natives.py b/.github/buildcheck/tests/test_natives.py index 86749a488..d272b3413 100644 --- a/.github/buildcheck/tests/test_natives.py +++ b/.github/buildcheck/tests/test_natives.py @@ -40,7 +40,8 @@ def platform(classifiers): def jobs(rows=ROWS, package_needs="[build]", smoke_targets=("linux-x86-64",), run_targets=("linux-x86-64",)): build = [line for r in rows for line in upload("natives-" + r["classifier"])] - fatjars = [line for t in smoke_targets for line in upload("s-" + t, f"f/llama-*-all-{t}-jar-with-dependencies.jar")] + fatjars = [line for t in smoke_targets + for line in upload("llama-fatjar-smoke-" + t, f"f/llama-*-all-{t}-jar-with-dependencies.jar")] smoke = [f" - run: .github/smoke-test-fatjar.sh f 'llama-*-all-{t}-jar-with-dependencies.jar'" for t in run_targets] return workflow.parse(workflow_text([("build", None, build), ("other", None, []), @@ -49,6 +50,25 @@ def jobs(rows=ROWS, package_needs="[build]", smoke_targets=("linux-x86-64",), ru ("smoke", "[package-fatjars]", smoke)])) +SMOKE_MATRIX = [" strategy:", " matrix:", " include:", "{rows}", + " steps:", " - uses: actions/download-artifact@v8", " with:", + " name: {download}", " path: f/", + " - run: .github/smoke-test-fatjar.sh f 'llama-*-all-${{ matrix.target }}-jar-with-dependencies.jar'"] + + +def matrix_jobs(targets, download="llama-fatjar-smoke-${{ matrix.target }}", smoke=None, + jar="f/llama-*-all-linux-x86-64-jar-with-dependencies.jar"): + """The workflow of jobs(), with the smoke job as a matrix over `targets`.""" + rows = "\n".join(f" - {{ target: {t}, runner: ubuntu-latest }}" for t in targets) + body = [line.format(rows=rows, download=download) if "{" in line and "matrix" not in line else line + for line in (smoke or SMOKE_MATRIX)] + body = [line for line in body if line] + build = [line for r in ROWS for line in upload("natives-" + r["classifier"])] + fatjars = upload("llama-fatjar-smoke-linux-x86-64", jar) + return workflow.parse(workflow_text([("build", None, build), ("package", "build", []), + ("package-fatjars", "package", fatjars), ("smoke", "package-fatjars", body)])) + + class ListTest(unittest.TestCase): def test_rows_skip_comments(self): @@ -108,7 +128,7 @@ def test_package_may_wait_transitively(self): wf = workflow_text([("build", None, [line for r in ROWS for line in upload("natives-" + r["classifier"])]), ("mid", "build", []), ("package", "mid", []), ("package-fatjars", "package", - upload("s", "f/llama-*-all-linux-x86-64-jar-with-dependencies.jar")), + upload("llama-fatjar-smoke-linux-x86-64", "f/llama-*-all-linux-x86-64-jar-with-dependencies.jar")), ("smoke", "package-fatjars", [" - run: .github/smoke-test-fatjar.sh f 'llama-*-all-linux-x86-64-jar-with-dependencies.jar'"])]) self.assertEqual(natives.check_workflow(ROWS, workflow.parse(wf)), []) @@ -119,6 +139,29 @@ def test_every_fatjar_target_is_uploaded_and_launched(self): self.assertEqual(natives.check_workflow(ROWS, jobs(run_targets=("linux-x86-64", "linux-s390x"))), ["publish.yml fat-jar smoke runs: linux-s390x is not in .github/natives.csv"]) + def test_a_smoke_upload_must_hold_its_own_target(self): + wf = matrix_jobs(["linux-x86-64"], jar="f/llama-*-all-linux-aarch64-jar-with-dependencies.jar") + self.assertEqual(natives.check_workflow(ROWS, wf), + ["publish.yml package-fatjars: llama-fatjar-smoke-linux-x86-64 does not upload the " + "linux-x86-64 fat jar"]) + + def test_a_matrix_launches_its_rows(self): + self.assertEqual(natives.check_workflow(ROWS, matrix_jobs(["linux-x86-64"])), []) + self.assertEqual(natives.check_workflow(ROWS, matrix_jobs([])), + ["publish.yml fat-jar smoke runs: missing linux-x86-64"]) + self.assertEqual(natives.check_workflow(ROWS, matrix_jobs(["linux-x86-64", "linux-s390x"])), + ["publish.yml fat-jar smoke runs: linux-s390x is not in .github/natives.csv"]) + + def test_a_matrix_must_download_each_rows_own_jar(self): + self.assertEqual(natives.check_workflow(ROWS, matrix_jobs(["linux-x86-64"], download="llama-fatjars")), + ["publish.yml: smoke does not download llama-fatjar-smoke-${{ matrix.target }}"]) + + def test_matrix_rows_count_only_where_the_smoke_run_uses_them(self): + # A matrix whose smoke line names a fixed jar launches that jar, not the rows. + lines = [line.replace("${{ matrix.target }}-jar", "linux-x86-64-jar") if "smoke-test" in line else line + for line in SMOKE_MATRIX] + self.assertEqual(natives.check_workflow(ROWS, matrix_jobs(["linux-s390x"], smoke=lines)), []) + def test_loader(self): text = 'List BACKEND_PRIORITY = Arrays.asList(\n "cuda13", "msvc",\n "metal", "opencl", "cpu");' self.assertEqual(natives.check_loader(ROWS, text), []) diff --git a/.github/smoke-native-macos.sh b/.github/smoke-native-macos.sh index ace6f7241..0d78c3eeb 100755 --- a/.github/smoke-native-macos.sh +++ b/.github/smoke-native-macos.sh @@ -13,7 +13,7 @@ # one path and produced a byte-level hybrid, the result — a library whose ad-hoc linker signature no # longer matched its own __TEXT pages, which macOS SIGKILLs on load — shipped in 5.0.6 and several # 5.0.7 snapshots with an all-green pipeline. Linux and Windows already had the equivalent gate -# (`smoke-fatjar-linux` / `smoke-fatjar-windows`, downstream of `package`); macOS had none. +# (the `smoke-fatjar` matrix, downstream of `package`); macOS had none. # # This is the macOS member of the cross-repo "no release asset is attached that CI has not run" # convention (workspace/policies/fat-jar-release-assets.md). It is NOT the shared diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 902d2c8a3..d4b347efc 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -21,7 +21,7 @@ jobs: steps: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 - with: { java-version: '21', distribution: temurin } + with: { java-version-file: .java-version, distribution: temurin } - uses: github/codeql-action/init@v4 with: { languages: java, queries: +security-and-quality } - name: Build with Maven diff --git a/.github/workflows/java-tests.yml b/.github/workflows/java-tests.yml new file mode 100644 index 000000000..c2a8c736d --- /dev/null +++ b/.github/workflows/java-tests.yml @@ -0,0 +1,122 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT + +# The model-backed Java suite against one native build, for the macOS and Windows test jobs of +# publish.yml (the three macOS arm64 builds and the two Windows x86-64 CPU builds). Called, never +# run on its own: the caller job keeps its id, its name and its `needs`, so the release gate and +# `package` see the same graph as before. A check shows as " / Java tests". +# +# The only differences between the callers are the inputs below; everything OS-specific (memory +# report, core dumps vs. Windows Error Reporting minidumps) is decided here by runner.os. The +# JDK is .java-version, the models come from the shared cache, so nothing is passed through +# `with:` that the caller's `env:` would have to repeat (a reusable workflow sees none of it). +name: Java tests +on: + workflow_call: + inputs: + runs-on: + description: The runner label (macos-14, macos-15, windows-2025-vs2026, ...). + type: string + required: true + natives-artifact: + description: The artifact holding the native build under test (its /// tree). + type: string + required: true + maven-args: + description: Extra arguments for `mvn test`, e.g. -Dnet.ladenthin.llama.test.ngl=0. + type: string + required: false + default: '' + error-artifact: + description: Name of the artifact uploaded on failure (crash logs, dumps, surefire reports). + type: string + required: true +permissions: + contents: read +jobs: + tests: + name: Java tests + runs-on: ${{ inputs.runs-on }} + steps: + - uses: actions/checkout@v7 + with: + persist-credentials: false + - name: Display CPU Info + shell: bash + run: bash .github/print-host-info.sh + - uses: actions/download-artifact@v8 + with: + name: ${{ inputs.natives-artifact }} + path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ + - uses: ./.github/actions/restore-models + - uses: actions/setup-java@v6 + with: + distribution: 'temurin' + java-version-file: .java-version + - name: Memory before tests + if: runner.os == 'macOS' + run: vm_stat && sysctl hw.memsize hw.physmem + - name: Memory before tests + if: runner.os == 'Windows' + shell: pwsh + run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List + - name: Enable core dumps + if: runner.os != 'Windows' + run: ulimit -c unlimited + - name: Enable WER LocalDumps for java.exe + # Windows Error Reporting writes minidumps when java.exe (or any other registered process) + # crashes via __fastfail / abort / unhandled SEH -- the Windows analogue of Linux core + # dumps, so that a JVM crash inside the JNI layer leaves a real native callstack instead of + # just surefire's "VM terminated without saying goodbye" line. DumpType=2 == + # MiniDumpWithFullMemory; the workspace dumps/ folder is globbed by the failure upload. + if: runner.os == 'Windows' + shell: pwsh + run: | + $key = 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\java.exe' + New-Item -Path $key -Force | Out-Null + New-Item -Path "${{ github.workspace }}\dumps" -ItemType Directory -Force | Out-Null + New-ItemProperty -Path $key -Name 'DumpFolder' -Value "${{ github.workspace }}\dumps" -PropertyType ExpandString -Force | Out-Null + New-ItemProperty -Path $key -Name 'DumpType' -Value 2 -PropertyType DWord -Force | Out-Null + New-ItemProperty -Path $key -Name 'DumpCount' -Value 5 -PropertyType DWord -Force | Out-Null + Get-ItemProperty -Path $key | Format-List + - name: Run tests + shell: bash + env: + MAVEN_ARGS_EXTRA: ${{ inputs.maven-args }} + # MAVEN_ARGS_EXTRA is split on purpose: it carries zero or more separate arguments. + run: mvn -e --no-transfer-progress -f llama/pom.xml $MAVEN_ARGS_EXTRA test + # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that + # fails makes Surefire record tests="0" for that class -- no entries at all, so it is + # invisible to a skip check. That is exactly how every model-gated class stayed silently + # muted on every test-java-* job for months. The floor is deliberately slack; the per-class + # zero check is the sensitive half. + - name: Verify tests actually ran + shell: bash + run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 + - name: Memory after tests + if: always() && runner.os == 'macOS' + run: vm_stat && sysctl hw.memsize hw.physmem + - name: Memory after tests + if: always() && runner.os == 'Windows' + shell: pwsh + run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List + # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). + - name: Print crash logs (on failure) + if: failure() + shell: bash + run: bash .github/print-crash-logs.sh llama + - if: failure() + uses: actions/upload-artifact@v7 + with: + name: ${{ inputs.error-artifact }} + path: | + ${{ github.workspace }}/llama/hs_err_pid*.log + ${{ github.workspace }}/llama/*.hprof + ${{ github.workspace }}/dumps/*.dmp + ${{ github.workspace }}/llama/target/surefire-reports/*.dump + ${{ github.workspace }}/llama/target/surefire-reports/*.dumpstream + ${{ github.workspace }}/llama/target/surefire-reports/*.txt + ${{ github.workspace }}/llama/target/surefire-reports/TEST-*.xml + ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/**/* + if-no-files-found: warn diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 066ad9686..1c34d0915 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -20,7 +20,9 @@ on: type: boolean default: true env: - JAVA_VERSION: '21' + # The JDK every job builds with is .java-version (setup-java's java-version-file), so the + # reusable workflows read the same one without an input. + # The Gradle builds (Android AAR, the consumer fixture, the llmservice app, the signing self-test). # AGP 9.4.x needs Gradle >= 9.6.0 (CLAUDE.md, Android app). Not seen by Dependabot: bump by hand. GRADLE_VERSION: '9.8.0' @@ -170,7 +172,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: @@ -280,7 +282,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin # .github/buildcheck/natives.py; its unit tests run in the shared-files job. - name: Natives list check (everything that names a natives jar agrees with natives.csv) @@ -316,7 +318,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - name: Install parent + core net.ladenthin:llama into the local repo (Java only) run: > @@ -340,7 +342,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - name: Install parent + core net.ladenthin:llama into the local repo (Java only) run: > @@ -375,7 +377,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) run: > mvn -B --no-transfer-progress -pl llama -am -DskipTests -Denforcer.skip=true @@ -443,7 +445,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - name: Install parent + core net.ladenthin:llama into the local repo (Java only) run: > @@ -494,7 +496,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) run: > mvn -B --no-transfer-progress -pl llama -am -DskipTests -Denforcer.skip=true @@ -719,7 +721,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install toolchain (GCC 14, mirrors upstream llama.cpp ARM release) run: | sudo apt-get update @@ -769,7 +771,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install s390x cross toolchain + qemu-user run: | sudo apt-get update @@ -821,7 +823,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install Vulkan SDK (headers + loader + glslc shader compiler) run: | sudo apt-get update @@ -857,7 +859,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install toolchain (GCC 14) + Vulkan SDK run: | sudo apt-get update @@ -985,7 +987,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: @@ -1108,7 +1110,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: @@ -1193,7 +1195,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: @@ -1289,7 +1291,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: temurin - uses: gradle/actions/setup-gradle@v6 with: @@ -1371,7 +1373,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Display CPU Info shell: bash run: bash .github/print-host-info.sh @@ -1409,7 +1411,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Display CPU Info shell: bash run: bash .github/print-host-info.sh @@ -1816,7 +1818,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install ROCm/HIP (TheRock wheels) # KEEP IN SYNC WITH UPSTREAM: ROCM_VERSION tracks the ubuntu-rocm job in llama.cpp's # .github/workflows/release.yml at the pinned GIT_TAG (GPU targets: see the build step). Since ROCm 7.14 @@ -1955,7 +1957,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install Intel oneAPI (DPC++ + MKL) run: | wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null @@ -1996,7 +1998,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install Intel oneAPI (DPC++ + MKL) run: | wget -qO- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null @@ -2109,7 +2111,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Install OpenCL dev + Intel OpenVINO 2026.4 (archive) run: | # Intel's OpenVINO APT repo only publishes up to ~2025 (the /openvino/2026 path 404s), and @@ -2203,7 +2205,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Display CPU Info shell: bash run: bash .github/print-host-info.sh @@ -2237,7 +2239,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Display CPU Info shell: bash run: bash .github/print-host-info.sh @@ -2279,7 +2281,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Memory before tests run: free -h - name: Enable core dumps @@ -2353,7 +2355,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version cache: maven - name: Test under vmlens (interleaving analysis) # Add each new test in the `vmlens` package to this -Dtest list (surefire @@ -2369,314 +2371,58 @@ jobs: path: llama/target/vmlens-report/ if-no-files-found: ignore + # The macOS and Windows Java test jobs: one reusable workflow (.github/workflows/java-tests.yml), + # called with the native build under test. The ids, names and needs stay here, so `package`, + # the release gate and every reader of this file see the same jobs as before. test-java-macos-arm64-metal: name: Java Tests macOS 14 arm64 (Metal) needs: [build-macos-arm64-metal, verify-model-cache] - runs-on: macos-14 - steps: - - uses: actions/checkout@v7 - - name: Display CPU Info - shell: bash - run: bash .github/print-host-info.sh - - uses: actions/download-artifact@v8 - with: - name: macos-14-metal - path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Memory before tests - run: vm_stat && sysctl hw.memsize hw.physmem - - name: Enable core dumps - run: ulimit -c unlimited - - name: Run tests - run: | - mvn -e --no-transfer-progress -f llama/pom.xml -Dnet.ladenthin.llama.test.ngl=0 test - # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that - # fails makes Surefire record tests="0" for that class -- no entries at all, so it is - # invisible to a skip check. That is exactly how every model-gated class stayed silently - # muted on every test-java-* job for months. The floor is deliberately slack; the per-class - # zero check is the sensitive half. - - name: Verify tests actually ran - run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 - - name: Memory after tests - if: always() - run: vm_stat && sysctl hw.memsize hw.physmem - # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - - name: Print crash logs (on failure) - if: failure() - shell: bash - run: bash .github/print-crash-logs.sh llama - - if: failure() - uses: actions/upload-artifact@v7 - with: - name: error-log-macos-14-metal - path: | - ${{ github.workspace }}/llama/hs_err_pid*.log - ${{ github.workspace }}/llama/*.hprof - ${{ github.workspace }}/llama/target/surefire-reports/*.dump - ${{ github.workspace }}/llama/target/surefire-reports/*.dumpstream - ${{ github.workspace }}/llama/target/surefire-reports/*.txt - ${{ github.workspace }}/llama/target/surefire-reports/TEST-*.xml - if-no-files-found: warn + uses: ./.github/workflows/java-tests.yml + with: + runs-on: macos-14 + natives-artifact: macos-14-metal + maven-args: -Dnet.ladenthin.llama.test.ngl=0 + error-artifact: error-log-macos-14-metal test-java-macos-arm64-no-metal: name: Java Tests macOS 15 arm64 (no Metal) needs: [build-macos-arm64-no-metal, verify-model-cache] - runs-on: macos-15 - steps: - - uses: actions/checkout@v7 - - name: Display CPU Info - shell: bash - run: bash .github/print-host-info.sh - - uses: actions/download-artifact@v8 - with: - name: macos-15-no-metal - path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Memory before tests - run: vm_stat && sysctl hw.memsize hw.physmem - - name: Enable core dumps - run: ulimit -c unlimited - - name: Run tests - run: | - mvn -e --no-transfer-progress -f llama/pom.xml test - # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that - # fails makes Surefire record tests="0" for that class -- no entries at all, so it is - # invisible to a skip check. That is exactly how every model-gated class stayed silently - # muted on every test-java-* job for months. The floor is deliberately slack; the per-class - # zero check is the sensitive half. - - name: Verify tests actually ran - run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 - - name: Memory after tests - if: always() - run: vm_stat && sysctl hw.memsize hw.physmem - # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - - name: Print crash logs (on failure) - if: failure() - shell: bash - run: bash .github/print-crash-logs.sh llama - - if: failure() - uses: actions/upload-artifact@v7 - with: - name: error-log-macos-15-no-metal - path: | - ${{ github.workspace }}/llama/hs_err_pid*.log - ${{ github.workspace }}/llama/*.hprof - ${{ github.workspace }}/llama/target/surefire-reports/*.dump - ${{ github.workspace }}/llama/target/surefire-reports/*.dumpstream - ${{ github.workspace }}/llama/target/surefire-reports/*.txt - ${{ github.workspace }}/llama/target/surefire-reports/TEST-*.xml - if-no-files-found: warn + uses: ./.github/workflows/java-tests.yml + with: + runs-on: macos-15 + natives-artifact: macos-15-no-metal + error-artifact: error-log-macos-15-no-metal test-java-macos-arm64-metal-15: name: Java Tests macOS 15 arm64 (Metal) needs: [build-macos-arm64-metal-15, verify-model-cache] - runs-on: macos-15 - steps: - - uses: actions/checkout@v7 - - name: Display CPU Info - shell: bash - run: bash .github/print-host-info.sh - - uses: actions/download-artifact@v8 - with: - name: natives-metal-macos-aarch64 - path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Memory before tests - run: vm_stat && sysctl hw.memsize hw.physmem - - name: Enable core dumps - run: ulimit -c unlimited - - name: Run tests - run: | - mvn -e --no-transfer-progress -f llama/pom.xml test - # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that - # fails makes Surefire record tests="0" for that class -- no entries at all, so it is - # invisible to a skip check. That is exactly how every model-gated class stayed silently - # muted on every test-java-* job for months. The floor is deliberately slack; the per-class - # zero check is the sensitive half. - - name: Verify tests actually ran - run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 - - name: Memory after tests - if: always() - run: vm_stat && sysctl hw.memsize hw.physmem - # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - - name: Print crash logs (on failure) - if: failure() - shell: bash - run: bash .github/print-crash-logs.sh llama - - if: failure() - uses: actions/upload-artifact@v7 - with: - name: error-log-macos-15-metal - path: | - ${{ github.workspace }}/llama/hs_err_pid*.log - ${{ github.workspace }}/llama/*.hprof - ${{ github.workspace }}/llama/target/surefire-reports/*.dump - ${{ github.workspace }}/llama/target/surefire-reports/*.dumpstream - ${{ github.workspace }}/llama/target/surefire-reports/*.txt - ${{ github.workspace }}/llama/target/surefire-reports/TEST-*.xml - if-no-files-found: warn + uses: ./.github/workflows/java-tests.yml + with: + runs-on: macos-15 + natives-artifact: natives-metal-macos-aarch64 + error-artifact: error-log-macos-15-metal test-java-windows-x86_64: name: Java Tests Windows 2025 x86_64 (default / Ninja) needs: [build-windows-x86_64, verify-model-cache] - runs-on: windows-2025-vs2026 - steps: - - uses: actions/checkout@v7 - - name: Display CPU Info - shell: bash - run: bash .github/print-host-info.sh - - uses: actions/download-artifact@v8 - with: - name: natives-cpu-windows-x86-64 - path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Memory before tests - run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List - shell: pwsh - - name: Enable WER LocalDumps for java.exe - # Windows Error Reporting writes minidumps when java.exe (or any other - # registered process) crashes via __fastfail / abort / unhandled SEH. - # We use it as the Windows analogue of Linux core dumps so that a JVM - # crash inside the JNI layer leaves us a real native callstack instead - # of just surefire's "VM terminated without saying goodbye" line. - # DumpType=2 == MiniDumpWithFullMemory; the workspace dumps/ folder is - # globbed by the failure-upload step below. - shell: pwsh - run: | - $key = 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\java.exe' - New-Item -Path $key -Force | Out-Null - New-Item -Path "${{ github.workspace }}\dumps" -ItemType Directory -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpFolder' -Value "${{ github.workspace }}\dumps" -PropertyType ExpandString -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpType' -Value 2 -PropertyType DWord -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpCount' -Value 5 -PropertyType DWord -Force | Out-Null - Get-ItemProperty -Path $key | Format-List - - name: Run tests - run: | - mvn -e --no-transfer-progress -f llama/pom.xml test - # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that - # fails makes Surefire record tests="0" for that class -- no entries at all, so it is - # invisible to a skip check. That is exactly how every model-gated class stayed silently - # muted on every test-java-* job for months. The floor is deliberately slack; the per-class - # zero check is the sensitive half. - - name: Verify tests actually ran - shell: bash - run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 - - name: Memory after tests - if: always() - run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List - shell: pwsh - # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - - name: Print crash logs (on failure) - if: failure() - shell: bash - run: bash .github/print-crash-logs.sh llama - - if: failure() - uses: actions/upload-artifact@v7 - with: - name: windows-output - path: | - ${{ github.workspace }}\llama\hs_err_pid*.log - ${{ github.workspace }}\llama\*.hprof - ${{ github.workspace }}\dumps\*.dmp - ${{ github.workspace }}\llama\target\surefire-reports\*.dump - ${{ github.workspace }}\llama\target\surefire-reports\*.dumpstream - ${{ github.workspace }}\llama\target\surefire-reports\*.txt - ${{ github.workspace }}\llama\target\surefire-reports\TEST-*.xml - ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/**/* - if-no-files-found: warn + uses: ./.github/workflows/java-tests.yml + with: + runs-on: windows-2025-vs2026 + natives-artifact: natives-cpu-windows-x86-64 + error-artifact: windows-output # Java/inference validation of the MSVC-built x86_64 DLL (the analogue of # test-java-windows-x86_64 for the default Ninja build). Loads the MSVC jllama.dll # via JNI and runs the full model-backed suite, so both Windows generators are - # validated end-to-end before the `msvc-windows` classifier JAR ships. + # validated end-to-end before the `msvc-windows` natives jar ships. test-java-windows-x86_64-msvc: name: Java Tests Windows 2025 x86_64 (MSVC classifier) needs: [build-windows-x86_64-msvc, verify-model-cache] - runs-on: windows-2025-vs2026 - steps: - - uses: actions/checkout@v7 - - name: Display CPU Info - shell: bash - run: bash .github/print-host-info.sh - - uses: actions/download-artifact@v8 - with: - name: natives-msvc-windows-x86-64 - path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Memory before tests - run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List - shell: pwsh - - name: Enable WER LocalDumps for java.exe - # Windows Error Reporting writes minidumps when java.exe (or any other - # registered process) crashes via __fastfail / abort / unhandled SEH. - # We use it as the Windows analogue of Linux core dumps so that a JVM - # crash inside the JNI layer leaves us a real native callstack instead - # of just surefire's "VM terminated without saying goodbye" line. - # DumpType=2 == MiniDumpWithFullMemory; the workspace dumps/ folder is - # globbed by the failure-upload step below. - shell: pwsh - run: | - $key = 'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\java.exe' - New-Item -Path $key -Force | Out-Null - New-Item -Path "${{ github.workspace }}\dumps" -ItemType Directory -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpFolder' -Value "${{ github.workspace }}\dumps" -PropertyType ExpandString -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpType' -Value 2 -PropertyType DWord -Force | Out-Null - New-ItemProperty -Path $key -Name 'DumpCount' -Value 5 -PropertyType DWord -Force | Out-Null - Get-ItemProperty -Path $key | Format-List - - name: Run tests - run: | - mvn -e --no-transfer-progress -f llama/pom.xml test - # Green tests are not the same as tests that ran. A class-level @BeforeAll assumption that - # fails makes Surefire record tests="0" for that class -- no entries at all, so it is - # invisible to a skip check. That is exactly how every model-gated class stayed silently - # muted on every test-java-* job for months. The floor is deliberately slack; the per-class - # zero check is the sensitive half. - - name: Verify tests actually ran - shell: bash - run: .github/verify-test-counts.sh llama/target/surefire-reports --min-total 1500 - - name: Memory after tests - if: always() - run: Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory,TotalVisibleMemorySize | Format-List - shell: pwsh - # Echo the crash logs into the job log (workspace/policies/ci-test-diagnostics.md 3.1). - - name: Print crash logs (on failure) - if: failure() - shell: bash - run: bash .github/print-crash-logs.sh llama - - if: failure() - uses: actions/upload-artifact@v7 - with: - name: windows-output-msvc - path: | - ${{ github.workspace }}\llama\hs_err_pid*.log - ${{ github.workspace }}\llama\*.hprof - ${{ github.workspace }}\dumps\*.dmp - ${{ github.workspace }}\llama\target\surefire-reports\*.dump - ${{ github.workspace }}\llama\target\surefire-reports\*.dumpstream - ${{ github.workspace }}\llama\target\surefire-reports\*.txt - ${{ github.workspace }}\llama\target\surefire-reports\TEST-*.xml - ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/**/* - if-no-files-found: warn + uses: ./.github/workflows/java-tests.yml + with: + runs-on: windows-2025-vs2026 + natives-artifact: natives-msvc-windows-x86-64 + error-artifact: windows-output-msvc # --------------------------------------------------------------------------- # Package and publish @@ -2749,7 +2495,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Build JARs # `assembly` additionally produces the fat jar-with-dependencies uber JAR # (llama--jar-with-dependencies.jar: library classes + Java runtime deps + @@ -2814,81 +2560,105 @@ jobs: compression-level: 0 # jars are already deflated retention-days: 7 # multi-GB artifact; release jobs consume it within the same run if-no-files-found: error - # Small single-jar artifacts so the smoke jobs don't download the multi-GB set. - - name: Upload Linux smoke jar + # Small single-jar artifacts, llama-fatjar-smoke-, one per all-backends fat jar, so the + # smoke-fatjar matrix does not download the multi-GB set. check-natives.py holds these, the + # matrix rows and the targets derived from natives.csv to the same list. + - name: Upload linux-x86-64 smoke jar uses: actions/upload-artifact@v7 with: - name: llama-fatjar-smoke-linux + name: llama-fatjar-smoke-linux-x86-64 path: fatjars/llama-*-all-linux-x86-64-jar-with-dependencies.jar compression-level: 0 retention-days: 7 if-no-files-found: error - - name: Upload Windows smoke jar + - name: Upload linux-aarch64 smoke jar uses: actions/upload-artifact@v7 with: - name: llama-fatjar-smoke-windows - path: fatjars/llama-*-all-windows-x86-64-jar-with-dependencies.jar + name: llama-fatjar-smoke-linux-aarch64 + path: fatjars/llama-*-all-linux-aarch64-jar-with-dependencies.jar compression-level: 0 retention-days: 7 if-no-files-found: error - # The two aarch64 fat jars are release assets exactly like the x86-64 pair, so they get the - # same treatment: package-fatjars emits four, and for a long time only the two x86-64 ones - # were ever launched. GitHub's free ARM runners (ubuntu-24.04-arm / windows-11-arm) are - # already used by the aarch64 build jobs, so there is no reason to leave these unrun. - - name: Upload Linux aarch64 smoke jar + - name: Upload windows-x86-64 smoke jar uses: actions/upload-artifact@v7 with: - name: llama-fatjar-smoke-linux-aarch64 - path: fatjars/llama-*-all-linux-aarch64-jar-with-dependencies.jar + name: llama-fatjar-smoke-windows-x86-64 + path: fatjars/llama-*-all-windows-x86-64-jar-with-dependencies.jar compression-level: 0 retention-days: 7 if-no-files-found: error - - name: Upload Windows arm64 smoke jar + - name: Upload windows-aarch64 smoke jar uses: actions/upload-artifact@v7 with: - name: llama-fatjar-smoke-windows-arm64 + name: llama-fatjar-smoke-windows-aarch64 path: fatjars/llama-*-all-windows-aarch64-jar-with-dependencies.jar compression-level: 0 retention-days: 7 if-no-files-found: error - # GPU-less runners: every GPU backend must fail its load cleanly (missing vendor - # runtimes) and the server must come up on the CPU backend — this exercises backend - # probing, per-backend extraction, and the fallback chain end-to-end through - # a real `java -jar`, then proves the server serves /health + /v1/chat/completions. - smoke-fatjar-linux: - name: Smoke test all-backends fat jar (Linux) + # Every all-backends fat jar, launched on a GPU-less runner of its OS/arch: every GPU backend + # must fail its load cleanly (missing vendor runtimes) and the server must come up on the CPU + # backend -- backend probing, per-backend extraction and the fallback chain end-to-end through a + # real `java -jar`, then /health + /v1/chat/completions. One row per fat-jar target; + # check-natives.py fails when the rows differ from the targets natives.csv derives, so a new + # all-- jar cannot ship unlaunched -- which is how the two aarch64 jars once did: + # built, signed and attached to every release while nothing ran them. That breaks the rule of + # workspace/policies/fat-jar-release-assets.md ("No release asset is attached that CI has not + # run"), which exists because a corrupt macOS dylib shipped in three releases under a green + # pipeline. fail-fast is off: one platform's failure must not hide another's. + smoke-fatjar: + name: Smoke test all-backends fat jar (${{ matrix.target }}) needs: [package-fatjars, verify-model-cache] - runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + include: + - { target: linux-x86-64, runner: ubuntu-latest } + - { target: linux-aarch64, runner: ubuntu-24.04-arm } + - { target: windows-x86-64, runner: windows-2025-vs2026 } + - { target: windows-aarch64, runner: windows-11-arm } + runs-on: ${{ matrix.runner }} + env: + JAR_GLOB: llama-*-all-${{ matrix.target }}-jar-with-dependencies.jar steps: - uses: actions/checkout@v7 + with: + persist-credentials: false - uses: actions/download-artifact@v8 with: - name: llama-fatjar-smoke-linux + name: llama-fatjar-smoke-${{ matrix.target }} path: fatjars/ - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - # Same floor, re-checked on the ASSEMBLED release asset rather than the module - # jars the `package` job verified: package-fatjars rewrites the zip (drops the other - # platforms, adds the backend trees), and this is the artifact users download. + java-version-file: .java-version + # The Java 8 floor, re-checked on the ASSEMBLED release asset rather than the module jars the + # `package` job verified: package-fatjars rewrites the zip per OS/arch (drops the other + # platforms, adds the backend trees), and this is the artifact users download. The script is + # bash-only, so the Linux rows check; the Windows jars are assembled from the same classes. - name: Verify Java 8 bytecode (no class newer than major 52) + if: runner.os == 'Linux' run: .github/verify-bytecode-version.sh --max-major 52 fatjars - name: Run fat-jar server smoke test - run: .github/smoke-test-fatjar.sh fatjars 'llama-*-all-linux-x86-64-jar-with-dependencies.jar' "models/${DRAFT_MODEL_NAME}" + if: runner.os != 'Windows' + run: .github/smoke-test-fatjar.sh fatjars "$JAR_GLOB" "models/${DRAFT_MODEL_NAME}" + - name: Run fat-jar server smoke test + if: runner.os == 'Windows' + shell: pwsh + run: .github/smoke-test-fatjar.ps1 -JarDir fatjars -JarGlob $env:JAR_GLOB -Model "models/$env:DRAFT_MODEL_NAME" # RPC over two JVMs from the same release asset: RpcServer in one, the default NativeServer # with --rpc in the other. Requires a chat completion, the model buffer on the RPC endpoint in # the load log (the layers really went over RPC), an accepted client on the server, and a clean # non-SIGABRT failure naming the endpoint when --rpc names a server nobody runs. - name: Run fat-jar RPC smoke test (two JVMs) - run: .github/smoke-rpc-fatjar.sh fatjars 'llama-*-all-linux-x86-64-jar-with-dependencies.jar' "models/${DRAFT_MODEL_NAME}" + if: matrix.target == 'linux-x86-64' + run: .github/smoke-rpc-fatjar.sh fatjars "$JAR_GLOB" "models/${DRAFT_MODEL_NAME}" - name: Upload server logs if: failure() uses: actions/upload-artifact@v7 with: - name: fatjar-smoke-linux-logs + name: fatjar-smoke-${{ matrix.target }}-logs path: | server-out.log server-err.log @@ -2915,13 +2685,13 @@ jobs: path: agent-assets/ - uses: actions/download-artifact@v8 with: - name: llama-fatjar-smoke-linux + name: llama-fatjar-smoke-linux-x86-64 path: agent-assets/ - uses: ./.github/actions/restore-models - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version # The agent is Java 21 (Atmosphere's floor), unlike the Java 8 core: its ceiling is 65. # One waiver: JLine ships its FFM terminal provider (org/jline/terminal/impl/ffm, 25 classes) # as Java 22 bytecode. It is discovered through META-INF/jline/providers/ffm, not loaded @@ -2943,103 +2713,6 @@ jobs: path: agent-*.log if-no-files-found: warn - smoke-fatjar-windows: - name: Smoke test all-backends fat jar (Windows) - needs: [package-fatjars, verify-model-cache] - runs-on: windows-2025-vs2026 - steps: - - uses: actions/checkout@v7 - - uses: actions/download-artifact@v8 - with: - name: llama-fatjar-smoke-windows - path: fatjars/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Run fat-jar server smoke test - shell: pwsh - run: .github/smoke-test-fatjar.ps1 -JarDir fatjars -JarGlob 'llama-*-all-windows-x86-64-jar-with-dependencies.jar' -Model "models/$env:DRAFT_MODEL_NAME" - - name: Upload server logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: fatjar-smoke-windows-logs - path: | - server-out.log - server-err.log - if-no-files-found: warn - - # The aarch64 halves of the same convention. Identical in shape to the two jobs above — the only - # differences are the runner and the jar glob — and they exist because `all-linux-aarch64` and - # `all-windows-aarch64` were built, GPG-signed and attached to every release without CI ever - # launching them, which is exactly what workspace/policies/fat-jar-release-assets.md forbids - # ("No release asset is attached that CI has not run"). That rule exists because a corrupt macOS - # dylib shipped in three releases under a fully green pipeline; these two jars were the remaining - # assets in the same blind spot. - smoke-fatjar-linux-aarch64: - name: Smoke test all-backends fat jar (Linux aarch64) - needs: [package-fatjars, verify-model-cache] - runs-on: ubuntu-24.04-arm - steps: - - uses: actions/checkout@v7 - - uses: actions/download-artifact@v8 - with: - name: llama-fatjar-smoke-linux-aarch64 - path: fatjars/ - - uses: ./.github/actions/restore-models - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - # Same floor, re-checked on this assembled release asset: package-fatjars rewrites the zip - # per OS/arch, so the aarch64 jar is a different artifact from the one smoke-fatjar-linux - # verifies even though the classes are identical. - - name: Verify Java 8 bytecode (no class newer than major 52) - run: .github/verify-bytecode-version.sh --max-major 52 fatjars - - name: Run fat-jar server smoke test - run: .github/smoke-test-fatjar.sh fatjars 'llama-*-all-linux-aarch64-jar-with-dependencies.jar' "models/${DRAFT_MODEL_NAME}" - - name: Upload server logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: fatjar-smoke-linux-aarch64-logs - path: | - server-out.log - server-err.log - if-no-files-found: warn - - smoke-fatjar-windows-arm64: - name: Smoke test all-backends fat jar (Windows arm64) - needs: [package-fatjars, verify-model-cache] - runs-on: windows-11-arm - steps: - - uses: actions/checkout@v7 - - uses: actions/download-artifact@v8 - with: - name: llama-fatjar-smoke-windows-arm64 - path: fatjars/ - - uses: ./.github/actions/restore-models - # temurin publishes a Windows/AArch64 JDK for this java-version; the build-windows-arm64 - # job already resolves it on this same runner. - - uses: actions/setup-java@v6 - with: - distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} - - name: Run fat-jar server smoke test - shell: pwsh - run: .github/smoke-test-fatjar.ps1 -JarDir fatjars -JarGlob 'llama-*-all-windows-aarch64-jar-with-dependencies.jar' -Model "models/$env:DRAFT_MODEL_NAME" - - name: Upload server logs - if: failure() - uses: actions/upload-artifact@v7 - with: - name: fatjar-smoke-windows-arm64-logs - path: | - server-out.log - server-err.log - if-no-files-found: warn - # --------------------------------------------------------------------------- # macOS member of the cross-repo "no release asset is attached that CI has not run" convention # (workspace/policies/fat-jar-release-assets.md; BitcoinAddressFinder and srcmorph run the shared @@ -3070,7 +2743,7 @@ jobs: - uses: actions/setup-java@v6 with: distribution: 'temurin' - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version - name: Run packaged-native smoke test run: .github/smoke-native-macos.sh jars 'llama-*-jar-with-dependencies.jar' @@ -3083,7 +2756,7 @@ jobs: steps: - uses: actions/checkout@v7 - uses: actions/setup-java@v6 - with: { java-version: '${{ env.JAVA_VERSION }}', distribution: temurin } + with: { java-version-file: .java-version, distribution: temurin } - uses: actions/download-artifact@v8 with: { name: jacoco-report, path: target/site/jacoco/ } continue-on-error: true @@ -3128,7 +2801,7 @@ jobs: publish-snapshot: name: Publish Snapshot to Central - needs: [check-snapshot, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] + needs: [check-snapshot, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] if: needs.check-snapshot.result == 'success' && inputs.publish_to_central runs-on: ubuntu-latest environment: maven-central @@ -3155,7 +2828,7 @@ jobs: - name: Set up Maven Central Repository uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: 'temurin' server-id: central server-username: MAVEN_USERNAME @@ -3325,7 +2998,7 @@ jobs: publish-release: name: Publish Release to Central if: needs.check-tag.result == 'success' && inputs.publish_to_central - needs: [check-tag, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar-linux, smoke-fatjar-windows, smoke-fatjar-linux-aarch64, smoke-fatjar-windows-arm64, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] + needs: [check-tag, crosscompile-linux-x86_64-cuda, crosscompile-android-aarch64-opencl, package-android-aar, test-android-emulator, code-style, test-java-llama-langchain4j, test-java-llama-kotlin, test-java-llama-atmosphere-agent, test-java-llama-atmosphere-agent-integration, package-fatjars, smoke-fatjar, smoke-fatjar-macos, smoke-agent-linux, vmlens, shared-files] runs-on: ubuntu-latest environment: maven-central permissions: @@ -3351,7 +3024,7 @@ jobs: - name: Set up Maven Central Repository uses: actions/setup-java@v6 with: - java-version: ${{ env.JAVA_VERSION }} + java-version-file: .java-version distribution: 'temurin' server-id: central server-username: MAVEN_USERNAME diff --git a/.github/workflows/sonarqube.yml b/.github/workflows/sonarqube.yml index ee1eb8f4a..6d7864970 100644 --- a/.github/workflows/sonarqube.yml +++ b/.github/workflows/sonarqube.yml @@ -20,7 +20,7 @@ jobs: - name: Set up JDK 21 uses: actions/setup-java@v6 with: - java-version: 21 + java-version-file: .java-version distribution: 'zulu' - name: Cache SonarQube packages uses: actions/cache@v6 diff --git a/.java-version b/.java-version new file mode 100644 index 000000000..aabe6ec39 --- /dev/null +++ b/.java-version @@ -0,0 +1 @@ +21 diff --git a/.java-version.license b/.java-version.license new file mode 100644 index 000000000..b918686f7 --- /dev/null +++ b/.java-version.license @@ -0,0 +1,3 @@ +SPDX-FileCopyrightText: 2026 Bernard Ladenthin + +SPDX-License-Identifier: MIT diff --git a/CHANGELOG.md b/CHANGELOG.md index dc3023517..76f269501 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,12 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by library are now such shared scripts). The workflow lost ~760 lines to two composite actions (`restore-models`, `install-sccache-windows`) and `print-host-info.sh`; `validate-models.bat` is gone (Windows runs the bash script). +- **Fewer copies of the same job in the workflow.** The four fat-jar smoke jobs are one `smoke-fatjar` + matrix (its rows checked against the targets `natives.csv` derives; the per-jar artifacts are now + `llama-fatjar-smoke-`), the macOS and Windows Java test jobs call one reusable workflow + (`.github/workflows/java-tests.yml`), and the JDK version is `.java-version`, read by every + `setup-java` step. Check names of those jobs changed (` / Java tests`, `Smoke test all-backends + fat jar ()`); no required status check referred to them. ### Fixed - **`ToolCallingIntegrationTest#requiredToolCallIsParsedFromStreamingResponse` failed on both Windows diff --git a/CLAUDE.md b/CLAUDE.md index 4e6ee2f85..1990dbfde 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -121,13 +121,29 @@ srcmorph), the signing self-test, `lombok.config` and the issue/PR templates. **Composite actions (`.github/actions/`, this repo only).** `restore-models` (restore the shared GGUF cache + `validate-models.sh`; the one place a job gets its models — 16 call sites) and -`install-sccache-windows` (10 call sites; the caller keeps `if:` and `continue-on-error`). Composite, -not reusable workflows: they replace steps without changing any job or check name. - -**Considered and not done** (measured after the above): a reusable workflow for the Java test jobs -(the three macOS jobs are 53 lines each with 7 differing, the two Windows jobs 77) and a matrix for -the four fat-jar smokes (32–47 lines each). Both would save about a hundred lines and rename every -affected check, which means updating branch protection — not worth it at this size. +`install-sccache-windows` (10 call sites; the caller keeps `if:` and `continue-on-error`). Composite +actions for steps; for whole jobs, the two shapes below. + +**Reusable workflow for the macOS/Windows Java tests (`.github/workflows/java-tests.yml`).** The +three macOS arm64 test jobs and the two Windows x86-64 ones were five copies of one job differing in +the runner, the natives artifact, one Maven flag and the failure-artifact name. They are now five +**callers** (`uses: ./.github/workflows/java-tests.yml` + four `with:` inputs) that keep their ids, +names and `needs`, so `package`, the release gate and `check-natives.py` see the same graph; the +OS-specific steps (memory report, core dumps vs. WER minidumps) are chosen inside by `runner.os`. +Two constraints shaped it: a called workflow sees **none** of the caller's `env:`, and `with:` cannot +read `env` — which is why the JDK version moved to **`.java-version`** (`setup-java`'s +`java-version-file`, used by every job and by `codeql.yml`/`sonarqube.yml`, so one file names it) +and why the callee needs no model names (the tests default to the CI model set). The Linux x86-64 +test job stays inline: it differs in far more than inputs. + +**Matrix for the fat-jar smokes (`smoke-fatjar`).** See "All-backends server fat jars" below. A +matrix and not a reusable workflow, because the rows differ only in data (target, runner). + +**Check names changed with both** — a called job shows as ` / Java tests`, a matrix row +as `Smoke test all-backends fat jar ()`. That was measured, not assumed, before doing it: +none of the four repositories has a required status check (each has only a ruleset — no deletion, +no force push, PR required — and classic branch protection is off), so no setting pins a check +name. Should required checks ever be added, name these new forms. ## Upgrading CUDA Version @@ -487,7 +503,7 @@ corrupt binary. **The end-to-end gate: `smoke-fatjar-macos`.** The three macOS Java test jobs each test the dylib *their own job* built, so until now nothing exercised the **packaged** artifact on macOS — Linux and -Windows had `smoke-fatjar-*` downstream of `package`, macOS had none, which is why this bug reached +Windows had fat-jar smoke jobs downstream of `package`, macOS had none, which is why this bug reached three releases with a fully green pipeline. The job (`needs: [package]`, `macos-15`, gates both publish jobs) downloads `llama-jars` and runs `.github/smoke-native-macos.sh`, which extracts `Mac/aarch64/metal/libjllama.dylib` from the **default fat jar** and asserts two things: `codesign --verify @@ -553,21 +569,24 @@ Mechanism: A load failure (missing vendor runtime → `UnsatisfiedLinkError`) moves to the next backend, ending at `cpu`. `net.ladenthin.llama.backend` forces one backend (fail-loud). A backend whose extra module is already resident from a previously failed attempt is skipped (by-name import cross-wiring). -3. **`publish.yml` wiring** — **all four** OS/arch fat jars are launched, one smoke job each: - `smoke-fatjar-linux` / `smoke-fatjar-windows` (x86-64) plus `smoke-fatjar-linux-aarch64` - (`ubuntu-24.04-arm`) / `smoke-fatjar-windows-arm64` (`windows-11-arm`). Each runs its jar via - real `java -jar` on a GPU-less runner (cached draft model, `--chat-template chatml`): poll - `/health` to 200, assert a `/v1/chat/completions` choice, and require the loader's - backend-selection log line — so every GPU backend failing its load and falling back to the - CPU backend is exercised on the actual release asset. The four jobs consume four small - single-jar artifacts (`llama-fatjar-smoke-{linux,windows,linux-aarch64,windows-arm64}`) rather - than the multi-GB `llama-fatjars` set. **The two aarch64 jobs close a real gap**: those jars were - built, GPG-signed and attached to every release while `publish.yml` referenced them zero times, - which is exactly what the cross-repo rule forbids — and that rule exists because a corrupt macOS - dylib shipped in three releases under a fully green pipeline. `publish-snapshot`/`publish-release` - `need` `package-fatjars` + **all four** smokes (fail-loud gating); `github-release-signed` and - `github-snapshot` additionally download `llama-fatjars` into their asset directory, then - **GPG-sign each fat jar** via `.github/sign-fatjars.sh` (a detached `.asc` alongside the +3. **`publish.yml` wiring** — **every** OS/arch fat jar is launched by one row of the + **`smoke-fatjar` matrix** (`fail-fast: false`; rows `linux-x86-64` on `ubuntu-latest`, + `linux-aarch64` on `ubuntu-24.04-arm`, `windows-x86-64` on `windows-2025-vs2026`, + `windows-aarch64` on `windows-11-arm`). Each row runs its jar via real `java -jar` on a + GPU-less runner (cached draft model, `--chat-template chatml`): poll `/health` to 200, assert a + `/v1/chat/completions` choice, and require the loader's backend-selection log line — so every + GPU backend failing its load and falling back to the CPU backend is exercised on the actual + release asset. Each row downloads one small single-jar artifact, `llama-fatjar-smoke-`, + rather than the multi-GB `llama-fatjars` set; the Linux rows also re-run the bytecode gate and + the `linux-x86-64` row the RPC smoke. `check-natives.py` holds the uploads (name **and** the jar + inside) and the matrix rows to the targets `natives.csv` derives, so a new fat jar cannot ship + unlaunched — **the aarch64 rows close a real gap**: those jars were built, GPG-signed and + attached to every release while `publish.yml` referenced them zero times, which is exactly what + the cross-repo rule forbids — and that rule exists because a corrupt macOS dylib shipped in + three releases under a fully green pipeline. `publish-snapshot`/`publish-release` `need` + `package-fatjars` + `smoke-fatjar` (a matrix job is done when every row is; fail-loud gating); + `github-release-signed` and `github-snapshot` additionally download `llama-fatjars` into their + asset directory, then **GPG-sign each fat jar** via `.github/sign-fatjars.sh` (a detached `.asc` alongside the `.sha256`), so the fat jars land signed on the tag release and the rolling `snapshot` pre-release. Both jobs declare `environment: maven-central` (where the signing key secret is scoped; it has no approval gate) and `checkout` the repo so the script is present. Signing @@ -1088,7 +1107,7 @@ drives start/stop, bind failure, start timeout, interrupt and the single-instanc `RpcServerTest` (native, model-free: lifecycle, restart on the same port, single instance, bind failure, unreachable `--rpc` load), `RpcIntegrationTest` (draft model: layers on the RPC server proven from the load log's `model buffer size` line naming the endpoint, then the stale-server -load). CI: `.github/smoke-rpc-fatjar.sh` in `smoke-fatjar-linux` runs **two JVMs from the release +load). CI: `.github/smoke-rpc-fatjar.sh` in the `linux-x86-64` row of `smoke-fatjar` runs **two JVMs from the release asset** — `RpcServer` in one, the default `NativeServer` with `--rpc` in the other — and requires a chat completion, the RPC model buffer in the log, an accepted client on the server, and a clean non-SIGABRT failure naming the endpoint for a server nobody runs. @@ -2101,7 +2120,7 @@ why a `release 9` `module-info` is fine. `--allow` is a repeatable glob matched this check reported a clean pass over a directory a failed build had left empty). It runs twice: in the `package` job over `llama/target` (every jar plus the default fat -jar, as early as they exist), and again in `smoke-fatjar-linux` over the downloaded `fatjars/` — +jar, as early as they exist), and again in the two Linux rows of `smoke-fatjar` over the downloaded fat jar — `package-fatjars` rewrites those zips, and they are the artifacts users actually download. **Surefire AND PIT both exclude `org.slf4j:slf4j-simple` from the test classpath** @@ -3318,7 +3337,7 @@ Structure (mirrors the consumer-test's plumbing): after upgrading Gradle to 9.x with AGP 9.2.0) that hits this project directly since `buildTypes.release` sets `isMinifyEnabled = true`; `9.4.0` carries that fix forward and the `.github/android-consumer-test` fixture is pinned the same way. AGP 9.4.x requires Gradle >= 9.6.0 - and JDK 17+; CI already runs JDK 21 everywhere (`env.JAVA_VERSION`), so only the `gradle-version` + and JDK 17+; CI already runs JDK 21 everywhere (`.java-version`), so only the `gradle-version` pin on the jobs that build this project (and the `.github/android-consumer-test` fixture) needed bumping (currently `9.8.0`). AGP 9.0+ has **built-in Kotlin support** (a runtime dependency on Kotlin Gradle plugin 2.2.10+), so the standalone `org.jetbrains.kotlin.android` plugin is no longer diff --git a/TODO.md b/TODO.md index dd33cbbbd..427b9180e 100644 --- a/TODO.md +++ b/TODO.md @@ -52,8 +52,9 @@ so everything below is genuinely still open. release too), but the app needs `android.permission.INTERNET` even for loopback, which the AAR deliberately does not declare and the emulator fixture does not have. A loopback test on the emulator needs that permission in the fixture's manifest only. -- **RPC smoke on the other fat-jar platforms.** `smoke-rpc-fatjar.sh` runs in `smoke-fatjar-linux` - only; the Java `RpcServerTest`/`RpcIntegrationTest` already run on every `test-java-*` job +- **RPC smoke on the other fat-jar platforms.** `smoke-rpc-fatjar.sh` runs in the `linux-x86-64` row + of the `smoke-fatjar` matrix only (widening it to `linux-aarch64` is a change of that step's `if:`; + Windows needs a PowerShell port of the script); the Java `RpcServerTest`/`RpcIntegrationTest` already run on every `test-java-*` job (Windows and macOS included), so this is about the packaged asset, not the code path. - **Authentication / TLS.** Upstream has none; the documented answer is a trusted network or a tunnel. Only worth doing if it lands upstream. From 818bef55da430e4ec57b361484b1d16325a3de3c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 06:31:01 +0000 Subject: [PATCH 04/22] Split CLAUDE.md: agent and app guidance next to their code, dropped patches to history The root CLAUDE.md is loaded into every session and had grown to 312 KB. The Atmosphere agent section (83 KB) moves to llama-atmosphere-agent/CLAUDE.md and the Android app section to android-llmservice/CLAUDE.md; Claude Code loads those only when working in that directory. The root keeps a short summary of what each means for the rest of the repository (release asset, CI gates, version bump, spec of record). The notes on the six dropped llama.cpp patches move to docs/history/dropped-llama-patches.md; the root keeps the standing drop-check and, in the s390x section, the one rule that outlived 0013 (no -DGGML_VXE=ON). Root CLAUDE.md: 312 KB -> 213 KB. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/workflows/publish.yml | 4 +- CLAUDE.md | 1147 +---------------- REUSE.toml | 2 + TODO.md | 2 +- android-llmservice/CLAUDE.md | 114 ++ docs/history/dropped-llama-patches.md | 128 ++ ...vestigation-jline-status-windows-redraw.md | 4 +- llama-atmosphere-agent/CLAUDE.md | 895 +++++++++++++ 8 files changed, 1179 insertions(+), 1117 deletions(-) create mode 100644 android-llmservice/CLAUDE.md create mode 100644 docs/history/dropped-llama-patches.md create mode 100644 llama-atmosphere-agent/CLAUDE.md diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 1c34d0915..9652a7170 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -24,7 +24,7 @@ env: # reusable workflows read the same one without an input. # The Gradle builds (Android AAR, the consumer fixture, the llmservice app, the signing self-test). - # AGP 9.4.x needs Gradle >= 9.6.0 (CLAUDE.md, Android app). Not seen by Dependabot: bump by hand. + # AGP 9.4.x needs Gradle >= 9.6.0 (android-llmservice/CLAUDE.md). Not seen by Dependabot: bump by hand. GRADLE_VERSION: '9.8.0' # The sccache/Depot compiler cache (CLAUDE.md, "CI build cache"). Inert without the token, which # stays per build job (SCCACHE_WEBDAV_TOKEN): at workflow level it would reach every job, @@ -784,7 +784,7 @@ jobs: # exactly right for what it is (a big-endian correctness gate for our own layer, not a perf target). # Do NOT "fix" a VXE-related compile error by adding -DGGML_VXE=ON: that define sets __VXE__ and # __VXE2__ together while -march stays at the toolchain default arch11, so every z14+ builtin is - # rejected. See the "`0013` was dropped at the b10948 bump" note in CLAUDE.md for the measured + # rejected. See the `0013` note in docs/history/dropped-llama-patches.md for the measured # comparison (the patch itself is gone -- upstream merged it as ggml-org/llama.cpp#28775). - name: Build libraries (cross-compile s390x) shell: bash diff --git a/CLAUDE.md b/CLAUDE.md index 1990dbfde..ce8a65854 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -895,121 +895,12 @@ Current patches: | `0014-common-log-callback-sink.patch` | **Gives `common_log` a callback sink: `common_log_set_callback(log, cb, user_data)` (`common/log.{h,cpp}`).** This is what `LlamaModel.setLogger` hooks. Before it, the Java logger was a `llama_log_set()` callback, which has two holes, both found while chasing `slot print_timing` lines interleaving with the Atmosphere agent's streamed answer: **(a)** every model load runs `common_init()`, which re-points `llama_log_set()` at `common_log_default_callback` (`common.cpp:394`), so `setLogger(…)` *before* `new LlamaModel(…)` silently lost the callback; **(b)** the server's own `SRV_*`/`SLT_*` macros are `LOG_INF` and write straight into `common_log`, which `llama_log_set()` never carried, so the per-request `slot …`/`srv …` lines could not be routed to Java at all (the reason `LlamaModelTest#testLogText/JSON` sat `@Disabled` for years). `common_log` upstream offers file, colors, prefix, timestamps, verbosity and JSONL but no hook. The patch adds one: while a callback is set, the worker thread hands every entry to it **instead of** printing to stdout/stderr (a `--log-file` still receives them); the callback gets the bare formatted message (no prefix/timestamp/colors) with the `ggml_log_callback` signature; swapping the callback pauses the worker first, so queued entries reach the *previous* sink (which is what makes `setLogger(format, null)` a synchronous drain). With the sink, `common_init()`'s `llama_log_set()` reset is harmless — it points at the default callback that feeds `common_log`, i.e. exactly the path into the sink — so the ordering problem (a) disappears without any re-install logic, and the `srv`/`slot` lines (b) arrive because they are `common_log` entries. `jllama.cpp`'s `setLogger` therefore sets `common_log_set_callback(common_log_main(), trampoline)` **plus** `llama_log_set(common_log_default_callback)` (so llama/ggml lines feed `common_log` even before the first load). Two consequences to know: the callback runs on `common_log`'s **worker thread**, a plain `std::thread` llama.cpp re-creates on every pause/resume and never attaches to the JVM — the trampoline attaches per call and detaches again (`get_jni_env_attaching`; a thread that exits while attached leaks a `JavaThread`, and this thread is not ours — the leak-free choice, not the cheapest: each attach creates a `java.lang.Thread` object, so a `thread_local` guard that detaches once at thread exit is the optimisation on file in `TODO.md`), `setLogger` must call `common_log_set_callback` **outside** `g_log_mutex`, because the pause joins the worker, which needs that mutex to read the callback, and `setLogger` callers are serialized by a **separate** `g_set_logger_mutex`: two unserialized swaps race on the worker's `std::thread` (one joins it while the other assigns a fresh thread over the still-joinable object = `std::terminate`, the whole JVM), which `LlamaLoggerTest#concurrentSetLoggerCallsDoNotRaceOnTheLogWorker` reproduced before the mutex existed. Two caveats the Javadoc carries: a caller must not hold a lock the *previous* callback needs (the drain runs it on the worker while the caller waits), and the verbosity threshold is process-wide and reset by **every** load (`common_params_parse` ends with `common_log_set_verbosity_thold(params.verbosity)`, default 3), so a load without `-lv` puts it back to 3 — a review assumed the opposite, and `LlamaLoggerTest#verbosityThresholdIsProcessWideAndEveryLoadSetsIt` now pins the measured behaviour. And the verbosity threshold applies *before* the sink: at the default (`3`) the Java logger sees errors, warnings and the server's INFO lines, while llama/ggml INFO lines (`common_log_get_verbosity` maps them to TRACE = 4) arrive only from `setLogVerbosity(4)` on — the same filtering the console gets, and a behaviour change for consumers who captured the unfiltered `llama_log_set()` stream before. **Runnable guards:** `src/test/cpp/test_common_log_callback.cpp` (6 tests over a private `common_log_init()` instance: delivery, bare text under prefix+timestamps, clear, swap-drains-to-old-sink, file kept, levels pass through) links the function on every platform, so a bump that drops the patch reds `C++ Tests` at link time; `LlamaLoggerTest` (model-free, needs only `libjllama`: a logger set before a deliberately failing load on a non-GGUF file sees the `srv … loading model` INFO line and llama's ERROR line, in TEXT and JSON) and the re-enabled `LlamaModelTest#testLogText/testLogJSON` plus `#testLoggerSetBeforeLoadSurvivesTheLoad` (vocab-only load) cover the Java side. Upstream-submittable ("common : add a callback sink to common_log for embedding hosts"); **not yet filed upstream**. Touches only `common/log.{h,cpp}`, which no other patch touches. **On a bump, check whether upstream added a hook of its own (grep `callback` in `common/log.h`) and, if so, DROP this patch and port `setLogger` to theirs rather than refreshing it.** | | `0015-rpc-embeddable-client-and-stoppable-server.patch` | **Makes ggml-rpc usable inside a JVM** (see "RPC backend" below). Upstream treats every client-side RPC problem as `GGML_ABORT` — a malformed endpoint, a server that is not running, a failed handshake — which in a JVM kills the application over a typo in `--rpc`. **(1)** The *registration* path reports failure instead: `rpc_dispatcher::start()` returns `false`, `try_get_dispatcher()` `nullptr`, `ggml_backend_rpc_get_device_count()` `0`, `ggml_backend_rpc_add_server()` `nullptr`, and `add_rpc_devices()` (`common/arg.cpp`) throws `std::invalid_argument` naming the server where it used to register `nullptr` and silently run without it. Every path *after* registration keeps the original contract through `get_dispatcher()`, which still aborts — a server that vanishes mid-inference stays fatal (TODO). **(2)** `add_server()`'s cache hit is re-checked (a stopped server used to pass registration and abort on the first tensor upload), and `ggml_backend_rpc_get_device_memory()` of a gone server returns 0/0 instead of aborting — RPC devices stay in ggml's process-wide registry forever, and `common_init()` queries the memory of **every** registered device at `-lv 4`, so a stopped server would otherwise take down an unrelated later load. **(3)** The server becomes stoppable: `ggml_backend_rpc_stop_server()` (disconnects the client being served, wakes `accept()` with a loopback connection — the one portable way) and `ggml_backend_rpc_server_listening()`; the loop now reaches its cleanup and frees its backends on every early return, but deliberately **not** `rpc_transport_shutdown()`, whose `WSACleanup()` would invalidate every other RPC socket in the process. **(4)** transport: `MSG_NOSIGNAL`/`SO_NOSIGPIPE`, `socket_t::shutdown()`, and the fds `connect()`/`create_server()`/`accept()` leaked on their error paths are closed. Touches `ggml/include/ggml-rpc.h`, `ggml/src/ggml-rpc/{ggml-rpc.cpp,transport.cpp,transport.h}` and one function in `common/arg.cpp` (`add_rpc_devices`, away from `0001`'s `common_params_parse` hunks). **Runnable guard: `src/test/cpp/test_rpc.cpp`** — links both new functions (a bump that drops the patch fails `C++ Tests` at link time on every platform) and drives registration, stop and the memory query over loopback. Upstream-submittable; **not yet filed upstream**. On a bump, check whether upstream grew its own stop function or non-aborting registration (grep `stop_server` / `GGML_ABORT("Failed to connect` in `ggml-rpc.cpp`). | -**`0010` was dropped at the b11080 bump.** Upstream merged -[ggml-org/llama.cpp#28518](https://github.com/ggml-org/llama.cpp/pull/28518) -("json: Fixed json enum handling", first tagged at **b11080**), which fixes the defect at its -root instead of at the emit site: `common_json_value` gains an `std::is_enum`-gated constructor -that delegates to `std::underlying_type`, and `common_json_is_value` now accepts enums. So an -unscoped enum no longer binds to `common_json_value(bool)` anywhere, and the one-line `(int)` -cast `0010` added to upstream's `get_res_model_info()` became a **redundant carry**. - -**This is the drop-check firing, and it is the case that makes the check worth running by hand.** -`0010` still applied cleanly at b11080 — upstream never touched the emit site — so the fail-loud -applier said nothing, exactly as `CLAUDE.md` warned it could not ("the applier detects 'does not -apply', never 'upstream already fixed this'"). The standing check is phrased as *"did upstream -cast the value themselves?"*; the honest reading is *"is the defect still there?"*, and it was -not. Dropped, not refreshed, per the `0009`/`0011`/`0013` precedent. - -**The runnable guard stays, re-pointed** (the `0011` precedent again): the `CommonJsonEnumTrap` -pair in `src/test/cpp/test_json_helpers.cpp` is now the `CommonJsonEnum` trio, and it pins -upstream's contract — an **uncast** enum serialises as its numeric value, an explicit -`static_cast` is equivalent, and a real `bool` is still a boolean (the new constructor sits -next to `common_json_value(bool)`, so that one is worth pinning too). A future bump that loses -the enum constructor reds `C++ Tests` on every platform, and the response is to reinstate both -the cast and the patch. `jllama.cpp` keeps the explicit casts at its own two `"vocab_type"` -sites: they are correct either way and survive such a revert. - -`.github/verify-patches-applied.sh` lost its third check with the patch — `0010` was the only -patch with no runnable guard, which is what that check existed for. The script keeps its two -generic assertions (every patch on disk is in the stamp; the patched tree is dirty), so it still -catches an applier that never ran or a tree reverted after the fact. - -**`0011` was dropped at the b11069 bump.** Upstream merged -[ggml-org/llama.cpp#29161](https://github.com/ggml-org/llama.cpp/pull/29161) -("common/peg : handle invalid utf-8 sequences in the AST", commit `3d82ef62d`, first tagged at -**b11063**) — an independent and broader fix for the same defect; the patch itself had never been -filed upstream. Where `0011` made the until-parser's `INVALID` branch honour `ctx.is_lenient()` by -returning what was scanned *before* the bad byte (so the text after it was lost), upstream now -**consumes** every undecodable run, in strict mode too, records its `{pos, len}` on the AST node -(`common_peg_invalid_utf8`, carried up through `common_peg_parse_result`), and -`common_chat_peg_mapper` emits `node.sanitized_text()`, which substitutes exactly one U+FFFD per run -(the Unicode "maximal subpart" rule: `\xE4\xB8` followed by `c` is one run, `\xFF\xFE` is two). -`common/unicode.cpp` now reports the valid-prefix length in `bytes_consumed` for `INVALID` / -`INCOMPLETE` results to make that possible. The lenient incomplete-at-end-of-input branch (withhold -the trailing bytes, more may arrive on a stream) is unchanged. The applier failed loud with "does not -apply" at `common/peg-parser.cpp:680` exactly as designed — the `INVALID` branch it patched no longer -exists. Dropped, not refreshed, per the `0009`/`0013` precedent. **The runnable guard stays, -re-pointed:** the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` now pin upstream's -replacement contract (invalid byte → U+FFFD with the text after it kept; a trailing incomplete -sequence still withheld), so a future upstream revert to `FAIL`, or a change to the replacement rule, -still reds `C++ Tests` on every platform. The strict-mode `FAIL` on invalid UTF-8 that `0011` -deliberately preserved is gone upstream as well — `tests/peg-parser/test-unicode.cpp`'s -`malformed UTF-8` block now asserts `SUCCESS` plus the sanitized text. **Behavioural note for -consumers:** a completion containing a stray byte now returns the *full* text with U+FFFD in place of -the byte, where the patched builds returned the text up to the byte. - -**`0013` was dropped at the b10948 bump.** Upstream merged this project's own PR -[ggml-org/llama.cpp#28775](https://github.com/ggml-org/llama.cpp/pull/28775) -("ggml-cpu(s390x): guard VXE-only repack helpers", commit `6978052`, first tagged at **b10948**): -`ggml/src/ggml-cpu/arch/s390/repack.cpp` now wraps `vxe_dot_acc` / `vxe_splat_granule` / `vxe_fold` -in the same `#if defined(__VXE__) || defined(__VXE2__)` guard the patch added — byte-identical apart -from the trailing `// __VXE__ || __VXE2__` comment the patch put on the `#endif`. So the non-VXE -s390x compile break the patch fixed (three `does not name a type` errors before any skipped code is -reached) no longer exists upstream, and the applier failed loud with "does not apply cleanly" at -configure time exactly as designed — the guard it wants to add is already there. Dropped, not -refreshed, per the `0009` precedent. - -**The s390x configuration this leaves in place is still ours to know, because it is what made the -patch necessary and it did not change.** `ggml/CMakeLists.txt` declares -`option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})`, so the `build-linux-s390x` job's -`-DGGML_NATIVE=OFF` (correct for a cross-build — an x86 host must not bake `-march=native` into an -s390x artifact) silently switches VXE off too: no `-mvx -mzvector`, `__VEC__` undefined, and -`ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__` -never fires. The job therefore ships a **scalar** s390x binary, which is right for what it is (a -big-endian *correctness* gate for this project's own layer, not a performance target). **Do not -"fix" a future VXE-related compile error with `-DGGML_VXE=ON`** — measured, not assumed: that -self-define sets `__VXE__` *and* `__VXE2__` together while `-march` stays at the toolchain default -`arch11`, so a handful of errors becomes dozens of `'__builtin_s390_vec_*' matching variant requires -z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which -compiles but raises the artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on -VXE2 emulation. If a comparable break resurfaces, re-check upstream's guard against this description -before reintroducing a local patch. - -**`0009` was dropped at the b10280 bump.** Upstream merged -[sheredom/subprocess.h#104](https://github.com/sheredom/subprocess.h/pull/104) — the exact fix this -patch submitted — via [ggml-org/llama.cpp#26606](https://github.com/ggml-org/llama.cpp/pull/26606) -("vendor: apply patches for subprocess.h"): `vendor/sheredom/subprocess.h` at b10280 already defines -the `SUBPROCESS_HAVE_CWD` compile-time probe and the `ENOSYS` fallback our patch added, so the -old-glibc build break `0009` fixed no longer exists upstream. The applier's idempotent -`git apply --reverse --check` could **not** auto-skip this one (unlike a byte-identical carry): the -same PR also rewrote the neighboring Windows argv-quoting logic and added a `__NetBSD__` branch next -to the `process_cwd` guard, shifting the patch's second hunk's context, so it failed loud with "does -not apply cleanly" at configure time exactly as designed — confirming the patch needed to be dropped, -not refreshed. Also folded into that PR: a fix for `-std=c++20` unknown to GCC 8 -([#105](https://github.com/sheredom/subprocess.h/pull/105)) and a `posix_spawn` exec-failure -reporting fix pre-glibc-2.24 ([#106](https://github.com/sheredom/subprocess.h/pull/106)) — neither -affects this project. If a regression surfaces on old-glibc builds (manylinux2014/manylinux_2_28), -re-check `vendor/sheredom/subprocess.h`'s `SUBPROCESS_HAVE_CWD` guard against this description before -reintroducing a local patch. - -**`0005` was dropped at the b9981 bump.** Upstream's own `server-context.cpp` picked up an -equivalent — and broader — fix for the same checkpoint-starvation problem: `create_checkpoint` -now tracks `id_task` and evicts a stale checkpoint that sits within `checkpoint_min_step` of a -newer one (instead of our pre-creation duplicate-position check), and the `do_checkpoint` gate -now exempts **every** near-prompt-end checkpoint from the min-step spacing (`near_prompt_end`, -unconditionally) rather than only recurrent/hybrid (`ctx_tgt_seq_rm_type` `FULL`/`RS`) models as -our patch scoped it. The upstream version is a strict superset of what `0005` did, so it applies -cleanly with the patch removed and needs no replacement. If a regression in agentic multi-turn -prefill behavior shows up, re-check `tools/server/server-context.cpp`'s `create_checkpoint` / -`do_checkpoint` logic against this description before reintroducing a local patch. - -**`0004` was dropped at the b9982 bump.** Upstream merged -[ggml-org/llama.cpp#23116](https://github.com/ggml-org/llama.cpp/pull/23116) verbatim: the -`b9981...b9982` diff carries the exact same `oaicompat_chat_params_parse` precedence change -(`reasoning_budget_tokens` > `thinking_budget_tokens` alias > server default, plus the per-request -`reasoning_budget_message` override) and the same two `tests/test-chat.cpp` regression cases -(`test_reasoning_budget_tokens_per_request` / `test_reasoning_budget_message_per_request`, -byte-identical body). No local patch needed — the tree already matches what `0004` used to add. +**Dropped patches** (`0004`, `0005`, `0009`, `0010`, `0011`, `0013` -- upstream fixed each defect +itself) are recorded in [`docs/history/dropped-llama-patches.md`](docs/history/dropped-llama-patches.md): +why each went, and the runnable guards that stayed behind re-pointed at upstream's contract. Read the +matching note before reintroducing a patch for the same defect. **The standing drop-check on every +bump:** the applier detects "does not apply", never "upstream already fixed this" -- for each +remaining patch ask whether the *defect* is still there, not only whether the patch still applies. ## RPC backend: `--rpc` client and the in-JVM `RpcServer` @@ -1770,7 +1661,10 @@ little-endian WAV writer (`tts_wav.hpp`), the JSON/token/embedding transforms, a which is where an endian bug in *our* code could hide. Model-backed **Java** tests are deliberately **not** run under emulation (a JVM + GGUF inference under `qemu-user` is slow and flaky); the Java↔JNI boundary uses host-native array copies (endian-transparent), so the C++ gate covers the actual risk. -`-DGGML_OPENMP=OFF` sidesteps cross-libgomp issues (ggml uses its own `std::thread` pool). s390x is a +`-DGGML_OPENMP=OFF` sidesteps cross-libgomp issues (ggml uses its own `std::thread` pool). +`-DGGML_NATIVE=OFF` also switches VXE off, so the binary is **scalar** on purpose; **do not "fix" a +VXE-related compile error with `-DGGML_VXE=ON`** (it needs `-march=z15` too and raises the hardware +floor) -- measured in the `0013` note of `docs/history/dropped-llama-patches.md`. s390x is a CPU platform like aarch64, so it ships as the natives jar `cpu-linux-s390x`, part of `llama-platform` (`OSInfo` maps `os.arch=s390x` → `Linux/s390x`). **Fail-loud** and in `package.needs` like every other build. (Upstream llama.cpp already supports s390x @@ -2422,1005 +2316,34 @@ releases as a signed Central Portal bundle upload (staging repo → zip → Publ ## Local coding agent with Atmosphere (`llama-atmosphere-agent/`, standalone) -A **copy-and-run general-purpose terminal agent** (Claude Code / OpenCode reduced to the essentials, offline) -that pairs [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in OpenAI-compatible -agent runtime with this project's `OpenAiCompatServer`. Like `android-llmservice/` it is a -**standalone Maven project, NOT a reactor module and NEVER on Maven Central** — it is an application, and it -needs Java 21 (Atmosphere's floor) while the core stays Java 8. CI builds it against the core it just -installed (`-Dllama.version=`); a user copies the folder and runs -`mvn compile exec:java -Dexec.args="…"` with no `-D` at all — the pom's `llama.version` names the -**released** core the READMEs describe (currently `5.2.0`, written as if released so the docs are -right the moment the release lands). The natives come from the -`natives` profile (`llama-platform`, active unless `-Dllama.natives=none`) plus the `gpu-natives` -profile (`-Dllama.classifier=`); CI passes `-Dllama.natives=none`, since it installs only -the classes and the model-backed job uses `-Dnet.ladenthin.llama.lib.path`. - -**Release asset: the agent jar WITHOUT the core.** `mvn -P assembly package` (the pom's `assembly` -profile, descriptor `src/assembly/agent-jar.xml`) builds -`llama-atmosphere-agent--jar-with-dependencies.jar` — named after the **core** version -it was built against, not the agent's own `1.0.0-SNAPSHOT`, because it only runs next to that core. -It excludes `net.ladenthin:llama` **with its whole runtime graph** (`useTransitiveFiltering`: Jackson 2, -slf4j-api, and Jackson 3's `jackson-annotations`, which resolves through the core's trail) plus -`jspecify` and `slf4j-simple`, all of which every core fat jar already bundles — so the asset is ~14 MB -(most of it Jetty, the ACP SDK with Reactor, and the Atmosphere console pages) instead of hundreds, the natives are not in the release twice, and there are never two SLF4J providers. -The manifest's `Class-Path` names the four `llama--all---…` fat jars and then the default -`llama--jar-with-dependencies.jar`, so `java -jar` works when the agent lies next to any of them -(missing entries are ignored; note that a manifest `Class-Path` is honoured under `java -cp` too). -**Rename a core fat jar and this list must follow** — `smoke-agent-linux` is what notices. -CI wiring (`publish.yml`): the model-free job builds it, writes the `.sha256` and uploads artifact -`llama-atmosphere-agent-jar`; `github-snapshot` / `github-release-signed` download it into the asset -directory next to `llama-fatjars`, so `sign-fatjars.sh` signs it (`*-jar-with-dependencies*.jar`) and -the one upload attaches it. **`smoke-agent-linux`** (`.github/smoke-agent-jar.sh`) runs the asset the -way the README tells a user to — `java -jar` next to the real `all-linux-x86-64` fat jar — and checks: -bytecode ≤ 65 (Java 21, unlike the core's 52 — with one `--allow` for JLine's FFM terminal provider `org/jline/terminal/impl/ffm/*`, 25 classes shipped as Java 22 bytecode that JLine discovers through `META-INF/jline/providers/ffm` and never loads on 21, where it picks its JNI provider; the first CI run of the smoke caught them), that the jar started **alone** fails with -`NoClassDefFoundError: net/ladenthin/llama/LlamaModel` (i.e. it really carries no core), `--help`, a -one-shot `2 + 2` answer and a `read_file` round that must surface a marker from `--workspace`, all on -the cached `TOOL_MODEL_NAME` with `--ngl 0`; then `--web` on port 0 (the console is `401` without the -token, the token link answers `302` with the `jllama_agent` cookie, the console page is served with it) -and `--acp` through `.github/smoke/agent_acp_smoke.py` (standard-library Python playing the editor: -handshake, modes, a streamed `4`, the announced commands, a `read_file` tool card that brings a marker -back, a clean exit when stdin closes, and nothing but JSON-RPC on stdout). **All three agent jobs gate -both publish jobs** (model-free, model-backed integration, smoke). - -**The console pages come from a jar the assembly otherwise leaves out.** Atmosphere's prebuilt AI -console lives in `atmosphere-spring-boot-starter` under `META-INF/resources/atmosphere/console/`. The -pom depends on that starter with a `*:*` exclusion (no Spring on the classpath), the assembly's main -dependency set excludes it, and a second dependency set unpacks **only** that resource path. Remove -either half and the jar either grows by the whole starter or `--web` starts with nothing to open -(`WebConsole.available()` says so at startup). - -**What Atmosphere is, for this purpose.** `org.atmosphere:atmosphere-ai` (4.0.71) ships -`BuiltInAgentRuntime` + `OpenAiCompatibleClient`: a zero-framework OpenAI client that *always* -streams (`stream:true`), accumulates `delta.tool_calls` by `index`, executes `ToolDefinition` -executors, re-submits the conversation (assistant `tool_calls` message **without** a `content` key, -then one `role:"tool"` message per call with `tool_call_id` + `name`), and loops until -`finish_reason` is not `tool_calls`. It reads `LLM_BASE_URL`/`LLM_MODEL`/`LLM_API_KEY` or takes -`AiConfig.configure(mode, model, apiKey, baseUrl)`; `GET /models` is best-effort; the Responses API -is used only when the base URL contains `api.openai.com`; `tool_choice`/`parallel_tool_calls`/ -`response_format` are not sent. It runs headless — `runtime.execute(AgentExecutionContext, -StreamingSession)` — so no Spring Boot, servlet container or `@Agent` scanning is involved; its -built-in `FileSystemTools` resolve the `AgentFileSystem` from `StreamingSession.injectables()`, -which is how the tools are confined to a workspace. The `@Agent`/`@AiTool` annotations and the -Spring Boot starter are a deployment layer on top of the same runtime. - -**Verified compatibility (verdict A — works unchanged).** Two test layers, both in the project: - -- `AtmosphereWireContractTest` + `LocalAgentTest` — **model-free, every PR, seconds**: the *real* - `OpenAiCompatServer` (routing, bearer auth, `/v1/models`, SSE framing) over a loopback socket with - a `ScriptedBackend` replaying llama.cpp-shaped chunks (role delta, `tool_calls` deltas with - `index`/`id`/`name` and fragmented `arguments`, `finish_reason:"tool_calls"`). Pins: one tool - round; four rounds incl. a parallel pair with interleaved fragments and the whole history kept; - chunk-by-chunk streaming and history replay; `temperature`/`max_tokens` on the wire; 401 on a - wrong key before the backend is reached; and the **one known gap** — an engine failure *after* the - stream started is an SSE `data: {"error":…}` under HTTP 200 (upstream llama-server does the same), - which Atmosphere's parser ignores (it reads only `choices[0]`), so the turn completes with the - text so far instead of erroring. That is a SHOULD for Atmosphere's `OpenAiCompatibleClient`, not - for this project. -- `AtmosphereToolLoopIntegrationTest` — **model-backed, CI only** (`test-java-llama-atmosphere-agent-integration`, - a publish gate): the same loop against the cached Qwen2.5-1.5B tool model - through the downloaded Linux natives — plain chat, streaming (≥ 2 chunks), a tool call whose result - is answered, a read→write→read loop that changes a temp file. Self-skips without the GGUF. - -**The one core change this needed:** `OpenAiBackend`, `ChunkSink` and -`OpenAiCompatServer(OpenAiBackend, OpenAiServerConfig)` are now **public** (they were the -package-private test seam). A sibling module cannot otherwise drive the real server without a model; -the alternative — a same-named package in the sibling's test tree — is a split package that breaks -the moment anything runs on the module path. - -**Layout.** `AgentOptions` (CLI parsing, pure), `AgentRunner` (the whole Atmosphere wiring, ~40 -lines: `AiConfig.configure` → `BuiltInAgentRuntime` → `AgentExecutionContext` + `ToolLoopPolicies`), -`ConsoleSession` (streams to stdout, prints `⚙ tool {args}` / `↳ result`, supplies the -`WorkspaceAgentFileSystem` via `injectables()`), `ShellTool` (opt-in `run_command`, `sh -c` / -`cmd /c` starting in the workspace, timeout kills the process tree, output tail-truncated), `LocalAgent` -(`--base-url` = external server, `--model` = in-process `LlamaModel` + loopback `OpenAiCompatServer` -with `enableJinja()` and `setLogVerbosity(2)` by default — llama.cpp logs to **stderr**, the console the -streamed answer shares, so the per-request `slot …` INFO lines would interleave with it; `--log-verbosity ` -/ `--verbose` override — one-shot `--prompt` or a `you>` REPL with `/clear` `/exit`). `.mvn/jvm.config` -pins `-Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8` for the `mvn exec:java` JVM: on Windows, -`common_init()` switches the console to UTF-8 (`SetConsoleOutputCP(CP_UTF8)`) after the JVM fixed its -stdout encoding from the old code page, which turned umlauts/emoji in answers into `�`/`?`. It needs a -sibling **`.mvn/jvm.config.license`** (the same two SPDX lines as the one next to the root `.mvn/jvm.config`): -a `jvm.config` takes no comments, so REUSE can only read its metadata from that file, and without it the -`REUSE Compliance Check` job fails on `main` — which is how it was found, the PR run having been cancelled. -Spotless (palantir) is configured in its own pom; the model-free CI job runs `spotless:check`. - -**The REPL layer (commands, approval, status line, rendering).** Eight small classes and one dependency -(`org.jline:jline`, one jar, no transitive deps): -`SlashCommands` (a line starting with `/` whose first word names a command is handled locally — -`/help /status /tools /mode /compact /clear /exit`; **an unknown `/command` goes to the model**, which -is why no escape syntax is needed for `/usr/bin/…`), `ApprovalMode` + `ConsoleApprovalStrategy` -(`[y]es/[n]o/[a]uto` per gated call), `StatusLine`, and `Ansi` + `MarkdownConsole`. Four points that -are decisions, not details: - -1. **The approval gate is Atmosphere's, not ours.** `AgentRunner.approval(strategy, policy)` attaches - `ToolApprovalPolicy.custom(...)` (gating `run_command`, `write_file`, `edit_file`, `delete`, - `rename` — reading tools never ask) and a `ConsoleApprovalStrategy`; `ToolExecutionHelper` then - blocks the tool loop before the executor runs and turns a denial into the tool result - `{"status":"cancelled","message":"Action cancelled by user"}` for the model. Do not reimplement - that message. `ApprovalWireTest` pins both halves over the real server. **The reading tools - (`ls`, `read_file`, `glob`, `grep`) never ask, and that is a decision, not an omission**: gating - them would make the question so frequent it stops being read. Because the gate is a list of - *names*, a tool upstream adds or renames would drop out of it and then run unasked — so - `READ_ONLY_TOOLS` names the other half explicitly and - `ConsoleApprovalStrategyTest.everyOfferedToolIsEitherGatedOrDeclaredReadOnly` asserts every offered - tool is in exactly one of the two sets, and that neither set names a tool nobody offers. Same class - as the stale `spotbugs-exclude.xml` entries: an allowlist that silently stops matching. -2. **One-shot (`--prompt`) denies a gated call** instead of auto-approving it — `--auto` is the - deliberate opt-in. Atmosphere itself fails closed when no strategy is wired, and this keeps that - direction: an unattended run must not be the most permissive one. -3. **`AgentTerminal` has exactly two implementations, chosen once at startup, and `--plain` picks the - line-oriented one on purpose.** `LocalAgent.usesFullTerminal(options, interactive)` is the single - place that decides: the cursor-controlling console needs someone typing **and** permission to move - the cursor, and `--plain` withholds the second even on a real terminal. That is not a fallback but - a supported mode — for a session that is piped, logged, recorded, or carried by something that - forwards lines rather than a screen. It gives up the pinned block, the spinner, history/completion - and typing-during-a-turn (`PlainTerminal.hasPendingInput()` is always false), and gains being - correct when the output is a file. A normal SSH session needs none of this: a remote terminal - reports its size and handles cursor control like a local one. `JLineTerminal` (a real - terminal: line editing, history, Tab completion of the command names, a status line pinned to the - bottom via JLine's `Status`, single-key answers through `enterRawMode`, streamed output via - `LineReader.printAbove` so the bottom block stays put) and `PlainTerminal` (a `PrintStream` plus a - `BufferedReader`: no cursor control at all, correct when the output is a file). `JLineTerminal.open` - returns **null** instead of throwing when there is no usable terminal — piped input, a dumb - terminal, a missing native provider — and the caller falls back. Every test drives `PlainTerminal`, - which is why none of them needs a TTY. Verified on Windows: JLine picks the `windows-vtp` provider, - so ANSI works there without the registry caveat. -4. **The answer is rendered append-only, one completed line at a time** (`MarkdownConsole`). Redrawing - on every token is what produces the known overdraw/truncation bugs in the Ink/Bubble-Tea based - clients and breaks when the output is piped. Only headings, bullets, fences and inline - `**bold**`/`` `code` `` are handled; italics deliberately are not (`*` is more often a glob than - emphasis). Colour is decided once in `Ansi.detect()` — `CLICOLOR_FORCE`, then `NO_COLOR`, then - `TERM=dumb`/`CLICOLOR=0`, else "is a terminal" via `Console.isTerminal()` (reflective: JDK 22+; - below that `System.console() != null`). -5. **`/loop` keeps its state in a file, not in the context, and stops on a text marker** (`TaskLoop`, - `LoopOptions`). Every step re-sends the task verbatim and drops the history, so the context cannot - grow (Claude Code's ralph-wiggum plugin does the same, and `AGENT-LOOP.md` in the workspace is the - memory). The stop signal is a line that is **exactly** `<>`, never a substring — - deliberately **not** a "done" tool: below 7B a model emits a malformed tool call far more often - than a malformed line, and mini-SWE-agent's SWE-bench results come from exactly this plain-sentinel - design. `--check ''` re-verifies the claim and feeds a failure back. Four guards, none of them - trusted to the model: step cap, wall-clock budget, stall detection (three steps with no file change - and no tool call), interval. **Order matters and a test pins it**: the marker is checked *before* - the stall detector, because the step that only answers "done" changes nothing and would otherwise - be reported as no progress. -6. **Three of the file tools are this project's, not Atmosphere's** (`WorkspaceTools`, `TextEdits`, - `WorkspaceSearch`) — `read_file`, `edit_file`, `grep`. They are **replacements, not additions**: - two tools that both claim to read a file is the worst case for tool selection. They call the same - `AgentFileSystem`, so workspace confinement, path validation and size limits stay Atmosphere's. - Each replacement has a measured reason, and all three are pinned by tests: - - `edit_file`: the framework matches against the **raw** file content, so a model's LF text never - matches a CRLF file — on Windows *every* edit fails silently. `TextEdits` normalizes before - matching and restores the file's own ending and byte-order mark. It also shows the nearest lines - on a miss and the line numbers on an ambiguous match (a failed edit drops the eventual success - rate from 90.5 % to 57.2 %), offers `replace_all`, and applies a batch of edits **all-or-nothing** - — a deviation from every shipping agent, which apply sequentially and leave a half-edited file. - - `grep`: the framework walks **alphabetically** with one global 2-second deadline and one global - 500-hit budget, so `.git`, `target` and `node_modules` consume both before `src` is reached. - `WorkspaceSearch` excludes them, groups by file with line numbers, and **states** truncation. - - `read_file`: `offset`/`limit` and numbered lines (whole-file reads measure 12.7 % against 18.0 % - task success in the SWE-agent ablations). The numbers are display only, which both the tool - description and the system prompt say — leaked line numbers in `old_string` are a known failure. - - `edit_file` **refuses a file that was not read** in this session (`WorkspaceTools.ReadTracker`). - Not a staleness check: an exact unambiguous match is safe regardless; this catches the model - inventing the text. - **Rejected on evidence, do not add later without new numbers:** a unified-diff/patch tool (Meta's - ablation: search-replace 42–53 % vs 26–30 % unified diff vs 20–26 % line diff on one model; a 7B - model collapses 54 → 33 → 14 %), fuzzy matching (turns a loud miss into a silent wrong-place edit), - an embedding index (Cursor's production effect is +0.3 %), and LSP tools (the one isolation study - finds them token-negative and *worse* at multi-file rename, because renames touch comments and - strings that semantic references exclude). -7. **The session transcript (`Transcript`) is not the conversation the model is sent, and must not be - merged with it.** The model's history is rewritten by `/compact` — a summary replaces the turns — - and has never carried a timestamp; the transcript only grows and stamps every entry. `/compact` - adds a note to it and changes nothing else, `/clear` empties it (the command means "forget this - session"), `/save [name]` writes it into the workspace, and `--transcript ` appends live so a - killed session still leaves what it had — that write failing is swallowed, because a record that - exists to survive a bad ending may not cause one. **A list, not a map keyed by the timestamp**: a - tool result and the answer after it regularly share a millisecond and a map would drop one - silently; insertion order already is time order. `ToolCallLog` stays as the separate, hard-cut - receipt for `/calls` — it answers "did that really run", which prose cannot. - **`/load` replays only `USER` and `AGENT` entries** as messages: a tool result outside its round is - not something a chat template has a place for, and inventing a shape for it would be worse than - letting the model call the tool again. The parser treats a line without a stamp as a continuation - of the entry above it, because an entry is not a line — an answer keeps its newlines when written, - and reading line by line would turn one answer into several. A file that is not a transcript yields - **no** entries rather than one wrong one, since anything it yielded would be replayed to the model - as if it had been said. - - **`JLineTerminal.open` refuses when there is no console, and that was a red build for months.** A system - terminal takes over the process's standard input, and where there is no console that input belongs to somebody - else. Inside a **Surefire fork it is the channel Surefire talks over**: a test that drives the agent - interactively reached `open()`, JLine grabbed the channel, and the run ended with - `[SUREFIRE] std/in stream corrupted` — **every test green, the build red**, which is exactly the shape that - hides. It also made `LocalAgentTest` take 23.6 s instead of 2.0 s, because it was blocking on reads that were - never going to arrive. The check is the same two-step `Ansi` uses for colour: `Console.isTerminal()` where it - exists (JDK 22+, where `System.console()` returns a console even for redirected streams), otherwise the mere - presence of a console. Piped input lands here too and has always been served by the plain console, so nothing - else changes. - -8. **Tool calls are carried into the conversation as a text note, and logged for `/calls`.** - `LocalAgent.withToolNotes` prefixes each turn's answer in the history with - `(tools I actually ran this turn: -> )`, and `ToolCallLog` - keeps the same data for the `/calls` command. **Why it is a note and not real `tool_calls` - messages:** `AbstractAgentRuntime.assembleMessages` rebuilds every history entry as - `new ChatMessage(h.role(), h.content())` — the tool-call array and the tool-call id never leave the - framework, so protocol-faithful replay through `context.history()` is impossible; content is what - survives. **Why it exists at all:** with only user text and assistant prose in the history, a 4B - model stopped calling tools after the third turn of a real session and *described* the work instead - — inventing JUnit tests, a Maven build and a `.bat` script, complete with exit codes, while the - workspace stayed empty. The system prompt also forbids claiming an action without the call. - **Placement was found by failing twice, so do not "simplify" it:** in front of the assistant's - answer made the model copy the record into its own replies (the user saw `(tools I actually ran - this turn: …)` as the first line of an answer); real `tool_calls` messages are impossible (see - above); a mid-history system message is cleanest but Mistral's template requires strict - user/assistant alternation and Gemma has no system role. It therefore rides in front of the **next - user message**, which every template accepts. - Pinned by `LocalAgentTest.aToolCallStaysInTheHistorySoTheNextTurnSeesItHappened`. - **Live feedback while a turn runs** (`LocalAgent.activityLine`, `ShellTool`'s line-by-line output): - the block's first row names the running tool and its own elapsed time, and shell output is printed - as it arrives. **The turn runs on its own thread, and it has to:** Atmosphere's `execute()` is - synchronous — it returns only once the whole turn including every tool round is done — so running - it on the console thread leaves nobody to refresh the line, and the block sits on - "… waiting for input …" for the entire turn (exactly the symptom that was reported). The approval - prompt then reads a key in raw mode on that worker thread while the console thread redraws four - times a second, so `TurnActivity` pauses the redraw for as long as the question is open. Reading the pipe incrementally is not only cosmetic — an unread pipe blocks the child once - it is full, which on Windows is roughly 4 KB. -9. **The context number in the status line is an estimate, marked `~`, and it moves during the turn.** - llama.cpp emits its usage chunk only when the client sets `stream_options.include_usage`, and - Atmosphere's client does not; `ConsoleSession.usage()` takes the real count when one arrives, - otherwise `LocalAgent.estimateTokens` uses four characters per token. The window size is - `--ctx-size` (in-process) or the server's `/props` (`ServerProps`), and is omitted rather than - guessed when neither answers. **The state row is a `Function`, not a - string**, and `awaitWithActivity` asks it again on every redraw: it used to be rendered once before - the turn and handed over fixed, so the figure stood still through every tool round and only moved - at the next `you>` — which is exactly when it no longer helps anyone decide whether to `/compact`. - `LocalAgent.liveTokens` adds `ConsoleSession.producedChars()` (streamed text **plus** every tool - call and result — all of it is in the prompt of the next model call of the *same* turn) to what the - request carried when it was sent, and yields to the server's own count as soon as one arrives. - `TaskLoop` passes a constant function, and its step label must be copied into a local first: a - lambda may not close over the loop counter. -10. **One call to `AgentTerminal.line` is one screen line**, and `ConsoleSessionTest` is what defends - it. The pinned block is reserved in **lines**, so a single "line" carrying twenty newlines moves the - screen twenty rows further than the terminal accounted for and the block is then drawn across the - output — reported twice, both times from a `write_file` call whose `content` argument was the file. - `ConsoleSession.describeArguments` folds and cuts **each argument value on its own** (80 chars) - before cutting the whole rendering (200), so a call carrying a whole file still shows the file - *name*; results and errors are folded the same way. `JLineTerminal.line` splits a multi-line string - as a backstop for a caller that forgets. Only the console is cut — the model gets everything, and - `ConsoleSession.rounds()` keeps the full arguments for the history note and `/calls`. -11. **The approval mode carries a glyph, and shift+tab switches it**: `ApprovalMode.symbol()` / - `badge()` render `⏸ manual` and `⏵⏵ auto` on the status line and in `/mode`, the transport symbols - the established terminal agents use for the same distinction; `ApprovalMode.next()` is the cycle - the key walks. The binding is `AgentTerminal.onCycleMode(Runnable)`, which **defaults to declining** - — only `JLineTerminal` overrides it, and the startup line advertises the key only when the bind - succeeded. Two details are not obvious: it is bound **both** through terminfo - (`InfoCmp.Capability.key_btab`) **and** to the literal `ESC [ Z`, because JLine's - `windows-vtp.caps` declares no `key_btab` at all while the terminal in virtual-terminal input mode - does send the sequence; and the shortcut fires **only while a line is being read**, so it switches - the mode between turns — which is when it is decided anyway. The REPL therefore holds the two - status numbers in an `AtomicLong`/`AtomicBoolean` rather than locals, so the widget (which runs - inside the reader) can re-render the pinned row with what the last turn left behind. - -12. **The input is framed into the pinned block, and the prompt stays there during a turn; typing - stops the turn.** The frame is two halves that must be read together: the **top** rule is the first - line of the reader's *prompt* (`rule() + newline + "> "`, rebuilt on every read because the window - can be resized) — **no, and the second attempt was wrong too.** Both are recorded because the - obvious fix is the one that fails. (1) Rule as the first line of a **two-line prompt**: - `ERASE_LINE_ON_FINISH` erases exactly **one** line, so every Enter leaves the rule behind and - holding Enter draws a column of them. (2) Rule as **ordinary output before each read**: nothing is - left behind on Enter any more, but one rule now stays in the scrollback per turn and travels up - with it. **There is no third option** — JLine's status region is below the prompt and never above - it, so a rule above the input can only be part of the prompt (1) or part of the scrollback (2). - The settled shape is therefore **one** rule, the first line of the status block, directly under the - input line; the prompt is `"> "`, one line, and `AgentTerminal.readLine`'s `prompt` argument is - consequently **ignored** here. - `ERASE_LINE_ON_FINISH` removes the input line on Enter and the reader thread echoes it above as - `› text`, so the transcript keeps what was asked. - - **`ScreenUseCasesTest` + `ScreenTerminalHarness` are how the resize and `/cls` cases are checkable, - and they are what every byte-level attempt before them could not do.** The harness subclasses JLine's - `LineDisciplineTerminal` and puts its own `ScreenTerminal` (a real VT interpreter, public API in the - shipped jar) behind it, so a test reads the **screen** — "the block is smeared across the output", - "an escape sequence is printed as text", "the block is drawn twice" are indistinguishable from - correct output in a byte stream, which is why two byte-level assertions in `JLineTerminalTest` had to - be deleted, one of them green with the fix it was written for switched off. The cases are the reports: - dragging wider and narrower (with nothing typed, with text in the input, one step at a time, with a - two- and a three-row block, with the real block's glyphs), `/cls` and Ctrl-L (block intact, and the - cursor and prompt on the row the block leaves for them), a window that over-reports its width, size - events arriving on another thread, and a block refresh racing the reader. Several assert the **cursor - row** rather than the content, which is what finally located two of the defects: content can look - plausible while the cursor is rows away from where a region reserved from the bottom expects it. - Three things are load-bearing. **(1) A reader runs in every test** — it runs for the whole session in - the application and owns the resize signal, so a test without one is a state the application cannot - be in. **(2) The block rebuild is invoked directly, not waited for** - (`refreshBlockForCurrentSize`): sleeping for the 120 ms poll passed alone and failed in a full run, - and a flaky test is worse than none — what that leaves uncovered is the polling thread itself, a loop - that compares two sizes and calls that method. **(3) A rule shows as `q` when it went out through a - *prompt*** (the DEC line-drawing set, which this screen renders literally) and as `─` through the - status region, so both forms count as a rule and a row of `q` also says which path it took. - **The JLine fixes are demonstrated red/green through it**, the library being just a property — and since - there are **eight** of them, one of which adds a method, the wiring needs a paragraph of its own. - `ScreenUseCasesTest` **skips itself** on a JLine that does not carry them, keyed on - `Status.repaint()` (the fifth fix and the only one that is a new method, so its presence stands in for the - whole set). That is CLAUDE.md's own rule applied late — "no project test may assert the fixed behaviour - while the build depends on an unfixed release" — and it had been broken: the pom's `jline.version` is the - **released** one, which is what CI builds against, and against it these cases fail. Measured: the whole - module is green with `-Djline.version=4.4.6-atmosphere` (the screen cases run) and green with the - released `4.4.6` as well, where they report as **skipped** rather than passing. - **Whether the fixes are still needed is measurable rather than arguable**, because the gate honours - `-Datmosphere.screen.tests.runAnyway=true`. Against the current console, 233 tests: released `4.4.6` **26 - red**; `statusfix4` (fixes 1–4) 8; `statusfix5` (+`repaint`) 4; `statusfix6` (+addressing every row) 1; - `statusfix7` (+padding one column short) **0**; `statusfix8` 0. (Those version names are the history of the - investigation; **what to build today is `4.4.6-atmosphere`**, the reviewed set — the recipe is at the end of - the investigation document.) So they are needed, by a wide margin — and the - **eighth is carried on the merits, not on a failing test**: `Status.resize` still erases rows above the bar - without it, but this console wipes and redraws after a width change, so nothing here observes it. It stays - because the rows above a bar are not the bar's to clear, an erase is unrecoverable where a scroll is not, and - every consumer that does not wipe needs it; its guard lives where the defect does, in `StatusRepaintTest`. - **Every change was then reverted on its own and measured**, which dropped one and corrected a mistake of mine: - the `xenl` entry for `windows-vtp` is **no longer caught by anything** (addressing every row and padding one - column short removed the dependence on the wrap), so it is out — an unmeasured change does not belong in a set - meant for submission. The `doDisplay()` → `display.resize(size)` change was first reported as unnecessary too, - wrongly: that revert had been made with a string replacement matching nothing, compiled into a second directory - layered in front of the first, which does not reliably win. Compiled properly, two tests fail without it. The - set is therefore **seven** changes in three files, each with a failing test in JLine's own style behind it; the - full matrix and both traps are in the investigation document. The marker cannot tell - the fifth build from the later ones (the sixth adds a *protected* method, the seventh changes only a padding - width), so on an older patched jar one or two cases fail rather than skipping — stated in the test rather than worked around. - **The gate is the whole class and there is no fixed list of affected cases, which is itself a finding.** - Against the released library between six and thirteen of them fail, a *different set each run*: the fourth - fix is a data race, and when its `ConcurrentModificationException` lands on the reader's signal thread it - ends that thread, after which no size change is reported at all and whichever cases were still to run fail - too. The skip is per test (`@BeforeEach`), not a `@BeforeAll` assumption, because a class-level one makes - Surefire record the class as **zero tests** — which reads as "nothing here" instead of "skipped", the same - trap that silently muted every model-backed test in this repository for months. - **What proves the fixes themselves are JLine's own tests**, in the clone's own style and next to its - others: `StatusRedisplayTest`, `StatusDelayedWrapTest`, `StatusConcurrencyTest`, `StatusRepaintTest` - (4 cases: an update with unchanged lines leaves damage on screen, `redraw()` does too because it is the - same diff, `repaint()` puts every reserved row back without scrolling anything, and a repaint before - anything was shown is not an error) and `StatusWrongWidthTest` (3 cases on a screen wider than the width - it reports: every reserved row keeps its own screen row, nothing is written beside the rule, and the rows - above the block stay empty, and — the seventh fix — a reported width one column too LARGE does not wrap the - bottom row). 103 of JLine's own tests are green with all eight fixes, `DisplayTest` and `ScreenTerminalTest` - included. - - - **The fourth fix is a data race, and it is the one that explains the reports that survived the other - three.** `refreshingTheBlockWhileTheReaderRedrawsNeverThrows` refreshes the block from three threads - while the size changes, and fails with a `ConcurrentModificationException` whose stack names the - defect: `AttributedString.join` iterating `Display.oldLines` from `Status.resize` (called by - `LineReaderImpl.handleSignal`) while another thread is inside `Status.update` replacing that same - `ArrayList`. Neither is synchronized and both are reachable from different threads *by design* — this - console keeps its block current from its own thread four times a second. **Keystrokes do not provoke - it; a size change does**, because the reader then resizes its display, resizes the region and - redisplays. Raised on the input pump it ends that thread, after which no size change is ever reported - again and the block keeps its width — which is the "rule wider than the window, wrapped" screen, - reported as a separate problem. The fix is `synchronized` on `Status`'s nine public entry points. - **And it is why the screen harness could not reproduce the reports it was built for**: - `ScreenTerminalOutputStream.write` is synchronized, so it serialises the byte stream, while the - unprotected state is JLine's own list — no stream lock reaches it. Two cases carry - `@Disabled` as the record of a defect that is still open — a narrower drag loses the edit line, and a - *three*-row block ends a wide drag with the rule three columns short, the state row shifted one - column and the prompt on two rows, while a two-row block comes out clean. - - **`JLineTerminalTest` is how any of this is checkable**: `JLineTerminal.over(Terminal, …)` takes a - terminal built over two streams, which renders exactly like a TTY, so the screen can be asserted on - the emitted bytes. Two things that cost an hour each and are not guessable: the test terminal needs - **`stdoutEncoding`** as well as `encoding`, or every `─` arrives as `?`; and a box character is - written as UTF-8 from `printAbove` but as the **DEC line-drawing set** (`ESC(0` + `q`s + `ESC(B`) - inside a *prompt*, so a counter that looks only for `─` passes against the exact bug it was - written for — verified by putting the two-line prompt back and watching the test go from 1 rule to 5. - - **Escape sequences drawn as text** (`[?1h` above the prompt, then a `1H` inside the rule) were two - further defects of the same family, and the second is the one that eventually **destroyed the - block**. First: `line()` wrote straight to the terminal whenever the reader was not inside - `readLine`, which is exactly when the next read emits its init sequence — once the reader thread - exists, **everything** now goes through `printAbove`. Second: three threads write to this terminal - as a matter of course — the turn (Atmosphere's thread) prints tool lines, the console thread - refreshes the block four times a second, and with an in-process model **llama.cpp logs to stderr**, - which is the same console and goes around JLine entirely. The first two are serialised by a - `writing` lock held across `line()` and `status()`; the third is fixed by - `LocalAgent.captureNativeLog`, which routes the native log through `LlamaModel.setLogger` - (the callback sink `patches/0014` added) into `terminal.line`, so it scrolls in above the prompt - like any other output instead of scrolling lines JLine never sees. **Honest limit:** the lock is - reasoned, not test-covered. Two attempts to pin it are recorded in the history of - `JLineTerminalTest` and both passed with the lock removed — even one that sliced every - `OutputStream.write` in half — because `PrintWriter` already makes a single call atomic and the - interleaving happens *between* calls, inside JLine. A test that is green either way is worse than - none, so it was deleted rather than kept. - - **`/cls` clears by SCROLLING, not by erasing, and `/clear` also wipes the history.** - `AgentTerminal.clearScreen()` defaults to doing nothing (a stream has no screen); `JLineTerminal` - prints a window's worth of blank lines through `printAbove`. That is the whole implementation, and it - replaces an erase-based one that took four attempts and produced a reported defect each time — which - is why the reasoning is kept in full. - - **What a clear has to do here is two things at once**: leave the screen blank *and* leave the input on - the row the pinned block leaves for it. Erasing does the first and undoes the second, because - `clear_screen` puts the cursor home and the reader draws its prompt where the cursor is. The - reports, in order: blank rows printed after the erase to push the input back down scrolled the - *erased* lines back into view (`ESC[2J` clears the visible area and leaves them in the scrollback); - a `cursor_address` smuggled into `printAbove`'s argument corrupted its bookkeeping — it moves up, - writes, and redraws the prompt below — and stranded a character above the prompt; and the erase on - its own left the input at the top left ("nach /cls ist der cursor auch ganz oben und nicht unten"). - The screen tests then showed that last state is worse than it looks: the reader redraws its prompt - as a **diff** against what it believes is on screen, an erase invalidates that belief, and the - measured result is **no prompt on screen at all** — cursor on row 0, the block still pinned at the - bottom, ten blank rows between them. - - **Scrolling has none of those problems because it is nothing but output.** Everything is pushed above - the window, so the screen is blank and what was written stays reachable with the scrollbar — which - erasing the scrollback (`ESC[3J`) would have broken anyway, and this console promises it. Nothing is - erased, so nothing can be pulled back into view. The reader's bookkeeping stays right, because - printing above the prompt is what `printAbove` is *for*. The block is never touched: it is pinned, so - it needs neither `status.reset()` nor a rebuild — three steps of erase-era repair went away with the - erase. And the cursor ends on its row **by construction**, since printing is what pushes it there — - which is also the mechanism behind "ein paar Mal Enter und alles sitzt wieder", so a screen that has - lost rows is repaired by a clear rather than left crooked. - - **Ctrl-L is the same command through another door, and this console now owns the binding.** JLine's - keymap dispatches Ctrl-L by *name* to the widget registered under `LineReader.CLEAR_SCREEN`, and - JLine's own widget wipes and redraws the line — i.e. it reproduced the identical defect, measured on - an interpreted screen. Replacing the map entry re-points the key without touching the keymap. **It - deliberately does not take the `writing` lock**: a widget runs on the reader's thread with the - reader's own lock held, while `line()` takes `writing` first and the reader's lock second, so - acquiring `writing` there inverts the order and hangs the session. What serialises it instead is the - reader's lock, which every `printAbove` needs; a concurrent block refresh can still interleave, which - is the exposure JLine's own Ctrl-L widget has today as well. - - **Where each half is pinned.** `ScreenUseCasesTest` asserts the part that is only visible on a screen - — after `/cls`, and again after Ctrl-L, the cursor is on `rows - 1 - blockRows`, the prompt is on that - row, and every row above it is blank (checked one row at a time, so a failure names the row). All four - were red before the change, with the cursor on row 0 and no prompt anywhere. - `JLineTerminalTest.aClearScrollsAWindowAndErasesNothing` pins the pair a pipe *can* see: a window's - worth of line feeds, and no `ESC[2J` at all. That test changed sides — it used to assert the exact - opposite ("a clear must not scroll") and was right for as long as clearing meant erasing; both sides - are recorded in it. Two byte-level assertions written for the erase (`erase display reached the - screen`, `the block is drawn again after the wipe`) were **deleted** rather than adapted: what they - described no longer happens, and the behaviour they were reaching for is asserted on the screen. - - **The screen is scrolled to the bottom once, before the first prompt** (`scrollToBottom`). The - reader draws its prompt at the cursor, i.e. after the last line printed, while only the status - block is pinned to the window — so on a half-empty screen the input floats in the middle with the - block far below it, and they only meet once output has scrolled the cursor down by itself. That is - why it looked right after a few turns and like an ordinary prompt at the start. Emitting - `rows - 1` newlines once makes it the state from the first prompt on; from then on every printed - line scrolls and the cursor stays on the last row. The cost is a screenful of blank lines above the - session, which is what a program that wants its input at the bottom *without* taking over the - screen has to pay. - - **The repeated `> Hallo` is a JLine defect, reproduced and fixed upstream-side.** Typing without - Enter and then dragging the window showed the prompt and buffer a dozen times side by side. It is - **one copy per size event** — a drag reports a new size per step — and an earlier reading of this - ("the repeats are written while typing and widening only reveals them, so the count follows - keystrokes") was an inference from a number rather than a measurement, and wrong. A three-row - `Status` is equally necessary: the same drag without one is clean, and typing without a resize is - clean too. Ruled out along the way: project code, JLine 4.4.6, `nativeSignals(false)`, a missing - terminfo capability, Windows itself (it reproduces on a virtual `xterm`), and the `lastStatusSize` - guard in `redisplay()` that was the documented hypothesis before the resize path was measured. The - cause is `handleSignal(WINCH)`'s status branch — see the next paragraph. Full record, including the - two measurements that were *inconclusive rather than negative*, in - [`docs/upstream-investigation-jline-status-windows-redraw.md`](docs/upstream-investigation-jline-status-windows-redraw.md). - **Nothing here can honestly fix it**; `--plain` pins nothing and is unaffected. - - **Every output line is folded by this console, so the CONSOLE never wraps one** (`JLineTerminal.fold`, to - one column less than the window). This is the fix for the whole family of drag artefacts and the reasoning - is the load-bearing part: a line the console wrapped is **one** logical line spanning two screen rows, and - Windows **joins such lines again when the window is widened**. The text above then needs fewer rows and - everything below moves **up** — including the block rows last rendered, which end up above the pinned region - where nothing ever writes again. One leftover per drag step, which is the reported staircase of rules - climbing "von unten rechts nach oben links"; narrowing does it in reverse and walks the input upwards. - **No program can observe a reflow or prevent one — but it can deny it a target:** a line that was never - soft-wrapped has nothing to join. - **The harness could not reproduce this, and that is why every grow test was green while the console was - not:** `ScreenTerminal` pulls scrollback down when it grows and **does not reflow at all**. The property is - testable, though, and that is what the two screen cases assert — after printing a long line, **no row on - screen reaches the last column** (red before: rows 8 and 9 were full width). Five unit tests pin the fold: - a short line is untouched, no piece exceeds the width, a double-width glyph is never split (a piece may come - out a column short instead), ANSI a caller put in survives and does not count towards the width, and - umlauts stay whole. Folding counts **screen columns** via `AttributedString.fromAnsi`, never characters — - an icon is one character and two columns, which has cost this class a defect before. - **The price is stated rather than hidden:** text keeps the line breaks it was printed with, so widening no - longer re-flows the conversation. That is the same trade this console already makes by rendering - append-only. - - **The pinned block is rebuilt whenever the window changes size, from a poll** (`startWatchingSize`, - every 120 ms). What JLine holds are the rows it was handed, so a rule built for a 113-column window - stays 113 columns wide: on a resize the row is padded with spaces or cut with an ellipsis, never - re-made, and the next row then continues on the same screen line — the three rows run together with - growing gaps, which is what was reported three times. Only the caller knows a rule is meant to span - the window. **Both halves are needed and the second is easy to miss:** `status.resize(size)` first, - or the pinned region cuts the rebuilt rule straight back to the old width. **Measured, not - reasoned:** a probe on the real console reproduced the report when it left the block alone and - rendered cleanly when it rebuilt all three rows per size event — after four harness theories had - been measured and discarded (buffer-vs-window width, reflow by joining the rows, a - wide→narrow→wide drag, an accumulating cursor drift), and with that probe recording window and - buffer at identical widths throughout and one size event per ~125 ms for a single drag. - **It carries no test, and a written one was deleted rather than kept:** the stream-backed harness - has no screen model, so all it can observe after a size change is that JLine re-emits the rule at - its *old* width without the poll and emits nothing with it — neither says the rule was redrawn at - the new width, and the first assertion built on that passed with the poll disabled. Green either - way is worse than none (the same call already made for two write-lock tests). The evidence is the - probe, on the console where it happens. **The poll DOES re-assert the region - (`status.resize(size)`), and that line was once removed for a measured reason before coming back for - another:** it writes to the terminal directly, so without `Status`'s methods being synchronized it landed - inside what the reader was drawing for the same size change and printed `36;1H` as text with the block - doubled. That race is the fourth JLine fix carried here, so this line depends on that fix and must not be - kept without it — `refreshingTheBlockWhileTheReaderRedrawsNeverThrows` is what holds the pair together. - - **The pinned region ADDRESSES each of its rows, which is the sixth JLine fix and the one that explains - the screens full of rule fragments.** One measurement settled a long chase: a status update emits - `ESC[8;1H` and then the rows back to back, padded to the reported width — **one cursor address for the - whole bar**. The second row begins on a new screen row only because writing the last column of the first - made the terminal wrap. A screen that is wider than the reported width (a window mid-drag, or one whose - terminal has reflowed its buffer) therefore never wraps, and **every reserved row lands on one screen row, - side by side**. And because the bar is reserved from the *bottom*, what the collapse pushes past the window - lands in the **output area above it**, where nothing writes again — one fragment per drag, which is why it - accumulated and why only `/cls` (which scrolls) cleaned it up. - `Display.addressesEveryRow()` (false by default, so no other display changes) switches off the - pending-wrap compensation, and `Status.MovingCursorDisplay` turns every cursor move into an absolute - address. **Two narrower variants were measured and are wrong**, both recorded in the code: addressing rows - at the top of the update loop makes even an unchanged update save and restore the cursor, which interleaves - with the reader's writes and printed the typed text **one character per screen row** (deterministic, 2/2); - addressing only row starts leaves a row that shares a prefix unaddressed, so the row above's pending wrap - is never finished and the state row came out shifted one column, `" state]"` (deterministic, 3/3). - **And it shipped with an off-by-one that JLine's own tests structurally could not see:** `Display` counts - positions in `columns + 1` per row (`columns1`) throughout, the address divided by `columns`, so row *N* - landed at column *N* — rule at 0, activity row at 1, state row at 2. Every existing test compares - **trimmed** rows, which is exactly what hides it; it took the interpreted screen here to catch it, and - `StatusWrongWidthTest.everyRowStartsAtColumnZero` now asserts on the raw row. - - **The prompt is never addressed at all, and that is the row that is still lost on a resize.** Bisected on - the interpreted screen with the real block: after `/cls` the cursor is right (16 of 16), after **JLine's - WINCH handling alone** it is wrong (19 of 20), and neither this console's row rebuild nor its repaint - changes it either way. The byte stream says why — the status region clears its band, re-establishes the - scroll region, restores the cursor, the block is addressed row by row, and then the prompt is written as a - bare `>` **wherever the cursor happens to be**. Nothing relates it to the region below. A growing window - moves the screen's content down by as many rows as it has scrollback to pull from, which need not be the - number of rows added, so the prompt ends up a row or two above the rule. - **Not patched, and the reason is a contract rather than cowardice:** a prompt directly above the pinned - region is what *this* application wants (it scrolls to the bottom once at startup), while JLine promises - only "the prompt is drawn at the cursor", which for a half-empty screen is correct. "Move the cursor to the - bottom of the scroll region on a resize" would be right here and wrong there. - `ScreenUseCasesTest.aGrowingWindowLeavesThePromptOnTheRowTheBlockLeavesForIt` carries the reproduction with - the bisection in its `@Disabled` text; `/cls` repairs it in one keystroke, which the same measurement - confirms. - - **The startup line names the terminal and the JLine patch level** (`LocalAgent.describeTerminal`), because - its absence cost two rounds of testing: a screenshot from a console says nothing about which library - produced it, and twice a defect was chased whose fix the jar in use did not contain. Probed by **method** - (`Status.repaint` for the fifth fix, `Display.addressesEveryRow` for the sixth), never by version string — - the patched builds overlay classes into the released jar and keep its manifest version, so the string - cannot tell them apart. - - **A screen model that REFLOWS, because the one that does not reported every drag case green** - (`ReflowingScreenHarness` + `ReflowResizeTest`, and `ReflowProbe` (in the agent's test tree, `…/atmosphere/probe/`) for the real console). JLine's - `ScreenTerminal` adjusts its buffer on a resize and **never reflows**; a real console does, and that single - missing behaviour is why several rounds of reproduction failed. The harness adds it and nothing else: it does - not parse VT (JLine's screen keeps doing that), it reads the screen on a width change, rebuilds the logical - lines, re-wraps them and writes the result back **behind JLine's back** — the honest channel, since a console's - reflow changes the screen without telling the program. Both of its rules are explicit so they can be checked: - a row **continues** into the next when its last cell is not blank (the terminal's own rule in practice), and - which edge keeps its content when joining frees rows is a **parameter** (`Anchor.BOTTOM`/`TOP`), not a guess. - `ReflowingScreenHarnessTest` puts the harness itself under test. **Its limit is stated:** the cursor is not - reflowed with the content, so it is evidence about what is ON the screen after a resize, not about where the - cursor lands — those cases stay with `ScreenTerminalHarness`. - **The insight that came with it:** folding makes nothing soft-wrapped *at the width it was printed at*, so - making the window NARROWER turns those same lines into wrapped ones and the next widening joins them. That is - why enlarging alone looked fine while alternating "zerhackt alles" — the shrink manufactures what the widening - then moves everything with. - - **The eighth JLine fix: `Status.resize` must not erase the rows above the bar.** The first thing the emulator - showed was not a reflow but an **erase** — after a shrink the conversation was simply gone. Reproduced without - any reflow at all, on the ordinary harness: three answers on a 100-column window, halve the width, and **two - are erased**. `Status.resize` clears a band when the geometry changes and pulled its start upwards by - `(ceil(oldColumns / columns) - 1) * statusLines` rows "to account for wrapped status lines" — six rows above a - three-row bar in a halved window, straight through the application's output. Since the seventh fix a status row - is padded one column short and **cannot wrap**, so there are no extra rows to account for. Red/green both - sides: `StatusRepaintTest.makingTheWindowNarrowerDoesNotEraseWhatIsAboveTheBar` fails with the compensation - restored and passes without it; `makingTheWindowNARROWERmustNotERASETheConversation` is red against - `statusfix7` and green against `statusfix8`. **This is the answer to "nach dem kleiner ziehen sehe ich es nicht - mehr":** the output was never scrolled away, it was erased by the bar's own housekeeping. - - **A status row must not write the last column, which is the seventh JLine fix.** `Status.update` padded every - row to the **full** reported width. A row padded to a width the screen does not have **wraps**: it occupies - two screen rows where the bar reserves one, so the bar's *last* row wraps past the bottom of the screen and - **scrolls** it — the bar moves up and leaves a copy of itself above, one per bad render. That is the reported - "ganz viel kleiner / größer abwechselnd zerhackt alles", a screen carrying several complete bars at different - widths. The `…` some of those rules ended in is JLine's own marker for a row wider than the region, which is - what pointed at it. - **Padding one column short costs nothing** now that every row is addressed rather than reached by wrapping - (the sixth fix), and it additionally stops the terminal from marking the row as **wrapped**, which is what let - a reflow join it to its neighbour. Measured with a screen reporting one column more than it has — the lag a - dragged console really shows: the state row came out as `" ▤ X:/tmp/…"`, its `[` eaten by the wrap; with eight - columns over, eight characters. Both are the same thing the earlier `"state]"` reports were. - **Our own half of that race was measured and is already closed:** a harness whose `getSize()` returns a - different width on every call (a drag that stops) produces no ellipsis and a whole state row, because this - console reads the size **once** per block build and hands that one read to both the region and the rows. The - case is kept as the guard for it. - **What the harness cannot show is the stack itself** — `ScreenTerminal` does not reflow, and JLine's diff skips - a render whose content has not changed, so only the first bad render damages it. What it does show, a bottom - row eaten by a wrap, is the same mechanism one render at a time. - - **The redraw after the size SETTLES is a repaint, not a diff** (`repaintBlockFromScratch`, 400 ms after - the last size event). This is the fix for "kleiner ziehen sah gut aus, größer macht noch Probleme", and it - rests on a mechanism worth knowing: JLine pads every region row to the width the terminal **reports** and - writes the rows one after another, relying on the terminal wrapping at the right margin to start the next. - A screen that is **wider** than the reported width therefore never wraps, and the next row continues on - the same screen line — a rule with the activity row cut short beside it, the next rule where that left - off, and the *last* row still perfectly correct (the region addresses its first row and then writes on, so - only the rows in between collapse). Reproduced deterministically with a harness that reports eight columns - fewer than its screen has. - **Two causes reach that state and only one recovers by itself.** A console reporting a width it has not - applied does it briefly, and the existing rebuild repairs it (measured, including a window that also grows - a row taller). **Windows also reflows its screen buffer when the window is widened**, joining rows it had - marked as wrapped — which is every region row, since each is padded to the last column — and JLine is - never told. Its `Display` still matches what it wrote, so every later update computes an **empty diff and - emits nothing** and the joined rows stay for the rest of the session. That is the half no earlier theory - explained: the artefact *persisting* while every redraw runs, and repairing itself when Enter is held - (printing does not go through the region's diff at all). - **The repaint is two passes:** the same number of rows, blank, then the real rows — the first makes the - second a real write and erases the rows on the way. **`Status.reset()` is the call that looks right and is - wrong:** it forgets the scroll region too, so the next update believes it must grow the region and scrolls - to make room — the stale rows were pushed *up* rather than cleared and the block stood on screen **twice**, - four rows apart (measured on the interpreted screen, the only place that is visible). An empty block has - the same problem for the same reason: it changes the region's height. Only the settle redraw repaints; a - drag reports a size every ~125 ms and repainting on each is bytes spent against a screen about to change - again. - **What it cannot reach:** the prompt has a display of its own with the same diff and no `reset()` a caller - can call, so a reflow that damages the prompt's row is still beyond repair from here — - `thePromptItselfStaysVisibleAfterEnlarging` records that. `/cls` and Ctrl-L do repair it, because they - only print. - - **A settled WIDTH change wipes the screen, in EITHER direction; a height change does not.** This is the last of the drag defects and - the only one that could not be prevented, only removed. A console **reflows** when the width changes — measured - on the reporter's, with a number rather than a screenshot, because a pasted screen cannot answer it (Windows - Terminal copies the scrollback and rejoins wrapped runs, so wrapping is invisible in a paste): a 32-row window - widened from 86 to 111 columns left the cursor on **row 28 of 32** where 31 is the last, exactly the three rows - that joining three wrapped lines freed; narrowed to 72 it stayed on 31. So the **top** keeps its content and - everything below moves **up** — and the block's rows are ordinary screen rows, so they are carried out of the - region while the next render draws a fresh block at the bottom. The carried-up copy then sits above the region - where nothing writes again: one per size event, one every ~125 ms during a drag, which is "beim größer ziehen - wieder hunderte male die Linie". - **Both halves of "cannot be prevented" were measured.** Taking the block off the screen at the first event of a - drag fixes an *alternating* drag but not a plain shrink-then-widen: the console reflows immediately while this - console learns of the size up to 120 ms later, so the block is unavoidably on screen then. Nothing can find the - copy afterwards either — a caller cannot read the screen. Scrolling removes it, which is what `/cls` does and - why that command was always the repair. The trade was the user's call: the visible conversation scrolls out of - view on a width drag — and is then **printed again**, which is the part that makes the wipe affordable. - Every line handed to `line()` is remembered **before** folding (a bounded ring of the last 300), and after the - wipe as many of the most recent as fit above the block are printed again, oldest first, counted in **screen - rows** rather than lines because folding can turn one line into several. Storing the text rather than the - drawing buys what a reader expects from a window they just made bigger: the conversation **re-flows** — a line - that needed two rows in a narrow window takes one in a wide one - (`whatIsPrintedAgainIsFoldedForTheWidthTheWindowHasNOW` measures both states). It was wiped without this for - one round and the report was a single sentence: "allerdings sehe ich den Verlauf nicht mehr". A height change - needs none of this (no width change, nothing re-wraps) and keeps the conversation where it is, with the prompt - printed back to its row instead. - **The redraw only prints; it must not remember.** `line()` does both, and routing the redraw through it put the - whole visible conversation into the ring a second time on every wipe — one turn ended up on screen four times - ("der Text erscheint zwar wieder, aber mehrmals"). `remember` and `print` are separate for that reason. It also - takes **two** width changes to see it: after the first wipe the ring holds the line twice but only one copy has - been printed, which is why the case that shipped with the redraw was green while the console was not. - **It was narrowed to "only when the width grew" once, on reasoning rather than evidence, and the next report - came from the other direction:** narrowing does not join lines, it **splits** them, and the bar's own rows — - built for the old width — no longer fit and are re-wrapped across several screen rows, so the bar needs more - rows than the region reserves and everything above it is pushed up ("beim kleiner ziehen wandert es nach oben - mit ganz vielen Zeilen"). Both directions leave rows behind that only scrolling removes. Note what the wipe is - **not**: the eighth JLine fix stops the library from *erasing* those rows, which is unrecoverable, while - scrolling them into the scrollback is not — that fix still matters for every consumer that does not wipe. - - **A height change moves the prompt off its row, and the settle prints it back** - (`pushThePromptBackToItsRow`). Measured on the interpreted screen with the real three-row block: growing 20 - rows to 26 left the prompt on row 19 where 22 is right, shrinking 20 to 14 left it on row 13 where 10 is - right — **half the height change in both directions**, and "half" is an artefact of that buffer state rather - than a rule (the screen moves content by as many rows as it has scrollback to give or room to delete, which - is what a console does). **A width change moves nothing at all**, which is folding paying off, so nothing is - printed for one. - **The growing half is fixed and the reason it works is the direction:** the drift cannot be computed, but it - is bounded by the height change, and printing moves the cursor down one row per line until it reaches its row - and then merely scrolls — so printing that many lines lands it correctly without knowing where it was. It is - the same repair `/cls` performs with a whole window's worth, which is why that command always worked. - Measured 19 → 22; `aGrowingWindowLeavesThePromptOnTheRowTheBlockLeavesForIt` was the open reproduction and is - now green twice in a row. The price is bounded by the change: up to that many blank rows enter the - conversation and up to that many lines of it scroll out of view. - **The shrinking half stays open**, because there the prompt must move *up*: it lands inside the pinned band - and the block draws over it ("nach dem kleiner ziehen sehe ich es nicht mehr"). Nothing a caller can emit - moves the cursor up without breaking the reader's bookkeeping — `cursor_address` inside `printAbove` was tried - twice and stranded characters above the prompt both times. - `aWindowThatGetsSHORTERLeavesThePromptOnTheRowTheBlockLeavesForIt` carries it, `/cls` repairs it. - **Measuring the cursor was ruled out, not overlooked:** a cursor-position report is a round trip through the - terminal's input, which the reader owns all session, so it would race the keyboard and could swallow a - keystroke or leave `[24;1R` in the input line — the exact class of defect this whole investigation has been - about. - - **Do not add a `WINCH` handler, and the reason is measured.** A resize drawing a row of - `> > > > >` across the screen looks like the pinned region not being told about the new size, so a - `Signal.WINCH` handler that resized and re-rendered it was added — and the user reported it - **worse**, not better. `LineReaderImpl.handleSignal(WINCH)` already calls `Status.resize(Size)`, - and the reader installs its own handler for as long as it is reading, which is the whole session; - ours therefore either never ran or ran *in addition*, putting a second writer on the terminal from - the signal thread at the exact moment the reader was redrawing. A probe driving a real - `terminal.raise(WINCH)` against a **pipe-backed** terminal shows JLine doing it correctly on its - own: scroll region reset, the rule re-cut to the new width, **one** prompt — but that probe was - measuring the wrong thing, because a pipe has no screen to redraw onto. Against JLine's own - `VirtualTerminal` (a real VT interpreter over a virtual screen) the same drag reproduces the report - exactly, on `xterm` and `windows-vtp` alike, and the cause is a **one-line JLine defect**: - `handleSignal(WINCH)`'s status branch calls `doDisplay()`, which replaces the `Display` with a fresh - one that believes the screen is blank, so the following `redisplay()` paints the prompt as new - content once per size event instead of as a diff (`display.resize(size)` is the fix, verified with - three new tests plus JLine's 72 existing ones). Still nothing to fix *here*, and no project test may - assert the fixed behaviour while the build depends on an unfixed release. - `fit()` measuring in **screen columns** (`AttributedString.columnLength`) rather than characters is - ours and does matter: an icon is one character and two columns, and a row wider than the window - wraps onto a second screen line, which the reserved region cannot survive. - - **Two things tried and removed, both because measurement said they did nothing.** (1) Skipping a - status write when the block is unchanged — removing the guard again left the emitted bytes - identical, because JLine already skips an unchanged block. (2) Re-cutting the rows when the window - width changed — `Status.resize()` re-cuts the rows it holds itself. Neither was kept with a comment - claiming a benefit it does not have. The stray `?1h` consequently still has **no established - cause**: the lock covers our writes, the reader's own are inside JLine. - - **Restoring the block after a wipe took three steps — and the whole repair went away with the wipe.** - It is kept as a record because it is the same mechanism that eventually retired the wipe itself. The - steps were `status.reset()`, then `status.update(List.of())`, then rendering it again from `requested` - (the text the caller gave, not the rendered rows); with only the first two, `Status` drew the - difference it computed against a belief the wipe had invalidated — observed as a single character - emitted where a whole block was missing. The reader's own prompt has exactly that problem and no - equivalent repair, which is why an erase left no prompt on screen at all. A clear that only scrolls - invalidates nothing, so the block is simply left alone. - - **The turn after an interrupted one is pinned end to end** (`InterruptedTurnTest`): with a scripted - backend behind the real `OpenAiCompatServer`, a turn is cut short by pending input and the next one - is then driven through — it reaches the server, carries the interrupted question in its history, - and answers. Reported as "it does not carry on by itself"; the mechanism works, so the cause of - that report is elsewhere and is **not** claimed to be fixed. Two things the writing of it settled: - a turn that finishes inside one activity tick is never even looked at for interruption (correct — - there is nothing to cut short), which is why the scripted backend has to be made slow or the test - proves nothing; and a second line typed during the replacement turn stops that one too, which is - the design and not a defect, but looks from the outside exactly like a turn that never started. - - **A blank line must not count as pending input.** `hasPendingInput()` ignores blank lines but leaves - them queued: counting them meant that holding Enter cancelled one turn per keystroke and produced - nothing, while dropping them would break the approval prompt, where an empty answer means yes. - `DISABLE_EVENT_EXPANSION` is set in the same builder because the reader's default treats `!` as a - shell history expansion, which silently rewrites a request like `git commit -m "fixed!"`. - **What cannot be done, asked and answered:** keep the block visible while the *user* scrolls the - terminal's scrollback. That needs the alternate screen buffer, i.e. a full-screen application, which - would give up the scrollback and the append-only property the whole console design rests on. - - **The prompt stays at the bottom during a turn, and typing stops the turn.** One thread inside - `JLineTerminal` (`startReading`) sits in `readLine` for the whole session and fills a queue; - **every** read in that class is served from it, because a terminal has one keyboard and two threads - reading it take turns at random. That is why `readKey` no longer reads a single key in raw mode: the - question is printed above the prompt and answered in the same input line (`y` + Enter). End of input - cannot be a queue value, so a sentinel is queued and **put back on every take** — otherwise the - second reader after Ctrl-D would see "nothing typed yet" instead of "no more input". - `AgentTerminal.hasPendingInput()` is the peek the turn loop peeks with; it deliberately does **not** - consume, so the line the interruption was triggered by is still there for the next `readLine` and - becomes the next message. The stop itself is `AgentRunner.start(...)` → - `runtime.executeWithHandle(...)`, whose handle closes the in-flight SSE stream (Atmosphere's own - "D-6 built-in hard-cancel"); that also replaced `turn()`'s hand-rolled worker thread, since - `executeWithHandle` dispatches on a virtual thread and returns at once. `awaitWithActivity` returns a - three-valued `TurnEnd` rather than a boolean, because *interrupted* must not be reported as the - *timed out* error the old `false` produced. **Order in that loop is load-bearing and a test pins the - behaviour**: the `activity.isPaused()` check comes first, so while an approval question is open a - typed line is its answer and not an interruption. **What this is not:** Claude Code injects a - mid-turn message into the running loop; `AgentExecutionContext` is a record whose request is built - once from `message()` + `history()`, with nothing to append to, so stop-and-resend is the achievable - equivalent — and it acts immediately instead of waiting out a tool loop. - -**Three front ends on one `AgentSession`: console, `--web`, `--acp`.** Everything that is not -presentation lives in `AgentSession` — history, the slash commands (`submit` returns `CONTINUE`/`EXIT`), -`/compact`, `/loop`, `/retry`, the transcript, the approval mode, cancellation (`cancel()`, -`stopRequested()`, `isBusy()`), and `runTurn` itself. A front end implements `SessionFrontend`: `line`, -a `renderer()` (a `StreamingSession` the turn forwards every event to), `await` (how it waits for a -turn — the console with its activity block, the others with `AgentSession.awaitQuietly`), `approvals()`, -`ask()` (a free question; `null` means the front end has none, and `/loop` then refuses rather than -guesses) and `commandOutput()` (a running command's lines). The console is `ConsoleFrontend` over the two -`AgentTerminal`s; `LocalAgent` is now only option parsing, the choice of front end and the console's -activity rendering. Three pieces are what keeps them from drifting apart: - -- **`TurnRecorder` records, the front end renders.** The recorder is the `StreamingSession` handed to - Atmosphere; it keeps text, tool rounds, token counts and the running tool, supplies the workspace - filesystem via `injectables()`, and forwards everything downstream. `ConsoleSession` is now just a - recorder with a `ConsoleRenderer`. So the tool note in the history, `/calls` and the transcript are - identical whichever front end showed the turn. -- **`ModeGatedApprovalStrategy` wraps every front end's strategy.** Atmosphere asks the strategy per - call; the wrapper reads the session's mode *at that moment*, so `/mode auto` in the browser, *Allow - all* in an editor and Shift+Tab in the console all take effect for the very next call, and a front - end with nobody to ask (`approvals()` = `null`) denies. `ConsoleApprovalStrategy` keeps its own AUTO - check as well; it is redundant under the wrapper and harmless. -- **`AgentSessionTest` + `RecordingFrontend`** drive the session with no terminal, no browser and no - editor, over the real `OpenAiCompatServer` and a scripted engine; the three front-end test classes - then only need to prove their own mapping. - -**`--web` (`WebServer`, `WebAgentEndpoint`, `WebFrontend`, `WebAccessGuard`, `WebConsole`).** Embedded -Jetty 12 (ee10 servlet + Jakarta WebSocket; `jetty-ee10-annotations` excluded, nothing here needs -annotation scanning) with an `AtmosphereServlet` whose annotation map is set **explicitly** to the one -`@AiEndpoint`. **The scanning trap, which cost an afternoon:** Atmosphere's `ClasspathScanner.preventOOM()` -turns classpath scanning **off** whenever `org.junit.jupiter.api.Assertions` is on the classpath, and in -servlet-initializer mode the explicit map is only processed *during* a scan — so the endpoint registered -fine from the jar and was missing in every test ("No AtmosphereHandler or WebSocketHandler installed"). -The init params `ANNOTATION_PACKAGE="all"` and `SCAN_CLASSPATH="false"` make the explicit map the path in -both. One `AgentSession` per process, shared by every tab (`WebAgentEndpoint.SESSION`); a message while a -turn runs cancels it first, `/stop` only cancels, and `connection.complete()` is always sent so the -console's input unlocks. Approvals use Atmosphere's own `/__approval//approve|deny` protocol through -`ApprovalStrategy.virtualThread(registry)`, which the prebuilt console renders as Approve / Deny. -`WebAccessGuard` is a Jetty `Handler.Wrapper` in front of everything: loopback `Host` when bound to -loopback (DNS rebinding), `Origin` must match `Host`, `?token=` exchanged for an `HttpOnly; -SameSite=Strict` cookie plus a redirect, else `Bearer` or cookie, else `401` — constant-time comparison. -`WebConsole` serves the console pages with a fresh CSP nonce per response and answers -`/api/console/info`. `WebServerTest` (11) drives it over the JDK `WebSocket` speaking the atmosphere.js -wire protocol (`AtmosphereTestClient`: `len|payload` frames, `X-atmo-protocol`); **create the client -object before `buildAsync`**, or the handshake message races the listener into a thrown exception and -`request(1)` is never called — a flaky test that looked like a server bug. - -**`--acp` (`AcpServer`, `AcpFrontend`).** The ACP Java SDK (`com.agentclientprotocol:acp-core` + -`acp-json-jackson2`, 0.18.0) over `StdioAcpAgentTransport`. Its sync handlers run on the SDK's own pool, -so a blocking `session/prompt` does not block `session/cancel` or the answer to a permission request. -One `AgentSession` per ACP session, built for the editor's `cwd` (`AgentOptions.withWorkspace`); the -model endpoint is opened once per process. **stdout is the protocol**: `run()` keeps the real stdout for -the transport and points `System.out` at stderr, so a stray `println` cannot corrupt the stream (the -smoke asserts every stdout line is JSON). Mapping: text → `agent_message_chunk`; `ToolStart` → -`tool_call` (`call_N`, `ToolKind` from the tool name, `locations` resolved against the workspace, raw -arguments), `ToolResult`/`ToolError` → `tool_call_update` (result cut at 4000 chars), running command -output → `tool_call_update` IN_PROGRESS at most every 500 ms with the last 40 lines; approvals → -`session/request_permission` with allow / always / reject, where *always* switches the session to AUTO -and sends `current_mode_update`, and no answer (timeout, cancelled dialog, editor gone) is a deny; the -approval modes are ACP session modes `manual`/`auto`; the slash commands (minus `/exit`, `/cls`) go out as -`available_commands_update` 100 ms after `session/new` on a virtual thread, because an update that -arrives before the `session/new` response names a session the editor does not know yet. -`AcpServerTest` (9) deliberately drives it with **plain JSON** (`AcpTestClient`), not the SDK's client — -an SDK on both ends agrees with itself even where it disagrees with the protocol. Tear-down closes the -editor side first: that is how an ACP session ends, and closing the server first logs an -`InterruptedIOException` "Transport error" per test. - -**Two defects fixed on the way (step 0), both pinned in `LocalAgentTest`:** the shell tool's live-output -sink captured the console *before* it existed, so every command's output came back as -`[output unavailable: NullPointerException]` (now an `AtomicReference` set once the terminal is up); and -`/clear` left the pending tool note and the message `/retry` would resend, so the first turn after a -clear still carried the old conversation. - -**The default system prompt is general-purpose on purpose — do not narrow it back.** Every model-facing -text is a resource, not a Java literal: `src/main/resources/net/ladenthin/llama/atmosphere/` holds -`system-prompt.txt`, `system-prompt-shell.txt`, `system-prompt-no-shell.txt` and `run-command-tool.txt` -(each with a `.license` sidecar for REUSE), loaded by `LocalAgent.prompt(name)` with `{placeholder}` -substitution; `AgentOptionsTest.promptResourcesLoadAndEveryPlaceholderIsFilled` fails on a missing file or -an unfilled placeholder. `LocalAgent.systemPrompt` -and the `ShellTool` description describe `run_command` as running *any* command line through the named -shell (`ShellTool.shellName()`), not limited to the workspace, and tell the model to run a command rather -than explain one. The earlier wording ("careful *coding agent*", `run_command` "to build, test or inspect -the project" / "build, test, grep or list files") made Qwen3-4B refuse "list the docker images" — "my -tools are only for files" — with the tool registered and `docker` on `PATH`; a fresh single-turn run -refused too, so it was the prompt, not the chat history. Without `--allow-shell` the prompt says commands -are unavailable and names the flag, so the model does not invent its own limitation. Pinned by -`AgentOptionsTest.defaultSystemPromptIsGeneralPurposeAndAllowsAnyCommandWithTheShell` and -`shellToolDescriptionDoesNotNarrowItToTheProject`. `ShellToolTest` runs on every platform: each test -picks its command line with `ShellTool.isWindows()` — the same detection `ShellTool.run` uses to choose -`cmd.exe /c` over `sh -c` — so `ls`/`dir /b`, `sleep 30`/`ping -n 30 127.0.0.1 >nul`, and the truncation -test counts the platform's line separator. It used plain POSIX commands before and failed 4 of 5 on -Windows; never skip it per OS, give a new test both command forms instead. - -**Version bump note.** The pom's `llama.version` property is the **release** version, not the -reactor's `-SNAPSHOT` (CI always overrides it, so a not-yet-published default never breaks CI). -`versions:set` does not touch this standalone pom, so at release time bump it by hand together with -the fat-jar filename `llama--jar-with-dependencies.jar` in the two READMEs (`README.md` -"Local coding agent" + the project's own README) — the same class as the `llama-langchain4j/README.md` -snippet. Before the release, run it against a pre-release core with `-Dllama.version=-SNAPSHOT` -(the pom keeps the Sonatype snapshot repository for exactly that). +A copy-and-run terminal agent (console, `--web`, `--acp`) pairing Atmosphere's OpenAI-compatible +agent runtime with this project's `OpenAiCompatServer`. A **standalone Maven project** (Java 21, +Atmosphere's floor), **not** a reactor module and **never** on Maven Central. Everything about the +agent itself -- the REPL, approval gate, file tools, JLine console and its eight carried JLine fixes, +the three front ends -- is in **[`llama-atmosphere-agent/CLAUDE.md`](llama-atmosphere-agent/CLAUDE.md)**, +which Claude Code loads when working in that directory. What matters from the rest of the repository: + +- **Release asset:** `llama-atmosphere-agent--jar-with-dependencies.jar`, built + **without** the core; its manifest `Class-Path` names the four `all--` fat jars and the + default fat jar. Renaming a core fat jar means updating that list -- `check-natives.py` holds it + to the fat-jar targets of `natives.csv`, and `smoke-agent-linux` launches the pair. +- **CI:** the model-free job, the model-backed integration job and `smoke-agent-linux` all gate both + publish jobs. +- **Version bump:** the pom's `llama.version` is the **release** version and `versions:set` does not + touch this standalone pom -- bump it by hand with the fat-jar filename in the two READMEs. +- **The one core change it needed:** `OpenAiBackend`, `ChunkSink` and + `OpenAiCompatServer(OpenAiBackend, OpenAiServerConfig)` are public, so the agent's tests can drive + the real server without a model. Keep them public. ## Android app "LLM Service" (`android-llmservice/`) -A shippable, **KISS fully-offline on-device chat app** consuming the `llama-android` AAR + -`llama-kotlin` façade. App label **"LLM Service"**, applicationId/namespace/package -**`net.ladenthin.android.llmservice`**. One Compose screen: pick a GGUF from the file system -(Storage Access Framework), stream a chat reply fully on-device, **tune sampling** via a Settings -sheet (⚙️ — temperature/Top-K/Top-P/Min-P/**repeat penalty**/repeat range/max tokens; the -repeat-penalty default is what stops the small-model repetition loop), **Stop/Copy/Regenerate/Clear** -the chat, watch an **in-app log** (bottom strip + full viewer with copy-all / save-as-`.txt` / clear), -switch UI language via a **flag picker (13 languages)**, and **Save/Load** the conversation to -**private local storage**. No `INTERNET`/storage permission — nothing leaves the device. Like `llama-android/` and -`.github/android-consumer-test/`, it is a **standalone plain-Gradle/AGP build, NOT a Maven -reactor module** (and NOT published to Maven Central — it is an app, not a library). It is the -"real app" counterpart to the headless `.github/android-consumer-test/` fixture (which only -compiles/loads the API); this one has a UI and is driven end-to-end. The name is intentionally -**generic/descriptive** (sidesteps trademark conflicts); the applicationId under the owner's -domain is the only string that must be globally unique. - -Structure (mirrors the consumer-test's plumbing): -- **`settings.gradle.kts`** — `rootProject.name = "android-llmservice"`; pins AGP `9.4.0` + the - Compose compiler plugin (`2.4.20`); `mavenLocal()` first so the freshly-built AAR + façade - resolve there in CI (Maven Central for real users). Stay at `>= 9.2.1`: `9.2.1` (not `9.2.0`) - first fixed a real R8 regression (`ClassNotFoundException` on `com.android.tools.r8.RecordTag` - after upgrading Gradle to 9.x with AGP 9.2.0) that hits this project directly since - `buildTypes.release` sets `isMinifyEnabled = true`; `9.4.0` carries that fix forward and the - `.github/android-consumer-test` fixture is pinned the same way. AGP 9.4.x requires Gradle >= 9.6.0 - and JDK 17+; CI already runs JDK 21 everywhere (`.java-version`), so only the `gradle-version` - pin on the jobs that build this project (and the `.github/android-consumer-test` fixture) - needed bumping (currently `9.8.0`). AGP 9.0+ has **built-in Kotlin support** (a runtime dependency on - Kotlin Gradle plugin 2.2.10+), so the standalone `org.jetbrains.kotlin.android` plugin is no longer - applied — applying it now fails the build with "no longer required for Kotlin support since - AGP 9.0" (`app/build.gradle.kts` line 7). The Compose compiler plugin still applies - separately and its 2.4.20 pin exceeds AGP's 2.2.10 floor, so no other version changed. -- **`app/build.gradle.kts`** — `namespace`/`applicationId` `net.ladenthin.android.llmservice`, - `minSdk 28` (AAR floor), `compileSdk 37` (raised from 35 — Compose/lifecycle/activity AAR - metadata now requires it), `targetSdk 35`, Jetpack Compose, `androidx.appcompat` (only for - the per-app language API), ABIs `arm64-v8a` + `x86_64`. Depends on `net.ladenthin:llama-android` + - `net.ladenthin:llama-kotlin` at `-PjllamaVersion=` (defaults to the last release - when built by hand). The release `signingConfig` reads an **upload keystore** (env vars - `JLLAMA_UPLOAD_STORE_FILE` / `_STORE_PASSWORD` / `_KEY_ALIAS` / `_KEY_PASSWORD`, or the matching `-P` - props) and **falls back to debug signing** when none is set, so forks/PRs/local builds stay green. -- **i18n:** every UI string is a resource; translations ship for **13 languages** - (`values-{de,es,fr,it,pt,ru,tr,ar,hi,zh-rCN,ja,ko}/strings.xml` + default English), listed in - `res/xml/locales_config.xml` and `Languages.kt`. The flag dropdown switches language in-app via - `AppCompatDelegate.setApplicationLocales` (AppCompat `autoStoreLocales` service persists it; - `MainActivity` extends `AppCompatActivity`, theme `Theme.LlmService` = AppCompat DayNight). The - ViewModel survives the locale-change recreation, so the chat isn't lost. `app_name` ("LLM Service") - is the one string deliberately NOT translated. -- **`ChatViewModel.kt`** — the logic: loads a `LlamaModel` (SAF `content://` copied into `filesDir` - because llama.cpp mmaps a real path), streams via `generateChatFlow` and forwards the `GenerationSettings` - sampling knobs to `InferenceParameters`; supports **stop** (`generation?.cancel()` — cancellation keeps - the partial reply, not an error) and **regenerate** (drop the trailing reply, re-run the last user turn, - shared `startGeneration` path); keeps a capped rolling in-app **log**. **`SessionStore.kt`** — private - local save/load: the conversation + model path as JSON in `filesDir/session.json` (`org.json`, no dep - added), readable only by the app. **`LlmServiceApp.kt`** — `Application` that **wipes the transient - working data** (the copied model `current-model.gguf` + cache) on every **cold start** - (`onCreate`) — a privacy guarantee independent of the OS calling `onDestroy`; `MainActivity` also - wipes best-effort on finish. Only the **opt-in saved session** persists. **`MainActivity.kt`** — - Compose UI with a **two-row app bar** (row 1 = horizontally-scrollable model name + a ❌ unload - button when a model is loaded, row 2 = all action icons) + SAF picker + flag menu + Save/Load + the ⚙️ Settings sheet - (sampling knobs + a **Model** section: CPU threads / context length applied via **Reload model**) + - the 🧾 log strip/viewer + Stop/Copy/Regenerate/Clear chat actions (+ **long-press any bubble to - copy**) + **prompt shortcut chips** + a **device-readiness card** (free RAM / storage / battery via - `DeviceInfo`) on the picker + the offline badge; reads optional `MODEL_PATH` / `CHAT_TEMPLATE` - intent extras as a **test hook** (the shipping UI never sets them). -- **`app/src/androidTest/kotlin/.../ChatFlowInstrumentedTest.kt`** — the real end-to-end test: - launches the activity with a preloaded model (bypassing the system SAF dialog, which only - UiAutomator could drive), types a prompt, taps Send, asserts a non-empty streamed reply. - Self-skips (JUnit `Assume`) when the adb-pushed model is absent. - -**Requirements / roadmap docs (keep in sync with the code).** The app has **no unit tests** for its -logic (only the one on-device `ChatFlowInstrumentedTest` above), so -[`android-llmservice/requirements.md`](android-llmservice/requirements.md) is the **spec of record** — -it enumerates, with stable `Rn.m` IDs, every feature the app implements today. Any change to app -behavior must update `requirements.md` in the same commit (add/amend/retire a row); the roadmap of -not-yet-built features lives in [`android-llmservice/TODO.md`](android-llmservice/TODO.md), and -`README.md`'s feature claims must match `requirements.md`. - -**Recommended real model: Gemma 3 4B Instruct** (a `*-it` GGUF; carries its own chat template, so -no override needed). Any instruct GGUF works. The CI on-device test uses the tiny cached draft model -with a forced `chatml` template just to prove tokens generate. - -**Key signing fact:** the Maven Central **GPG key cannot sign an APK/AAB** (different -cryptosystem). Android needs a Java keystore (PKCS12/JKS + RSA) **upload key**; Play App Signing -manages the final app-signing key. CI wires it from the optional `ANDROID_UPLOAD_KEYSTORE_BASE64` -+ password/alias secrets. - -**CI wiring** — **two split jobs** in `.github/workflows/publish.yml` (both `needs:` the two Android -native jobs; the test job additionally `needs: verify-model-cache`), so the installable artifacts are -**always** produced even when the on-device UI test is flaky/slow: -- **`build-android-llmservice`** — installs the core + `llama-kotlin` to mavenLocal (`llama-kotlin`'s - core dep is provided-scope, so the desktop JAR is never pulled into the APK), stages the CI-built - Android natives, publishes the CPU AAR to mavenLocal, builds the **release `.aab`** (signed with the - upload key when the `ANDROID_UPLOAD_KEYSTORE_BASE64` secret is present, else debug-signed) + the - installable **`.apk`**, and uploads them (`android-llmservice-aab` / `android-llmservice-apk`). - **No emulator** — a flaky emulator can never block getting the APK. -- **`test-android-llmservice`** — a **separate, non-gating** check that repeats the setup, then boots - the KVM x86_64 emulator and runs the Compose UI test via **`.github/run-android-llmservice-test.sh`** - (adb-pushes the cached `AMD-Llama-135m` draft model, `connectedDebugAndroidTest`). It can go **red on - its own** without stopping the build/artifacts; keep it **non-required** in branch protection to keep - it optional (visible-but-non-blocking). - -Both emulator jobs (`test-android-llmservice` + `test-android-emulator`) prepend a **free-disk step** -before the emulator (delete every restored model except `DRAFT_MODEL_NAME` + large unused preinstalled -toolchains — the latter via `ggml-org/free-disk-space` with `android`, `large-packages`, `tool-cache` -and `swap-storage` switched **off**, since the emulator needs the SDK, mesa and the JDK) because the AVD userdata partition needs ~7.4 GB and the full ~10 GB GGUF cache restore -otherwise FATALs the emulator ("Not enough space to create userdata partition"). Neither -llmservice job is **yet a publish gate** (not in the `publish-snapshot`/`publish-release` `needs:` -graphs) so a Compose/AGP version-pin hiccup can't block a library release. -`android-llmservice/**/build/` is git-ignored. +A fully-offline on-device chat app consuming the `llama-android` AAR + `llama-kotlin` façade +(applicationId `net.ladenthin.android.llmservice`), a **standalone Gradle/AGP build**, not a reactor +module and not published to Maven Central. Details -- structure, AGP/Gradle pins, signing, i18n -- +are in **[`android-llmservice/CLAUDE.md`](android-llmservice/CLAUDE.md)**. Two rules that reach +outside the directory: **`requirements.md` is the spec of record** (any behaviour change updates it in +the same commit), and its two CI jobs, `build-android-llmservice` and `test-android-llmservice`, are +**not** publish gates (both are named with a reason in `.github/release-gate-exemptions.txt`). ## Dependency Convergence Pinning diff --git a/REUSE.toml b/REUSE.toml index 950bce38d..9c6a5b66d 100644 --- a/REUSE.toml +++ b/REUSE.toml @@ -10,6 +10,8 @@ path = [ "SECURITY.md", "SUPPORT.md", "CLAUDE.md", + "llama-atmosphere-agent/CLAUDE.md", + "android-llmservice/CLAUDE.md", "CLAUDE_TEST_GENERATION_PROMPT.md", "models/README.md", "docs/RELEASE.md", diff --git a/TODO.md b/TODO.md index 427b9180e..55d66b1ad 100644 --- a/TODO.md +++ b/TODO.md @@ -219,7 +219,7 @@ the patch was ever filed, so the `ContentOnlyParseUtf8` guard now pins upstream' `0010` followed at b11080: upstream gave `common_json_value` an enum constructor via ggml-org/llama.cpp#28518, fixing at the root the enum-to-bool trap the patch cast around, so it became a redundant carry — note that it still *applied* cleanly, which is why the by-hand drop-check exists. -All four drops are recorded in `CLAUDE.md` under the patch table.) +All four drops are recorded in `docs/history/dropped-llama-patches.md`.) ### llama.cpp upstream feature exposure (queued, deferred by policy) diff --git a/android-llmservice/CLAUDE.md b/android-llmservice/CLAUDE.md new file mode 100644 index 000000000..6f15732a7 --- /dev/null +++ b/android-llmservice/CLAUDE.md @@ -0,0 +1,114 @@ +# CLAUDE.md — `android-llmservice/` (the "LLM Service" Android app) + +Guidance for working in this directory. Claude Code loads it when files here are read; the +repository-wide rules are in [`../CLAUDE.md`](../CLAUDE.md), which keeps a short summary. + + +A shippable, **KISS fully-offline on-device chat app** consuming the `llama-android` AAR + +`llama-kotlin` façade. App label **"LLM Service"**, applicationId/namespace/package +**`net.ladenthin.android.llmservice`**. One Compose screen: pick a GGUF from the file system +(Storage Access Framework), stream a chat reply fully on-device, **tune sampling** via a Settings +sheet (⚙️ — temperature/Top-K/Top-P/Min-P/**repeat penalty**/repeat range/max tokens; the +repeat-penalty default is what stops the small-model repetition loop), **Stop/Copy/Regenerate/Clear** +the chat, watch an **in-app log** (bottom strip + full viewer with copy-all / save-as-`.txt` / clear), +switch UI language via a **flag picker (13 languages)**, and **Save/Load** the conversation to +**private local storage**. No `INTERNET`/storage permission — nothing leaves the device. Like `llama-android/` and +`.github/android-consumer-test/`, it is a **standalone plain-Gradle/AGP build, NOT a Maven +reactor module** (and NOT published to Maven Central — it is an app, not a library). It is the +"real app" counterpart to the headless `.github/android-consumer-test/` fixture (which only +compiles/loads the API); this one has a UI and is driven end-to-end. The name is intentionally +**generic/descriptive** (sidesteps trademark conflicts); the applicationId under the owner's +domain is the only string that must be globally unique. + +Structure (mirrors the consumer-test's plumbing): +- **`settings.gradle.kts`** — `rootProject.name = "android-llmservice"`; pins AGP `9.4.0` + the + Compose compiler plugin (`2.4.20`); `mavenLocal()` first so the freshly-built AAR + façade + resolve there in CI (Maven Central for real users). Stay at `>= 9.2.1`: `9.2.1` (not `9.2.0`) + first fixed a real R8 regression (`ClassNotFoundException` on `com.android.tools.r8.RecordTag` + after upgrading Gradle to 9.x with AGP 9.2.0) that hits this project directly since + `buildTypes.release` sets `isMinifyEnabled = true`; `9.4.0` carries that fix forward and the + `.github/android-consumer-test` fixture is pinned the same way. AGP 9.4.x requires Gradle >= 9.6.0 + and JDK 17+; CI already runs JDK 21 everywhere (`.java-version`), so only the `gradle-version` + pin on the jobs that build this project (and the `.github/android-consumer-test` fixture) + needed bumping (currently `9.8.0`). AGP 9.0+ has **built-in Kotlin support** (a runtime dependency on + Kotlin Gradle plugin 2.2.10+), so the standalone `org.jetbrains.kotlin.android` plugin is no longer + applied — applying it now fails the build with "no longer required for Kotlin support since + AGP 9.0" (`app/build.gradle.kts` line 7). The Compose compiler plugin still applies + separately and its 2.4.20 pin exceeds AGP's 2.2.10 floor, so no other version changed. +- **`app/build.gradle.kts`** — `namespace`/`applicationId` `net.ladenthin.android.llmservice`, + `minSdk 28` (AAR floor), `compileSdk 37` (raised from 35 — Compose/lifecycle/activity AAR + metadata now requires it), `targetSdk 35`, Jetpack Compose, `androidx.appcompat` (only for + the per-app language API), ABIs `arm64-v8a` + `x86_64`. Depends on `net.ladenthin:llama-android` + + `net.ladenthin:llama-kotlin` at `-PjllamaVersion=` (defaults to the last release + when built by hand). The release `signingConfig` reads an **upload keystore** (env vars + `JLLAMA_UPLOAD_STORE_FILE` / `_STORE_PASSWORD` / `_KEY_ALIAS` / `_KEY_PASSWORD`, or the matching `-P` + props) and **falls back to debug signing** when none is set, so forks/PRs/local builds stay green. +- **i18n:** every UI string is a resource; translations ship for **13 languages** + (`values-{de,es,fr,it,pt,ru,tr,ar,hi,zh-rCN,ja,ko}/strings.xml` + default English), listed in + `res/xml/locales_config.xml` and `Languages.kt`. The flag dropdown switches language in-app via + `AppCompatDelegate.setApplicationLocales` (AppCompat `autoStoreLocales` service persists it; + `MainActivity` extends `AppCompatActivity`, theme `Theme.LlmService` = AppCompat DayNight). The + ViewModel survives the locale-change recreation, so the chat isn't lost. `app_name` ("LLM Service") + is the one string deliberately NOT translated. +- **`ChatViewModel.kt`** — the logic: loads a `LlamaModel` (SAF `content://` copied into `filesDir` + because llama.cpp mmaps a real path), streams via `generateChatFlow` and forwards the `GenerationSettings` + sampling knobs to `InferenceParameters`; supports **stop** (`generation?.cancel()` — cancellation keeps + the partial reply, not an error) and **regenerate** (drop the trailing reply, re-run the last user turn, + shared `startGeneration` path); keeps a capped rolling in-app **log**. **`SessionStore.kt`** — private + local save/load: the conversation + model path as JSON in `filesDir/session.json` (`org.json`, no dep + added), readable only by the app. **`LlmServiceApp.kt`** — `Application` that **wipes the transient + working data** (the copied model `current-model.gguf` + cache) on every **cold start** + (`onCreate`) — a privacy guarantee independent of the OS calling `onDestroy`; `MainActivity` also + wipes best-effort on finish. Only the **opt-in saved session** persists. **`MainActivity.kt`** — + Compose UI with a **two-row app bar** (row 1 = horizontally-scrollable model name + a ❌ unload + button when a model is loaded, row 2 = all action icons) + SAF picker + flag menu + Save/Load + the ⚙️ Settings sheet + (sampling knobs + a **Model** section: CPU threads / context length applied via **Reload model**) + + the 🧾 log strip/viewer + Stop/Copy/Regenerate/Clear chat actions (+ **long-press any bubble to + copy**) + **prompt shortcut chips** + a **device-readiness card** (free RAM / storage / battery via + `DeviceInfo`) on the picker + the offline badge; reads optional `MODEL_PATH` / `CHAT_TEMPLATE` + intent extras as a **test hook** (the shipping UI never sets them). +- **`app/src/androidTest/kotlin/.../ChatFlowInstrumentedTest.kt`** — the real end-to-end test: + launches the activity with a preloaded model (bypassing the system SAF dialog, which only + UiAutomator could drive), types a prompt, taps Send, asserts a non-empty streamed reply. + Self-skips (JUnit `Assume`) when the adb-pushed model is absent. + +**Requirements / roadmap docs (keep in sync with the code).** The app has **no unit tests** for its +logic (only the one on-device `ChatFlowInstrumentedTest` above), so +[`android-llmservice/requirements.md`](requirements.md) is the **spec of record** — +it enumerates, with stable `Rn.m` IDs, every feature the app implements today. Any change to app +behavior must update `requirements.md` in the same commit (add/amend/retire a row); the roadmap of +not-yet-built features lives in [`android-llmservice/TODO.md`](TODO.md), and +`README.md`'s feature claims must match `requirements.md`. + +**Recommended real model: Gemma 3 4B Instruct** (a `*-it` GGUF; carries its own chat template, so +no override needed). Any instruct GGUF works. The CI on-device test uses the tiny cached draft model +with a forced `chatml` template just to prove tokens generate. + +**Key signing fact:** the Maven Central **GPG key cannot sign an APK/AAB** (different +cryptosystem). Android needs a Java keystore (PKCS12/JKS + RSA) **upload key**; Play App Signing +manages the final app-signing key. CI wires it from the optional `ANDROID_UPLOAD_KEYSTORE_BASE64` ++ password/alias secrets. + +**CI wiring** — **two split jobs** in `.github/workflows/publish.yml` (both `needs:` the two Android +native jobs; the test job additionally `needs: verify-model-cache`), so the installable artifacts are +**always** produced even when the on-device UI test is flaky/slow: +- **`build-android-llmservice`** — installs the core + `llama-kotlin` to mavenLocal (`llama-kotlin`'s + core dep is provided-scope, so the desktop JAR is never pulled into the APK), stages the CI-built + Android natives, publishes the CPU AAR to mavenLocal, builds the **release `.aab`** (signed with the + upload key when the `ANDROID_UPLOAD_KEYSTORE_BASE64` secret is present, else debug-signed) + the + installable **`.apk`**, and uploads them (`android-llmservice-aab` / `android-llmservice-apk`). + **No emulator** — a flaky emulator can never block getting the APK. +- **`test-android-llmservice`** — a **separate, non-gating** check that repeats the setup, then boots + the KVM x86_64 emulator and runs the Compose UI test via **`.github/run-android-llmservice-test.sh`** + (adb-pushes the cached `AMD-Llama-135m` draft model, `connectedDebugAndroidTest`). It can go **red on + its own** without stopping the build/artifacts; keep it **non-required** in branch protection to keep + it optional (visible-but-non-blocking). + +Both emulator jobs (`test-android-llmservice` + `test-android-emulator`) prepend a **free-disk step** +before the emulator (delete every restored model except `DRAFT_MODEL_NAME` + large unused preinstalled +toolchains — the latter via `ggml-org/free-disk-space` with `android`, `large-packages`, `tool-cache` +and `swap-storage` switched **off**, since the emulator needs the SDK, mesa and the JDK) because the AVD userdata partition needs ~7.4 GB and the full ~10 GB GGUF cache restore +otherwise FATALs the emulator ("Not enough space to create userdata partition"). Neither +llmservice job is **yet a publish gate** (not in the `publish-snapshot`/`publish-release` `needs:` +graphs) so a Compose/AGP version-pin hiccup can't block a library release. +`android-llmservice/**/build/` is git-ignored. diff --git a/docs/history/dropped-llama-patches.md b/docs/history/dropped-llama-patches.md new file mode 100644 index 000000000..9dcb7904f --- /dev/null +++ b/docs/history/dropped-llama-patches.md @@ -0,0 +1,128 @@ + + +# Dropped llama.cpp patches + +The local patches under `llama/patches/` that were retired because upstream fixed the defect +themselves. Moved here from `CLAUDE.md`, which lists only the patches still carried. Each note +records why the patch went and what guard, if any, stays behind -- read the matching note before +reintroducing a patch for the same defect. + +**`0010` was dropped at the b11080 bump.** Upstream merged +[ggml-org/llama.cpp#28518](https://github.com/ggml-org/llama.cpp/pull/28518) +("json: Fixed json enum handling", first tagged at **b11080**), which fixes the defect at its +root instead of at the emit site: `common_json_value` gains an `std::is_enum`-gated constructor +that delegates to `std::underlying_type`, and `common_json_is_value` now accepts enums. So an +unscoped enum no longer binds to `common_json_value(bool)` anywhere, and the one-line `(int)` +cast `0010` added to upstream's `get_res_model_info()` became a **redundant carry**. + +**This is the drop-check firing, and it is the case that makes the check worth running by hand.** +`0010` still applied cleanly at b11080 — upstream never touched the emit site — so the fail-loud +applier said nothing, exactly as `CLAUDE.md` warned it could not ("the applier detects 'does not +apply', never 'upstream already fixed this'"). The standing check is phrased as *"did upstream +cast the value themselves?"*; the honest reading is *"is the defect still there?"*, and it was +not. Dropped, not refreshed, per the `0009`/`0011`/`0013` precedent. + +**The runnable guard stays, re-pointed** (the `0011` precedent again): the `CommonJsonEnumTrap` +pair in `src/test/cpp/test_json_helpers.cpp` is now the `CommonJsonEnum` trio, and it pins +upstream's contract — an **uncast** enum serialises as its numeric value, an explicit +`static_cast` is equivalent, and a real `bool` is still a boolean (the new constructor sits +next to `common_json_value(bool)`, so that one is worth pinning too). A future bump that loses +the enum constructor reds `C++ Tests` on every platform, and the response is to reinstate both +the cast and the patch. `jllama.cpp` keeps the explicit casts at its own two `"vocab_type"` +sites: they are correct either way and survive such a revert. + +`.github/verify-patches-applied.sh` lost its third check with the patch — `0010` was the only +patch with no runnable guard, which is what that check existed for. The script keeps its two +generic assertions (every patch on disk is in the stamp; the patched tree is dirty), so it still +catches an applier that never ran or a tree reverted after the fact. + +**`0011` was dropped at the b11069 bump.** Upstream merged +[ggml-org/llama.cpp#29161](https://github.com/ggml-org/llama.cpp/pull/29161) +("common/peg : handle invalid utf-8 sequences in the AST", commit `3d82ef62d`, first tagged at +**b11063**) — an independent and broader fix for the same defect; the patch itself had never been +filed upstream. Where `0011` made the until-parser's `INVALID` branch honour `ctx.is_lenient()` by +returning what was scanned *before* the bad byte (so the text after it was lost), upstream now +**consumes** every undecodable run, in strict mode too, records its `{pos, len}` on the AST node +(`common_peg_invalid_utf8`, carried up through `common_peg_parse_result`), and +`common_chat_peg_mapper` emits `node.sanitized_text()`, which substitutes exactly one U+FFFD per run +(the Unicode "maximal subpart" rule: `\xE4\xB8` followed by `c` is one run, `\xFF\xFE` is two). +`common/unicode.cpp` now reports the valid-prefix length in `bytes_consumed` for `INVALID` / +`INCOMPLETE` results to make that possible. The lenient incomplete-at-end-of-input branch (withhold +the trailing bytes, more may arrive on a stream) is unchanged. The applier failed loud with "does not +apply" at `common/peg-parser.cpp:680` exactly as designed — the `INVALID` branch it patched no longer +exists. Dropped, not refreshed, per the `0009`/`0013` precedent. **The runnable guard stays, +re-pointed:** the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` now pin upstream's +replacement contract (invalid byte → U+FFFD with the text after it kept; a trailing incomplete +sequence still withheld), so a future upstream revert to `FAIL`, or a change to the replacement rule, +still reds `C++ Tests` on every platform. The strict-mode `FAIL` on invalid UTF-8 that `0011` +deliberately preserved is gone upstream as well — `tests/peg-parser/test-unicode.cpp`'s +`malformed UTF-8` block now asserts `SUCCESS` plus the sanitized text. **Behavioural note for +consumers:** a completion containing a stray byte now returns the *full* text with U+FFFD in place of +the byte, where the patched builds returned the text up to the byte. + +**`0013` was dropped at the b10948 bump.** Upstream merged this project's own PR +[ggml-org/llama.cpp#28775](https://github.com/ggml-org/llama.cpp/pull/28775) +("ggml-cpu(s390x): guard VXE-only repack helpers", commit `6978052`, first tagged at **b10948**): +`ggml/src/ggml-cpu/arch/s390/repack.cpp` now wraps `vxe_dot_acc` / `vxe_splat_granule` / `vxe_fold` +in the same `#if defined(__VXE__) || defined(__VXE2__)` guard the patch added — byte-identical apart +from the trailing `// __VXE__ || __VXE2__` comment the patch put on the `#endif`. So the non-VXE +s390x compile break the patch fixed (three `does not name a type` errors before any skipped code is +reached) no longer exists upstream, and the applier failed loud with "does not apply cleanly" at +configure time exactly as designed — the guard it wants to add is already there. Dropped, not +refreshed, per the `0009` precedent. + +**The s390x configuration this leaves in place is still ours to know, because it is what made the +patch necessary and it did not change.** `ggml/CMakeLists.txt` declares +`option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})`, so the `build-linux-s390x` job's +`-DGGML_NATIVE=OFF` (correct for a cross-build — an x86 host must not bake `-march=native` into an +s390x artifact) silently switches VXE off too: no `-mvx -mzvector`, `__VEC__` undefined, and +`ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__` +never fires. The job therefore ships a **scalar** s390x binary, which is right for what it is (a +big-endian *correctness* gate for this project's own layer, not a performance target). **Do not +"fix" a future VXE-related compile error with `-DGGML_VXE=ON`** — measured, not assumed: that +self-define sets `__VXE__` *and* `__VXE2__` together while `-march` stays at the toolchain default +`arch11`, so a handful of errors becomes dozens of `'__builtin_s390_vec_*' matching variant requires +z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which +compiles but raises the artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on +VXE2 emulation. If a comparable break resurfaces, re-check upstream's guard against this description +before reintroducing a local patch. + +**`0009` was dropped at the b10280 bump.** Upstream merged +[sheredom/subprocess.h#104](https://github.com/sheredom/subprocess.h/pull/104) — the exact fix this +patch submitted — via [ggml-org/llama.cpp#26606](https://github.com/ggml-org/llama.cpp/pull/26606) +("vendor: apply patches for subprocess.h"): `vendor/sheredom/subprocess.h` at b10280 already defines +the `SUBPROCESS_HAVE_CWD` compile-time probe and the `ENOSYS` fallback our patch added, so the +old-glibc build break `0009` fixed no longer exists upstream. The applier's idempotent +`git apply --reverse --check` could **not** auto-skip this one (unlike a byte-identical carry): the +same PR also rewrote the neighboring Windows argv-quoting logic and added a `__NetBSD__` branch next +to the `process_cwd` guard, shifting the patch's second hunk's context, so it failed loud with "does +not apply cleanly" at configure time exactly as designed — confirming the patch needed to be dropped, +not refreshed. Also folded into that PR: a fix for `-std=c++20` unknown to GCC 8 +([#105](https://github.com/sheredom/subprocess.h/pull/105)) and a `posix_spawn` exec-failure +reporting fix pre-glibc-2.24 ([#106](https://github.com/sheredom/subprocess.h/pull/106)) — neither +affects this project. If a regression surfaces on old-glibc builds (manylinux2014/manylinux_2_28), +re-check `vendor/sheredom/subprocess.h`'s `SUBPROCESS_HAVE_CWD` guard against this description before +reintroducing a local patch. + +**`0005` was dropped at the b9981 bump.** Upstream's own `server-context.cpp` picked up an +equivalent — and broader — fix for the same checkpoint-starvation problem: `create_checkpoint` +now tracks `id_task` and evicts a stale checkpoint that sits within `checkpoint_min_step` of a +newer one (instead of our pre-creation duplicate-position check), and the `do_checkpoint` gate +now exempts **every** near-prompt-end checkpoint from the min-step spacing (`near_prompt_end`, +unconditionally) rather than only recurrent/hybrid (`ctx_tgt_seq_rm_type` `FULL`/`RS`) models as +our patch scoped it. The upstream version is a strict superset of what `0005` did, so it applies +cleanly with the patch removed and needs no replacement. If a regression in agentic multi-turn +prefill behavior shows up, re-check `tools/server/server-context.cpp`'s `create_checkpoint` / +`do_checkpoint` logic against this description before reintroducing a local patch. + +**`0004` was dropped at the b9982 bump.** Upstream merged +[ggml-org/llama.cpp#23116](https://github.com/ggml-org/llama.cpp/pull/23116) verbatim: the +`b9981...b9982` diff carries the exact same `oaicompat_chat_params_parse` precedence change +(`reasoning_budget_tokens` > `thinking_budget_tokens` alias > server default, plus the per-request +`reasoning_budget_message` override) and the same two `tests/test-chat.cpp` regression cases +(`test_reasoning_budget_tokens_per_request` / `test_reasoning_budget_message_per_request`, +byte-identical body). No local patch needed — the tree already matches what `0004` used to add. diff --git a/docs/upstream-investigation-jline-status-windows-redraw.md b/docs/upstream-investigation-jline-status-windows-redraw.md index cd139a5e5..de482681d 100644 --- a/docs/upstream-investigation-jline-status-windows-redraw.md +++ b/docs/upstream-investigation-jline-status-windows-redraw.md @@ -134,7 +134,7 @@ console API rather than through `System.out`. - Nothing in `llama-atmosphere-agent` causes it, and it cannot be fixed there without reimplementing `handleSignal` in a `LineReaderImpl` subclass. An earlier attempt to help by installing a `WINCH` handler made the display **worse**, because the line reader owns that signal for the whole session — - see `CLAUDE.md`, "Do not add a `WINCH` handler". + see `llama-atmosphere-agent/CLAUDE.md`, "Do not add a `WINCH` handler". - No project test asserts the fixed behaviour: it would fail against the JLine release the build depends on. `JLineTerminalTest` therefore keeps pinning what *is* true today. - `--plain` is unaffected and is the escape hatch: it pins nothing and redraws nothing. @@ -229,7 +229,7 @@ the new width on **every** size event — every run ends with exactly one row ca So the emitted sequences are correct for a conformant VT, and what is left is the real console's own handling of lines drawn at earlier widths: Windows Terminal reflows wrapped lines on widening, and an `ESC[2J` erase leaves them in the scrollback (the same property that made `/cls` scroll old rows back -into view — see `CLAUDE.md`). Nothing in this project draws those rows a second time: only +into view — see `llama-atmosphere-agent/CLAUDE.md`). Nothing in this project draws those rows a second time: only `status()` and `clearScreen()` rebuild the block, and neither runs on a resize. **The workaround is Ctrl-L (or `/cls`)** — it wipes the visible area and redraws the block, which is diff --git a/llama-atmosphere-agent/CLAUDE.md b/llama-atmosphere-agent/CLAUDE.md new file mode 100644 index 000000000..fe4447c8f --- /dev/null +++ b/llama-atmosphere-agent/CLAUDE.md @@ -0,0 +1,895 @@ +# CLAUDE.md — `llama-atmosphere-agent/` (local coding agent with Atmosphere) + +Guidance for working in this directory. Claude Code loads it when files here are read; the +repository-wide rules are in [`../CLAUDE.md`](../CLAUDE.md), which keeps a short summary of what +this project means for the rest of the repository (release asset, CI gates, version bump). + + +A **copy-and-run general-purpose terminal agent** (Claude Code / OpenCode reduced to the essentials, offline) +that pairs [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in OpenAI-compatible +agent runtime with this project's `OpenAiCompatServer`. Like `android-llmservice/` it is a +**standalone Maven project, NOT a reactor module and NEVER on Maven Central** — it is an application, and it +needs Java 21 (Atmosphere's floor) while the core stays Java 8. CI builds it against the core it just +installed (`-Dllama.version=`); a user copies the folder and runs +`mvn compile exec:java -Dexec.args="…"` with no `-D` at all — the pom's `llama.version` names the +**released** core the READMEs describe (currently `5.2.0`, written as if released so the docs are +right the moment the release lands). The natives come from the +`natives` profile (`llama-platform`, active unless `-Dllama.natives=none`) plus the `gpu-natives` +profile (`-Dllama.classifier=`); CI passes `-Dllama.natives=none`, since it installs only +the classes and the model-backed job uses `-Dnet.ladenthin.llama.lib.path`. + +**Release asset: the agent jar WITHOUT the core.** `mvn -P assembly package` (the pom's `assembly` +profile, descriptor `src/assembly/agent-jar.xml`) builds +`llama-atmosphere-agent--jar-with-dependencies.jar` — named after the **core** version +it was built against, not the agent's own `1.0.0-SNAPSHOT`, because it only runs next to that core. +It excludes `net.ladenthin:llama` **with its whole runtime graph** (`useTransitiveFiltering`: Jackson 2, +slf4j-api, and Jackson 3's `jackson-annotations`, which resolves through the core's trail) plus +`jspecify` and `slf4j-simple`, all of which every core fat jar already bundles — so the asset is ~14 MB +(most of it Jetty, the ACP SDK with Reactor, and the Atmosphere console pages) instead of hundreds, the natives are not in the release twice, and there are never two SLF4J providers. +The manifest's `Class-Path` names the four `llama--all---…` fat jars and then the default +`llama--jar-with-dependencies.jar`, so `java -jar` works when the agent lies next to any of them +(missing entries are ignored; note that a manifest `Class-Path` is honoured under `java -cp` too). +**Rename a core fat jar and this list must follow** — `smoke-agent-linux` is what notices. +CI wiring (`publish.yml`): the model-free job builds it, writes the `.sha256` and uploads artifact +`llama-atmosphere-agent-jar`; `github-snapshot` / `github-release-signed` download it into the asset +directory next to `llama-fatjars`, so `sign-fatjars.sh` signs it (`*-jar-with-dependencies*.jar`) and +the one upload attaches it. **`smoke-agent-linux`** (`.github/smoke-agent-jar.sh`) runs the asset the +way the README tells a user to — `java -jar` next to the real `all-linux-x86-64` fat jar — and checks: +bytecode ≤ 65 (Java 21, unlike the core's 52 — with one `--allow` for JLine's FFM terminal provider `org/jline/terminal/impl/ffm/*`, 25 classes shipped as Java 22 bytecode that JLine discovers through `META-INF/jline/providers/ffm` and never loads on 21, where it picks its JNI provider; the first CI run of the smoke caught them), that the jar started **alone** fails with +`NoClassDefFoundError: net/ladenthin/llama/LlamaModel` (i.e. it really carries no core), `--help`, a +one-shot `2 + 2` answer and a `read_file` round that must surface a marker from `--workspace`, all on +the cached `TOOL_MODEL_NAME` with `--ngl 0`; then `--web` on port 0 (the console is `401` without the +token, the token link answers `302` with the `jllama_agent` cookie, the console page is served with it) +and `--acp` through `.github/smoke/agent_acp_smoke.py` (standard-library Python playing the editor: +handshake, modes, a streamed `4`, the announced commands, a `read_file` tool card that brings a marker +back, a clean exit when stdin closes, and nothing but JSON-RPC on stdout). **All three agent jobs gate +both publish jobs** (model-free, model-backed integration, smoke). + +**The console pages come from a jar the assembly otherwise leaves out.** Atmosphere's prebuilt AI +console lives in `atmosphere-spring-boot-starter` under `META-INF/resources/atmosphere/console/`. The +pom depends on that starter with a `*:*` exclusion (no Spring on the classpath), the assembly's main +dependency set excludes it, and a second dependency set unpacks **only** that resource path. Remove +either half and the jar either grows by the whole starter or `--web` starts with nothing to open +(`WebConsole.available()` says so at startup). + +**What Atmosphere is, for this purpose.** `org.atmosphere:atmosphere-ai` (4.0.71) ships +`BuiltInAgentRuntime` + `OpenAiCompatibleClient`: a zero-framework OpenAI client that *always* +streams (`stream:true`), accumulates `delta.tool_calls` by `index`, executes `ToolDefinition` +executors, re-submits the conversation (assistant `tool_calls` message **without** a `content` key, +then one `role:"tool"` message per call with `tool_call_id` + `name`), and loops until +`finish_reason` is not `tool_calls`. It reads `LLM_BASE_URL`/`LLM_MODEL`/`LLM_API_KEY` or takes +`AiConfig.configure(mode, model, apiKey, baseUrl)`; `GET /models` is best-effort; the Responses API +is used only when the base URL contains `api.openai.com`; `tool_choice`/`parallel_tool_calls`/ +`response_format` are not sent. It runs headless — `runtime.execute(AgentExecutionContext, +StreamingSession)` — so no Spring Boot, servlet container or `@Agent` scanning is involved; its +built-in `FileSystemTools` resolve the `AgentFileSystem` from `StreamingSession.injectables()`, +which is how the tools are confined to a workspace. The `@Agent`/`@AiTool` annotations and the +Spring Boot starter are a deployment layer on top of the same runtime. + +**Verified compatibility (verdict A — works unchanged).** Two test layers, both in the project: + +- `AtmosphereWireContractTest` + `LocalAgentTest` — **model-free, every PR, seconds**: the *real* + `OpenAiCompatServer` (routing, bearer auth, `/v1/models`, SSE framing) over a loopback socket with + a `ScriptedBackend` replaying llama.cpp-shaped chunks (role delta, `tool_calls` deltas with + `index`/`id`/`name` and fragmented `arguments`, `finish_reason:"tool_calls"`). Pins: one tool + round; four rounds incl. a parallel pair with interleaved fragments and the whole history kept; + chunk-by-chunk streaming and history replay; `temperature`/`max_tokens` on the wire; 401 on a + wrong key before the backend is reached; and the **one known gap** — an engine failure *after* the + stream started is an SSE `data: {"error":…}` under HTTP 200 (upstream llama-server does the same), + which Atmosphere's parser ignores (it reads only `choices[0]`), so the turn completes with the + text so far instead of erroring. That is a SHOULD for Atmosphere's `OpenAiCompatibleClient`, not + for this project. +- `AtmosphereToolLoopIntegrationTest` — **model-backed, CI only** (`test-java-llama-atmosphere-agent-integration`, + a publish gate): the same loop against the cached Qwen2.5-1.5B tool model + through the downloaded Linux natives — plain chat, streaming (≥ 2 chunks), a tool call whose result + is answered, a read→write→read loop that changes a temp file. Self-skips without the GGUF. + +**The one core change this needed:** `OpenAiBackend`, `ChunkSink` and +`OpenAiCompatServer(OpenAiBackend, OpenAiServerConfig)` are now **public** (they were the +package-private test seam). A sibling module cannot otherwise drive the real server without a model; +the alternative — a same-named package in the sibling's test tree — is a split package that breaks +the moment anything runs on the module path. + +**Layout.** `AgentOptions` (CLI parsing, pure), `AgentRunner` (the whole Atmosphere wiring, ~40 +lines: `AiConfig.configure` → `BuiltInAgentRuntime` → `AgentExecutionContext` + `ToolLoopPolicies`), +`ConsoleSession` (streams to stdout, prints `⚙ tool {args}` / `↳ result`, supplies the +`WorkspaceAgentFileSystem` via `injectables()`), `ShellTool` (opt-in `run_command`, `sh -c` / +`cmd /c` starting in the workspace, timeout kills the process tree, output tail-truncated), `LocalAgent` +(`--base-url` = external server, `--model` = in-process `LlamaModel` + loopback `OpenAiCompatServer` +with `enableJinja()` and `setLogVerbosity(2)` by default — llama.cpp logs to **stderr**, the console the +streamed answer shares, so the per-request `slot …` INFO lines would interleave with it; `--log-verbosity ` +/ `--verbose` override — one-shot `--prompt` or a `you>` REPL with `/clear` `/exit`). `.mvn/jvm.config` +pins `-Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8` for the `mvn exec:java` JVM: on Windows, +`common_init()` switches the console to UTF-8 (`SetConsoleOutputCP(CP_UTF8)`) after the JVM fixed its +stdout encoding from the old code page, which turned umlauts/emoji in answers into `�`/`?`. It needs a +sibling **`.mvn/jvm.config.license`** (the same two SPDX lines as the one next to the root `.mvn/jvm.config`): +a `jvm.config` takes no comments, so REUSE can only read its metadata from that file, and without it the +`REUSE Compliance Check` job fails on `main` — which is how it was found, the PR run having been cancelled. +Spotless (palantir) is configured in its own pom; the model-free CI job runs `spotless:check`. + +**The REPL layer (commands, approval, status line, rendering).** Eight small classes and one dependency +(`org.jline:jline`, one jar, no transitive deps): +`SlashCommands` (a line starting with `/` whose first word names a command is handled locally — +`/help /status /tools /mode /compact /clear /exit`; **an unknown `/command` goes to the model**, which +is why no escape syntax is needed for `/usr/bin/…`), `ApprovalMode` + `ConsoleApprovalStrategy` +(`[y]es/[n]o/[a]uto` per gated call), `StatusLine`, and `Ansi` + `MarkdownConsole`. Four points that +are decisions, not details: + +1. **The approval gate is Atmosphere's, not ours.** `AgentRunner.approval(strategy, policy)` attaches + `ToolApprovalPolicy.custom(...)` (gating `run_command`, `write_file`, `edit_file`, `delete`, + `rename` — reading tools never ask) and a `ConsoleApprovalStrategy`; `ToolExecutionHelper` then + blocks the tool loop before the executor runs and turns a denial into the tool result + `{"status":"cancelled","message":"Action cancelled by user"}` for the model. Do not reimplement + that message. `ApprovalWireTest` pins both halves over the real server. **The reading tools + (`ls`, `read_file`, `glob`, `grep`) never ask, and that is a decision, not an omission**: gating + them would make the question so frequent it stops being read. Because the gate is a list of + *names*, a tool upstream adds or renames would drop out of it and then run unasked — so + `READ_ONLY_TOOLS` names the other half explicitly and + `ConsoleApprovalStrategyTest.everyOfferedToolIsEitherGatedOrDeclaredReadOnly` asserts every offered + tool is in exactly one of the two sets, and that neither set names a tool nobody offers. Same class + as the stale `spotbugs-exclude.xml` entries: an allowlist that silently stops matching. +2. **One-shot (`--prompt`) denies a gated call** instead of auto-approving it — `--auto` is the + deliberate opt-in. Atmosphere itself fails closed when no strategy is wired, and this keeps that + direction: an unattended run must not be the most permissive one. +3. **`AgentTerminal` has exactly two implementations, chosen once at startup, and `--plain` picks the + line-oriented one on purpose.** `LocalAgent.usesFullTerminal(options, interactive)` is the single + place that decides: the cursor-controlling console needs someone typing **and** permission to move + the cursor, and `--plain` withholds the second even on a real terminal. That is not a fallback but + a supported mode — for a session that is piped, logged, recorded, or carried by something that + forwards lines rather than a screen. It gives up the pinned block, the spinner, history/completion + and typing-during-a-turn (`PlainTerminal.hasPendingInput()` is always false), and gains being + correct when the output is a file. A normal SSH session needs none of this: a remote terminal + reports its size and handles cursor control like a local one. `JLineTerminal` (a real + terminal: line editing, history, Tab completion of the command names, a status line pinned to the + bottom via JLine's `Status`, single-key answers through `enterRawMode`, streamed output via + `LineReader.printAbove` so the bottom block stays put) and `PlainTerminal` (a `PrintStream` plus a + `BufferedReader`: no cursor control at all, correct when the output is a file). `JLineTerminal.open` + returns **null** instead of throwing when there is no usable terminal — piped input, a dumb + terminal, a missing native provider — and the caller falls back. Every test drives `PlainTerminal`, + which is why none of them needs a TTY. Verified on Windows: JLine picks the `windows-vtp` provider, + so ANSI works there without the registry caveat. +4. **The answer is rendered append-only, one completed line at a time** (`MarkdownConsole`). Redrawing + on every token is what produces the known overdraw/truncation bugs in the Ink/Bubble-Tea based + clients and breaks when the output is piped. Only headings, bullets, fences and inline + `**bold**`/`` `code` `` are handled; italics deliberately are not (`*` is more often a glob than + emphasis). Colour is decided once in `Ansi.detect()` — `CLICOLOR_FORCE`, then `NO_COLOR`, then + `TERM=dumb`/`CLICOLOR=0`, else "is a terminal" via `Console.isTerminal()` (reflective: JDK 22+; + below that `System.console() != null`). +5. **`/loop` keeps its state in a file, not in the context, and stops on a text marker** (`TaskLoop`, + `LoopOptions`). Every step re-sends the task verbatim and drops the history, so the context cannot + grow (Claude Code's ralph-wiggum plugin does the same, and `AGENT-LOOP.md` in the workspace is the + memory). The stop signal is a line that is **exactly** `<>`, never a substring — + deliberately **not** a "done" tool: below 7B a model emits a malformed tool call far more often + than a malformed line, and mini-SWE-agent's SWE-bench results come from exactly this plain-sentinel + design. `--check ''` re-verifies the claim and feeds a failure back. Four guards, none of them + trusted to the model: step cap, wall-clock budget, stall detection (three steps with no file change + and no tool call), interval. **Order matters and a test pins it**: the marker is checked *before* + the stall detector, because the step that only answers "done" changes nothing and would otherwise + be reported as no progress. +6. **Three of the file tools are this project's, not Atmosphere's** (`WorkspaceTools`, `TextEdits`, + `WorkspaceSearch`) — `read_file`, `edit_file`, `grep`. They are **replacements, not additions**: + two tools that both claim to read a file is the worst case for tool selection. They call the same + `AgentFileSystem`, so workspace confinement, path validation and size limits stay Atmosphere's. + Each replacement has a measured reason, and all three are pinned by tests: + - `edit_file`: the framework matches against the **raw** file content, so a model's LF text never + matches a CRLF file — on Windows *every* edit fails silently. `TextEdits` normalizes before + matching and restores the file's own ending and byte-order mark. It also shows the nearest lines + on a miss and the line numbers on an ambiguous match (a failed edit drops the eventual success + rate from 90.5 % to 57.2 %), offers `replace_all`, and applies a batch of edits **all-or-nothing** + — a deviation from every shipping agent, which apply sequentially and leave a half-edited file. + - `grep`: the framework walks **alphabetically** with one global 2-second deadline and one global + 500-hit budget, so `.git`, `target` and `node_modules` consume both before `src` is reached. + `WorkspaceSearch` excludes them, groups by file with line numbers, and **states** truncation. + - `read_file`: `offset`/`limit` and numbered lines (whole-file reads measure 12.7 % against 18.0 % + task success in the SWE-agent ablations). The numbers are display only, which both the tool + description and the system prompt say — leaked line numbers in `old_string` are a known failure. + - `edit_file` **refuses a file that was not read** in this session (`WorkspaceTools.ReadTracker`). + Not a staleness check: an exact unambiguous match is safe regardless; this catches the model + inventing the text. + **Rejected on evidence, do not add later without new numbers:** a unified-diff/patch tool (Meta's + ablation: search-replace 42–53 % vs 26–30 % unified diff vs 20–26 % line diff on one model; a 7B + model collapses 54 → 33 → 14 %), fuzzy matching (turns a loud miss into a silent wrong-place edit), + an embedding index (Cursor's production effect is +0.3 %), and LSP tools (the one isolation study + finds them token-negative and *worse* at multi-file rename, because renames touch comments and + strings that semantic references exclude). +7. **The session transcript (`Transcript`) is not the conversation the model is sent, and must not be + merged with it.** The model's history is rewritten by `/compact` — a summary replaces the turns — + and has never carried a timestamp; the transcript only grows and stamps every entry. `/compact` + adds a note to it and changes nothing else, `/clear` empties it (the command means "forget this + session"), `/save [name]` writes it into the workspace, and `--transcript ` appends live so a + killed session still leaves what it had — that write failing is swallowed, because a record that + exists to survive a bad ending may not cause one. **A list, not a map keyed by the timestamp**: a + tool result and the answer after it regularly share a millisecond and a map would drop one + silently; insertion order already is time order. `ToolCallLog` stays as the separate, hard-cut + receipt for `/calls` — it answers "did that really run", which prose cannot. + **`/load` replays only `USER` and `AGENT` entries** as messages: a tool result outside its round is + not something a chat template has a place for, and inventing a shape for it would be worse than + letting the model call the tool again. The parser treats a line without a stamp as a continuation + of the entry above it, because an entry is not a line — an answer keeps its newlines when written, + and reading line by line would turn one answer into several. A file that is not a transcript yields + **no** entries rather than one wrong one, since anything it yielded would be replayed to the model + as if it had been said. + + **`JLineTerminal.open` refuses when there is no console, and that was a red build for months.** A system + terminal takes over the process's standard input, and where there is no console that input belongs to somebody + else. Inside a **Surefire fork it is the channel Surefire talks over**: a test that drives the agent + interactively reached `open()`, JLine grabbed the channel, and the run ended with + `[SUREFIRE] std/in stream corrupted` — **every test green, the build red**, which is exactly the shape that + hides. It also made `LocalAgentTest` take 23.6 s instead of 2.0 s, because it was blocking on reads that were + never going to arrive. The check is the same two-step `Ansi` uses for colour: `Console.isTerminal()` where it + exists (JDK 22+, where `System.console()` returns a console even for redirected streams), otherwise the mere + presence of a console. Piped input lands here too and has always been served by the plain console, so nothing + else changes. + +8. **Tool calls are carried into the conversation as a text note, and logged for `/calls`.** + `LocalAgent.withToolNotes` prefixes each turn's answer in the history with + `(tools I actually ran this turn: -> )`, and `ToolCallLog` + keeps the same data for the `/calls` command. **Why it is a note and not real `tool_calls` + messages:** `AbstractAgentRuntime.assembleMessages` rebuilds every history entry as + `new ChatMessage(h.role(), h.content())` — the tool-call array and the tool-call id never leave the + framework, so protocol-faithful replay through `context.history()` is impossible; content is what + survives. **Why it exists at all:** with only user text and assistant prose in the history, a 4B + model stopped calling tools after the third turn of a real session and *described* the work instead + — inventing JUnit tests, a Maven build and a `.bat` script, complete with exit codes, while the + workspace stayed empty. The system prompt also forbids claiming an action without the call. + **Placement was found by failing twice, so do not "simplify" it:** in front of the assistant's + answer made the model copy the record into its own replies (the user saw `(tools I actually ran + this turn: …)` as the first line of an answer); real `tool_calls` messages are impossible (see + above); a mid-history system message is cleanest but Mistral's template requires strict + user/assistant alternation and Gemma has no system role. It therefore rides in front of the **next + user message**, which every template accepts. + Pinned by `LocalAgentTest.aToolCallStaysInTheHistorySoTheNextTurnSeesItHappened`. + **Live feedback while a turn runs** (`LocalAgent.activityLine`, `ShellTool`'s line-by-line output): + the block's first row names the running tool and its own elapsed time, and shell output is printed + as it arrives. **The turn runs on its own thread, and it has to:** Atmosphere's `execute()` is + synchronous — it returns only once the whole turn including every tool round is done — so running + it on the console thread leaves nobody to refresh the line, and the block sits on + "… waiting for input …" for the entire turn (exactly the symptom that was reported). The approval + prompt then reads a key in raw mode on that worker thread while the console thread redraws four + times a second, so `TurnActivity` pauses the redraw for as long as the question is open. Reading the pipe incrementally is not only cosmetic — an unread pipe blocks the child once + it is full, which on Windows is roughly 4 KB. +9. **The context number in the status line is an estimate, marked `~`, and it moves during the turn.** + llama.cpp emits its usage chunk only when the client sets `stream_options.include_usage`, and + Atmosphere's client does not; `ConsoleSession.usage()` takes the real count when one arrives, + otherwise `LocalAgent.estimateTokens` uses four characters per token. The window size is + `--ctx-size` (in-process) or the server's `/props` (`ServerProps`), and is omitted rather than + guessed when neither answers. **The state row is a `Function`, not a + string**, and `awaitWithActivity` asks it again on every redraw: it used to be rendered once before + the turn and handed over fixed, so the figure stood still through every tool round and only moved + at the next `you>` — which is exactly when it no longer helps anyone decide whether to `/compact`. + `LocalAgent.liveTokens` adds `ConsoleSession.producedChars()` (streamed text **plus** every tool + call and result — all of it is in the prompt of the next model call of the *same* turn) to what the + request carried when it was sent, and yields to the server's own count as soon as one arrives. + `TaskLoop` passes a constant function, and its step label must be copied into a local first: a + lambda may not close over the loop counter. +10. **One call to `AgentTerminal.line` is one screen line**, and `ConsoleSessionTest` is what defends + it. The pinned block is reserved in **lines**, so a single "line" carrying twenty newlines moves the + screen twenty rows further than the terminal accounted for and the block is then drawn across the + output — reported twice, both times from a `write_file` call whose `content` argument was the file. + `ConsoleSession.describeArguments` folds and cuts **each argument value on its own** (80 chars) + before cutting the whole rendering (200), so a call carrying a whole file still shows the file + *name*; results and errors are folded the same way. `JLineTerminal.line` splits a multi-line string + as a backstop for a caller that forgets. Only the console is cut — the model gets everything, and + `ConsoleSession.rounds()` keeps the full arguments for the history note and `/calls`. +11. **The approval mode carries a glyph, and shift+tab switches it**: `ApprovalMode.symbol()` / + `badge()` render `⏸ manual` and `⏵⏵ auto` on the status line and in `/mode`, the transport symbols + the established terminal agents use for the same distinction; `ApprovalMode.next()` is the cycle + the key walks. The binding is `AgentTerminal.onCycleMode(Runnable)`, which **defaults to declining** + — only `JLineTerminal` overrides it, and the startup line advertises the key only when the bind + succeeded. Two details are not obvious: it is bound **both** through terminfo + (`InfoCmp.Capability.key_btab`) **and** to the literal `ESC [ Z`, because JLine's + `windows-vtp.caps` declares no `key_btab` at all while the terminal in virtual-terminal input mode + does send the sequence; and the shortcut fires **only while a line is being read**, so it switches + the mode between turns — which is when it is decided anyway. The REPL therefore holds the two + status numbers in an `AtomicLong`/`AtomicBoolean` rather than locals, so the widget (which runs + inside the reader) can re-render the pinned row with what the last turn left behind. + +12. **The input is framed into the pinned block, and the prompt stays there during a turn; typing + stops the turn.** The frame is two halves that must be read together: the **top** rule is the first + line of the reader's *prompt* (`rule() + newline + "> "`, rebuilt on every read because the window + can be resized) — **no, and the second attempt was wrong too.** Both are recorded because the + obvious fix is the one that fails. (1) Rule as the first line of a **two-line prompt**: + `ERASE_LINE_ON_FINISH` erases exactly **one** line, so every Enter leaves the rule behind and + holding Enter draws a column of them. (2) Rule as **ordinary output before each read**: nothing is + left behind on Enter any more, but one rule now stays in the scrollback per turn and travels up + with it. **There is no third option** — JLine's status region is below the prompt and never above + it, so a rule above the input can only be part of the prompt (1) or part of the scrollback (2). + The settled shape is therefore **one** rule, the first line of the status block, directly under the + input line; the prompt is `"> "`, one line, and `AgentTerminal.readLine`'s `prompt` argument is + consequently **ignored** here. + `ERASE_LINE_ON_FINISH` removes the input line on Enter and the reader thread echoes it above as + `› text`, so the transcript keeps what was asked. + + **`ScreenUseCasesTest` + `ScreenTerminalHarness` are how the resize and `/cls` cases are checkable, + and they are what every byte-level attempt before them could not do.** The harness subclasses JLine's + `LineDisciplineTerminal` and puts its own `ScreenTerminal` (a real VT interpreter, public API in the + shipped jar) behind it, so a test reads the **screen** — "the block is smeared across the output", + "an escape sequence is printed as text", "the block is drawn twice" are indistinguishable from + correct output in a byte stream, which is why two byte-level assertions in `JLineTerminalTest` had to + be deleted, one of them green with the fix it was written for switched off. The cases are the reports: + dragging wider and narrower (with nothing typed, with text in the input, one step at a time, with a + two- and a three-row block, with the real block's glyphs), `/cls` and Ctrl-L (block intact, and the + cursor and prompt on the row the block leaves for them), a window that over-reports its width, size + events arriving on another thread, and a block refresh racing the reader. Several assert the **cursor + row** rather than the content, which is what finally located two of the defects: content can look + plausible while the cursor is rows away from where a region reserved from the bottom expects it. + Three things are load-bearing. **(1) A reader runs in every test** — it runs for the whole session in + the application and owns the resize signal, so a test without one is a state the application cannot + be in. **(2) The block rebuild is invoked directly, not waited for** + (`refreshBlockForCurrentSize`): sleeping for the 120 ms poll passed alone and failed in a full run, + and a flaky test is worse than none — what that leaves uncovered is the polling thread itself, a loop + that compares two sizes and calls that method. **(3) A rule shows as `q` when it went out through a + *prompt*** (the DEC line-drawing set, which this screen renders literally) and as `─` through the + status region, so both forms count as a rule and a row of `q` also says which path it took. + **The JLine fixes are demonstrated red/green through it**, the library being just a property — and since + there are **eight** of them, one of which adds a method, the wiring needs a paragraph of its own. + `ScreenUseCasesTest` **skips itself** on a JLine that does not carry them, keyed on + `Status.repaint()` (the fifth fix and the only one that is a new method, so its presence stands in for the + whole set). That is CLAUDE.md's own rule applied late — "no project test may assert the fixed behaviour + while the build depends on an unfixed release" — and it had been broken: the pom's `jline.version` is the + **released** one, which is what CI builds against, and against it these cases fail. Measured: the whole + module is green with `-Djline.version=4.4.6-atmosphere` (the screen cases run) and green with the + released `4.4.6` as well, where they report as **skipped** rather than passing. + **Whether the fixes are still needed is measurable rather than arguable**, because the gate honours + `-Datmosphere.screen.tests.runAnyway=true`. Against the current console, 233 tests: released `4.4.6` **26 + red**; `statusfix4` (fixes 1–4) 8; `statusfix5` (+`repaint`) 4; `statusfix6` (+addressing every row) 1; + `statusfix7` (+padding one column short) **0**; `statusfix8` 0. (Those version names are the history of the + investigation; **what to build today is `4.4.6-atmosphere`**, the reviewed set — the recipe is at the end of + the investigation document.) So they are needed, by a wide margin — and the + **eighth is carried on the merits, not on a failing test**: `Status.resize` still erases rows above the bar + without it, but this console wipes and redraws after a width change, so nothing here observes it. It stays + because the rows above a bar are not the bar's to clear, an erase is unrecoverable where a scroll is not, and + every consumer that does not wipe needs it; its guard lives where the defect does, in `StatusRepaintTest`. + **Every change was then reverted on its own and measured**, which dropped one and corrected a mistake of mine: + the `xenl` entry for `windows-vtp` is **no longer caught by anything** (addressing every row and padding one + column short removed the dependence on the wrap), so it is out — an unmeasured change does not belong in a set + meant for submission. The `doDisplay()` → `display.resize(size)` change was first reported as unnecessary too, + wrongly: that revert had been made with a string replacement matching nothing, compiled into a second directory + layered in front of the first, which does not reliably win. Compiled properly, two tests fail without it. The + set is therefore **seven** changes in three files, each with a failing test in JLine's own style behind it; the + full matrix and both traps are in the investigation document. The marker cannot tell + the fifth build from the later ones (the sixth adds a *protected* method, the seventh changes only a padding + width), so on an older patched jar one or two cases fail rather than skipping — stated in the test rather than worked around. + **The gate is the whole class and there is no fixed list of affected cases, which is itself a finding.** + Against the released library between six and thirteen of them fail, a *different set each run*: the fourth + fix is a data race, and when its `ConcurrentModificationException` lands on the reader's signal thread it + ends that thread, after which no size change is reported at all and whichever cases were still to run fail + too. The skip is per test (`@BeforeEach`), not a `@BeforeAll` assumption, because a class-level one makes + Surefire record the class as **zero tests** — which reads as "nothing here" instead of "skipped", the same + trap that silently muted every model-backed test in this repository for months. + **What proves the fixes themselves are JLine's own tests**, in the clone's own style and next to its + others: `StatusRedisplayTest`, `StatusDelayedWrapTest`, `StatusConcurrencyTest`, `StatusRepaintTest` + (4 cases: an update with unchanged lines leaves damage on screen, `redraw()` does too because it is the + same diff, `repaint()` puts every reserved row back without scrolling anything, and a repaint before + anything was shown is not an error) and `StatusWrongWidthTest` (3 cases on a screen wider than the width + it reports: every reserved row keeps its own screen row, nothing is written beside the rule, and the rows + above the block stay empty, and — the seventh fix — a reported width one column too LARGE does not wrap the + bottom row). 103 of JLine's own tests are green with all eight fixes, `DisplayTest` and `ScreenTerminalTest` + included. + + + **The fourth fix is a data race, and it is the one that explains the reports that survived the other + three.** `refreshingTheBlockWhileTheReaderRedrawsNeverThrows` refreshes the block from three threads + while the size changes, and fails with a `ConcurrentModificationException` whose stack names the + defect: `AttributedString.join` iterating `Display.oldLines` from `Status.resize` (called by + `LineReaderImpl.handleSignal`) while another thread is inside `Status.update` replacing that same + `ArrayList`. Neither is synchronized and both are reachable from different threads *by design* — this + console keeps its block current from its own thread four times a second. **Keystrokes do not provoke + it; a size change does**, because the reader then resizes its display, resizes the region and + redisplays. Raised on the input pump it ends that thread, after which no size change is ever reported + again and the block keeps its width — which is the "rule wider than the window, wrapped" screen, + reported as a separate problem. The fix is `synchronized` on `Status`'s nine public entry points. + **And it is why the screen harness could not reproduce the reports it was built for**: + `ScreenTerminalOutputStream.write` is synchronized, so it serialises the byte stream, while the + unprotected state is JLine's own list — no stream lock reaches it. Two cases carry + `@Disabled` as the record of a defect that is still open — a narrower drag loses the edit line, and a + *three*-row block ends a wide drag with the rule three columns short, the state row shifted one + column and the prompt on two rows, while a two-row block comes out clean. + + **`JLineTerminalTest` is how any of this is checkable**: `JLineTerminal.over(Terminal, …)` takes a + terminal built over two streams, which renders exactly like a TTY, so the screen can be asserted on + the emitted bytes. Two things that cost an hour each and are not guessable: the test terminal needs + **`stdoutEncoding`** as well as `encoding`, or every `─` arrives as `?`; and a box character is + written as UTF-8 from `printAbove` but as the **DEC line-drawing set** (`ESC(0` + `q`s + `ESC(B`) + inside a *prompt*, so a counter that looks only for `─` passes against the exact bug it was + written for — verified by putting the two-line prompt back and watching the test go from 1 rule to 5. + + **Escape sequences drawn as text** (`[?1h` above the prompt, then a `1H` inside the rule) were two + further defects of the same family, and the second is the one that eventually **destroyed the + block**. First: `line()` wrote straight to the terminal whenever the reader was not inside + `readLine`, which is exactly when the next read emits its init sequence — once the reader thread + exists, **everything** now goes through `printAbove`. Second: three threads write to this terminal + as a matter of course — the turn (Atmosphere's thread) prints tool lines, the console thread + refreshes the block four times a second, and with an in-process model **llama.cpp logs to stderr**, + which is the same console and goes around JLine entirely. The first two are serialised by a + `writing` lock held across `line()` and `status()`; the third is fixed by + `LocalAgent.captureNativeLog`, which routes the native log through `LlamaModel.setLogger` + (the callback sink `patches/0014` added) into `terminal.line`, so it scrolls in above the prompt + like any other output instead of scrolling lines JLine never sees. **Honest limit:** the lock is + reasoned, not test-covered. Two attempts to pin it are recorded in the history of + `JLineTerminalTest` and both passed with the lock removed — even one that sliced every + `OutputStream.write` in half — because `PrintWriter` already makes a single call atomic and the + interleaving happens *between* calls, inside JLine. A test that is green either way is worse than + none, so it was deleted rather than kept. + + **`/cls` clears by SCROLLING, not by erasing, and `/clear` also wipes the history.** + `AgentTerminal.clearScreen()` defaults to doing nothing (a stream has no screen); `JLineTerminal` + prints a window's worth of blank lines through `printAbove`. That is the whole implementation, and it + replaces an erase-based one that took four attempts and produced a reported defect each time — which + is why the reasoning is kept in full. + + **What a clear has to do here is two things at once**: leave the screen blank *and* leave the input on + the row the pinned block leaves for it. Erasing does the first and undoes the second, because + `clear_screen` puts the cursor home and the reader draws its prompt where the cursor is. The + reports, in order: blank rows printed after the erase to push the input back down scrolled the + *erased* lines back into view (`ESC[2J` clears the visible area and leaves them in the scrollback); + a `cursor_address` smuggled into `printAbove`'s argument corrupted its bookkeeping — it moves up, + writes, and redraws the prompt below — and stranded a character above the prompt; and the erase on + its own left the input at the top left ("nach /cls ist der cursor auch ganz oben und nicht unten"). + The screen tests then showed that last state is worse than it looks: the reader redraws its prompt + as a **diff** against what it believes is on screen, an erase invalidates that belief, and the + measured result is **no prompt on screen at all** — cursor on row 0, the block still pinned at the + bottom, ten blank rows between them. + + **Scrolling has none of those problems because it is nothing but output.** Everything is pushed above + the window, so the screen is blank and what was written stays reachable with the scrollbar — which + erasing the scrollback (`ESC[3J`) would have broken anyway, and this console promises it. Nothing is + erased, so nothing can be pulled back into view. The reader's bookkeeping stays right, because + printing above the prompt is what `printAbove` is *for*. The block is never touched: it is pinned, so + it needs neither `status.reset()` nor a rebuild — three steps of erase-era repair went away with the + erase. And the cursor ends on its row **by construction**, since printing is what pushes it there — + which is also the mechanism behind "ein paar Mal Enter und alles sitzt wieder", so a screen that has + lost rows is repaired by a clear rather than left crooked. + + **Ctrl-L is the same command through another door, and this console now owns the binding.** JLine's + keymap dispatches Ctrl-L by *name* to the widget registered under `LineReader.CLEAR_SCREEN`, and + JLine's own widget wipes and redraws the line — i.e. it reproduced the identical defect, measured on + an interpreted screen. Replacing the map entry re-points the key without touching the keymap. **It + deliberately does not take the `writing` lock**: a widget runs on the reader's thread with the + reader's own lock held, while `line()` takes `writing` first and the reader's lock second, so + acquiring `writing` there inverts the order and hangs the session. What serialises it instead is the + reader's lock, which every `printAbove` needs; a concurrent block refresh can still interleave, which + is the exposure JLine's own Ctrl-L widget has today as well. + + **Where each half is pinned.** `ScreenUseCasesTest` asserts the part that is only visible on a screen + — after `/cls`, and again after Ctrl-L, the cursor is on `rows - 1 - blockRows`, the prompt is on that + row, and every row above it is blank (checked one row at a time, so a failure names the row). All four + were red before the change, with the cursor on row 0 and no prompt anywhere. + `JLineTerminalTest.aClearScrollsAWindowAndErasesNothing` pins the pair a pipe *can* see: a window's + worth of line feeds, and no `ESC[2J` at all. That test changed sides — it used to assert the exact + opposite ("a clear must not scroll") and was right for as long as clearing meant erasing; both sides + are recorded in it. Two byte-level assertions written for the erase (`erase display reached the + screen`, `the block is drawn again after the wipe`) were **deleted** rather than adapted: what they + described no longer happens, and the behaviour they were reaching for is asserted on the screen. + + **The screen is scrolled to the bottom once, before the first prompt** (`scrollToBottom`). The + reader draws its prompt at the cursor, i.e. after the last line printed, while only the status + block is pinned to the window — so on a half-empty screen the input floats in the middle with the + block far below it, and they only meet once output has scrolled the cursor down by itself. That is + why it looked right after a few turns and like an ordinary prompt at the start. Emitting + `rows - 1` newlines once makes it the state from the first prompt on; from then on every printed + line scrolls and the cursor stays on the last row. The cost is a screenful of blank lines above the + session, which is what a program that wants its input at the bottom *without* taking over the + screen has to pay. + + **The repeated `> Hallo` is a JLine defect, reproduced and fixed upstream-side.** Typing without + Enter and then dragging the window showed the prompt and buffer a dozen times side by side. It is + **one copy per size event** — a drag reports a new size per step — and an earlier reading of this + ("the repeats are written while typing and widening only reveals them, so the count follows + keystrokes") was an inference from a number rather than a measurement, and wrong. A three-row + `Status` is equally necessary: the same drag without one is clean, and typing without a resize is + clean too. Ruled out along the way: project code, JLine 4.4.6, `nativeSignals(false)`, a missing + terminfo capability, Windows itself (it reproduces on a virtual `xterm`), and the `lastStatusSize` + guard in `redisplay()` that was the documented hypothesis before the resize path was measured. The + cause is `handleSignal(WINCH)`'s status branch — see the next paragraph. Full record, including the + two measurements that were *inconclusive rather than negative*, in + [`docs/upstream-investigation-jline-status-windows-redraw.md`](../docs/upstream-investigation-jline-status-windows-redraw.md). + **Nothing here can honestly fix it**; `--plain` pins nothing and is unaffected. + + **Every output line is folded by this console, so the CONSOLE never wraps one** (`JLineTerminal.fold`, to + one column less than the window). This is the fix for the whole family of drag artefacts and the reasoning + is the load-bearing part: a line the console wrapped is **one** logical line spanning two screen rows, and + Windows **joins such lines again when the window is widened**. The text above then needs fewer rows and + everything below moves **up** — including the block rows last rendered, which end up above the pinned region + where nothing ever writes again. One leftover per drag step, which is the reported staircase of rules + climbing "von unten rechts nach oben links"; narrowing does it in reverse and walks the input upwards. + **No program can observe a reflow or prevent one — but it can deny it a target:** a line that was never + soft-wrapped has nothing to join. + **The harness could not reproduce this, and that is why every grow test was green while the console was + not:** `ScreenTerminal` pulls scrollback down when it grows and **does not reflow at all**. The property is + testable, though, and that is what the two screen cases assert — after printing a long line, **no row on + screen reaches the last column** (red before: rows 8 and 9 were full width). Five unit tests pin the fold: + a short line is untouched, no piece exceeds the width, a double-width glyph is never split (a piece may come + out a column short instead), ANSI a caller put in survives and does not count towards the width, and + umlauts stay whole. Folding counts **screen columns** via `AttributedString.fromAnsi`, never characters — + an icon is one character and two columns, which has cost this class a defect before. + **The price is stated rather than hidden:** text keeps the line breaks it was printed with, so widening no + longer re-flows the conversation. That is the same trade this console already makes by rendering + append-only. + + **The pinned block is rebuilt whenever the window changes size, from a poll** (`startWatchingSize`, + every 120 ms). What JLine holds are the rows it was handed, so a rule built for a 113-column window + stays 113 columns wide: on a resize the row is padded with spaces or cut with an ellipsis, never + re-made, and the next row then continues on the same screen line — the three rows run together with + growing gaps, which is what was reported three times. Only the caller knows a rule is meant to span + the window. **Both halves are needed and the second is easy to miss:** `status.resize(size)` first, + or the pinned region cuts the rebuilt rule straight back to the old width. **Measured, not + reasoned:** a probe on the real console reproduced the report when it left the block alone and + rendered cleanly when it rebuilt all three rows per size event — after four harness theories had + been measured and discarded (buffer-vs-window width, reflow by joining the rows, a + wide→narrow→wide drag, an accumulating cursor drift), and with that probe recording window and + buffer at identical widths throughout and one size event per ~125 ms for a single drag. + **It carries no test, and a written one was deleted rather than kept:** the stream-backed harness + has no screen model, so all it can observe after a size change is that JLine re-emits the rule at + its *old* width without the poll and emits nothing with it — neither says the rule was redrawn at + the new width, and the first assertion built on that passed with the poll disabled. Green either + way is worse than none (the same call already made for two write-lock tests). The evidence is the + probe, on the console where it happens. **The poll DOES re-assert the region + (`status.resize(size)`), and that line was once removed for a measured reason before coming back for + another:** it writes to the terminal directly, so without `Status`'s methods being synchronized it landed + inside what the reader was drawing for the same size change and printed `36;1H` as text with the block + doubled. That race is the fourth JLine fix carried here, so this line depends on that fix and must not be + kept without it — `refreshingTheBlockWhileTheReaderRedrawsNeverThrows` is what holds the pair together. + + **The pinned region ADDRESSES each of its rows, which is the sixth JLine fix and the one that explains + the screens full of rule fragments.** One measurement settled a long chase: a status update emits + `ESC[8;1H` and then the rows back to back, padded to the reported width — **one cursor address for the + whole bar**. The second row begins on a new screen row only because writing the last column of the first + made the terminal wrap. A screen that is wider than the reported width (a window mid-drag, or one whose + terminal has reflowed its buffer) therefore never wraps, and **every reserved row lands on one screen row, + side by side**. And because the bar is reserved from the *bottom*, what the collapse pushes past the window + lands in the **output area above it**, where nothing writes again — one fragment per drag, which is why it + accumulated and why only `/cls` (which scrolls) cleaned it up. + `Display.addressesEveryRow()` (false by default, so no other display changes) switches off the + pending-wrap compensation, and `Status.MovingCursorDisplay` turns every cursor move into an absolute + address. **Two narrower variants were measured and are wrong**, both recorded in the code: addressing rows + at the top of the update loop makes even an unchanged update save and restore the cursor, which interleaves + with the reader's writes and printed the typed text **one character per screen row** (deterministic, 2/2); + addressing only row starts leaves a row that shares a prefix unaddressed, so the row above's pending wrap + is never finished and the state row came out shifted one column, `" state]"` (deterministic, 3/3). + **And it shipped with an off-by-one that JLine's own tests structurally could not see:** `Display` counts + positions in `columns + 1` per row (`columns1`) throughout, the address divided by `columns`, so row *N* + landed at column *N* — rule at 0, activity row at 1, state row at 2. Every existing test compares + **trimmed** rows, which is exactly what hides it; it took the interpreted screen here to catch it, and + `StatusWrongWidthTest.everyRowStartsAtColumnZero` now asserts on the raw row. + + **The prompt is never addressed at all, and that is the row that is still lost on a resize.** Bisected on + the interpreted screen with the real block: after `/cls` the cursor is right (16 of 16), after **JLine's + WINCH handling alone** it is wrong (19 of 20), and neither this console's row rebuild nor its repaint + changes it either way. The byte stream says why — the status region clears its band, re-establishes the + scroll region, restores the cursor, the block is addressed row by row, and then the prompt is written as a + bare `>` **wherever the cursor happens to be**. Nothing relates it to the region below. A growing window + moves the screen's content down by as many rows as it has scrollback to pull from, which need not be the + number of rows added, so the prompt ends up a row or two above the rule. + **Not patched, and the reason is a contract rather than cowardice:** a prompt directly above the pinned + region is what *this* application wants (it scrolls to the bottom once at startup), while JLine promises + only "the prompt is drawn at the cursor", which for a half-empty screen is correct. "Move the cursor to the + bottom of the scroll region on a resize" would be right here and wrong there. + `ScreenUseCasesTest.aGrowingWindowLeavesThePromptOnTheRowTheBlockLeavesForIt` carries the reproduction with + the bisection in its `@Disabled` text; `/cls` repairs it in one keystroke, which the same measurement + confirms. + + **The startup line names the terminal and the JLine patch level** (`LocalAgent.describeTerminal`), because + its absence cost two rounds of testing: a screenshot from a console says nothing about which library + produced it, and twice a defect was chased whose fix the jar in use did not contain. Probed by **method** + (`Status.repaint` for the fifth fix, `Display.addressesEveryRow` for the sixth), never by version string — + the patched builds overlay classes into the released jar and keep its manifest version, so the string + cannot tell them apart. + + **A screen model that REFLOWS, because the one that does not reported every drag case green** + (`ReflowingScreenHarness` + `ReflowResizeTest`, and `ReflowProbe` (in the agent's test tree, `…/atmosphere/probe/`) for the real console). JLine's + `ScreenTerminal` adjusts its buffer on a resize and **never reflows**; a real console does, and that single + missing behaviour is why several rounds of reproduction failed. The harness adds it and nothing else: it does + not parse VT (JLine's screen keeps doing that), it reads the screen on a width change, rebuilds the logical + lines, re-wraps them and writes the result back **behind JLine's back** — the honest channel, since a console's + reflow changes the screen without telling the program. Both of its rules are explicit so they can be checked: + a row **continues** into the next when its last cell is not blank (the terminal's own rule in practice), and + which edge keeps its content when joining frees rows is a **parameter** (`Anchor.BOTTOM`/`TOP`), not a guess. + `ReflowingScreenHarnessTest` puts the harness itself under test. **Its limit is stated:** the cursor is not + reflowed with the content, so it is evidence about what is ON the screen after a resize, not about where the + cursor lands — those cases stay with `ScreenTerminalHarness`. + **The insight that came with it:** folding makes nothing soft-wrapped *at the width it was printed at*, so + making the window NARROWER turns those same lines into wrapped ones and the next widening joins them. That is + why enlarging alone looked fine while alternating "zerhackt alles" — the shrink manufactures what the widening + then moves everything with. + + **The eighth JLine fix: `Status.resize` must not erase the rows above the bar.** The first thing the emulator + showed was not a reflow but an **erase** — after a shrink the conversation was simply gone. Reproduced without + any reflow at all, on the ordinary harness: three answers on a 100-column window, halve the width, and **two + are erased**. `Status.resize` clears a band when the geometry changes and pulled its start upwards by + `(ceil(oldColumns / columns) - 1) * statusLines` rows "to account for wrapped status lines" — six rows above a + three-row bar in a halved window, straight through the application's output. Since the seventh fix a status row + is padded one column short and **cannot wrap**, so there are no extra rows to account for. Red/green both + sides: `StatusRepaintTest.makingTheWindowNarrowerDoesNotEraseWhatIsAboveTheBar` fails with the compensation + restored and passes without it; `makingTheWindowNARROWERmustNotERASETheConversation` is red against + `statusfix7` and green against `statusfix8`. **This is the answer to "nach dem kleiner ziehen sehe ich es nicht + mehr":** the output was never scrolled away, it was erased by the bar's own housekeeping. + + **A status row must not write the last column, which is the seventh JLine fix.** `Status.update` padded every + row to the **full** reported width. A row padded to a width the screen does not have **wraps**: it occupies + two screen rows where the bar reserves one, so the bar's *last* row wraps past the bottom of the screen and + **scrolls** it — the bar moves up and leaves a copy of itself above, one per bad render. That is the reported + "ganz viel kleiner / größer abwechselnd zerhackt alles", a screen carrying several complete bars at different + widths. The `…` some of those rules ended in is JLine's own marker for a row wider than the region, which is + what pointed at it. + **Padding one column short costs nothing** now that every row is addressed rather than reached by wrapping + (the sixth fix), and it additionally stops the terminal from marking the row as **wrapped**, which is what let + a reflow join it to its neighbour. Measured with a screen reporting one column more than it has — the lag a + dragged console really shows: the state row came out as `" ▤ X:/tmp/…"`, its `[` eaten by the wrap; with eight + columns over, eight characters. Both are the same thing the earlier `"state]"` reports were. + **Our own half of that race was measured and is already closed:** a harness whose `getSize()` returns a + different width on every call (a drag that stops) produces no ellipsis and a whole state row, because this + console reads the size **once** per block build and hands that one read to both the region and the rows. The + case is kept as the guard for it. + **What the harness cannot show is the stack itself** — `ScreenTerminal` does not reflow, and JLine's diff skips + a render whose content has not changed, so only the first bad render damages it. What it does show, a bottom + row eaten by a wrap, is the same mechanism one render at a time. + + **The redraw after the size SETTLES is a repaint, not a diff** (`repaintBlockFromScratch`, 400 ms after + the last size event). This is the fix for "kleiner ziehen sah gut aus, größer macht noch Probleme", and it + rests on a mechanism worth knowing: JLine pads every region row to the width the terminal **reports** and + writes the rows one after another, relying on the terminal wrapping at the right margin to start the next. + A screen that is **wider** than the reported width therefore never wraps, and the next row continues on + the same screen line — a rule with the activity row cut short beside it, the next rule where that left + off, and the *last* row still perfectly correct (the region addresses its first row and then writes on, so + only the rows in between collapse). Reproduced deterministically with a harness that reports eight columns + fewer than its screen has. + **Two causes reach that state and only one recovers by itself.** A console reporting a width it has not + applied does it briefly, and the existing rebuild repairs it (measured, including a window that also grows + a row taller). **Windows also reflows its screen buffer when the window is widened**, joining rows it had + marked as wrapped — which is every region row, since each is padded to the last column — and JLine is + never told. Its `Display` still matches what it wrote, so every later update computes an **empty diff and + emits nothing** and the joined rows stay for the rest of the session. That is the half no earlier theory + explained: the artefact *persisting* while every redraw runs, and repairing itself when Enter is held + (printing does not go through the region's diff at all). + **The repaint is two passes:** the same number of rows, blank, then the real rows — the first makes the + second a real write and erases the rows on the way. **`Status.reset()` is the call that looks right and is + wrong:** it forgets the scroll region too, so the next update believes it must grow the region and scrolls + to make room — the stale rows were pushed *up* rather than cleared and the block stood on screen **twice**, + four rows apart (measured on the interpreted screen, the only place that is visible). An empty block has + the same problem for the same reason: it changes the region's height. Only the settle redraw repaints; a + drag reports a size every ~125 ms and repainting on each is bytes spent against a screen about to change + again. + **What it cannot reach:** the prompt has a display of its own with the same diff and no `reset()` a caller + can call, so a reflow that damages the prompt's row is still beyond repair from here — + `thePromptItselfStaysVisibleAfterEnlarging` records that. `/cls` and Ctrl-L do repair it, because they + only print. + + **A settled WIDTH change wipes the screen, in EITHER direction; a height change does not.** This is the last of the drag defects and + the only one that could not be prevented, only removed. A console **reflows** when the width changes — measured + on the reporter's, with a number rather than a screenshot, because a pasted screen cannot answer it (Windows + Terminal copies the scrollback and rejoins wrapped runs, so wrapping is invisible in a paste): a 32-row window + widened from 86 to 111 columns left the cursor on **row 28 of 32** where 31 is the last, exactly the three rows + that joining three wrapped lines freed; narrowed to 72 it stayed on 31. So the **top** keeps its content and + everything below moves **up** — and the block's rows are ordinary screen rows, so they are carried out of the + region while the next render draws a fresh block at the bottom. The carried-up copy then sits above the region + where nothing writes again: one per size event, one every ~125 ms during a drag, which is "beim größer ziehen + wieder hunderte male die Linie". + **Both halves of "cannot be prevented" were measured.** Taking the block off the screen at the first event of a + drag fixes an *alternating* drag but not a plain shrink-then-widen: the console reflows immediately while this + console learns of the size up to 120 ms later, so the block is unavoidably on screen then. Nothing can find the + copy afterwards either — a caller cannot read the screen. Scrolling removes it, which is what `/cls` does and + why that command was always the repair. The trade was the user's call: the visible conversation scrolls out of + view on a width drag — and is then **printed again**, which is the part that makes the wipe affordable. + Every line handed to `line()` is remembered **before** folding (a bounded ring of the last 300), and after the + wipe as many of the most recent as fit above the block are printed again, oldest first, counted in **screen + rows** rather than lines because folding can turn one line into several. Storing the text rather than the + drawing buys what a reader expects from a window they just made bigger: the conversation **re-flows** — a line + that needed two rows in a narrow window takes one in a wide one + (`whatIsPrintedAgainIsFoldedForTheWidthTheWindowHasNOW` measures both states). It was wiped without this for + one round and the report was a single sentence: "allerdings sehe ich den Verlauf nicht mehr". A height change + needs none of this (no width change, nothing re-wraps) and keeps the conversation where it is, with the prompt + printed back to its row instead. + **The redraw only prints; it must not remember.** `line()` does both, and routing the redraw through it put the + whole visible conversation into the ring a second time on every wipe — one turn ended up on screen four times + ("der Text erscheint zwar wieder, aber mehrmals"). `remember` and `print` are separate for that reason. It also + takes **two** width changes to see it: after the first wipe the ring holds the line twice but only one copy has + been printed, which is why the case that shipped with the redraw was green while the console was not. + **It was narrowed to "only when the width grew" once, on reasoning rather than evidence, and the next report + came from the other direction:** narrowing does not join lines, it **splits** them, and the bar's own rows — + built for the old width — no longer fit and are re-wrapped across several screen rows, so the bar needs more + rows than the region reserves and everything above it is pushed up ("beim kleiner ziehen wandert es nach oben + mit ganz vielen Zeilen"). Both directions leave rows behind that only scrolling removes. Note what the wipe is + **not**: the eighth JLine fix stops the library from *erasing* those rows, which is unrecoverable, while + scrolling them into the scrollback is not — that fix still matters for every consumer that does not wipe. + + **A height change moves the prompt off its row, and the settle prints it back** + (`pushThePromptBackToItsRow`). Measured on the interpreted screen with the real three-row block: growing 20 + rows to 26 left the prompt on row 19 where 22 is right, shrinking 20 to 14 left it on row 13 where 10 is + right — **half the height change in both directions**, and "half" is an artefact of that buffer state rather + than a rule (the screen moves content by as many rows as it has scrollback to give or room to delete, which + is what a console does). **A width change moves nothing at all**, which is folding paying off, so nothing is + printed for one. + **The growing half is fixed and the reason it works is the direction:** the drift cannot be computed, but it + is bounded by the height change, and printing moves the cursor down one row per line until it reaches its row + and then merely scrolls — so printing that many lines lands it correctly without knowing where it was. It is + the same repair `/cls` performs with a whole window's worth, which is why that command always worked. + Measured 19 → 22; `aGrowingWindowLeavesThePromptOnTheRowTheBlockLeavesForIt` was the open reproduction and is + now green twice in a row. The price is bounded by the change: up to that many blank rows enter the + conversation and up to that many lines of it scroll out of view. + **The shrinking half stays open**, because there the prompt must move *up*: it lands inside the pinned band + and the block draws over it ("nach dem kleiner ziehen sehe ich es nicht mehr"). Nothing a caller can emit + moves the cursor up without breaking the reader's bookkeeping — `cursor_address` inside `printAbove` was tried + twice and stranded characters above the prompt both times. + `aWindowThatGetsSHORTERLeavesThePromptOnTheRowTheBlockLeavesForIt` carries it, `/cls` repairs it. + **Measuring the cursor was ruled out, not overlooked:** a cursor-position report is a round trip through the + terminal's input, which the reader owns all session, so it would race the keyboard and could swallow a + keystroke or leave `[24;1R` in the input line — the exact class of defect this whole investigation has been + about. + + **Do not add a `WINCH` handler, and the reason is measured.** A resize drawing a row of + `> > > > >` across the screen looks like the pinned region not being told about the new size, so a + `Signal.WINCH` handler that resized and re-rendered it was added — and the user reported it + **worse**, not better. `LineReaderImpl.handleSignal(WINCH)` already calls `Status.resize(Size)`, + and the reader installs its own handler for as long as it is reading, which is the whole session; + ours therefore either never ran or ran *in addition*, putting a second writer on the terminal from + the signal thread at the exact moment the reader was redrawing. A probe driving a real + `terminal.raise(WINCH)` against a **pipe-backed** terminal shows JLine doing it correctly on its + own: scroll region reset, the rule re-cut to the new width, **one** prompt — but that probe was + measuring the wrong thing, because a pipe has no screen to redraw onto. Against JLine's own + `VirtualTerminal` (a real VT interpreter over a virtual screen) the same drag reproduces the report + exactly, on `xterm` and `windows-vtp` alike, and the cause is a **one-line JLine defect**: + `handleSignal(WINCH)`'s status branch calls `doDisplay()`, which replaces the `Display` with a fresh + one that believes the screen is blank, so the following `redisplay()` paints the prompt as new + content once per size event instead of as a diff (`display.resize(size)` is the fix, verified with + three new tests plus JLine's 72 existing ones). Still nothing to fix *here*, and no project test may + assert the fixed behaviour while the build depends on an unfixed release. + `fit()` measuring in **screen columns** (`AttributedString.columnLength`) rather than characters is + ours and does matter: an icon is one character and two columns, and a row wider than the window + wraps onto a second screen line, which the reserved region cannot survive. + + **Two things tried and removed, both because measurement said they did nothing.** (1) Skipping a + status write when the block is unchanged — removing the guard again left the emitted bytes + identical, because JLine already skips an unchanged block. (2) Re-cutting the rows when the window + width changed — `Status.resize()` re-cuts the rows it holds itself. Neither was kept with a comment + claiming a benefit it does not have. The stray `?1h` consequently still has **no established + cause**: the lock covers our writes, the reader's own are inside JLine. + + **Restoring the block after a wipe took three steps — and the whole repair went away with the wipe.** + It is kept as a record because it is the same mechanism that eventually retired the wipe itself. The + steps were `status.reset()`, then `status.update(List.of())`, then rendering it again from `requested` + (the text the caller gave, not the rendered rows); with only the first two, `Status` drew the + difference it computed against a belief the wipe had invalidated — observed as a single character + emitted where a whole block was missing. The reader's own prompt has exactly that problem and no + equivalent repair, which is why an erase left no prompt on screen at all. A clear that only scrolls + invalidates nothing, so the block is simply left alone. + + **The turn after an interrupted one is pinned end to end** (`InterruptedTurnTest`): with a scripted + backend behind the real `OpenAiCompatServer`, a turn is cut short by pending input and the next one + is then driven through — it reaches the server, carries the interrupted question in its history, + and answers. Reported as "it does not carry on by itself"; the mechanism works, so the cause of + that report is elsewhere and is **not** claimed to be fixed. Two things the writing of it settled: + a turn that finishes inside one activity tick is never even looked at for interruption (correct — + there is nothing to cut short), which is why the scripted backend has to be made slow or the test + proves nothing; and a second line typed during the replacement turn stops that one too, which is + the design and not a defect, but looks from the outside exactly like a turn that never started. + + **A blank line must not count as pending input.** `hasPendingInput()` ignores blank lines but leaves + them queued: counting them meant that holding Enter cancelled one turn per keystroke and produced + nothing, while dropping them would break the approval prompt, where an empty answer means yes. + `DISABLE_EVENT_EXPANSION` is set in the same builder because the reader's default treats `!` as a + shell history expansion, which silently rewrites a request like `git commit -m "fixed!"`. + **What cannot be done, asked and answered:** keep the block visible while the *user* scrolls the + terminal's scrollback. That needs the alternate screen buffer, i.e. a full-screen application, which + would give up the scrollback and the append-only property the whole console design rests on. + + **The prompt stays at the bottom during a turn, and typing stops the turn.** One thread inside + `JLineTerminal` (`startReading`) sits in `readLine` for the whole session and fills a queue; + **every** read in that class is served from it, because a terminal has one keyboard and two threads + reading it take turns at random. That is why `readKey` no longer reads a single key in raw mode: the + question is printed above the prompt and answered in the same input line (`y` + Enter). End of input + cannot be a queue value, so a sentinel is queued and **put back on every take** — otherwise the + second reader after Ctrl-D would see "nothing typed yet" instead of "no more input". + `AgentTerminal.hasPendingInput()` is the peek the turn loop peeks with; it deliberately does **not** + consume, so the line the interruption was triggered by is still there for the next `readLine` and + becomes the next message. The stop itself is `AgentRunner.start(...)` → + `runtime.executeWithHandle(...)`, whose handle closes the in-flight SSE stream (Atmosphere's own + "D-6 built-in hard-cancel"); that also replaced `turn()`'s hand-rolled worker thread, since + `executeWithHandle` dispatches on a virtual thread and returns at once. `awaitWithActivity` returns a + three-valued `TurnEnd` rather than a boolean, because *interrupted* must not be reported as the + *timed out* error the old `false` produced. **Order in that loop is load-bearing and a test pins the + behaviour**: the `activity.isPaused()` check comes first, so while an approval question is open a + typed line is its answer and not an interruption. **What this is not:** Claude Code injects a + mid-turn message into the running loop; `AgentExecutionContext` is a record whose request is built + once from `message()` + `history()`, with nothing to append to, so stop-and-resend is the achievable + equivalent — and it acts immediately instead of waiting out a tool loop. + +**Three front ends on one `AgentSession`: console, `--web`, `--acp`.** Everything that is not +presentation lives in `AgentSession` — history, the slash commands (`submit` returns `CONTINUE`/`EXIT`), +`/compact`, `/loop`, `/retry`, the transcript, the approval mode, cancellation (`cancel()`, +`stopRequested()`, `isBusy()`), and `runTurn` itself. A front end implements `SessionFrontend`: `line`, +a `renderer()` (a `StreamingSession` the turn forwards every event to), `await` (how it waits for a +turn — the console with its activity block, the others with `AgentSession.awaitQuietly`), `approvals()`, +`ask()` (a free question; `null` means the front end has none, and `/loop` then refuses rather than +guesses) and `commandOutput()` (a running command's lines). The console is `ConsoleFrontend` over the two +`AgentTerminal`s; `LocalAgent` is now only option parsing, the choice of front end and the console's +activity rendering. Three pieces are what keeps them from drifting apart: + +- **`TurnRecorder` records, the front end renders.** The recorder is the `StreamingSession` handed to + Atmosphere; it keeps text, tool rounds, token counts and the running tool, supplies the workspace + filesystem via `injectables()`, and forwards everything downstream. `ConsoleSession` is now just a + recorder with a `ConsoleRenderer`. So the tool note in the history, `/calls` and the transcript are + identical whichever front end showed the turn. +- **`ModeGatedApprovalStrategy` wraps every front end's strategy.** Atmosphere asks the strategy per + call; the wrapper reads the session's mode *at that moment*, so `/mode auto` in the browser, *Allow + all* in an editor and Shift+Tab in the console all take effect for the very next call, and a front + end with nobody to ask (`approvals()` = `null`) denies. `ConsoleApprovalStrategy` keeps its own AUTO + check as well; it is redundant under the wrapper and harmless. +- **`AgentSessionTest` + `RecordingFrontend`** drive the session with no terminal, no browser and no + editor, over the real `OpenAiCompatServer` and a scripted engine; the three front-end test classes + then only need to prove their own mapping. + +**`--web` (`WebServer`, `WebAgentEndpoint`, `WebFrontend`, `WebAccessGuard`, `WebConsole`).** Embedded +Jetty 12 (ee10 servlet + Jakarta WebSocket; `jetty-ee10-annotations` excluded, nothing here needs +annotation scanning) with an `AtmosphereServlet` whose annotation map is set **explicitly** to the one +`@AiEndpoint`. **The scanning trap, which cost an afternoon:** Atmosphere's `ClasspathScanner.preventOOM()` +turns classpath scanning **off** whenever `org.junit.jupiter.api.Assertions` is on the classpath, and in +servlet-initializer mode the explicit map is only processed *during* a scan — so the endpoint registered +fine from the jar and was missing in every test ("No AtmosphereHandler or WebSocketHandler installed"). +The init params `ANNOTATION_PACKAGE="all"` and `SCAN_CLASSPATH="false"` make the explicit map the path in +both. One `AgentSession` per process, shared by every tab (`WebAgentEndpoint.SESSION`); a message while a +turn runs cancels it first, `/stop` only cancels, and `connection.complete()` is always sent so the +console's input unlocks. Approvals use Atmosphere's own `/__approval//approve|deny` protocol through +`ApprovalStrategy.virtualThread(registry)`, which the prebuilt console renders as Approve / Deny. +`WebAccessGuard` is a Jetty `Handler.Wrapper` in front of everything: loopback `Host` when bound to +loopback (DNS rebinding), `Origin` must match `Host`, `?token=` exchanged for an `HttpOnly; +SameSite=Strict` cookie plus a redirect, else `Bearer` or cookie, else `401` — constant-time comparison. +`WebConsole` serves the console pages with a fresh CSP nonce per response and answers +`/api/console/info`. `WebServerTest` (11) drives it over the JDK `WebSocket` speaking the atmosphere.js +wire protocol (`AtmosphereTestClient`: `len|payload` frames, `X-atmo-protocol`); **create the client +object before `buildAsync`**, or the handshake message races the listener into a thrown exception and +`request(1)` is never called — a flaky test that looked like a server bug. + +**`--acp` (`AcpServer`, `AcpFrontend`).** The ACP Java SDK (`com.agentclientprotocol:acp-core` + +`acp-json-jackson2`, 0.18.0) over `StdioAcpAgentTransport`. Its sync handlers run on the SDK's own pool, +so a blocking `session/prompt` does not block `session/cancel` or the answer to a permission request. +One `AgentSession` per ACP session, built for the editor's `cwd` (`AgentOptions.withWorkspace`); the +model endpoint is opened once per process. **stdout is the protocol**: `run()` keeps the real stdout for +the transport and points `System.out` at stderr, so a stray `println` cannot corrupt the stream (the +smoke asserts every stdout line is JSON). Mapping: text → `agent_message_chunk`; `ToolStart` → +`tool_call` (`call_N`, `ToolKind` from the tool name, `locations` resolved against the workspace, raw +arguments), `ToolResult`/`ToolError` → `tool_call_update` (result cut at 4000 chars), running command +output → `tool_call_update` IN_PROGRESS at most every 500 ms with the last 40 lines; approvals → +`session/request_permission` with allow / always / reject, where *always* switches the session to AUTO +and sends `current_mode_update`, and no answer (timeout, cancelled dialog, editor gone) is a deny; the +approval modes are ACP session modes `manual`/`auto`; the slash commands (minus `/exit`, `/cls`) go out as +`available_commands_update` 100 ms after `session/new` on a virtual thread, because an update that +arrives before the `session/new` response names a session the editor does not know yet. +`AcpServerTest` (9) deliberately drives it with **plain JSON** (`AcpTestClient`), not the SDK's client — +an SDK on both ends agrees with itself even where it disagrees with the protocol. Tear-down closes the +editor side first: that is how an ACP session ends, and closing the server first logs an +`InterruptedIOException` "Transport error" per test. + +**Two defects fixed on the way (step 0), both pinned in `LocalAgentTest`:** the shell tool's live-output +sink captured the console *before* it existed, so every command's output came back as +`[output unavailable: NullPointerException]` (now an `AtomicReference` set once the terminal is up); and +`/clear` left the pending tool note and the message `/retry` would resend, so the first turn after a +clear still carried the old conversation. + +**The default system prompt is general-purpose on purpose — do not narrow it back.** Every model-facing +text is a resource, not a Java literal: `src/main/resources/net/ladenthin/llama/atmosphere/` holds +`system-prompt.txt`, `system-prompt-shell.txt`, `system-prompt-no-shell.txt` and `run-command-tool.txt` +(each with a `.license` sidecar for REUSE), loaded by `LocalAgent.prompt(name)` with `{placeholder}` +substitution; `AgentOptionsTest.promptResourcesLoadAndEveryPlaceholderIsFilled` fails on a missing file or +an unfilled placeholder. `LocalAgent.systemPrompt` +and the `ShellTool` description describe `run_command` as running *any* command line through the named +shell (`ShellTool.shellName()`), not limited to the workspace, and tell the model to run a command rather +than explain one. The earlier wording ("careful *coding agent*", `run_command` "to build, test or inspect +the project" / "build, test, grep or list files") made Qwen3-4B refuse "list the docker images" — "my +tools are only for files" — with the tool registered and `docker` on `PATH`; a fresh single-turn run +refused too, so it was the prompt, not the chat history. Without `--allow-shell` the prompt says commands +are unavailable and names the flag, so the model does not invent its own limitation. Pinned by +`AgentOptionsTest.defaultSystemPromptIsGeneralPurposeAndAllowsAnyCommandWithTheShell` and +`shellToolDescriptionDoesNotNarrowItToTheProject`. `ShellToolTest` runs on every platform: each test +picks its command line with `ShellTool.isWindows()` — the same detection `ShellTool.run` uses to choose +`cmd.exe /c` over `sh -c` — so `ls`/`dir /b`, `sleep 30`/`ping -n 30 127.0.0.1 >nul`, and the truncation +test counts the platform's line separator. It used plain POSIX commands before and failed 4 of 5 on +Windows; never skip it per OS, give a new test both command forms instead. + +**Version bump note.** The pom's `llama.version` property is the **release** version, not the +reactor's `-SNAPSHOT` (CI always overrides it, so a not-yet-published default never breaks CI). +`versions:set` does not touch this standalone pom, so at release time bump it by hand together with +the fat-jar filename `llama--jar-with-dependencies.jar` in the two READMEs (`README.md` +"Local coding agent" + the project's own README) — the same class as the `llama-langchain4j/README.md` +snippet. Before the release, run it against a pre-release core with `-Dllama.version=-SNAPSHOT` +(the pom keeps the Sonatype snapshot repository for exactly that). From efc67da3791e6c6aa55f661ffc6810567b4d8859 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 06:32:58 +0000 Subject: [PATCH 05/22] Restore six line continuations lost in the workflow cleanup 551f527d stripped the trailing backslash from six run-script lines whose next line starts with || or &&, which makes each script a bash syntax error: build-webui (every native build needs it), the two AAR/consumer checks of package-android-aar and github-snapshot. Found by running bash -n over every run block; actionlint does not parse the scripts without shellcheck. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/workflows/publish.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 9652a7170..f3fae613b 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -604,7 +604,7 @@ jobs: # all, which is why the naive grep used to be sufficient.) Assert the ACTIVE # #define and a non-zero asset count instead -- verified against both paths. N=$(sed -n 's/.*std::array.*/\1/p' "$OUT/ui.h" | head -1) - if grep -qE '^[[:space:]]*#define[[:space:]]+LLAMA_UI_HAS_ASSETS' "$OUT/ui.h" + if grep -qE '^[[:space:]]*#define[[:space:]]+LLAMA_UI_HAS_ASSETS' "$OUT/ui.h" \ && [ -n "$N" ] && [ "$N" -gt 0 ]; then echo "LLAMA_UI_HAS_ASSETS: present, $N assets embedded" else @@ -1039,10 +1039,10 @@ jobs: for entry in $ENTRIES; do unzip -l "$AAR" | grep -q "${entry}$" || { echo "::error::$AAR is missing $entry"; exit 1; } done - unzip -p "$AAR" AndroidManifest.xml | grep -q 'android:minSdkVersion="28"' + unzip -p "$AAR" AndroidManifest.xml | grep -q 'android:minSdkVersion="28"' \ || { echo "::error::$AAR manifest lost minSdkVersion 28"; exit 1; } unzip -p "$AAR" classes.jar > /tmp/aar-classes.jar - unzip -l /tmp/aar-classes.jar | grep -q "net/ladenthin/llama/LlamaModel.class" + unzip -l /tmp/aar-classes.jar | grep -q "net/ladenthin/llama/LlamaModel.class" \ || { echo "::error::$AAR classes.jar lost LlamaModel"; exit 1; } if unzip -l /tmp/aar-classes.jar | grep -qE "module-info\.class|net/ladenthin/llama/(Linux|Mac|Windows)/"; then echo "::error::$AAR classes.jar carries module-info or desktop native resources"; exit 1 @@ -1065,11 +1065,11 @@ jobs: APK=.github/android-consumer-test/app/build/outputs/apk/release/app-release.apk test -f "$APK" for abi in arm64-v8a x86_64; do - unzip -l "$APK" | grep -q "lib/$abi/libjllama.so" + unzip -l "$APK" | grep -q "lib/$abi/libjllama.so" \ || { echo "::error::consumer APK is missing lib/$abi/libjllama.so"; exit 1; } done # The AAR's consumer proguard.txt must have carried the binding through R8. - unzip -p "$APK" "classes*.dex" | grep -aq "Lnet/ladenthin/llama/LlamaModel;" + unzip -p "$APK" "classes*.dex" | grep -aq "Lnet/ladenthin/llama/LlamaModel;" \ || { echo "::error::R8 stripped net.ladenthin.llama.LlamaModel — consumer proguard rules broken"; exit 1; } - name: Upload AARs uses: actions/upload-artifact@v7 @@ -2979,7 +2979,7 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - gh release view snapshot --repo ${{ github.repository }} 2>/dev/null + gh release view snapshot --repo ${{ github.repository }} 2>/dev/null \ || gh release create snapshot \ --repo ${{ github.repository }} \ --prerelease \ From 1de7e2560264418d4fc67f0432bdb7bfea82f999 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 06:33:55 +0000 Subject: [PATCH 06/22] CI: shared-files checks the workflow jobs kept identical across the repositories An entry .github/workflows/publish.yml# in .github/shared-files.sha256 hashes one job of the workflow (its header and body, not the comment lines before the next job). startgate, shared-files, verify-signing-key, check-snapshot and check-tag are identical in all four publish.yml files, and verify-signing-key-gradle, github-snapshot and github-release in the three Maven-only ones; they were identical by convention only and are now checked like files, without moving them into a reusable workflow. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/buildcheck/sharedfiles.py | 50 +++++++++++++--- .github/buildcheck/tests/test_sharedfiles.py | 61 +++++++++++++++++++- .github/shared-files.sha256 | 13 ++++- CHANGELOG.md | 4 ++ CLAUDE.md | 9 ++- 5 files changed, 122 insertions(+), 15 deletions(-) diff --git a/.github/buildcheck/sharedfiles.py b/.github/buildcheck/sharedfiles.py index 1c771ee40..5caf6cbe2 100644 --- a/.github/buildcheck/sharedfiles.py +++ b/.github/buildcheck/sharedfiles.py @@ -5,7 +5,11 @@ streambuffer, checked by every repository's own `shared-files` job. Each repository lists the shared files it carries, with their SHA-256, in -.github/shared-files.sha256 -- `sha256sum` format, so `sha256sum -c` reads it as well. The job +.github/shared-files.sha256 -- `sha256sum` format. An entry can also name ONE JOB of a workflow, +`.github/workflows/publish.yml#startgate`: jobs such as `startgate` or `check-tag` are kept identical +inside four otherwise different workflows, and a job entry hashes just that job's text (its header +and body, without the comment lines between it and the next job, which belong to the next one). +`sha256sum -c` reads the file entries only. The job * fails when a listed file is missing or its content no longer matches its line: a shared file was edited here alone. Edit it in every repository that lists it, then update each manifest (`check-shared-files.py --write` rewrites the hashes of this repository's lines); @@ -23,6 +27,8 @@ import sys import urllib.request +from . import workflow + REPOS = ("java-llama.cpp", "BitcoinAddressFinder", "srcmorph", "streambuffer") OWNER = "bernardladenthin" MANIFEST = ".github/shared-files.sha256" @@ -46,19 +52,39 @@ def parse(text): return entries -def sha256(path): - with open(path, "rb") as f: - return hashlib.sha256(f.read()).hexdigest() +def content(root, entry): + """The bytes an entry stands for: a file, or for `#` that one job's text. + None when the file or the job does not exist.""" + path, _, job = entry.partition("#") + full = os.path.join(root, path) + if not os.path.isfile(full): + return None + with open(full, "rb") as f: + data = f.read() + if not job: + return data + jobs = workflow.parse(data.decode("utf-8")) + if job not in jobs: + return None + lines = jobs[job].lines + while lines and (not lines[-1].strip() or lines[-1].startswith(" #")): + lines = lines[:-1] + return ("\n".join(lines) + "\n").encode("utf-8") + + +def sha256(root, entry): + data = content(root, entry) + return None if data is None else hashlib.sha256(data).hexdigest() def verify(root, entries): - """Failures: every listed file exists and still has its listed hash.""" + """Failures: every listed file (or job) exists and still has its listed hash.""" failures = [] for path, digest in entries.items(): - full = os.path.join(root, path) - if not os.path.isfile(full): + actual = sha256(root, path) + if actual is None: failures.append(f"{path} is listed in {MANIFEST} but does not exist") - elif sha256(full) != digest: + elif actual != digest: failures.append(f"{path} differs from its line in {MANIFEST}: a shared file was changed in " f"this repository alone -- change every copy, then update every manifest") return failures @@ -69,7 +95,13 @@ def rewrite(text, root): out = [] for line in text.splitlines(): match = LINE.match(line) - out.append(f"{sha256(os.path.join(root, match.group(2)))} {match.group(2)}" if match else line) + if not match: + out.append(line) + continue + digest = sha256(root, match.group(2)) + if digest is None: + raise ValueError(f"{match.group(2)} is listed in {MANIFEST} but does not exist") + out.append(f"{digest} {match.group(2)}") return "\n".join(out) + "\n" diff --git a/.github/buildcheck/tests/test_sharedfiles.py b/.github/buildcheck/tests/test_sharedfiles.py index e78d610a3..bf09e9c4f 100644 --- a/.github/buildcheck/tests/test_sharedfiles.py +++ b/.github/buildcheck/tests/test_sharedfiles.py @@ -81,6 +81,64 @@ def test_unknown_arguments(self): self.assertEqual(self.run_main("--nope")[0], 2) +WORKFLOW = """name: t +on: + push: +jobs: + first: + runs-on: ubuntu-latest + steps: + - run: echo one + + # a comment about the second job belongs to the second job + second: + needs: first + runs-on: ubuntu-latest + steps: + - run: echo two +""" +FIRST = " first:\n runs-on: ubuntu-latest\n steps:\n - run: echo one\n" + + +class JobEntryTest(unittest.TestCase): + + def setUp(self): + self.root = tempfile.mkdtemp() + os.makedirs(os.path.join(self.root, ".github", "workflows")) + self.write(WORKFLOW) + + def write(self, text): + with open(os.path.join(self.root, ".github/workflows/w.yml"), "w", encoding="utf-8") as f: + f.write(text) + + def test_a_job_entry_hashes_the_job_alone(self): + self.assertEqual(sharedfiles.content(self.root, ".github/workflows/w.yml#first"), FIRST.encode()) + self.assertEqual(sharedfiles.sha256(self.root, ".github/workflows/w.yml#first"), + hashlib.sha256(FIRST.encode()).hexdigest()) + + def test_the_comment_before_the_next_job_is_not_part_of_the_job(self): + self.write(WORKFLOW.replace("belongs to the second job", "was reworded")) + self.assertEqual(sharedfiles.content(self.root, ".github/workflows/w.yml#first"), FIRST.encode()) + + def test_a_change_inside_the_job_is_seen_and_one_elsewhere_is_not(self): + entry = ".github/workflows/w.yml#first" + entries = {entry: sharedfiles.sha256(self.root, entry)} + self.write(WORKFLOW.replace("echo two", "echo three")) + self.assertEqual(sharedfiles.verify(self.root, entries), []) + self.write(WORKFLOW.replace("echo one", "echo changed")) + self.assertIn("differs from its line", sharedfiles.verify(self.root, entries)[0]) + + def test_a_missing_job_or_workflow_is_reported(self): + entries = {".github/workflows/w.yml#gone": A, ".github/workflows/x.yml#first": A} + failures = sharedfiles.verify(self.root, entries) + self.assertEqual(len(failures), 2) + self.assertTrue(all("does not exist" in f for f in failures), failures) + + def test_write_refuses_an_entry_that_does_not_exist(self): + with self.assertRaises(ValueError): + sharedfiles.rewrite(f"{A} .github/workflows/w.yml#gone\n", self.root) + + class CompareTest(unittest.TestCase): def test_matches_by_path_then_by_a_unique_file_name(self): @@ -104,7 +162,8 @@ def test_the_shared_build_checks_list_themselves(self): with open(os.path.join(REPO, sharedfiles.MANIFEST), encoding="utf-8") as f: entries = sharedfiles.parse(f.read()) for path in (".github/buildcheck/sharedfiles.py", ".github/check-shared-files.py", - ".github/buildcheck/tests/test_sharedfiles.py"): + ".github/buildcheck/tests/test_sharedfiles.py", + ".github/workflows/publish.yml#shared-files"): self.assertIn(path, entries) diff --git a/.github/shared-files.sha256 b/.github/shared-files.sha256 index da88e36dd..7193eedd5 100644 --- a/.github/shared-files.sha256 +++ b/.github/shared-files.sha256 @@ -7,16 +7,18 @@ # .github/workflows/publish.yml (.github/check-shared-files.py, see .github/buildcheck/sharedfiles.py): # a copy changed here alone fails it, a copy another repository lists with a different hash warns. # To change a shared file: change it in every repository listing it, then run -# `python3 .github/check-shared-files.py --write` in each. `sha256sum -c` reads this file too. +# `python3 .github/check-shared-files.py --write` in each. An entry `#` stands for one +# job of a workflow (kept identical inside otherwise different workflows); `sha256sum -c` reads the +# file entries only. 82171de97c6621b4c1f3e9cd66afd49990f6d31331857423b37f2b9a2f35fbc0 .github/buildcheck/__init__.py 3d1a9f2e93709941261a9655bc19bf50d5becefd8f6a6c2f5564096a757996bb .github/buildcheck/workflow.py 5a70eee1bb8af180edfd2a4bae61be61fac15b409f5df6a8bb9869425d7fc381 .github/buildcheck/releasegate.py -86545e924c95b69a16f4b9f759a38779b441aa04d8b635a52eb029a196e39a11 .github/buildcheck/sharedfiles.py +95285b6e9fdd30a71e97aff236e4a23d42101bc182955cae7ca1e4993939321a .github/buildcheck/sharedfiles.py 0059120aab539c6ce8055675ae0d3b040097f80320051e40385072f68845ddb0 .github/buildcheck/tests/__init__.py 561efd3f26d7728e475c8ce83c10b4761a6703b585b7b2a7ab052df10f73166c .github/buildcheck/tests/helpers.py 4985f25fa177bd79a79335742adfd0da0431e101124b4e7e214906b56ae903de .github/buildcheck/tests/test_workflow.py cbe504ad081ba31ec16e11c11f879c64179ca7741bf0b1d1318a7fc969ed67dd .github/buildcheck/tests/test_releasegate.py -7cbef54c09c6612503da7bd3733b4ab51c6f34666ef062885a7586e075a907de .github/buildcheck/tests/test_sharedfiles.py +f040b63c801e1d8b1b27b0fe8ae733d6fb5cf1b166e79470a6c3e3282a1bcb50 .github/buildcheck/tests/test_sharedfiles.py 7fe208dd33184b4543328f97b339c95e23c408b120e6bfa6095de4182aeb9a19 .github/check-release-gate.py 1604ccdea83998814a3c1f98f746272222039867b0c7d692355e61495c1c648b .github/check-shared-files.py 7da53fdb9537ccf7d3f3740be31f295bacb914fe186304b9ed8a65fd3d3d413e .github/print-crash-logs.sh @@ -29,3 +31,8 @@ ab45f5c102b47dd16c325d4d9c283d158ba90c05f484eac45b2767885c4462f9 .github/signin ebfcc0adf59f5858bbe4dc077c906304a197f72a55256f0d5aac669bee5e871f .github/PULL_REQUEST_TEMPLATE.md d3ca255bda928a293883c7c9c1718dde3f7e69e230af54f9fa9c08c804d7dfd3 .github/sign-fatjars.sh 42f1842270af691bdfe561355bee4eb9ae326383f1852db19763abb888d6b90e llama/lombok.config +f26d05f25a154b84d8f281bb92857811749e2225cdbc167362de930c48b532d1 .github/workflows/publish.yml#startgate +14d4f71b0f60f64913413fbabbafd786672215f5baa5dc2f97c0877565b373a2 .github/workflows/publish.yml#shared-files +fcb6cd0e2a71205b918f545f2680d378c790b774b9852129c5f9c7f3f8eb9860 .github/workflows/publish.yml#verify-signing-key +09230b04c5f14bdda50a0c862152ff197a656cbc3fd0d110899c5f8150f8aa9a .github/workflows/publish.yml#check-snapshot +51e987d59efd9887b43e345edc0e0f3db21686879e3ca6ea9f0fdbc493d37121 .github/workflows/publish.yml#check-tag diff --git a/CHANGELOG.md b/CHANGELOG.md index 76f269501..08f3a08fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,10 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by library are now such shared scripts). The workflow lost ~760 lines to two composite actions (`restore-models`, `install-sccache-windows`) and `print-host-info.sh`; `validate-models.bat` is gone (Windows runs the bash script). +- **Workflow jobs kept identical across the repositories are checked too**: a + `.github/shared-files.sha256` entry `.github/workflows/publish.yml#` hashes one job (`startgate`, + `shared-files`, `verify-signing-key`, `check-snapshot`, `check-tag`, and where present + `verify-signing-key-gradle`, `github-snapshot`, `github-release`). - **Fewer copies of the same job in the workflow.** The four fat-jar smoke jobs are one `smoke-fatjar` matrix (its rows checked against the targets `natives.csv` derives; the per-jar artifacts are now `llama-fatjar-smoke-`), the macOS and Windows Java test jobs call one reusable workflow diff --git a/CLAUDE.md b/CLAUDE.md index ce8a65854..662393700 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -104,8 +104,13 @@ name). Introducing it found `vmlens` gating nothing in all four repos; it now ga **Shared files (all four repos).** Files kept byte-identical in java-llama.cpp, BitcoinAddressFinder, srcmorph and streambuffer are listed with their SHA-256 in each repo's **`.github/shared-files.sha256`** -(`sha256sum` format, so `sha256sum -c` reads it too). The `shared-files` job — itself identical in all -four `publish.yml` files, and gating both publish jobs — **fails** when a listed file changed here +(`sha256sum` format). An entry can also name **one job of a workflow**, +`.github/workflows/publish.yml#startgate`: `startgate`, `shared-files`, `verify-signing-key`, +`check-snapshot` and `check-tag` are identical in all four `publish.yml` files (plus +`verify-signing-key-gradle`, `github-snapshot` and `github-release` in the three Maven-only ones), and +such an entry hashes just that job -- so a job kept identical by convention is now checked like a +file, without moving it into a reusable workflow (no check name or `needs:` changes). The +`shared-files` job, gating both publish jobs, **fails** when a listed file changed here alone, and **warns** when another repo's default branch lists the same file with a different hash (a sync is one change per repo and lands in four steps, so a warning, not a failure). **The manifest is the reference for what must stay equal**: changing a shared file means changing every copy, then From d40c7e969ba420222a9933b406cd9402ad81fda8 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 06:37:21 +0000 Subject: [PATCH 07/22] CI: warn where a Maven dependency or plugin differs from a sibling repository check-versions.py (shared build-check library, run in the shared-files job) compares every groupId:artifactId the POMs use -- plugins, dependencies, annotation-processor paths and the Spotless formatter version, ${...} resolved -- with the default branches of the three sibling repositories and warns per difference. Dependabot bumps each repository on its own; this is where the drift now shows instead of in a hand-kept table. Warnings only: a bump lands in four pull requests. The repositories' own net.ladenthin artifacts are left out. All 33 coordinates the four repositories share agree today. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/buildcheck/__init__.py | 2 +- .github/buildcheck/tests/test_versions.py | 120 +++++++++++++++++++ .github/buildcheck/versions.py | 139 ++++++++++++++++++++++ .github/check-versions.py | 16 +++ .github/shared-files.sha256 | 7 +- .github/workflows/publish.yml | 2 + CHANGELOG.md | 3 + CLAUDE.md | 15 ++- 8 files changed, 300 insertions(+), 4 deletions(-) create mode 100644 .github/buildcheck/tests/test_versions.py create mode 100644 .github/buildcheck/versions.py create mode 100755 .github/check-versions.py diff --git a/.github/buildcheck/__init__.py b/.github/buildcheck/__init__.py index e49ad24b7..cd57e0d9f 100644 --- a/.github/buildcheck/__init__.py +++ b/.github/buildcheck/__init__.py @@ -3,7 +3,7 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 """Build checks as a library, so buildcheck/tests can test them. Standard library only. -Two kinds of module live here. workflow.py, releasegate.py and sharedfiles.py (with their tests +Two kinds of module live here. workflow.py, releasegate.py, sharedfiles.py and versions.py (with their tests and the check-*.py entry points next to this package) are kept BYTE-IDENTICAL in java-llama.cpp, BitcoinAddressFinder, srcmorph and streambuffer: each repository lists them in .github/shared-files.sha256, which its `shared-files` job checks (see sharedfiles.py). Every other diff --git a/.github/buildcheck/tests/test_versions.py b/.github/buildcheck/tests/test_versions.py new file mode 100644 index 000000000..b2f03ca0a --- /dev/null +++ b/.github/buildcheck/tests/test_versions.py @@ -0,0 +1,120 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +import io +import os +import unittest +from unittest import mock + +from buildcheck import versions +from buildcheck.tests.helpers import REPO + + +def pom(body, properties=""): + return (f'{properties}' + f'{body}') + + +PARENT = pom("", "6.1.3${junit.version}2.99.0") +CHILD = pom(""" + org.junit.jupiterjunit-jupiter + ${junit.version} + anested${nested} + aunresolved${nowhere} + amanaged + net.ladenthinllama5.2.0 + +maven-jar-plugin3.5.0 + com.google.errorprone + error_prone_core2.50.0 + + com.diffplug.spotlessspotless-maven-plugin3.10.3 + ${palantir} + """) + + +class CoordinatesTest(unittest.TestCase): + + def test_reads_plugins_dependencies_and_processor_paths_with_properties_resolved(self): + self.assertEqual(versions.coordinates([PARENT, CHILD]), { + "org.junit.jupiter:junit-jupiter": {"6.1.3"}, + "a:nested": {"6.1.3"}, + "org.apache.maven.plugins:maven-jar-plugin": {"3.5.0"}, + "com.google.errorprone:error_prone_core": {"2.50.0"}, + "com.diffplug.spotless:spotless-maven-plugin": {"3.10.3"}, + "com.palantir.javaformat:palantir-java-format": {"2.99.0"}, + }) + + def test_a_repository_using_two_versions_of_one_coordinate_keeps_both(self): + other = pom('org.junit.jupiter' + 'junit-jupiter6.0.0') + self.assertEqual(versions.coordinates([PARENT, CHILD, other])["org.junit.jupiter:junit-jupiter"], + {"6.0.0", "6.1.3"}) + + +class CompareTest(unittest.TestCase): + + def test_only_coordinates_both_use_and_only_differences(self): + own = {"j:u": {"1"}, "s:p": {"2"}, "only:here": {"3"}} + others = {"r1": {"j:u": {"1"}, "s:p": {"9"}}, "r2": {"only:there": {"4"}}} + self.assertEqual(versions.compare(own, others), + ["s:p is 2 here and 9 in r1: bump it in every repository"]) + + +class MainTest(unittest.TestCase): + + def run_main(self, siblings): + out, err = io.StringIO(), io.StringIO() + + def fetcher(repo, path): + if repo not in siblings: + raise OSError("unreachable") + return siblings[repo] + with mock.patch.object(versions, "current_repo", return_value="streambuffer"), \ + mock.patch.object(versions, "read_local", return_value=[PARENT, CHILD]): + code = versions.main(["x"], "/nowhere", fetcher, out, err) + return code, out.getvalue(), err.getvalue() + + def test_differences_are_warnings_and_never_fail(self): + bumped = pom('org.junit.jupiter' + 'junit-jupiter6.1.4') + code, out, err = self.run_main({"srcmorph": bumped, "BitcoinAddressFinder": CHILD.replace( + "${junit.version}", "6.1.3").replace("${nested}", "6.1.3")}) + self.assertEqual(code, 0) + self.assertIn("::warning::org.junit.jupiter:junit-jupiter is 6.1.3 here and 6.1.4 in srcmorph", err) + self.assertIn("could not read java-llama.cpp's pom.xml", err) + self.assertNotIn("BitcoinAddressFinder:", err) + self.assertIn("1 version differences", out) + + def test_a_missing_pom_skips_that_file_not_the_repository(self): + out, err = io.StringIO(), io.StringIO() + bumped = pom('org.junit.jupiter' + 'junit-jupiter6.1.4') + + def fetcher(repo, path): + if repo == "srcmorph" and path == "pom.xml": + return bumped + raise OSError("404") + with mock.patch.object(versions, "current_repo", return_value="streambuffer"), \ + mock.patch.object(versions, "read_local", return_value=[PARENT, CHILD]): + self.assertEqual(versions.main(["x"], "/nowhere", fetcher, out, err), 0) + self.assertIn("could not read srcmorph's srcmorph/pom.xml", err.getvalue()) + self.assertIn("6.1.4 in srcmorph", err.getvalue()) + + def test_arguments_are_refused(self): + self.assertEqual(versions.main(["x", "--offline"], REPO, err=io.StringIO()), 2) + + +class RepositoryTest(unittest.TestCase): + + def test_this_repository_s_poms_are_listed_and_readable(self): + repo = versions.current_repo(REPO) + if repo not in versions.POMS: + self.skipTest(f"checked out as {repo}, not under its repository name") + found = versions.coordinates(versions.read_local(REPO, repo)) + self.assertTrue(found, "no versioned coordinate found -- the POM list in versions.POMS is stale") + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/buildcheck/versions.py b/.github/buildcheck/versions.py new file mode 100644 index 000000000..2935a7614 --- /dev/null +++ b/.github/buildcheck/versions.py @@ -0,0 +1,139 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""The Maven dependencies and plugins java-llama.cpp, BitcoinAddressFinder, srcmorph and streambuffer +have in common, compared across the repositories: a warning per coordinate whose version differs. + +The four repositories pin the same build tooling (JUnit, Error Prone, NullAway, Spotless, SpotBugs, +JaCoCo, the Maven plugins, ...), and Dependabot bumps each repository on its own, so they drift one +pull request at a time. This check makes the drift visible in every repository's `shared-files` +job. It compares what the POMs actually USE -- every `groupId:artifactId` of a plugin, dependency or +annotation-processor path with its version, plus the formatter version Spotless is configured with, +`${property}` references resolved -- not property +names, which differ between repositories and include values that must differ (the Java release). + +Warnings, never failures: a bump lands in four pull requests, and the first one merged would +otherwise turn the other three repositories red. The repository's own artifacts +(`net.ladenthin:*`) are left out: a sibling depends on the last RELEASE of java-llama.cpp while +java-llama.cpp itself is at the next snapshot. +""" + +import re +import sys +import urllib.request +import xml.etree.ElementTree as ET + +from .sharedfiles import OWNER, REPOS, current_repo + +NS = "{http://maven.apache.org/POM/4.0.0}" +POMS = { + "java-llama.cpp": ("pom.xml", "llama/pom.xml", "llama-langchain4j/pom.xml", "llama-kotlin/pom.xml", + "llama-platform/pom.xml", "llama-atmosphere-agent/pom.xml"), + "BitcoinAddressFinder": ("pom.xml",), + "srcmorph": ("pom.xml", "srcmorph/pom.xml", "srcmorph-cli/pom.xml", "srcmorph-maven-plugin/pom.xml"), + "streambuffer": ("pom.xml",), +} +OWN_GROUP = "net.ladenthin" +DEFAULT_PLUGIN_GROUP = "org.apache.maven.plugins" +VERSIONED = ("plugin", "dependency", "path", "annotationProcessorPath") +# Versions a plugin's names without coordinates: Spotless takes its formatter as +# . The versions plugin cannot see those, which is how palantir fell +# behind unnoticed more than once (workspace crossrepostatus, "Tool versions"). +CONFIGURED = {"palantirJavaFormat": "com.palantir.javaformat:palantir-java-format"} +RAW_URL = "https://raw.githubusercontent.com/{owner}/{repo}/HEAD/{path}" +PROPERTY = re.compile(r"\$\{([^}]+)\}") + + +def coordinates(pom_texts): + """groupId:artifactId -> set of versions used across the POMs of one repository. Properties + are looked up across all of them (a child module uses its parent's), up to a few levels deep; + a version that does not resolve to a literal is left out.""" + roots = [ET.fromstring(text) for text in pom_texts] + props = {} + for root in roots: + for element in root.findall(NS + "properties/*"): + props[element.tag[len(NS):]] = (element.text or "").strip() + + def resolve(value): + for _ in range(5): + match = PROPERTY.fullmatch(value) + if not match or match.group(1) not in props: + break + value = props[match.group(1)] + return None if "${" in value else value + + found = {} + for root in roots: + for element in root.iter(): + tag = element.tag[len(NS):] + if tag in CONFIGURED: + version = resolve((element.findtext(NS + "version") or "").strip()) + if version: + found.setdefault(CONFIGURED[tag], set()).add(version) + continue + if tag not in VERSIONED: + continue + artifact, version = element.findtext(NS + "artifactId"), element.findtext(NS + "version") + if not artifact or not version: + continue + group = (element.findtext(NS + "groupId") or DEFAULT_PLUGIN_GROUP).strip() + version = resolve(version.strip()) + if version and group != OWN_GROUP: + found.setdefault(f"{group}:{artifact.strip()}", set()).add(version) + return found + + +def compare(own, others): + """Warnings for every coordinate another repository uses in a different version.""" + warnings = [] + for repo, theirs in sorted(others.items()): + for coordinate in sorted(set(own) & set(theirs)): + if own[coordinate] != theirs[coordinate]: + warnings.append(f"{coordinate} is {', '.join(sorted(own[coordinate]))} here and " + f"{', '.join(sorted(theirs[coordinate]))} in {repo}: bump it in every repository") + return warnings + + +def fetch(repo, path): + with urllib.request.urlopen(RAW_URL.format(owner=OWNER, repo=repo, path=path), timeout=20) as response: + return response.read().decode("utf-8") + + +def read_local(root, repo): + texts = [] + for path in POMS[repo]: + with open(f"{root}/{path}", encoding="utf-8") as f: + texts.append(f.read()) + return texts + + +def main(argv, root, fetcher=fetch, out=sys.stdout, err=sys.stderr): + """check-versions.py -- compare with the siblings' default branches (warnings only)""" + if argv[1:]: + print(main.__doc__, file=err) + return 2 + repo = current_repo(root) + if repo not in POMS: + print(f"::error::{repo} is not one of {', '.join(REPOS)}", file=err) + return 2 + own = coordinates(read_local(root, repo)) + others = {} + for sibling in REPOS: + if sibling == repo: + continue + texts = [] + for path in POMS[sibling]: + try: + texts.append(fetcher(sibling, path)) + except Exception as e: # noqa: BLE001 -- a sibling that cannot be read is a warning, never a red + print(f"::warning::could not read {sibling}'s {path}: {e}", file=err) + if texts: + others[sibling] = coordinates(texts) + warnings = compare(own, others) + for warning in warnings: + print(f"::warning::{warning}", file=err) + shared = set(own).intersection(*others.values()) if others else set() + print(f"{len(own)} versioned coordinates here, {len(shared)} used by every repository read, " + f"{len(warnings)} version differences (compared with {', '.join(sorted(others)) or 'no other repository'})", + file=out) + return 0 diff --git a/.github/check-versions.py b/.github/check-versions.py new file mode 100755 index 000000000..69745035a --- /dev/null +++ b/.github/check-versions.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Warn about Maven dependencies and plugins whose version differs between this repository and its +siblings. See buildcheck/versions.py. Usage: check-versions.py +""" + +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from buildcheck import versions # noqa: E402 + +if __name__ == "__main__": + sys.exit(versions.main(sys.argv, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))) diff --git a/.github/shared-files.sha256 b/.github/shared-files.sha256 index 7193eedd5..2b09695fa 100644 --- a/.github/shared-files.sha256 +++ b/.github/shared-files.sha256 @@ -10,7 +10,7 @@ # `python3 .github/check-shared-files.py --write` in each. An entry `#` stands for one # job of a workflow (kept identical inside otherwise different workflows); `sha256sum -c` reads the # file entries only. -82171de97c6621b4c1f3e9cd66afd49990f6d31331857423b37f2b9a2f35fbc0 .github/buildcheck/__init__.py +e43a52dd303a363512b43b13b2b941cba16ac4f039617c6e53810dd17eb4a4ca .github/buildcheck/__init__.py 3d1a9f2e93709941261a9655bc19bf50d5becefd8f6a6c2f5564096a757996bb .github/buildcheck/workflow.py 5a70eee1bb8af180edfd2a4bae61be61fac15b409f5df6a8bb9869425d7fc381 .github/buildcheck/releasegate.py 95285b6e9fdd30a71e97aff236e4a23d42101bc182955cae7ca1e4993939321a .github/buildcheck/sharedfiles.py @@ -21,6 +21,9 @@ cbe504ad081ba31ec16e11c11f879c64179ca7741bf0b1d1318a7fc969ed67dd .github/buildc f040b63c801e1d8b1b27b0fe8ae733d6fb5cf1b166e79470a6c3e3282a1bcb50 .github/buildcheck/tests/test_sharedfiles.py 7fe208dd33184b4543328f97b339c95e23c408b120e6bfa6095de4182aeb9a19 .github/check-release-gate.py 1604ccdea83998814a3c1f98f746272222039867b0c7d692355e61495c1c648b .github/check-shared-files.py +e5f1679dc7151b2676b2f587860bbaa17e1b89b44a5a4e9703eb3499248c4a06 .github/buildcheck/versions.py +d454d33b61089b5f7bd6d439e3e5b58e01e7b01eeba6b4c2f7ed3360bbb158f0 .github/buildcheck/tests/test_versions.py +0c78f43dd181f079d35edde7cbf25bb22eac92c907845d8dc8eec78e812aa097 .github/check-versions.py 7da53fdb9537ccf7d3f3740be31f295bacb914fe186304b9ed8a65fd3d3d413e .github/print-crash-logs.sh c8c02dc39aeacc154fad74071443c6513666aa94ca9f9c5fab49c5f39dea7ee6 .github/verify-signing-key.sh e0b2a4f331eae326b61c71f78d6272d06d99140a85dedb7710f93402de862466 .github/verify-bytecode-version.sh @@ -32,7 +35,7 @@ ebfcc0adf59f5858bbe4dc077c906304a197f72a55256f0d5aac669bee5e871f .github/PULL_R d3ca255bda928a293883c7c9c1718dde3f7e69e230af54f9fa9c08c804d7dfd3 .github/sign-fatjars.sh 42f1842270af691bdfe561355bee4eb9ae326383f1852db19763abb888d6b90e llama/lombok.config f26d05f25a154b84d8f281bb92857811749e2225cdbc167362de930c48b532d1 .github/workflows/publish.yml#startgate -14d4f71b0f60f64913413fbabbafd786672215f5baa5dc2f97c0877565b373a2 .github/workflows/publish.yml#shared-files +068c3b1d9db88437927e5005eead1e1952fc611450dca705a213fd946e4547b4 .github/workflows/publish.yml#shared-files fcb6cd0e2a71205b918f545f2680d378c790b774b9852129c5f9c7f3f8eb9860 .github/workflows/publish.yml#verify-signing-key 09230b04c5f14bdda50a0c862152ff197a656cbc3fd0d110899c5f8150f8aa9a .github/workflows/publish.yml#check-snapshot 51e987d59efd9887b43e345edc0e0f3db21686879e3ca6ea9f0fdbc493d37121 .github/workflows/publish.yml#check-tag diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index f3fae613b..1930c724c 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -109,6 +109,8 @@ jobs: run: python3 -m unittest discover -s .github/buildcheck/tests -t .github - name: Release gate (every job gates both publish jobs, or release-gate-exemptions.txt says why) run: python3 .github/check-release-gate.py + - name: Maven versions (warns where a dependency or plugin differs from a sibling repository) + run: python3 .github/check-versions.py # --------------------------------------------------------------------------- # GPG signing-key preflight (standalone, no `needs:` — runs in parallel at the diff --git a/CHANGELOG.md b/CHANGELOG.md index 08f3a08fa..8e7eb4a07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,9 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by `.github/shared-files.sha256` entry `.github/workflows/publish.yml#` hashes one job (`startgate`, `shared-files`, `verify-signing-key`, `check-snapshot`, `check-tag`, and where present `verify-signing-key-gradle`, `github-snapshot`, `github-release`). +- **Maven versions are compared with the sibling repositories**: `check-versions.py` (in the + `shared-files` job) warns where a dependency or plugin -- incl. annotation-processor paths and the + Spotless formatter version -- is used in another version than in a sibling's default branch. - **Fewer copies of the same job in the workflow.** The four fat-jar smoke jobs are one `smoke-fatjar` matrix (its rows checked against the targets `natives.csv` derives; the per-jar artifacts are now `llama-fatjar-smoke-`), the macOS and Windows Java test jobs call one reusable workflow diff --git a/CLAUDE.md b/CLAUDE.md index 662393700..5819afdcf 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -94,6 +94,7 @@ shapes this repo writes and rejects anything else loudly. | `workflow.py` | — | the job graph of a workflow (`needs`, uploads, transitive closure) | | `releasegate.py` | `check-release-gate.py` (`shared-files`) | every job gates **both** publish jobs, unless `.github/release-gate-exemptions.txt` names it with a reason | | `sharedfiles.py` | `check-shared-files.py` (`shared-files`) | the files kept byte-identical across the four sibling repos | +| `versions.py` | `check-versions.py` (`shared-files`) | **warns** where a Maven dependency or plugin is used in another version than in a sibling repo | **The release gate.** A job nothing waits for can go red while a release still ships — the natives builds `package` once forgot to wait for, and the aarch64 fat jars that were signed and attached @@ -119,7 +120,19 @@ job says so, and the history shows it — but nobody can edit a shared script wi is shared. Duplicating the library across repos is therefore deliberate. A separate actions/library repo was considered and rejected: every consumer would need a pinned SHA per use (Scorecard's pinned-dependencies rule), a release process of its own, and a cross-repo checkout; copies with a -checksum keep each repo self-contained. Shared today: the `buildcheck` modules above marked shared in +checksum keep each repo self-contained. + +**Maven versions across the four repos (`check-versions.py`).** They pin the same build tooling and +Dependabot bumps each repo on its own, so the versions drift one pull request at a time; the +`shared-files` job therefore compares every `groupId:artifactId` the POMs use (plugins, dependencies, +annotation-processor paths, and the formatter version Spotless is configured with -- `${...}` +resolved) with the siblings' default branches and **warns** per difference. Warnings only, because a +bump lands in four pull requests and the first one merged must not turn the other three red; the +repos' own artifacts (`net.ladenthin:*`) are left out. It compares what is used, not property names: +those differ between repos and include values that must differ (`maven.compiler.release`). The POM +list per repo is `versions.POMS` -- a new module's POM goes there, in all four copies. + +Shared today: the `buildcheck` modules above marked shared in `__init__.py` (+ their tests and CLIs), `print-crash-logs.sh`, `verify-signing-key.sh`, `verify-bytecode-version.sh`, `sign-fatjars.sh` (jllama + srcmorph), `smoke-fatjar-cli.sh` (BAF + srcmorph), the signing self-test, `lombok.config` and the issue/PR templates. From f6f5c75a7df77527818ca788f6d46fa6bd5bce42 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 1 Oct 2026 07:08:45 +0000 Subject: [PATCH 08/22] CI: bash -n over every bash run script of the workflows and actions The shared-files job now parses every run: script of .github/workflows and the composite actions that runs in bash (shell decided as the runner does: step shell, job and workflow defaults, else PowerShell on a Windows runner and bash elsewhere). A broken script -- such as a lost line continuation that leaves a line starting with || -- fails within minutes instead of in the job that runs it. New shared buildcheck module runscripts.py with tests and the check-run-scripts.py CLI, listed in the shared-files manifest of all four repositories. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2 --- .github/buildcheck/__init__.py | 10 +- .github/buildcheck/runscripts.py | 273 ++++++++++++++++++++ .github/buildcheck/tests/test_runscripts.py | 127 +++++++++ .github/check-run-scripts.py | 16 ++ .github/shared-files.sha256 | 7 +- .github/workflows/publish.yml | 2 + CHANGELOG.md | 3 + CLAUDE.md | 13 + 8 files changed, 444 insertions(+), 7 deletions(-) create mode 100644 .github/buildcheck/runscripts.py create mode 100644 .github/buildcheck/tests/test_runscripts.py create mode 100755 .github/check-run-scripts.py diff --git a/.github/buildcheck/__init__.py b/.github/buildcheck/__init__.py index cd57e0d9f..add4f0f54 100644 --- a/.github/buildcheck/__init__.py +++ b/.github/buildcheck/__init__.py @@ -3,11 +3,11 @@ # SPDX-License-Identifier: MIT OR Apache-2.0 """Build checks as a library, so buildcheck/tests can test them. Standard library only. -Two kinds of module live here. workflow.py, releasegate.py, sharedfiles.py and versions.py (with their tests -and the check-*.py entry points next to this package) are kept BYTE-IDENTICAL in java-llama.cpp, -BitcoinAddressFinder, srcmorph and streambuffer: each repository lists them in -.github/shared-files.sha256, which its `shared-files` job checks (see sharedfiles.py). Every other -module is the repository's own. +Two kinds of module live here. workflow.py, releasegate.py, sharedfiles.py, versions.py and +runscripts.py (with their tests and the check-*.py entry points next to this package) are kept +BYTE-IDENTICAL in java-llama.cpp, BitcoinAddressFinder, srcmorph and streambuffer: each repository +lists them in .github/shared-files.sha256, which its `shared-files` job checks (see sharedfiles.py). +Every other module is the repository's own. Run the tests from the repository root: python3 -m unittest discover -s .github/buildcheck/tests -t .github diff --git a/.github/buildcheck/runscripts.py b/.github/buildcheck/runscripts.py new file mode 100644 index 000000000..ea6850588 --- /dev/null +++ b/.github/buildcheck/runscripts.py @@ -0,0 +1,273 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 +"""Every `run:` script of the workflows and composite actions that runs in bash, parsed with +`bash -n` -- so a broken script fails the `shared-files` job in the first minutes of a run instead +of the job that executes it, possibly hours later or only on a release path. + +The case this exists for: a cleanup lost the trailing backslash of six continued lines, leaving a +line that starts with `||` or `-e` -- in a step only a publish run or a native build reaches. Neither +actionlint (it parses scripts only when shellcheck is installed) nor a review saw it; `bash -n` sees +every one of them. + +Which scripts are bash is decided the way the runner decides it: the step's `shell:`, else the job's +and then the workflow's `defaults.run.shell`, else the runner's default -- PowerShell on a Windows +runner, bash everywhere else. A runner chosen by an expression (a matrix, a workflow input) counts +as Windows only when the step's `if:` says so; a step without `shell:` on such a job runs on every +OS of the matrix and has to be valid bash anyway. Scripts in `sh`, `bash -el {0}` or a bash given by +path are bash; pwsh, powershell, cmd and python are skipped. + +Like workflow.py this reads the two-space layout the workflows are written in, not general YAML. +The run value is taken as YAML defines it: a literal block (`|`) line for line, a folded block +(`>`) and a plain scalar folded into one line, a single-quoted scalar unquoted. A double-quoted +scalar is skipped (its escapes are not worth a YAML parser). Every `${{ ... }}` expression becomes +a word, which is what the runner substitutes before bash sees the script. +""" + +import glob +import os +import re +import shutil +import subprocess +import sys + +RUN = re.compile(r"^(?P\s*(?:- )?)run:(?:\s+(?P.*?))?\s*$") +BLOCK_HEADER = re.compile(r"^(?P