Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ the first two items break every existing consumer at upgrade.
fails the stream through `onError`.

### Added
- **llama.cpp b11881** (from b11639, in eleven steps: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815, b11820, b11827, b11881). What the
- **llama.cpp b11883** (from b11639, in twelve steps: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815, b11820, b11827, b11881, b11883). What the
range brings to a consumer: `POST /tts` (llama.cpp b11781, #26603, speech from a TTS-capable model through the
embedded server) and the realtime transcription WebSocket `/v1/realtime` (b11780, #30636), both served by
`NativeServer` -- classic and attach mode -- and not by the Java transport or `LlamaModel` yet; a
Expand Down Expand Up @@ -167,6 +167,9 @@ the first two items break every existing consumer at upgrade.
mat-vec kernels on the Intel XMX engines for short batches (#29864, `GGML_SYCL_ENABLE_XMX_ESIMD`), an
OpenCL depthwise `CONV_2D_DW` (#30900), the MiniCPM-OCR 1.0 projector (#30979), a DFlash drafter
fix for qwen4exp (#30927) and fewer graph inputs for glm5-next (#30932) -- nothing here changed.
The b11883 step (2026-10-25, 2 commits, tagged): a grammar nested deeper than upstream's new limit is
refused instead of overflowing the stack while it is parsed (#29755, for #29495), and quantizing 4D
tensors counts their rows with `ggml_nrows()` and finds MoE experts by name (#30820). No patch changed.
The b11881 step (2026-10-25, 54 commits; upstream had not tagged the number yet, so the pin names the
commit `fa3645feb`): llama.cpp's server entry point is one function per role now (#31113), which
`NativeServer` does not show -- `0006` and `0007` were rewritten onto it, and attach mode registers
Expand Down
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.

Current llama.cpp pinned version: **b11881**
Current llama.cpp pinned version: **b11883**

## Natives jars: library jars and GPU module jars (`.github/natives.csv`)

Expand Down Expand Up @@ -214,7 +214,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1
a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static
platforms (macOS, s390x) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own
CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT`
and `GGML_CUDA_GRAPHS_DEFAULT` at b11881 (unchanged since b11639) -- because adding ggml first skips that block; re-check with
and `GGML_CUDA_GRAPHS_DEFAULT` at b11883 (unchanged since b11639) -- because adding ggml first skips that block; re-check with
`grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once:
ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of
its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which
Expand Down Expand Up @@ -1213,7 +1213,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
ships no UI):
```bash
# needs node/npm + network for the asset build; the embed step is plain cmake -P
git clone --depth 1 --branch b11881 https://github.com/ggml-org/llama.cpp /tmp/lc
git clone --depth 1 --branch b11883 https://github.com/ggml-org/llama.cpp /tmp/lc
( cd /tmp/lc/tools/ui && npm ci && npm run build )
mkdir -p webui-generated /tmp/ui-gen
cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \
Expand Down Expand Up @@ -1253,7 +1253,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
- `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
as the repo secret **`DEPOT_TOKEN`**.

Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11881`), the
Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11883`), the
~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
*changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
Expand Down Expand Up @@ -2732,7 +2732,7 @@ the parameterized `CApiVocabOnly` case expands to 22 instances).

#### Upstream source location (in CMake build tree)

llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11881`.
llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11883`.

**GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely
by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
**Build:**
![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)
![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)
[![llama.cpp b11881](https://img.shields.io/badge/llama.cpp-%23b11881-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11881)
[![llama.cpp b11883](https://img.shields.io/badge/llama.cpp-%23b11883-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11883)
[![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)
![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)
[![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev)
Expand Down
1 change: 1 addition & 0 deletions docs/history/llama-cpp-breaking-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -917,3 +917,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
| b11820–b11827 | patches | **All 12 apply unchanged** (the fail-loud applier on a fresh build directory at `970b51ce7`; `jllama-build-info` lists 0001, 0002, 0003, 0006, 0007, 0008, 0014, 0015, 0017, 0018, 0020, 0021; `check-patches.py`: 12 patches, 88 hunks, 0 problems). No patch-target file is in the range (the intersection of the 48 paths the patches touch with `git diff --name-only b11820..b11827`, 23 files, is empty), so the drop-check answers of the b11820 row stand. Across b11815..b11827 the root `CMakeLists.txt`, `.github/`, `vendor/`, `ggml/CMakeLists.txt`, `ggml/src/CMakeLists.txt`, `tools/server/CMakeLists.txt` and `tools/mtmd/CMakeLists.txt` are unchanged, so every `KEEP IN SYNC WITH UPSTREAM` pin of `publish.yml` stays (ROCm 10.0.0, Vulkan 1.4.357.0, DLE 2026.1.4 + oneDNN 2026.0.2 + Level Zero 1.33.1, CCCL v3.4.3, OpenVINO 2026.4.1, BoringSSL 0.20261020.0). |
| b11827–b11881 | `tools/server/server.cpp`, `tools/server/server-common.{cpp,h}`, `tools/server/server-models.{cpp,h}`, `tools/server/server-context.{cpp,h}`, `tools/server/server-decision.cpp`, `tools/server/server-task.h`, `common/{arg.cpp,chat.{cpp,h},download.{cpp,h},file-lock.{cpp,h},hf-cache.{cpp,h},json-schema.cpp,jinja/value.cpp,speculative.{cpp,h},common.cpp,parsers/deepseek.cpp}`, `src/` (new `models/deepseek41.cpp`, `llama-kv-cache-dsv4.*`, `llama-vocab.*`, `models/dflash.cpp`), `tools/mtmd/` (new `models/soprano.cpp`, `mtmd-helper-gen.cpp`, `mtmd.h`), `ggml/include/ggml.h`, `ggml/src/ggml.c`, `ggml/src/ggml-rpc/transport.cpp`, the GPU backends, `vendor/cpp-httplib/CMakeLists.txt` (LibreSSL only), `tools/ui/` (54 commits, `fa3645feb` = b11881, 2026-10-22 .. 2026-10-25; **not tagged upstream at the time of the bump**, pinned through `JLLAMA_GIT_COMMIT`) | **No project source change; two patches rewritten (next row).** #31113 ("server: refactor server entrypoint") split `llama_server()` into `server_run()` and one function per role (`server_run_router`, `server_run_instance`, `server_run_download`), added `server_run_opts` (`install_signal_handlers`, `resolve_model`; llama-cli's overload clears both), factored the route table out as `server_register_routes()` (with the GCP shim) and `server_register_stream_routes()`, and moved `server_child` from `server-models.{cpp,h}` to `server_role` in `server-common.{cpp,h}` (same environment variables, same stdin/stdout protocol); the `server_state` enum moved from `server-context.h` to `server-common.h`. Nothing in `src/main/cpp` names any of them. #30242: JSON nested more than 128 levels is refused before it is walked -- `common_chat_templates_apply`'s inputs (tools, tool-call arguments, `chat_template_kwargs`, `json_schema`), the `json_schema` of `/completion` and the `/v1/systemone` body -- with `std::invalid_argument`, which every entry point here already maps to `InvalidRequestException`. #29312: `-hf` downloads take a lock shared with the hf CLI (`common/file-lock.cpp`, compiled into llama-common by upstream's own list), `common_download_file_single` and `hf_cache::finalize_file` gained defaulted parameters; not called here. Additive only in the reviewed headers: `common_chat_msg::extra_json` (DeepSeek V4.1 templates), `mtmd_audio_stream_get_n_rows[_flushed]`, `MTMD_GEN_AUDIO_TYPE_SOPRANO` (`tts_engine.cpp` accepts every type but `NONE`), `ggml_prec_get_acc/src`, `server_task_result_cmpl_partial::audio_commit_drained`. `include/llama.h`, `common/common.h`, `tools/mtmd/mtmd-helper.h`, `server-schema.cpp` and `server-task.cpp` are not in the range, so the server request-field set, its bounds and the response keys are identical. No option of `common/arg.cpp` changed (the 50 lines route the HF snapshot finalisation through the cancellable lock). The root `CMakeLists.txt` is unchanged (`GGML_LLAMAFILE_DEFAULT` / `GGML_CUDA_GRAPHS_DEFAULT` the same), and so is upstream's `release.yml`, so every `KEEP IN SYNC WITH UPSTREAM` pin of `publish.yml` stays; `vendor/cpp-httplib` moved LibreSSL to 4.4.0, BoringSSL (what this project builds) is unchanged. **Verified at b11881**: `jllama` and `jllama_test` built from the pin (configure: `llama.cpp b11881 is fetched by commit fa3645feb…`, `pin check: fetched commit count 11881 = b11881`, no compiler warning), `ctest` **692/692**, `NativeLibraryLoadSmokeTest`, `LlamaLoggerTest`, `RpcServerTest` and `BackendLoadTest` 32/32 against the b11881 library. |
| b11827–b11881 | patches | **`0006` and `0007` rewritten, the other ten apply unchanged** (the fail-loud applier on a fresh tree at `fa3645feb`; `check-patches.py`: 12 patches, 94 hunks, 0 problems). Six patch-target files are in the range: `common/arg.cpp` (`0001`, `0015`; the HF-lock change is away from both), `ggml/src/ggml-rpc/transport.cpp` (`0015`; #30399 adds an RDMA bounds check, and RDMA is off here), `tools/server/server-context.{cpp,h}` (`0002`, `0003`, `0021`; #30864 and #30994 touch the rerank template and the realtime route, the `server_state` enum moved out of the header), `tools/server/server-models.cpp` (`0008`; `server_child` moved out, `update_args` unchanged) and `tools/server/server.cpp` (`0001`, `0006`, `0007`), where #31113 broke `0006` and `0007`. **`0006`:** the embedded mode clears upstream's new `server_run_opts::install_signal_handlers` instead of guarding the `sigaction` block, and the `llama_server_shutdown_handler_reset` moved into the two role functions that install a handler, declared after the last local the handler captures -- `models_routes` in the router (declared after `ctx_http` there, so a reset next to `ctx_http` would have run after `models_routes` was destroyed) and `ctx_http` in the instance; the forwarded-argv parse, the mutex and `llama_server_request_shutdown()` are unchanged. **`0007`:** upstream now has the route table as a function of its own, so the patch's `llama_server_register_common_routes()` and its removal block are gone (4 hunks to 2) and `llama_server_attach()` registers `server_register_routes()` plus `server_register_stream_routes()` with the instance handlers, exactly what `server_run_instance()` registers; the verbatim-copy invariant of the b10519 row no longer exists. Guards: `NativeServerShutdown` (3, the real `llama_server()` in router mode), `CApiServer.TheEmbeddedServerStartsInRouterModeAndStops` and `CApi.TheAttachedServerServesTheModelAndLeavesItUsable` green. Drop-checks re-read at `fa3645feb`: `0001` -- `argv = utf8.ptrs.data();` still at `common/arg.cpp:1296`, no `common_params_parse_main`; `0002` -- `params_base.load_progress_callback = load_progress_callback;` unguarded at `server-context.cpp:1286`; `0003` -- no `slot_prompt_similarity` in `server-context.h`; `0008` -- no `LLAMA_SERVER_WORKER_CMD` upstream; `0014` -- `common/log.h` names only `common_log_default_callback`; `0015` -- `GGML_ABORT("Failed to connect to %s\n"` at `ggml-rpc.cpp:650`; `0017` -- the four `_mm_prefetch(&x[ib] + sizeof(...))` calls at `quants.c:846/847/864/865`; `0018` -- `free(buffer->context);` at `amx.cpp:46`; `0020` -- `ggml_gallocr_needs_realloc()` compares the leaf count only; `0021` -- the `else` branch at `server-context.cpp:1942` still takes `params_base.lora_adapters` without the clear. |
| b11881–b11883 | `src/llama-grammar.{cpp,h}`, `src/llama-quant.cpp` | **None needed.** Two commits: a recursion limit in the grammar parser (#29755) and the 4D-tensor/expert detection fix of the quantizer (#30820); no header the project includes changed, all twelve patches apply unchanged (`git apply` on `5c965fd8c`), the commit count is 11883. |
4 changes: 2 additions & 2 deletions llama/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -356,8 +356,8 @@ endif()
# hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable
# asserts that the fetched commit's count IS the number. Clear it once the tag exists;
# `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b<nnnn>` prints the hash.
set(JLLAMA_GIT_TAG b11881)
set(JLLAMA_GIT_COMMIT "fa3645feb5f47cbb30df5e6d031c22543d43d240") # the commit b11881 names (not tagged upstream yet); kept after the tag exists
set(JLLAMA_GIT_TAG b11883)
set(JLLAMA_GIT_COMMIT "5c965fd8ca0b8ecd24f4034c29f9c00c1c16aa43") # the commit the tag b11883 names; kept as the pin (a tag name can move)
if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$")
message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b<nnnn>, got '${JLLAMA_GIT_TAG}'")
endif()
Expand Down
4 changes: 2 additions & 2 deletions llama/legal/natives/NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ texts are not needed here.

| Component | Version | Compiled into | SPDX | Texts | Source |
|---|---|---|---|---|---|
| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11881 fa3645feb5f47cbb30df5e6d031c22543d43d240 | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE |
| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11883 5c965fd8ca0b8ecd24f4034c29f9c00c1c16aa43 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE |
| nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp |
| cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE |
| BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE |
Expand All @@ -47,7 +47,7 @@ texts are not needed here.
| WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE |
| WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md |
| WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE |
| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11881 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv |
| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11883 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv |
| WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static |
| CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE |
| CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,28 @@
* library was compiled against, exposed as a compile-time constant so callers can render a badge or
* emit a startup log line without loading the native library.
*
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11881"}) that mirrors the
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11883"}) that mirrors the
* {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is
* absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a
* lightweight version badge in Android or other UIs.</p>
*
* <p>For the <em>authoritative</em> value that is baked into the native binary — the build number
* plus the resolved upstream commit, e.g. {@code "b11881-<commit>"} — call
* plus the resolved upstream commit, e.g. {@code "b11883-<commit>"} — call
* {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own
* {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires
* the native library to be loaded).</p>
*/
public final class LlamaCppVersion {

/**
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b11881"}.
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b11883"}.
*
* <p>Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the
* "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the
* compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the
* value actually linked into the native binary.</p>
*/
public static final String LLAMA_CPP_VERSION = "b11881";
public static final String LLAMA_CPP_VERSION = "b11883";

// Constants holder — not instantiable.
private LlamaCppVersion() {}
Expand Down
Loading