From 442a4ad1d1ccef31dbc4a8a6cca46f01940313c5 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Sun, 25 Oct 2026 20:44:07 +0100 Subject: [PATCH] Upgrade llama.cpp from b11883 to b11887 Four upstream commits, tagged. #31035 caps the DRY sequence breakers: llama.h gains LLAMA_DRY_MAX_SEQ_BREAKERS (64), the request schema drops duplicate dry_sequence_breakers and refuses more than 64 distinct ones (each costs a scan of the vocabulary when the sampler is built, so a long list exhausted CPU and memory), the sampler stops at 64 with a warning. Two C++ tests pin the new contract (duplicates dropped in order; 64 distinct plus a duplicate accepted, one more refused), and withDrySequenceBreakers' Javadoc names it. #31028 merges Qwen4-exp tensors on conversion; #30977 and #31157 touch upstream CI and tests only. All twelve patches apply unchanged on 6050b467c (commit count 11887); CMakeLists.txt, common/, the license files, vendor/ and the WebUI lockfile are unchanged. Checked locally: check-patches (12 patches, 94 hunks, 0 problems), check-natives, spotless:check, clang-format 23.1.3 on test_server.cpp; the build and tests are CI's. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VszvDmRRPP1UcwozsUMY1L --- CHANGELOG.md | 7 +++++- CLAUDE.md | 14 +++++------ README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 1 + llama/CMakeLists.txt | 4 ++-- llama/legal/natives/NOTICE.md | 4 ++-- .../llama/parameters/InferenceParameters.java | 3 ++- .../llama/value/LlamaCppVersion.java | 8 +++---- llama/src/test/cpp/test_server.cpp | 24 +++++++++++++++++++ 9 files changed, 49 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ce6c2ed3..78dbefc45 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -136,7 +136,7 @@ the first two items break every existing consumer at upgrade. fails the stream through `onError`. ### Added -- **llama.cpp b11883** (from b11639, in twelve steps: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815, b11820, b11827, b11881, b11883). What the +- **llama.cpp b11887** (from b11639, in thirteen steps: b11760, b11761, b11799, b11800, b11803, b11809, b11814, b11815, b11820, b11827, b11881, b11883, b11887). What the range brings to a consumer: `POST /tts` (llama.cpp b11781, #26603, speech from a TTS-capable model through the embedded server) and the realtime transcription WebSocket `/v1/realtime` (b11780, #30636), both served by `NativeServer` -- classic and attach mode -- and not by the Java transport or `LlamaModel` yet; a @@ -167,6 +167,11 @@ the first two items break every existing consumer at upgrade. mat-vec kernels on the Intel XMX engines for short batches (#29864, `GGML_SYCL_ENABLE_XMX_ESIMD`), an OpenCL depthwise `CONV_2D_DW` (#30900), the MiniCPM-OCR 1.0 projector (#30979), a DFlash drafter fix for qwen4exp (#30927) and fewer graph inputs for glm5-next (#30932) -- nothing here changed. + The b11887 step (2026-10-25, 4 commits, tagged): a request's `dry_sequence_breakers` (`InferenceParameters + .withDrySequenceBreakers`) lose their duplicates and more than 64 distinct ones are refused as an invalid + request -- each costs a scan of the vocabulary when the sampler is built, so a long list exhausted CPU and + memory (#31035; the sampler itself stops at 64 for `--dry-sequence-breaker` with a warning); Qwen4-exp + models with merged tensors (#31028). No patch changed. The b11883 step (2026-10-25, 2 commits, tagged): a grammar nested deeper than upstream's new limit is refused instead of overflowing the stack while it is parsed (#29755, for #29495), and quantizing 4D tensors counts their rows with `ggml_nrows()` and finds MoE experts by name (#30820). No patch changed. diff --git a/CLAUDE.md b/CLAUDE.md index 166c806ee..079d770dd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11883** +Current llama.cpp pinned version: **b11887** ## Natives jars: library jars and GPU module jars (`.github/natives.csv`) @@ -214,7 +214,7 @@ measurements behind the Windows step are in `docs/handover/local-agent-report-b1 a `llama-common`/`cpp-httplib` that came out shared (one more file to ship). On the static platforms (macOS, s390x) all of it stays inside `libjllama`. **Every ggml default llama.cpp's own CMakeLists changes before its `add_subdirectory(ggml)` is repeated** -- `GGML_LLAMAFILE_DEFAULT` - and `GGML_CUDA_GRAPHS_DEFAULT` at b11883 (unchanged since b11639) -- because adding ggml first skips that block; re-check with + and `GGML_CUDA_GRAPHS_DEFAULT` at b11887 (unchanged since b11639) -- because adding ggml first skips that block; re-check with `grep -n _DEFAULT CMakeLists.txt` at the new tag on every bump. The second one was missing once: ggml's own default for CUDA graphs is OFF, and a CUDA module built without it lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070, prompt processing -20%; Vulkan, which @@ -1213,7 +1213,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11883 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11887 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -1253,7 +1253,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11883`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`JLLAMA_GIT_TAG b11887`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -2709,7 +2709,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" | File | Tests | Scope | |------|-------|-------| | `src/test/cpp/test_utils.cpp` | 168 | Upstream helpers: `server_tokens`, `server_grammar_trigger`, `gen_tool_call_id`, `json_value`, `json_get_nested_values`, UTF-8 helpers, `format_response_rerank`, `format_embeddings_response_oaicompat`, `oaicompat_completion_params_parse`, `oaicompat_chat_params_parse`, `are_lora_equal`, `strip_flag_from_argv`, `token_piece_value`, `json_is_array_and_contains_numbers`, `format_oai_sse`, `format_oai_resp_sse`, `format_anthropic_sse`, `parse_lora_request`, `common_chat_parse` over malformed UTF-8 (the `ContentOnlyParseUtf8` guard, which pins upstream #29161's one-U+FFFD-per-invalid-run contract — formerly the guard for the dropped `patches/0011`) | -| `src/test/cpp/test_server.cpp` | 206 | Upstream result types: `server_slot_stats` (the `timings` JSON payload; replaced `result_timings` in b10408), `task_params::to_json()` (incl. `dry_sequence_breakers`, `preserved_tokens`, `timings_per_token`), `completion_token_output`, `server_task_result_cmpl_partial` (non-oaicompat + `to_json_oaicompat` + logprobs + `to_json_oaicompat_chat` + `to_json_anthropic` + dispatcher), `server_task_result_cmpl_final` (non-oaicompat + `to_json_oaicompat` + `to_json_oaicompat_chat` + `to_json_oaicompat_chat_stream` + `to_json_anthropic` + `to_json_anthropic_stream` + tool_calls + dispatcher), `server_task_result_embd`, `server_task_result_rerank`, `server_task_result_metrics` (`to_metrics()` = the `/metrics` Prometheus exposition text; its `to_json()` has been unused since b10519 and returns `json{}` = JSON null), `server_task_result_slots` (`to_json()` = the `/slots` array, fed by the b10519 `SERVER_TASK_TYPE_SLOT_GET` task), `server_task_result_slot_save_load`, `server_task_result_slot_erase`, `server_task_result_apply_lora`, `server_task_result_get_lora`, `server_task_result_error`, `format_error_response`, `server_task::need_sampling()`, `server_task::n_tokens()`, `server_schema::eval_llama_cmpl_schema()` (parsing pipeline + grammar routing + error paths + per-request `dry_*` and `sse_ping_interval` field round-trips incl. hard-limit + server-default inheritance), `response_fields` projection | +| `src/test/cpp/test_server.cpp` | 208 | Upstream result types: `server_slot_stats` (the `timings` JSON payload; replaced `result_timings` in b10408), `task_params::to_json()` (incl. `dry_sequence_breakers`, `preserved_tokens`, `timings_per_token`), `completion_token_output`, `server_task_result_cmpl_partial` (non-oaicompat + `to_json_oaicompat` + logprobs + `to_json_oaicompat_chat` + `to_json_anthropic` + dispatcher), `server_task_result_cmpl_final` (non-oaicompat + `to_json_oaicompat` + `to_json_oaicompat_chat` + `to_json_oaicompat_chat_stream` + `to_json_anthropic` + `to_json_anthropic_stream` + tool_calls + dispatcher), `server_task_result_embd`, `server_task_result_rerank`, `server_task_result_metrics` (`to_metrics()` = the `/metrics` Prometheus exposition text; its `to_json()` has been unused since b10519 and returns `json{}` = JSON null), `server_task_result_slots` (`to_json()` = the `/slots` array, fed by the b10519 `SERVER_TASK_TYPE_SLOT_GET` task), `server_task_result_slot_save_load`, `server_task_result_slot_erase`, `server_task_result_apply_lora`, `server_task_result_get_lora`, `server_task_result_error`, `format_error_response`, `server_task::need_sampling()`, `server_task::n_tokens()`, `server_schema::eval_llama_cmpl_schema()` (parsing pipeline + grammar routing + error paths + per-request `dry_*` and `sse_ping_interval` field round-trips incl. hard-limit + server-default inheritance), `response_fields` projection | | `src/test/cpp/test_json_helpers.cpp` | 72 | All functions in `json_helpers.hpp`: `get_result_error_message`, `get_result_error_code` (the HTTP code of a failed task result, from which `jllama.cpp` picks `InvalidRequestException` or `LlamaException`), `results_to_json`, `rerank_results_to_json` (incl. missing/out-of-range `index` rejection and the `tokens_evaluated` pass-through), `parse_encoding_format`, `extract_embedding_prompt`, `is_infill_request`, `parse_slot_prompt_similarity`, `parse_positive_int_config`, `wrap_stream_chunk`, `server_metrics_to_json`, `route_error_message` | | `src/test/cpp/test_log_helpers.cpp` | 13 | All functions in `log_helpers.hpp`: `log_level_name`, `format_log_as_json` | | `src/test/cpp/test_common_log_callback.cpp` | 7 | **The runnable guard for `patches/0014`**: `common_log_set_callback()` on a private `common_log_init()` instance (never `common_log_main()`, so the process-wide logger the other tests print through is untouched) — delivery of level + bare text, no prefix/timestamp even when both are on (what `common_init()` does), clearing stops delivery, a swap drains queued entries to the *previous* sink (the property behind `LlamaModel.setLogger(format, null)` being a synchronous flush), a `--log-file` keeps being written alongside the sink, every `ggml_log_level` passes through unchanged, and the sink is not gated on upstream's `common_log_set_stdo()` console switch (b11612: entries reach the callback with stdout/stderr output off). The Java half (`LlamaLoggerTest`, model-free) proves the JNI trampoline on top of it. | @@ -2727,12 +2727,12 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" | `src/test/cpp/test_gallocr_leafs.cpp` | 1 | **The runnable guard for `patches/0020`.** Two graphs with the same two nodes in the same order (`n1 = add(x, x)`, `n2 = mul(w, n1)`) and only the leafs moved: one expands `n1` before `n2` and meets the input first, the other expands only `n2`, whose first source is the weight -- the shape of the llama.cpp case, where every node and node source still matches its slot, so the old check passed it. Reserved with the first, the second must get an allocation for the input inside the allocator's buffer and compute the right product on it. No model, every platform. **Verified both ways:** with the fix removed from the source tree the test dies with SIGSEGV (rc 139); a first version that swapped the operands of one `add` passed without the fix, because a swapped *node source* was already caught -- a guard has to move only the leafs. | | `src/test/cpp/test_lora_cache.cpp` | 5 | **The runnable guard for `patches/0021`.** Writes a random llama model (the shared `tiny_llama_gguf.hpp`, also used by `test_c_api.cpp`) and a random LoRA adapter on K, V and the FFN output of every layer, loads them with one slot and `--cache-ram 0`, and answers the same prompt greedily. After `jllama_set_lora_adapters` (scale 1 -> 0 and back) and after a request with its own adapters, the warm slot must answer exactly like an emptied one. One case guards the others against passing vacuously: the adapter changes the answer, and a repeated request reuses the cache (`timings.cache_n > 0`; not `tokens_cached`, which counts every token the slot holds and is positive after any request). After each adapter change the next request must take nothing from the cache (`timings.cache_n == 0`), so those three cases are red without the patch whether or not the random model happens to answer differently. **Verified both ways:** without the patch 3 of 4 red (the warm slot answered from the old scales), with it 4 of 4 green in three runs; no model, every platform. A second fixture (`LoraCacheFreshSlot`: two slots, `--slot-save-path`) saves slot 0 (by its file name inside that directory) after a request and restores it into the never-used slot 1, whose request must take all of the prompt but at most two tokens from it and answer like slot 0 -- red before the `init()` hunk (`Expected: (prompt_n) <= (2), actual: 77 vs 2`), green with it (F240). | -**Current total: 671 tests** (`TEST`/`TEST_F`/`TEST_P` definitions; ctest counts 692 cases, because +**Current total: 673 tests** (`TEST`/`TEST_F`/`TEST_P` definitions; ctest counts 694 cases, because the parameterized `CApiVocabOnly` case expands to 22 instances). #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11883`. +llama.cpp is fetched via CMake FetchContent, pinned to `JLLAMA_GIT_TAG b11887`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 1d4bd9c5a..b05bd7297 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11883](https://img.shields.io/badge/llama.cpp-%23b11883-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11883) +[![llama.cpp b11887](https://img.shields.io/badge/llama.cpp-%23b11887-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11887) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.1%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index e58ec503a..cd9f6e935 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -918,3 +918,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11827–b11881 | `tools/server/server.cpp`, `tools/server/server-common.{cpp,h}`, `tools/server/server-models.{cpp,h}`, `tools/server/server-context.{cpp,h}`, `tools/server/server-decision.cpp`, `tools/server/server-task.h`, `common/{arg.cpp,chat.{cpp,h},download.{cpp,h},file-lock.{cpp,h},hf-cache.{cpp,h},json-schema.cpp,jinja/value.cpp,speculative.{cpp,h},common.cpp,parsers/deepseek.cpp}`, `src/` (new `models/deepseek41.cpp`, `llama-kv-cache-dsv4.*`, `llama-vocab.*`, `models/dflash.cpp`), `tools/mtmd/` (new `models/soprano.cpp`, `mtmd-helper-gen.cpp`, `mtmd.h`), `ggml/include/ggml.h`, `ggml/src/ggml.c`, `ggml/src/ggml-rpc/transport.cpp`, the GPU backends, `vendor/cpp-httplib/CMakeLists.txt` (LibreSSL only), `tools/ui/` (54 commits, `fa3645feb` = b11881, 2026-10-22 .. 2026-10-25; **not tagged upstream at the time of the bump**, pinned through `JLLAMA_GIT_COMMIT`) | **No project source change; two patches rewritten (next row).** #31113 ("server: refactor server entrypoint") split `llama_server()` into `server_run()` and one function per role (`server_run_router`, `server_run_instance`, `server_run_download`), added `server_run_opts` (`install_signal_handlers`, `resolve_model`; llama-cli's overload clears both), factored the route table out as `server_register_routes()` (with the GCP shim) and `server_register_stream_routes()`, and moved `server_child` from `server-models.{cpp,h}` to `server_role` in `server-common.{cpp,h}` (same environment variables, same stdin/stdout protocol); the `server_state` enum moved from `server-context.h` to `server-common.h`. Nothing in `src/main/cpp` names any of them. #30242: JSON nested more than 128 levels is refused before it is walked -- `common_chat_templates_apply`'s inputs (tools, tool-call arguments, `chat_template_kwargs`, `json_schema`), the `json_schema` of `/completion` and the `/v1/systemone` body -- with `std::invalid_argument`, which every entry point here already maps to `InvalidRequestException`. #29312: `-hf` downloads take a lock shared with the hf CLI (`common/file-lock.cpp`, compiled into llama-common by upstream's own list), `common_download_file_single` and `hf_cache::finalize_file` gained defaulted parameters; not called here. Additive only in the reviewed headers: `common_chat_msg::extra_json` (DeepSeek V4.1 templates), `mtmd_audio_stream_get_n_rows[_flushed]`, `MTMD_GEN_AUDIO_TYPE_SOPRANO` (`tts_engine.cpp` accepts every type but `NONE`), `ggml_prec_get_acc/src`, `server_task_result_cmpl_partial::audio_commit_drained`. `include/llama.h`, `common/common.h`, `tools/mtmd/mtmd-helper.h`, `server-schema.cpp` and `server-task.cpp` are not in the range, so the server request-field set, its bounds and the response keys are identical. No option of `common/arg.cpp` changed (the 50 lines route the HF snapshot finalisation through the cancellable lock). The root `CMakeLists.txt` is unchanged (`GGML_LLAMAFILE_DEFAULT` / `GGML_CUDA_GRAPHS_DEFAULT` the same), and so is upstream's `release.yml`, so every `KEEP IN SYNC WITH UPSTREAM` pin of `publish.yml` stays; `vendor/cpp-httplib` moved LibreSSL to 4.4.0, BoringSSL (what this project builds) is unchanged. **Verified at b11881**: `jllama` and `jllama_test` built from the pin (configure: `llama.cpp b11881 is fetched by commit fa3645feb…`, `pin check: fetched commit count 11881 = b11881`, no compiler warning), `ctest` **692/692**, `NativeLibraryLoadSmokeTest`, `LlamaLoggerTest`, `RpcServerTest` and `BackendLoadTest` 32/32 against the b11881 library. | | b11827–b11881 | patches | **`0006` and `0007` rewritten, the other ten apply unchanged** (the fail-loud applier on a fresh tree at `fa3645feb`; `check-patches.py`: 12 patches, 94 hunks, 0 problems). Six patch-target files are in the range: `common/arg.cpp` (`0001`, `0015`; the HF-lock change is away from both), `ggml/src/ggml-rpc/transport.cpp` (`0015`; #30399 adds an RDMA bounds check, and RDMA is off here), `tools/server/server-context.{cpp,h}` (`0002`, `0003`, `0021`; #30864 and #30994 touch the rerank template and the realtime route, the `server_state` enum moved out of the header), `tools/server/server-models.cpp` (`0008`; `server_child` moved out, `update_args` unchanged) and `tools/server/server.cpp` (`0001`, `0006`, `0007`), where #31113 broke `0006` and `0007`. **`0006`:** the embedded mode clears upstream's new `server_run_opts::install_signal_handlers` instead of guarding the `sigaction` block, and the `llama_server_shutdown_handler_reset` moved into the two role functions that install a handler, declared after the last local the handler captures -- `models_routes` in the router (declared after `ctx_http` there, so a reset next to `ctx_http` would have run after `models_routes` was destroyed) and `ctx_http` in the instance; the forwarded-argv parse, the mutex and `llama_server_request_shutdown()` are unchanged. **`0007`:** upstream now has the route table as a function of its own, so the patch's `llama_server_register_common_routes()` and its removal block are gone (4 hunks to 2) and `llama_server_attach()` registers `server_register_routes()` plus `server_register_stream_routes()` with the instance handlers, exactly what `server_run_instance()` registers; the verbatim-copy invariant of the b10519 row no longer exists. Guards: `NativeServerShutdown` (3, the real `llama_server()` in router mode), `CApiServer.TheEmbeddedServerStartsInRouterModeAndStops` and `CApi.TheAttachedServerServesTheModelAndLeavesItUsable` green. Drop-checks re-read at `fa3645feb`: `0001` -- `argv = utf8.ptrs.data();` still at `common/arg.cpp:1296`, no `common_params_parse_main`; `0002` -- `params_base.load_progress_callback = load_progress_callback;` unguarded at `server-context.cpp:1286`; `0003` -- no `slot_prompt_similarity` in `server-context.h`; `0008` -- no `LLAMA_SERVER_WORKER_CMD` upstream; `0014` -- `common/log.h` names only `common_log_default_callback`; `0015` -- `GGML_ABORT("Failed to connect to %s\n"` at `ggml-rpc.cpp:650`; `0017` -- the four `_mm_prefetch(&x[ib] + sizeof(...))` calls at `quants.c:846/847/864/865`; `0018` -- `free(buffer->context);` at `amx.cpp:46`; `0020` -- `ggml_gallocr_needs_realloc()` compares the leaf count only; `0021` -- the `else` branch at `server-context.cpp:1942` still takes `params_base.lora_adapters` without the clear. | | b11881–b11883 | `src/llama-grammar.{cpp,h}`, `src/llama-quant.cpp` | **None needed.** Two commits: a recursion limit in the grammar parser (#29755) and the 4D-tensor/expert detection fix of the quantizer (#30820); no header the project includes changed, all twelve patches apply unchanged (`git apply` on `5c965fd8c`), the commit count is 11883. | +| b11883–b11887 | `include/llama.h`, `src/llama-sampler.cpp`, `tools/server/server-schema.cpp`, `src/models/qwen4exp.cpp`, `src/llama-arch.{cpp,h}` | **None needed in the code; a contract moved.** #31035 adds `LLAMA_DRY_MAX_SEQ_BREAKERS` (64) to `llama.h`: the request schema drops duplicate `dry_sequence_breakers` and refuses more than 64 distinct ones (an invalid request), the sampler stops at 64 with a warning. Pinned by `test_server.cpp` (`DrySequenceBreakers_DuplicatesDropped`, `DrySequenceBreakers_DistinctLimit`), noted in `withDrySequenceBreakers`' Javadoc. #31028 merges Qwen4-exp tensors on conversion (new tensor names); #30977/#31157 touch upstream CI and tests only. All twelve patches apply unchanged (`git apply` on `6050b467c`), the commit count is 11887. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 976fdac34..30c34ff76 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -356,8 +356,8 @@ endif() # hash, every other consumer keeps the number, and the pin check after FetchContent_MakeAvailable # asserts that the fetched commit's count IS the number. Clear it once the tag exists; # `llama-next-version.sh` says when, and `llama-next-version.sh --resolve b` prints the hash. -set(JLLAMA_GIT_TAG b11883) -set(JLLAMA_GIT_COMMIT "5c965fd8ca0b8ecd24f4034c29f9c00c1c16aa43") # the commit the tag b11883 names; kept as the pin (a tag name can move) +set(JLLAMA_GIT_TAG b11887) +set(JLLAMA_GIT_COMMIT "6050b467c07a95c7b87debd4ddac2f8ba8d9b807") # the commit the tag b11887 names; kept as the pin (a tag name can move) if(NOT JLLAMA_GIT_TAG MATCHES "^b[0-9]+$") message(FATAL_ERROR "JLLAMA_GIT_TAG must be a llama.cpp release number b, got '${JLLAMA_GIT_TAG}'") endif() diff --git a/llama/legal/natives/NOTICE.md b/llama/legal/natives/NOTICE.md index eb77fd975..2d003d9f4 100644 --- a/llama/legal/natives/NOTICE.md +++ b/llama/legal/natives/NOTICE.md @@ -22,7 +22,7 @@ texts are not needed here. | Component | Version | Compiled into | SPDX | Texts | Source | |---|---|---|---|---|---| -| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11883 5c965fd8ca0b8ecd24f4034c29f9c00c1c16aa43 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | +| llama.cpp (ggml / llama / mtmd / llama-common / tools/server / CPU+Metal+GPU kernels / WebUI own code) | b11887 6050b467c07a95c7b87debd4ddac2f8ba8d9b807 (texts unchanged since fa3645feb) | all 23 jars + both AARs | MIT | `llama.cpp` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/LICENSE | | nlohmann/json | 3.12.0 + vendor/nlohmann/json-deep-nesting.patch | library jars | MIT | `nlohmann-json` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/licenses/LICENSE-jsonhpp | | cpp-httplib | v0.62.0 | library jars | MIT | `cpp-httplib` | https://github.com/ggml-org/llama.cpp/blob/fa3645feb5f47cbb30df5e6d031c22543d43d240/vendor/cpp-httplib/LICENSE | | BoringSSL (ssl + crypto; static; OPENSSL_NO_ASM) | 0.20261020.0 = d3bf4795ad5b79d646cddf2fec687da5b9b502db | library jars except cpu-android-* and cpu-linux-s390x | Apache-2.0 | `boringssl` | https://boringssl.googlesource.com/boringssl/+/refs/tags/0.20261020.0/LICENSE | @@ -47,7 +47,7 @@ texts are not needed here. | WebUI vendored big-integer | in tree | library jars (embedded ui.cpp) | Unlicense | `webui-big-integer` | tools/ui/src/lib/vendors/big-integer/LICENSE | | WebUI vendored decimal.js | in tree | library jars (embedded ui.cpp) | MIT | `webui-decimal.js` | tools/ui/src/lib/vendors/decimal.js/LICENCE.md | | WebUI vendored nerdamer-prime | in tree | library jars (embedded ui.cpp) | MIT | `webui-nerdamer-prime` | tools/ui/src/lib/vendors/nerdamer-prime/LICENSE | -| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11883 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | +| WebUI npm bundle (JS + CSS incl. KaTeX fonts + workbox) | tools/ui/package-lock.json at b11887 | library jars (every library build job downloads webui-generated) (upper bound (tree-shaking / server-only deps)) | 337 pkgs: MIT 271, ISC 43, BSD-3-Clause 9, Apache-2.0 5, BSD-2-Clause 2, MPL-2.0 OR Apache-2.0 1, Unlicense 1, 5 without field (MIT files); 19 MIT packages ship no license file, the MIT text follows them | `webui-npm` | webui-npm/packages.csv | | WebUI third-party logos (huggingface.ico / exa.ico / context7.png / github-*.png) | - | library jars (yes) | trademarks (no license file) | `(not shipped)` | tools/ui/static | | CCCL (CUB / Thrust / libcu++ headers) | v3.4.3 | cuda13-linux-x86-64 + cuda13-windows-x86-64 | Apache-2.0 WITH LLVM-exception AND Apache-2.0 AND BSD-3-Clause (CUB) | `cccl` | https://github.com/NVIDIA/cccl/blob/v3.4.3/LICENSE | | CUDA Toolkit headers / libdevice | CUDA 13.4 | cuda13 modules (yes) | LicenseRef-NVIDIA-CUDA-EULA | `(not shipped)` | - | diff --git a/llama/src/main/java/net/ladenthin/llama/parameters/InferenceParameters.java b/llama/src/main/java/net/ladenthin/llama/parameters/InferenceParameters.java index a6fe4e8b0..e045d80b7 100644 --- a/llama/src/main/java/net/ladenthin/llama/parameters/InferenceParameters.java +++ b/llama/src/main/java/net/ladenthin/llama/parameters/InferenceParameters.java @@ -778,7 +778,8 @@ public InferenceParameters withDryPenaltyLastN(int dryPenaltyLastN) { * tokens at which DRY restarts matching, so repetition is not penalized across them (llama.cpp * default: {@code ["\n", ":", "\"", "*"]}). Empty input is a no-op (returns {@code this}), so when * this wither is not called nothing is emitted and the server's default breakers apply. Only takes - * effect when {@link #withDryMultiplier(float)} is non-zero. + * effect when {@link #withDryMultiplier(float)} is non-zero. The server drops duplicates and refuses + * more than 64 distinct breakers as an invalid request (each one costs a scan of the vocabulary). * * @param breakers the sequence-breaker strings * @return a new instance with the breaker array set, or {@code this} if {@code breakers} is empty diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 04638d820..55ba3d1be 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -9,13 +9,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11883"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11887"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11883-"} — call + * plus the resolved upstream commit, e.g. {@code "b11887-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -23,14 +23,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11883"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11887"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11883"; + public static final String LLAMA_CPP_VERSION = "b11887"; // Constants holder — not instantiable. private LlamaCppVersion() {} diff --git a/llama/src/test/cpp/test_server.cpp b/llama/src/test/cpp/test_server.cpp index 1615ca29d..1072c1ab3 100644 --- a/llama/src/test/cpp/test_server.cpp +++ b/llama/src/test/cpp/test_server.cpp @@ -1975,6 +1975,30 @@ TEST(ParamsFromJsonCmpl, DrySequenceBreakers_NonEmpty_RoundTrip) { EXPECT_EQ(p.sampling.dry_sequence_breakers[3], "*"); } +// Since b11887 (#31035) the schema drops duplicate breakers, keeping the first occurrence of each, +// and refuses more than LLAMA_DRY_MAX_SEQ_BREAKERS distinct ones: every distinct breaker costs a +// scan of the whole vocabulary when the sampler is built, so a long list exhausted CPU and memory. +TEST(ParamsFromJsonCmpl, DrySequenceBreakers_DuplicatesDropped) { + const auto p = parse_params({{"dry_sequence_breakers", json::array({"\n", ":", "\n", "*", ":"})}}); + ASSERT_EQ(p.sampling.dry_sequence_breakers.size(), 3u); + EXPECT_EQ(p.sampling.dry_sequence_breakers[0], "\n"); + EXPECT_EQ(p.sampling.dry_sequence_breakers[1], ":"); + EXPECT_EQ(p.sampling.dry_sequence_breakers[2], "*"); +} + +TEST(ParamsFromJsonCmpl, DrySequenceBreakers_DistinctLimit) { + json breakers = json::array(); + for (int i = 0; i < LLAMA_DRY_MAX_SEQ_BREAKERS; ++i) { + breakers.push_back("b" + std::to_string(i)); + } + // a duplicate does not count against the limit + breakers.push_back("b0"); + EXPECT_EQ(parse_params({{"dry_sequence_breakers", breakers}}).sampling.dry_sequence_breakers.size(), + static_cast(LLAMA_DRY_MAX_SEQ_BREAKERS)); + breakers.push_back("one too many"); + EXPECT_THROW(parse_params({{"dry_sequence_breakers", breakers}}), std::invalid_argument); +} + TEST(ParamsFromJsonCmpl, LoraNotArray_Throws) { EXPECT_THROW(parse_params({{"lora", "not-an-array"}}), std::invalid_argument); }