diff --git a/CHANGELOG.md b/CHANGELOG.md index 70b1da1..d92d42a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,8 +17,35 @@ The release procedure (prompt template and step-by-step instructions) lives in [ Windows the `` block meant for a `pom.xml` carried 3x CRLF and 3x LF. All lines end in `\n` now. `CalibrationReportTest` renders the block in a child JVM started with `-Dline.separator=\r\n`, so the Linux CI sees what only a Windows build showed before (red without the fix). +- **...and that fix left `spotbugs:check` red.** SpotBugs reports `VA_FORMAT_STRING_USES_NEWLINE` for every + `String.format` pattern containing `\n` and asks for `%n` — the platform separator this block must not + carry, so the rule and the requirement are in direct conflict. Nothing could have shown it: the reactor + still dies earlier on the unresolvable `net.ladenthin:llama:6.0.0` (see `TODO.md`), so no CI run reaches + SpotBugs, and `main` was red on a check that never executed. `appendPasteBlock` appends the newline as a + literal through a new `appendXmlNumber` helper now — the shape `appendJsonNumber`/`appendYamlNumber` + already use for the same reason — and formats the numbers at the call site through the `FORMAT_*` constants + the JSON and YAML renderers share. No pattern carries a newline, the block keeps its LF endings, and no + suppression was added. ### Changed +- **The `srcmorph-selftest` profile names its model directory once, as `ai.modelDir`.** All 38 + `` entries of `srcmorph-maven-plugin/pom.xml` carried the absolute `X:/Modelle/...`, a drive + that no longer exists on the owner's machine, so the profile could not run anywhere as committed — and a + machine-specific absolute path sat in a published POM. They read `${ai.modelDir}/.gguf` now, the + property defaults to the owner's model drive, and another machine runs the profile with + `-Dai.modelDir=/path/to/models`. `docs/ai-index-benchmark/tools/calibrate-bench-pom.xml` keeps its single + overridable `bench.model` and only follows the drive. +- **`LlamaCppJniAiGenerationProvider` no longer explains `gpuLayers = -1` in terms of a "CPU build" and a + "GPU build".** That was jllama's pre-6.0.0 artifact layout, where the classifier chose the backend. Since + 6.0.0 the library jar is CPU-only and a GPU backend joins it as an additive module jar, so the comment's + premise was gone while its conclusion stayed right: `-1` leaves llama.cpp's own default, which offloads + everything once a GPU device is registered. Measured on an RTX 3070 Laptop (Qwen3-4B-Instruct-2507-Q4_K_M, + identical config, only the classpath differing): 63 t/s prefill and 7.1 t/s decode without the module jar, + 3474 / 86.8 with it, and `gpuLayers` -1 vs 99 within noise either way. +- Removed four unused constants and one string concatenation from `LlamaCppJniAiGenerationProviderTest` + (six Error Prone `UnusedVariable` / `StringConcatToTextBlock` warnings). The wire-key assertions they + belonged to moved to the model-free `LlamaCppJniProviderSupportTest` with the core-module extraction, + where the same four constants are declared and used; these were fossils, so no coverage is lost. - Build tooling bumped in step with the sibling repositories: `anthropics/claude-code-action` v1.0.258 and `github/codeql-action` v4.39.1. maven-clean-plugin 3.5.0, maven-resources-plugin 3.5.0, maven-install-plugin 3.2.0, maven-deploy-plugin 3.2.0 and maven-dependency-plugin 3.11.0 are now declared, diff --git a/TODO.md b/TODO.md index bca8c67..844651c 100644 --- a/TODO.md +++ b/TODO.md @@ -34,6 +34,16 @@ everything below is genuinely still open. a self-hosted runner, or a manual pre-release pass on one GPU box per backend. Worth knowing which half is covered before reading the green check as "the CUDA jar runs". + **`cuda13-windows-x86-64` has had that pass (2026-10-23, against `llama 6.0.0-SNAPSHOT`).** On an + RTX 3070 Laptop the `-P gpu-cuda` fat jar (258 MB, carrying `Windows/x86_64/cuda13/` next to the six + CPU library directories) ran a real `GenerateFileIndex` over the production prompt set through the + CLI *and* through `srcmorph:generate`, both at 3474 t/s prefill / 86.8 t/s decode, with the loader + naming the module (`with GPU module(s) cuda13`) and ggml registering `[CUDA, RPC, CPU]`. The same + config on the default CPU fat jar is 63 / 7.1 with `[RPC, CPU]`, so the jar demonstrably loads its + backend rather than falling back. **Still unlaunched: `rocm`, `sycl`, `openvino` and `vulkan`** — + no AMD, Intel or second GPU here. Vulkan is reachable on any NVIDIA box and is the cheapest one to + add next. + - **Follow java-llama.cpp and drop the per-classifier fat jars? (owner's decision)** Since its modular natives java-llama.cpp ships no fat jar at all: the thin jar on Central is the deliverable, a GPU is `jbang --deps net.ladenthin:llama:: …`, and its smoke jobs launch the diff --git a/docs/ai-index-benchmark/tools/calibrate-bench-pom.xml b/docs/ai-index-benchmark/tools/calibrate-bench-pom.xml index d8554c6..8959386 100644 --- a/docs/ai-index-benchmark/tools/calibrate-bench-pom.xml +++ b/docs/ai-index-benchmark/tools/calibrate-bench-pom.xml @@ -6,7 +6,7 @@ SPDX-License-Identifier: Apache-2.0 4.0.0 benchbench1pom - X:/Modelle/granite-4.0-h-small-UD-Q4_K_XL.gguf + M:/Modelle/granite-4.0-h-small-UD-Q4_K_XL.gguf 163840 0.04 diff --git a/srcmorph-maven-plugin/pom.xml b/srcmorph-maven-plugin/pom.xml index 573002e..a8e1107 100644 --- a/srcmorph-maven-plugin/pom.xml +++ b/srcmorph-maven-plugin/pom.xml @@ -102,6 +102,12 @@ SPDX-License-Identifier: Apache-2.0 ${project.basedir}/src/site/ai + + M:/Modelle + true @@ -879,7 +885,7 @@ SPDX-License-Identifier: Apache-2.0 Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL-C64k (throughput-optimal alternative; gpt-oss-20B-c96k is now the ACTIVE default) + MoE 30B total / ~3.3B active: code-specialized AND fast on CPU; fewer active params than Gemma-4-26B-A4B; non-thinking; Apache-2.0. - + Locally present (X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf, + + Locally present (${ai.modelDir}/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf, 17.7 GB); 64K context - generous but CPU-feasible (256K prefill is unusable on CPU). - Needs Qwen sampling (temp 0.7 / top_p 0.8 / top_k 20 / repeat 1.05). Use when: this is the go-to for most code summarization on CPU. @@ -911,7 +917,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/lmstudio-community/Mistral-7B-Instruct-v0.3-GGUF/blob/main/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf --> Mistral-7B-Instruct-v0.3-Q4_K_M-C16k - X:/Modelle/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf + ${ai.modelDir}/Mistral-7B-Instruct-v0.3-Q4_K_M.gguf 16384 2048 1.0 @@ -924,7 +930,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf --> Meta-Llama-3.1-8B-Instruct-Q5_K_M-C32k - X:/Modelle/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf + ${ai.modelDir}/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf 32768 1536 1.0 @@ -937,7 +943,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/Codestral-22B-v0.1-GGUF/blob/main/Codestral-22B-v0.1-Q4_K_M.gguf --> Codestral-22B-v0.1-Q4_K_M-C32k - X:/Modelle/Codestral-22B-v0.1-Q4_K_M.gguf + ${ai.modelDir}/Codestral-22B-v0.1-Q4_K_M.gguf 32768 1536 1.0 @@ -950,7 +956,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/Codestral-22B-v0.1-GGUF/blob/main/Codestral-22B-v0.1-Q6_K.gguf --> Codestral-22B-v0.1-Q6_K-C32k - X:/Modelle/Codestral-22B-v0.1-Q6_K.gguf + ${ai.modelDir}/Codestral-22B-v0.1-Q6_K.gguf 32768 1536 1.0 @@ -963,7 +969,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF/blob/main/qwen2.5-coder-7b-instruct-q5_k_m.gguf --> Qwen2.5-Coder-7B-Instruct-Q5_K_M-C16k - X:/Modelle/qwen2.5-coder-7b-instruct-q5_k_m.gguf + ${ai.modelDir}/qwen2.5-coder-7b-instruct-q5_k_m.gguf 16384 2048 1.0 @@ -979,7 +985,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-GGUF --> Ministral-3-8B-Instruct-2512-Q4_K_M-C32k - X:/Modelle/Ministral-3-8B-Instruct-2512-Q4_K_M.gguf + ${ai.modelDir}/Ministral-3-8B-Instruct-2512-Q4_K_M.gguf 32768 2048 0.1 @@ -996,7 +1002,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512-GGUF --> Ministral-3-14B-Instruct-2512-Q4_K_M-C32k - X:/Modelle/Ministral-3-14B-Instruct-2512-Q4_K_M.gguf + ${ai.modelDir}/Ministral-3-14B-Instruct-2512-Q4_K_M.gguf 32768 2048 0.1 @@ -1012,7 +1018,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-GGUF --> Ministral-3-8B-Instruct-2512-Q5_K_M-C32k - X:/Modelle/Ministral-3-8B-Instruct-2512-Q5_K_M.gguf + ${ai.modelDir}/Ministral-3-8B-Instruct-2512-Q5_K_M.gguf 32768 2048 0.1 @@ -1029,7 +1035,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512-GGUF --> Ministral-3-14B-Instruct-2512-Q5_K_M-C32k - X:/Modelle/Ministral-3-14B-Instruct-2512-Q5_K_M.gguf + ${ai.modelDir}/Ministral-3-14B-Instruct-2512-Q5_K_M.gguf 32768 2048 0.1 @@ -1046,7 +1052,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF --> Gemma-4-26B-A4B-it-UD-Q4_K_M-C32k - X:/Modelle/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf + ${ai.modelDir}/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf 32768 8192 1.0 @@ -1068,7 +1074,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/gemma-4-12B-it-GGUF --> Gemma-4-12B-it-Q4_K_M-C32k - X:/Modelle/gemma-4-12B-it-Q4_K_M.gguf + ${ai.modelDir}/gemma-4-12B-it-Q4_K_M.gguf 32768 8192 1.0 @@ -1090,7 +1096,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive --> Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M-C32k - X:/Modelle/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf + ${ai.modelDir}/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf 32768 8192 1.0 @@ -1114,7 +1120,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive --> Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q6_K_P-C32k - X:/Modelle/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf + ${ai.modelDir}/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf 32768 8192 1.0 @@ -1135,7 +1141,7 @@ SPDX-License-Identifier: Apache-2.0 non-thinking (emits no blocks, so no enable_thinking flag is needed or set), Apache-2.0. Unsloth Dynamic UD-Q4_K_XL quant, single-file GGUF (17.7 GB) at - X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. + ${ai.modelDir}/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. Official Qwen3-Coder sampling (model card): temperature 0.7, top_p 0.8, top_k 20, repetition_penalty 1.05. Context: native window is 262144; 64 K is a deliberate, generous @@ -1147,7 +1153,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF --> Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL-C64k - X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf + ${ai.modelDir}/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf Qwen2.5-Coder-14B-Instruct-Q4_K_M-C32k - X:/Modelle/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf + ${ai.modelDir}/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf 32768 2048 0.7 @@ -1189,7 +1195,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF/resolve/main/Qwen3-4B-Instruct-2507-Q5_K_M.gguf --> Qwen3-4B-Instruct-2507-Q5_K_M-C32k - X:/Modelle/Qwen3-4B-Instruct-2507-Q5_K_M.gguf + ${ai.modelDir}/Qwen3-4B-Instruct-2507-Q5_K_M.gguf 32768 2048 0.7 @@ -1208,7 +1214,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/ibm-granite_granite-4.0-h-tiny-GGUF/resolve/main/ibm-granite_granite-4.0-h-tiny-Q4_K_M.gguf --> Granite-4.0-H-Tiny-Q4_K_M-C128k - X:/Modelle/ibm-granite_granite-4.0-h-tiny-Q4_K_M.gguf + ${ai.modelDir}/ibm-granite_granite-4.0-h-tiny-Q4_K_M.gguf 131072 2048 0.2 @@ -1224,7 +1230,7 @@ SPDX-License-Identifier: Apache-2.0 https://huggingface.co/bartowski/ibm-granite_granite-4.0-h-small-GGUF/resolve/main/ibm-granite_granite-4.0-h-small-Q4_K_M.gguf --> Granite-4.0-H-Small-Q4_K_M-C128k - X:/Modelle/ibm-granite_granite-4.0-h-small-Q4_K_M.gguf + ${ai.modelDir}/ibm-granite_granite-4.0-h-small-Q4_K_M.gguf 131072 2048 0.2 @@ -1240,7 +1246,7 @@ SPDX-License-Identifier: Apache-2.0 ============================================================ --> EXP-Qwen3.5-4B - X:/Modelle/Qwen3.5-4B-Q4_K_M.gguf + ${ai.modelDir}/Qwen3.5-4B-Q4_K_M.gguf 16384 1536 0.7 @@ -1254,7 +1260,7 @@ SPDX-License-Identifier: Apache-2.0 EXP-Qwen2.5-Coder-7B - X:/Modelle/qwen2.5-coder-7b-instruct-q5_k_m.gguf + ${ai.modelDir}/qwen2.5-coder-7b-instruct-q5_k_m.gguf 16384 1536 0.7 @@ -1314,7 +1320,7 @@ SPDX-License-Identifier: Apache-2.0 ============================================================ --> gpt-oss-20B-c16k - X:/Modelle/gpt-oss-20b-UD-Q4_K_XL.gguf + ${ai.modelDir}/gpt-oss-20b-UD-Q4_K_XL.gguf 16384 ${ai.gpuLayers} ${ai.mainGpu} @@ -1332,7 +1338,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-c48k - X:/Modelle/gpt-oss-20b-UD-Q4_K_XL.gguf + ${ai.modelDir}/gpt-oss-20b-UD-Q4_K_XL.gguf 49152 ${ai.gpuLayers} ${ai.mainGpu} @@ -1350,7 +1356,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-c96k - X:/Modelle/gpt-oss-20b-UD-Q4_K_XL.gguf + ${ai.modelDir}/gpt-oss-20b-UD-Q4_K_XL.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1400,7 +1406,7 @@ SPDX-License-Identifier: Apache-2.0 ============================================================ --> gpt-oss-20B-mxfp4 - X:/Modelle/gpt-oss-20b-mxfp4.gguf + ${ai.modelDir}/gpt-oss-20b-mxfp4.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1418,7 +1424,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-Q4_K_M - X:/Modelle/gpt-oss-20b-Q4_K_M.gguf + ${ai.modelDir}/gpt-oss-20b-Q4_K_M.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1436,7 +1442,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-Q5_K_M - X:/Modelle/gpt-oss-20b-Q5_K_M.gguf + ${ai.modelDir}/gpt-oss-20b-Q5_K_M.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1454,7 +1460,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-Q8_0 - X:/Modelle/gpt-oss-20b-Q8_0.gguf + ${ai.modelDir}/gpt-oss-20b-Q8_0.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1472,7 +1478,7 @@ SPDX-License-Identifier: Apache-2.0 gpt-oss-20B-F16 - X:/Modelle/gpt-oss-20b-F16.gguf + ${ai.modelDir}/gpt-oss-20b-F16.gguf 98304 ${ai.gpuLayers} ${ai.mainGpu} @@ -1519,7 +1525,7 @@ SPDX-License-Identifier: Apache-2.0 rules and are map-reduced (cheap small chunks, maxChunks-bounded). --> granite-4.0-h-tiny-bigwindow - X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf + ${ai.modelDir}/granite-4.0-h-tiny-Q4_K_M.gguf 393216 ${ai.gpuLayers} ${ai.mainGpu} @@ -1556,7 +1562,7 @@ SPDX-License-Identifier: Apache-2.0 huge-java / huge-sql rules below (priority above big-window). --> granite-4.0-h-1b-fastchunk - X:/Modelle/granite-4.0-h-1b-Q4_K_M.gguf + ${ai.modelDir}/granite-4.0-h-1b-Q4_K_M.gguf 16384 ${ai.gpuLayers} ${ai.mainGpu} @@ -1575,7 +1581,7 @@ SPDX-License-Identifier: Apache-2.0 EXP-Qwen3-Coder-30B - X:/Modelle/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf + ${ai.modelDir}/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf 16384 1536 0.7 @@ -1593,7 +1599,7 @@ SPDX-License-Identifier: Apache-2.0 ============================================================ --> EXP-DeepSeek-Coder-V2-Lite - X:/Modelle/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf + ${ai.modelDir}/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf 16384 1536 0.3 @@ -1606,7 +1612,7 @@ SPDX-License-Identifier: Apache-2.0 EXP-Seed-Coder-8B - X:/Modelle/Seed-Coder-8B-Instruct-Q4_K_M.gguf + ${ai.modelDir}/Seed-Coder-8B-Instruct-Q4_K_M.gguf 16384 1536 0.3 @@ -1619,7 +1625,7 @@ SPDX-License-Identifier: Apache-2.0 EXP-Granite-4.0-H-Tiny - X:/Modelle/granite-4.0-h-tiny-Q4_K_M.gguf + ${ai.modelDir}/granite-4.0-h-tiny-Q4_K_M.gguf 16384 1536 0.0 @@ -1636,7 +1642,7 @@ SPDX-License-Identifier: Apache-2.0 Same 16384 window + 1536 output as H-Tiny for benchmark comparability. --> EXP-Granite-4.0-H-Small - X:/Modelle/granite-4.0-h-small-UD-Q4_K_XL.gguf + ${ai.modelDir}/granite-4.0-h-small-UD-Q4_K_XL.gguf 16384 1536 0.0 @@ -1649,7 +1655,7 @@ SPDX-License-Identifier: Apache-2.0 EXP-Qwen3-4B-Instruct-2507 - X:/Modelle/Qwen3-4B-Instruct-2507-Q4_K_M.gguf + ${ai.modelDir}/Qwen3-4B-Instruct-2507-Q4_K_M.gguf 16384 1536 0.7 diff --git a/srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrationReport.java b/srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrationReport.java index f5c4883..7155e09 100644 --- a/srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrationReport.java +++ b/srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrationReport.java @@ -32,10 +32,10 @@ public final class CalibrationReport { /** File name of the machine-readable YAML report, written alongside {@link #JSON_FILE_NAME}. */ public static final String YAML_FILE_NAME = "srcmorph-calibration.yaml"; - /** Format of the two throughput figures; identical to the one {@link #renderXml()} pastes. */ + /** Format of the two throughput figures, shared by {@link #renderXml()} and the JSON/YAML reports. */ private static final String FORMAT_TOKENS_PER_SECOND = "%.1f"; - /** Format of the chars-per-token figure; identical to the one {@link #renderXml()} pastes. */ + /** Format of the chars-per-token figure, shared by {@link #renderXml()} and the JSON/YAML reports. */ private static final String FORMAT_CHARS_PER_TOKEN = "%.2f"; /** Format of the load duration, which has no counterpart in the XML block. */ @@ -238,16 +238,42 @@ private static void appendPasteBlock( final StringBuilder out, final String modelKey, final AiCalibrationMeasurement m) { out.append("\n"); out.append("\n"); - out.append(String.format( - Locale.ROOT, - " %.1f\n", - m.prefillTokensPerSecond())); - out.append(String.format( - Locale.ROOT, " %.1f\n", m.decodeTokensPerSecond())); - out.append(String.format(Locale.ROOT, " %.2f\n", m.charsPerToken())); + appendXmlNumber( + out, + "prefillTokensPerSecond", + String.format(Locale.ROOT, FORMAT_TOKENS_PER_SECOND, m.prefillTokensPerSecond())); + appendXmlNumber( + out, + "decodeTokensPerSecond", + String.format(Locale.ROOT, FORMAT_TOKENS_PER_SECOND, m.decodeTokensPerSecond())); + appendXmlNumber(out, "charsPerToken", String.format(Locale.ROOT, FORMAT_CHARS_PER_TOKEN, m.charsPerToken())); out.append("\n"); } + /** + * Appends one indented {@code value} line of the paste block, ending in {@code \n}. + * + *

The line separator is a literal here and never part of a {@link String#format} pattern: + * SpotBugs reports {@code VA_FORMAT_STRING_USES_NEWLINE} for a pattern containing {@code \n} and + * wants {@code %n}, which is exactly the platform-dependent separator this block must not carry + * (see the {@code renderXml()} contract and {@code CalibrationReportTest}). Keeping the newline + * out of the pattern satisfies both. The value arrives already rendered for the same reason as in + * {@link #appendJsonNumber(StringBuilder, String, String)}.

+ * + * @param out the buffer + * @param tag the XML element name + * @param value the already-formatted value + */ + private static void appendXmlNumber(final StringBuilder out, final String tag, final String value) { + out.append(" <") + .append(tag) + .append('>') + .append(value) + .append("\n"); + } + /** One model's key paired with its {@link AiCalibrationMeasurement}. */ @ToString @EqualsAndHashCode diff --git a/srcmorph/src/main/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProvider.java b/srcmorph/src/main/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProvider.java index e4dd002..5023a25 100644 --- a/srcmorph/src/main/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProvider.java +++ b/srcmorph/src/main/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProvider.java @@ -90,8 +90,17 @@ private LlamaModel model() { if (config.cacheReuse() > 0) { modelParameters.setCacheReuse(config.cacheReuse()); } - // Offload model layers to the GPU when explicitly configured (>= 0). -1 leaves the - // binding/native-build default (CPU build stays CPU; GPU build uses its own default). + // Offload model layers to the GPU when explicitly configured (>= 0). -1 leaves llama.cpp's + // own default, which offloads everything when a GPU device is registered and stays on the + // CPU when none is -- so a GPU needs no gpuLayers, only the matching natives jar on the + // classpath. Since jllama 6.0.0 there is no "CPU build" or "GPU build" to speak of: the + // library jar is CPU-only and a GPU backend joins it as an additive module jar + // (-Dllama.classifier=cuda13-windows-x86-64 and friends, see srcmorph/pom.xml). + // Measured on an RTX 3070 Laptop with Qwen3-4B-Instruct-2507-Q4_K_M, same config except + // the classpath: without the module jar 63 t/s prefill and 7.1 t/s decode (ggml registers + // [RPC, CPU]); with it 3474 / 86.8 (ggml registers [CUDA, RPC, CPU]). gpuLayers -1 and 99 + // were within noise of each other in both cases, which is what "llama.cpp's own default" + // means here. if (config.gpuLayers() >= 0) { modelParameters.setGpuLayers(config.gpuLayers()); } diff --git a/srcmorph/src/test/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProviderTest.java b/srcmorph/src/test/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProviderTest.java index dd7635e..4b13068 100644 --- a/srcmorph/src/test/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProviderTest.java +++ b/srcmorph/src/test/java/net/ladenthin/srcmorph/provider/LlamaCppJniAiGenerationProviderTest.java @@ -20,21 +20,6 @@ public class LlamaCppJniAiGenerationProviderTest { private static final String MODEL_PATH = NativeLlamaAvailability.modelPath(); - /** - * JSON key of the DRY penalty window as the binding renders it. Quoted so the assertions cannot - * accidentally match a different key that merely contains this one as a substring. - */ - private static final String PARAM_DRY_PENALTY_LAST_N = "\"dry_penalty_last_n\""; - - /** JSON key of the repeat-penalty window, quoted for the same reason. */ - private static final String PARAM_REPEAT_LAST_N = "\"repeat_last_n\""; - - /** Chat-template kwarg key for Qwen-style thinking, as the provider spells it. */ - private static final String KWARG_ENABLE_THINKING = "enable_thinking"; - - /** Chat-template kwarg key for the gpt-oss reasoning-effort level. */ - private static final String KWARG_REASONING_EFFORT = "reasoning_effort"; - private static final AiMdHeader HEADER = new AiMdHeader( "Test.java", AiMdHeaderCodec.HEADER_VERSION_1_0, @@ -63,13 +48,16 @@ private static AiGenerationRequest request(final String source) { public void generate_realProvider_returnsNonEmptyResponse() throws Exception { NativeLlamaAvailability.assumeAvailable(); final AiPromptSupport promptSupport = new AiPromptSupport(CommonTestFixtures.createFilePromptDefinitions()); - final String source = "package com.example;\n" + "\n" - + "public class Test {\n" - + "\n" - + " public String hello(final String name) {\n" - + " return \"Hello \" + name;\n" - + " }\n" - + "}\n"; + final String source = """ + package com.example; + + public class Test { + + public String hello(final String name) { + return "Hello " + name; + } + } + """; try (LlamaCppJniAiGenerationProvider provider = new LlamaCppJniAiGenerationProvider(config(32768, 128), promptSupport)) {