docs: state the real glibc floor of the s390x jar; follow-ups in TODO

CHANGELOG.md

@@ -45,8 +45,11 @@ the first two items break every existing consumer at upgrade.
    `OpenAiServerConfig.Builder.port(int)`) where a fixed port matters, and point clients that were
    configured for `http://localhost:8080` at the new port.
 5. **Platform floors:** Windows 10 or newer (Universal CRT), macOS 15.0, glibc 2.28 on Linux
-   (RHEL 8, Ubuntu 20.04, Debian 10 and later), Android API 28; the ROCm jars need a ROCm 10
-   runtime, the CUDA jars CUDA 13. 32-bit Windows and the `msvc-windows-x86-64` jar no longer ship.
+   x86-64 and aarch64 (RHEL 8, Ubuntu 20.04, Debian 10 and later), Android API 28. The s390x jar is
+   the exception: glibc 2.38 and the libstdc++ of GCC 13 (Ubuntu 24.04, SLES 15 SP6, Debian 13; not
+   RHEL 8/9, not Ubuntu 22.04), because it is cross-compiled on `ubuntu-latest`. The ROCm jars
+   need a ROCm 10 runtime, the CUDA jars CUDA 13. 32-bit Windows and the `msvc-windows-x86-64` jar
+   no longer ship.
 6. **Rejected requests** come back as `InvalidRequestException` (a `LlamaException`, so existing
    handlers still match) with the bare reason as the message; the OpenAI-compatible server answers
    `400` for them.

README.md

@@ -259,7 +259,7 @@ platforms you target, e.g. `cpu-linux-x86-64`.
 |---|---|---|---|
 | `cpu-linux-x86-64` | CPU | Linux x86-64 | A JDK 8+ JVM; glibc ≥ 2.28 (manylinux_2_28: RHEL 8, Ubuntu 20.04, Debian 10 and later). Ships one CPU backend module per instruction-set level (x86-64 baseline, SSE4.2, AVX, AVX2, AVX-512, AVX-VNNI, AMX), of which ggml loads the best for the running CPU at start-up — a CPU without AVX2 works, and an AVX-512/AMX machine uses its kernels. |
 | `cpu-linux-aarch64` | CPU | Linux aarch64 | glibc ≥ 2.28 (RHEL 8, Ubuntu 20.04, Debian 10, Amazon Linux 2023 and later) — built in the manylinux_2_28 image on an arm64 runner. Ships one CPU backend module per ARM feature level (armv8.0 up to armv9.2: dotprod, fp16, SVE, i8mm, SVE2, SME), chosen at start-up like the x86-64 ones. |
-| `cpu-linux-s390x` | CPU | Linux s390x (IBM Z, big-endian) | A JDK 8+ JVM. |
+| `cpu-linux-s390x` | CPU | Linux s390x (IBM Z, big-endian) | A JDK 8+ JVM; **glibc ≥ 2.38 and the libstdc++ of GCC 13** (`GLIBCXX_3.4.32`): Ubuntu 24.04, SLES 15 SP6, Debian 13 and later. **Not RHEL 8 or 9, not Ubuntu 22.04** -- the load fails there with `GLIBC_2.38 not found`. Cross-compiled on `ubuntu-latest`, not in a manylinux image, so the 2.28 floor of the other Linux CPU jars does not apply. |
 | `cpu-windows-x86-64` | CPU | Windows x86-64 | A JDK 8+ JVM; **Windows 10 or newer**. Ships one CPU backend module per instruction-set level (as the Linux x86-64 jar: x86-64 baseline, SSE4.2, AVX, AVX2, AVX-512, AVX-VNNI, AMX), of which ggml loads the best for the running CPU at start-up. Built with clang, hybrid CRT: the STL and vcruntime are static (no `msvcp140.dll` / `vcruntime140.dll`, no VC++ redistributable), the Universal CRT is the OS one (`ucrtbase.dll`), so Microsoft's UCRT security updates apply. |
 | `cpu-windows-aarch64` | CPU | Windows on ARM (Snapdragon X / Surface) | A JDK 8+ JVM; **Windows 10 or newer**. Built natively on `windows-11-arm` with `clang-cl`, same hybrid CRT as the x86-64 jar; one CPU backend module (ggml builds no ARM variant set for Windows), so the Vulkan and OpenCL module jars can join it. |
 | `metal-macos-aarch64` | Metal + CPU | macOS aarch64 (Apple silicon) | A JDK 8+ JVM. |

TODO.md

@@ -485,6 +485,62 @@ ROCm anyway. So this needs a real Radeon machine, or a deliberate decision to sh
 of a green link plus `verify-native-deps.py`'s dependency allowlist. Do not quietly treat "the job is
 green" as the measurement.
 
+### Linux s390x: glibc 2.38 floor (after the release)
+
+**Measured 2026-10-25** on the CI artifacts of Publish runs 40217818548 and 40227501358.
+
+What each Linux CPU jar references:
+
+| jar | GLIBC | GLIBCXX | built with |
+|---|---|---|---|
+| `cpu-linux-x86-64` | 2.27 | 3.4.25 | GCC 14.2.1, manylinux_2_28 |
+| `cpu-linux-aarch64` | 2.27 | 3.4.25 | GCC 14.2.1, manylinux_2_28 |
+| `cpu-linux-s390x` | **2.38** | **3.4.32** | GCC 13.3.0, Ubuntu 24.04 cross toolchain |
+
+The 2.38 symbols are `__isoc23_strtol` / `strtoul` / `sscanf` and their relatives, and `fmod`. The
+2.34 symbols are `pthread_*` and `dl*`. All of them are only the newest symbol versions of the
+cross sysroot (glibc 2.39); no code needs them.
+
+The load was tested as a consumer does it: `llama` + `cpu-linux-s390x` from `llama-jars` on the
+classpath, then `LlamaLoader.initialize()`, in Docker with s390x emulated by qemu.
+
+- **Fails** on RHEL 8.10 (glibc 2.28) with `UnsatisfiedLinkError … GLIBC_2.38 not found`. `ldd`
+  also lists `GLIBCXX_3.4.26`..`3.4.32` and `CXXABI_1.3.13`, so libstdc++ blocks it as well.
+- **Fails** on RHEL 9 (2.34) and Ubuntu 22.04 (2.35).
+- **Loads** on Ubuntu 24.04.
+- **`ldd` clean** on SLES 15 SP6 (BCI, glibc 2.38), Debian 13 and Ubuntu 25.04.
+- **Control:** `cpu-linux-x86-64` and `cpu-linux-aarch64` load on RHEL 8.
+
+**Not a decision but a side effect.** `GLIBC_CEILING` in `.github/buildcheck/nativedeps.py` holds
+only the manylinux directories. Every other Linux build "inherits the floor of the ubuntu runner";
+CLAUDE.md justifies that for GPU modules only. `1559fe4c` never mentions s390x. The CI test cannot
+see the gap: its `ctest` runs under qemu against the same 2.39 sysroot. The README and the
+CHANGELOG now state the real floor.
+
+Follow-ups, none of them before the release (no build change now):
+
+- [ ] **Pin `build-linux-s390x` to `runs-on: ubuntu-24.04`** instead of `ubuntu-latest`. The output
+      stays identical (24.04 is `ubuntu-latest` today), but a runner image change can no longer
+      raise the floor to ~2.42 without anyone noticing.
+- [ ] **`"Linux/s390x/cpu": (2, 38)` in `GLIBC_CEILING`.** This freezes today's floor, so
+      `verify-native-deps.py` fails loudly on any further rise instead of passing it. Extend
+      `test_nativedeps.py` with it. Optionally add a GLIBCXX ceiling too; the check reads GLIBC
+      only today.
+- [ ] **The real fix: floor 2.28 like the other Linux jars.** Then set `(2, 28)`, and put back the
+      README row and CHANGELOG item 5.
+      - **Candidates** (measure them, do not guess):
+        - (a) build in `quay.io/pypa/manylinux_2_28_s390x` (gcc-toolset-14) as aarch64 does. No
+          GitHub s390x runner exists: either emulate (slow) or check whether IBM's runner offer for
+          open-source projects applies;
+        - (b) cross-compile against a glibc 2.28 s390x sysroot (AlmaLinux 8) with a newer cross GCC.
+      - **libstdc++:** either way it needs the gcc-toolset treatment (newer parts linked statically,
+        `GLIBCXX` ≤ 3.4.25).
+      - **Acceptance:** on real LinuxONE hardware, cfarm191/192 (`workspacePrivate/cfarm`), with
+        RHEL 8 in Docker as the quick check.
+- **Ruled out:** `ubuntu-22.04` as the runner (glibc 2.35 still misses RHEL 8/9, and its cross GCC 11
+  cannot build llama.cpp, which needs GCC >= 12). `-static-libstdc++` alone is ruled out too: it
+  fixes GLIBCXX but not GLIBC.
+
 ### Hardware and test-fixture coverage
 
 - **Real arm64 Android / Adreno OpenCL device:** run the AAR + Kotlin facade and LLM Service app