docs(binaries): archive the payload, not just the binary - #307
Conversation
BINARIES.md has said since 2026-08-30 that "a binary is not a rollback on its own" — the MLX library is dlopen'd from build/lib/ollama and the binary does not carry it. The "Adding one" procedure never acted on that: it archived the binary alone. So every row in the table was a one-way door in practice. The payload for each archived build existed in exactly one place, the checkout's build/lib/ollama, and was destroyed by the next build that moved a pin. That became concrete today. Staging 0.34.0-maxusai-8a7ba949 on :11435 meant overwriting the only copy of the 0.33.2 payload; rolling back would have been a ~25 minute cold rebuild at 2b95b4a, itself possible only because Xcode's separately-downloadable Metal toolchain happened to get installed this morning. Both pairings are now archived — payload-0.33.2-maxusai-2b95b4a5/ and payload-0.34.0-maxusai-8a7ba949/ beside the binaries — so rolling between them is a copy. The procedure is updated to archive both halves, and to archive the OUTGOING pairing before overwriting it. Also adds the 0.34.0-maxusai-8a7ba949 row. It is recorded as built and staged, NOT serving: preflight PASS 19/12 against the mlx-metal-0-34-0 profile (#306), and #301's whitespace bound confirmed behaviourally on real documents across the MLX pin move, but the :11435 kickstart has not run. The 0.33.2 row loses "current" and says what it actually is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The kickstart ran at 13:17. Verified three ways rather than by version alone: :11435 reports 0.34.0-maxusai-8a7ba949 on a new pid, a live MLX load emitted "MLX version"=0.32.2-61-gd9add9d, and the dlopen path resolved /opt/github/MaxusAI/ollama/build/lib/ollama — so the Go and MLX halves are the pairing that was measured, not a skew. The 0.33.2 row becomes the rollback target and now says what is new about it: both halves are archived, so that rollback is a copy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 0.33.2 row carries "goldens NOT recalibrated at this MLX pin" as a caution. It turns out they did not need to be. Golden parity passes on this build for 31b, 26b and 12b, with max sampled element deltas of 0.109, 0.051 and 0.063. The 12b golden is the striking one: generated 2026-08-07 and never recalibrated, it is three MLX pins behind (27fec909, c793734e, d9add9d1) and still matches to four decimal places on mean, std and norm_mean. The 31b and 26b goldens were recalibrated once at 27fec909 (d53d33a) and have held through both moves since. So the vision encoder path has been numerically stable across every pin move this fork has made, and this is the first row that can say its goldens were checked rather than assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Follow-up: the payload archive turned out to be load-bearing within hours of being created, which is worth recording as evidence rather than argument. Three separate experiments on 2026-09-18 required standing
Under the binary-only procedure this PR replaces, each would have required a ~25 minute cold rebuild at With both halves archived, restoring the old pairing is a copy. The finding in #310 exists because of that. Deployment is confirmed and the row here reflects it: |
BINARIES.md has said since 2026-08-30 that "a binary is not a rollback on its own" — the MLX library is
dlopen'd frombuild/lib/ollamaand the binary does not carry it. The "Adding one" procedure never acted on that: it archived the binary alone.So every row in the table was a one-way door in practice. The payload for each archived build existed in exactly one place — the checkout's
build/lib/ollama— and was destroyed by the next build that moved a pin.Why this stopped being theoretical
Staging
0.34.0-maxusai-8a7ba949on:11435meant overwriting the only copy of the 0.33.2 payload. Rolling back would have been a ~25 minute cold rebuild at2b95b4a5— itself possible only because Xcode's separately-downloadable Metal toolchain happened to be installed this morning. Without that, the rollback path was closed entirely.Both pairings are now archived beside the binaries:
Rolling between those two is now a copy. The rows above
0.33.2still need a checkout and rebuild, and the section says so rather than implying otherwise.Changes
0.34.0-maxusai-8a7ba949— llama.cppb10864, MLXd9add9d1, sha256867ee5d6…0.33.2row loses "current", which it no longer isOn the new row's wording
It says built and staged, NOT yet serving. Preflight is
PASS 19/12against themlx-metal-0-34-0profile (#306), and #301's whitespace bound is confirmed behaviourally on real documents across the MLX pin move — but the:11435kickstart has not run, and the row should not claim a deployment that hasn't happened. It needs updating to "current" once it does.🤖 Generated with Claude Code