llm: OLLAMA_IGPU_DIRECT_IO=0 opts out of forced direct I/O on integrated GPUs

[glennneuber]

Adds an opt-out for the --load-mode dio flag that ollama forces on Linux integrated CUDA/ROCm devices. Written on 2026-09-17 to satisfy gate clause 3(b); what a day of measurement since then did to its rationale is recorded below, because the honest case for it is now weaker than the one in the gate document.

What it does

OLLAMA_IGPU_DIRECT_IO=0 suppresses --load-mode dio. Default is unchanged (on), so this is inert unless set.

if envconfig.IntegratedGPUDirectIO(true) {
    for _, g := range gpus {
        if runtime.GOOS == "linux" && g.Integrated && (strings.EqualFold(g.Library, "CUDA") || strings.EqualFold(g.Library, "ROCm")) {
            return append(params, "--load-mode", "dio")
        }
    }
}

Covered by TestAppendLoadModeArgsIntegratedDirectIOOptOut, mutation-tested: replacing the guard with if true turns it red. Verified on hardware — the dio-on runner line carries --load-mode dio, the dio-off line does not.

Why the rationale changed

The AMD upgrade gate treats --direct-io as an unvalidated risk on gfx1151 and clause 3 asks for it to be absent, opt-out-able, or validated. This patch was the "opt-out-able" answer.

Measurement since (#304 addendum, #317) has exonerated direct I/O:

• It does not cause the vision regression that made 0.34.x unusable on gfx1151 — that was upstream's HIP prop.integrated defect, and OLLAMA_IGPU_DIRECT_IO=0 reproduces the regression exactly.
• It is a large benefit on this host: 1203 completed model loads in a V3 window with dio on, against 3 with it off.
• The nearest thing to disabling the underlying host-buffer path, GGML_CUDA_NO_PINNED=1, aborts the runner on load (signal: aborted, core dumped).

So this knob is a diagnostic convenience, not a safety valve. Its real value turned out to be as an experimental control: it let us isolate dio as a variable and rule it out, which is how the gate's central suspicion of the last month was retired.

Why land it anyway

• It made a refutation possible, and will again — an A/B on the load path with no rebuild is worth having.
• The default is unchanged, the guard is tested, and the blast radius is one flag on one code path.
• docs: ROCm gate — #17459 and #17475 do not reproduce on gfx1151 #304 cites it as satisfying clause 3(b). If it does not land, that clause reverts to unsatisfied on a technicality, for a flag we have since shown to be harmless and useful.

Drop it if the gate is retired or upstream exposes its own opt-out.

🤖 Generated with Claude Code

[glennneuber]

The vision-suite A/B is now done, on a healthy build, and the arms are provably different

Closing the loop on this PR's open item. Every earlier dio A/B was run on unpatched 0.34.1, where the HIP prop.integrated defect (#317 / compat 906) was destroying vision output in both arms — "dio does not hurt quality" measured while quality was already gone is not a result.

The first re-run was also invalid, and this PR's own instrumentation caught it. I ran it against the hip906 image, which carries 906 but predates this PR, so OLLAMA_IGPU_DIRECT_IO=0 was ignored and both arms launched --load-mode dio. The scores were identical because the configuration was identical. The guard that exposed it is the runner's actual flag line — worth keeping in any future A/B of an env-gated launcher flag, because two identical arms look exactly like a null result.

Valid run. Image built from main @ 16649e8c (0.34.1 + compat 906 + this knob + #320), run_engine_compare.sh, think off, OLLAMA_NUM_PARALLEL=2, same digests and runner both arms:

arm	container env	runner flag	qwen3.6:35b-a3b	gemma4:31b-it
dio on	unset (default)	--load-mode dio	0.972	0.960
dio off	OLLAMA_IGPU_DIRECT_IO=0	(no --load-mode flag)	0.972	0.960

Fine-text 6/6 · 6/6 · 6/6, invoice 5/5 · 5/5, grounding 4/5 — identical in both arms, and both at the post-906 baseline.

Conclusion: no measurable quality effect either way, so the default stays on. That is what this PR shipped, but the reasoning behind it has changed: it now rests on a quality measurement from a build whose vision path is known-good, rather than on inference from a broken one. The knob's value is as a diagnostic control — it is precisely how dio was ruled out as a cause of the gfx1151 regression before the real cause was found.

Had the arms diverged against dio, this would have needed a follow-up PR flipping the default, not an edit here — the PR was already merged when the measurement completed.

🤖 Generated with Claude Code