fix(LOAD-MODELOPT-NVFP4-BORROW): one owner per device resident

[iiLaurens]

Row

LOAD-MODELOPT-NVFP4-BORROW — one row per PR. Spec:
.agents/specs/nvfp4-one-owner-per-resident.md.

Before starting

• Issue/PR search: no open issue described this leak. Issue A DFlash2 run reserved 156 GiB of VM for a ~40 GiB workload, and the global OOM killer took coredns with it — so the fleet reported the device dead while the host never rebooted #1647 (ModelOpt
  borrow) is a different defect in the same row and closed in flow. Filed
  ISSUE-LOCAL-01M3QDT6A8JEM8WNJTPR16MXHR.
• Pull request shape selected at row claim: one PR for the spec and the
  implementation.
• Row: LOAD-MODELOPT-NVFP4-BORROW in .agents/engine-matrix.md; the row spec
  load-modelopt-nvfp4-borrow.md owns ResidentNvfp4's upload and adoption path.
• Anchors inspected: Nvfp4Weight (qwen3_5_weights.h:672-712), OwnedTensor::d_dev
  (:195), both ResidentNvfp4 copies (dense_nvfp4_gemm.h:319, qwen3_5.cpp:1449),
  and every release site (dense_nvfp4_gemm.h:448,667; qwen3_5.cpp:2952,3132,6890;
  laguna.cpp:650).

What changed

Nvfp4Weight published one device allocation on two owning handles: the
type-specific d_packed/d_scale and the generic packed.d_dev/scale.d_dev
slot that AdoptDeviceBytesAsHost keys on. Every Marlin repack builder reset only
the type-specific pair, so the d_dev alias kept a full packed+scale device copy
alive for the process lifetime — once per repacked weight, and once per expert per
projection on an MoE model. That undoes the repack's whole point ("then FREE the
fp4 originals") on exactly the 24 GiB cards the NVFP4 arms serve from.

The change makes d_dev the one owner. The type-specific members are deleted (a
remaining reader fails to compile), both upload copies publish into d_dev
directly, and Nvfp4Weight::ReleaseResident() is the single release: it drops an
adopted host twin view first (that view holds the same block on a host-addressable
device), then resets both slots. scripts/check-fp4-resident-consistency.py, which
pinned the old two-handle contract, is updated in the same change: its publication
clause now requires a backend-owned allocation and a new clause requires the
release to reset BOTH slots with the twin drop first.

Evidence

RED, before the fix (the new case, against unmodified main):

$ ./build/tests/test_load_direct_upload -tc="fp4 resident: releasing the resident frees both device buffers"
CHECK( b.frees == 2 ) is NOT correct!
  values: CHECK( 0 == 2 )
test cases: 1 | 0 passed | 1 failed | 16 skipped

The invariant is that a release through the weight's own resident state frees both
buffers. Pre-fix it freed zero: the surviving packed.d_dev alias owned the block.

GREEN, after the fix:

$ ./build/tests/test_load_direct_upload
test cases:  17 |  17 passed | 0 failed | 0 skipped
assertions: 206 | 206 passed | 0 failed

Checker, including its mutation suite:

$ python3 scripts/check-fp4-resident-consistency.py
OK: 2 ResidentNvfp4 definition(s) across 2 file(s) count the upload, copy,
publish d_dev, and adopt — per buffer, for both packed and scale — and
ReleaseResident() resets both d_dev slots.

$ python3 -m unittest tests.scripts.test_check_fp4_resident_consistency
Ran 53 tests ... OK

The new release clause goes red under each live-tree mutation of
qwen3_5_weights.h: drop packed.d_dev.reset(), drop scale.d_dev.reset(), drop
either adopted-twin drop, move a twin drop after its reset; and the publication
clause under = nullptr or a bare alias of another object's d_dev.

• python3 scripts/check-agent-record.py OK; commit style and trailer
  contracts OK for both commits
• tests that cover this change: tests/vllm/test_load_direct_upload.cpp,
  tests/scripts/test_check_fp4_resident_consistency.py
• public docs unchanged (no user-facing fact changed)
• full ctest --test-dir build: 795 of 809 pass, and every one of the 14
  failures reproduces on a clean origin/main (b45a94273) build:
  test_model_loader_gguf, test_gliner2_e2e, test_gdn_v_head_permute,
  test_deepseek_v4_exl3_loader, test_qwen4_exp_layer_loop,
  test_glm4_moe_lite_paged_engine, test_bench,
  test_bench_kv_cache_dtype, test_bench_eos_chat_template, test_capi,
  test_qwen3_dense_async_serving, test_qwen3_paged_engine (each run as a
  base binary, exit 1), plus test_cpu_threadpool (segfault/abort 3/3 on
  base) and test_serve_low_tools (fails on base). test_serve_deepseek_v4_mm
  hangs in accept() on this box and is Not Run in the base ctest.
• full scripts/agent-preflight.sh — RED ON THIS BOX AT BASE. Each failing
  checker was re-run on a clean origin/main worktree and fails there too;
  none of the 35 red gates touches a file this PR changes.

End-to-end A/B on the official NVFP4 safetensors checkpoint

The unit gate proves the release frees; this proves the release MATTERS on a real
artifact. ukisai/Swift-1.5-Qwen3.8-27b-NVFP4 (21.94 GB, 5 shards, public and
ungated), the ModelOpt NVFP4 + FP8 checkpoint the local GGUF is converted from,
run on the local sm_120a card (RTX PRO 4000 Blackwell, 24 GiB) under the file
mutex, same command, same binary build recipe, only the branch differs:

$ VT_LOAD_STATS=1 ./build-cuda/examples/vllm-cli --model /tmp/models/Swift-1.5-Qwen3.8-27b-NVFP4 \
    --device cuda --kv-cache-dtype fp8 --max-num-seqs 1 --kv-cache-memory 2000000000 \
    --max-tokens 8 --prompt "The capital of France is" --temperature 0

Tree	Result
This branch (fix)	exit 0: generated 8 tokens (" Paris.\nThe capital of Germany is"), 1.064 tok/s, device_upload=12.044 GiB
main + the unrelated PermuteVHeads CUDA kernel (PR #3359)	exit 1: engine-fatal: EngineCore busy loop threw: vt cuda: marlin_repack: malloc bqt: out of memory at device_upload=8.619 GiB
main + the fp8-KV prefill change (PR #3360)	the same OOM, same device_upload=8.619 GiB

Two independent trees without the fix die in the NVFP4 Marlin repack; the fix
reaches the same repack and finishes. That is the leak: BuildMarlinDenseResident
repacks a weight and frees the fp4 originals only through the type-specific handle,
and the surviving d_dev alias keeps a full packed+scale copy alive per repacked
weight — so the first forward runs out of card. The repack's own comment says the
free is the point ("we do this repack lazily on first forward and then FREE the
fp4 originals"), and on a 24 GiB card a second copy is the difference between
serving and OOM.

Speed claims

• This PR makes NO speed claim. It frees a leaked allocation; no timing was
  taken and none is implied.

Honest gaps

• No end-to-end GiB/RSS number. The gate proves the release frees both buffers
  through the fake backend; it does not measure a repacking serve's process RSS.
• The mutation evidence is author-run. A fresh reviewer over the immutable head
  still owes its own mutation pass.
• scripts/agent-preflight.sh on this box is red at base (35 gates): record
  drift in .agents/completed/ and docs/benchmarks/, supported-models drift,
  the oracle pin mismatch (e126687a9a vs a7c23ac96d in upstream-sync.md),
  and one environment FAIL (file is not installed for the release-archive
  test). Every one reproduces on clean main, and none touches a changed file.

Refs ISSUE-LOCAL-01M3QDT6A8JEM8WNJTPR16MXHR

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:opencode-go/deepseek-v4.1-flash [pi]

[iiLaurens]

Human comment:
This was a memory leak in VRAM that prevented me from loading the Qwen 3.8 27B NVFP4 model on my 24GB GPU. It probably also affects other models. This fix allowed me to succesfully load and use the NVFP4 model.