dasLLAMA vulkan: the resident plan's auto-arm headroom follows the platform - the 2 GiB floor where no OS reports room (Linux), the WDDM share on Windows - so the Qwen3.6-35B-A3B arms unpinned on a 16 GB Linux card; the tier names a driver without the four-wide decode; the Linux RTX 5080 scaling pass ledgered

[borisbat]

Behavior change: on a box where no OS reports video-memory room (Linux), the whole-model resident driver's auto arm keeps the 2 GiB floor as its headroom instead of the larger of 2 GiB and 27% of the tier's cap; Windows keeps the share.

Why. The review of #3988 ran the branch on a Linux RTX 5080: unpinned, the driver declined the Qwen3.6-35B-A3B UD-IQ2_XXS ("needs 16464 MB of 10406 MB (weights alone 11167 MB)") and the per-op rails served at 2.5x lower rates, while DASLLAMA_GPU_VRAM_MB=14000 armed it whole. The room query the plan reads is the Windows memory manager's, so on Linux it never answers and the plan took a headroom sized for a WDDM demotion collapse that Linux does not have.

What changes.

• plan_headroom(budget, wddm): the WDDM share on Windows, the 2 GiB floor elsewhere; plan_room calls it with the platform. A decline taken with no OS answer names the headroom it stood on and DASLLAMA_GPU_VRAM_MB as the pin that reads no room.
• The tier warns at device init when the driver reports no VK_NV_cooperative_matrix_decode_vector: the cm2 tiles then run the scalar decode arm, and a MoE prefill window runs about half again as long (160 ms with the four-wide arm against 237 without, the 5060 Ti's 35B window).
• followup_vulkan.md items 44 and 45: the driver's K-quant admission riding the CPU repack flag (the review's second note), and the scaling diagnosis below.

Observable behavior. On a rented Linux RTX 5080 (driver 580.173, 8 vCPU), the 35B unpinned: declined on master (per-op rails 648.4 / 25.4) -> armed at ctx 44159 with this branch, pp512 3361.1 +- 41.6 / tg128 139.8 +- 0.1, the same rate as pinned at 14000 MB (armed at ctx 102754: 3356.2 / 140.1). On Windows nothing moves: plan_headroom(16 GiB, true) is still 27%.

Where to look. plan_headroom / plan_room in dasllama_gpu_resident.das; the device-init warning in dasllama_vulkan_common.das; test_plan_room in tests/test_gpu_serving_declines.das pins both arms and the plan's wiring.

Validation, the Linux rows, the scaling diagnosis

Validation

• tests/test_gpu_serving_declines.das 17 of 17 (the plan-room cell asserts the WDDM arm at 27% of 16 GiB, the floor winning on a 4 GiB cap, the off-WDDM 2 GiB, and the plan taking this platform's headroom). Lint clean on both rails; the review-md gate OK.
• The pod (RunPod community cloud, RTX 5080 16303 MiB, Ubuntu 24.04, NVIDIA 580.173, clang, LLVM 22, the Linux CI's configure with GLFW off): the merged code unpinned declined exactly as the reviewer's box did; this branch unpinned armed at ctx 44159; the model file's sha256 is the model-set row's (2e8f5f70...), the reviewer's byte count 10756586464 the same file.

Same-box rows (the pod, -t 8, debug-jit)

engine	pp512	tg128
llama.cpp b10660 Vulkan release, -ngl 99 -fa 1	5352.6 +- 24.1	147.7 +- 0.4
llama.cpp b10886 (current)	5461.2 +- 5.5	148.1 +- 0.3
dasLLAMA resident, pinned 14000 (ctx 102754)	3356.2 +- 43.1	140.1 +- 0.2
dasLLAMA resident, unpinned with this branch (ctx 44159)	3361.1 +- 41.6	139.8 +- 0.1
dasLLAMA per-op rails (master unpinned, declined)	648.4 +- 44.6	25.4 +- 2.5

The reviewer's 32-core box read 3660 / 145 for our pinned row and 5083.8 / 140.3 for llama.cpp b10660 built from source; the host accounts for the difference.

Scaling 5060 Ti (Windows 616.56) -> 5080 (Linux 580.173); bandwidth 448 -> 960 GB/s, 36 -> 84 SMs

	5060 Ti	5080	ratio
llama.cpp b10660 pp512 / tg128	2853.1 / 71.6	5352.6 / 147.7	1.88x / 2.06x
dasLLAMA pp512 / tg128	2995.8 / 107.7	3361.1 / 139.8	1.12x / 1.30x
the 512-row window, DASLLAMA_GPU_PROF=1	160181 us	145782 us	1.10x
the same with the four-wide decode off on the 5060 Ti	236883 us	145782 us	1.62x
the token	9397 us	7778 us	1.21x

Three causes, from the per-role profile (ledger item 45 carries every role):

1. The four-wide decode twin is off on Linux. The driver does not list VK_NV_cooperative_matrix_decode_vector (it lists _matrix, _matrix2, _vector), so every cm2 tile runs the scalar arm - and so do llama.cpp's there, at a 32-deep expert-tile k step where its four-wide arm runs 64. Same box, same source (b10660), the 5060 Ti's 35B pp512: llama.cpp four-wide 2828.9, scalar at k step 32 2385-2393, scalar at k step 64 2080; ours twin 2995.8, scalar at k step 64 2049. Arm- and k-step-matched the two engines read the same, so on this axis llama.cpp's Linux edge is its 32-deep k step (1.15x for it), and the tiles alone stay 1.2-1.3x behind at either matched arm (its scalar decode is written for the driver's two-wide commoning: a pair's shared work once, the element selected last). A local llama.cpp reference built with a glslc that does not know the extension (Vulkan SDK 1.4.350) runs the scalar arm at k step 64 by default and under-reads the reference by 27% - its banner says NV_coopmat2, the four-wide build's NV_coopmat2v (the bring-up document's reference section now says so). Arm-matched, the expert tiles scale 1.80 to 1.90x; the probe reads 0.71 ms per iq2xxs gate/up plane on the 5080 no-twin against 0.73 twin-on / 1.33 twin-off on the 5060 Ti.
2. Flat window roles on either arm: the deltanet scan (13037 -> 12654 us, a serial recurrence), the shared expert's k5/k6 tiles (3518 -> 3379, 1582 -> 1452), the router's 32 fixed workgroups (2276 -> 2371), and the routed tiles' real schedule of ~105 single-workgroup pieces per plane, one wave on either card.
3. Decode's per-dispatch floor: the expert GEMV moves 3.3 MB per layer in 8.5 us on the 5080 (3.4 of transfer, ~5 fixed) and 11.3 on the 5060 Ti (7.4 + 3.9); some 600 dispatches per token; every small kernel slower on the wider card. The GEMV probe itself scales with bandwidth (iq2xxs 410 -> 881 GB/s, iq3xxs 414 -> 886).

The pod's card ran 2910-2925 MHz at 250-263 W of a 307 W limit under load. Host over GPU time on the pod: 176 us per token.

Not done

• The kernel levers of cause 1 (a 32-deep k step for the scalar arm, then the decode body's pair form) and of causes 2 and 3 (column-split schedule pieces, an LPT order, the shared-expert tile pick at 84 SMs, the scan's parallelism; fewer dispatches per decode layer) - research first, their own slices, ledger item 45.
• A device-side servability predicate for the driver's K-quant admission (ledger item 44) - needs a box whose backend does not repack.
• The 5080 rows are pod rows, not board rows.

[Copilot]

🟢 Approval recommended

The behavior change is well-contained, tests cover the new headroom rules, and the remaining feedback is limited to minor documentation wording.

Pull request overview

This PR adjusts dasLLAMA’s Vulkan resident-driver auto-arming behavior so that when the OS does not report per-process VRAM “room” (notably on Linux), the plan uses a fixed 2 GiB headroom floor instead of a WDDM-oriented percentage share; Windows retains the WDDM share behavior. It also improves diagnosability for performance regressions when the driver lacks the cooperative-matrix decode-vector extension.

Changes:

• Introduces plan_headroom(budget, wddm) and updates plan_room / resident decline messaging to reflect platform-specific headroom and the DASLLAMA_GPU_VRAM_MB pin behavior.
• Adds a Vulkan-tier init warning when VK_NV_cooperative_matrix_decode_vector is not reported (cm2 decode falls back to scalar path).
• Updates tests and documentation/architecture notes to match the new headroom rules and observed driver capability behavior.

File summaries

File	Description
modules/dasLLAMA/dasllama/dasllama_gpu_resident.das	Adds platform-sensitive headroom helper and updates plan/decline messaging for “no OS room” cases.
modules/dasLLAMA/tests/test_gpu_serving_declines.das	Expands test_plan_room coverage to assert the new plan_headroom behavior and platform wiring.
modules/dasLLAMA/dasllama/dasllama_vulkan_common.das	Logs a warning when cm2 decode-vector is requested but not supported by the driver.
modules/dasLLAMA/ENVIRONMENT.md	Clarifies semantics of DASLLAMA_GPU_VRAM_MB w.r.t. pinning/headroom/OS room queries.
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_RESIDENCY.md	Updates residency-plan architecture doc to match the revised headroom policy and decline messaging.
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_GEMM.md	Documents decode-vector fallback cost and the new init-time warning.
modules/dasLLAMA/followup_vulkan.md	Adds follow-up ledger items describing K-quant admission coupling and scaling diagnosis notes.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

The new runtime warning message hardcodes specific driver version claims that are likely to become stale and should be made version-agnostic.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

modules/dasLLAMA/dasllama/dasllama_vulkan_common.das:1273

• The new device-init warning hardcodes specific driver branches/versions ("Linux 580" vs "Windows 616"). That’s likely to become stale quickly and may mislead users on other drivers; the actionable part is the missing extension and the expected perf impact. Consider making the message version-agnostic and (optionally) pointing to the env knob being requested.

• Files reviewed: 7/7 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The new warning/docs describe the scalar decode slowdown as “about a third” despite the PR’s own cited measurements indicating ~1.5× (~48%) longer, which makes the user-facing messaging inconsistent.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 8/8 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The changes are scoped, test-backed, and align code/docs around platform-specific VRAM headroom behavior, with only minor doc wording nits noted.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_GEMM.md:151

• The parenthetical compares two window times but the second value (“237”) is missing its unit, which makes the sentence ambiguous to read/scans as a typo.

• Files reviewed: 9/9 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The behavior change is narrowly scoped, is covered by targeted tests, and the associated diagnostics/documentation updates are consistent with the implementation.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 0 new
• Review effort level: Lite