From eec4988cd8b7d7dadeec900d1e596469e20720b9 Mon Sep 17 00:00:00 2001 From: Digant Desai Date: Fri, 30 Oct 2026 16:28:30 -0700 Subject: [PATCH] Update [ghstack-poisoned] --- backends/native/runtime/vulkan/TODO.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/backends/native/runtime/vulkan/TODO.md b/backends/native/runtime/vulkan/TODO.md index 2b6b1de54df..f4c8d3bd7ad 100644 --- a/backends/native/runtime/vulkan/TODO.md +++ b/backends/native/runtime/vulkan/TODO.md @@ -14,3 +14,23 @@ HuggingFace Llama 3.2 1B 8da4w, 2K export: peak GPU memory is 2794 MiB, against 2265 MiB with buffer SDPA disabled and 2263 MiB for the ET-VK `.pte`. The +528 MiB comes from enabling buffer SDPA. Give these temporaries lifetimes in the memory plan, measure again, and attribute whatever remains. + +## Lower int4 weight-only linear for any N + +`FuseQuantizedLinear` leaves an int4 weight-only linear unlowered when N is not +a multiple of 4, to match `VK_CHECK_COND(N % 4 == 0)` in ET-VK's +`prepack_q4_w_4x8_nc_buffer`, and the engine then rejects the quantized weight. +Round N up to a multiple of 4 there (the pack shader already fills n >= N with +zeros) and mask the partial last tile on store. Until then Vulkan fails the op +sample tests `linear_int4wo_*.linear_feature_sizes_2` and +`linear_int4wo_*.linear_unbatched_0`, where N is 1 and 43. + +## Rank-4 linear input + +Vulkan returns wrong outputs for the op sample test +`linear_float.linear_leading_batch_1`, an `nn.Linear(67, 43)` on a +(2, 4, 8, 67) input: in f32 the fitted rel_bias is -0.48, and f16 returns +values up to 6.1e4. Both are consistent with only one of the two outer batches +being written. The rank-3 inputs (4, 8, 67) and (2, 7, 256) pass. Find where +the outer dimension is lost: the engine's linear lowering, or ET-VK's linear on +a rank-4 tensor.