dasLLAMA vulkan: a MoE that fits the card serves on the whole-model resident driver - the routed block on the device, the shared expert's K-quant planes, the decode GEMV lane split; Qwen3.6-35B-A3B whole at 1.05x / 1.50x of llama.cpp, Qwen3-30B-A3B 1.00x / 1.15x, the Qwen1.5-MoE twin 1.03x / 0.96x - #3988
Merged
Conversation
… qwen2moe-class layer's shexp triple, resident under the shexp mark, runs as ONE region over every position of the window on the same arm the routed experts take - the f16-fed cm2 chain with an identity slot map and unit combine weights where the tier admits it, the quant form over the Q8_0 image elsewhere - and the reduce applies the per-row sigmoid gate in the CPU form's order; before this the prefill ran the shared expert's three GEMMs on the CPU, 582 ms of a 945 ms window on Qwen1.5-MoE-A2.7B (`--prof`, the Q4_K_M mint, RTX 5060 Ti), the per-op prefill's largest term on every shared-expert MoE; the twin now reads pp512 1430.4 against 538.1 (llama.cpp 5099.8), its window 360 ms, the shared expert 59 ms on the device, decode unchanged; `shexp_gpu_prefill_layers()` counts the layers the arm served (the tests' engage witness); the resident-driver comment that kept the prefill shexp off the dense rail's three submits now names the chain the shexp takes; the GEMM doc's expert-chain section carries the arm and its number; the twin joins the model-set table as a recipe row (`llama-quantize --allow-requantize <Q8_0> <out> Q4_K_M`); tests: test_vulkan_moe_cm2.das gains the shared expert's call shape (one region over every row, the identity map at unit weight, k = 1, at 40 rows and at 300 rows over a 2048-wide dim, the tight bar with its poison), and the new stocked file test_gpu_moe_shexp.das holds the model-level form on the twin - the same prompt and fed tokens through the tier's route lever, device arm against the CPU form, within 0.12 of the max logit at the prefill and every step with the one-step-off control and the engage counter, at 64 and 600 tokens; the bar is the MoE class of CPU-vs-device noise, not the arm's: the arms agree to 1e-8 at layer 0 and part from layer 1 as the device chain's summation order moves the residual by ulps, the next requant rounds a few elements the other way and the router flips a near-tie (measured 0.05-0.10 under both tile modes; the shared expert's rows themselves sit within 1e-4 of the CPU form on the sdot4 feed and 0.5% on the f16 feed, and two route-off sessions are bit-identical) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… layer's resident q8 triple rides inside the span beside the routed slots - its feed is the same normed row requantized into its own stacks' images, its metas are one fixed region the host writes once at record (the top-k never touches them), its gate logit is one more router GEMV row over the layer's gate vector landing past the router's logits, and one combine (`DecCombineSh`, the routed slots plus sigmoid(gate) times the shared down row onto the residual; an ungated shared expert adds at weight one) replaces the plain one on such a layer; `span_model_ok` now declines only gemma4's dense shared expert, the span's per-layer serve gate asks the shexp mark, and `span_tokens()` counts the tokens the span served (the tests' engage witness); before this the span declined every shared-expert model to the per-layer path, where a resident layer's shared expert ran on the CPU (`sh_gpu` serves the shexp rail only beside CPU experts) - on the Qwen1.5-MoE twin that was 18 of a 26 ms token beside 7.6 ms in 46 submits - the twin now reads tg128 134.0 against 38.5 (llama.cpp 173.8), the token 7.5 ms, prefill unchanged at 1435.9; the decode doc's span section carries the arm; `vk_span_reset` forgets a recorded span so a second model in one process (the tests' shape) records anew; tests: test_vulkan_dec_tail.das gains the span with a gated shared expert against the CPU reference (the shared q8 triple beside the routed pair, the gate logit past the router's, one combine, two positions, the poisoned-element control and the no-shared-expert reference that must miss the device row; `vk_drop_model_state` first, since the span records once per model, and `ffn_up` takes a base), and test_gpu_moe_shexp.das holds the span serving every fed step on the twin's device arm and none on the CPU arm Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lan counts every MoE layer's expert triple (placed in the arena sliced per expert, as the per-op walk gathers it), its shared expert's q8 triple and the f32 router plane (every MoE layer's rows, a gated shared expert's gate row beside them), sized BEFORE the per-op reserves so a fitting plan forgoes the streamed slot and the decode mirrors; the window chain gains a routed block over the FFN-normed rows - the router GEMM (`RouterGemm`, the span's GEMV batched 16 positions by 16 rows), the per-row select (`TopKRows`, the decode top-k's core `TopKCore` over each row, position-major picks and weights), the device bucket schedule (`MoeSched`, one workgroup: thread e counts expert e's slots, exclusive scans place the buckets and the tiles, the three planes' 4-word records and per-wg maps land in the host fill's layout at a fixed map offset, the map's tail past the real workgroup count carries `SCHED_NONE`, on which a cm2 tile workgroup reads a zero-row region and returns before its first barrier, so the dispatch is an upper bound and nothing is indirect; the slot map lands in slot order, the CPU walk's), the existing f16 gather + cm2 tiles + act over a whole window of bucket rows with 32 rows of tile slack, and the combine onto the FFN rows (`MoeCombineSh` over the shared expert's rows already there from the dense tail, scaled by the gate's sigmoid; the plain combine from zero on a layer without one); the token command gains the same block in decode form - the router GEMV and top-k over the driver's routing smalls, the top-k writing the three expert GEMVs' slot regions with the arena's slab-local bases, gate/up/act/down over k regions, the combine into ffnout - so a resident MoE takes neither the span nor the per-op rails, and the resident prefill fills the one mirror the token command reads; the MoE seats install separately (`install_moe_gpu_resident_moe`: `rdec_moe_ok`, the f16-fed cm2 admission per expert triple; `rdec_prepare_moe`; `rdec_set_moe_layer`), the plan names the router shapes the kernels do not serve (a non-softmax gate, a router or selection bias, biased or mx4 stacks, past 256 experts or 64 slots), an MoE takes the split add+rms arm (the router reads the normed row the fused twins never store), `DASLLAMA_GPU_RESIDENT=0` keeps the per-op rails for any model and `set_gpu_resident_route` is its in-process twin; the Qwen1.5-MoE-A2.7B Q4_K_M twin now reads pp512 5069.2 against 1435.9 (llama.cpp 5099.8, 0.99x) and tg128 142.2 against 134.0 (173.8, 0.82x), the whole model resident at 10107 MB with a 3069 MB mirror at ctx 16368; the prefill doc gains the routed block's section (2.2af), the decode doc the MoE token command's (2.2ag), the residency doc the MoE plan, the ledger's item 43 its status; tests: test_vulkan_kernels.das gains the four routing kernels against CPU oracles with a poison per bar (the router GEMM at a second layer's offset, the per-row top-k against moe_select_core renormalized and scaled, the schedule against its CPU twin word for word over the whole planes with empty experts and the sentinel tails, the gated combine over two accumulating windows), the new stocked file test_gpu_resident_moe.das holds the twin's forced feed against the all-CPU chain at one and two windows (0.15 of the max logit - the routed chain's noise class, measured 0.05-0.095 with the one-step-off control 3.7-9.1 logits off - and the census witnesses: the schedule and the per-row select once per MoE layer per window, the token command's top-k once per layer at its recording), test_gpu_serving_declines.das the MoE names and the layer helpers, and test_gpu_moe_shexp.das pins the driver off for its load so the per-op arm's claims stand Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… router GEMM is a 32 x 32 tile of positions by router rows per workgroup, each invocation a 2 x 2 block, K through shared in 64-wide steps (the 16 x 16 one-output tile stepping K by 32 was barrier-bound at 243 us per layer for 268 MFLOP; 142 now), and the bucket schedule's two slot walks stage the picks through workgroup memory in 256-slot chunks instead of reading every slot from global memory per thread (258 us per layer to 163); the Qwen3-30B-A3B UD-IQ2_XXS window 230.7 -> 221.2 ms, the row 2176.7 / 124.4 -> 2270.5 / 124.6 (llama.cpp 3520.0 / 116.6), the Qwen1.5-MoE twin 5069.2 -> 5195.2 (llama.cpp 5099.8), decode unchanged; the router tile takes a 64-multiple row width, which the MoE seats' reach now names; the plan doc carries both files' per-role windows and tokens and the reading of llama.cpp's coopmat2 mul_mat_id for the expert tiles, the window's remaining term; the kernel cells' router fixture widens to the tile's 64-step (dim 128) and dispatches by the 32-wide grid Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the routing retile, the reference engine's mul_mat_id figures at the 30B expert shape, and the reading of its coopmat2 mul_mat_id for slice 5 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…'s routed block registers after its deltanet head through its own seat (`rdec_set_moe_experts`, the seat the attention form now calls after its quad), the deltanet registration builds the layer with the shared expert as its dense triple (or the bare tail sets where a layer has none), and the window chain's tail already served both heads; the NextN name leaves the MoE decline list - the whole-model driver serves a dense hybrid's trunk with the block on the host, and an MoE's the same way; the Qwen3.6-35B-A3B UD-IQ2_XXS (10.76 GB, the fit the UD-IQ3_S files at 13.7 GB cannot make on the 16 GB card) rides whole at 11110 MB with a 1992 MB mirror at ctx 102011: pp512 2252.3 / tg128 95.8 against llama.cpp's 2853.1 / 71.6 (0.79x / 1.34x), the per-op tier's 638.3 / 41.1 before the arc; the decode doc's MoE section carries the recurrent form; tests: test_gpu_resident_moe.das takes its fixture as a parameter and gains the hybrid at one and two windows (the conv tail and the state across the seam, the same bar and controls) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s by a shift: every `unpack8(lane)[i & 1]` byte2 select of the iq2xxs, iq3xxs, iq3s and iq2s decodes (the scalar callback and the four-wide twin alike) reads its lane and shifts instead, and the IQ2_XXS / IQ3_XXS sign index comes out of the sub-block's aux32 word assembled from its two lanes rather than from two selected bytes - the select reads the same lane, but a decode built on selects ran 1.1x to 1.5x slower than the shift form; measured on a new probe arm over the Qwen3-30B-A3B expert schedule (`harness/vk_gemm_probe.das -- moe:<fmt>`: 128 buckets of 32 rows, the gate/up and down shapes, dispatched to the device schedule's bound with its sentinel tail, both decode arms interleaved; `moesk:<fmt>` the skewed 1/(rank+8) profile beside the split m+s schedule, the next slice's research row), per gate/up plane on the RTX 5060 Ti: iq2xxs 955 -> 749 us, iq3xxs 728 -> 585, iq3s 1141 -> 767, iq2s 830 -> 772, against llama.cpp's coopmat2 `mul_mat_id` tile at 754 / 788 / 870 / 797 on the same uniform buckets (its coopmat2 branch carries no integer-dot expert tile, so its harness figures are all cm2), while iq2xs 664, iq4xs 632, k4 728 and q8 892 already sat under its 744 / 959 / 1009 / 998; the 30B UD-IQ2_XXS window 221.2 -> 187.7 ms (the expert tiles 166 -> 133 ms), pp512 2270.5 -> 2670.7 (llama.cpp 3520.0: 0.65x -> 0.76x), tg128 124.6 unchanged; the Qwen3.6-35B UD-IQ2_XXS 2252.3 -> 2542.7 (2853.1: 0.79x -> 0.89x), tg128 95.3; the Qwen1.5-MoE twin (k4) 5208.4 / 143.1, unchanged; the GEMM doc's decode section carries the rule with the figures, the Vulkan doc's MoE block section the retiled router and the staged slot walks it had lost, the ledger's twin item its status; the kernel file 110 of 110 with every rewritten tile at 0 of 89600 off its oracle Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dow's real buckets: a real router is skewed (on the Qwen3-30B-A3B window 69 of 128 experts route, nine hold over 128 rows, the largest 467, and the s column alone ran 175 tiles where a 32/128 ladder runs 87 - the profile's new bucket report, `DASLLAMA_GPU_PROF=1`, reads the last MoE layer's picks back and prints both counts), so the device schedule now cuts a bucket past the s column into m columns with the last one partial, unless the remainder past whole columns fits the s column, which then takes it (`sched_ladder_m_rows`), and writes both piece lists in one pass - the s pieces' records at [0, ne), the m pieces' at [ne, 2 ne), each dispatch's map at its own offset, the two tile counts scanned as one packed word, four sentinel tails - for two dispatches per expert plane, the m column's first, under separate hazard bits (`VHZ_GATE_M`, `VHZ_UP_M`, `VHZ_MDN_M`) so the two co-run and the reader's barrier covers both; the m stamps take the s tile's partial-column fast path (`STILE` generalized to the stamp's column: the B layout's rows rounded up to it, the store clamped through `tensorLayout2DPad`), and every f16 plane the tiles read carries 128 rows of slack (`TILE_READ_SLACK`, the kernel cells' fixtures); the first cut - whole 128-row chunks on m, the remainders on s - gained 2% on a 1 / (rank + 8) profile because the mid-sized buckets stayed on the s tile, which is what the ladder fixes: on the probe's real-shape profile (`harness/vk_gemm_probe.das -- moesk:iq2xxs`, both forms beside each other) the gate/up plane 842 -> 593 us, the down plane 915 -> 650; the 30B UD-IQ2_XXS window 187.7 -> 153.9 ms (the expert tiles 133 -> 97.7; llama.cpp's window 142.3), pp512 2670.7 -> 3242.0 (3520.0: 0.76x -> 0.92x), tg128 123.6; the Qwen3.6-35B UD-IQ2_XXS 2542.7 -> 2837.7 (2853.1: 0.89x -> 0.99x), tg128 95.9; the Qwen1.5-MoE twin's window 99.2 -> 94.5 ms, its row 5152.9 / 142.5 (the first measured rep after the warmup reads 101 ms on every model, a driver warm-up the bench's one warmup does not absorb); the resident driver's own tile pick for the MoE block (`moe_tc`) is gone, the per-op tier keeps `cm2_tile_cols`; the Vulkan doc's MoE block section carries the ladder and the report, the GEMM doc's tile-pick section the partial-column path on both columns; the schedule cell's CPU twin follows the two lists over a whole m column with an s remainder, partial m columns, s-column buckets and empty experts, with the m map's last sentinel as its poison; the kernel file 110 of 110 (every m tile 0 of 89600 off its oracle on partial columns), the resident MoE file 6 of 6 on both fixtures, the dense hybrid file 15 of 15 and the per-op MoE file 3 of 3 on the same m tile Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d off the reference engine's own per-op tables (`GGML_VK_PERF_LOGGER=1`): the twin's shared expert was held as a q8 transcode of the file's Q4_K / Q6_K, twice the bytes its three GEMVs read per token, so the loader now keeps the shared expert's K-quant planes beside the transcode the CPU and Metal chains read (`wsh*_fmt` tags, `wshk*_offs` planes, the kq repack list, `IMAGE_VERSION` 36) and the whole-model driver plans, tallies and places those - the twin's image 10107 -> 9756 MB, its token 7.06 -> 6.12 ms (the shared GEMVs 2.39 -> 1.48 ms, the experts' own requant gone since they share the feed), tg128 142.5 -> 162.7 (llama.cpp 173.8: 0.82x -> 0.94x), pp512 5172.7; on the 30B window the reference beats us on nothing but the router, the schedule and the gate/up tiles, so the schedule's two per-expert slot walks (thread e scanning every slot twice) became an atomic tally and an atomic cursor over workgroup memory (8.1 -> 0.46 ms per window; the rows within a bucket land in an order nothing downstream reads - the tiles compute rows apart and the combine reads each slot's row through the map - and the kernel cell checks the map as a permutation of each bucket's rows) and the router tile is 64 x 32 with 4 x 2 blocks whose next K step's rows are fetched into registers as float4 while the current step computes (6.6 -> 4.3 ms); the 30B window 153.9 -> 144.9 ms against the reference's 142.3, pp512 3448.9 (3520.0: 0.98x), tg128 124.6 (1.07x); with all three the twin reads 5348.3 / 163.4 (5099.8 / 173.8: 1.05x / 0.94x) and the Qwen3.6-35B UD-IQ2_XXS 2946.0 / 95.2 (2853.1 / 71.6: 1.03x / 1.33x); the docs carry the three: the residency doc's MoE plan, the decode doc's feed paragraph, the Vulkan doc's MoE block section, the tests guide's routing cell; the kernel file 110 of 110, the resident MoE file 6 of 6 on both fixtures, the per-op MoE file 3 of 3, the model-free suite (74 files) and the image suite's mechanics and smol arms green, and the twin through the image rail - a mint run, then a mapped run of that image - at the same 9756 MB resident image Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t, in both chains: the add+rms after the routed block (`ClsArComb`, its f16 twin `ClsArCombF16B` where the next layer's head takes the f16 feed) adds the shared expert's down rows at the sigmoid of the gate logit and the k weighted expert rows through the slot map straight into the residual, then norms the row for the next layer - the residual step read those rows anyway, so the window and the token command each lose one dispatch per MoE layer, and `MoeCombineSh` goes; the fused step sums the FFN row first (the gated shared row, then the slots in order) and adds it to the residual, the order the two kernels took, because the natural order moved the rounding enough to flip a router near-tie downstream (one step of the Qwen3.6-35B two-window cell read 1.50 logits off the CPU chain against its 1.39 bar where the chain's order reads 0.39063567, the pre-fold value to the digit); the rows on the RTX 5060 Ti against llama.cpp b357x: Qwen3-30B-A3B UD-IQ2_XXS 3455.7 / 127.7 (3520.0 / 116.6: 0.98x / 1.10x, the window 143.7 ms against 142.3), Qwen3.6-35B UD-IQ2_XXS 2976.1 / 99.9 (2853.1 / 71.6: 1.04x / 1.40x), the Qwen1.5-MoE twin 5461.8 / 165.4 (5099.8 / 173.8: 1.07x / 0.95x; its token's ar2 360 us where ar2 + comb read 199 + 233); the cm2 tile's fast path picks its store layout by the column, not by the stamp: a whole column stores through the clamp-Undefined layout on every stamp and only the partial column an s / m stamp loads unclamped stores through the clamp-Constant one, so the per-stamp `FLO` typedef (fifty-five lines over the tile stamps and the toy pair) goes - the clamp on a whole column measured free (48.0 against 48.1 TFLOP/s on the k4 m tile, `harness/vk_gemm_probe.das -- cm2:k4`), so the change is the layout's meaning and the deletion, not a rate; the Vulkan doc's MoE block section and the decode doc's token section carry the fold and its sum order, the GEMM doc's store rule the column form, the ledger's item 43 the rows and what is still open; tests: the routing cell's combine arm becomes the combine-folded residual step against one CPU oracle (the gated and ungated shared expert, no shared expert; the residual rows, the f32 normed rows and the f16 twin within the half's rounding, a poison per bar), the kernel file 55 of 55, the resident MoE file green on the twin and the 35B at both lengths Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ows as float4 at a stride of 17 with each lane's two rows 16 apart - the scalar stage at a 68-float stride put four of every sixteen lanes of a weight read on one bank and the tile ran at about 17 FMAs per cycle per SM - and adds a float4's four products in k order, the scalar loop's sums to the bit: 4.29 -> 2.43 ms per Qwen3-30B-A3B window (the reference's 1.2); the combine-folded residual step loads eight slots' rows together, then four, then one at a time, the sums still in slot order - the token command's one-row form is latency, and the plain slot loop read 360 us per Qwen1.5-MoE token where the groups read about 290, while a four-group alone read 490 on the 30B's eight slots against the plain loop's 437 (the compiler's unroll served eight and left four to a scalar tail; the eight-then-four form reads 440); a slot-major pass through the row stash was the wrong shape (+2.4 ms on the 30B window, its tg128 127.7 -> 121.7); the FFN-norm requant of a MoE layer with no shared expert is skipped, the dense triple being its only reader and the gather taking the f32 rows (540 us per 30B window that nothing read); the rows on the RTX 5060 Ti against llama.cpp b357x: the 30B 3482.6 / 126.2 (3520.0 / 116.6: 0.99x / 1.08x, the window 143.4 ms against 142.3), the Qwen3.6-35B 2962.8 / 99.0 (2853.1 / 71.6: 1.04x / 1.38x), the twin 5395.3 / 163.9 (5099.8 / 173.8: 1.06x / 0.94x; its token 6.05 ms on the device in every profile, the bench's tg wandering 163-169 across the day on the same kernels); the Vulkan doc's MoE block section carries the router's bank rule, the slot groups and the requant skip, the decode doc's token section the one-row step's figures, the ledger's item 43 the rows and what is still open; tests: the resident MoE file gains the 30B as its third fixture - the MoE with no shared expert, the residual step's add partner off - at one and two windows, and every fixture's per-step differences read the same digits through all three levers (the 35B two-window step 4 at 0.39063567 on every run of the pass); the kernel file 55 of 55 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…p by the row length: the family put a whole subgroup on one row, a lane per 32-block, so a MoE's expert rows (K 512 to 1408, 16 to 44 blocks) left most lanes idle and the 30B's expert GEMVs ran at 40-54% of the card's bandwidth where the dense rows sat at 75-100%; `gemv_lanes_per_row` picks 8 lanes to 24 blocks, 8 for the grid formats and 16 for the k-lattice to 48, 16 to 96, and past that the whole subgroup for the k-lattice and 16 for the grid formats, the push block carries the lanes (0 = one row, the q8 GEMV's one form), the fold is `subgroupClusteredAdd` with every lane reducing and a dead row at zero so the clusters stay whole, and `gemv_enc` sizes the grid to match for every decode site (the resident token command's dense and expert GEMVs, the deltanet head's, the classifier head's, the per-op tier's expert and shared-expert chains); the rule is `harness/vk_gemv_probe.das`'s new lanes sweep (`<n> <d>`, three splits per format, DRAM-bound planes) over K 512 / 768 / 1408 / 2048 / 2560 / 4096 / 5632 on the RTX 5060 Ti - at K 768 iq2s 148 -> 337 GB/s, iq2xxs 213 -> 376, k4 403 -> 414; at K 512 iq2s 100 -> 297, iq2xxs 142 -> 351; at K 1408 k6 386 -> 399, iq2s 198 -> 368; k6 loses 4% on 16 lanes past 96 blocks, which is why the k-lattice keeps the whole subgroup there - and llama.cpp's mat-vec has the same bytes in flight by the other axis (K over 16 threads, two to four rows per thread); the rows against llama.cpp b357x, tg128: Qwen3-30B-A3B UD-IQ2_XXS 126.2 -> 132.0 (116.6: 1.13x; its token 7.17 -> 6.79 ms, e_down 1085 -> 806 us - a token's expert dispatch is 3-4 MB, where launch and ramp cost what the transfer does, so the steady rates do not arrive whole), Qwen3.6-35B UD-IQ2_XXS 99.0 -> 107.1 (71.6: 1.50x), the Qwen1.5-MoE twin 163.9 -> 166.8 (173.8: 0.96x), the dense Qwen3-4B Q4_K_M a wash (its token 8032 against 8041 us on the one-row form at K 2560); the pp rows do not dispatch the family; the GEMM doc gains sec.2.2ah with the rule and the sweep, the ledger's item 43 the rows and the dispatch-size reading; tests: the kernel file's GEMV family cell runs the 13 formats at all three splits against the oracle (55 of 55), and the lane-to-block map sets a row's summation order, so every resident-vs-CPU step moved inside its class - the resident MoE file's three fixtures, the hybrid file's five twins, the qwen2 bias file and the per-op shexp file all green Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ments the branch added, 37 restated a section a citation already names and go (the MoE block's, the token command's, the residency plan's, the decode span's, the GEMV lane split's), the site contracts are spelled `//!` one-liners, two facts land where they belong - the residual step has no Q8 requant leaf (a ninth binding, past the hazard rail's eight), stated in the Vulkan doc's MoE block section, and the GEMM probe's `moe:` / `moesk:` arms join `ARCHITECTURE_MEASUREMENT.md`'s probe axes with `run_moe_fmt_shape` citing them - and two names carry what a comment did: `GemvArgs.lpr` is `lanes_per_row`, and the span's `she`, an offset where every sibling `she` is a stride, is `sh_off`; the router stage's `float4[1088]` / `[544]` are `int(ROUTER_TP * ROUTER_LD)` / `int(ROUTER_TE * ROUTER_LD)`, and the kernel file's sixteen `rows + 128` fixtures read `TILE_READ_SLACK`; `REVIEW_GPU_VULKAN.md`'s stamp-count rule gains the third role table the branch added (`rd_moe_tail_names`, `g_rdq_moe`) - an MoE stamp added or dropped passed the rule as written Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…HITECTURE_GPU_VULKAN_MOE.md` carries sections 2.2af (the MoE block of the prefill window) and 2.2ag (the whole-model driver's MoE token command) whole, with their anchors and numbers, moved out of `ARCHITECTURE_GPU_VULKAN.md` and `ARCHITECTURE_GPU_VULKAN_DECODE.md`, which the arc had grown past the 300-line cap (the decode companion read 320); every citer follows - the fifteen `[arch]` annotations on the routing kernels, the residual step's classes and the two chains' MoE seats, and the section cross-references in the sibling companions, the ledger and the routing heads of `ARCHITECTURE.md`, `ARCHITECTURE_GPU.md` and both source documents Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…K-quant planes moved the layout closure, and the gate's two constants take the version and the hash it printed Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…by an xor-shuffle butterfly at 8 and 16 lanes per row and by the subgroup add at the whole subgroup (the clustered add read a subgroup feature the tier never checks; `harness/vk_gemv_probe.das` the same GB/s within noise), the decode profile's sampler counts a recurrent MoE layer as the deltanet head's six stamps plus its MoE tail by name (the 35B's per-role lines aggregated where they read "roles unaggregated") and clears the MoE tail table with the other accumulators on a model drop, and the prefill profile counts a recurrent MoE layer's routed stamps too and prints them on their own `dnmoe` line (the 35B's window read "roles unaggregated" by exactly its thirty recurrent MoE layers' eight stamps); the residual step's push block derives the gate logit's index from the row stride (a MoE layer's gate logit is the row's last) and names its slots; the decode span's record keys on the routing shape it depends on (experts, slots, the shared expert's width and gate) beside the plane offset and the layer count; the router plane checks the device's storage range at its sizing site; the prefill's shared expert logs why it stays on the CPU rail once per reason per model (a dense shared expert, planes not resident, a prefill under 32 positions, the route lever); the whole-model driver names the route lever by its spelling and says when the tier installed no MoE seats; three input-scaled buffers reserve before they resize; `TILE_READ_SLACK` is one public constant in `dasllama_gpu_tier.das` the residency plan (which under-counted the two f16 planes by 96 rows), the prefill's planes and the GEMM probe all read; the engine's names after the style audit (`router_slot`, `has_dense_triple`, `row_lane` / `lanes_per_row`, `ROUTER_K4` / `ROUTER_TH`, `slots`, `b_over`, one `MOE_MAX_EXPERTS`, per-field `//!` tails); the kernel cells: every routing output under a sentinel fill before its dispatch, the residual step proven to move off its input and its f32 and f16 normed-row bars each red a poison (the f16 bar's derivation stated), the router GEMM at nine rows (fewer than the tile's half), the schedule oracle spelling the tile ladder out instead of calling the shipped rule and taking its thresholds from `SCHED_S_ROWS` / `SCHED_M_ROWS`, the k4 cm2 cell dispatching two workgroups past its schedule over sentinel map words on all four arms, the routing cell skipping (not feinting) without a device, and a new cell pinning `gemv_lanes_per_row` per format class and row length; the decode-span shared-expert cell runs the ungated arm after a span reset; the resident MoE file asserts the driver's admission on a cm2 device (skipping only without the tile family), pins the route lever for its load and takes a fixture struct, the shared-expert file asserts the lever kept the driver off, the declines cell names the expert-stack bias, the clamped activation and the layer output scale, the Vulkan serving census gains the Q4_K_M twin (the routing classes' only census reach on a 16 GB card) and arms the device before its coopmat mode sweep (a family filter that skipped every earlier row read a null device state), the resident MoE file's logits bar reads 0.20 of the step's max logit with the noise class it measured (a router near-tie on the 35B at two windows flips under any change of the device's summation order and reads 0.155-0.16; the one-step-off controls 0.42 and above), and `performance/model_specs.das` gains the two UD-IQ2_XXS fixtures at their pinned upstream revisions with bytes and sha256; the GEMM probe's `moe:` / `moesk:` arm alternates two output planes under fresh hazards per dispatch (one plane serialized the chain on its write-after-write), reads the engine's schedule constants and ladder, sums its skew residual and names its forms by the enum's words; the documents: the GEMM companion's sec.2.2ah carries the butterfly, iq3s's k-lattice split and the probe's box, the MoE companion names the profile behind every figure and the packed tile count's bound, the Vulkan companion's slack figure reads 128, the GPU role table's tier row lists the resident MoE seats and the MTP range reads 2.28-2.39, every routing head names sec.2.2ah, the image range reads 2.1n, `PERF_LEDGER.md` records the shared expert's K-quant planes (device image 10107 -> 9756 MB, token 7.06 -> 6.12 ms, the planar image +537 MB until the CPU chain reads them), `HOW_TO_ADD_A_FORMAT.md` gains the lane-rule step, the plan and ledger item 43 name their instruments and boxes and the reference build's op rows; the checklists after their self-review: `REVIEW_GPU_VULKAN.md` bans the `unpack8` select its GEMM section measured slower, defines the Vulkan-tier gate, widens the drop-sweep and the KHR rule's trigger and escape; `REVIEW_GPU.md`'s weight-role rule reads the property (never written after arming at every site that binds it), the tier-SPI carve carries its duty, the census rule states its outcome, the cache-key rule leads with its criterion; `REVIEW.md` defines a model dimension and lists the runtime companion; `REVIEW_PLACEMENT.md` splits host-side conversion from kernel-side decode and calls a test's oracle a witness; `REVIEW_GPU_RACE.md`'s race and window rules read their properties; `tests/REVIEW.md`'s census rule reads the property and its two cells rules are one; `tests/REVIEW_KERNEL_CELLS.md`'s precision and kargs triggers widen; `harness/REVIEW.md`'s alternate rule reads the property; `tests/CLAUDE.md`'s census follows every changed cell Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pert tagging pass (the closure moved; no byte moved, the version stays 36), and the MoE arc's plan document leaves the tree with the rest of the plans folder - the arc's rows and readings live in the ledger's item 43, the MoE companion and the PR Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tp_force_reject_every(n)` rejects every n-th draft the greedy walk verifies whatever the verify said (the sampled walk is untouched), so a fixture that accepts every draft still walks the arm; the GLM-4.5-Air MTP cell - the one non-recurrent MTP model, the only reach of the step's shortcut reject (the verify's row 0 stands, no re-forward) - read 0 of 20 drafts rejected on its own once the decode GEMV's lane split moved a draft near-tie, so it now runs its code fixture plain (its own reject count in the assert text) and again with every third draft rejected, both streams token-for-token against plain decode and the forced run's rejects at a third of its drafts or more; the MTP companion's sec.2.28 names the CPU step's shortcut reject and the seam, the test census gains the file's entry; test_mtp.das 10 of 10 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ons stack = 524288`, the deep-engine test rule of skills/internal/tests_in_repo.md, which cites this very file as the structural example): its gpt-oss-20b cell - the file's deepest forward, mx4 experts with sinks and biases - tripped "stack overflow" at parity_gen_ok's entry on the RTX 5060 Ti box, the cell's first run there (every earlier stocked pass on that box skipped it on the model gate); the same cell overflowed with the GPU tier off and on master's own module sources, while `benchmarks/lcpp_bench.das` runs the file on the CPU green, so the budget, not the engine, was the difference; the cell now passes on the box Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…PU tier armed, the deltanet planes' pattern: the whole-model resident driver is their one reader, so a CPU-only load keeps the q8 transcode alone and a CPU box's image is what it was before the arc, while a tier-armed load carries both copies until the per-op and CPU rails read the K-quant planes too (the Qwen1.5-MoE twin serves 11291 MB of weights CPU-only against 11781 tier-armed, the 490 MB; the resident image stays 9756 MB, the two stocked files 3 of 3 and 9 of 9, the vulkan bake round trip 4 of 4); `REVIEW.das` gains the Vulkan checklist's four mechanical checks - `khr_stage16` stays abstract on the cm2 template, the add+rms row slab triple (`AR_MAX_DIM`, `ArBase.row`, the servability gate's dim cap) agrees, every superblock format on the cm2 template ships its KHR class, its dispatch stamp and its arm in each of the prefill's three KHR ladders (q8 the one named exemption: no superblock format, its cm2 tiles carry no KHR arm), and no `vkCreateComputePipelines` outside the `[vk_dispatch]` lens in the engine, harness and test folders - each red on a planted defect (a body on the stage, 4096 against 8192, the k4 arm dropped from `khr_cls_ensure`, a hand call in the engine) and green on the tree, the four rules naming their check; ledger item 43's open list gains the two probe arms the checklist's race rule asks for (the router tile's float4 stage against its scalar stage, the residual step's slot groups against the plain loop), ruled to ship as stated claims Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e module gate's four Vulkan checks - what it reads and the names it licenses (`Q8Cm2T` alone for the KHR set: no superblock format, its cm2 tiles carry no KHR arm; the two llama.cpp shader ports under performance/ outside the pipeline walk as reference measurements of another engine's kernels) - the line the module checklist asks of a new `REVIEW.das` check; the Vulkan checklist's KHR-set rule reads over `kq_sb` formats, the set the check walks, its two collapsed rules wrap at the file's width Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
A newly added test leaves a global resident-route lever forced on after load, which can affect later tests and reduce suite isolation.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the dasLLAMA Vulkan GPU tier to serve Mixture-of-Experts (MoE) models via the whole-model resident driver by default when the model fits VRAM, moving routing/top‑k/scheduling and the routed FFN block onto the device for both prefill and decode. It also adds/extends tests, documentation, performance harness probes, and module-level mechanical review gates to cover the new MoE residency path and related Vulkan invariants.
Changes:
- Enable whole-model resident serving for fitting MoE models (new routed device block for prefill and token decode), plus associated residency planning and shared-expert K‑quant handling.
- Add/extend Vulkan kernel/unit + stocked model tests (resident MoE, shared-expert span, decline reasons, mode sweeps, MTP seam coverage).
- Add Vulkan mechanical checks to
modules/dasLLAMA/REVIEW.dasand update architecture/docs/ledgers/probes accordingly.
File summaries
| File | Description |
|---|---|
| modules/dasLLAMA/tests/test_vulkan_moe_cm2.das | Extends cm2 MoE expert-chain kernel test with a “one-region” shared-expert call shape. |
| modules/dasLLAMA/tests/test_vulkan_dec_tail.das | Adds decode-span shared-expert (gated/ungated) coverage and span reset dependency. |
| modules/dasLLAMA/tests/test_parity.das | Increases stack budget for deep forward path in parity suite. |
| modules/dasLLAMA/tests/test_mtp.das | Adds forced-reject seam usage to cover depth-1 reject branch deterministically. |
| modules/dasLLAMA/tests/test_kernel_coverage.das | Refactors coopmat mode sweep into a helper and adds resident-MoE fixture to census. |
| modules/dasLLAMA/tests/test_gpu_serving_declines.das | Expands resident-driver decline coverage for MoE shapes/features and messaging. |
| modules/dasLLAMA/tests/test_gpu_resident_moe.das | New stocked test validating resident MoE correctness vs CPU chain + kernel census witnesses. |
| modules/dasLLAMA/tests/test_gpu_moe_shexp.das | New stocked test validating shared-expert prefill-on-GPU route lever vs CPU form. |
| modules/dasLLAMA/tests/run.das | Registers new GPU MoE tests in suites/area lists. |
| modules/dasLLAMA/tests/REVIEW.md | Tightens checklist language around kernel-unit test obligations and skip discipline. |
| modules/dasLLAMA/tests/REVIEW_KERNEL_CELLS.md | Generalizes “new kargs field” rule to “new selectable branch” (field or sentinel). |
| modules/dasLLAMA/tests/CLAUDE.md | Updates test census documentation for new/expanded Vulkan + GPU-resident MoE coverage. |
| modules/dasLLAMA/tests/_vkd_toy.das | Removes unused FLO typedefs from toy cm2 classes. |
| modules/dasLLAMA/REVIEW.md | Updates module checklist docs list and clarifies resize+reserve rule wording. |
| modules/dasLLAMA/REVIEW.das | Updates image layout stamp and adds four Vulkan mechanical checks (KHR stage abstract, slab triple, KHR set completeness, no hand pipelines). |
| modules/dasLLAMA/REVIEW_PLACEMENT.md | Clarifies placement rules (host vs kernel-side conversion; shared helpers vs witnesses; init require rule). |
| modules/dasLLAMA/REVIEW_GPU.md | Clarifies GPU binding “weight role” rule and Vulkan census expectations; tightens cache-key guidance. |
| modules/dasLLAMA/REVIEW_GPU_VULKAN.md | Updates Vulkan checklist to align with new REVIEW.das checks and new decode/GEMM guidance. |
| modules/dasLLAMA/REVIEW_GPU_RACE.md | Generalizes timing-arm guidance and clarifies hand-binding definition. |
| modules/dasLLAMA/performance/profile_common.das | Adds DASLLAMA_GPU_RESIDENT to recorded benchmark environment variables. |
| modules/dasLLAMA/performance/model_specs.das | Adds model spec entries for new resident-MoE/shared-expert test fixtures. |
| modules/dasLLAMA/PERF_LEDGER.md | Records landed shared-expert K‑quant plane minting/resident-driver usage results. |
| modules/dasLLAMA/HOW_TO_ADD_A_FORMAT.md | Documents new decode GEMV lane-split rule and the probe process to decide it. |
| modules/dasLLAMA/harness/vk_gemv_probe.das | Adds lanes_per_row axis and prints split timings; uses gemv_wgs. |
| modules/dasLLAMA/harness/REVIEW.md | Tightens probe-arm requirement to include the alternate it’s ranked against. |
| modules/dasLLAMA/followup_vulkan.md | Updates Vulkan followup ledger notes to reflect lane-shift decode work and MoE residency landing. |
| modules/dasLLAMA/ENVIRONMENT.md | Documents DASLLAMA_GPU_RESIDENT behavior change (default on, 0 keeps per-op rails). |
| modules/dasLLAMA/dasllama/dasllama_vulkan_common.das | Extends Vulkan resident state for MoE/span shared expert, new hazards, and drop-state resets. |
| modules/dasLLAMA/dasllama/dasllama_math_vulkan.das | Installs resident-driver MoE seats into the GPU tier SPI. |
| modules/dasLLAMA/dasllama/dasllama_load.das | Adds shared-expert K‑quant tagging, offset allocation, and conditional load alongside q8 transcode. |
| modules/dasLLAMA/dasllama/dasllama_layout.das | Includes shared-expert K‑quant planes in repack region collection. |
| modules/dasLLAMA/dasllama/dasllama_image.das | Bumps IMAGE_VERSION to reflect new shared-expert K‑quant plane storage. |
| modules/dasLLAMA/dasllama/dasllama_gpu_tier.das | Adds resident-driver MoE seats, resident-route lever, tile read slack, and shared-expert decline logging. |
| modules/dasLLAMA/dasllama/dasllama_env.das | Adds gpu_resident env knob (DASLLAMA_GPU_RESIDENT) defaulting to on. |
| modules/dasLLAMA/dasllama/dasllama_common.das | Adds shared-expert K‑quant metadata and MTP forced-reject seam. |
| modules/dasLLAMA/dasllama/dasllama_blocks.das | Extends decode span to support shared expert; adds span token counter and shared-expert prefill-on-GPU accounting. |
| modules/dasLLAMA/ARCHITECTURE.md | Updates architecture index to route MoE Vulkan resident content to a new companion doc. |
| modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md | Documents new moe:/moesk: probe axes for expert schedule measurement. |
| modules/dasLLAMA/ARCHITECTURE_GPU.md | Updates GPU role table narrative and companion routing for Vulkan resident MoE. |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN.md | Routes MoE block sections to new MoE companion and documents REVIEW.das Vulkan checks. |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_RESIDENCY.md | Updates residency plan to account for MoE routed block + shared-expert K‑quant planes. |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_MOE.md | New document: specifies resident prefill MoE block and token-command routed block mechanisms. |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_GEMM.md | Updates decode-byte extraction rule and adds decode GEMV lane-split section (2.2ah). |
| modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_DECODE.md | Updates decode-era doc to include shared-expert-in-span behavior and MoE routing references. |
| modules/dasLLAMA/ARCHITECTURE_GPU_MTP.md | Documents reject-branch mechanics + the new forced-reject seam. |
Review details
- Files reviewed: 51/51 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…he module gate's four Vulkan checks read - the abstract KHR stage on the cm2 template, the add+rms row slab and its constant, one superblock format template with its KHR class, dispatch stamp and three ladder arms, the q8 template the KHR set licenses, and the servability gate's dim cap - so the covered fixture stays green, and its planted half gains one violation per check with the finding's needle: the stage given a body, a second superblock template with no KHR class or arms, the constant off its slab, and a hand-built pipeline in the harness folder (the CI lane's `Test review-md discovery` step reds the four checks on the old fixture, whose anchors they could not find) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The new check_no_hand_pipelines mechanical gate can be bypassed by whitespace-spelled vkCreateComputePipelines calls and should match the function name independent of spacing.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
modules/dasLLAMA/REVIEW.das:900
check_no_hand_pipelinesonly matches the exact substringvkCreateComputePipelines(, so a call spelled with whitespace (e.g.vkCreateComputePipelines () would bypass the gate. Since this is meant to be a mechanical check, it should match the function name regardless of spacing.
- Files reviewed: 52/52 changed files
- Comments generated: 0 new
- Review effort level: Lite
Contributor
|
Ran this branch on Linux against the fixture the PR reports on - The routed block works, and on this card it clears the PR's figures. The resident driver
One blocker, one note, one rebase. 1. On Linux the default path never reaches the driver, and gives up 2.5x for itSame build, same box, same command; the only difference is
What makes this worth raising rather than filing as a constant to tune: the cap-context path Three things would each help independently:
2. Note: the new shared-expert gate reads a CPU kernel flag
Not a blocker - a repacking backend clears it, which is how the numbers above were taken. But 3. The branch needs a rebase to build on GCC 15+Neither |
Collaborator
Author
|
Thank you for running it on a Linux box - that is the first Linux reading of the routed block, and the 5080 row (3659.95 / 144.91 armed) goes into the PR body beside this box's. 3. Rebase - done; the branch now sits on master past 2. The shared-expert gate and 1. The headroom without an OS answer - agreed on all three points. |
Collaborator
Author
|
The follow-up is #3990: the platform-aware headroom (your item 1; the 5080 pod arms the 35B unpinned at ctx 44159, 3361 / 140), item 2 as ledger item 44, and the scaling pass against your numbers as item 45 - the four-wide decode twin is off on the Linux 580 driver (no VK_NV_cooperative_matrix_decode_vector), which is where a third of the prefill went. The rest of the reading is in the PR body. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Behavior change: a MoE that fits the card whole now serves on the whole-model resident driver by default (
DASLLAMA_GPU=1);DASLLAMA_GPU_RESIDENT=0orset_gpu_resident_route(false)keeps the per-op rails.Why. The per-op tier left a fitting MoE's window and token on the host between layers: the Qwen1.5-MoE twin read pp512 1435.9 and tg128 134.0 against llama.cpp's 5099.8 / 173.8, and the two Qwen MoE files that fit a 16 GB card at UD-IQ2_XXS ran the CPU routing per layer.
What changes.
unpack8select.modules/dasLLAMA/REVIEW.dasgains four mechanical checks from the Vulkan checklist: the cm2 template's KHR stage stays abstract, the add+rms row slab triple agrees (AR_MAX_DIM,ArBase.row, the servability gate's dim cap), every superblock format on the cm2 template ships its KHR class, dispatch stamp and three ladder arms (q8 the one named exemption), and novkCreateComputePipelinesoutside the[vk_dispatch]lens in the engine, harness and test folders. Each is red on a planted defect and green on the tree; the four rules name their check, the two the gate enforces whole (the abstract stage, the slab triple) collapse to the contract's weakening-the-check form, and the Vulkan companion's opening carries each check's charter line with the names it licenses (Q8Cm2Tfor the KHR set; the two llama.cpp shader ports underperformance/outside the pipeline walk).Observable behavior.
DASLLAMA_GPU_PROF=1prints the MoE tail's per-role stamps by name and the last MoE layer's bucket sizes.Where to look.
dasllama_gpu_resident.das(the plan and the placement), the routing kernels indasllama_vulkan_classes.das(RouterGemm,TopKRows,MoeSched,ArCombBase),pf_moe_blockindasllama_vulkan_prefill.dasandrd_encode_moeindasllama_vulkan_decode.das; the risky spot is the device bucket schedule's map offsets and sentinel tails, held word for word to a CPU twin intests/test_vulkan_kernels.das.Validation, claims, ledger
Validation
Rates:
benchmarks/lcpp_bench.daswith-jit --for-debug-purposes -r 3 -p 512 -n 128 -t 16underDASLLAMA_GPU=1 DASLLAMA_IMAGE=0 DASLLAMA_ALLOW_UNTUNED=1on the RTX 5060 Ti; llama.cpp b10660 (build-vulkan-357)llama-bench -ngl 99 -fa 1 -t 16 -r 3on the same box. Debug-jit readings, not board cells; the board's Vulkan rows are owed (see Not done).Kernel file
tests/test_vulkan_kernels.das112/112 (every cm2 format's l/m/s and KHR arms, the routing cells, the lane-rule pin);tests/test_vulkan_dec_tail.das5/5,tests/test_vulkan_moe_cm2.das8/8,tests/test_gpu_serving_declines.das17/17,tests/test_model_specs.das12/12.Stocked model files under
DASLLAMA_PARITY_FULL=1 DASLLAMA_GPU=1:test_gpu_resident_moe.das9/9 (the twin, the 35B hybrid, the 30B, each at one and two windows),test_gpu_resident_hybrid.das15/15,test_gpu_resident_qwen2.das3/3,test_gpu_moe_shexp.das3/3. The resident fixtures' logs carryresident driver armedand noresident override passed a call; the mirror codec was the f16 default (the f32 armDASLLAMA_VK_KV32=1did not run). The q8 models the changed GEMV path serves are the hybrid file's Qwen3.5-0.8B-Q8_0 and the qwen2 file's Qwen2.5-0.5B-Q8_0 (forced-feed logits parity on the resident driver); the kq models are the three MoE fixtures.cm2:<fmt>probe rows for the 13 kq cm2 formats (both decode arms in one process, two rounds), since the sharedcm2_tile/runchanged (STILE, the zero-row early return): the four-wide twin is faster than the scalar arm on every format, shape and tile (gate / down / q-wo at the l and m tiles; e.g. k4 gate l 0.524 vs 0.720 ms, iq2s down m 0.592 vs 1.118 ms), so noDECV4/DECVECfix is owed.moe:/moesk:rows retaken on the two-output-plane arm (per gate/up plane, uniform 32-row buckets): iq2xxs 724 us, iq3xxs 569, iq3s 766, iq2s 736, iq2xs 651, iq4xs 632, k4 746, q8 885 (llama.cpp's cm2mul_mat_idtile 754 / 788 / 870 / 797 / 744 / 959 / 1009 / 998); the skewed profile's ladder m+s 563 us against the whole s column's 815 (gate/up), 652 against 896 (down). The GEMM companion's sec.2.2k, the MoE companion's sec.2.2af and ledger item 43 carry these figures.The dense decode is unmoved by the GEMV fold: the Qwen3-4B Q4_K_M token profile reads 8036 us against 8032 / 8041 before it (
DASLLAMA_GPU_PROF=1,vk_rdec gpu avg/token).The 35B hybrid's
DASLLAMA_GPU_PROF=1profile aggregates on both eras after the sampler fixes: the token command 9397 us (MoE tail by name), the window 160181 us with 10 MoE layers on the attention head and 30 on the deltanet head reported on their own line.The
moe:/moesk:arm's buffer, size and hazard-bit order and itsBatchArgspush block were checked by hand againstKqCm2BatchT+MoeCmBase(wq 0, ws 1, sched 2, xf 3, y 5).tests/run.das -- --suite coverage --arm coverage-vk --family qwen2moe(the Q8_0 carrier on the per-op rails, the Q4_K_M twin on the resident driver): 4 passed, 2 skipped (the filtered arms), every new class dispatched -router_gemm_cls_spv24,topk_rows_cls_spv24,moe_sched_cls_spv24,cls_ar_comb_spv25,cls_ar_comb_f16_b_spv23,dec_combine_sh_cls_spv21. The unfiltered census (the whole zoo) did not run; the census's coopmat mode sweep read a null device state under a family filter that skipped every earlier row, fixed here.The dasllama lanes at the batch tip:
preflight --only dasllama-model-freePASS (1954.7 s).--only dasllama-stockedcannot finish on this box: preflight caps every test gate at 3600 s and the stocked suite runs about two hours here, so the lane cut after 26 of its files (25 green, one red below) and printed a negative elapsed. The suite then ran whole through the module's own runner,tests/run.das -- --suite stockedunderDASLLAMA_PARITY_FULL=1 DASLLAMA_GPU=1, no--exclude, at the tip before the rebase: 44 files in 88 minutes, 43 green, 121 cells skipped loudly (filtered arms, absent devices, models the box does not stock), one red -test_parity.das's gpt-oss-20b cell, fixed in the next bullet.That red was a das stack overflow at the cell's generate call, 18 to 35 s after the load, and the fix is the one line the module's deep-engine test rule asks for:
options stack = 524288intests/test_parity.das(the file had none while 84 of the module's test files do, and the gpt-oss forward - mx4 experts, sinks, biases - is its deepest chain). The cell was the model's first run on this box; every earlier stocked pass here skipped it on the model gate. The controls that placed it before the fix: the same cell with the GPU tier off overflowed too, and so did the cell on master's own module sources, whilebenchmarks/lcpp_bench.dasran the file on the CPU green (8 of 8 opening tokens, pp32 30.5 t/s, tg8 17.4 t/s) - the budget, not the engine, was the difference. With the line the cell passes both fixtures token-for-token (2 of 2, 121 s, tier on as in the suite).The one red that run found is fixed here:
tests/test_mtp.das's GLM-4.5-Air cell (the only non-recurrent MTP model, so the only reach of the depth-1 step's shortcut reject) wanted at least one draft rejected on its own and read 0 of 20 - the GEMV lane split moved a draft near-tie and the code fixture now accepts every draft. The engine gainsset_mtp_force_reject_every(n), a test seam that rejects every n-th draft of the depth-1 step's greedy walk whatever the verify said (the sampled walk is untouched); the cell runs its fixture plain, then with every third draft rejected, and holds both streams token-for-token to plain decode, asserting the forced run's reject count at a third of its drafts or more.test_mtp.dasalone: 10 of 10 (387 s).The MoE arc's plan document leaves the tree with the rest of the plans folder (master removed the folder while this branch was open; the rebase kept master's wording on every sentence both sides had rewritten, and no
plans/path remains anywhere in the tree). The arc's rows and readings live in ledger item 43, the MoE companion and this body.performance/fetch_models.das -- -o Qwen1.5-MoE-A2.7B-Chat-Q4_K_M-local: 1 ok, 0 fetched, 0 pending, 0 failed;-o UD-IQ2_XXS: 2 ok (both new rows verified against the stocked files' bytes and sha256; the unscoped verify over the whole stocked set did not run - a hash pass over every model); the twin's recipellama-quantize --allow-requantize Qwen1.5-MoE-A2.7B-Chat.Q8_0.gguf Qwen1.5-MoE-A2.7B-Chat-Q4_K_M-local.gguf Q4_K_M, 9496236768 bytes.ARCHITECTURE_MEASUREMENT.md's one-benchmark-rig section gained one paragraph (themoe:/moesk:arms); its 12 other citers' contracts are unchanged.Dupes report: the exact matches (
f16r,silu,q16,unpack_f16) predate the branch;bucket_start/bucket_rowsin the kernel file are justified siblings (a bucket's first row and its row count over the same slot list).Woodpecker (codex) ran twice: the first round's three findings (the clustered-add feature bit, the recurrent-MoE stamp count, the MoE profile table's reset) are fixed here; the second round, at the batch tip, reported no substantiated regression.
The make-pr chain's fast tier ran once on the batch tip: untracked, format, lint (49 files on both rails), hash-refs, md-ascii, ast-verify, ci-das and compile-sweep green; the review-md gate was red on the image layout stamp (the loader's shexp tagging pass is in the closure), re-stamped and rerun alone; the docs, tests-cpp, tests-interp, tests-jit and tests-aot tiers the red fast tier skipped ran once each afterwards at the tip before the rebase: lint PASS (83 files on both rails), docs PASS (all seven gates, sphinx 537 s), tests-cpp PASS; tests-interp 14348 tests, 14332 passed, 1 failed -
tests/watchdog/test_watchdog.dason the lane's 60 s per-file cap (169 s; it passes alone); tests-jit 14217 tests, 14202 passed, 2 failed - the same watchdog file (163 s) andtests/jit_tests/cross_target_folds.dasat 61.1 s against the same 60 s cap, no assert failed in either; tests-aot SKIPPED by preflight there (the DLL-flavor host pins the runtime DLL thetest_aotrelink needs). After the rebase onto master (which had removedplans/andhistory/and changed the dasbind module's C++), the tree's binary was rebuilt (a clean-first of the runtime DLL target ahead of it, the /Z7 post-merge link tax) and the gates ran again at the final tip: lint PASS (51 files on both rails), format, untracked and hash-refs PASS, md-ascii PASS, docs PASS (all seven gates, sphinx 490 s), the review-md gate OK on bothREVIEW.dasfiles, and tests-aot on atest_aothost built at the tip: 13308 tests, 13269 passed, 22 failed, all 22 in the threetests/module_cachefiles master changed on 2026-09-09 (test_descriptor_manifest.das11,test_module_groups.das9,test_require_module_now.das2 - the module-cache tests under the static AOT runner, the class the previous PR saw ontest_descriptor_manifest.dasalone); this branch touches none of them. The dasllama suites were not rerun after the rebase (master's 42 commits are TTS, a browser example and the arm64 extern marshalling; nothing under the Vulkan tier); their runs above are at the tip before it, on the binary before it.The resident MoE file's logits bar moved from 0.15 to 0.20 of the step's max logit: the GEMV fold's summation order flipped a router near-tie on the 35B at two windows (that step read 0.155 of the max logit against the 0.15 bar; the same step read 0.16 under the residual step's natural order and 0.04 under the chain's), so the class of a legitimate order change is 0.15-0.16 there, 0.07-0.11 on the twin and the 30B; the one-step-off controls read 0.42 and above on every step, so the bar still discriminates by 2x. The file's
//!carries the reading.Claims - stated, not tested
DASLLAMA_GPU_PROF=1profiles across processes, not an in-process interleaved race; a break would read as the window or token profile moving on the next profile run. The two probe arms the checklist's race rule asks for sit on ledger item 43's open list (ruled to ship as stated claims).REVIEW.daschecks were proven on planted defects in the tree (a body onkhr_stage16,AR_MAX_DIMat 4096 against the slab's 8192, the k4 arm dropped fromkhr_cls_ensure, a handvkCreateComputePipelinesin the engine): four findings, one per cell, then the files restored. The review-md discovery test (utils/internal/review-md/test_walkers.das) pins them too: its dasllama fixture carries the anchors the four checks read, and its planted half fires one violation per check with the finding's needle (the CI step red on the first push because the old fixture had no anchors; 6 of 6 after).pf_roles_per_layer's MoE arms,pf_moe_bucket_report,rdq_sample's MoE accounting,rd_moe_tail_names) are witnessed by theDASLLAMA_GPU_PROF=1runs on the 35B hybrid and the twin (the token and window per-role lines aggregate, no "roles unaggregated"), not by a test; that witness is what caught the recurrent-MoE stamp counts in both the token sampler and the window report, fixed here.L.xe_own's arm (an expert feed in a form the shared expert's is not) is dead on all three fixtures: the twin's shared expert rides its K-quant planes like its experts, the 30B has none, the 35B's is a K-quant too.vk_max_storage_range(); no stocked model reaches it (the 30B's plane is 51 MB).Not done
rd_tail_sets_barefrom a deltanet layer) has no fixture; the attention twin of the arm is the 30B's.moe:,moesk:, the gemv lanes sweep) have no oracle - measurement-only, the serving kernels they time are the kernel cells'.test_gpu_moe_shexp.das3 of 3,test_gpu_resident_moe.das9 of 9, the vulkan bake round trip (--suite image-vulkan) 4 of 4, the review-md gate OK.