[begcdn]

Overview

Adds LLM_ARCH_LAGUNA to the existing build_amax_src1_f16 condition in build_moe_ffn, so Laguna applies the same f16 activation rescaling already used by Mistral4, Mellum and Flint.

Without it, Laguna MoE models overflow on Metal when the prefill microbatch is 32 tokens or larger (#30896): kernel_mul_mm_id stages the f32 activation through half in shared memory, so gate/up outputs above 65504 become inf in the down projection. At smaller microbatches a different kernel path is used, which is why the failure only appears at ubatch >= 32.

Verified locally on a Laguna-XS-2.1 slice: the src1 amax into ffn_moe_down dropped from 103671.5 to 1018.9 with this change, and the unflagged Metal MUL_MAT_ID path produces NaN for src1 ~1e5 at n >= 32 while CPU does not.

Additional information

Ref: #30896, #30584

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - an AI agent (Devin) located the bug mechanism, wrote the one-line change and the local verification probes; the patch follows the existing pattern from graph : add precision handling for MELLUM architecture #30584.

[CISC]

Split axis needs to be fixed https://github.com/ggml-org/llama.cpp/actions/runs/40192579226/job/121460068820?pr=31147#step:8:381

[ggerganov]

@begcdn Apply this patch:

diff --git a/src/llama-arch.cpp b/src/llama-arch.cpp
index 39e29d09a..3a5afe36f 100644
--- a/src/llama-arch.cpp
+++ b/src/llama-arch.cpp
@@ -1338,6 +1338,7 @@ bool llm_arch_supports_sm_tensor(const llm_arch & arch) {
         case LLM_ARCH_MISTRAL4:
         case LLM_ARCH_MELLUM: // [TAG_MOE_AMAX_SCALE]
         case LLM_ARCH_FLINT:  // [TAG_MOE_AMAX_SCALE]
+        case LLM_ARCH_LAGUNA: // [TAG_MOE_AMAX_SCALE]
         case LLM_ARCH_KIMI_LINEAR:
         case LLM_ARCH_BAILINGMOE3:
         case LLM_ARCH_KIMI_K3: