[ncylich]

Summary

The Gemma-4 audio encoder was quarantined to CPU+GPU on Apple devices because its CoreML graph produced wrong output on the Neural Engine. Root cause: the audio attention masked-fill constant is attention_invalid_logits_value = -1e9, which overflows to -inf in the FLOAT16 NPU graph (fp16 max is 65504). Fully-masked padded rows then softmax to NaN, and partial rows diverge on the ANE.

This routes the gemma4 audio encoder to the ANE and makes it numerically correct there:

1. Clamp attention_invalid_logits_value to a finite, fp16-safe -1e4 before export. Logits are soft-capped to ±50, so masked positions still exp() to exactly 0 — masking is unchanged on every backend — but no value ever overflows fp16. (Same finite-mask pattern Parakeet already uses.)
2. Emit npu_audio_compute_units = "CPU_AND_NE" in the bundle manifest for gemma4 (as already done for parakeet_tdt).

Validation (Mac M4 Pro, int8 audio encoder, real test.wav)

Correctness on valid output tokens vs PyTorch fp32 ground truth:

backend	cos vs torch fp32
GPU (old path)	0.9998
ANE (this PR)	0.9983

Before the fix the ANE diverged and CPU-only returned NaN; both are resolved.

Speed (median, fully warm):

backend	latency
GPU	~123 ms
ANE	~84 ms (1.46x faster)

The finite mask also lets the encoder compile cleanly onto the ANE — the old -inf graph silently partition-fell-back and was never truly resident.

Notes

• Vision encoder is intentionally left on CPU+GPU: it is ~2.4x slower on the ANE (a 2,520-token ViT is GPU-favorable) and the GPU output already matches torch.
• Deploying picks this up automatically on the next cactus transpile … --npu of the gemma4 bundle.

Signed-off-by: Noah Cylich noahcylich@gmail.com