Skip to content

Fix per-matrix CQ4 reciprocals in Metal batch transform - #784

Merged
jakmro merged 1 commit into
cactus-compute:mainfrom
clumsylad21:fix/issue-773-metal-cq4-recip
Aug 13, 2026
Merged

jakmro merged 1 commit into
cactus-compute:mainfrom
clumsylad21:fix/issue-773-metal-cq4-recip

Conversation

@clumsylad21

Copy link
Copy Markdown
Contributor

Fixes #773

Summary

Fix the Metal CQ4 batched transform to use each matrix's own
input_scale_recip instead of reusing the first matrix's reciprocal
buffer across the batch.

Also adds a regression test covering batched CQ4 projections with
distinct reciprocal scales.

Root cause

cactus_metal_encode_transform_batch() bound only Ws[0]'s
input_scale_recip, while left_signs, right_signs, and
permutation were already bound separately for each matrix.

As a result, cq4_transform_batch applied the first matrix's
reciprocal scaling to every projection in the batch.

This was effectively masked for the uncalibrated CQ4 weights, whose
input scales are unit-valued, but breaks calibrated weights where the
per-matrix scales differ.

Validation

The targeted Metal regression test now passes:

✓ PASS │ matmul_cq4_metal_distinct_recip

The test uses three CQ4 matrices with deliberately distinct reciprocal
scales and checks the batched Metal path against independent Metal
execution.

I also reproduced the original model-level failure on the current
Cactus main baseline (c90a1af) using the calibrated Gemma CQ4 bundle.

Before the fix:

pred=16, gold=18, chars=4350

With only the per-matrix reciprocal fix:

pred=18, gold=18, chars=1018

The uncalibrated/calibrated CPU runs did not show this backend-specific
regression.

@jakmro

jakmro commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Hey @clumsylad21, thanks for this PR, it just needs a DCO sign off

Signed-off-by: Bhavyansh Sharma <66562679+clumsylad21@users.noreply.github.com>
@clumsylad21
clumsylad21 force-pushed the fix/issue-773-metal-cq4-recip branch from 5e4c102 to 6742fea Compare August 13, 2026 20:17
@clumsylad21

Copy link
Copy Markdown
Contributor Author

Thanks @jakmro ! Added the DCO sign-off.

@jakmro
jakmro merged commit 22b86a6 into cactus-compute:main Aug 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants