Repository navigation
Fix per-matrix CQ4 reciprocals in Metal batch transform - #784
Merged
jakmro merged 1 commit intoAug 13, 2026
Merged
Conversation
Collaborator
|
Hey @clumsylad21, thanks for this PR, it just needs a DCO sign off |
Signed-off-by: Bhavyansh Sharma <66562679+clumsylad21@users.noreply.github.com>
clumsylad21
force-pushed
the
fix/issue-773-metal-cq4-recip
branch
from
August 13, 2026 20:17
5e4c102 to
6742fea
Compare
Contributor
Author
|
Thanks @jakmro ! Added the DCO sign-off. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #773
Summary
Fix the Metal CQ4 batched transform to use each matrix's own
input_scale_recipinstead of reusing the first matrix's reciprocalbuffer across the batch.
Also adds a regression test covering batched CQ4 projections with
distinct reciprocal scales.
Root cause
cactus_metal_encode_transform_batch()bound onlyWs[0]'sinput_scale_recip, whileleft_signs,right_signs, andpermutationwere already bound separately for each matrix.As a result,
cq4_transform_batchapplied the first matrix'sreciprocal scaling to every projection in the batch.
This was effectively masked for the uncalibrated CQ4 weights, whose
input scales are unit-valued, but breaks calibrated weights where the
per-matrix scales differ.
Validation
The targeted Metal regression test now passes:
The test uses three CQ4 matrices with deliberately distinct reciprocal
scales and checks the batched Metal path against independent Metal
execution.
I also reproduced the original model-level failure on the current
Cactus main baseline (
c90a1af) using the calibrated Gemma CQ4 bundle.Before the fix:
With only the per-matrix reciprocal fix:
The uncalibrated/calibrated CPU runs did not show this backend-specific
regression.