Repository navigation
[executorch][native] Add GGUF q4_k and q6_k packages to the op samples - #24366
digantdesai wants to merge 4 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24366
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 820b01d with merge base 0f8e096 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
Stack from ghstack (oldest at bottom):
PTN carries GGUF k-quant constants, but no op package exercised them. A
Ggufscheme, asgguf_q4_kandgguf_q6_k, writes random valid blocks to a .gguf with gguf-py and loads them back as a checkpoint would.golden-int8_actnow comes from per-token int8 pre-hooks on the weight-only model, which also work for GGUF, soScheme.weight_onlyreplacesScheme.act_quant. FACTO also keeps the tuples whose input features fit 256-element blocks.Authored with Claude Code.
Differential Revision: D128254890
NOTE FOR REVIEWERS: This PR has internal Meta-specific changes or comments, please review them on Phabricator!
Differential Revision: D128254890