Skip to content

enable native MPS GPTQ quantization - #3011

Merged
Qubitium merged 1 commit into
mainfrom
mlx
Aug 7, 2026
Merged

enable native MPS GPTQ quantization#3011
Qubitium merged 1 commit into
mainfrom
mlx

Conversation

@Qubitium

@Qubitium Qubitium commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • stop models.auto from globally forcing PYTORCH_ENABLE_MPS_FALLBACK=1 on macOS
  • remove obsolete fallback requirements from GPTQ, GPTAQ, and FOEM quantization
  • add macOS regression coverage for environment preservation and native MPS GPTQ quantization

Why

Modern PyTorch MPS implements the GPTQ Hessian and quantization operations used here. The old guard forced unsupported operations onto CPU even when the Metal implementation was available, preventing native MPS execution and making Hessian math substantially slower on Apple Silicon.

Fallback remains available as an explicit PyTorch opt-in for users running older builds with unsupported operations.

Validation

  • tests/test_mps_gptq.py: 2 passed on Apple M4 Max with PYTORCH_ENABLE_MPS_FALLBACK unset
  • complete GPTQ quantization produced finite MPS weights without CPU fallback
  • focused GPTQ configuration tests: 2 passed
  • git diff --check
  • Ruff passes for the new regression test

@Qubitium
Qubitium marked this pull request as ready for review August 7, 2026 14:42
@Qubitium
Qubitium merged commit 3403789 into main Aug 7, 2026
6 checks passed
@Qubitium
Qubitium deleted the mlx branch August 7, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant