[jhapriyavart10]

Summary

This PR corrects progressive multi-block weight compensation in _gptq_correct_group() when Hadamard CQ quantization runs with GPTQ (use_gptq=True).

Previously, each 128-column block used slices of a single global inverse Hessian. This is correct for the first block, but for subsequent blocks the required inverse is that of the remaining unquantized sub-Hessian.

This PR replaces the static block slicing with an upper-Cholesky formulation that preserves the existing Hessian preprocessing while providing the correct progressive compensation update.

Problem & Root Cause

For progressive Optimal Brain Surgeon (OBS) / GPTQ, after block B is quantized, the remaining weights must be compensated using the Hessian of the remaining unquantized columns.

The existing implementation computes:

m_bb = h_inv[start:stop, start:stop]
m_bs = h_inv[start:stop, stop:]
update = np.linalg.solve(m_bb + 1e-6 * I, m_bs)

where h_inv is the inverse of the original full Hessian.

A slice of an inverse matrix is generally not the inverse of the corresponding submatrix:

(H^-1)[R, R] != (H[R, R])^-1

For progressive blocks after the first one, this causes the compensation update to ignore the Schur-complement contribution from previously quantized columns.

The defect was reproduced with deterministic correlated positive-definite Hessians. The old implementation matches the OBS reference for block 0, but diverges on subsequent blocks; the regression case reaches a maximum update difference of approximately 1.198.

Fix

The global inverse Hessian is factored once:

U = np.linalg.cholesky(np.linalg.inv(H)).T

so that, in exact arithmetic:

H^-1 = U^T U

For each progressive block, the corresponding upper-triangular blocks are used:

u_bb = u_factor[start:stop, start:stop]
u_bs = u_factor[start:stop, stop:]
update = solve_triangular(u_bb, u_bs, lower=False)

The existing compensation application remains unchanged:

work[:, stop:] -= (work[:, start:stop] - recon) @ update

This formulation is algebraically equivalent to the required progressive OBS update in exact arithmetic and was validated numerically against an independent progressive sub-Hessian reference in float32.

The redundant per-block +1e-6 I perturbation is removed. The existing Hessian-level preprocessing and damping remain unchanged.

Regression Tests

Added deterministic coverage in test_cq.py.

test_gptq_multiblock_compensation_matches_progressive_obs

Uses:

• correlated positive-definite Hessian
• K = 384
• block size 128
• three progressive blocks

The test independently computes the expected OBS update from the inverse of the remaining sub-Hessian and compares it with the production Cholesky implementation.

It also demonstrates that the previous static-inverse formulation fails the same correctness tolerance on a later block.

test_quantize_hadamard_with_gptq_multiblock

Verifies that quantize_hadamard() actually executes GPTQ with a non-null Hessian and that compensation affects subsequent block quantization decisions.

Validation

• Targeted CQ tests: 10 passed
• Runnable conversion tests: 81 passed
• Native test_parakeet_capture.py: environment-blocked on Windows because the compiled native runtime library is unavailable
• git diff --check: clean

The new regression coverage is CPU/Python-only and does not require ARM, Metal, or external model checkpoints.

Performance

Measured on K = 512, block size 128, float32, identical data, 10 warmup iterations, and 100 timed iterations.

Overall factorization + compensation measurements:

• Previous implementation: median 35.825 ms
• New implementation: median 34.120 ms

Measured difference:

• median: approximately 4.76% faster

The block-solve component alone is substantially cheaper with triangular solves, but the overall pipeline measurement is the relevant comparison.

Scope / Non-Goals

This PR intentionally changes only:

• python/cactus/convert/quantization/cq.py
• python/cactus/convert/tests/test_cq.py

It does not modify:

• CLI calibration-manifest forwarding
• model adapters or conversion policies
• C++ engine code
• graph execution
• kernels
• serialized weight format

Relationship to Issue #767 and PR #790

This PR is related to Issue #767.

Issue #767 reports CQ4 degradation for a finetuned Needle checkpoint. During investigation, two distinct paths were identified:

1. the public CLI does not forward --calibration-manifest, which is being addressed separately in draft PR Add calibration manifest support to convert CLI #790;
2. when calibrated GPTQ executes on multi-block tensors, the progressive compensation logic in cq.py uses the incorrect static inverse formulation.

This PR addresses the second issue.

PR #790 addresses the separate CLI calibration-manifest path. The two changes are complementary.

Because the reporter's exact finetuned checkpoint is not publicly available, this PR does not claim that it reproduces or completely resolves the reporter's exact token-garbage output.

Related: #767

[Copilot]

Copilot review overview

🟡 Changes recommended

The new eager SciPy import can break base installations where SciPy is not installed, including transpilation paths that import this module.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Eager SciPy import breaks base installation imports · New

What changed in this PR

Corrects progressive GPTQ compensation for multi-block Hadamard CQ quantization using an upper-Cholesky formulation, with deterministic regression coverage.

Changes:

• Replaces static inverse-Hessian slicing with triangular solves.
• Adds progressive OBS equivalence and GPTQ integration tests.
• Introduces an eager SciPy import in cq.py.

File	Description
python/​cactus/​convert/​quantization/​cq.py	Updates GPTQ compensation implementation.
python/​cactus/​convert/​tests/​test_cq.py	Adds multi-block compensation regression tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jhapriyavart10]

Good catch. Because scipy is declared under the optional convert dependency extra in pyproject.toml, importing solve_triangular at module import time could break base installations when cq.py is loaded indirectly.

I've pushed commit 3f133da0 moving from scipy.linalg import solve_triangular lazily inside _gptq_correct_group() within the existing try: block. This matches the existing lazy-import pattern used in cq.py for gammaln and hadamard, preserves the existing fallback semantics, and leaves the GPTQ mathematical formulation unchanged.

The targeted CQ tests and runnable conversion test suite both still pass.