Repository navigation
Conversation
This commit support auto_percdamp for Qwen GPTQ Quantizer Co-Authored-By: Cline TICO-DCO-1.0-Signed-off-by: Evgenii Maltsev <e.maltsev@samsung.com>
|
Qwen3-vl-4B evaluation with using auto_percdamp. Commit: 658a7704a
Run command:Additional configuration:
Logs:Run command: |
|
Thanks for adding opt-in automatic per-layer damping and bounded Cholesky retries. The overall direction looks reasonable, especially given the different calibration characteristics of Qwen3-VL's vision and language stages. However, I found three issues that I think should be addressed before merging. 1. Memory regression in the default
|
|
@mhs4670go, thank you for reviewing this PR! I've fixed your comments:
|
|
Thanks for addressing the previous review comments. Clearing Remaining issueWhen this operation fails: hinv = torch.linalg.cholesky(hinv, upper=True) # _factorize_hessian_inverse()the assignment does not complete, so There is an additional reference path when Consequently, the retry path can retain an extra full-size matrix precisely when it is trying to recover from a numerical failure. For an 8192 × 8192 FP64 matrix, one unnecessary allocation is 512 MiB, excluding workspaces. This can turn an otherwise recoverable numerical failure into an OOM in a memory-constrained run; I am not claiming that a full-model GPU OOM was reproduced. Suggested implementationCould we isolate each attempt in its own helper and retain only diagnostic text, rather than the exception object, in the outer retry loop? Both changes matter: the helper gives each attempt its own temporary variables, and dropping the exception prevents its traceback from retaining that helper frame. Python clears the The following is a suggested replacement for the auto-mode helper. Add Suggested code def _factorize_hessian_inverse_once(
self,
h: torch.Tensor,
diag: torch.Tensor,
diag_mean: torch.Tensor,
percdamp: float,
) -> torch.Tensor:
"""Run one attempt; keep its temporary tensors in this frame only."""
h_try = h.clone()
h_try[diag, diag] += percdamp * diag_mean
chol = torch.linalg.cholesky(h_try)
del h_try
hinv = torch.cholesky_inverse(chol)
del chol
hinv = torch.linalg.cholesky(hinv, upper=True)
self._validate_hinv(hinv, percdamp)
return hinv
def _factorize_hessian_inverse(
self,
h: torch.Tensor,
percdamp: float,
percdamp_max: Optional[float] = None,
retry_factor: float = 2.5,
max_retries: int = 5,
) -> torch.Tensor:
"""Retry auto-mode factorization without retaining failed attempts."""
if percdamp_max is None:
raise ValueError(
"percdamp_max is required for the retry path; the static "
"single-attempt path is inlined in fasterquant."
)
if percdamp_max < percdamp:
raise ValueError(
"percdamp_max must be >= percdamp. "
f"got {percdamp_max} < {percdamp}"
)
diag, diag_mean = self._damping_precheck(h, percdamp)
cur_percdamp = float(percdamp)
ceiling = float(percdamp_max)
attempts = 0
last_error_text = ""
while True:
attempts += 1
try:
hinv = self._factorize_hessian_inverse_once(
h, diag, diag_mean, cur_percdamp
)
except torch.OutOfMemoryError:
# Damping cannot fix an OOM. Preserve the original exception.
raise
except RuntimeError as exc:
# Keep diagnostic text only, not the exception or traceback.
# Python clears `exc` when this handler exits, allowing the
# failed helper frame and its temporaries to be released
# before the next attempt allocates anything.
last_error_text = f"{type(exc).__name__}: {exc}"
else:
self.percdamp_used = cur_percdamp
return hinv
# Keep the existing escalation policy and final ceiling attempt.
if cur_percdamp >= ceiling:
break
if attempts >= max_retries:
cur_percdamp = ceiling
else:
cur_percdamp = min(ceiling, cur_percdamp * retry_factor)
raise RuntimeError(
"GPTQ Hessian factorization failed or became numerically unstable "
f"after {attempts} attempt(s). "
f"start_percdamp={percdamp}, last_percdamp={cur_percdamp}, "
f"retry_ceiling={percdamp_max}. "
"The final attempt used the configured retry ceiling. "
"Increase percdamp or raise the retry ceiling "
"(auto_percdamp_retry_max). "
f"Last error: {last_error_text}"
) from NoneThis keeps the current damping schedule, the final attempt at the configured ceiling, Diagnostic trade-off: the final numerical-failure error includes the last exception's type and message, but deliberately does not chain the original exception. Retaining that exception would retain its traceback and defeat the cleanup. OOM exceptions are still re-raised unchanged. Neither |
What
Adds an opt-in automatic per-layer damping policy (
percdamp_mode="auto") to the Qwen3-VL GPTQ quantizer. Instead of using a single globalpercdampvalue for every layer, auto mode derives a per-layer starting damping factor from the Qwen3-VL stage type, calibration volume, effective token/patch observations, and Hessian diagonal spread — and can retry unstable Cholesky factorizations with escalated damping instead of aborting the whole run.Default behavior is unchanged:
percdamp_mode="static"preserves the existing GPTQ behavior bit-for-bit.Motivation
Qwen3-VL mixes heterogeneous stages (vision patch embed, vision blocks, mergers/deepstack mergers, text layers, LM head) that are calibrated with very different numbers of batches and effective observations. A single global damping factor is a poor fit: vision stages replaying fewer calibration batches tend to produce ill-conditioned Hessians, and a Cholesky failure on a single layer previously aborted the entire quantization run. Auto damping normalizes regularization across stages and makes runs resilient via bounded damping escalation.
Usage:
Co-Authored-By: Cline
TICO-DCO-1.0-Signed-off-by: Evgenii Maltsev e.maltsev@samsung.com