Repository navigation

Improve layerwise GPTQ memory and speed: Hessian headroom, batch probe and offload fixes - #2945

Merged

cjluo-nv merged 6 commits into

mainfrom

chenjiel/gptq-hessian-headroom

Oct 27, 2026

Merged

Improve layerwise GPTQ memory and speed: Hessian headroom, batch probe and offload fixes#2945
cjluo-nv merged 6 commits into
mainfrom
chenjiel/gptq-hessian-headroom

Commits on Oct 26, 2026
- Place GPTQ Hessians by measured headroom instead of a 65% usage cap
  
  cjluo-nv
  and
  claude
  committed
  a53f43f
- Size the layerwise calibration batch by probing each kind of decoder layer
  
  cjluo-nv
  and
  claude
  committed
  919fa71
- Raise the layerwise calibration token cap to 32,768
  
  cjluo-nv
  and
  claude
  committed
  2c89473
- Keep offloaded layers' quantizer state off the GPU and stop layerwise OOM retries
  
  cjluo-nv
  and
  claude
  committed
  84aec95
- Address review: offload writeback, probe OOM, forward peak, shared helpers
  
  cjluo-nv
  and
  claude
  committed
  3472896
- Probe every GPU a layer kind runs on; count only allocatable cache
  
  cjluo-nv
  and
  claude
  committed
  8c7a668