Skip to content

Add calibration manifest support to convert CLI - #790

Draft
Lulzx wants to merge 1 commit into
cactus-compute:mainfrom
Lulzx:fix/cq-calibration-manifest
Draft

Lulzx wants to merge 1 commit into
cactus-compute:mainfrom
Lulzx:fix/cq-calibration-manifest

Conversation

@Lulzx

@Lulzx Lulzx commented Aug 16, 2026

Copy link
Copy Markdown

Summary

  • expose --calibration-manifest on the public cactus convert command and forward it through local CQ conversion
  • make cached output fail clearly instead of silently ignoring a supplied calibration manifest
  • report zero-sample and unusable-Hessian RTN fallbacks separately, with strict-mode failures when GPTQ calibration cannot be applied
  • document the manifest format and add CLI, report, malformed-Hessian, and tiny Needle end-to-end calibration coverage

Root cause

The internal converter already accepted calibration manifests, but the public CLI did not expose or forward the option. This made finetuned Needle conversions run without representative activation data. In addition, cached bundles could bypass calibration silently, and positive-sample Hessians that could not be applied fell back to RTN without a visible warning.

Impact

Finetuned checkpoints can now supply representative language data through the supported CLI. Conversion summaries distinguish successful GPTQ calibration from both zero-sample and unusable-Hessian fallbacks, making accidental RTN conversion visible and actionable.

Validation

  • PYTHONPATH=python python3 -m pytest python/tests/test_cli_calibration.py python/cactus/convert/tests -q (77 passed)
  • focused Ruff checks passed, excluding three pre-existing violations in untouched lines (E401, F401, F841)
  • python3 -m compileall -q python/cactus/cli python/cactus/convert python/tests/test_cli_calibration.py
  • git diff --check

Fixes #767

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Finetuned Needle checkpoint converts to a CQ4 bundle that emits token garbage (stock converts fine)

1 participant