exp(acestep): int8 DiT fake-quant quality gate (OPT-0089) - #62
Merged
Merged
Conversation
Additive ledger entry and experiment record only; no existing record content changed. Registers the measured iPhone-memory feasibility gate before any requantization or comparison work, per the optimization-history discipline in AGENTS.md. Distinct mechanism from abandoned OPT-0058 (no activation quantization, no kernel changes).
…llocation Test-file pin refresh only, per the established convention.
Streams the hosted rev7 dense DiT package shard-by-shard, fake-quantizes the 264 rank-two GEMM tensors (symmetric int8, fp16 scale per 32-K block, round-to-nearest, clamp +/-127) in their native tile layouts, and emits a content-addressed package whose manifest differs only in the 48 shard SHA-256 values plus its own SHA-256 (byte length preserved), plus a per-tensor quantization-error JSON report. Norms, biases, scale-shift tables, and constants are byte-identical passthrough. models-local output stays gitignored; only the script is committed.
…results Evidence positive (numerical feasibility only, no listening claim), disposition benchmark-only. Records the per-tensor quantization-error report (264 tensors, NRMSE 0.00515-0.00634, no outliers), the byte-exact fp16 determinism reproduction of the pinned seed-12345 baseline WAV, the fake-quant vs fp16 waveform/spectral comparison on two prompt/seed pairs (NRMSE 0.0669 / 0.2268, trajectory divergence rather than noise-like corruption), and the iPhone residency projection (~1.862 GB int8 phase peak vs the observed ~1.920 GB kill boundary). Updates only the OPT-0089 row and record; no prior ledger content changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measured feasibility gate for an int8-weight DiT on iPhone, registered as OPT-0089 (benchmark-only; no kernel, runtime, or production change). Motivated by a real iPhone 17 Safari tab OOM-killed during DiT weight upload at 1,789,925,376 / 3,020,808,192 bytes (layer 14/24): the fp16 rev7 dense DiT cannot fit the tab. Before any int8 kernel work, this measures PURE quantization damage with zero kernel changes.
What this PR contains
scripts/requantize-dit-int8.py— streams the hosteddit-revision7package shard-by-shard, fake-quantizes the 264 rank-two GEMM tensors (symmetric int8, fp16 scale per 32-K block, round-to-nearest, clamp ±127) in their native tile layouts, dequantizes in place, and emits a content-addressed package whose manifest differs only in the 48 shard SHA-256s + its own SHA-256 (byte length preserved at 254,357). Norms/biases/tables/constants byte-identical. Fake-quant packageef8355b9…stays in gitignoredmodels-local/— weights are NOT in this PR.quant-error.json,waveform-metrics.json), following the OPT-0088 registration convention; next-available-ID test pins refreshed.Measured results
Per-tensor damage (weight level): uniform, NRMSE 0.00515–0.00634 (median 0.00559), min SNR 43.96 dB, no outlier tensor or family → no mixed-precision fp16-retention map needed.
End-to-end, unchanged production graph, fresh profile per run, local model origins; fp16 determinism gate reproduced the pinned seed-12345 baseline WAV byte-exactly (
095267d7…) before any fake-quant run:The damage is trajectory divergence of the 8-evaluation sampler, not noise-like corruption (zero non-finite samples, exact peak parity, flat per-second profiles; the latin per-second max 5.19 is a near-silent ending-decay small-denominator artifact).
Memory projection: int8 resident = 1.51 GB weights + 94 MB scales; projected DiT phase peak ≈1.862 GB tracked GPU vs the observed ≈1.920 GB iPhone kill point (conditioning already released before upload) — plausibly fits, ≈58 MB margin; fallback levers identified (quantize 130 MB reference-shared, int4 MLP).
Verdict: int8-per-block-32 does not collapse generation; kernel work justified under a new OPT id; listening gate mandatory before any product claim.
Validation
npm run acestep:testgreen (2025 passed, 27 skipped) after registration and after close.Do not merge without review; benchmark-only record.