Repository navigation
[quantization] Standardize FP eager vs. Quant eager benchmarks - #1073
Merged
Merged
Conversation
mhs4670go
marked this pull request as draft
October 21, 2026 08:23
mhs4670go
marked this pull request as ready for review
October 25, 2026 13:02
Thsi commit standardize eager benchmarks. TICO-DCO-1.0-Signed-off-by: seongwoo <mhs4670go@naver.com>
Keep eager quality evaluation as the default without an execution backend option. Update the shared protocol, example configurations, documentation, and regression tests; preserve the independent calibration/evaluation/export sequence-length settings. TICO-DCO-1.0-Signed-off-by: seongwoo <mhs4670go@naver.com>
Update positive export tests to provide explicit static inputs while retaining missing-input negative coverage. Validate complete indexed probe arrays and per-output parity records against graph boundaries, reject malformed evidence, and recompute requested Torch/Circle parity from saved arrays. Add regressions for symmetric evidence loss and stale parity summaries. Narrow dynamic wrapper/buffer/report types and correct import layout without changing quantized arithmetic, frozen qparams, or comparison tolerances. Prepare artifact directories before optional full-text probes. Validation: 32 standalone comparator CLI cases pass; both historical false-positive cases reproduce in the parent version. Changed Python sources parse with Python 3.10 grammar and compile. Project tests and canonical UFMT/Pylint/mypy verification remain blocked by unavailable dependencies in this environment. TICO-DCO-1.0-Signed-off-by: ChatGPT <noreply@openai.com>
Validate complete runtime evidence against graph boundary contracts, including dynamic shape signatures, execution dtypes, and recomputed Torch parity. Preserve calibration length by default and make the historical bridge explicit. Extend static-input and HF RoPE regressions and finish repository formatting. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
Preserve the HF generation backend contract without retaining another Llama model. Calibrate Qwen cache boundaries with teacher-forced dataset tokens and apply consistent past-KV quantization to dynamic and static caches. Reject uncalibrated active cache paths without repairing frozen state. Correct padded perplexity scoring and add deterministic CPU HF/Circle checks. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
Avoid cropping physical eager sliding-cache masks a second time. Resolve supported eager calibration from native and recipe budgets independently of static export profiles, retaining static and legacy capacity checks. Cover padded cached decode and real PLE/shared-KV calibration beyond the static profile. Calibrate the cached-inference fixture inside its original three-token budget and retain the upstream past-observer guard. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
added 3 commits
October 26, 2026 02:04
Rename cpu_followup to cpu_validation and update child invocations, helper/test imports, and documented commands. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
Remove historical calibration diagnostics and their unused fixture artifacts. Expose connected replay with complete finite, dtype-aware saved evidence checks. Describe checkpoint readiness and revision selection without PR-specific history. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
Translate the CPU validation guidance to English. TICO-DCO-1.0-Signed-off-by: SEONGWOO CHAE <mhs4670go@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thsi commit standardize eager benchmarks.
Related: #1072
TICO-DCO-1.0-Signed-off-by: seongwoo mhs4670go@naver.com