Fix swiglu_decode intermediate comparison to use chained reference#105
Open
albiol2004 wants to merge 1 commit intoamd:develfrom
Open
Fix swiglu_decode intermediate comparison to use chained reference#105albiol2004 wants to merge 1 commit intoamd:develfrom
albiol2004 wants to merge 1 commit intoamd:develfrom
Conversation
The decode test verified `intermediate` against golden_ref["intermediate"] (CPU-computed silu(golden_left) * golden_right), while the prefill test uses a chained reference built from the observed AIE left_swished and right buffers. That inconsistency surfaces as spurious failures at rectangular FFN shapes (e.g. embedding=1024, hidden=3584): the AIE SiLU LUT rounds near-zero outputs to exactly 0.0 where fp32 CPU silu keeps a tiny negative value, and the subsequent multiply against a large-magnitude right operand amplifies that sub-tolerance drift into "got 0.0, expected -1.27"-style mismatches. The AIE kernels are numerically correct, the observed intermediate matches observed_left_swished * observed_right exactly, and the final output already passes at a tighter tolerance. Only the verification methodology was wrong. Switch to the prefill-style chained reference and tighten tolerance to (rel=0.04, abs=0.4), same as the output check and prefill intermediate. Add (1024, 3584) to the parametrization so rectangular decode is covered in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aligns
swiglu_decode/test.pywithswiglu_prefill/test.pyby verifying theintermediatebuffer against a chained reference built from the observed AIEleft_swishedandrightbuffers, instead of against the CPU-computedgolden_ref["intermediate"].The golden-reference path amplifies legitimate, sub-tolerance bf16 drift from upstream stages (e.g. SiLU of very-negative inputs where the AIE LUT rounds to
0.0while fp32 CPU silu preserves a tiny negative value) through the multiplication against a large-magnituderightoperand, producing spurious "got 0.0, expected -1.27"-style failures. The AIE kernels themselves are numerically correct, the observedintermediatematchesobserved_left_swished * observed_rightexactly, and the finaloutputalready passes at a tighter tolerance than the intermediate stage.This issue surfaces at rectangular FFN shapes (e.g.
embedding_dim=1024, hidden_dim=3584) where the statistics of the SiLU input distribution make near-zero LUT outputs more common than at the previously-tested square2048²shape. Adds(1024, 3584)to the parametrization so regressions in rectangular decode are caught.Added
(1024, 3584)parametrization iniron/operators/swiglu_decode/test.py, reflecting Qwen3.5-0.8B FFN dims so rectangular decode is covered alongside the existing square smoke test.Changed
iron/operators/swiglu_decode/test.py: verifyintermediateagainst a chained reference (observed_left_swished * observed_right) rather thangolden_ref["intermediate"], matching the approach already inswiglu_prefill/test.py. Tightens the tolerance torel_tol=0.04, abs_tol=0.4accordingly (same values used for the output check and for prefill).Removed
Testing
Verified on NPU2 (Strix,
aie2p):pytest iron/operators/swiglu_decode/test.py -v --iterations 1: both2048×2048and1024×3584pass.pytest iron/operators/ -m "not extensive" --iterations 1: no regressions.PR Merge Checklist
develcommit and pointing todevel.