QA: Check for a leaked class fixture at the class boundary - #48976
tarekziade wants to merge 4 commits into
Conversation
MemoryCleanupMixin's leak check is per-test, so a model setUpClass parks on the class is already inside every test's baseline and every test reports zero.
The fallback `mamba2_chunk_scan` built each contraction as a broadcast product followed by a sum, materialising the un-summed tensor first. G's is (batch, chunks, chunk, chunk, heads, state) in float32 — 4 GiB per sequence at bamba-9B's shapes, so BambaModelIntegrationTest OOMs asking for 8 GiB on a 22.3 GiB runner for a 10-token generate, with 18.2 GiB of that card holding the model setUpClass loaded. Largest intermediate drops 4.000 GiB -> 0.031 GiB.
2a5126e to
c47ddcb
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: bamba, falcon_h1, granitemoehybrid, mamba2, nemotron_h, zamba2 |
CI recapDashboard: View test results in Grafana
|
|
run-slow: bamba, falcon_h1, granitemoehybrid, mamba2, nemotron_h, zamba2 |
AMD CIThis comment contains models: ["models/bamba", "models/falcon_h1", "models/granitemoehybrid", "models/mamba2", "models/nemotron_h", "models/zamba2"] |
Nvidia CIThis comment contains models: ["models/bamba", "models/falcon_h1", "models/granitemoehybrid", "models/mamba2", "models/nemotron_h", "models/zamba2"] |
CI Results (AMD)Commit Info
Model CI Report❌ 9 new failed tests from this PR 😭
|
CI Results (Nvidia)Commit Info
Model CI Report❌ 14 new failed tests from this PR 😭
|
2 participants
What does this PR do?
MemoryCleanupMixinleak check is per-test, so a modelsetUpClassparks on the class is already inside every test's baseline and every test reports zero.This patch also fixes Mamba2 which had that issue (in mamba2_chunk_scan)