Skip to content

[None][infra] Waive 12 failed cases for main in post-merge 2889 - #17407

Open
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260807-073530
Open

[None][infra] Waive 12 failed cases for main in post-merge 2889#17407
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260807-073530

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by @mzweilz)
Target branch: main
Bug(s): 6572800, 6572835, 6572843

Waive entries added

examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_flux.py::test_flux_accuracy_against_golden[flux1-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_flux.py::test_flux_accuracy_against_golden[flux2-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_ltx2.py::test_ltx2_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_qwen_image.py::test_qwenimage_feature_accuracy_against_golden[cuda-graph] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_qwen_image.py::test_qwenimage_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan21-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-cuda-graph] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-fp8-blockwise] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-nvfp4] SKIP (https://nvbugs/6572800)
perf/test_perf_sanity.py::test_e2e[disagg_upload-e2e-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL] SKIP (https://nvbugs/6572843)
unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "MEGAMOE_CUTEDSL" SKIP (https://nvbugs/6572835)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Dev Engineer Review

  • Added 12 waiver entries in tests/integration/test_lists/waives.txt.
  • Entries cover visual-generation accuracy tests, one GB300 disaggregated performance test, and one single-GPU MegaMoE CUTEDSL test.
  • Each entry references bug 6572800, 6572835, or 6572843.
  • The change is additive and has no exported API changes.
  • The waiver format and scope are consistent with the stated objectives.
  • No duplicate or invalid entries were identified from the provided change summary.

QA Engineer Review

  • Modified tests/integration/test_lists/waives.txt.
  • Added 12 waiver entries. No entries were removed.
  • The change affects test waivers only. It does not modify test implementations.
  • CBTS coverage data is unavailable.
  • Verdict: needs follow-up.

Bug(s): 6572800, 6572835, 6572843
Requested by: @mzweilz

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@mzweilz

mzweilz commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

/bot skip --comment "waive only"

@mzweilz
mzweilz enabled auto-merge (squash) August 7, 2026 07:36
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The integration waiver list now skips selected visual-generation accuracy tests, one GB300 DeepSeek-R1 FP4 disaggregated performance test, and one single-GPU MegaMoE CUTEDSL test. Each entry references an NVBug issue.

Changes

Integration test waivers

Layer / File(s) Summary
Add integration test skip waivers
tests/integration/test_lists/waives.txt
Adds skip entries for Cosmos3, Flux, LTX2, Qwen Image, Wan, GB300 DeepSeek-R1 FP4 disaggregated performance, and single-GPU MegaMoE CUTEDSL tests.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

Suggested reviewers: brnguyen2, qijune, bowenfu

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the infrastructure change: waiving 12 failed post-merge cases for the main branch.
Description check ✅ Passed The description explains the waiver purpose, target branch, bug references, requestor, and all 12 affected test cases.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Around line 133-148: Add the repository-standard NVIDIA copyright header at
the top of waives.txt, using the waiver-list parser’s accepted comment syntax
and setting the copyright year to 2026. Preserve all existing waiver entries
unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c01298a2-c929-432a-a2d4-33fafc077a48

📥 Commits

Reviewing files that changed from the base of the PR and between f8190db and 5d1668c.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Comment on lines +133 to +148
examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_nano_t2i_lpips_against_golden SKIP (https://nvbugs/6418815)
examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_nano_t2v_lpips_against_golden SKIP (https://nvbugs/6437341)
examples/visual_gen/test_visual_gen_flux.py::test_flux_accuracy_against_golden[flux1-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_flux.py::test_flux_accuracy_against_golden[flux2-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_ltx2.py::test_ltx2_cuda_graph_trtllm_backend SKIP (https://nvbugs/6463822)
examples/visual_gen/test_visual_gen_ltx2.py::test_ltx2_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[cfg2_ulysses2] SKIP (https://nvbugs/6535765)
examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[ulysses4] SKIP (https://nvbugs/6535765)
examples/visual_gen/test_visual_gen_qwen_image.py::test_qwenimage_feature_accuracy_against_golden[cuda-graph] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_qwen_image.py::test_qwenimage_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan22_t2v_lpips_against_golden SKIP (https://nvbugs/6535765)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan21-nvfp4] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-cuda-graph] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-fp8-blockwise] SKIP (https://nvbugs/6572800)
examples/visual_gen/test_visual_gen_wan.py::test_wan_feature_accuracy_against_golden[wan22-nvfp4] SKIP (https://nvbugs/6572800)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Add the required NVIDIA copyright header.

tests/integration/test_lists/waives.txt is modified, but it has no NVIDIA copyright header. Add the repository-standard header at the top of the file and set the year to 2026. Use the comment form accepted by the waiver-list parser.

As per coding guidelines, source files must contain the NVIDIA copyright header with the year of the latest meaningful modification.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/waives.txt` around lines 133 - 148, Add the
repository-standard NVIDIA copyright header at the top of waives.txt, using the
waiver-list parser’s accepted comment syntax and setting the copyright year to
2026. Preserve all existing waiver entries unchanged.

Source: Coding guidelines

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #64547 [ skip ] triggered by Bot. Commit: 5d1668c Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #64547 [ skip ] completed with state SUCCESS. Commit: 5d1668c
Skipping testing for commit 5d1668c

Link to invocation

@brnguyen2 brnguyen2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — the comments below are optional touch-ups, not blockers.

Verified the 12 entries: test IDs, params, and file paths all resolve against the tests on disk (e.g. flux1-nvfp4/flux2-nvfp4 from the FLUX_MODEL_SPECS×FLUX_FEATURE_PROFILES cross product), no duplicates, and the file stays sorted.

One thing worth a second look before merging: 10 of the 12 are visual_gen *_accuracy_against_golden cases failing in the same post-merge run under a single bug, and per that bug they are first-time failures with an identified first-bad commit — that pattern reads more like one regression than 10 independent flakes. Waiving them all removes the signal; it would be good to make sure the owner of the suspect commit is looped in so the waivers don't sit indefinitely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants