Skip to content

[https://nvbugs/6428087][fix] Extend the PP send payload with the flag (backward-compatible 2/3-tuple… - #16145

Open
trtllm-agent wants to merge 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6428087
Open

[https://nvbugs/6428087][fix] Extend the PP send payload with the flag (backward-compatible 2/3-tuple…#16145
trtllm-agent wants to merge 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6428087

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Root cause: PP send/recv drops SampleStateTorch.use_host_stop_criteria, so non-last PP ranks default to False and call process_draft_tokens with an empty finish_reasons list from the last rank's host fast path.
  • Fix: Extend the PP send payload with the flag (backward-compatible 2/3-tuple handling) so all PP ranks pick the same update branch; remove the waiver.
  • Automated fix generated by repair-bot

Test plan

  • Verify fix on the same GPU type as the original failure
  • Check for regressions in related tests

Links

Dev Engineer Review

  • Propagates SampleStateTorch.use_host_stop_criteria across pipeline-parallel ranks.
  • Supports both legacy 2-tuple payloads and new 3-tuple payloads.
  • Removes the waiver for NVBug 6428087 and updates the related waiver reference to NVBug 6427411.
  • The changes are consistent with the reported bug fix. No public API changes are introduced.

QA Engineer Review

  • Only tests/integration/test_lists/waives.txt changed.
  • Removed the waived TestDeepSeekV3Lite::test_nvfp4_4gpus CUTEDSL/tp2pp2 configuration linked to NVBug 6428087.
  • Added the revised configuration linked to NVBug 6427411.
  • No test-db/ or qa/ files were modified.
  • Verdict: needs follow-up because CBTS coverage data is unavailable.

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4a1bd0c0-f6bb-49a1-9a27-164e0a57c707

📥 Commits

Reviewing files that changed from the base of the PR and between 1cef02e and c180e55.

📒 Files selected for processing (2)
  • tensorrt_llm/_torch/pyexecutor/py_executor.py
  • tests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt
🚧 Files skipped from review as they are similar to previous changes (1)
  • tensorrt_llm/_torch/pyexecutor/py_executor.py

Walkthrough

The PR extends the SAMPLE_STATE PP ring broadcast to propagate use_host_stop_criteria alongside existing state data. It also replaces a DeepSeekV3Lite test waiver configuration and NVBUG reference.

Changes

PP ring broadcast and test waiver

Layer / File(s) Summary
Propagate use_host_stop_criteria in PP ring broadcast
tensorrt_llm/_torch/pyexecutor/py_executor.py
The send path adds use_host_stop_criteria to the SAMPLE_STATE payload. The receive path applies the value to sample_state when present.
Update DeepSeekV3Lite waiver
tests/integration/test_lists/waives.txt
The waiver uses a different CUTEDSL/tp2pp2 configuration and changes the linked NVBUG from 6428087 to 6427411.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: qijune, bowenfu, yingguo-trt

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the bug fix and the PP payload change.
Description check ✅ Passed The description explains the root cause and fix, lists test coverage, and links the NVBug; the template checklist is not included but is non-critical.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@trtllm-agent

Copy link
Copy Markdown
Collaborator Author

NVBug 6428087 is closed as Bug - Fixed. The linked bug appears resolved elsewhere or for a reason that does not prove this PR is redundant. This PR should be judged on its own merits; repair-bot is not auto-closing it.

The greedy host stop-criteria optimization (PR NVIDIA#15920) added a per-batch
flag `use_host_stop_criteria` on `SampleStateTorch` that gates whether
`update_requests` uses the host fast path or calls
`process_draft_tokens`. In the PP execution loop, `SampleStateTorch` is
constructed on non-last PP ranks via `_forward_step_inter_pp` without
setting this flag, and only `sample_state.host` and per-request result
diffs are shipped via ring send/recv from the last PP rank. Consequently,
when the last PP rank determined `use_host_stop_criteria=True` and
therefore skipped writing finish_reasons, earlier PP ranks still saw the
default False and entered `process_draft_tokens`, which then indexed an
empty finish_reasons list and raised
`IndexError: list index out of range` from `finish_if_reason`.

Extend the PP send payload with the flag (backward-compatible: recv
accepts both 2- and 3-tuple payloads, and send only appends the flag when
the sampler exposes the attribute) so that all ranks pick the same
branch. Also remove the associated waiver.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants