[https://nvbugs/6428087][fix] Extend the PP send payload with the flag (backward-compatible 2/3-tuple… - #16145
[https://nvbugs/6428087][fix] Extend the PP send payload with the flag (backward-compatible 2/3-tuple…#16145trtllm-agent wants to merge 2 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThe PR extends the ChangesPP ring broadcast and test waiver
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
NVBug 6428087 is closed as Bug - Fixed. The linked bug appears resolved elsewhere or for a reason that does not prove this PR is redundant. This PR should be judged on its own merits; repair-bot is not auto-closing it. |
The greedy host stop-criteria optimization (PR NVIDIA#15920) added a per-batch flag `use_host_stop_criteria` on `SampleStateTorch` that gates whether `update_requests` uses the host fast path or calls `process_draft_tokens`. In the PP execution loop, `SampleStateTorch` is constructed on non-last PP ranks via `_forward_step_inter_pp` without setting this flag, and only `sample_state.host` and per-request result diffs are shipped via ring send/recv from the last PP rank. Consequently, when the last PP rank determined `use_host_stop_criteria=True` and therefore skipped writing finish_reasons, earlier PP ranks still saw the default False and entered `process_draft_tokens`, which then indexed an empty finish_reasons list and raised `IndexError: list index out of range` from `finish_if_reason`. Extend the PP send payload with the flag (backward-compatible: recv accepts both 2- and 3-tuple payloads, and send only appends the flag when the sampler exposes the attribute) so that all ranks pick the same branch. Also remove the associated waiver. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
c9b9bea to
c180e55
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Summary
Test plan
Links
Dev Engineer Review
SampleStateTorch.use_host_stop_criteriaacross pipeline-parallel ranks.QA Engineer Review
tests/integration/test_lists/waives.txtchanged.TestDeepSeekV3Lite::test_nvfp4_4gpusCUTEDSL/tp2pp2 configuration linked to NVBug 6428087.test-db/orqa/files were modified.