Repository navigation
CollectiveX: restore nccl-ep zero-copy under CUDA graphs and FP8 in the default matrix / CollectiveX：恢复 nccl-ep 在 CUDA graph 下的零拷贝，并让 FP8 重新进入默认矩阵 - #4594
Merged
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…he default matrix #4105 moved nccl-ep onto TensorRT-LLM's configuration and in doing so undid two pieces of NVIDIA's enablement: - Low-latency zero-copy (#3370) was turned off for every graphed row, because TensorRT-LLM dispatches into a plain tensor under capture. Graphed rows, the default, therefore ran the staged dispatch. Low-latency now zero-copies wherever the library can (rank-major, BF16, inside one NVLink/MNNVL domain), graphed or eager. Scale-out, expert-major and DS-FP8 still stage. - FP8 low-latency decode (#3726) left the default matrix through OFF_PATH_PRECISIONS, dropping 9 cases. OFF_PATH_PRECISIONS is empty again and cases.json regains those 9 cases (220 requested, 196 runnable EP). Graphed zero-copy rows no longer match TensorRT-LLM, so they are library (candidate) rows; eager and scale-out low-latency rank-major BF16 rows keep the tensorrt-llm contract. Contract tests and docs follow. 中文：恢复 nccl-ep 在 CUDA graph 下的零拷贝，并让 FP8 重新进入默认矩阵 #4105 让 nccl-ep 采用 TensorRT-LLM 的配置，同时撤销了 NVIDIA 的两项启用工作： - 低延迟零拷贝（#3370）在所有 graph 行上被关闭，因为 TensorRT-LLM 在 capture 下写入普通张量。 默认的 graph 行因此改走 staging dispatch。现在低延迟在库支持的地方都使用零拷贝（rank-major、 BF16、单个 NVLink/MNNVL 域内），graph 与 eager 均如此。scale-out、expert-major 与 DS-FP8 仍走 staging。 - FP8 低延迟 decode（#3726）通过 OFF_PATH_PRECISIONS 被移出默认矩阵，少了 9 个用例。 OFF_PATH_PRECISIONS 重新为空，cases.json 恢复这 9 个用例（共 220 个请求、196 个可运行 EP 用例）。 graph 零拷贝行不再与 TensorRT-LLM 一致，因此标为库行（candidate）；eager 与 scale-out 的低延迟 rank-major BF16 行保留 tensorrt-llm 约定。契约测试与文档同步更新。
…rect its engine labels - COLLX_NCCL_LL_ZERO_COPY=0 stages every low-latency dispatch, so the graphed in-domain rank-major BF16 row can run TensorRT-LLM's graphed staged dispatch again and carries engine_contract tensorrt-llm. A row carries the contract only when its zero-copy choice matches TensorRT-LLM's for the regime (zero-copy eager in-domain, staged under capture or scale-out), so the opt-out makes the eager in-domain row a library row. HT ignores the knob. The row drops -zc and records ll_zero_copy_opt_out in implementation.library_config. Ranks inherit it like the other COLLX_ knobs (srun --export=ALL). - Docs and comments say scale-out, matching _internode (EP larger than the scale-up domain): GB200 and GB300 EP16 span nodes inside one MNNVL domain and zero-copy. methodology.md, methodology-summary.md and the adapter comments no longer call every nccl-ep low-latency row a deployed-engine row. Removed a duplicated comment. - Tests cover the graphed zero-copy candidate row, eager and graphed scale-out, GB-style EP16 inside an MNNVL domain, and the knob. - README: case counts after rebasing onto the MI300X/MI325X change (220 requested, 196 runnable EP, 6 unsupported) and the MoRI/UCCL-EP coverage rows that change implies. 中文：为 nccl-ep 低延迟零拷贝增加关闭开关，并修正其引擎标签 - COLLX_NCCL_LL_ZERO_COPY=0 让所有低延迟 dispatch 走 staging，使 graph 模式下单个 scale-up 域内的 rank-major BF16 行可以再次运行 TensorRT-LLM 的 graph staging dispatch，并带有 engine_contract tensorrt-llm。只有当行的零拷贝选择与 TensorRT-LLM 在该模式下的选择一致时（域内 eager 用零拷贝， capture 下或 scale-out 用 staging），该行才带此契约，因此开启该开关后域内 eager 行成为库行。HT 忽略该开关。该行去掉 -zc，并在 implementation.library_config 中记录 ll_zero_copy_opt_out。各 rank 与其他 COLLX_ 开关一样继承该变量（srun --export=ALL）。 - 文档与注释改用 scale-out，与 _internode（EP 大于 scale-up 域）一致：GB200 与 GB300 的 EP16 跨节点 但位于同一 MNNVL 域内，使用零拷贝。methodology.md、methodology-summary.md 与适配器注释不再把所有 nccl-ep 低延迟行称为已部署引擎行。删除一处重复注释。 - 测试覆盖 graph 零拷贝 candidate 行、eager 与 graph 的 scale-out 行、MNNVL 域内的 GB 式 EP16，以及该开关。 - README：rebase 到 MI300X/MI325X 变更之后的用例数量（220 个请求、196 个可运行 EP、6 个不支持）， 以及该变更带来的 MoRI/UCCL-EP 覆盖行。
Oseltamivir
force-pushed
the
cx-nccl-ep-restore
branch
from
October 28, 2026 06:44
a2e3b39 to
cdd2176
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#4105 moved nccl-ep onto TensorRT-LLM's configuration and in doing so undid two pieces of NVIDIA's enablement. This restores both.
nccl-ep-v02-ll-rm,zero_copy: false). Low-latency now zero-copies wherever the library can, graphed or eager: rank-major BF16 inside one NVLink/MNNVL scale-up domain. Scale-out (EP larger than the scale-up domain; GB200/GB300 EP16 spans nodes but is not scale-out), expert-major and DS-FP8 still stage, as the library requires. High-throughput zero-copy was never affected.OFF_PATH_PRECISIONS, which dropped its 9 cases from the default matrix.OFF_PATH_PRECISIONSis empty again andcases.jsonregains exactly those 9 cases (220 requested, 196 runnable EP after rebasing onto CollectiveX: run the MI300X and MI325X pools on the vLLM ROCm image / CollectiveX：在 vLLM ROCm 镜像上运行 MI300X 与 MI325X 资源池 #4581).tensorrt-llmcontract andproductionmaturity only when its zero-copy choice matches TensorRT-LLM's: zero-copy eager in-domain, staged under capture or scale-out. Graphed zero-copy rows, the new default, are library (candidate) rows.COLLX_NCCL_LL_ZERO_COPY=0stages every low-latency dispatch, so TensorRT-LLM's graphed staged row can still be measured (production,tensorrt-llm). The row drops-zcand recordslibrary_config.ll_zero_copy_opt_out.Contract tests and docs (
nccl-ep.md, methodology, summary, README) follow.Validation
pytest collectivex/tests: 240 passed, 12 skipped.AI model disclosure
claude-opus-5-5, run through Claude Code): the change, tests and this description.中文
摘要
#4105 让 nccl-ep 采用 TensorRT-LLM 的配置，同时撤销了 NVIDIA 的两项启用工作。本 PR 恢复这两项。
nccl-ep-v02-ll-rm，zero_copy: false）。现在低延迟在库支持的地方都使用零拷贝，graph 与 eager 均如此：单个 NVLink/MNNVL scale-up 域内的 rank-major BF16。scale-out（EP 大于 scale-up 域；GB200/GB300 的 EP16 跨节点但不属于 scale-out）、expert-major 与 DS-FP8 按库的要求仍走 staging。高吞吐模式的零拷贝从未受影响。OFF_PATH_PRECISIONS，默认矩阵因此少了 9 个用例。OFF_PATH_PRECISIONS重新为空，cases.json恢复这 9 个用例（rebase 到 CollectiveX: run the MI300X and MI325X pools on the vLLM ROCm image / CollectiveX：在 vLLM ROCm 镜像上运行 MI300X 与 MI325X 资源池 #4581 之后共 220 个请求、196 个可运行 EP 用例）。tensorrt-llm契约与production成熟度。graph 零拷贝行（新的默认配置）是库行（candidate）。COLLX_NCCL_LL_ZERO_COPY=0让所有低延迟 dispatch 走 staging，因此仍可测量 TensorRT-LLM 的 graph staging 行（production、tensorrt-llm）。该行去掉-zc，并记录library_config.ll_zero_copy_opt_out。契约测试与文档（
nccl-ep.md、methodology、摘要、README）同步更新。验证
pytest collectivex/tests：240 通过，12 跳过。