Repository navigation
perf(amd): DSv4.1-Flash ATOM AgentX DPA + prefill TBO from c32 on the 1022 nightly / DSv4.1-Flash ATOM AgentX 更新至 1022 nightly 并从 c32 起开启 DPA + prefill TBO - #3965
Conversation
…ightly Move dsv41flash-fp4-mi355x-atom-agentic-dspark to rocm/atom-dev:nightly_202610131505 (ATOM 661ef7661, with ROCm/ATOM#2376 V4.1 DPA and prefill TBO). Run TP2 c32/64/128/256 and TP4 c32 with DP attention, prefill TBO, DP session affinity and the prefill delayer; drop TP2 c6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
yhl-amd
added
the
full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - { tp: 2, ep: 1, dp-attn: false, kv-offloading: none, spec-decoding: draft_model, conc-list: [1, 2, 4, 8, 16, 128, 256], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/atom/mi355x-fp4-mtp/agentic.yaml } | ||
| - { tp: 2, ep: 1, dp-attn: true, kv-offloading: none, spec-decoding: draft_model, conc-list: [32, 64, 128, 256], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/atom/mi355x-fp4-mtp/agentic.yaml } |
There was a problem hiding this comment.
🔴 Operators lose the ability to select the new tp2 c128/c256 DP-attention points individually via test-config --exp-names, which now errors out for them. The new dp-attn:true row (amd-master.yaml:1536, conc-list includes 128, 256) overlaps the plain-TP row (:1535, now also listing 128, 256) at tp=2. Single-node exp-name generation in infx/matrix/generate.py:1196-1200 builds the exp-name without dp-attn, so both rows produce the identical exp-name dsv41flash_tp2_conc128_kvnone_spec-draft_model. filter_exp_names (generate.py:1572) then raises "Experiment name(s) matched multiple rows" for that name, unlike multinode's multinode_agentic_exp_name which appends a dpa tag. Fix: make single-node agentic exp-name include dp-attn (e.g. append a dpa suffix when Fields.DP_ATTN is true), covering every tp/conc pair where a plain and DP-attention row can overlap.
Why this was flagged
Trigger: running python -m infx.matrix.generate test-config --config-keys dsv41flash-fp4-mi355x-atom-agentic-dspark --exp-names dsv41flash_tp2_conc128_kvnone_spec-draft_model (the documented per-point validation workflow in .github/workflows/README.md) after this PR. generate.py:1196-1200 computes identical exp-name strings for the tp2/conc128 plain-TP row (amd-master.yaml:1535) and the tp2/conc128 DP-attention row (amd-master.yaml:1536), because dp-attn is not part of the formula. filter_exp_names (generate.py:1572) counts 2 matches and raises ValueError('Experiment name(s) matched multiple rows'), so the new c128/c256 points this PR adds cannot be exercised individually. On base, this entry had no tp/conc pair shared between dp-attn true/false rows, so exp-names there were unique; actual per-job result files stay distinct (benchmark-tmpl.yml:254 embeds dpa in RESULT_FILENAME_BASE), so only exp-name-based selection/identification breaks, not result storage.
Verification: The single-node agentic exp-name formula at inferencex-e2e/infx/matrix/generate.py:1196-1200 encodes only tp, conc, kv-offload and spec, not dp-attn. amd-master.yaml:1535 (dp-attn: false) and :1536 (dp-attn: true) both have tp=2, kv-offloading: none, spec: draft_model, and overlap at conc 128 and 256, producing the identical exp-name dsv41flash_tp2_conc128_kvnone_spec-draft_model (and conc256).
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40362093275 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40362093275 |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/atom/mi355x-fp4-mtp/agentic.yaml # inferencex-e2e/configs/amd-master.yaml # inferencex-e2e/perf-changelog.yaml
yhl-amd
changed the title
…1013-dpa # Conflicts: # inferencex-e2e/perf-changelog.yaml
rocm/atom-dev:nightly_202610221638 (sha256:32c81e56e468...; ATOM 0.1.9.dev11+gce46f6877, ROCm 7.2.4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
yhl-amd
changed the title
… without TBO Merge origin/main, move the image to rocm/atom-dev:nightly_202610231458 (sha256:05322b86...) and drop --enable-tbo from the DP-attention points: TBO does not support SWA bounded replay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use the ATOM 2c772a5 release image (ROCm/ATOM run 39986380362, sha256:01a19e8b...) instead of nightly_202610231458. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
max-num-seqs 128 queued half of the 256 sessions (TTFT 490 s); raise it to 256 and capture a 256 decode graph on that point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…seqs as B300 vLLM Move to the ATOM 11a2d56 release image (ROCm/ATOM run 40072622970, sha256:42716923...) and set max-num-seqs per point (per DP rank) like the B300 vLLM AgentX recipe: 1 at c1, 2x concurrency for c2-c16, concurrency from c32 up; capture sizes stop at each point's max-num-seqs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1013-dpa # Conflicts: # inferencex-e2e/perf-changelog.yaml
max-num-seqs is twice the concurrency at c1 too (TP2 and TP4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop every max-num-seqs setting; decode graphs go back to the base list (sizes through 32 plus 48, 64, 128), with 256 added on the c256 points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…OM AgentX on nightly_202610261558 Image rocm/atom-dev:nightly_202610261558 (ATOM 713eb06, digest sha256:57477e7a...) adds ROCm/ATOM#2696/#2697, which fix the hang every DP-attention point hit on nightly_202610240802, and #2719 (trace labels). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ith FP8 KV, TP and DEP2
Moves
dsv41flash-fp4-mi355x-atom-agentic-dsparkto the 2026-10-22 ATOM nightly and runs DeepSeek-V4.1 DP attention with prefill TBO from concurrency 32.rocm/atom-dev:nightly_202610121516→rocm/atom-dev:nightly_202610221638(digestsha256:32c81e56e4680065b6ea3e0ca8363a00cd3ccb932d8990e83d799b8bbb58a4b3; ATOM0.1.9.dev11+gce46f6877, ROCm 7.2.4), in the master config and the AToMesh frontend image. It adds feat(v41): enable DPA and prefill TBO ROCm/ATOM#2376 (V4.1 DP attention and prefill TBO), Fix V4 FP4 graph replay with fixed-capacity metadata ROCm/ATOM#2543 and fix(v4): FP4 decode bounds drop the DP-padded empty tail ROCm/ATOM#2556 (FP4 fixes for DP-padded batches), [DP] Balance DP-attention prefill steps by estimated chunk cost ROCm/ATOM#2551 (DP-attention prefill balanced by chunk cost) perf(v41): run decoder SWA bounded replay under DP attention ROCm/ATOM#2569 (decoder SWA bounded replay under DP attention), [Scheduler] Take prompt hashing and the prefix walk off the GPU's critical path ROCm/ATOM#2588 and [Scheduler] #2588 follow-ups: stop the cold prefix walk at its first miss, ship prompt chains beside the seqs ROCm/ATOM#2599 (prompt hashing and the prefix walk off the GPU critical path), [ATOM] Enable DPA comm-fused MoE reduce-scatter with compact ragged gathers ROCm/ATOM#2429 (DP-attention comm-fused MoE reduce-scatter) and perf: reuse speculative output buffers and combine status D2H ROCm/ATOM#2670 (reused speculative output buffers).--enable-dp-attention --enable-tbo. They set:GPU_MAX_HW_QUEUES=5;ATOM_DP_SESSION_AFFINITY=1, with AIPerf sending its session id (AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID=1);--dp-load-balance least_tokenswithATOM_DP_LB_REQ_EQUIV=512. ATOM's defaultleast_requestsignoresATOM_DP_LB_REQ_EQUIV;ATOM_ENABLE_PREFILL_DELAYER=1andATOM_PREFILL_DECODE_INTERVAL=10.max-num-seqsstays 128 on every point; other server settings are unchanged.14 points in total.
中文
把
dsv41flash-fp4-mi355x-atom-agentic-dspark换成 10-22 的 ATOM nightly，并从并发 32 开始跑 DeepSeek-V4.1 的 DP attention + prefill TBO。rocm/atom-dev:nightly_202610121516→rocm/atom-dev:nightly_202610221638（ATOM0.1.9.dev11+gce46f6877，ROCm 7.2.4），包含 feat(v41): enable DPA and prefill TBO ROCm/ATOM#2376（V4.1 DPA 和 prefill TBO）、Validate official vLLM Router with srt-slurm #2543、Runner reconciliation #2556、[AMD] Add kimik3-fp4-mi355x-vllm-agentic-dspark (AgentX, conc 1/2/4/8) #2551（DPA 下按成本平衡 prefill）、Fix Kimi K3 B200 AIPerf metrics configuration #2569（DPA 下也启用 decoder SWA bounded replay）、[CI] Add weekly overview snapshot sweep / [CI] 新增每周 overview snapshot 定时 sweep #2588 和 [Power] feat: stamp whole-deployment semantics with a schema version / 为功耗指标加盖整机语义版本号 #2599（prompt hash 与前缀查找移出 GPU 关键路径）、[Klaud Cold] qwen3.5-fp8-mi300x-sglang-agentic-mtp: first Qwen3.5 MI300X AgentX recipe, MTP with golden-AL simulated acceptance / MI300X 首个 Qwen3.5 AgentX 配方（MTP + 黄金 AL 模拟接受） #2429（DPA 下通信融合的 MoE reduce-scatter）和 feat(agentx): add Kimi-K3 FP4 MI355X ATOM DSpark recipe #2670（复用投机解码输出缓冲）。--enable-dp-attention --enable-tbo。设置：GPU_MAX_HW_QUEUES=5；ATOM_DP_SESSION_AFFINITY=1，AIPerf 会发送 session id；--dp-load-balance least_tokens配合ATOM_DP_LB_REQ_EQUIV=512。ATOM 默认的least_requests不读ATOM_DP_LB_REQ_EQUIV；ATOM_ENABLE_PREFILL_DELAYER=1，ATOM_PREFILL_DECODE_INTERVAL=10。max-num-seqs都保持 128，其它 server 参数不变。共 14 个点。
AI model disclosure
claude-opus-5-5): made the config, recipe and changelog changes and prepared this PR.🤖 Generated with Claude Code