Skip to content

[Klaud Cold] Update DeepSeek-V4-Pro FP4 MI355X ATOM AgentX DSpark image to nightly_202610261558 / [Klaud Cold] 将 DeepSeek-V4-Pro FP4 MI355X ATOM AgentX DSpark 镜像更新至 nightly_202610261558 #4577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Klaud-Cold wants to merge 4 commits into from
+12 −18
Closed

[Klaud Cold] Update DeepSeek-V4-Pro FP4 MI355X ATOM AgentX DSpark image to nightly_202610261558 / [Klaud Cold] 将 DeepSeek-V4-Pro FP4 MI355X ATOM AgentX DSpark 镜像更新至 nightly_202610261558 #4577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e7060af
perf(amd): update DSv4-Pro FP4 MI355X ATOM AgentX image to nightly_20…
github-actions[bot] Oct 27, 2026
09dc099
fix(amd): disable EPLB on DSv4-Pro FP4 MI355X ATOM DEP points
github-actions[bot] Oct 27, 2026
4211e6b
Merge remote-tracking branch 'origin/main' into klaud/auto-4565dac9cf…
github-actions[bot] Oct 28, 2026
591909c
perf-changelog: add DSv4-Pro FP4 MI355X ATOM AgentX image update entry
github-actions[bot] Oct 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 3 additions & 17 deletions ...rencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv4/atom/mi355x-fp4-mtp/agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ base:
gpus_per_node: 8
frontend:
type: atomesh
container_image: rocm/atom-dev@sha256:ef65fed021823c29d37c636101e6242b7b97b970f5e3f5e8badd672f73935d61
container_image: rocm/atom-dev@sha256:57477e7a4b5156fcc7b21bab082eba98577752465f58479ac2ead8514f882a61
enable_multiple_frontends: false
observability:
enabled: false
Expand Down Expand Up @@ -55,7 +55,8 @@ base:

# One variant per point. KV is FP8 at every point. DEP pins sessions to one
# DP-attention rank so later turns reach their prefix KV, and captures every
# small per-rank decode batch.
# small per-rank decode batch. EPLB stays off: its first rebalance hangs the
# DEP8 server under ATOM's routing rewrite (ROCm/ATOM#2725).
override_tp8_c1:
roles:
agg:
Expand Down Expand Up @@ -118,8 +119,6 @@ override_tp8_c48_dep:
dp-load-balance: least_tokens
moe-backend: mega
cudagraph-capture-sizes: '[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,32,48,64,128]'
enable-eplb: true
eplb-config: '{"load_window_size":100,"rebalance_interval":200}'
env:
ATOM_DP_SESSION_AFFINITY: '1'
ATOM_DP_LB_REQ_EQUIV: '512'
Expand All @@ -129,7 +128,6 @@ override_tp8_c48_dep:
ATOM_MEGA_MASK_PAD_ROWS: '1'
ATOM_MEGA_DECODE_MTPR: '256'
AITER_MEGA_FIXED_SLOT_MAX_MTPR: '511'
ATOM_EPLB_MAX_REBALANCES: '1'
benchmark:
env:
CONC: '48'
Expand Down Expand Up @@ -175,8 +173,6 @@ override_tp8_c96_dep:
dp-load-balance: least_tokens
moe-backend: mega
cudagraph-capture-sizes: '[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,32,48,64,128,256,512]'
enable-eplb: true
eplb-config: '{"load_window_size":100,"rebalance_interval":200}'
env:
ATOM_DP_SESSION_AFFINITY: '1'
ATOM_DP_LB_REQ_EQUIV: '512'
Expand All @@ -186,7 +182,6 @@ override_tp8_c96_dep:
ATOM_MEGA_MASK_PAD_ROWS: '1'
ATOM_MEGA_DECODE_MTPR: '512'
AITER_MEGA_FIXED_SLOT_MAX_MTPR: '1023'
ATOM_EPLB_MAX_REBALANCES: '1'
benchmark:
env:
CONC: '96'
Expand All @@ -205,8 +200,6 @@ override_tp8_c128_dep:
dp-load-balance: least_tokens
moe-backend: mega
cudagraph-capture-sizes: '[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,32,48,64,128]'
enable-eplb: true
eplb-config: '{"load_window_size":100,"rebalance_interval":200}'
env:
ATOM_DP_SESSION_AFFINITY: '1'
ATOM_DP_LB_REQ_EQUIV: '512'
Expand All @@ -216,7 +209,6 @@ override_tp8_c128_dep:
ATOM_MEGA_MASK_PAD_ROWS: '1'
ATOM_MEGA_DECODE_MTPR: '512'
AITER_MEGA_FIXED_SLOT_MAX_MTPR: '1023'
ATOM_EPLB_MAX_REBALANCES: '2'
benchmark:
env:
CONC: '128'
Expand Down Expand Up @@ -258,8 +250,6 @@ override_tp8_c256_dep_lmcache:
dp-load-balance: least_tokens
moe-backend: mega
cudagraph-capture-sizes: '[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,32,48,64,128]'
enable-eplb: true
eplb-config: '{"load_window_size":100,"rebalance_interval":200}'
env:
ATOM_DP_SESSION_AFFINITY: '1'
ATOM_DP_LB_REQ_EQUIV: '512'
Expand All @@ -269,7 +259,6 @@ override_tp8_c256_dep_lmcache:
ATOM_MEGA_MASK_PAD_ROWS: '1'
ATOM_MEGA_DECODE_MTPR: '512'
AITER_MEGA_FIXED_SLOT_MAX_MTPR: '1023'
ATOM_EPLB_MAX_REBALANCES: '4'
ATOM_KV_OFFLOAD: lmcache_mp
# Braces doubled: srtctl expands {node}/{node_id} in env values with str.format_map.
ATOM_KV_OFFLOAD_EXTRA_CONFIG: '{{"lmcache.mp.host": "tcp://127.0.0.1", "lmcache.mp.port": 8750, "lmcache.mp.tp_rank_collapse": true}}'
Expand Down Expand Up @@ -317,8 +306,6 @@ override_tp8_c384_dep_lmcache:
dp-load-balance: least_tokens
moe-backend: mega
cudagraph-capture-sizes: '[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,32,48,64,72,80,88,96,104,112,120,128]'
enable-eplb: true
eplb-config: '{"load_window_size":100,"rebalance_interval":200}'
env:
ATOM_DP_SESSION_AFFINITY: '1'
ATOM_DP_LB_REQ_EQUIV: '512'
Expand All @@ -328,7 +315,6 @@ override_tp8_c384_dep_lmcache:
ATOM_MEGA_MASK_PAD_ROWS: '1'
ATOM_MEGA_DECODE_MTPR: '1024'
AITER_MEGA_FIXED_SLOT_MAX_MTPR: '2047'
ATOM_EPLB_MAX_REBALANCES: '5'
ATOM_KV_OFFLOAD: lmcache_mp
# Braces doubled: srtctl expands {node}/{node_id} in env values with str.format_map.
ATOM_KV_OFFLOAD_EXTRA_CONFIG: '{{"lmcache.mp.host": "tcp://127.0.0.1", "lmcache.mp.port": 8750, "lmcache.mp.tp_rank_collapse": true}}'
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1016,7 +1016,7 @@ dsv4-fp4-mi355x-vllm-agentic-mtp:
# 1019 GB budget of dram-utilization 0.34. Every point runs ATOM's bounded
# state replay with prompt-end state checkpoints only.
dsv4-fp4-mi355x-atom-agentic-mtp:
image: rocm/atom-dev:nightly_202610141644
image: rocm/atom-dev:nightly_202610261558@sha256:57477e7a4b5156fcc7b21bab082eba98577752465f58479ac2ead8514f882a61
model: deepseek-ai/DeepSeek-V4-Pro-0813
model-prefix: dsv4
runner: cluster:mi355x-amds
Expand Down
8 changes: 8 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11480,3 +11480,11 @@
- "Update the Muse Spark Open B300 curve to lmsysorg/sglang:nightly-dev-20261021-c1c287a4 and set SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0 at every concurrency; run TP8/EP8 at concurrency 1, 4, 8, 10, 14, 16, 20, 24 and TP4/EP4 HiCache at concurrency 16, 28, 32, 40, 44, 48."
- "Use shortest-prefill-first at TP8 concurrency 10-24, stream interval 20 at TP8 concurrency 4, 20, 24 and TP4 concurrency 28-48, SGLANG_TRTLLM_MHA_DECODE_SEQ_LEN_SPLITS=4 at TP4 concurrency 40-48, and at concurrency 44 and 48 request admission 24 and 26 with FCFS, no prefill-decode interval, HiCache write_back and SWA/full ratio 0.1."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4417

- config-keys:
- dsv4-fp4-mi355x-atom-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Update the ATOM image from nightly_202610141644 to nightly_202610261558 and disable EPLB on the DEP points."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4577
Loading