[Klaud Cold] Update DeepSeek-V4.1-Flash FP4 H200 SGLang AgentX DSpark image to nightly-dev-cu13-20261028-586355a8 / [Klaud Cold] 将 DeepSeek-V4.1-Flash FP4 H200 SGLang AgentX DSpark 镜像更新至 nightly-dev-cu13-20261028-586355a8 by Klaud-Cold · Pull Request #4597 · SemiAnalysisAI/InferenceX

Klaud-Cold · 2026-10-28T04:14:29Z

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-cu13-20261013-7224fce1 to lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8.
Baseline: 2026-10-20 · lmsysorg/sglang:nightly-dev-cu13-20261013-7224fce1
Mean latency · Sources: API 1, API 2, API 3, API 4

Point	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP4 EP1 a8dc9d	4,128	31.33	1,593.2	3.99
AgentX c1 TP8 EP1 8ceb83	2,139.51	15.93	1,599.27	3.75
AgentX c2 TP4 EP1 eaa0d5	4,537.76	38.13	1,429.77	4.21
AgentX c2 TP8 EP1 fc923b	2,381.7	19.52	1,259.04	3.99
AgentX c4 TP4 EP1 f2293f	6,296.82	47.54	1,353.14	4.97
AgentX c4 TP8 EP1 e29a5b	3,260.75	24.79	1,352.04	4.6
AgentX c8 TP4 EP1 0eee46	10,476.3	82.15	1,538.47	7.88
AgentX c8 TP8 EP1 a7b50f	5,478.52	42.47	1,557.53	7.05
AgentX c16 TP4 EP1 4cb20a	12,187.27	100.89	6,439.99	18.61
AgentX c16 TP8 EP1 f8a420	6,590.5	53.97	5,500.09	17.19

Eval	Score ↑	Samples
gsm8k/em_strict · c16	97.5%	1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-cu13-20261013-7224fce1 更新为 lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8。
**基线：**2026-10-20 · lmsysorg/sglang:nightly-dev-cu13-20261013-7224fce1
平均延迟 · 来源： API 1, API 2, API 3, API 4；数值及异常说明见上表。
**停止：**随时关闭此 PR 即可立即停止该候选：Klaud 会取消其运行并删除其分支。

…dev-cu13-20261028-586355a8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-28T04:14:37Z

Thanks for the contribution!

Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

**审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
**PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
**合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

Klaud-Cold · 2026-10-28T04:15:59Z

Initial attempt · Passed · Run 40631236971 / attempt 1 · 2026-10-28 06:03 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · c7296e6af320 · Mean latency
Change: At ref a4d6073, pin SGLang nightly-dev-cu13-20261028-586355a8 by digest with recipe flags unchanged; source diff sgl-project/sglang 7224fce1...586355a8 (1135 commits) keeps every used flag, while SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE/LAYOUT become deprecated aliases mapped to the same --engram-config CPU per-rank placement (#46893) and SGLANG_DSV41_REASONING_EFFORT moves to the deepseek-recipe encoder (#45315), validated at boot; SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE stays default False.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP4 EP1 854522	31.96 (N/A)	1,027.76 (N/A)	4.03 (N/A)
AgentX c1 TP8 EP1 301550	16.52 (N/A)	1,007.84 (N/A)	3.72 (N/A)

Note: All rows: Δ N/A: no matched baseline.
Note: AgentX c1 TP8 EP1 301550: 1 request error.

Eval	Score ↑	Samples
gsm8k/em_strict · gsm8k/em_strict - c16	97.42% (-0.08 pp)	1,319 each

Next: Append the perf-changelog entry and dispatch the full AgentX final sweep (TP4/TP8 c1-c16 plus TP4 c16 GSM8K) on h200-dgxc.

中文

初次尝试 · 已通过 · Run 40631236971 / attempt 1 · 2026-10-28 06:03 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · c7296e6af320 · 平均延迟
**变更：**在 ref a4d6073 上，按 digest 固定 SGLang nightly-dev-cu13-20261028-586355a8，配方参数不变；源码对比 sgl-project/sglang 7224fce1...586355a8（1135 个提交）保留所有已用参数，SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE/LAYOUT 变为弃用别名并映射到相同的 --engram-config CPU per-rank 放置（#46893），SGLANG_DSV41_REASONING_EFFORT 移至 deepseek-recipe 编码器（#45315）并在启动时校验；SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE 保持默认 False。实测数值及异常说明见上表。
**下一步：**追加 perf-changelog 条目，并在 h200-dgxc 上调度完整的 AgentX 最终扫描（TP4/TP8 c1-c16 及 TP4 c16 GSM8K）。

Klaud-Cold · 2026-10-28T04:16:15Z

parked · Session 1 is waiting on 40631236971. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40631236971。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

Klaud-Cold · 2026-10-28T06:00:47Z

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 2 个会话将在其规划运行的新运行中继续。

…park image update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-10-28T06:06:28Z

Final full sweep · Failed · Run 40641632969 / attempt 1 · 2026-10-28 08:27 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · 767303780e5b · Mean latency
Change: Final sweep at 7673037 with recipe flags unchanged: 7 of 10 AgentX points and TP4 c16 GSM8K passed; TP4 c4 failed after a healthy server boot because aiperf got HTTP 429 from Hugging Face loading semianalysisai/cc-traces-weka-062126, and fail-fast cancelled TP8 c4/c8.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP4 EP1 a8dc9d	32.17 (+2.7%)	1,017.75 (-36.1%)	4.01 (+0.5%)
AgentX c2 TP4 EP1 eaa0d5	38.31 (+0.5%)	839.76 (-41.3%)	4.14 (-1.7%)
AgentX c4 TP4 EP1 f2293f	N/A	N/A	N/A
AgentX c8 TP4 EP1 0eee46	84.16 (+2.4%)	1,045.75 (-32%)	7.52 (-4.6%)
AgentX c16 TP4 EP1 4cb20a	109.66 (+8.7%)	4,375.73 (-32.1%)	17.16 (-7.8%)
AgentX c1 TP8 EP1 8ceb83	16.59 (+4.2%)	998.46 (-37.6%)	3.72 (-0.8%)
AgentX c2 TP8 EP1 fc923b	20.71 (+6.1%)	752.14 (-40.3%)	3.9 (-2.3%)
AgentX c4 TP8 EP1 e29a5b	N/A	N/A	N/A
AgentX c8 TP8 EP1 a7b50f	N/A	N/A	N/A
AgentX c16 TP8 EP1 f8a420	57.94 (+7.4%)	3,796.8 (-31%)	16.38 (-4.7%)

Note: AgentX c1 TP4 EP1 a8dc9d: 2 request errors.
Note: AgentX c4 TP4 EP1 f2293f: failed.
Note: AgentX c4 TP4 EP1 f2293f, AgentX c4 TP8 EP1 e29a5b, AgentX c8 TP8 EP1 a7b50f: request errors unavailable.
Note: AgentX c4 TP4 EP1 f2293f, AgentX c4 TP8 EP1 e29a5b, AgentX c8 TP8 EP1 a7b50f: Δ N/A: point unavailable.
Note: AgentX c2 TP8 EP1 fc923b: 1 request error.
Note: AgentX c4 TP8 EP1 e29a5b, AgentX c8 TP8 EP1 a7b50f: cancelled.

Eval	Score ↑	Samples
gsm8k/em_strict · gsm8k/em_strict - c16	97.42% (-0.08 pp)	1,319 each

Next: Hugging Face rate limit is infrastructure, not the image: park and let recovery rerun the failed and cancelled jobs of run 40641632969 on the same head after 45 minutes as infrastructure retry 1.

中文

最终完整 sweep · 失败 · Run 40641632969 / attempt 1 · 2026-10-28 08:27 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · 767303780e5b · 平均延迟
**变更：**在 7673037 上运行最终扫描，配方参数不变：10 个 AgentX 点中 7 个及 TP4 c16 GSM8K 通过；TP4 c4 在服务器正常启动后失败，原因是 aiperf 加载 semianalysisai/cc-traces-weka-062126 时收到 Hugging Face HTTP 429 限流，fail-fast 随后取消了 TP8 c4/c8。实测数值及异常说明见上表。
**下一步：**Hugging Face 限流属于基础设施问题而非镜像问题：暂停会话，由恢复流程在 45 分钟后于同一 head 上重跑 run 40641632969 的失败和被取消任务，作为基础设施重试 1。

Klaud-Cold · 2026-10-28T06:06:36Z

parked · Session 2 is waiting on 40641632969. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40641632969。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

github-actions · 2026-10-28T08:22:21Z

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40641632969

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40641632969

Klaud-Cold · 2026-10-28T08:24:31Z

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 3 个会话将在其规划运行的新运行中继续。

Klaud-Cold · 2026-10-28T08:27:31Z

parked · Session 3 is waiting for recovery to rerun the failed jobs of 40641632969 after 2026-10-28 09:12 UTC. Recovery starts the next session once they end.

中文

parked · 第 3 个会话正在等待恢复流程在 2026-10-28 09:12 UTC 后重跑 40641632969 的失败作业。运行结束后，恢复流程将启动下一个会话。

Klaud-Cold · 2026-10-28T11:14:30Z

Final full sweep · Passed · Run 40641632969 / attempt 2 · 2026-10-28 11:14 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · 767303780e5b · Mean latency
Change: Validate the complete updated-image family at this head.

Point	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
AgentX c1 TP4 EP1 a8dc9d	32.17 (+2.7%)	1,017.75 (-36.1%)	4.01 (+0.5%)
AgentX c1 TP8 EP1 8ceb83	16.59 (+4.2%)	998.46 (-37.6%)	3.72 (-0.8%)
AgentX c2 TP4 EP1 eaa0d5	38.31 (+0.5%)	839.76 (-41.3%)	4.14 (-1.7%)
AgentX c2 TP8 EP1 fc923b	20.71 (+6.1%)	752.14 (-40.3%)	3.9 (-2.3%)
AgentX c4 TP4 EP1 f2293f	50.62 (+6.5%)	884.95 (-34.6%)	5 (+0.6%)
AgentX c4 TP8 EP1 e29a5b	24.99 (+0.8%)	881.84 (-34.8%)	4.49 (-2.4%)
AgentX c8 TP4 EP1 0eee46	84.16 (+2.4%)	1,045.75 (-32%)	7.52 (-4.6%)
AgentX c8 TP8 EP1 a7b50f	44.13 (+3.9%)	1,003.09 (-35.6%)	6.63 (-6%)
AgentX c16 TP4 EP1 4cb20a	109.66 (+8.7%)	4,375.73 (-32.1%)	17.16 (-7.8%)
AgentX c16 TP8 EP1 f8a420	57.94 (+7.4%)	3,796.8 (-31%)	16.38 (-4.7%)

Note: AgentX c1 TP4 EP1 a8dc9d: 2 request errors.
Note: AgentX c2 TP8 EP1 fc923b: 1 request error.
Note: AgentX c4 TP8 EP1 e29a5b: 3 request errors.

Eval	Score ↑	Samples
gsm8k/em_strict · c16	97.42% (-0.08 pp)	1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 40641632969 / attempt 2 · 2026-10-28 11:14 UTC
lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8@sha256:b4f09a2ccbcfcc262d49600c3cd7f8581a793ea7d104947257b6a95500953b90 · 767303780e5b · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

Klaud-Cold · 2026-10-28T11:14:41Z

validated · Repairs: unknown · Runs: 40631236971, 40641608731, 40641632969
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：未知 · 运行：40631236971, 40641608731, 40641632969
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

claude · 2026-10-28T11:15:08Z

Claude finished @Klaud-Cold's task in 12s —— View job

LGTM - no blocking issues found

The PR pins the public Docker Hub image lmsysorg/sglang:nightly-dev-cu13-20261028-586355a8 by digest, and the new inferencex-e2e/perf-changelog.yaml entry is at the end of the file. The final full sweep (run 40641632969, attempt 2) passed all 10 points, and GSM8K scored 97.42%, down 0.08 pp from the baseline.
· Branch

claude

Code review found no issues

No high-confidence issues detected in this change.

chore(dsv41flash): update H200 SGLang AgentX DSpark image to nightly-…

a4d6073

…dev-cu13-20261028-586355a8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-project-automation Bot added this to InferenceX Board Oct 28, 2026

github-actions Bot added the klaud-parked Klaud parked until its queued or running benchmark runs end label Oct 28, 2026

github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 28, 2026

github-actions Bot and others added 2 commits October 28, 2026 06:03

Merge main into klaud/auto-1d1b27d97ebf722d-18b8f9b76fb1d4d1

c3a772e

chore(dsv41flash): add perf-changelog entry for H200 SGLang AgentX DS…

7673037

…park image update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 28, 2026

github-actions Bot added klaud-parked Klaud parked until its queued or running benchmark runs end and removed klaud-in-progress Klaud is working on this candidate or will resume it labels Oct 28, 2026

github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 28, 2026

github-actions Bot added klaud-parked Klaud parked until its queued or running benchmark runs end and removed klaud-in-progress Klaud is working on this candidate or will resume it labels Oct 28, 2026

Klaud-Cold marked this pull request as ready for review October 28, 2026 11:14

Klaud-Cold requested a review from a team October 28, 2026 11:14

Klaud-Cold requested review from Ankur-singh, kedarpotdar-nv, nvpohanh and wzhao18 as code owners October 28, 2026 11:14

Klaud-Cold requested a review from RohitNagraj as a code owner October 28, 2026 11:14

github-actions Bot added klaud-finished Klaud verified the full sweep; ready for review and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 28, 2026

claude Bot reviewed Oct 28, 2026

View reviewed changes

Conversation

Klaud-Cold commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

github-actions Bot commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

Klaud-Cold commented Oct 28, 2026

Uh oh!

claude Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Klaud-Cold commented Oct 28, 2026 •

edited

Loading

Klaud-Cold commented Oct 28, 2026 •

edited

Loading

Klaud-Cold commented Oct 28, 2026 •

edited

Loading

claude Bot commented Oct 28, 2026 •

edited

Loading