[zijiexia]

The SGLang DSpark drafter stores its KV cache in bf16 by default. On B300 TP2 that took 21 GB of HBM and 71 GB of the 370 GB HiCache host pool per rank, leaving about 10% less GPU KV than vLLM, whose DSpark drafter uses FP8 KV. At high concurrency the smaller GPU cache lowered the GPU prefix hit rate and cost throughput against vLLM.

This PR sets speculative-draft-kv-cache-dtype: fp8_e4m3 in the base args of the GB200 MiniMax-M3.1-Flash SGLang AgentX recipe, so the freed HBM and host memory go to target KV. The changelog entry is not append-only, so the whole curve reruns on the new recipe. The same change for B300 is in #4524.

Risk: FP8 draft KV can lower real DSpark acceptance. Throughput runs use the golden synthetic AL, so the sweep's eval is what checks this.

中文

SGLang 的 DSpark drafter 默认用 bf16 存 KV cache。在 B300 TP2 上，它每个 rank 占用 21 GB HBM 和 370 GB HiCache host 池中的 71 GB，GPU KV 容量比 vLLM 少约 10%（vLLM 的 DSpark drafter 用 FP8 KV）。高并发时 GPU cache 更小，GPU prefix 命中率下降，吞吐落后于 vLLM。

本 PR 在 GB200 MiniMax-M3.1-Flash SGLang AgentX recipe 的 base args 中设置 speculative-draft-kv-cache-dtype: fp8_e4m3，把省下的 HBM 和 host 内存留给 target KV。changelog 条目不是 append-only，整条曲线会用新 recipe 重跑。B300 的同样改动在 #4524。

风险：FP8 draft KV 可能降低真实的 DSpark 接受率。吞吐测试使用 golden synthetic AL，因此需要由 sweep 中的 eval 来验证。

AI model disclosure

Prepared with Claude Code using claude-opus-5-5. Reviewed by the human author before submission.

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

🔴 After merge, every GB200 MiniMax-M3.1-Flash SGLang point serves the DSpark draft below its shipped precision, which CONTRIBUTING.md's draft-precision rule forbids. The draft-only speculative-draft-kv-cache-dtype: fp8_e4m3 lowers draft KV from the pinned image's bf16 default. Target KV stays at its own default, so the dtype is not applied to both models alike. Fix: remove the draft-only override and keep the framework default draft KV. If FP8 KV is wanted, use an upstream kv-cache-dtype that the pinned image applies to target and draft alike, then reword the changelog entry.

Why this was flagged

agentic.yaml:37 adds speculative-draft-kv-cache-dtype: fp8_e4m3 to the base args, so every override (tp4 c1-c32, tp4_dram c32-c64, tp4_ep4_dram c72/c80) inherits it. The PR's own description says the SGLang DSpark drafter defaults to bf16 KV. The recipe sets no target kv-cache-dtype, so this is a draft-only override that lowers precision. CONTRIBUTING.md:86-87 lists "Dtype or KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint" as forbidden. CONTRIBUTING.md:100-104 says passing evals does not exempt this, because draft-precision losses show up in acceptance rate, which target evals do not measure. So the PR's stated safeguard ("the sweep's eval is what checks this") does not hold. On base, the recipe served the draft at the framework default and was compliant. After merge, the whole rerun curve fails Check 13 (.github/codeowner-signoff-verify-prompt.md:600-606).

Verification: Every point of minimaxm31flash-fp4-gb200-sglang-agentic-dspark (nvidia-master.yaml:5814-5831) uses this recipe's base args. agentic.yaml:37 adds speculative-draft-kv-cache-dtype: fp8_e4m3 under base.roles.agg.args. The recipe sets no target kv-cache-dtype anywhere. CONTRIBUTING.md:86-87 forbids "Dtype or KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint."

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40634228473

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40634228473

[claude]

Code review found no new issues

No new issues were found in this update; 1 finding from earlier reviews is still open above.

Still open from earlier reviews (1):

• 🔴 inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/gb200-fp4-mtp/agentic.yaml:37 — After merge, every GB200 MiniMax-M3.1-Flash SGLang point serves the DSpark draft below its shipped precision, which CON…

If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (minimaxm31flash-fp4-gb200-sglang-agentic-dspark) will happen in sgl-project/sglang#47027, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (84ee06a11) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#47027 has been merged. This PR's AgentX config (minimaxm31flash-fp4-gb200-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (84ee06a11) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.