Skip to content

[AgentX] MiniMax-M3.1-Flash GB200 SGLang: store DSpark draft KV in FP8 / [AgentX] MiniMax-M3.1-Flash GB200 SGLang：DSpark draft KV 改用 FP8 #4599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zijiexia wants to merge 3 commits into
base: main
Choose a base branch
from
+10 −0
Open

[AgentX] MiniMax-M3.1-Flash GB200 SGLang: store DSpark draft KV in FP8 / [AgentX] MiniMax-M3.1-Flash GB200 SGLang：DSpark draft KV 改用 FP8 #4599

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d367812
Store DSpark draft KV in FP8 on the GB200 SGLang AgentX curve
zijiexia Oct 28, 2026
84ee06a
Set the changelog PR link
zijiexia Oct 28, 2026
c350a66
Merge remote-tracking branch 'origin/main' into minimaxm31flash-gb200…
zijiexia Oct 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ...enchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/gb200-fp4-mtp/agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ base:
speculative-algorithm: DSPARK
speculative-dspark-block-size: 7
speculative-use-block-verification: true
# FP8 draft KV, as vLLM's DSpark drafter uses; bf16 costs ~10 GB of HBM per rank.
speculative-draft-kv-cache-dtype: fp8_e4m3
Comment on lines +36 to +37

@claude claude Bot Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 After merge, every GB200 MiniMax-M3.1-Flash SGLang point serves the DSpark draft below its shipped precision, which CONTRIBUTING.md's draft-precision rule forbids. The draft-only speculative-draft-kv-cache-dtype: fp8_e4m3 lowers draft KV from the pinned image's bf16 default. Target KV stays at its own default, so the dtype is not applied to both models alike. Fix: remove the draft-only override and keep the framework default draft KV. If FP8 KV is wanted, use an upstream kv-cache-dtype that the pinned image applies to target and draft alike, then reword the changelog entry.

Why this was flagged

agentic.yaml:37 adds speculative-draft-kv-cache-dtype: fp8_e4m3 to the base args, so every override (tp4 c1-c32, tp4_dram c32-c64, tp4_ep4_dram c72/c80) inherits it. The PR's own description says the SGLang DSpark drafter defaults to bf16 KV. The recipe sets no target kv-cache-dtype, so this is a draft-only override that lowers precision. CONTRIBUTING.md:86-87 lists "Dtype or KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint" as forbidden. CONTRIBUTING.md:100-104 says passing evals does not exempt this, because draft-precision losses show up in acceptance rate, which target evals do not measure. So the PR's stated safeguard ("the sweep's eval is what checks this") does not hold. On base, the recipe served the draft at the framework default and was compliant. After merge, the whole rerun curve fails Check 13 (.github/codeowner-signoff-verify-prompt.md:600-606).

Verification: Every point of minimaxm31flash-fp4-gb200-sglang-agentic-dspark (nvidia-master.yaml:5814-5831) uses this recipe's base args. agentic.yaml:37 adds speculative-draft-kv-cache-dtype: fp8_e4m3 under base.roles.agg.args. The recipe sets no target kv-cache-dtype anywhere. CONTRIBUTING.md:86-87 forbids "Dtype or KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default for that checkpoint."

cuda-graph-backend-prefill: breakable
tokenizer-worker-num: 8
watchdog-timeout: 3600
Expand Down
8 changes: 8 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11499,3 +11499,11 @@
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4509
scenario-type: [agentic-coding]
append-only: true

- config-keys:
- minimaxm31flash-fp4-gb200-sglang-agentic-dspark
description:
- "Store the DSpark draft KV cache in FP8 (speculative-draft-kv-cache-dtype fp8_e4m3) instead of the bf16 default, matching the vLLM DSpark drafter; this frees HBM and HiCache host pool space for target KV."
- "Rerun the whole curve so every point uses the FP8 draft KV cache."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4599
scenario-type: [agentic-coding]
Loading