Skip to content

[AgentX] MiniMax-M3.1-Flash B200 SGLang: store DSpark draft KV in FP8 / [AgentX] MiniMax-M3.1-Flash B200 SGLang：DSpark draft KV 改用 FP8 #4598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
functionstackx merged 4 commits into from
Oct 28, 2026
Merged

[AgentX] MiniMax-M3.1-Flash B200 SGLang: store DSpark draft KV in FP8 / [AgentX] MiniMax-M3.1-Flash B200 SGLang：DSpark draft KV 改用 FP8 #4598

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6a7bb46
Store DSpark draft KV in FP8 on the B200 SGLang AgentX curve
zijiexia Oct 28, 2026
0185c53
Set the changelog PR link
zijiexia Oct 28, 2026
bb3cf1b
Merge remote-tracking branch 'origin/main' into minimaxm31flash-b200-…
zijiexia Oct 28, 2026
10a66f0
Merge remote-tracking branch 'origin/main' into minimaxm31flash-b200-…
zijiexia Oct 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ...benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/b200-fp4-mtp/agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ base:
speculative-algorithm: DSPARK
speculative-dspark-block-size: 7
speculative-use-block-verification: true
# FP8 draft KV, as vLLM's DSpark drafter uses; bf16 costs ~10 GB of HBM per rank.
speculative-draft-kv-cache-dtype: fp8_e4m3
Comment on lines +36 to +37

@claude claude Bot Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Every point of this B200 SGLang submission now runs the DSpark draft cheaper than the pinned image's default, which CONTRIBUTING.md forbids. The override sets the draft KV to fp8_e4m3, while the PR says SGLang's drafter default is bf16. The target KV dtype stays unset. CONTRIBUTING.md:86-87 forbids "KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default". Fix: drop speculative-draft-kv-cache-dtype and keep the image's default draft KV. Free HBM some other way, for example a target kv-cache-dtype that the draft inherits consistently.

Why this was flagged

The recipe base args at agentic.yaml:37 apply to every override (tp4_c1 through tp4_ep4_dram_c80), and nvidia-master.yaml:5635-5639 binds all of them for minimaxm31flash-fp4-b200-sglang-agentic-dspark. The recipe sets no target kv-cache-dtype, so only the draft's KV is quantized. That is not the "applied consistently to target and draft" allowance at CONTRIBUTING.md:60-62. On the base branch the draft KV uses the pinned image's default, bf16 per the PR's own account. After this change, all published points come from a draft run below shipped precision. CONTRIBUTING.md:100-104 says matching an upstream recipe (here, vLLM's FP8 drafter) or passing evals does not exempt this. It also says synthetic AgentX acceptance hides the hit. These throughput runs use golden synthetic AL, so the acceptance loss the PR names as a risk never shows up in the measured numbers. The codeowner Check 13 should FAIL this as a demonstrated draft-precision violation.

Verification: In agentic.yaml, line 37 of the diff adds speculative-draft-kv-cache-dtype: fp8_e4m3 under base.roles.agg.args, so all 13 overrides inherit it. No override sets a target kv-cache-dtype. The PR description says the default is bf16. So this is a draft-only override that sets precision below the framework default. CONTRIBUTING.md:86-87 forbids exactly that.

cuda-graph-backend-prefill: breakable
tokenizer-worker-num: 8
watchdog-timeout: 3600
Expand Down
8 changes: 8 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11507,3 +11507,11 @@
- "Rerun the whole curve so it is measured on one recipe revision."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4600
scenario-type: [agentic-coding]

- config-keys:
- minimaxm31flash-fp4-b200-sglang-agentic-dspark
description:
- "Store the DSpark draft KV cache in FP8 (speculative-draft-kv-cache-dtype fp8_e4m3) instead of the bf16 default, matching the vLLM DSpark drafter; this frees HBM and HiCache host pool space for target KV."
- "Rerun the whole curve so every point uses the FP8 draft KV cache."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4598
scenario-type: [agentic-coding]
Loading