Repository navigation
Use latest configs for MiniMax M3.1 GB200 AgentX / 更新 MiniMax M3.1 GB200 AgentX 配置 - #4464
Conversation
|
Thanks for the contribution!
中文感谢你的贡献!
|
| # (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c). | ||
| minimaxm31flash-fp4-gb200-vllm-agentic-dspark: | ||
| image: vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347628973fb80519a919a2f2740b6645b62cba3fe2bd87b25f111bd0e948 | ||
| image: vllm/vllm-openai:minimax-m31-flash |
There was a problem hiding this comment.
🟡 (optional) GB200 AgentX runs now pull whatever image the floating minimax-m31-flash tag points to, so results can change without any config change. The digest pin was dropped at nvidia-master.yaml:5621. The B200, H100 and H200 siblings still pin the same tag to @ sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c. Fix: pin the image to a digest, e.g. vllm/vllm-openai:minimax-m31-flash@ sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c (the multiarch manifest with the arm64 cu134 build), or keep the previous arm64 digest.
Why this was flagged
The base pinned vllm/vllm-openai:minimax-m31-flash-arm64-cu134@ sha256:5d76347628973fb80519a919a2f2740b6645b62cba3fe2bd87b25f111bd0e948. HEAD uses the bare tag vllm/vllm-openai:minimax-m31-flash (nvidia-master.yaml:5621). This is a model-named tag, not a version, and the vLLM day-zero tags get re-pushed when fixes land. Every GB200 sweep, and any /use <run_id> reuse, then resolves whatever the tag points to when the job pulls. So the published AgentX numbers no longer map to a fixed engine build. Sibling entries at nvidia-master.yaml:5281, 5304 and 5601 still pin the same tag by digest, so only this key loses reproducibility. CONTRIBUTING.md's compliance rules are written against "the pinned upstream image", and with a floating tag that baseline is undefined.
Verification: Trigger: Docker Hub's vllm/vllm-openai:minimax-m31-flash tag is re-pushed after this merges. nvidia-master.yaml:5621 changes to the bare image: vllm/vllm-openai:minimax-m31-flash. enroot_uri in infx/launch/backends/slurm/squash.py:76-87 keeps an immutable digest only when "@ sha256:" is present. GB200 run events therefore lose the engine provenance they had on the base branch.
| kv-transfer-config: '{"kv_connector":"SimpleCPUOffloadConnector","kv_role":"kv_both","kv_connector_extra_config":{"cpu_bytes_to_use":450000000000,"lazy_offload":true}}' | ||
| benchmark: | ||
| env: |
There was a problem hiding this comment.
🟡 nit (optional): Nothing ties the hard-coded 450 GB DRAM offload budget to the matrix value. Published results could report a different DRAM allocation than the server actually used. The overrides set cpu_bytes_to_use 450000000000 but omit KV_OFFLOADING and TOTAL_CPU_DRAM_GB from benchmark.env. So the binder check at single_node.py:144-146 is skipped. Fix: add KV_OFFLOADING: dram and TOTAL_CPU_DRAM_GB: '450' to both overrides' benchmark.env, as the B200 sibling does. Then a drift fails the bind instead of mislabeling results.
Why this was flagged
The matrix derives total-cpu-dram-gb in agentic_dram_offload_gb (generate.py:421-459) from dram-utilization 1.0, gb200-nv available-cpu-dram-mib 860_160 (runners.yaml:579) and 2 of 4 GPUs, which gives 450. The workflow exports that as TOTAL_CPU_DRAM_GB (benchmark-tmpl.yml:142). Results record it as allocated_cpu_dram_gb (results/agentic/init.py:192). The recipe repeats the number by hand in kv-transfer-config cpu_bytes_to_use at agg-variants.yaml:62 and :79. validate_recipe (single_node.py:144-146) only compares CONC, KV_OFFLOADING and TOTAL_CPU_DRAM_GB when the recipe's benchmark.env declares them. These overrides declare only CONC. The B200 sibling override_tp4_c32_dram declares KV_OFFLOADING: dram and TOTAL_CPU_DRAM_GB: '1073', so its bind fails on a mismatch. Today both sides give 450, so current results are correct. But if dram-utilization or the runner's available-cpu-dram-mib changes later, the server keeps offloading 450 GB while results publish the new allocation, with no error.
Verification: validate_recipe in inferencex-e2e/infx/srt_slurm/single_node.py:144-146 only compares a value when the recipe declares it. In the new agg-variants.yaml, the benchmark.env of both overrides holds only CONC, while "cpu_bytes_to_use":450000000000 is hard-coded at lines 62 and 79. Today the matrix value is 450, so it matches the recipe. If either input changes and the recipe does not, the bind no longer fails.
| - { tp: 2, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agg-variants.yaml:override_tp2_c32_dram } | ||
| - { tp: 2, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [40], srt-recipe: agg-variants.yaml:override_tp2_c40_dram } | ||
| - search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 4, 8, 16, 32], srt-recipe: agentic.yaml } |
There was a problem hiding this comment.
🟡 (optional) The zip_override_dep4 block in gb200-fp4-mtp/agentic.yaml, with its vllm-router frontend, stays in the repo after this change even though no config uses it. The TP4+EP4 dp-attn row was the only master row that selected that variant, and this diff deletes it. AGENTS.md says to delete recipes that no active config uses. Fix: delete zip_override_dep4 from minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml, keeping zip_override_tp4, which line 5636 still uses. Merging that block into agg-variants.yaml would also remove the duplicated base.
Why this was flagged
On the base, the row { tp: 4, ep: 4, dp-attn: true, ..., conc-list: [64, 128], router: vllm-router, srt-recipe: agentic.yaml } selected the DEP4 variant of benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml (zip_override_dep4 with CONC ['64','128'], data-parallel-size 4 and setup_script vllm-router.sh). The diff removes that row, and only the TP4 conc 1-32 row stays at nvidia-master.yaml:5636. The recipe file is not touched, so zip_override_dep4 is now an orphan that matches no master row. AGENTS.md "Deprecating benchmark configs" says to delete recipes and assets that no active config uses. On top of that, the new agg-variants.yaml copies agentic.yaml's whole base block, so one recipe design now lives in two files that can drift apart.
Verification: Triggering condition: merging as-is. The diff deletes the only master row that used the DEP4 variant in nvidia-master.yaml. The recipe minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml is untouched and still has zip_override_dep4. So the dep4 block is now a dead variant. AGENTS.md:74 says "Delete recipes, setup scripts and other assets that no active config uses any more".
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40636603138 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40636603138 |
fb725b7 to
c3f13d3
Compare
使用最新配置提交 MiniMax M3.1 AgentX。
c3f13d3 to
50407d2
Compare
# Conflicts: # inferencex-e2e/perf-changelog.yaml
# Conflicts: # inferencex-e2e/perf-changelog.yaml
# Conflicts: # inferencex-e2e/perf-changelog.yaml
# Conflicts: # inferencex-e2e/perf-changelog.yaml
# Conflicts: # inferencex-e2e/perf-changelog.yaml
…e point Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # inferencex-e2e/perf-changelog.yaml
|
/use 40636603138 |
|
@xinli-sw staged run 40636603138. This run remains available across future |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
中文
Signed: |
❌❌❌ REJECTED ❌❌❌@RohitNagraj Two checks block the merge. First, nothing shows that the selected DeepGEMM MegaMoE kernel actually ran in the evals. Second, the new ❌ Check 2 (Evals pass): FAIL — No eval shows that the MegaMoE path actually ran. The three disaggregated configs select ❌ Check 18 (Recipe/image/sweep consistency): FAIL — (f2) The new
Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — ✅ Check 1 (Sweep on in-PR commit): PASS — At pinned head ✅ Check 3 (Upstream recipe): PASS — The published vLLM recipe comes from vllm-project/vllm#64283, MERGED 2026-10-28T19:17:13Z. Its ✅ Check 4 (Reuse command): PASS — ✅ Check 5 (Latest checklist): PASS — All 18 current template items are present. The unchecked draft-precision and Pareto items have notes in the additional detail section. ✅ Check 6 (Upstream images, engine-first): PASS — Both entries use the upstream ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (No architecture hacks): PASS — No ✅ Check 9 (Chat template): PASS — The AgentX benchmarks replay traces through the shared ✅ Check 10 (No engine patches): PASS — The reachable ✅ Check 11 (Golden AL): PASS — Job logs show the connector injecting ➖ Check 12 (Append-only): N/A — The new changelog entry does not set ➖ Check 15 (PR description): N/A — The PR body says only "Use latest configs for MiniMax M3.1 GB200 AgentX submission" and makes no verifiable technical claims. ✅ Check 16 (NVIDIA overview): PASS — MiniMax-M3.1-Flash has a vLLM entry in the Framework Recipes table of the NVIDIA overview. ➖ Check 17 (Metrics publication): N/A — These are Dynamo-vLLM configs. The Dynamo TRT-LLM metrics controls do not apply. Assessed commit: |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
|
confirmed that https://github.com/SemiAnalysisAI/InferenceX/actions/runs/40636603138 uses magemoe in evals |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
|
Ingest report — ✅ Results ingested into the Ingest workflow · Merge Ingest run · Benchmark source run
benchmarkFiles 9, rawBenchmarkRows 9, coveredByPerJobFiles 0, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 7914, gpuMetricSamplesNew 0, powerPublicationPoints 9, changelogEntriesWritten 1 Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun. 中文Ingest 报告 — ✅ 结果已写入 上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。 |
Use latest configs for MiniMax M3.1 GB200 AgentX submission.
中文
更新 MiniMax M3.1 GB200 AgentX 提交配置。
AI model disclosure
Prepared by Codex with delegated agents. Exact model/version identifiers could not be verified from the runtime.