Skip to content

config: append GB300 AgentX c5120 and c6656 points #4126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 4 commits into from
Oct 16, 2026
Merged

config: append GB300 AgentX c5120 and c6656 points #4126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6c52e2d
config: append GB300 AgentX c5120 and c6656 with harness 1.0.10
cquil11 Oct 15, 2026
ff4aa0a
docs: link higher-concurrency points to PR 4126
cquil11 Oct 15, 2026
d6cd9c0
fix(ci): isolate each multi-node launcher environment
cquil11 Oct 15, 2026
aa51e35
chore: sync main and preserve PR 4126 sweep reuse
cquil11 Oct 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/benchmark-multinode-tmpl.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ jobs:
- name: Prepare launcher Python
env: *uv-setup-env
run: |
launch_venv="$RUNNER_TEMP/infx-launch-venv"
launch_venv="$(mktemp -d "$RUNNER_TEMP/infx-launch-venv.XXXXXX")"
uv venv --quiet --python 3.12 "$launch_venv"
uv pip install --quiet --exclude-newer PT12H --python "$launch_venv/bin/python" \
-r "$GITHUB_WORKSPACE/.result-tooling/inferencex-e2e/pyproject.toml"
Expand Down
24 changes: 24 additions & 0 deletions ...hmarks/multi_node/srt-slurm-recipes/dsv41flash/vllm/gb300-fp4/agentx/disagg-variants.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,3 +217,27 @@ override_8p1d_dep2_dep16_c4096:
max-num-seqs: 72
max-num-batched-tokens: 720
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":432,"cudagraph_capture_sizes":[6,12,18,24,30,36,42,48,54,60,66,72,78,84,90,96,102,108,114,120,126,132,138,144,150,156,162,168,174,180,186,192,198,204,210,216,222,228,234,240,246,252,258,264,270,276,282,288,294,300,306,312,318,324,330,336,342,348,354,360,366,372,378,384,390,396,402,408,414,420,426,432]}'

override_10p1d_dep2_dep16_c6656:
name: "dsv41flash-fp4-gb300-dynamo-vllm-agentx-disagg-10p1d-dep2-dep16-c6656"
roles:
prefill:
nodes: 5
workers: 10
decode:
args:
max-num-seqs: 132
max-num-batched-tokens: 1320
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":792,"cudagraph_capture_sizes":[6,12,18,24,30,36,42,48,54,60,66,72,78,84,90,96,102,108,114,120,126,132,138,144,150,156,162,168,174,180,186,192,198,204,210,216,222,228,234,240,246,252,258,264,270,276,282,288,294,300,306,312,318,324,330,336,342,348,354,360,366,372,378,384,390,396,402,408,414,420,426,432,438,444,450,456,462,468,474,480,486,492,498,504,510,516,522,528,534,540,546,552,558,564,570,576,582,588,594,600,606,612,618,624,630,636,642,648,654,660,666,672,678,684,690,696,702,708,714,720,726,732,738,744,750,756,762,768,774,780,786,792]}'

override_8p1d_dep2_dep16_c5120:
name: "dsv41flash-fp4-gb300-dynamo-vllm-agentx-disagg-8p1d-dep2-dep16-c5120"
roles:
prefill:
nodes: 4
workers: 8
decode:
args:
max-num-seqs: 70
max-num-batched-tokens: 700
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":420,"cudagraph_capture_sizes":[6,12,18,24,30,36,42,48,54,60,66,72,78,84,90,96,102,108,114,120,126,132,138,144,150,156,162,168,174,180,186,192,198,204,210,216,222,228,234,240,246,252,258,264,270,276,282,288,294,300,306,312,318,324,330,336,342,348,354,360,366,372,378,384,390,396,402,408,414,420]}'
Comment on lines +233 to +243

@claude claude Bot Oct 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The two new recipes (override_8p1d_dep2_dep16_c5120, override_10p1d_dep2_dep16_c6656) will likely get killed by SLURM before finishing, unlike the existing 8p1d point. override_8p1d_dep2_dep16_c4096 at line 207-210 sets slurm: time_limit: "16:00:00" because the base 6h default (line 27) is too short for an 8p1d run; the new c5120 entry has the identical 4-node/8-worker prefill topology but higher concurrency (5120 vs 4096) and keeps no time_limit override, silently falling back to 6:00:00. The larger 10p1d c6656 entry has the same gap. Fix: add slurm: time_limit overrides (>= 16:00:00, scaled for concurrency) to override_8p1d_dep2_dep16_c5120 and override_10p1d_dep2_dep16_c6656.

Why this was flagged

The recipe base at disagg-variants.yaml:27 sets slurm.time_limit to "6:00:00". override_8p1d_dep2_dep16_c4096 (same file, line 207-210) overrides it to "16:00:00" because an 8-prefill-worker DEP2/DEP16 disagg run needs that long. The new override_8p1d_dep2_dep16_c5120 (line 233) has the identical prefill topology (nodes:4, workers:8) plus higher target concurrency (5120 vs 4096) but adds no slurm block, so it inherits the 6h default. override_10p1d_dep2_dep16_c6656 (line 221), an even larger 10p1d topology, has the same omission. On the base branch there is no c5120/c6656 point at all; after merge these SLURM jobs are likely to hit the 6h wall and be terminated mid-run, producing failed or truncated benchmark curve points instead of the intended data.

Verification: The new c5120 point is structurally the same run as the existing c4096 point that the author deliberately gave a 16h time limit, but it inherits the 6h base default and will be SLURM-killed before finishing.

30 changes: 30 additions & 0 deletions inferencex-e2e/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9187,6 +9187,36 @@ dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-disagg:
tp: 16
ep: 16
dp-attn: true
- spec-decoding: mtp
kv-offloading: dram
kv-offload-backend: { name: vllm-simple }
srt-recipe: disagg-variants.yaml:override_8p1d_dep2_dep16_c5120
conc-list: [5120]
prefill:
num-worker: 8
tp: 2
ep: 2
dp-attn: true
decode:
num-worker: 1
tp: 16
ep: 16
dp-attn: true
- spec-decoding: mtp
kv-offloading: dram
kv-offload-backend: { name: vllm-simple }
srt-recipe: disagg-variants.yaml:override_10p1d_dep2_dep16_c6656
conc-list: [6656]
prefill:
num-worker: 10
tp: 2
ep: 2
dp-attn: true
decode:
num-worker: 1
tp: 16
ep: 16
dp-attn: true

# One serving GPU, stock ARM64 release, native AgentX client and proportional host cache.
qwen3.8next-fp4-gb300-sglang-agentic-mtp:
Expand Down
9 changes: 9 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10013,3 +10013,12 @@
- "Add a HiCache host-DRAM KV offload arm at concurrency 2, 4 and 8 with 160 GB per rank, alongside the GPU-resident sweep; the few-GB GPU KV pool left by 67 GB of weights per H100 evicts agentic prefixes between turns."
- "Keep the batch-8 cap on running requests and decode graphs from the GPU-resident sweep."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4181

- config-keys:
- dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-disagg
scenario-type:
- agentic-coding
description:
- "Append GB300 DeepSeek-V4.1-Flash Dynamo + vLLM AgentX c5120 (8P1D) and c6656 (10P1D) points using the agentx-harness 1.0.10 release already pinned on main."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4126
append-only: true
Loading