Repository navigation
config: append GB300 AgentX c5120 and c6656 points - #4126
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
cquil11
added
the
non-canary-full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| override_8p1d_dep2_dep16_c5120: | ||
| name: "dsv41flash-fp4-gb300-dynamo-vllm-agentx-disagg-8p1d-dep2-dep16-c5120" | ||
| roles: | ||
| prefill: | ||
| nodes: 4 | ||
| workers: 8 | ||
| decode: | ||
| args: | ||
| max-num-seqs: 70 | ||
| max-num-batched-tokens: 700 | ||
| compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","max_cudagraph_capture_size":420,"cudagraph_capture_sizes":[6,12,18,24,30,36,42,48,54,60,66,72,78,84,90,96,102,108,114,120,126,132,138,144,150,156,162,168,174,180,186,192,198,204,210,216,222,228,234,240,246,252,258,264,270,276,282,288,294,300,306,312,318,324,330,336,342,348,354,360,366,372,378,384,390,396,402,408,414,420]}' |
There was a problem hiding this comment.
🔴 The two new recipes (override_8p1d_dep2_dep16_c5120, override_10p1d_dep2_dep16_c6656) will likely get killed by SLURM before finishing, unlike the existing 8p1d point. override_8p1d_dep2_dep16_c4096 at line 207-210 sets slurm: time_limit: "16:00:00" because the base 6h default (line 27) is too short for an 8p1d run; the new c5120 entry has the identical 4-node/8-worker prefill topology but higher concurrency (5120 vs 4096) and keeps no time_limit override, silently falling back to 6:00:00. The larger 10p1d c6656 entry has the same gap. Fix: add slurm: time_limit overrides (>= 16:00:00, scaled for concurrency) to override_8p1d_dep2_dep16_c5120 and override_10p1d_dep2_dep16_c6656.
Why this was flagged
The recipe base at disagg-variants.yaml:27 sets slurm.time_limit to "6:00:00". override_8p1d_dep2_dep16_c4096 (same file, line 207-210) overrides it to "16:00:00" because an 8-prefill-worker DEP2/DEP16 disagg run needs that long. The new override_8p1d_dep2_dep16_c5120 (line 233) has the identical prefill topology (nodes:4, workers:8) plus higher target concurrency (5120 vs 4096) but adds no slurm block, so it inherits the 6h default. override_10p1d_dep2_dep16_c6656 (line 221), an even larger 10p1d topology, has the same omission. On the base branch there is no c5120/c6656 point at all; after merge these SLURM jobs are likely to hit the 6h wall and be terminated mid-run, producing failed or truncated benchmark curve points instead of the intended data.
Verification: The new c5120 point is structurally the same run as the existing c4096 point that the author deliberately gave a 16h time limit, but it inherits the 6h base default and will be SLURM-killed before finishing.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38782902301 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38782902301 |
|
/use 38782902301 |
|
@cquil11 staged run 38782902301. This run remains available across future |
cquil11
changed the title
Append c5120 (8P1D) and c6656 (10P1D) to the existing GB300 DeepSeek-V4.1-Flash Dynamo + vLLM AgentX curve, preserving their serving settings and dedicated frontend/client node. Use the released harness and its default exact timing backend; no runtime harness patch or t-digest override.
The multi-node launcher creates a unique temporary Python environment to avoid collisions with directories left by previous jobs. Supersedes the memory experiment in #4074.
Both points passed the full one-hour sweep on agentx-harness 1.0.10, including exact exports, artifact uploads and cleanup. Export audits retained metric coverage and reconciled request and inter-chunk sample counts, with no OOM, processing backlog or child-worker migrations: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/38782902301/attempts/2.
Reuse the green sweep with
/use 38782902301; the changelog appends only these two new points.