Repository navigation
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
9337772 to
2ca177e
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🔴
inferencex-e2e/perf-changelog.yaml— This PR adds two new benchmark points (c5120, c6656) with no inferencex-e2e/perf-changelog.yaml entry, so operators have no record these configs need a rerun to validate on current hardware. AGENTS.md requires every recipe addition to come with a changelog block, and the PR description itself says these points "still need hardware confirmation," but nothing records that need. Fix: add a perf-changelog.yaml block for this PR listing the dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-disagg config-key additions (c5120, c6656), per AGENTS.md:140-141.Why this was flagged
AGENTS.md:140 states every recipe addition or modification requires a new inferencex-e2e/perf-changelog.yaml entry (append-only, one PR = one block). The diff adds override_8p1d_dep2_dep16_c5120 and override_10p1d_dep2_dep16_c6656 to disagg-variants.yaml and wires matching entries into nvidia-master.yaml (dsv41flash-fp4-gb300-dynamo-vllm-agentic-dspark-disagg scenarios list) but touches no perf-changelog.yaml file. On the base branch, every prior recipe addition (e.g. override_6p1d_dep2_dep16_c2884, override_4p1d_dep2_dep16_c1228) is expected to have a matching changelog block; this PR skips that bookkeeping, leaving no record that these new high-concurrency points require a rerun/hardware confirmation, which the PR author's own description says is still outstanding.
Verification: AGENTS.md:140 requires a new inferencex-e2e/perf-changelog.yaml entry for every recipe addition. Commit 9337772 adds override_8p1d_dep2_dep16_c5120 and override_10p1d_dep2_dep16_c6656 to disagg-variants.yaml and nvidia-master.yaml, but touches only those two files; perf-changelog.yaml is untouched and the working tree is clean. The prior commit introducing these points, #3941 (f516173), did append a changelog block, so this omission breaks the established pattern.
cquil11
added
the
non-canary-full-sweep-enabled
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要，改动通常也很小。请让你的 agent 清理 PR，使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38661720446 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38661720446 |
|
Superseded by #4126, which adds c5120 and c6656 from current main using released agentx-harness 1.0.10 and the default exact timing backend. The experimental runtime patch and t-digest override are no longer needed. |
Append GB300 DeepSeek-V4.1-Flash Dynamo + vLLM AgentX points at c5120 (8P1D) and c6656 (10P1D), preserving their original decode settings and the dedicated frontend/client node. The append-only sweep selects only these two additions; c4096 is tested separately in #4071.
These two new recipes use AgentX 1.0.9, a runtime patch from agentx-harness #52 to keep each SPAWN subagent on one client worker, and
AIPERF_METRICS_LIST_BACKEND=tdigestto bound retained timing samples. T-digest uses approximate list-metric percentiles, omits their per-phase/time-window summaries, and computes timing curves from request boundaries instead of retained per-chunk timing. This is a memory experiment, not a claim of complete metric parity. Existing points retain their current unpatched harness configuration and timing backend.The patch is applied to the isolated PyPI installation, fails the job if it does not apply, and logs its SHA256 for verification. It contains the routing implementation from commit
b5d7652b50aceb9e0c4ca77daf9ae8f472ac77fd.The full one-hour sweep passed for both additions, with the patch and t-digest setting verified on the actual clients. c5120 completed 608,372 requests (2 empty-response validation errors); c6656 completed 779,900 (zero profiling errors). All completed records were already processed when each profiling phase ended, and final AIPerf exports finished about 3 minutes later. Both jobs uploaded their final benchmark results.