From 13c5e26a323b7e408c1da47a9bdb70587608a26f Mon Sep 17 00:00:00 2001 From: weireweire <20922698+weireweire@users.noreply.github.com> Date: Wed, 28 Oct 2026 13:05:26 +0800 Subject: [PATCH] chore: bump srt-slurm to v2.65.2 Move the srt-slurm submodule from v2.59.1 (ae3d6877) to v2.65.2 (59920ea1). v2.65.2 adds NVIDIA/srt-slurm#629: Dynamo frontends since ai-dynamo/dynamo#16454 answer /health status-only, so the readiness wait never counted any workers on Dynamo 1.7 images. DynamoFrontend.probe_ready now asks /health?instances=true only when /health has no instances key. 573-participating-gpus.patch still applies, and src/srtctl/core/power is unchanged between the pins, so it stays. Validation: infx generate for every config key writes 2023 recipes that the new srtctl validates, byte-identical to the v2.59.1 output; all patches apply to v2.65.2; infx.matrix.changed lists no config keys. --- .../benchmarks/multi_node/srt-slurm-recipes/RECIPES.md | 2 +- inferencex-e2e/configs/CONFIGS.md | 2 +- inferencex-e2e/utils/srt-slurm | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md index 2cf40a553a..f2087cfb7f 100644 --- a/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md +++ b/inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md @@ -2,7 +2,7 @@ InferenceX owns the recipes in this directory. For every NVIDIA srt-slurm launch, the srt driver ([`infx/launch/drivers/srt/checkout.py`](../../../infx/launch/drivers/srt/checkout.py)) makes a job-local Git clone of the pinned submodule and copies this entire tree into `recipes/`. It records the actual revision in `srt-slurm-sha.txt`; power lanes copy that revision into `power-producer-sha.txt` for result validation. -The shared version is the Git submodule pointer at [`utils/srt-slurm`](../../../utils/srt-slurm), currently [v2.59.1](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.59.1) (`ae3d6877c46f3147e3a5dafa46ee4cd89322e52f`). Update that submodule pointer when upgrading, then run the recipe and integration checks. Do not add model-specific checkout branches to launchers. +The shared version is the Git submodule pointer at [`utils/srt-slurm`](../../../utils/srt-slurm), currently [v2.65.2](https://github.com/NVIDIA/srt-slurm/releases/tag/v2.65.2) (`59920ea16a13017a990e449d034a495c0b05bc57`). Update that submodule pointer when upgrading, then run the recipe and integration checks. Do not add model-specific checkout branches to launchers. InferenceX requires srt-slurm 2.0 or newer and `schema: 2` recipes. Legacy recipe layouts are unsupported; migrate them before adding them to this tree. diff --git a/inferencex-e2e/configs/CONFIGS.md b/inferencex-e2e/configs/CONFIGS.md index 9a1aaca4cd..c329222f93 100644 --- a/inferencex-e2e/configs/CONFIGS.md +++ b/inferencex-e2e/configs/CONFIGS.md @@ -287,7 +287,7 @@ schema; unknown keys fail. measurement fails the job under `REQUIRE_POWER=1` and otherwise records an invalid verdict without energy metrics. NVIDIA clusters run `dcgm-exporter`; AMD clusters use srt-slurm's - [`kind: custom` schema](https://github.com/NVIDIA/srt-slurm/blob/ae3d6877c46f3147e3a5dafa46ee4cd89322e52f/docs/power-telemetry.md#gpu-exporter-labels-and-metrics), + [`kind: custom` schema](https://github.com/NVIDIA/srt-slurm/blob/59920ea16a13017a990e449d034a495c0b05bc57/docs/power-telemetry.md#gpu-exporter-labels-and-metrics), which requires `gpu_labels` and `gpu_metrics` and rejects unknown keys such as the former `power_profile`. The AMD exporter reads [`runners/srt-slurm/exporters/amd-power.json`](../runners/srt-slurm/exporters/amd-power.json); diff --git a/inferencex-e2e/utils/srt-slurm b/inferencex-e2e/utils/srt-slurm index ae3d6877c4..59920ea16a 160000 --- a/inferencex-e2e/utils/srt-slurm +++ b/inferencex-e2e/utils/srt-slurm @@ -1 +1 @@ -Subproject commit ae3d6877c46f3147e3a5dafa46ee4cd89322e52f +Subproject commit 59920ea16a13017a990e449d034a495c0b05bc57