Skip to content

Report InferenceCluster provisioning stages - #607

Merged
haarchri merged 6 commits into
modelplaneai:mainfrom
RedbackThomson:inference-cluster-stage-conditions
Nov 19, 2026
Merged

haarchri merged 6 commits into
modelplaneai:mainfrom
RedbackThomson:inference-cluster-stage-conditions

Conversation

@RedbackThomson

@RedbackThomson RedbackThomson commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Description of your changes

Fixes #603.

This implements the stage conditions proposed in #603. Each compose-<cloud>-cluster function reports ClusterReady, NodePoolsReady and status.nodePools on its own XR, with the same logic on all eight clouds; only the resources grouped under each pool differ. compose-inference-cluster relays them as the cloud cluster XR reports them and derives Accepted and ServingStackReady. A stage is Pending on an earlier one only until it's composed. After that it reports its own state, so on a running cluster the serving stack stays Available while a new node pool provisions or Accepted fails.

Breaking for anything reading conditions: ClusterRunning becomes Available, and BackendReady becomes ServingStackReady, because "backend" now means a ModelEndpoint a gateway routes to. Its BackendHealthy, Installing and WaitingForCluster reasons become Available, Provisioning and Pending. ClusterReady no longer covers node pools. Nothing in Modelplane reads either condition, and Ready is unchanged.

The first commit fixes a bug this builds on. The kubeconfig ClusterProviderConfig was composed only while the cloud cluster XR was Ready, so one unready node group dropped it while every Object on the cluster still used it; it's now kept while it's observed. The checks that fail the reconcile (missing classes or reservations, invalid node pools, unsupported devices) now report on Accepted rather than ClusterReady.

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes.
  • Signed off every commit with git commit -s.

@github-actions

Copy link
Copy Markdown

Docs preview: https://modelplane-docs-pr-607.vercel.app (ready once the site's Content workflow finishes)

@RedbackThomson RedbackThomson changed the title Document InferenceCluster provisioning status Report InferenceCluster provisioning stages Oct 30, 2026
@RedbackThomson
RedbackThomson marked this pull request as ready for review October 30, 2026 20:58
@negz

negz commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Since we're touching this, it occurs to me that "backend" is old terminology for the serving stack. Could we rename the condition to ServingStackReady with a STACK column, or to StackReady?

These days "backend" mostly means a ModelEndpoint, the thing a gateway routes to (see ModelEndpoint, ModelService and the routing docs). So BackendReady on an InferenceCluster reads like "can the gateway reach my endpoints". What it actually reports is the ServingStack, which the docs call the "serving stack" throughout, as does the new BACKEND printer column's description. I don't feel strongly about which name, just that it shouldn't be "backend".

@RedbackThomson
RedbackThomson force-pushed the inference-cluster-stage-conditions branch 2 times, most recently from 503468e to 53461db Compare November 2, 2026 17:41
@RedbackThomson

Copy link
Copy Markdown
Contributor Author

/review

1 similar comment
@negz

negz commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

/review

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is close. The stage model follows #603, and what's left are questions about how the stages behave outside first provisioning.

The main ones are whether the fallback for a cloud cluster XR that doesn't report the new conditions should exist at all, and what a stage that already finished should report when an earlier one goes False on a running cluster. The others cover which failures report Failed, how the stage boundaries meet #550, and some test structure.

functions/compose-inference-cluster/tests/test_fn.py line 3403: Nit: the observed EKSCluster is now written out inline in four cases, three of them new. CONTRIBUTING asks for a helper once a resource appears in three or more cases, and an observed resource gets one separate from the desired one, such as _observed_eks_cluster(*, conditions=..., node_pools=...).

Reviewed by modelplane-review d7f2116. Reply, or react 👍 or 👎, on each comment to help tune it.

Comment thread apis/eksclusters/definition.yaml Resolved
Comment thread apis/inferenceclusters/definition.yaml Resolved
@RedbackThomson
RedbackThomson force-pushed the inference-cluster-stage-conditions branch from 53461db to dd68374 Compare November 3, 2026 00:17
@RedbackThomson
RedbackThomson force-pushed the inference-cluster-stage-conditions branch 2 times, most recently from 69062a6 to 608645e Compare November 17, 2026 20:58

@haarchri haarchri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - side note please rebase and add it for the recent merged implementations too: #594

compose-inference-cluster composed the ClusterProviderConfig that
reaches a provisioned cluster only while the cloud cluster XR was
Ready. A cloud cluster XR is unready whenever any resource it composes
is, so one node group or Helm release going unready on a running
cluster dropped the ClusterProviderConfig from desired state, and
Crossplane would delete it while every Object on the cluster still
used it: the namespaces, PoolObservations, ModelReplicas and
ModelRoutes.

Once composed, the ClusterProviderConfig is now kept for as long as it
is observed, the way the ServingStack already is.

Towards modelplaneai#603.

Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
Provisioning a cluster spends most of its time building node pools,
but neither the cloud cluster XRs nor InferenceCluster report anything
per pool: the only signal is the XR's own Ready, False until every
pool and every other composed resource is done.

Each cloud cluster XR, and InferenceCluster, gets a status.nodePools
list keyed by pool name. A pool's state is Pending until its cloud
resources are composed, Provisioning until their provider reports them
Ready, then Ready, or Failed while the provider reports that its last
operation on one of them failed, with the provider's error as the
message. The system pool is left out because it is part of the base
cluster rather than a pool the user asked for.

Towards modelplaneai#603.

Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
While it provisioned, a cloud cluster XR reported nothing beyond its
own Ready, which stays False until every resource it composes is
Ready. A cluster still creating its control plane looked the same as
one finishing its last GPU node pool, or one whose node group had
failed to create.

Each compose-<cloud>-cluster function now splits what it composes into
the node pools and the cluster: everything else, such as the network,
control plane, add-ons and system nodes, including any role: System
pool. ClusterReady is True once every cluster resource is marked
ready, and NodePoolsReady once every pool's resources are.
NodePoolsReady is Pending on ClusterReady only while no pool is Ready,
so on a running cluster the pools keep reporting their own state when
the cluster or a new pool provisions. status.nodePools reports each
pool as Pending until its resources are composed, then Provisioning,
then Ready.

A stage or pool is Failed while the provider reports a resource's
LastAsyncOperation False, with the provider's error, prefixed with the
resource's name, as the message. Synced=False doesn't count as a
failure, because a managed resource is also Synced=False while it
waits on a reference to resolve, which is routine while a cluster
provisions. upjet sets LastAsyncOperation only on resources it creates
asynchronously, so a failed IAM role, Helm release or
provider-kubernetes Object stays Provisioning.

ClusterReady's Provisioning message counts the cluster's resources that
are ready, such as "12 of 52 resources ready". compose-gke-cluster also
reports ClusterReady while it waits for the GCP ProviderConfig it reads
the project from, rather than composing nothing and reporting nothing.

Towards modelplaneai#603.

Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
InferenceCluster reported ClusterReady and BackendReady, with no
messages. ClusterReady covered the whole cloud cluster XR, node pools
included, so it read Provisioning for most of a cluster's creation and
said nothing about which part was slow or had failed.

InferenceCluster now reports Accepted, ClusterReady, NodePoolsReady and
ServingStackReady. While a new cluster provisions, each is Pending
until the one before it is True, so the first condition that isn't
True names the stage it's in. Once a stage has started it reports its
own state, so a running cluster's ServingStackReady stays True while a
new node pool provisions.

Accepted carries the checks that fail the reconcile, such as
WaitingForClasses, InvalidNodePool and UnsupportedDevices, which
reported on ClusterReady. While it's False, a stage that hasn't been
composed is Pending, and a running cluster's stages keep reporting their
own state. ClusterReady, NodePoolsReady and status.nodePools are relayed
from the cloud cluster XR as it reports them, including the Unknown
status Crossplane gives them when the cloud cluster XR's function fails,
while the XR's Ready stays True. An Existing cluster reports both stages
with the reason Existing and has no status.nodePools.

ClusterRunning becomes Available. BackendReady becomes
ServingStackReady, because "backend" now means a ModelEndpoint a
gateway routes to, and its BackendHealthy, Installing and
WaitingForCluster reasons become Available, Provisioning and Pending.
Nothing in Modelplane reads either condition. Ready, and the
status fields the scheduler and other functions read, are unchanged.

kubectl get ic shows each stage's reason in CLUSTER, POOLS and STACK
columns.

Towards modelplaneai#603.

Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
The getting started guide told readers to wait on Ready while a
cluster provisioned, with no way to see how far it had got. The
InferenceCluster page now describes the stage conditions, their
reasons and status.nodePools, and the guide points to it from the
provisioning step.

Fixes modelplaneai#603.

Signed-off-by: Nicholas Thomson <RedbackThomson@users.noreply.github.com>
@RedbackThomson
RedbackThomson force-pushed the inference-cluster-stage-conditions branch from 608645e to 09df6a1 Compare November 19, 2026 16:59
@haarchri
haarchri merged commit 0f30f2d into modelplaneai:main Nov 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InferenceCluster conditions should report progressive status until they're Ready

3 participants