Repository navigation
Provision InferenceClusters on CoreWeave - #655
Conversation
|
/review |
|
Docs preview: https://modelplane-docs-pr-655.vercel.app (ready once the site's Content workflow finishes) |
There was a problem hiding this comment.
I think this needs another round before it merges. The gateway annotation and the copied API token are design questions to settle first.
Most of the change follows the other provisioned clouds, and the questions are where CKS differs. Nothing has run on a real CKS cluster yet. One run would settle the GPU assumptions the description flags, and show whether the readiness and autoscaling problems raised inline happen in practice.
Reviewed by modelplane-review d148f8c. Reply, or react 👍 or 👎, on each comment to help tune it.
cfeec63 to
8af20de
Compare
|
/review |
There was a problem hiding this comment.
The changes since my last review look close to ready, with a few questions left. The main one is whether the system pool's new readiness check can mark a serving cluster un-Ready when CKS replaces its one node.
The VPC Usage, the compose-usages step and the reworked gateway Service settings raise nothing new. The other findings are docs and code comments the changes made untrue.
The docs commit's message still says ModelCache uses shared-vast, and the CoreWeaveCluster commit says every pool starts with at least one node, where only autoscaled pools do.
Reviewed by modelplane-review 84d2a55. Reply, or react 👍 or 👎, on each comment to help tune it.
|
/review |
There was a problem hiding this comment.
The changes since my last review look ready to merge, apart from two wording points inline.
The system pool's readiness switch and deleting the StorageClass with the XR answer the earlier threads, and the new test cases cover both paths the switch can take.
Reviewed by modelplane-review 3017043. Reply, or react 👍 or 👎, on each comment to help tune it.
|
I've run this end to end on a real CoreWeave org in US-EAST-04A. An On the live cluster:
Only unit tests cover what changed after that run: the system pool's |
UpCloud's load balancer needs - TCP frontend mode and names that survive
the cloud controller's 64-character truncation - were plumbed as
serviceAnnotations through the InferenceCluster status and ServingStack
spec, with a {gateway} template and a per-Gateway EnvoyProxy that
restated the externalTrafficPolicy pin. The annotation also broke an
InferenceGateway with spec.tls: its Service has two ports, and the
controller matches config frontends to ports by name, so the single
unnamed frontend left the load balancer one nameless frontend on port 0
with no backend.
Instead, a cloud half now declares what its load balancers need on every
gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's
KubernetesServiceSpec, which join() folds into the GatewayClass-level
EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with
mergeType: StrategicMerge, and on UpCloud alone overrides the config
annotation with a named TCP frontend per listener and names its Service.
The controller already prefixes every load balancer with the cluster
UUID, so a Service name that differs from the cluster gateway's within
the truncated remainder is enough; the UUID-built name, the {gateway}
template and UpCloudCluster's clusterId status all go. This is the
mechanism modelplaneai#655 introduces for CoreWeave, applied ahead of it so both
clouds share one shape.
The autoscaler fork lists every node group in the cluster each refresh
and gives any group missing from its --nodes list a minimum of one node
and a maximum of the cluster plan's limit, so it could scale the system
group down to one node or a fixed GPU pool up on pending pods, fighting
Crossplane as it reset each back to its nodeCount. Whenever any pool
autoscales, the group list now also pins the system group and every
fixed pool at their node count. compose_autoscaler_secret takes the
Autoscaler compose() already narrowed, a test now covers the node-group
sentinel that keeps groups composed through a Ready regression on
clusters without an autoscaler, and the node pool docs stop suggesting a
System pool beside the injected one, note that Premium and Starter plans
ignore diskSizeGb, and drop the stale GPU-pools-only wording.
Works around UpCloudLtd/upcloud-cloud-controller-manager#14.
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
UpCloud's load balancer needs - TCP frontend mode and names that survive
the cloud controller's 64-character truncation - were plumbed as
serviceAnnotations through the InferenceCluster status and ServingStack
spec, with a {gateway} template and a per-Gateway EnvoyProxy that
restated the externalTrafficPolicy pin. The annotation also broke an
InferenceGateway with spec.tls: its Service has two ports, and the
controller matches config frontends to ports by name, so the single
unnamed frontend left the load balancer one nameless frontend on port 0
with no backend.
Instead, a cloud half now declares what its load balancers need on every
gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's
KubernetesServiceSpec, which join() folds into the GatewayClass-level
EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with
mergeType: StrategicMerge, and on UpCloud alone overrides the config
annotation with a named TCP frontend per listener and names its Service.
The controller already prefixes every load balancer with the cluster
UUID, so a Service name that differs from the cluster gateway's within
the truncated remainder is enough; the UUID-built name, the {gateway}
template and UpCloudCluster's clusterId status all go. This is the
mechanism modelplaneai#655 introduces for CoreWeave, applied ahead of it so both
clouds share one shape.
The autoscaler fork lists every node group in the cluster each refresh
and gives any group missing from its --nodes list a minimum of one node
and a maximum of the cluster plan's limit, so it could scale the system
group down to one node or a fixed GPU pool up on pending pods, fighting
Crossplane as it reset each back to its nodeCount. Whenever any pool
autoscales, the group list now also pins the system group and every
fixed pool at their node count. compose_autoscaler_secret takes the
Autoscaler compose() already narrowed, a test now covers the node-group
sentinel that keeps groups composed through a Ready regression on
clusters without an autoscaler, and the node pool docs stop suggesting a
System pool beside the injected one, note that Premium and Starter plans
ignore diskSizeGb, and drop the stale GPU-pools-only wording.
Works around UpCloudLtd/upcloud-cloud-controller-manager#14.
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
|
I ran what changed since my last comment on a fresh cluster in US-EAST-04A:
|
|
/review |
There was a problem hiding this comment.
The changes since my last review look close to ready, with a few questions on the second NodePool Object and the new DCGM exporter.
Splitting an autoscaled pool across two Objects looks sound, and your live run covers it. I'm unsure a fixed pool switched to autoscaling at zero nodes can ever scale up. I've also asked where the CKS-specific collector rule belongs, and flagged a docs page the exporter makes untrue.
The "Provision InferenceClusters on CoreWeave" commit message still says the CoreWeave half installs only cert-manager and the DRA driver, and doesn't mention the collector change.
Reviewed by modelplane-review 3017043. Reply, or react 👍 or 👎, on each comment to help tune it.
Signed-off-by: Nic Cope <nicc@rk0n.org>
|
/review |
There was a problem hiding this comment.
The changes since my last review look close to ready, with one question on whether an autoscaled pool's seed node still keeps it off zero.
The released provider pin and the updated telemetry and collector docs answer the earlier threads. Since a CKS pool that reaches zero stays there, the default minNodes decides whether an autoscaled pool keeps working.
The PR description still says the PR pins v0.1.0-bf0e690d until provider-coreweave has a release, and could say it pins v0.1.0.
Reviewed by modelplane-review ad611b9. Reply, or react 👍 or 👎, on each comment to help tune it.
|
A second run in US-EAST-04A, with a CPU system pool (
The provider pin is now the released |
7f9ba17 to
4cc0412
Compare
|
/review |
There was a problem hiding this comment.
The changes since my last review look close to ready, with a few docs points from the move to RTX PRO 6000 and one test gap.
The new minNodes default settles the earlier thread on autoscaled pools reaching zero. What's left is mostly the getting started pages, which no longer fit the CoreWeave path in a few places.
The docs commit's message says the pages note that node pools appear only once the cluster is running, but this round removed that note.
Reviewed by modelplane-review 337445e. Reply, or react 👍 or 👎, on each comment to help tune it.
There was a problem hiding this comment.
The changes since my last review raise nothing new, and the PR looks ready to merge.
Reviewed by modelplane-review 337445e. Reply, or react 👍 or 👎, on each comment to help tune it.
dennis-upbound
left a comment
There was a problem hiding this comment.
Thanks Nic, this reads really well and follows the shape of the other cluster XRs closely. I like the Usage chain for teardown and keeping the kubeconfig's observed data when the token can't be read. The two-Object trick for targetNodes took me a minute, but the comment explains it well, and moving the Service settings into gateway_proxy is a nice general fix. A few small things inline, mostly API consistency and a docs note about the token. None of it blocks for me. LGTM.
Modelplane can't provision clusters on CoreWeave. Teams there have to create a CKS cluster themselves and register it with source Existing. CoreWeaveCluster provisions a CKS cluster in a dedicated VPC using provider-coreweave. CKS differs from the other clouds in two ways that shape the function. CKS has no cloud API for node pools. A pool is a NodePool resource inside the cluster, so every pool, the system pool included, is a provider-kubernetes Object composed once the cluster is Ready. The Objects keep their Delete policy: removing a pool from the spec must remove its nodes. compose-usages runs after the function to hold the provider-kubernetes ProviderConfig until every Object is gone, and Usages hold the kubeconfig Secret and the cluster until the ProviderConfig is gone, so teardown can't strand the Objects without a way to reach the cluster. Another Usage holds the VPC until the cluster is gone: CoreWeave's API answers a request to delete a VPC a cluster still uses with NotFound, which provider-coreweave takes to mean the VPC is already gone, leaking it. A pool's Object is Ready once CKS validates the pool rather than once its nodes arrive, so autoscaling and node replacement don't flap the cluster's readiness. The exception is the system pool until the CoreWeaveCluster is first Ready: its Object waits for a node, so the serving stack doesn't install onto a cluster with nowhere to run it. CKS's managed autoscaler owns an autoscaled pool's targetNodes, and CKS rejects a NodePool without it. One Object can't manage such a pool: it would force its targetNodes back over the autoscaler's, and leaving the field out of its manifest would remove it while the Object was its only field manager. provider-kubernetes applies each Object under a field manager named for the Object, so an autoscaled pool gets two. One creates and deletes the NodePool with a seed targetNodes, and the other, without targetNodes, keeps the rest of the pool current. CKS's autoscaler can't scale a pool up from zero nodes when its pods' devices bind through DRA, as Modelplane's do, so the seed is at least one node and minNodes defaults to the seed. A pool reaches zero, and stays there, only if minNodeCount asks for it. CKS publishes no kubeconfig. Its managed API endpoint accepts the same CoreWeave API access token the provider uses, behind a publicly trusted certificate, so the function reads the token from the Secret the CoreWeave ProviderConfig references and renders a kubeconfig from it and the cluster's observed endpoint. This copies the token into a Secret in the XR's namespace. CoreWeave manages the GPU stack, so GPU pools carry the label that keeps the managed GPU Operator's device plugin off them, leaving the serving stack's DRA driver as the only allocator. CKS provides a ReadWriteMany StorageClass, shared-vast, on every cluster, but it retains a volume after its claim is deleted, which would leave a deleted ModelCache's weights behind on billed storage. The function composes modelplane-rwx-fs for ModelCache instead, a copy of the observed shared-vast that deletes the volume. It's copied rather than written out because its parameters name the org's storage tenant and the zone. Towards modelplaneai#94. Signed-off-by: Nic Cope <nicc@rk0n.org>
An InferenceCluster can't provision a cluster on CoreWeave, even though CoreWeaveCluster can now build one. This adds CoreWeave as a cluster source and a provisioning provider. An InferenceCluster with source CoreWeave composes a CoreWeaveCluster from its zone, Kubernetes version, credentials and the pools' classes, then wires the cluster's kubeconfig into a ServingStack like Vultr and Civo do, with no identity layered on top. ModelCache gets the StorageClass the CoreWeaveCluster reports. The serving stack's CoreWeave half installs cert-manager, the NVIDIA DRA driver and a DCGM exporter. CoreWeave runs a managed NVIDIA GPU Operator and NFD on every CKS cluster, and documents that users mustn't install a second GPU Operator. It also installs NVIDIA's DRA driver for ComputeDomains only, so Modelplane installs it again with GPU allocation on. CoreWeave runs a DCGM exporter too, in cw-exporters, but it attributes a GPU's series only to pods the device plugin allocated it to. The serving stack installs NVIDIA's standalone exporter with DRA attribution, as on Nebius, and the collector's GPU job skips cw-exporters so it doesn't collect every GPU series twice. The two exporters run side by side on a CKS node without disturbing each other's profiling metrics. Towards modelplaneai#94. Signed-off-by: Nic Cope <nicc@rk0n.org>
CoreWeave Kubernetes Service gives a LoadBalancer Service a private address unless the Service carries the service.beta.kubernetes.io/coreweave-load-balancer-type: public annotation. Both of Modelplane's gateways need public addresses: an InferenceGateway reaches cluster gateways across clouds, and callers reach the InferenceGateway from outside the cluster. Neither function that composes a gateway knows which cloud it runs on. A cloud half of the serving stack can now declare the Service settings its load balancers need, as a subset of Envoy Gateway's KubernetesServiceSpec. The serving stack builds the GatewayClass's EnvoyProxy from them, beside the externalTrafficPolicy pin every cloud gets. The cluster gateway uses that EnvoyProxy as is. The InferenceGateway's own EnvoyProxy now sets mergeType StrategicMerge, so Envoy Gateway merges it onto the GatewayClass's rather than replacing it, and its Service gets the same settings and pin without the function knowing the cloud. The CoreWeave half declares the public annotation. CKS assigns a LoadBalancer Service its address once, when Envoy Gateway creates it. A Service created before the EnvoyProxy exists has no public annotation, and keeps the private address it gets even after the annotation arrives. The cluster Gateway now waits for the EnvoyProxy to be Ready before it's first composed. Towards modelplaneai#94. Signed-off-by: Nic Cope <nicc@rk0n.org>
The docs list CoreWeave CKS as a planned provider with no Crossplane provider, and say nothing about how to provision a cluster there. This marks CKS as provisioning-supported, links provider-coreweave, and adds CoreWeave to the getting started guide and the InferenceCluster and InferenceClass concept pages, with manifests for each. They note what differs from the other clouds: the CoreWeave API token also authenticates Modelplane to the cluster. Towards modelplaneai#94. Signed-off-by: Nic Cope <nicc@rk0n.org>
|
A final run in US-EAST-04A followed the CoreWeave getting started guide end to end with its own manifests: two It found a race, now fixed: Envoy Gateway created the second cluster's gateway Service before the EnvoyProxy carrying CKS's public annotation existed, and CKS gave the Service a private address it kept after the annotation arrived. The cluster Gateway now waits for the EnvoyProxy to be Ready before it's first composed. A recreated cluster created the EnvoyProxy 16 seconds before the Gateway and got a public address on the first try. The run also reproduced a VPC update loop in provider-coreweave v0.1.0 (upbound/provider-upjet-coreweave#6). The fix is in progress upstream in coreweave/terraform-provider-coreweave#554. |
UpCloud's load balancer needs - TCP frontend mode and names that survive
the cloud controller's 64-character truncation - were plumbed as
serviceAnnotations through the InferenceCluster status and ServingStack
spec, with a {gateway} template and a per-Gateway EnvoyProxy that
restated the externalTrafficPolicy pin. The annotation also broke an
InferenceGateway with spec.tls: its Service has two ports, and the
controller matches config frontends to ports by name, so the single
unnamed frontend left the load balancer one nameless frontend on port 0
with no backend.
Instead, a cloud half now declares what its load balancers need on every
gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's
KubernetesServiceSpec, which join() folds into the GatewayClass-level
EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with
mergeType: StrategicMerge, names its Service inference-gateway on every
cloud - one code path, with only the annotation depending on the cloud -
and on UpCloud alone overrides the config annotation with a named TCP
frontend per listener. On UpCloud the name is also load-bearing: the
controller already prefixes every load balancer with the cluster UUID,
so a Service name that differs from the cluster gateway's within the
truncated remainder is enough; the UUID-built name, the {gateway}
template and UpCloudCluster's clusterId status all go. On every other
cloud the rename replaces Envoy Gateway's generated
envoy-modelplane-system-inference-gateway-<hash> Service, so an
existing gateway gets a new LoadBalancer Service, with a new load
balancer and address. This is the mechanism modelplaneai#655 introduced for
CoreWeave, so both clouds share one shape.
The autoscaler fork lists every node group in the cluster each refresh
and gives any group missing from its --nodes list a minimum of one node
and a maximum of the cluster plan's limit, so it could scale the system
group down to one node or a fixed GPU pool up on pending pods, fighting
Crossplane as it reset each back to its nodeCount. Whenever any pool
autoscales, the group list now also pins the system group and every
fixed pool at their node count. compose_autoscaler_secret takes the
Autoscaler compose() already narrowed, a test now covers the node-group
sentinel that keeps groups composed through a Ready regression on
clusters without an autoscaler, and the node pool docs stop suggesting a
System pool beside the injected one, note that Premium and Starter plans
ignore diskSizeGb, and drop the stale GPU-pools-only wording.
Works around UpCloudLtd/upcloud-cloud-controller-manager#14.
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
UpCloud's load balancer needs - TCP frontend mode and names that survive
the cloud controller's 64-character truncation - were plumbed as
serviceAnnotations through the InferenceCluster status and ServingStack
spec, with a {gateway} template and a per-Gateway EnvoyProxy that
restated the externalTrafficPolicy pin. The annotation also broke an
InferenceGateway with spec.tls: its Service has two ports, and the
controller matches config frontends to ports by name, so the single
unnamed frontend left the load balancer one nameless frontend on port 0
with no backend.
Instead, a cloud half now declares what its load balancers need on every
gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's
KubernetesServiceSpec, which join() folds into the GatewayClass-level
EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with
mergeType: StrategicMerge, names its Service inference-gateway on every
cloud - one code path, with only the annotation depending on the cloud -
and on UpCloud alone overrides the config annotation with a named TCP
frontend per listener. On UpCloud the name is also load-bearing: the
controller already prefixes every load balancer with the cluster UUID,
so a Service name that differs from the cluster gateway's within the
truncated remainder is enough; the UUID-built name, the {gateway}
template and UpCloudCluster's clusterId status all go. On every other
cloud the rename replaces Envoy Gateway's generated
envoy-modelplane-system-inference-gateway-<hash> Service, so an
existing gateway gets a new LoadBalancer Service, with a new load
balancer and address. This is the mechanism modelplaneai#655 introduced for
CoreWeave, so both clouds share one shape.
The autoscaler fork lists every node group in the cluster each refresh
and gives any group missing from its --nodes list a minimum of one node
and a maximum of the cluster plan's limit, so it could scale the system
group down to one node or a fixed GPU pool up on pending pods, fighting
Crossplane as it reset each back to its nodeCount. Whenever any pool
autoscales, the group list now also pins the system group and every
fixed pool at their node count. compose_autoscaler_secret takes the
Autoscaler compose() already narrowed, a test now covers the node-group
sentinel that keeps groups composed through a Ready regression on
clusters without an autoscaler, and the node pool docs stop suggesting a
System pool beside the injected one, note that Premium and Starter plans
ignore diskSizeGb, and drop the stale GPU-pools-only wording.
Works around UpCloudLtd/upcloud-cloud-controller-manager#14.
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
* Add provider-upcloud to the project dependencies Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Add UpCloudCluster to provision UpCloud Managed Kubernetes clusters Modelplane can't provision clusters on UpCloud. This adds a namespaced UpCloudCluster XR and a compose-upcloud-cluster function that provision a UKS cluster on a dedicated SDN network, with a fixed system node group and a KubernetesNodeGroup per user pool. UKS has no inline pools, so unlike Civo and Vultr the system pool is its own Ready-gated node group. KubernetesNodeGroup has no name field, so the function pins each group's external name to its pool name, keeping UKS node group names deterministic. The UKS kubeconfig embeds static client certificates, so the connection secret alone reaches the cluster. fetchKubeconfig must be true or the provider publishes no kubeconfig at all. UKS has no server-side autoscaler and the cluster autoscaler's UpCloud cloud provider is an unmerged upstream PR (kubernetes/autoscaler#10234), so pools with maxNodeCount are scaled by UpCloud's fork image installed through the upstream chart. UpCloud provisions no API credentials on the cluster, so spec.autoscaler names a token Secret that an Object copies to kube-system, resolved server-side through patchesFrom so the token never passes through the function. The newest fork image is built for Kubernetes 1.29, older than any UKS version this XR provisions; it talks to the UpCloud API rather than version-skewed Kubernetes APIs, but the skew is outside the autoscaler's support policy. No ModelCache storage is composed: UpCloud File Storage is managed NFS without a CSI driver, so it only supports statically provisioned volumes. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Provision InferenceClusters on UpCloud An InferenceCluster could not be provisioned on UpCloud. This wires the new UpCloudCluster XR into the platform: source: UpCloud on InferenceCluster (zone, version, credentials and the autoscaler's token Secret), an upcloud provisioning block on InferenceClass naming the GPU server plan, and a hand-written UpCloud serving-stack half. UKS GPU node images ship the NVIDIA driver and container toolkit at the default root, so the half matches Nebius's shape: NFD and the DRA driver, no GPU Operator. With no operator there is no DCGM exporter either, so the modelplane_gpu_* and energy series are empty on UpCloud, the same gap Nebius has. The UKS kubeconfig embeds static client certificates, so the ClusterProviderConfig carries no identity. No cache StorageClass is relayed: UpCloud File Storage is NFS without a CSI driver. Fabric is not mapped; UKS has no EFA or InfiniBand analog. A CEL rule requires the upcloud.autoscaler Secret reference when a pool sets maxNodeCount, failing at admission rather than on the composed UpCloudCluster's own validation, where the user wouldn't see it. The upcloud e2e leg fakes GPU nodes the way compose-upcloud-cluster shapes them and audits the stack's placement on them. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Document the UpCloud provider UpCloud wasn't in the providers table or any of the provisioning docs. This adds the UKS row and tabs across the platform and getting-started pages, the UpCloud manifests they embed, and the API-reference example. The getting-started path includes a model deployment, which the other recent clouds still lack. The docs state what UpCloud doesn't support: no ModelCache (File Storage is NFS without a CSI driver, so only static volumes), no fabric, no GPU metrics (no GPU Operator, so no DCGM exporter), and GPU plans in fi-hel2 only with account-gated quota. Pools with maxNodeCount need the autoscaler token Secret, which the concepts manifest shows how to create. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * feat(e2e): disk size configurable, loadbalancer annotations Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * feat(nodes): take care that system is reserved, take care of autoscaler Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * feat(ss): add dcgm exporter and add tcp for lb Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * feat(parity): update branch after #556 merged, role from accelerator, cpu pools, draDriver tags, xrds update for accelerator, e2e leg, tests Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Declare gateway Service settings in the serving stack's cloud half UpCloud's load balancer needs - TCP frontend mode and names that survive the cloud controller's 64-character truncation - were plumbed as serviceAnnotations through the InferenceCluster status and ServingStack spec, with a {gateway} template and a per-Gateway EnvoyProxy that restated the externalTrafficPolicy pin. The annotation also broke an InferenceGateway with spec.tls: its Service has two ports, and the controller matches config frontends to ports by name, so the single unnamed frontend left the load balancer one nameless frontend on port 0 with no backend. Instead, a cloud half now declares what its load balancers need on every gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's KubernetesServiceSpec, which join() folds into the GatewayClass-level EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with mergeType: StrategicMerge, names its Service inference-gateway on every cloud - one code path, with only the annotation depending on the cloud - and on UpCloud alone overrides the config annotation with a named TCP frontend per listener. On UpCloud the name is also load-bearing: the controller already prefixes every load balancer with the cluster UUID, so a Service name that differs from the cluster gateway's within the truncated remainder is enough; the UUID-built name, the {gateway} template and UpCloudCluster's clusterId status all go. On every other cloud the rename replaces Envoy Gateway's generated envoy-modelplane-system-inference-gateway-<hash> Service, so an existing gateway gets a new LoadBalancer Service, with a new load balancer and address. This is the mechanism #655 introduced for CoreWeave, so both clouds share one shape. The autoscaler fork lists every node group in the cluster each refresh and gives any group missing from its --nodes list a minimum of one node and a maximum of the cluster plan's limit, so it could scale the system group down to one node or a fixed GPU pool up on pending pods, fighting Crossplane as it reset each back to its nodeCount. Whenever any pool autoscales, the group list now also pins the system group and every fixed pool at their node count. compose_autoscaler_secret takes the Autoscaler compose() already narrowed, a test now covers the node-group sentinel that keeps groups composed through a Ready regression on clusters without an autoscaler, and the node pool docs stop suggesting a System pool beside the injected one, note that Premium and Starter plans ignore diskSizeGb, and drop the stale GPU-pools-only wording. Works around UpCloudLtd/upcloud-cloud-controller-manager#14. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Stop installing the Prometheus stack on UpCloud the UpCloud half predates it and still carried the chart. The UpCloud e2e leg caught the consequence: its node-exporter DaemonSet tolerated both inference pool taints and landed on the fake GPU and CPU nodes. It goes the way #624 took it - the chart, the e2e leg's node-exporter fan-out expectation, and its key in the serving stack's UpCloud test cases. GPU telemetry is unaffected: the collector scrapes dcgm-exporter directly and never read this Prometheus. The gateway's LoadBalancer Service name is now pinned on every cloud rather than only on UpCloud, and UpCloud's frontend list derives from the same listener list the Gateway composes, so a listener added later reaches both. The gateway-level load balancer config replaces the class-level JSON value whole, so each side now points at the other for a key added to one. New tests pin what nothing covered: the UpCloud entry in the stacks' GATEWAY_SERVICE table, and the autoscaler pinning a fixed pool beside an autoscaled one. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> * Update the Vale styles hash for new upstream releases The style packages resolve to GitHub releases/latest zips, so the fixed-output hash breaks whenever one publishes a release. Rerun with the hash Nix reports, as the derivation's comment prescribes. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> --------- Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Description of your changes
Fixes #94.
Modelplane can't provision clusters on CoreWeave. This PR adds
CoreWeaveas anInferenceClustersource, backed by a newCoreWeaveClusterXR built on provider-coreweave.CKS has no cloud API for node pools. A pool is a
NodePoolresource inside the cluster, socompose-coreweave-clustercomposes each pool as a provider-kubernetesObjectonce the cluster is running. Usages block deleting the cluster and its kubeconfig until those Objects finish deleting. CKS's autoscaler sets an autoscaled pool'stargetNodes, so that pool'sObjectonly creates and deletes it. A secondObject, under its own field manager, applies everything buttargetNodes. Its apply leavestargetNodesin place and updates the pool's bounds, labels and taints.CKS doesn't publish a kubeconfig. Its API endpoint accepts the same CoreWeave API token the provider uses. The function renders a kubeconfig from the cluster's endpoint and the token in the CoreWeave ProviderConfig's Secret. That puts a copy of the org's token in a Secret beside the
CoreWeaveCluster, which is unavoidable until provider-kubernetes and provider-helm can read the token from provider-coreweave's Secret (crossplane-contrib/provider-kubernetes#603).CoreWeave manages the GPU driver, GPU Operator and NFD, so the serving stack's CoreWeave half installs cert-manager, the NVIDIA DRA driver and a DCGM exporter, and GPU pools get the label that keeps the managed device plugin off them. CoreWeave runs its own DCGM exporter too, but it only attributes GPUs the device plugin allocated. The collector skips it. CKS's own RWX class,
shared-vast, retains a volume after its claim goes, so ModelCache uses a composed copy withreclaimPolicy: Delete.CKS gives LoadBalancer Services private addresses unless annotated for a public one. A cloud half of the serving stack can now declare the Service settings its load balancers need, which go into the GatewayClass's EnvoyProxy. The InferenceGateway's EnvoyProxy sets
mergeType: StrategicMergeto inherit them, so neither gateway function knows which cloud it's on.A Usage blocks deleting the VPC until the cluster finishes deleting, because deleting a VPC a cluster still uses can leak it (upbound/provider-upjet-coreweave#4). Like the AKS and EKS autoscalers, CKS's can't scale a pool up from zero nodes when its pods' GPUs bind through DRA (#345). The XRD's
minNodeCountdescription says to keep it at 1 or higher.I have:
nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.