diff --git a/content/blog/2026-11-06-modelplane-v0-7/index.mdx b/content/blog/2026-11-06-modelplane-v0-7/index.mdx new file mode 100644 index 0000000..2d5bad1 --- /dev/null +++ b/content/blog/2026-11-06-modelplane-v0-7/index.mdx @@ -0,0 +1,160 @@ +--- +title: "Modelplane v0.7: CoreWeave, and a steadier fleet" +description: "Modelplane v0.7 provisions inference clusters on CoreWeave, and smooths out rollouts, node drains, and new gateways across the fleet." +date: "2026-11-06" +authors: + - name: "Nic Cope" + title: "Sr Principal Engineer, Upbound" + url: "https://github.com/negz" + avatar: "/authors/nic.jpg" + github: "https://github.com/negz" + linkedin: "https://www.linkedin.com/in/ncope/" + bio: "Nic is a Senior Principal Engineer at Upbound and a core maintainer of Modelplane and Crossplane. He's spent the last 15+ years working on cloud and infrastructure, including designing and building Crossplane from v0.1 through v2.0, and is now bringing that work to AI inference with Modelplane." +tags: ["release", "inference", "control-plane", "coreweave"] +cover: "/blog/modelplane-v0-7/cover.png" +draft: false +pinned: true +--- + +Modelplane v0.7 is out, and it brings CoreWeave into the fleet. A platform team +can now provision CoreWeave Kubernetes Service (CKS) clusters right alongside +EKS, AKS, GKE, and the other neoclouds Modelplane supports. AMD GPU pools also +share nodes between small models now, as NVIDIA pools have since v0.6. + +With KubeCon in Salt Lake City this week, we spent the rest of the release +running real fleets and smoothing out what we found. Rollouts finish on a full +GPU pool, a model's endpoint picker stays up through a node drain, and GPU metrics tell +you which deployment is using each GPU. + +## CoreWeave + +Set an `InferenceCluster`'s source to `CoreWeave` and Modelplane provisions a +CKS cluster with its VPC, a system node pool, and the GPU node pools you list, +then installs the serving stack. Here's one node of eight NVIDIA RTX PRO 6000 +Blackwell GPUs, from the getting-started guide: + +```yaml +apiVersion: modelplane.ai/v1alpha1 +kind: InferenceClass +metadata: + name: rtxp6000-8x +spec: + description: "CoreWeave rtxp6000-8x, 8x NVIDIA RTX PRO 6000 Blackwell 96GB" + provisioning: + provider: CoreWeave + coreweave: + # CKS instance types vary by zone. + instanceType: rtxp6000-8x + accelerator: + type: nvidia-rtx-pro-6000 + count: 8 + devices: + - name: gpu + claim: DRA + driver: gpu.nvidia.com + deviceClassName: gpu.nvidia.com + count: 8 + attributes: + architecture: { string: Blackwell } + cudaComputeCapability: { version: "12.0.0" } + capacity: + memory: { value: "97887Mi" } # RTX PRO 6000's real reported VRAM +--- +apiVersion: modelplane.ai/v1alpha1 +kind: InferenceCluster +metadata: + name: coreweave-us-east-04 + labels: + modelplane.ai/region: us-east-04 +spec: + cluster: + source: CoreWeave + coreweave: + zone: US-EAST-04A + nodePools: + - name: gpu-rtxp6000 + className: rtxp6000-8x + nodeCount: 1 + minNodeCount: 1 + maxNodeCount: 1 +``` + +The ML team doesn't change a thing. A `ModelDeployment` that runs on EKS or +Nebius runs on CoreWeave too, on any pool whose devices match its selectors. + +Thanks to CoreWeave for the GPU capacity we built and tested this on. Bringing +up a new cloud always shakes out a few bugs, and working through them with +CoreWeave's engineers has been a lot of fun. Their networking team is tracking +one where deleting a VPC right after its cluster can leave the VPC behind +([provider-upjet-coreweave#4](https://github.com/upbound/provider-upjet-coreweave/issues/4)). +They also spotted our provider updating some VPCs in a tight loop and asked us +about it +([provider-upjet-coreweave#6](https://github.com/upbound/provider-upjet-coreweave/issues/6)). +We've sent them a fix in +[terraform-provider-coreweave#554](https://github.com/coreweave/terraform-provider-coreweave/pull/554). + +## Rollouts that finish on a full pool + +GPUs are expensive, so pools tend to run full. That used to make rolling out a +new engine version awkward. An engine's Deployment started a new pod before +stopping an old one, and on a pool with no free GPU the new pod waited for a GPU +the old pod would never release. Modelplane now replaces one copy at a time, on +the GPU the old copy frees. LeaderWorkerSet and Grove gangs already worked this +way. + +A deployment also reports ready only once every copy runs the current spec, +rather than partway through a rollout. Switching an engine from `Standalone` to +a `Leader` and `Worker` gang cleans up the old Deployment too, so it doesn't +hold on to the GPUs its replacement needs. + +## Serving through node drains + +Draining a node used to take a model offline for minutes. Each replica's +endpoint picker, which chooses the engine copy that serves each request, ran as +one pod, and the cluster's gateway won't route without it. On Nebius a +replacement picker took about four minutes to start. Modelplane now runs two +pickers per replica, spread across nodes, with a disruption budget that keeps a +drain from evicting both. + +New gateways are safer to route to, too. The first `ModelRoute` on a new +`InferenceGateway` could report ready while Envoy AI Gateway was still rolling +the gateway, so requests to it returned HTTP 500s. A gateway now waits for that +rollout before it reports ready. It also keeps a placeholder route attached, so +the first real route doesn't trigger another rollout. + +## Which deployment is using that GPU? + +GPU metrics have the same `deployment`, `replica`, `role`, and `engine` labels +as engine metrics. A dashboard can break GPU utilisation down by deployment and +show it next to that deployment's queue depth and latency. + +Inference clusters are lighter too. Modelplane's telemetry collector scrapes +engines and GPUs itself, which means clusters don't need a Prometheus stack of +their own anymore. + +## Try it on your laptop + +The new [local demo](https://docs.modelplane.ai/getting-started/local-demo/) +runs Modelplane on two kind clusters on your laptop, with fake GPUs and models +small enough for a CPU. You play both teams, building a small fleet and then +upgrading a model behind the same name. + +When you're ready for real GPUs, the +[getting-started guide](https://docs.modelplane.ai/getting-started/) covers +Vultr and Civo end to end, and a new +[recipe](https://docs.modelplane.ai/recipes/kimi-k3.1-pro/) serves Moonshot AI's +Kimi K3.1 Pro with SGLang on one 8x B300 node. + +## Come find us at KubeCon + +The full release notes are on +[GitHub](https://github.com/modelplaneai/modelplane/releases/tag/v0.7.0). +Modelplane is Apache 2.0 at +[github.com/modelplaneai/modelplane](https://github.com/modelplaneai/modelplane), +and now has a +[charter](https://github.com/modelplaneai/modelplane/blob/main/CHARTER.md) +setting out the project's scope, plus a +[governance](https://github.com/modelplaneai/modelplane/blob/main/GOVERNANCE.md) +model adapted from Crossplane's. Questions are welcome in +[Slack](https://slack.modelplane.ai). If you're at KubeCon this week, come find +us at the Upbound booth. diff --git a/public/blog/modelplane-v0-7/cover.png b/public/blog/modelplane-v0-7/cover.png new file mode 100644 index 0000000..b7ec273 Binary files /dev/null and b/public/blog/modelplane-v0-7/cover.png differ