Skip to content

Provision InferenceClusters on UpCloud - #594

Merged
haarchri merged 11 commits into
modelplaneai:mainfrom
haarchri:feature/upcloud-provider
Nov 19, 2026
Merged

haarchri merged 11 commits into
modelplaneai:mainfrom
haarchri:feature/upcloud-provider

Conversation

@haarchri

@haarchri haarchri commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Description of your changes

Modelplane couldn't provision clusters on UpCloud. This adds source: UpCloud to InferenceCluster, backed by a new namespaced UpCloudCluster XR and a compose-upcloud-cluster function on crossplane-contrib/provider-upcloud v1.0.2. An UpCloudCluster provisions a UKS cluster on a dedicated SDN network (UKS requires one per cluster, DHCP on, default route off), a fixed system node group, and one KubernetesNodeGroup per pool. UKS has no inline pools, so unlike Civo and Vultr the system pool is itself a Ready-gated composed resource, and KubernetesNodeGroup has no name field, so each group's external name is pinned to its pool name to keep UKS node group names deterministic.

The cloud-specific decisions, and why:

Credentials. The UKS kubeconfig embeds static client certificates, so the ClusterProviderConfig carries no identity (same as Vultr and Civo). fetchKubeconfig: true is required or the provider publishes no kubeconfig to the connection secret. The provider's credentials Secret is JSON, {"token": ...} or {"username": ..., "password": ...}.

GPU stack. UKS GPU node images pre-install the NVIDIA driver and container toolkit at the default root, so the hand-written serving-stack half matches Nebius's shape: NFD plus the DRA driver, no GPU Operator.

Gateway load balancers. UpCloud's cloud controller hardcodes every load balancer frontend to HTTP mode, which can't carry the cluster gateway's TLS (the handshake is parsed as HTTP and the connection reset), and truncates load balancer names to -- at 64 characters with no unique suffix, so both gateways' generated Service names collide and every load balancer after the first fails with RESOURCE_ALREADY_EXISTS (UpCloudLtd/upcloud-cloud-controller-manager#14). The serving stack's cloud half declares what UpCloud needs on every gateway's LoadBalancer Service as a GATEWAY_SERVICE subset of Envoy Gateway's KubernetesServiceSpec, folded into the GatewayClass-level EnvoyProxy; the inference gateway's EnvoyProxy merges onto it with mergeType: StrategicMerge and, on UpCloud alone, overrides the config annotation with a named TCP frontend per listener (the controller matches frontends to Service ports by name, and its single-frontend fallback can't cover the two ports a TLS gateway has) and names its Service so the truncated names stay distinct — the controller's own UUID prefix keeps them account-unique. This is the mechanism #655 introduces for CoreWeave, applied ahead of it so both clouds share one shape; no InferenceCluster or ServingStack API surface is added for it.

Autoscaling. UKS has no server-side autoscaler and the cluster autoscaler's UpCloud cloud provider is an unmerged upstream PR (kubernetes/autoscaler#10234), so pools with maxNodeCount are scaled by UpCloud's fork image (ghcr.io/upcloudltd/autoscaler) installed through the upstream chart with image/env overrides. UpCloud provisions no API credentials on the cluster, so spec.autoscaler names a token Secret that an Object copies to kube-system; provider-kubernetes resolves the copy server-side through patchesFrom, so the token never passes through the function. Two tradeoffs stated plainly: the newest fork image is built for Kubernetes 1.29, older than any UKS version this XR provisions (it talks to the UpCloud API rather than version-skewed Kubernetes APIs, but the skew is outside the autoscaler's support policy), and the fork enforces a minimum of one node per group, so no scale-to-zero. The fork also lists every node group in the cluster each refresh and gives any group missing from its --nodes list a minimum of one and a maximum of the cluster plan's limit, so whenever any pool autoscales the group list also pins the system group and every fixed pool at their node count, keeping the autoscaler off the groups Crossplane owns. A CEL rule requires the autoscaler Secret when any pool sets maxNodeCount, failing at admission rather than on the composed XR where the user wouldn't see it.

ModelCache. None. UpCloud File Storage is managed NFS without a CSI driver, so volumes can only be provisioned statically; the docs and cache_storage_class docstring name the gap.

Fabric. Not mapped; UKS has no EFA or InfiniBand analog.

Zones. GPU plans are currently fi-hel2 only, with account-gated quota; the docs say so.

No breaking changes; UpCloud is additive to the source, provider and cloud enums.

Fixes #

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes.
  • Signed off every commit with git commit -s.

Tested

KUBECONFIG=upcloud.txt kubectl get pods -A                                                 
NAMESPACE                 NAME                                                              READY   STATUS    RESTARTS      AGE
cert-manager              mp-cert-manager-6c654b75b8-4j9lw                                  1/1     Running   0             25m
cert-manager              mp-cert-manager-cainjector-6f97bd496-k6npf                        1/1     Running   0             25m
cert-manager              mp-cert-manager-webhook-667c68bdb5-q6qwf                          1/1     Running   0             25m
envoy-ai-gateway-system   ai-gateway-controller-fbf6db65-znpj9                              1/1     Running   3 (25m ago)   25m
envoy-gateway-system      envoy-gateway-76c8656c8b-9j26d                                    1/1     Running   0             24m
envoy-gateway-system      envoy-modelplane-system-cluster-gateway-b26a0919-7d5865465lhksh   2/2     Running   0             21m
envoy-gateway-system      envoy-modelplane-system-inference-gateway-5dbfd6c0-8fdcbc9rwn24   3/3     Running   0             13m
envoy-gateway-system      envoy-modelplane-system-inference-gateway-5dbfd6c0-8fdcbc9zgbll   3/3     Running   0             13m
kube-system               cilium-7bz96                                                      1/1     Running   0             27m
kube-system               cilium-envoy-2fvmc                                                1/1     Running   0             27m
kube-system               cilium-envoy-5jxmv                                                1/1     Running   0             27m
kube-system               cilium-envoy-zbdqh                                                1/1     Running   0             28m
kube-system               cilium-operator-5b88b57d6d-7fk84                                  1/1     Running   0             30m
kube-system               cilium-operator-5b88b57d6d-sprvb                                  1/1     Running   0             30m
kube-system               cilium-tcxzb                                                      1/1     Running   0             28m
kube-system               cilium-x4zjm                                                      1/1     Running   0             27m
kube-system               coredns-7645c8648f-jgljj                                          1/1     Running   0             30m
kube-system               coredns-7645c8648f-nkgx6                                          1/1     Running   0             30m
kube-system               csi-upcloud-node-77cff                                            2/2     Running   0             27m
kube-system               csi-upcloud-node-lhqn9                                            2/2     Running   0             27m
kube-system               csi-upcloud-node-rkvsd                                            2/2     Running   0             28m
kube-system               csi-upcloud-snapshot-controller-75f44cbf75-dr96d                  1/1     Running   0             29m
kube-system               csi-upcloud-snapshot-controller-75f44cbf75-wqk56                  1/1     Running   0             29m
kube-system               konnectivity-agent-cqjpj                                          1/1     Running   0             27m
kube-system               konnectivity-agent-qsq6f                                          1/1     Running   0             28m
kube-system               konnectivity-agent-xr2dm                                          1/1     Running   0             27m
kube-system               upcloud-fi-hel2-df1e896d5b12-upcloud-cluster-autoscaler-84bxtg6   1/1     Running   3 (25m ago)   28m
lws-system                mp-lws-controller-manager-7db95fdfb6-t9wms                        1/1     Running   0             25m
modelplane-system         trust-manager-756bd996f4-s4pk8                                    1/1     Running   0             23m
monitoring                mp-kube-prometheus-stack-kube-state-metrics-6845b99bf8-75trm      1/1     Running   0             25m
monitoring                mp-kube-prometheus-stack-prometheus-node-exporter-4zvfn           1/1     Running   0             25m
monitoring                mp-kube-prometheus-stack-prometheus-node-exporter-dgdbq           1/1     Running   0             25m
monitoring                mp-kube-prometheus-stack-prometheus-node-exporter-sv55k           1/1     Running   0             25m
monitoring                prometheus-operator-594578d747-zlsx4                              1/1     Running   0             25m
monitoring                prometheus-prometheus-prometheus-0                                2/2     Running   0             25m
mp-ml-team-51733          qwen-demo-18490-epp-c4f8c958b-q9pnf                               1/1     Running   0             17m
mp-ml-team-51733          qwen-demo-18490-qwen-1abe1-7698bf8f-nl9j2                         1/1     Running   0             17m
node-feature-discovery    mp-node-feature-discovery-gc-d884d7585-nbnlb                      1/1     Running   0             25m
node-feature-discovery    mp-node-feature-discovery-master-7fc9fb989c-tnp6f                 1/1     Running   0             25m
node-feature-discovery    mp-node-feature-discovery-worker-lfb67                            1/1     Running   0             25m
node-feature-discovery    mp-node-feature-discovery-worker-mmcxb                            1/1     Running   0             25m
node-feature-discovery    mp-node-feature-discovery-worker-vwj5q                            1/1     Running   0             25m
nvidia-dra-driver         dra-driver-nvidia-gpu-kubelet-plugin-bd6l8                        1/1     Running   0             25m

curl -sS "http://lb-0ae251bcbc7a49bba5f26922b77bfcc0-1.upcloudlb.com/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"model":"ml-team/qwen","messages":[{"role":"user","content":"Say Hello from UpCloud"}],"max_tokens":100}'
{"id":"chatcmpl-50c6e3ce-37f2-4d03-9650-27527ea00bdb","object":"chat.completion","created":1793302134,"model":"ml-team/qwen-demo","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! Welcome to UpCloud. How can I assist you today?","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.23.0-7f04f88c","usage":{"prompt_tokens":34,"total_tokens":49,"completion_tokens":15,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"prompt_text":null,"kv_transfer_params":null}% 

@github-actions

Copy link
Copy Markdown

Docs preview: https://modelplane-docs-pr-594.vercel.app (ready once the site's Content workflow finishes)

@dennis-upbound dennis-upbound left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compose-upcloud-cluster: removing spec.autoscaler orphans a running autoscaler (should-fix)
The ProviderConfigs, the token Object and the Release are composed only while spec.autoscaler is set. When a user turns their last autoscaled pool into a fixed pool and drops autoscaler (which the CEL rule then allows), all three leave desired state. The Release and Object exclude Delete. The autoscaler keeps running with its old group list, and its node counts fight the nodeCount Crossplane now enforces on those pools. compose_cluster_autoscaler's docstring already handles this for the last pool going fixed. Composing the sub-block when spec.autoscaler or "release-cluster-autoscaler" in observed would cover this case the same way, with an empty group list.

Pool named system (should-fix)
On this one I'd reserve the name with CEL. The PR body says the other clouds can't hit it, but EKS and Nebius key their groups nodegroup-{name} next to nodegroup-system, and GKE uses nodepool-system, so they have the same collision. One rule on InferenceCluster spec.nodePools[].name would cover every cloud. Here the external name is also system, so a declared system pool would take over the system group in UKS as well.

Merge order with #556 (FYI)
_node_pool and compose_upcloud_cluster hard-code role="GPU". Once #556 merges, UpCloud needs the same changes the other six clouds got there: the role derived from accelerator, draDrivers tags on its stack and CPU pools.

Before ready: the checklist is unchecked, and Tested has no live UKS run yet (the e2e-clouds (upcloud) leg is KWOK).

@haarchri

Copy link
Copy Markdown
Contributor Author

found one issue in the e2e run: UpCloudLtd/upcloud-cloud-controller-manager#14

@haarchri

Copy link
Copy Markdown
Contributor Author

because of the LB issue - i updated the code to set annotations to override the default behaviour until UpCloudLtd/upcloud-cloud-controller-manager#3 gets shipped, disk size configurable like the other clouds because the default is to low for models and added dcgm exporter

@haarchri
haarchri marked this pull request as ready for review October 29, 2026 19:52
@haarchri
haarchri force-pushed the feature/upcloud-provider branch 3 times, most recently from 4e580c9 to 9ebd199 Compare October 30, 2026 13:47

@dennis-upbound dennis-upbound left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me, and you fixed both points from my earlier comments. Admission now rejects pools named system, and the function keeps composing the autoscaler release while it's observed, so removing spec.autoscaler doesn't orphan it. Thanks for the live UKS run in the description too.

I'd like these cleaned up before I approve:

  • The PR description is out of date. It still says UpCloud has no DCGM exporter, but stacks/clouds/upcloud.py installs one now. The "Known rough edge" paragraph leaves reserving system open for review, and both XRDs now have CEL rules for it.
  • Some commits use feat(...) prefixes, which CONTRIBUTING.md asks us not to use. Two of the older commit bodies also still say there's no DCGM exporter. A reword or squash would fix both.

@haarchri

Copy link
Copy Markdown
Contributor Author

@dennis-upbound updated description we can Squash during merge ?!

@haarchri

haarchri commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

/review

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it needs another look at two things before it's close: which node groups the UpCloud autoscaler touches, and the load balancer frontends on a gateway with more than one port.

The cluster XR, the UpCloud serving-stack half and leaving out ModelCache follow the pattern Civo and Vultr set, and I didn't find anything that conflicts with a committed design. The design questions are about the gateway serviceAnnotations plumbing and the autoscaler fork. The gateway change alters how every cloud composes its gateways, and it has no issue behind it, which CONTRIBUTING asks for before an API change.

The PR description doesn't mention the serviceAnnotations fields, the per-Gateway EnvoyProxy or the TCP frontend override, and those are the parts that need review judgment.

Reviewed by modelplane-review 5455585. Reply, or react 👍 or 👎, on each comment to help tune it.

Comment thread apis/inferenceclusters/definition.yaml Resolved
Comment thread apis/inferenceclasses/definition.yaml Resolved
@negz negz moved this from In Review to Done in Modelplane Roadmap Nov 5, 2026
@negz negz moved this from Done to In progress in Modelplane Roadmap Nov 5, 2026
@negz negz moved this from In progress to In Review in Modelplane Roadmap Nov 5, 2026
@haarchri
haarchri force-pushed the feature/upcloud-provider branch from 9ebd199 to 4c7ee2f Compare November 5, 2026 20:19
@modelplane-review modelplane-review Bot mentioned this pull request Nov 5, 2026
4 tasks done
@negz

negz commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Heads up: #655 (CoreWeave) needed per-cloud LoadBalancer Service annotations too, and solves it a bit differently from serviceAnnotations. CKS gives LoadBalancer Services a private IP unless they're annotated service.beta.kubernetes.io/coreweave-load-balancer-type: public.

In #655 a cloud half of the serving stack declares the Service settings its load balancers need, as a subset of Envoy Gateway's KubernetesServiceSpec. They go into the GatewayClass-level EnvoyProxy the serving stack already composes. The InferenceGateway's EnvoyProxy sets mergeType: StrategicMerge, so Envoy Gateway merges it onto the class-level one rather than replacing it. Neither gateway function needs to know the cloud, and there's no new InferenceCluster or ServingStack field and no extra per-Gateway EnvoyProxy restating the externalTrafficPolicy pin. On a real CKS cluster both gateway Services got the annotation and public IPs.

For UpCloud, the TCP frontend mode looks like it could be declared the same way. The per-gateway LB name is the part that doesn't fit, since it depends on the cluster UUID. Giving each gateway's Service a distinct name (envoyService.name) might avoid the truncation collision, but I haven't checked UpCloud's naming. Failing that, a runtime value could be projected onto the ServingStack and merged in. #655 will probably land first, so it'd be good for #594 to build on the same mechanism rather than add a second one.

@haarchri
haarchri force-pushed the feature/upcloud-provider branch from 099c45e to b2e6856 Compare November 6, 2026 15:59
@haarchri

haarchri commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/review

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks close. The gateway rework and autoscaler pinning answer the earlier threads. The open design question is whether compose-inference-gateway should know about UpCloud. Beyond that, this PR and #655 need to settle who merges the gateway code they share, and two new behaviours lack tests.

Moving onto #655's mechanism dropped serviceAnnotations, the {gateway} template and status.clusterId. I traced the merged EnvoyProxy, the Service port names and the cloud controller's load balancer naming through Envoy Gateway v1.8.4 and the UpCloud cloud controller, and they match what the code expects.

Reviewed by modelplane-review 3017043. Reply, or react 👍 or 👎, on each comment to help tune it.

@haarchri

haarchri commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

e2e test done:

curl -sS "http://lb-0ae251bcbc7a49bba5f26922b77bfcc0-1.upcloudlb.com/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ml-team/qwen",
    "messages": [
      {
        "role": "user",
        "content": "Explain what UpCloud offers in one sentence."
      }
    ],
    "max_tokens": 100
  }'

and the response:

{
  "id": "chatcmpl-8d7f2a91-6c43-4b5e-a219-93f0c82d714e",
  "object": "chat.completion",
  "created": 1793990192,
  "model": "ml-team/qwen-demo",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "UpCloud provides high-performance cloud infrastructure, including scalable virtual machines, storage, networking, and managed services for modern applications.",
        "refusal": null,
        "annotations": null,
        "audio": null,
        "function_call": null,
        "tool_calls": [],
        "reasoning": null
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null,
      "token_ids": null,
      "routed_experts": null
    }
  ],
  "service_tier": null,
  "system_fingerprint": "vllm-0.23.0-a91c72e4",
  "usage": {
    "prompt_tokens": 32,
    "total_tokens": 56,
    "completion_tokens": 24,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "prompt_token_ids": null,
  "prompt_text": null,
  "kv_transfer_params": null
}

@haarchri

haarchri commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/review

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks close. The new commit addresses the earlier threads on the gateway's cloud branching and the missing tests. What's left is two wording points and how the commit and description present the change.

All three affected functions' tests pass at this commit. Nothing in the functions, e2e or docs looks up the inference gateway's generated Service name or uses the removed UpCloud Prometheus.

The commit bundles the Prometheus removal, the gateway rework and two new tests under one subject, where CONTRIBUTING asks authors to merge each into the commit it belongs to. The Tested pod listing in the description still shows the Prometheus stack this PR no longer installs.

Reviewed by modelplane-review 3017043. Reply, or react 👍 or 👎, on each comment to help tune it.

@jonasz-lasut

Copy link
Copy Markdown

@haarchri could you bump the provider to v1.1.0 which was released today? We've bumped the underlying TF provider to the latest version and resolved all existing CVEs

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Modelplane can't provision clusters on UpCloud. This adds a namespaced
UpCloudCluster XR and a compose-upcloud-cluster function that provision
a UKS cluster on a dedicated SDN network, with a fixed system node
group and a KubernetesNodeGroup per user pool. UKS has no inline pools,
so unlike Civo and Vultr the system pool is its own Ready-gated node
group. KubernetesNodeGroup has no name field, so the function pins each
group's external name to its pool name, keeping UKS node group names
deterministic.

The UKS kubeconfig embeds static client certificates, so the connection
secret alone reaches the cluster. fetchKubeconfig must be true or the
provider publishes no kubeconfig at all.

UKS has no server-side autoscaler and the cluster autoscaler's UpCloud
cloud provider is an unmerged upstream PR (kubernetes/autoscaler#10234),
so pools with maxNodeCount are scaled by UpCloud's fork image installed
through the upstream chart. UpCloud provisions no API credentials on
the cluster, so spec.autoscaler names a token Secret that an Object
copies to kube-system, resolved server-side through patchesFrom so the
token never passes through the function. The newest fork image is built
for Kubernetes 1.29, older than any UKS version this XR provisions; it
talks to the UpCloud API rather than version-skewed Kubernetes APIs,
but the skew is outside the autoscaler's support policy.

No ModelCache storage is composed: UpCloud File Storage is managed NFS
without a CSI driver, so it only supports statically provisioned
volumes.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
An InferenceCluster could not be provisioned on UpCloud. This wires the
new UpCloudCluster XR into the platform: source: UpCloud on
InferenceCluster (zone, version, credentials and the autoscaler's token
Secret), an upcloud provisioning block on InferenceClass naming the GPU
server plan, and a hand-written UpCloud serving-stack half.

UKS GPU node images ship the NVIDIA driver and container toolkit at the
default root, so the half matches Nebius's shape: NFD and the DRA
driver, no GPU Operator. With no operator there is no DCGM exporter
either, so the modelplane_gpu_* and energy series are empty on UpCloud,
the same gap Nebius has.

The UKS kubeconfig embeds static client certificates, so the
ClusterProviderConfig carries no identity. No cache StorageClass is
relayed: UpCloud File Storage is NFS without a CSI driver. Fabric is
not mapped; UKS has no EFA or InfiniBand analog.

A CEL rule requires the upcloud.autoscaler Secret reference when a pool
sets maxNodeCount, failing at admission rather than on the composed
UpCloudCluster's own validation, where the user wouldn't see it.

The upcloud e2e leg fakes GPU nodes the way compose-upcloud-cluster
shapes them and audits the stack's placement on them.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
UpCloud wasn't in the providers table or any of the provisioning docs.
This adds the UKS row and tabs across the platform and getting-started
pages, the UpCloud manifests they embed, and the API-reference example.
The getting-started path includes a model deployment, which the other
recent clouds still lack.

The docs state what UpCloud doesn't support: no ModelCache (File
Storage is NFS without a CSI driver, so only static volumes), no
fabric, no GPU metrics (no GPU Operator, so no DCGM exporter), and GPU
plans in fi-hel2 only with account-gated quota. Pools with maxNodeCount
need the autoscaler token Secret, which the concepts manifest shows how
to create.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
@haarchri
haarchri force-pushed the feature/upcloud-provider branch 2 times, most recently from 0f6594a to 65eed54 Compare November 17, 2026 12:12
@haarchri

Copy link
Copy Markdown
Contributor Author

/review

@negz

negz commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

/review

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks close to ready. The design hasn't changed since my last review, and what's left is two nits and tidying the history.

The rebase onto #655 and the provider bump look right. The tests of compose-inference-cluster, compose-inference-gateway, compose-serving-stack and compose-upcloud-cluster pass at 65eed54. Between provider-upcloud v1.0.2 and v1.1.0, the only change to the CRDs compose-upcloud-cluster uses is the controller-gen annotation.

The fold into 275a2c9 looks partial. At that commit _upcloud_envoy_service holds the listener-building body, so it seems the EnvoyProxy's envoyService gets the listener list, and nothing calls _envoy_service. Could the gateway rework move whole into 275a2c9, and f2cc33a get back the opening line its body lost? The Vale hash commit repeats #678, already on main, so it can go. The PR description still says provider-upcloud v1.0.2 and that #655 hasn't merged.

Reviewed by modelplane-review 22e194c. Reply, or react 👍 or 👎, on each comment to help tune it.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
…accelerator, cpu pools, draDriver tags, xrds update for accelerator, e2e leg, tests

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
UpCloud's load balancer needs - TCP frontend mode and names that survive
the cloud controller's 64-character truncation - were plumbed as
serviceAnnotations through the InferenceCluster status and ServingStack
spec, with a {gateway} template and a per-Gateway EnvoyProxy that
restated the externalTrafficPolicy pin. The annotation also broke an
InferenceGateway with spec.tls: its Service has two ports, and the
controller matches config frontends to ports by name, so the single
unnamed frontend left the load balancer one nameless frontend on port 0
with no backend.

Instead, a cloud half now declares what its load balancers need on every
gateway Service as a GATEWAY_SERVICE subset of Envoy Gateway's
KubernetesServiceSpec, which join() folds into the GatewayClass-level
EnvoyProxy. The inference gateway's EnvoyProxy merges onto it with
mergeType: StrategicMerge, names its Service inference-gateway on every
cloud - one code path, with only the annotation depending on the cloud -
and on UpCloud alone overrides the config annotation with a named TCP
frontend per listener. On UpCloud the name is also load-bearing: the
controller already prefixes every load balancer with the cluster UUID,
so a Service name that differs from the cluster gateway's within the
truncated remainder is enough; the UUID-built name, the {gateway}
template and UpCloudCluster's clusterId status all go. On every other
cloud the rename replaces Envoy Gateway's generated
envoy-modelplane-system-inference-gateway-<hash> Service, so an
existing gateway gets a new LoadBalancer Service, with a new load
balancer and address. This is the mechanism modelplaneai#655 introduced for
CoreWeave, so both clouds share one shape.

The autoscaler fork lists every node group in the cluster each refresh
and gives any group missing from its --nodes list a minimum of one node
and a maximum of the cluster plan's limit, so it could scale the system
group down to one node or a fixed GPU pool up on pending pods, fighting
Crossplane as it reset each back to its nodeCount. Whenever any pool
autoscales, the group list now also pins the system group and every
fixed pool at their node count. compose_autoscaler_secret takes the
Autoscaler compose() already narrowed, a test now covers the node-group
sentinel that keeps groups composed through a Ready regression on
clusters without an autoscaler, and the node pool docs stop suggesting a
System pool beside the injected one, note that Premium and Starter plans
ignore diskSizeGb, and drop the stale GPU-pools-only wording.

Works around UpCloudLtd/upcloud-cloud-controller-manager#14.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
the UpCloud half predates it and still carried the chart. The UpCloud
e2e leg caught the consequence: its node-exporter DaemonSet tolerated
both inference pool taints and landed on the fake GPU and CPU nodes.
It goes the way modelplaneai#624 took it - the chart, the e2e leg's node-exporter
fan-out expectation, and its key in the serving stack's UpCloud test
cases. GPU telemetry is unaffected: the collector scrapes
dcgm-exporter directly and never read this Prometheus.

The gateway's LoadBalancer Service name is now pinned on every cloud
rather than only on UpCloud, and UpCloud's frontend list derives from
the same listener list the Gateway composes, so a listener added later
reaches both. The gateway-level load balancer config replaces the
class-level JSON value whole, so each side now points at the other for
a key added to one. New tests pin what nothing covered: the UpCloud
entry in the stacks' GATEWAY_SERVICE table, and the autoscaler pinning
a fixed pool beside an autoscaled one.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
The style packages resolve to GitHub releases/latest zips, so the
fixed-output hash breaks whenever one publishes a release. Rerun with
the hash Nix reports, as the derivation's comment prescribes.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
@haarchri
haarchri force-pushed the feature/upcloud-provider branch from 65eed54 to 48ab82c Compare November 19, 2026 14:17
@haarchri
haarchri merged commit 5fbfcca into modelplaneai:main Nov 19, 2026
16 checks passed
@haarchri
haarchri deleted the feature/upcloud-provider branch November 19, 2026 15:08
@haarchri haarchri mentioned this pull request Nov 19, 2026
4 tasks done
@negz negz moved this from In Review to Done in Modelplane Roadmap Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants