Skip to content

Add the add-inference-cluster-provider agent skill - #654

Open
haarchri wants to merge 1 commit into
modelplaneai:mainfrom
haarchri:feature/skill-inference-cluster
Open

haarchri wants to merge 1 commit into
modelplaneai:mainfrom
haarchri:feature/skill-inference-cluster

Conversation

@haarchri

@haarchri haarchri commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Description of your changes

Adding a new provisioned cloud as an InferenceCluster source is a wide change that touches a Cluster XR, a compose--cluster function, dispatch in compose-inference-cluster, the serving-stack cloud half, the e2e leg, and docs.

first PR implemented with: #594
starting point for: #331

Fixes #

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes.
  • Signed off every commit with git commit -s.

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>

@modelplane-review modelplane-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a step back on the approach. Much of the skill describes the UpCloud work in #594, which is still open, as code on main, and several claims about main are already wrong. #191 deleted the last function skill because it had drifted from the code the same way. It's also not clear a maintainer has chosen a skill over the other options #331 lists. Settling whether this waits for #594, and how it keeps matching the code as more clouds land, matters more than the inline fixes.

The commit subject feat(skill): add inference cluster has a typed prefix and no body, where CONTRIBUTING asks for no prefix and a body. The PR description leaves Fixes # blank without saying what problem the skill solves.

Reviewed by modelplane-review d148f8c. Reply, or react 👍 or 👎, on each comment to help tune it.


# Add a provisioned InferenceCluster provider

You are adding `source: <Cloud>` to `InferenceCluster` so Modelplane provisions

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has a maintainer picked the skill option on #331? The issue lists a contract, a skill, conformance tests and one canonical implementation. It notes that "docs and skills inform but don't enforce", and that tests catch the behavioural drift a doc can't. A check like hack/check_function_tests.py could enforce some of these rules, such as gating and orphan management. Could the contract go in CONTRIBUTING or a design, with this skill pointing at it?

|---|---|
| Kubeconfig auth | the kubeconfig alone (AKS, Vultr, Civo client certs; EKS via ClusterAuth) / identity layered on the ProviderConfig (GKE, Nebius) |
| Dependents gating | one pass, children reference the cluster by selector (AKS, EKS, GKE, Nebius) / gated on cluster `Ready` with `_dependents_observed()` because they need an observed ID (Vultr, Civo) |
| Autoscaling | server-side per pool (AKS, GKE, Nebius) / in-cluster cluster-autoscaler release (EKS, Civo) / fixed pools |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't Vultr server-side too? compose-vultr-cluster/function/fn.py says VKE serves node pool autoscaling itself, and sets the pool's autoScaler block. No provider on main uses fixed pools, so the "Options in the repo today" header promises an example that doesn't exist.

| Dependents gating | one pass, children reference the cluster by selector (AKS, EKS, GKE, Nebius) / gated on cluster `Ready` with `_dependents_observed()` because they need an observed ID (Vultr, Civo) |
| Autoscaling | server-side per pool (AKS, GKE, Nebius) / in-cluster cluster-autoscaler release (EKS, Civo) / fixed pools |
| GPU stack | generated from AICR (EKS, AKS, GKE) / hand-written half: driver in image (Nebius, UpCloud), managed GPU Operator add-on (Vultr), nothing pre-installed (Civo) |
| GPU metrics | DCGM exporter from the GPU operator (EKS, AKS, GKE, Civo) or the managed add-on (Vultr) / standalone `dcgm-exporter` chart when the driver is in the image (UpCloud) / gap stated (Nebius, which predates the standalone chart) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't Nebius already run the standalone exporter? stacks/clouds/nebius.py composes the dcgm-exporter chart with DCGM_METRICS_CSV, kubernetesDRA and the PCI affinity terms, and the e2e leg expects mp-dcgm-exporter on Nebius. So "gap stated (Nebius...)" here and "Nebius's gap predates the standalone chart" at line 238 look wrong. Line 234 points at upcloud.py, which isn't on main. Could these point at nebius.py instead?


## Phase 1: Add the dependency (commit 1)

1. `crossplane-project.yaml`: add the provider under `spec.dependsOn`, in the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few names don't match main (here and elsewhere):

  • crossplane-project.yaml lists providers under spec.dependencies, not spec.dependsOn.
  • The cache method is observed_cache_storage_class(), not cache_storage_class() (lines 342 and 345).
  • The Vale vocabulary is at .vale/styles/config/vocabularies/Modelplane/accept.txt, as CONTRIBUTING says, not under docs/utils/ (line 304).

An agent following these would look for names and paths that don't exist.

(`System`, `GPU`, `CPU`), size, `nodeCount`, optional
`minNodeCount`/`maxNodeCount`, `diskSizeGb` when the cloud unbundles disks
from the plan, `gpu` required for GPU pools via CEL, and `name != 'system'`
via CEL — `system` is the injected system pool's name on every provisioned

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the name != 'system' CEL rule live? I can't find a rule on 'system' in any XRD under apis/. InferenceCluster's nodePools[].name has only a pattern and maxLength. So "the same rule guards InferenceCluster's nodePools" looks untrue, and an agent would add API validation nobody has agreed to. Could the skill drop the rule, or could a separate PR add it to the existing XRDs first?

with `common.py` and the stack file (`standard.py` / `dynamo.py`). You don't
touch those, the generator, or `generated/aicr/` (EKS, AKS, GKE only). The
half supplies the hardware surface the stack contract needs: cert-manager,
kube-prometheus-stack, NFD unless the cloud runs one, the GPU Operator only

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should a cloud half still supply kube-prometheus-stack? No stack under compose-serving-stack/function/stacks/ has it, and generate.py lists it as dropped because nothing in Modelplane reads an in-cluster Prometheus (#551). An agent following this would add back a component the repo removed on purpose.

- `docs/content/platform/providers.md`: provisioning column ✓, `repolink` to
the provider actually used, cloud added to the "Provisioning supported"
sentence.
- `docs/content/platform/inference-cluster.md`: source lists, a tab for

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this list need scale-the-platform.md and scale-the-model.md? Each provisioned cloud has a tab on both pages, backed by getting-started/<cloud>/platform-scale.yaml and model-deployment-west.yaml. A hand-kept list like this drifts. Could it point to the git grep sweep from Phase 3 instead?

credentials (secret + `clusterproviderconfig-<cloud>.yaml`), platform (with
a realistic wait), gateway tabs.
- `docs/content/getting-started/deploying-a-model.md`: a tab for
`getting-started/<cloud>/model-deployment.yaml`. Vultr and Civo don't have

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this out of date? docs/manifests/getting-started/{vultr,civo}/model-deployment.yaml both exist, and deploying-a-model.md has tabs for both (#615). Lines 301 to 303 describe an api-key comment that clusterproviderconfig-civo.yaml no longer has. Could the skill leave out gaps the repo has since closed?

## Phase 4b: An apply-able example

Give people one folder they can apply to try the provider end to end. Build
`examples/<cloud>/` at the repo root, modelled on an existing cloud's kit:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want every new provider to build this kit? .gitignore excludes /examples/, so there's no existing kit to model on, and reviewers never see the scripts Phase 5 relies on for real-cloud validation. It also creates a second copy of docs/manifests/getting-started/<cloud>/ to keep in step. Could Phase 5 use the published manifests plus a few kubectl wait commands instead, or could this live outside the shared skill?

`reviewing-pull-requests` skill's standards to your own change.

Main moves while the PR is open. After every rebase, re-run the Phase 3
`git grep` parity sweep over the newest existing cloud: UpCloud had to

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the skill record one PR's history? This, the UKS boot disk size at lines 59 to 62 and the LB naming detail at lines 63 to 69 are true of one cloud at one point in time, and will go stale. Phase 0 items 10 and 11 run three or four times longer than their siblings because of them. Could they become general questions, such as "is the default boot disk big enough?", without the UpCloud specifics?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant