Repository navigation
Conversation
|
Docs preview: https://modelplane-docs-pr-673.vercel.app (ready once the site's Content workflow finishes) |
The upstream cluster-autoscaler's AWS provider doesn't model the DRA ResourceSlices a node would publish, so an EKS node group at zero never wakes for a pod whose GPU or CPU claim binds through DRA. The AKS and CoreWeave XRDs already warn about the same limit on minNodeCount; EKSCluster didn't, and InferenceCluster's minNodeCount had no description at all. The EKSCluster reference example and the Llama 3.1 8B EKS recipe went further and showed a GPU pool that scales to zero. This adds the warning to EKSCluster and InferenceCluster minNodeCount, regenerates the models and schema lock, and keeps the EKS examples at one node. InferenceCluster lists AKS for GPU pools only, matching AKSCluster. It doesn't reject minNodeCount: 0, since the issue hasn't settled which providers that should cover. Towards modelplaneai#345. Signed-off-by: ChethanUK <chethanuk@outlook.com>
c4a5f79 to
17992ad
Compare
|
i wonder If we should try to switch to EKS AutoMode with Karpenter DRA: https://aws.amazon.com/de/about-aws/whats-new/2026/11/amazon-eks-auto-mode-karpenter-dra/ |
|
/review |
There was a problem hiding this comment.
I think this needs another look at the EKS defaults before it's ready to merge. EKSCluster still defaults minNodeCount to 0, so an InferenceCluster pool that leaves the field out ends up in the state the new warning describes. Separately, the InferenceCluster description raises a question about providers: which clouds it lists, and whether it should list them at all.
Reviewed by modelplane-review 22e194c. Reply, or react 👍 or 👎, on each comment to help tune it.
| description: >- | ||
| Minimum number of nodes for autoscaling. Set to 1 or | ||
| higher for groups that must always be available. | ||
| Minimum number of nodes for autoscaling. Set it to 1 |
There was a problem hiding this comment.
Should we change the default here rather than only warn about it? minNodeCount still defaults to 0 and maxNodeCount to 8, and the InferenceCluster function passes pool.minNodeCount straight through (compose-inference-cluster/function/fn.py:1464) to scalingConfig.minSize. So a pool with only nodeCount: 1 gets a 0 to 8 node group, and once it idles to zero it can't come back.
Could the default follow nodeCount, as CoreWeave's does? The PR leaves runtime rejection until #345 settles, but a default isn't a rejection.
| minNodeCount: | ||
| type: integer | ||
| description: >- | ||
| Minimum node count for autoscaling. Keep GPU pools on |
There was a problem hiding this comment.
Are AKS CPU pools safe at zero? It looks like they bind through DRA just as EKS ones do: compose-aks-cluster labels them for dra-driver-cpu (fn.py:106-111), and engine pods claim through the dra.cpu DeviceClass. The AKS function's reason for the GPU limit is that no live node publishes ResourceSlices, which would hold for CPU slices too. GKE only models slices for nodes carrying the gke-nvidia-gpu-dra-driver label, so GKE CPU pools may be in the same position. If so, could the CPU clause list them too?
| type: integer | ||
| description: >- | ||
| Minimum node count for autoscaling. Keep GPU pools on | ||
| EKS, AKS and CoreWeave, and CPU pools on EKS and |
There was a problem hiding this comment.
Should the provider-neutral API name clouds here? This list repeats facts the EKS, AKS and CoreWeave XRDs already carry, and goes stale whenever a provider's autoscaler gains DRA support. What if this said that some providers' autoscalers can't scale a DRA pool up from zero, and pointed to each provider's minNodeCount? Users would then look one level down to learn whether the limit hits their cloud. If the list stays, the question above about AKS and GKE CPU pools applies.
Description of your changes
Towards #345.
EKS GPU and CPU pools can't scale up from zero nodes. The upstream cluster-autoscaler's AWS provider doesn't model the DRA ResourceSlices a new node would publish, so a pod whose claim binds through DRA waits forever on an empty group. The AKS and CoreWeave XRDs already say so on
minNodeCount. EKSCluster didn't, InferenceCluster'sminNodeCounthad no description, and both the EKSCluster reference example and the Llama 3.1 8B EKS recipe showed a GPU pool scaling to zero.This adds the warning to EKSCluster and InferenceCluster
minNodeCountand keeps the EKS examples at one node. It doesn't reject or warn onminNodeCount: 0at runtime, since the issue hasn't settled which providers that should cover.I regenerated the models and schema lock with the Crossplane CLI build that
nix/deps.nixpins (same hash), since nix isn't available here.I have:
RunRan the docs manifest validator and the compose-inference-cluster and compose-eks-cluster tests; nix isn't available here.nix flake check(or./nix.sh flake check) and made sure it passes.Added or updated tests covering any composition function changes.No composition function changes.git commit -s.