Repository navigation
Document that EKS GPU and CPU pools can't scale up from zero #673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -528,6 +528,13 @@ spec: | |
| minimum: 0 | ||
| minNodeCount: | ||
| type: integer | ||
| description: >- | ||
| Minimum node count for autoscaling. Keep GPU pools on | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are AKS CPU pools safe at zero? It looks like they bind through DRA just as EKS ones do: |
||
| EKS, AKS and CoreWeave, and CPU pools on EKS and | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should the provider-neutral API name clouds here? This list repeats facts the EKS, AKS and CoreWeave XRDs already carry, and goes stale whenever a provider's autoscaler gains DRA support. What if this said that some providers' autoscalers can't scale a DRA pool up from zero, and pointed to each provider's |
||
| CoreWeave, at 1 or higher: their cluster autoscalers | ||
| can't scale a pool whose devices bind through DRA up | ||
| from zero nodes, so a pod scheduled to an empty pool | ||
| waits forever. | ||
| minimum: 0 | ||
| maxNodeCount: | ||
| type: integer | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,5 +19,5 @@ spec: | |
| nodeCount: 1 | ||
| zones: | ||
| - us-west-2a | ||
| minNodeCount: 0 | ||
| minNodeCount: 1 | ||
| maxNodeCount: 4 | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we change the default here rather than only warn about it?
minNodeCountstill defaults to 0 andmaxNodeCountto 8, and the InferenceCluster function passespool.minNodeCountstraight through (compose-inference-cluster/function/fn.py:1464) toscalingConfig.minSize. So a pool with onlynodeCount: 1gets a 0 to 8 node group, and once it idles to zero it can't come back.Could the default follow
nodeCount, as CoreWeave's does? The PR leaves runtime rejection until #345 settles, but a default isn't a rejection.