Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
version: "2"
run:
allow-parallel-runners: true
# The e2e suite is behind this tag, so without it lint would silently skip the
# whole suite.
build-tags:
- e2e
linters:
default: none
enable:
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ setup-test-e2e: ## Set up a Kind cluster for e2e tests if it does not exist

.PHONY: test-e2e
test-e2e: setup-test-e2e manifests generate fmt vet ## Run the e2e tests. Expected an isolated environment using Kind.
KIND_CLUSTER=$(KIND_CLUSTER) go test ./test/e2e/ -v -ginkgo.v
KIND_CLUSTER=$(KIND_CLUSTER) go test -tags e2e ./test/e2e/ -v -ginkgo.v
$(MAKE) cleanup-test-e2e

.PHONY: cleanup-test-e2e
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,9 @@ the standard `nvidia.com/gpu` resource limit, so scheduling and provisioning rea
the same number. Do not set `nodeName` or a provider `nodeSelector` yourself — the
placement controller owns those.

> `kubectl logs`/`exec` do not work against a virtual node (it doesn't serve the
> kubelet API); read workload output on the provider side.
> `kubectl logs` works, `-f` and `--tail` included: the manager serves the one kubelet
> route the API server proxies for logs. `--timestamps`/`--previous`/`--since` are
> ignored, and `kubectl exec` still does not work — use the provider's shell for that.

## Getting started

Expand Down
138 changes: 58 additions & 80 deletions api/v1alpha1/groupversion_info.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,91 +87,72 @@ const (
// boxes it owns — ownerReferences alone would not support a label-selector query.
SandboxSetLabel = "nebula.inftyai.com/sandboxset"

// AcceleratorTypeLabel carries the requested accelerator TYPE only (e.g.
// "a100-40gb" or "h100"). The COUNT is expressed separately as a standard
// resource request/limit on the container (nvidia.com/gpu for the NVIDIA
// accelerators the wired providers serve today) — so scheduling fit and
// provisioning read the same number, and there is no bespoke count grammar. It
// is a label (not an annotation) so Pods can be selected/validated by
// accelerator type; label values forbid ":", which is exactly why the count
// could never live here. The name is provider-neutral (accelerator, not GPU)
// so it also fits non-GPU accelerators (TPUs, etc.) when such a provider lands.
// The type is matched case-insensitively against the provider catalog, so
// "a100", "A100" both resolve; the provider's canonical casing is what actually
// gets provisioned (see catalog.Base.MapAccelerator). Read type+count together
// AcceleratorTypeLabel carries the accelerator TYPE only (e.g. "a100-40gb",
// "h100"). The COUNT is a standard container resource request/limit
// (nvidia.com/gpu today), so scheduling fit and provisioning read the same number
// and there is no bespoke count grammar. A label rather than an annotation so Pods
// can be selected by accelerator type — and label values forbid ":", which is why
// the count could never live here. The name says "accelerator", not GPU, so TPUs
// and friends fit when such a provider lands. Matched case-insensitively against
// the catalog ("a100" and "A100" both resolve); the provider's canonical casing is
// what gets provisioned (see catalog.Base.MapAccelerator). Read type+count together
// via util.AcceleratorRequest.
AcceleratorTypeLabel = "nebula.inftyai.com/accelerator-type"

// CapacityTypeAnnotation carries the optimizer-chosen purchase tier
// (Spot/OnDemand). It is the one provisioning input that cannot be
// read off the Pod's own spec, so the placement controller writes it here
// when it ungates the Pod. The virtual kubelet — which provisions solely from
// the Pod — reads it back on CreatePod. Empty means "let the provider use its
// default" (e.g. Modal is OnDemand-only and ignores it).
// CapacityTypeAnnotation carries the chosen purchase tier (Spot/OnDemand). It is a
// provisioning input the Pod spec cannot express, so the placement controller
// writes it when it ungates the Pod and the virtual kubelet — which provisions
// solely from the Pod — reads it back on CreatePod. Empty means "use the provider's
// default" (Modal is OnDemand-only and ignores it).
CapacityTypeAnnotation = "nebula.inftyai.com/capacity-type"

// RegionAnnotation carries the optimizer-chosen provider region. Like
// CapacityTypeAnnotation it is a provisioning input absent from the Pod's own
// spec, so the placement controller writes it when it ungates the Pod and the
// virtual kubelet reads it back on CreatePod into ProvisionRequest.Region.
// Empty/absent means "the provider's configured default region" — region-simple
// providers (Modal, RunPod) ignore it.
// RegionAnnotation carries the chosen provider region. Same flow as
// CapacityTypeAnnotation: stamped at ungate, read on CreatePod into
// ProvisionRequest.Region. Absent means the provider's default region;
// region-simple providers (Modal, RunPod) ignore it.
RegionAnnotation = "nebula.inftyai.com/region"

// BlocklistTTLAnnotation carries the pool's FailoverPolicy.BlocklistTTL down to
// the virtual kubelet. Like the two annotations above it is a provisioning-time
// input the Pod cannot otherwise express: the TTL is a NodePool policy, but the
// VK handler (which provisions per-Pod and never sees the pool) needs it to know
// how long to blocklist a placement that just failed. The placement controller
// stamps it when it ungates the Pod; the handler reads it on a Provision failure
// to bound the block it records. Absent/unparseable means the handler's built-in
// default TTL.
// BlocklistTTLAnnotation carries the pool's FailoverPolicy.BlocklistTTL down to the
// virtual kubelet. The TTL is NodePool policy, but the VK handler provisions per-Pod
// and never sees the pool, so it needs the value here to bound the block it records
// after a Provision failure. Stamped at ungate; absent/unparseable means the
// handler's built-in default.
BlocklistTTLAnnotation = "nebula.inftyai.com/blocklist-ttl"

// EndpointAnnotation carries the reachable address of the external instance (a
// public DNS name, an IP, or a URL, in the provider's own form).
// It is the ONLY way to reach the workload, so it must be visible on the Pod:
// PodIP cannot hold it because the API server validates PodIP as a literal IP
// and rejects a DNS name (the common AWS case), so the endpoint rides an
// annotation instead. Written by the virtual kubelet as soon as it knows the
// address, which is provider-dependent and NOT tied to the phase: a provider
// that mints a connect URL at create time (Modal) publishes it from CreatePod,
// before the instance is Running; one whose address only exists after boot (AWS)
// publishes it from the poll loop. Absent until then, and never cleared once
// written. Unlike the provisioning-input annotations above (which the placement
// controller stamps and VK reads), this flows the other way — VK writes it for
// operators/tooling to read.
// EndpointAnnotation carries the reachable address of the external instance (a DNS
// name, an IP, or a URL, in the provider's own form). It is the only way to reach
// the workload, and PodIP cannot hold it — the API server validates PodIP as a
// literal IP and rejects a DNS name, the common AWS case — so it rides an
// annotation. The virtual kubelet writes it as soon as it knows the address, which
// is NOT tied to the phase: a provider that mints a connect URL at create time
// (Modal) publishes from CreatePod, before Running; one whose address only exists
// after boot (AWS) publishes from the poll loop. Absent until then, never cleared.
// This one flows outward — VK writes, operators read.
EndpointAnnotation = "nebula.inftyai.com/endpoint"

// TerminateInstanceFinalizer is held by every NodeClaim to guarantee teardown.
// The virtual kubelet owns the happy path (DeletePod → provider.Terminate,
// keyed on the Pod-derived claim name), but its teardown is edge-triggered and
// its instance tracking is in-memory, so a Pod force-deleted during a VK outage
// would leak a paid instance. This finalizer makes teardown level-triggered:
// the cluster-scoped claim outlives the namespaced Pod, so on delete the
// NodeClaim controller resolves the provider, finds the instance by claim name
// via List, and Terminates it before releasing the finalizer — independent of
// VK liveness (see docs/architecture.md §3).
// TerminateInstanceFinalizer is held by every NodeClaim to guarantee teardown. VK
// owns the happy path (DeletePod → provider.Terminate), but its teardown is
// edge-triggered and its tracking in-memory, so a Pod force-deleted during a VK
// outage would leak a paid instance. This finalizer makes teardown
// level-triggered: the cluster-scoped claim outlives the namespaced Pod, so on
// delete the NodeClaim controller resolves the provider, finds the instance by
// claim name via List, and Terminates before releasing — independent of VK
// liveness (see docs/architecture.md §3).
TerminateInstanceFinalizer = "nebula.inftyai.com/terminate-instance"
)

// Pod status reasons the virtual kubelet stamps on the Pods it reports, projecting
// the external instance's lifecycle onto standard Pod status (pkg/vnode/status.go
// is the only writer).
// the external instance's lifecycle onto standard Pod status (pkg/vnode/status.go is
// the only writer).
//
// They live here, not privately in pkg/vnode, for two reasons. They are a CONTRACT
// between packages: the Pod phase is lossy — Provisioning and "booting" both
// surface as PodPending — so the reason is the only thing separating "no instance
// exists yet" from "an instance exists and is coming up", and the NodeClaim
// controller keys its teardown guard off exactly that distinction (see
// desiredPhase). A rename on the writing side that the reading side did not follow
// would still compile, still pass tests, and silently leak paid instances: every
// booting instance would read as Provisioning, so a Pod that vanished mid-boot
// would be left running behind the cache-lag grace window. And they are user-facing
// — operators match on status.reason in jsonpath and alerts — so every value is
// public API whether or not Nebula's own code currently reads it. That is why the
// whole set is here rather than the subset with in-tree readers: these are the
// values status.reason can take, and a reader should find them in one place.
// They are public rather than private to pkg/vnode because they are a CONTRACT
// between packages. The Pod phase is lossy — provisioning and booting both surface as
// PodPending — so the reason is the only thing separating "no instance yet" from "an
// instance exists and is coming up", and the NodeClaim controller keys its teardown
// guard off exactly that (see desiredPhase). A rename on the writing side that the
// reader missed would still compile, still pass tests, and silently leak paid
// instances. They are also user-facing (operators match status.reason in jsonpath and
// alerts), so the whole set lives here — not just the values with in-tree readers.
const (
// PodReasonProvisioning: capacity has not been allocated yet. Stamped by CreatePod
// before it calls Provision, and HELD if Provision returns an id without reserving
Expand All @@ -181,17 +162,14 @@ const (
// as capacity is committed: at once for a provider that allocates synchronously
// (AWS), otherwise when the first poll observes the instance.
PodReasonProvisioning = "Provisioning"
// PodReasonInitializing: the instance EXISTS at the provider but is not yet
// reachable — it is booting (EC2 "pending"), running-but-not-yet-passing its
// reachability checks (running, <2/2, EC2's own "Initializing" status), or a Modal
// sandbox whose readiness probe has not passed. It mirrors that EC2 status-check
// term. Provisioning is done; the instance is coming up. Distinct from
// Provisioning so a Pod stuck here points at a slow boot / failing status checks,
// not a stuck allocation — and so the NodeClaim controller can tell that an
// instance exists. The virtual kubelet stamps it only on EVIDENCE of existence:
// either the provider observed the instance in its List, or Provision reported it
// reserved (capacity committed, not merely requested). That is what makes it
// trustworthy for the claim to key Bound off.
// PodReasonInitializing: the instance EXISTS but is not yet reachable — booting
// (EC2 "pending"), running with reachability checks outstanding (<2/2, EC2's own
// "Initializing" term, which this mirrors), or a Modal sandbox whose probe has not
// passed. Distinct from Provisioning so a Pod stuck here points at a slow boot
// rather than a stuck allocation, and so the NodeClaim controller can tell an
// instance exists. VK stamps it only on EVIDENCE of existence: the provider
// observed the instance in List, or Provision reported it reserved (capacity
// committed, not merely requested). That is what makes it safe to key Bound off.
PodReasonInitializing = "Initializing"
// PodReasonRunning: the provider reports the instance running.
PodReasonRunning = "Running"
Expand Down
Loading
Loading