feat: support exec with shell

README.md

@@ -92,9 +92,10 @@ the standard `nvidia.com/gpu` resource limit, so scheduling and provisioning rea
 the same number. Do not set `nodeName` or a provider `nodeSelector` yourself — the
 placement controller owns those.
 
-> `kubectl logs` works, `-f` and `--tail` included: the manager serves the one kubelet
-> route the API server proxies for logs. `--timestamps`/`--previous`/`--since` are
-> ignored, and `kubectl exec` still does not work — use the provider's shell for that.
+> `kubectl logs` and `kubectl exec` both work on Modal, `-f`/`--tail` and `-it`
+> included: the manager serves the two kubelet routes the API server proxies.
+> `--timestamps`/`--previous`/`--since` and `-c` are ignored, and a terminal resize is
+> not forwarded. On providers that do not support them yet, both answer NotFound.
 
 ## Getting started
 
@@ -104,6 +105,8 @@ placement controller owns those.
 - See [docs/architecture.md](docs/architecture.md) for design details.
 - See [docs/status.md](docs/status.md) for how instance lifecycle becomes Pod and
   NodeClaim status, per provider.
+- See [docs/kubelet-api.md](docs/kubelet-api.md) for how `kubectl logs` and `kubectl exec`
+  reach a Pod with no kubelet.
 - See [docs/metrics.md](docs/metrics.md) for what is instrumented and how to query it.
 
 ## License

docs/add-a-provider.md

@@ -87,7 +87,7 @@ carry weight:
   `kubectl logs -f` client disconnects, and on a long-polling API that is one open
   request per abandoned client if it does not land.
 
-See [status.md § Logs](status.md#logs) for what the kubelet side does with the stream,
+See [kubelet-api.md](kubelet-api.md) for what the kubelet side does with the stream,
 including which kubectl flags are ignored and why.
 
 ## 2. Add the price/availability catalog

docs/architecture.md

@@ -339,7 +339,7 @@ serves one HTTPS listener in the manager pod and every virtual node advertises i
 `manager.Runnable`, hence leader-scoped — correct, because only the leader holds the
 tracked Pods a log request resolves against. A provider opts in by implementing
 `provider.LogStreamer`; one that does not answers `NotFound`. See
-[status.md § Logs](status.md#logs) for the transport, the trust model, and which kubectl
+[kubelet-api.md](kubelet-api.md) for the transport, the trust model, and which kubectl
 flags are honoured.
 
 Exec, attach, stats, and port-forward are not implemented — those need an agent inside

docs/deploy.md

@@ -133,7 +133,7 @@ Manager flags worth knowing (edit `config/manager/manager.yaml` `args`):
 The endpoint needs `POD_IP` (projected via `fieldRef` in `config/manager/manager.yaml`)
 because virtual nodes advertise the leader's Pod IP, not a Service. Running the manager
 off-cluster leaves it unset, and logs degrade to unsupported. See
-[status.md § Logs](status.md#logs).
+[kubelet-api.md](kubelet-api.md).
 
 ---
 

docs/kubelet-api.md

@@ -0,0 +1,95 @@
+# Logs and exec
+
+`kubectl logs` (`-f` included) and `kubectl exec` (`-it` included) both work against a
+Nebula Pod. It is worth spelling out how, because none of the usual kubelet machinery is
+present.
+
+- [The transport](#the-transport)
+- [The provider seam](#the-provider-seam)
+- [What logs honour, and the one heuristic](#what-logs-honour-and-the-one-heuristic)
+- [Containers are not addressable](#containers-are-not-addressable)
+- [Exec needs no agent in the image](#exec-needs-no-agent-in-the-image)
+
+---
+
+## The transport
+
+Neither is a control-plane read: the API server proxies both to the kubelet of the node
+the Pod is on, dialing the address in the Node's `status.addresses` and the port in
+`status.daemonEndpoints`. A virtual node has no kubelet, so the manager serves those
+routes itself (`pkg/vnode/kubelet.go`) and every virtual node advertises the manager's
+Pod IP and that port. Consequences worth knowing:
+
+- The endpoint is **leader-scoped and dialed by Pod IP**, not through a Service. The
+  tracked Pods live in one process's memory, so a Service balancing across replicas
+  would send requests to a replica that answers `NotFound`.
+- It serves TLS with a self-signed, in-memory certificate — what the API server
+  expects of a kubelet, which does not verify it unless
+  `--kubelet-certificate-authority` is set. Client certificates are **not** verified
+  by default, because which CA signs the API server's kubelet client cert is not
+  portable across distributions. Anything that can reach the port can therefore read the
+  logs of, and **run commands in**, any Pod on these virtual nodes, with no RBAC check:
+  keep it closed with a NetworkPolicy, or pass `--kubelet-client-ca` to require mTLS.
+- No POD_IP (running the manager off-cluster) means no endpoint. Logs and exec degrade
+  to unsupported; nothing else is affected.
+
+## The provider seam
+
+Both are optional: a provider opts in by implementing `provider.LogStreamer` and
+`provider.Executor`, and one that does not answers `NotFound` rather than carrying a
+stub. Modal implements both; AWS implements neither. Each seam is deliberately minimal —
+logs are one stream from the instance's first byte, stdout and stderr merged; exec only
+STARTS the command and hands back its streams — so every kubectl option is honoured
+once, for all providers, in `pkg/vnode/logs.go` and `pkg/vnode/exec.go`.
+
+## What logs honour, and the one heuristic
+
+A provider stream has no EOF while the instance lives and no marker for "you have now
+caught up", which the real kubelet gets for free from a file on disk. So:
+
+| flag | behaviour |
+|---|---|
+| (none) | ends at the first silent gap (1s) or a 30s ceiling, whichever comes first |
+| `--follow` | runs until the instance exits, the client disconnects, or the manager shuts down; a silent gap does NOT end it |
+| `--tail=N` | the backlog is buffered into a ring of the last N lines and only those are emitted, then `--follow` continues from there |
+| `--limit-bytes` | hard cap on bytes handed to the client, applied last |
+| `--timestamps`, `--previous`, `--since`, `--since-time` | accepted and **ignored** |
+
+The idle gap is the heuristic, and it is unavoidable: the alternative for a one-shot
+read is hanging until the workload exits, which for a long-running server is forever.
+The ceiling covers the opposite case, a workload chatty enough that the stream never
+goes idle — it truncates rather than failing, and `--follow` has no ceiling.
+
+`--tail` costs what buffering costs: the full backlog still crosses the provider's API,
+because there is no seek. Only the delivery is trimmed, bounded by N lines.
+
+The ignored flags cannot be served from this seam rather than merely being unfinished.
+`--timestamps` would have to invent a receive-time stamp, attributing the backlog's
+whole history to the moment it was fetched. `--previous` and `--since` need a
+per-container restart history and time-indexed storage that no provider exposes. They
+are ignored rather than rejected so a habitual `--since=1h` prints the full stream
+instead of an error.
+
+## Containers are not addressable
+
+`-c` is accepted and ignored, for logs and exec alike: a Nebula Pod maps to exactly one
+external instance with one console, so there is no per-container stream to select or
+shell to enter. Honouring the name would mean rejecting `kubectl logs pod` (which sends
+no container) or lying about a second container's output.
+
+## Exec needs no agent in the image
+
+The provider's own worker starts the command, so `kubectl exec -it pod -- bash` works
+against an unmodified user image — including the `sleep infinity` placeholder a Sandbox
+runs. What the exec does need is a **running** instance: a sandbox still queued for
+capacity has no container to run in, and the attempt fails rather than waiting.
+
+Nebula only pumps bytes: stdin is forwarded and closed at EOF (so `exec -i -- cat < f`
+ends), stdout and stderr are copied back, and a non-zero exit reaches kubectl as
+`command terminated with exit code N` rather than a server error. Two gaps, both from the
+provider side:
+
+| behaviour | why |
+|---|---|
+| terminal **resize** is ignored | no provider exposes a window-size call, so `-it` opens at the remote default and stays there |
+| under `-t`, **stderr is merged into stdout** | a PTY is one stream; kubectl forbids asking for both anyway |

docs/status.md

@@ -26,7 +26,7 @@ enters the system.
   - [AWS](#aws)
   - [Modal](#modal)
   - [fake](#fake)
-- [Logs](#logs)
+- [Logs and exec](#logs-and-exec)
 - [What is not observable](#what-is-not-observable)
 
 ---
@@ -212,7 +212,8 @@ only two signals and has to record a third fact itself.
   one stream), so the sandbox console reaches kubectl over the kubelet log route.
   Two Modal-specific limits: the stream always replays from the sandbox's FIRST byte
   (there is no seek, which is why `--tail` is served by buffering — see
-  [Logs](#logs)), and v2 sandboxes report stdio as unsupported, which surfaces as an
+  [kubelet-api.md](kubelet-api.md#what-logs-honour-and-the-one-heuristic)), and v2
+  sandboxes report stdio as unsupported, which surfaces as an
   error rather than an empty stream.
 - **Exit codes are lossy.** The control plane's result carries eight statuses
   (`SUCCESS`, `FAILURE`, `INIT_FAILURE`, `INTERNAL_FAILURE`, `TERMINATED`,
@@ -239,71 +240,12 @@ paths without a real backend, so it has no boot or readiness phase to model.
 
 ---
 
-## Logs
-
-`kubectl logs` works against a Nebula Pod, `-f` included. It is worth spelling out
-how, because none of the usual kubelet machinery is present.
-
-**The transport.** `kubectl logs` is not a control-plane read: the API server proxies
-it to the kubelet of the node the Pod is on, dialing the address in the Node's
-`status.addresses` and the port in `status.daemonEndpoints`. A virtual node has no
-kubelet, so the manager serves that one route itself
-(`pkg/vnode/kubelet.go`) and every virtual node advertises the manager's Pod IP and
-that port. Consequences worth knowing:
-
-- The endpoint is **leader-scoped and dialed by Pod IP**, not through a Service. The
-  tracked Pods live in one process's memory, so a Service balancing across replicas
-  would send requests to a replica that answers `NotFound`.
-- It serves TLS with a self-signed, in-memory certificate — what the API server
-  expects of a kubelet, which does not verify it unless
-  `--kubelet-certificate-authority` is set. Client certificates are **not** verified
-  by default, because which CA signs the API server's kubelet client cert is not
-  portable across distributions. Anything that can reach the port can therefore read
-  the logs of any Pod on these virtual nodes: keep it closed with a NetworkPolicy, or
-  pass `--kubelet-client-ca` to require mTLS.
-- No POD_IP (running the manager off-cluster) means no endpoint. Logs degrade to
-  unsupported; nothing else is affected.
-
-**The provider seam.** Log support is optional: a provider opts in by implementing
-`provider.LogStreamer`, and one that does not answers `NotFound` rather than carrying
-a stub. The seam is deliberately option-free — one stream, from the instance's first
-byte, following until the instance exits, stdout and stderr merged — so every kubectl
-option is honoured once, for all providers, in `pkg/vnode/logs.go`.
-
-**What is honoured, and the one heuristic.** A provider stream has no EOF while the
-instance lives and no marker for "you have now caught up", which the real kubelet gets
-for free from a file on disk. So:
-
-| flag | behaviour |
-|---|---|
-| (none) | ends at the first silent gap (1s) or a 30s ceiling, whichever comes first |
-| `--follow` | runs until the instance exits, the client disconnects, or the manager shuts down; a silent gap does NOT end it |
-| `--tail=N` | the backlog is buffered into a ring of the last N lines and only those are emitted, then `--follow` continues from there |
-| `--limit-bytes` | hard cap on bytes handed to the client, applied last |
-| `--timestamps`, `--previous`, `--since`, `--since-time` | accepted and **ignored** |
-
-The idle gap is the heuristic, and it is unavoidable: the alternative for a one-shot
-read is hanging until the workload exits, which for a long-running server is forever.
-The ceiling covers the opposite case, a workload chatty enough that the stream never
-goes idle — it truncates rather than failing, and `--follow` has no ceiling.
-
-`--tail` costs what buffering costs: the full backlog still crosses the provider's API,
-because there is no seek. Only the delivery is trimmed, bounded by N lines.
-
-The ignored flags cannot be served from this seam rather than merely being unfinished.
-`--timestamps` would have to invent a receive-time stamp, attributing the backlog's
-whole history to the moment it was fetched. `--previous` and `--since` need a
-per-container restart history and time-indexed storage that no provider exposes. They
-are ignored rather than rejected so a habitual `--since=1h` prints the full stream
-instead of an error.
-
-**Containers are not addressable.** `-c` is accepted and ignored: a Nebula Pod maps to
-exactly one external instance with one console, so there is no per-container stream to
-select. Honouring the name would mean rejecting `kubectl logs pod` (which sends no
-container) or lying about a second container's output.
-
-**`kubectl exec` still does not work.** It needs an agent inside the container, which
-is not part of the project today; the route answers `NotImplemented`.
+## Logs and exec
+
+Both are read paths onto a live instance rather than status, and they share one
+transport, so they live in [docs/kubelet-api.md](kubelet-api.md): how the manager serves
+the kubelet routes, the trust model on that port, and which `kubectl logs` and `kubectl
+exec` options are honoured.
 
 ## What is not observable
 

internal/controller/sandbox_controller.go

@@ -204,10 +204,10 @@ func (r *SandboxReconciler) buildPod(sbx *nebulav1alpha1.Sandbox) *corev1.Pod {
 				// and takes the instance down: the box would surface as Failed seconds after
 				// being provisioned.
 				//
-				// A PLACEHOLDER process, not a control surface. There is no agent in the
-				// container, so `kubectl exec`/`logs` against a sandbox do not work yet.
-				// Whatever restores them replaces this command; a bare `sleep` keeps any
-				// bootstrap from having to inject a binary into an arbitrary user image.
+				// A PLACEHOLDER process, not a control surface. `kubectl exec` and
+				// `kubectl logs` do not go through it: the provider starts the command
+				// itself, so a bare `sleep` is enough and no binary has to be injected
+				// into an arbitrary user image.
 				//
 				// No tension with a user-supplied command: SandboxSpec has no command field.
 				Command:   []string{"sleep", "infinity"},