tests: capture a template's VM in the runtime adapter's guest

tests/runtime-adapter/README.md

@@ -36,9 +36,9 @@ Each configuration enables one side of `nvx`:
 
 | Configuration | Cargo options | `nvx` features | Modules | Binary |
 | --- | --- | --- | --- | --- |
-| Protocol-only | none | none | `frame`, `root`, `probe`, `exec` | none |
-| Host-only | `--features host` | `host` | `frame`, `root`, `probe`, `exec`, `host`, `runtime`, `initramfs`, `staged`, `sharing`, `layered`, `hook` | `runtime-adapter-host` |
-| Guest-only | `--features guest --target x86_64-unknown-linux-musl` | `guest` | `frame`, `root`, `probe`, `exec`, `guest`, `worker` | `runtime-adapter-guest`, a static executable |
+| Protocol-only | none | none | `frame`, `root`, `probe`, `exec`, `template` | none |
+| Host-only | `--features host` | `host` | `frame`, `root`, `probe`, `exec`, `template`, `host`, `runtime`, `initramfs`, `staged`, `sharing`, `layered`, `hook` | `runtime-adapter-host` |
+| Guest-only | `--features guest --target x86_64-unknown-linux-musl` | `guest` | `frame`, `root`, `probe`, `exec`, `template`, `guest`, `worker` | `runtime-adapter-guest`, a static executable |
 
 ## Framing
 
@@ -664,6 +664,90 @@ or with 1 after a diagnostic on standard error; and
 from inside the root, and which reports what the execution sees and may do;
 see [Probes](#probes).
 
+### Template mode
+
+A template's VM captures a snapshot of its guest before the guest assembles any
+root, so that every clone that a restore of the snapshot starts attaches a root
+of its own. The host opens no application session with it, and waits for no
+`Ready`: a committed capture ends the VM. Its kernel command line has
+`nvx_sandbox=1`, so NVX's `/init` runs the time ABI's boot step, then hands the
+VM to the guest agent's PID 1 of a layered root, which starts the worker, as in
+every VM with a layered root. OpenVMM adds `nvx_snapshot_tier=platform` to the
+kernel command line of a VM that captures a platform snapshot.
+
+On x86_64, whose time ABI a capture needs, the worker reads the kernel command
+line before anything else, with `template::boot`, which splits it as the kernel
+does, with `nvx::protocol::cmdline::parameters`. A command line in which the
+kernel finds no parameter `nvx_snapshot_tier=platform` boots cold: the worker
+goes on as above, whatever else the command line holds. With it, the VM is a
+template's, whose snapshot holds the platform's invariant configuration alone,
+which every clone shares. So the worker accepts only the parameters of the
+platform tier's allowlist, `template::is_platform_token`, which OpenVMM's
+capture of a platform snapshot allows too, byte for byte:
+
+- `earlycon=xe9`, `console=hvc0`, `console=hvc1`, `reboot=t`, `panic=-1`,
+  `nvx_sandbox=1`, `nvx_config=0xd0010000,65536`,
+  `nvx_snapshot_tier=platform`, and `nvx_control_tty=hvc2`;
+- `nr_cpus=`, `maxcpus=`, `virtio_mmio.device=`, `virtnet_ip=`,
+  `virtnet_mask=`, `virtnet_gw=`, `virtnet_dns=`, `virtnet_ip6=`, and
+  `virtnet_gw6=`, with any value that the host chooses.
+
+It refuses any other parameter, and a double quote, which the kernel drops from
+a parameter, or `--`, after which the kernel hands the rest of the command line
+to init, though init may still read it from `/proc/cmdline`: OpenVMM refuses
+both in the command line of a VM with a control console. Otherwise, the worker,
+through `worker::capture::capture_on_terminal`:
+
+1. opens the control tty, in raw mode, as the worker of a VM that boots cold
+   does;
+2. attaches the control session with `nvx::guest::nvxs::GuestConnection::attach`,
+   and waits until the tty has sent the attachment, so that the snapshot holds
+   the guest attached. It reads nothing from the session before the capture,
+   and serves no application session;
+3. prepares the guest for the capture with
+   `nvx::guest::clone::CapturePreparation`, which closes the worker's fence of
+   workload admission, checks that nothing is mounted from the `distro` or the
+   scratch block device, at the MMIO windows from which the worker assembles a
+   root, from an overlay, or from OpenVMM's host filesystem device, whose tag
+   is `microvm`, then waits for the time ABI's boot checks, runs its capture
+   checks, and suppresses the stall detectors;
+4. requests the capture with `CaptureBarrier::capture`, with the platform
+   tier's request, which keeps no scratch.
+
+Closing the worker's fence, `worker::capture::Admission`, acquires it: a close
+fails while the fence is closed, whoever closed it, and leaves it closed. The
+worker of a template's VM admits no execution before the capture, nor, in a
+clone, before repair acknowledged the restore. The worker returns from the
+capture only once the capture ended without a snapshot, or failed, or in a
+clone, which a restore of the snapshot started. It does not repair a clone yet:
+the clone ends with its fence still closed and its stall detectors still
+suppressed. The worker, and with it the VM, ends with one of these statuses,
+after a diagnostic on standard error:
+
+| Status | When |
+| --- | --- |
+| 1 | The control tty cannot be opened or put in raw mode, or the attachment cannot be sent |
+| 20 | The kernel command line names the platform tier, but holds another parameter, a double quote, or `--`, or the worker cannot read it |
+| 21 | The fence of workload admission cannot be closed |
+| 22 | Something is mounted from the template's devices or from an overlay, or the mount table cannot be checked |
+| 23 | The time ABI's boot checks did not complete |
+| 24 | The time ABI's capture checks failed |
+| 25 | The stall detectors cannot be suppressed |
+| 26 | The VM generation ID cannot be read |
+| 27 | The capture request failed, which leaves the fence closed |
+| 28 | The stall detectors' settings cannot be restored once the capture ended without a snapshot |
+| 29 | A stage of the capture failed that this build of the worker does not know |
+| 30 | The capture ended without a snapshot, as without a destination, or once OpenVMM rejected it, and the template continued |
+| 31 | A restore of the template's snapshot started this VM, a clone, which the worker does not repair yet |
+
+The statuses from 21 to 29 are `template::CAPTURE_STATUS_BASE` plus the number
+of the stage that failed, in the order in which
+`nvx::guest::clone::CaptureStage` declares the stages. A stage whose failure
+may leave the worker some of the permission for the time ABI's ports fails with
+the time ABI's port error, and the worker then starts no thread or process
+before it ends. A violation of the time ABI in enforcing mode stops the VM with
+the time ABI's own status, from 193 to 195, instead.
+
 ## Host program
 
 `host::Client` is the host program's client of the guest agent. It runs over
@@ -1214,15 +1298,24 @@ independent adapter's does:
   guest binary is a static executable: it has no program interpreter, needs no
   shared library, and keeps its stack non-executable.
 
-The unit tests of `frame`, `root`, `probe`, and `exec` run in every
+The unit tests of `frame`, `root`, `probe`, `exec`, and `template` run in every
 configuration. They cover each kind's encoding, messages in pieces of any size,
 a message of each kind split at every byte, the headers that the framing refuses
 before their body arrives, and the root configurations, readinesses, parts of
 mapping tables, failures, probes, facts, and messages of executions that decode,
 and those that don't, and the host's side of an execution, which still sends the
 cancellation of one that it gives up on in the step that ends it, without grace
 or settlement time, and which cancels one whose stream outgrows the 16 MiB that
-it keeps. In the guest-only configuration, the unit tests of `worker::probes`
+it keeps. Those of `template` check the platform tier's allowlist, token by
+token, against the tokens that OpenVMM's capture of a platform snapshot allows
+and refuses, and template mode: a command line in which the kernel finds no
+platform tier boots cold, whatever else it holds, and one with it boots a
+template's VM if it holds the tier's tokens alone, between any of the kernel's
+whitespace, and is refused otherwise, for a parameter outside the allowlist at
+its start, in its middle, or at its end, or within a platform token's value,
+after a vertical tab or the byte 0xa0, which the kernel takes for whitespace
+and a split at ASCII whitespace does not, for a double quote, and for `--`. In
+the guest-only configuration, the unit tests of `worker::probes`
 cover what the probes read: the mount table with its escapes, the kernel's
 records of a process, `os-release`, and the guest's memory, and the probes and
 arguments that the worker refuses. Those of
@@ -1237,7 +1330,12 @@ with a real-time signal, and check that PID 1 takes both stops and goes on
 until it collects the worker's status.
 Those of `worker::entries` cover the names of capabilities, the fields of a
 process's status, the outcomes of attempts, the entry's refusals, and the
-probe's arguments.
+probe's arguments. Those of `worker::capture` check that closing the worker's
+fence acquires it, the status of each stage of a capture, which no other
+outcome of the worker shares, a template's devices, and that the worker
+attaches its session, and the stream sent the attachment, before the capture
+runs, without reading anything of the session, and that no capture runs once
+the attachment cannot be written or sent.
 
 `tests/guest_session.rs` checks, in the guest-only configuration, that the
 guest agent serves the control session with the adapter's own framing. A test

tests/runtime-adapter/src/bin/runtime-adapter-guest.rs

@@ -20,7 +20,11 @@
 //!   SIBLING is the process ID of PID 1's unrelated child, which its probes check. It exits with 0
 //!   once the host stopped it, with 11 to 16 once a stage of the root failed, and with 1 once its
 //!   session failed or the broker closed the tty, each but the first after a diagnostic on
-//!   standard error.
+//!   standard error. In a template's VM, whose kernel command line names the platform snapshot
+//!   tier, the worker captures the VM before it assembles any root instead, on x86_64, as
+//!   `runtime_adapter::worker::capture` describes: it exits with 20 to 31, or with 1 once its
+//!   session failed, each after a diagnostic on standard error, unless a committed capture ended
+//!   the VM; see `runtime_adapter::template`.
 //! - `runtime-adapter-guest idle` sleeps until a signal ends it.
 //! - `runtime-adapter-guest entries ROOT CGROUP` runs the executions of the worker's `entries`
 //!   probe in the root ROOT, each placed in the cgroup CGROUP, in a mount namespace of its own, and
@@ -135,12 +139,17 @@ fn init(tty: &std::path::Path) -> ExitCode {
 }
 
 /// Runs the worker on the tty `tty`, with the process ID of PID 1's unrelated child, if any, and
-/// ends with its status.
+/// ends with its status. In a template's VM, the worker captures the VM instead; see
+/// `capture_template`.
 #[cfg(target_os = "linux")]
 fn worker(tty: &std::path::Path, sibling: Option<u32>) -> ExitCode {
     use runtime_adapter::worker::{self, LibraryRoot, Worked};
 
     let mut operations = LibraryRoot::new(sibling);
+    #[cfg(target_arch = "x86_64")]
+    if let Some(status) = capture_template(tty, &mut operations) {
+        return ExitCode::from(status);
+    }
     let worked = worker::work_on_terminal(tty, &mut operations);
     let status = match worked {
         Ok(worked) => {
@@ -164,6 +173,45 @@ fn worker(tty: &std::path::Path, sibling: Option<u32>) -> ExitCode {
     ExitCode::from(status)
 }
 
+/// Reads the kernel command line through `operations`, before anything else, and captures a
+/// template's VM on the tty `tty`, as `runtime_adapter::worker::capture` describes. Returns the
+/// status with which the worker ends then, after a diagnostic on standard error, or `None` in a
+/// VM that boots cold, whose worker serves the host.
+///
+/// A command line that the worker cannot read, or that `runtime_adapter::template::boot` refuses,
+/// ends the worker with `COMMAND_LINE_STATUS`.
+#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
+fn capture_template(
+    tty: &std::path::Path,
+    operations: &mut runtime_adapter::worker::LibraryRoot,
+) -> Option<u8> {
+    use runtime_adapter::template::{self, Boot, COMMAND_LINE_STATUS};
+    use runtime_adapter::worker::capture;
+    use runtime_adapter::worker::{RootOperations, SESSION_STATUS};
+
+    let boot = operations
+        .command_line()
+        .and_then(|command_line| template::boot(&command_line).map_err(|error| error.to_string()));
+    match boot {
+        Ok(Boot::Cold) => return None,
+        Ok(Boot::Template) => {}
+        Err(problem) => {
+            eprintln!("runtime-adapter-guest: {problem}");
+            return Some(COMMAND_LINE_STATUS);
+        }
+    }
+    Some(match capture::capture_on_terminal(tty) {
+        Ok(templated) => {
+            eprintln!("runtime-adapter-guest: {templated}");
+            templated.status()
+        }
+        Err(error) => {
+            eprintln!("runtime-adapter-guest: {error}");
+            SESSION_STATUS
+        }
+    })
+}
+
 /// Runs the executions of the worker's `entries` probe in the root `root`, each placed in the cgroup
 /// `workload`, and reports their facts.
 #[cfg(target_os = "linux")]

tests/runtime-adapter/src/lib.rs

@@ -5,8 +5,9 @@
 //! Each configuration enables one side of `nvx`:
 //!
 //! - protocol-only, without features: the control protocol, the contract versions, and the other
-//!   items that `nvx` has without features, with the adapter's own [`frame`]s, and the bodies of
-//!   its messages about a layered [`root`], its [`probe`]s, and [`exec`]utions;
+//!   items that `nvx` has without features, with the adapter's own [`frame`]s, the bodies of its
+//!   messages about a layered [`root`], its [`probe`]s, and [`exec`]utions, and the kernel command
+//!   line and statuses of its [`template`] mode;
 //! - host-only, with the `host` feature: the host API too, and the host program, in the `host`,
 //!   `runtime`, `initramfs`, `staged`, `sharing`, `layered`, and `hook` modules and the
 //!   `runtime-adapter-host` binary;
@@ -43,6 +44,10 @@
 //! and its guest binding, from one loop that never blocks. The executions enter the root with the
 //! publication of listeners that the root configuration names, and the host gives them the URL of
 //! the network's proxy, if any, in their environment.
+//!
+//! A VM whose kernel command line names the platform snapshot tier is a template's, which the
+//! worker captures before it assembles any root, so that every clone of the snapshot attaches a
+//! root of its own, in [`template`] mode.
 
 pub mod exec;
 pub mod frame;
@@ -64,6 +69,7 @@ pub mod runtime;
 pub mod sharing;
 #[cfg(feature = "host")]
 pub mod staged;
+pub mod template;
 #[cfg(all(feature = "guest", target_os = "linux"))]
 pub mod worker;