diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 18df974e7..15946d05f 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -103,10 +103,10 @@ on Linux and Windows. | `nvx::host::nvxs` | `HostConnection`: the host's role of the NVXS session over a platform transport, on which `nvx::host::session` and other host applications run their protocols. It attaches with the launch capability, waiting while the broker asks it to, sends application bytes in as many DATA records as they take, and returns each DATA payload from the guest, with deadlines; a receive that times out keeps a partial record | | `nvx::host::process` | Verified process references. A `ProcessRef` names one process by its ID, its start time, and the `BootId` of the host boot during which it started. `Process::open` checks a reference and holds its process through a pidfd on Linux or a process handle on Windows, whether or not the caller started it, so that waits sleep until the process exits and termination never reaches a process that reuses its ID. A reference from an earlier boot names an exited process, and a process whose identity this user cannot check, such as another user's, makes `Process::open` fail rather than report an exit. Linux hosts without pidfds poll the reference instead, and cannot terminate the process | | `nvx::host::supervisor` | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock, as a launch whose `LaunchId` the caller reserved beforehand, a fresh one for each launch, or that a clone's lease reserves: it writes the journal's marker, which names the launch, spawns OpenVMM with the claimed log, records its `ProcessRef` right after, and returns a `Starting` launch that the caller commits once the VMM serves its control endpoint, or abandons. A launch that fails never leaves an unrecorded VMM running. `VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that the VMM inherits under their numbers, besides its standard streams, and it inherits nothing else of the caller's: on Linux, they must be close-on-exec, a flag that only the VMM's own process clears; on Windows, they join the VMM's explicit handle list. `Vmm::stop` runs one graceful step of the caller's and waits for the VMM's exit event until a deadline, and takes a bounded forced step through the VMM's verified identity only on request. `Vmm::terminate` takes that forced step alone, within the caller's bound, for a VM whose guest the caller must not ask anything, such as one of another runtime identity. `reattach` recovers an interrupted launch, and holds the recorded VMM again after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate`. A clone's launch takes its locks in one order, the template store's, then the sandbox's, which guards its clone binding and launch journal, then the scratch image's claim, and holds none for the VM's lifetime; its lease lasts through crashes and reattaches, until a release ends it once a stop, a termination, or a reattach has verified the VM's exit | -| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | +| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; a network exactly when the template has a NIC; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and which nothing formats, copies, or reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the sandbox's directory, which the release removes; and a fresh capability. The image and the scratch device must present the `BlockGeometry` of the template's, which a restore compares, and be, at the paths by which OpenVMM opens them, the regular files that the checks found, which are not links or other reparse points, and which no one else may change. Neither the image nor a scratch image may be a file that the sandbox's store owns, such as `clone-scratch.raw`, its log, or its records, by any path, since the store's transitions replace or remove them, and a link or a directory under one of the store's names is refused. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | | `nvx::host::seal` | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it: the platform scheme, the device or volume serial number, the file ID, the kind, the length, the modification time, and on Linux the change time, with a versioned serde form. A seal detects accidental change, not tampering; see [Distro bindings](#distro-bindings) | | `nvx::host::session`, `platform`, `report` | The authenticated control session, which carries NVX's control protocol, `NVXC`, one frame per DATA record of a `HostConnection`; Linux and Windows primitives; and OpenVMM's outcome report | -| `nvx::host::sandbox` | The launch profile of `nvx-rs sandbox`, with native input validation: `SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and `SandboxSettings` resource limits (`Option` for memory and `Option` for processes), and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. The plan of an adapter that boots its own guest, `SandboxPlan::adapter`, holds no launch settings and no export plan. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` arguments, with `--microvm-workload-identity` when it has launch settings, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. An adapter's plan renders the caller's tokens before those of its root, and refuses tokens that its root or OpenVMM's shares own. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx-rs sandbox` and `nvx-rs run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx-rs run` shares, and the managed exec environment; see [Sandbox input contract](#sandbox-input-contract). `distro`: the `DistroBinding` of a distro layer, an EROFS layer or a GPT disk, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | +| `nvx::host::sandbox` | The launch profile of `nvx-rs sandbox`, with native input validation: `SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and `SandboxSettings` resource limits (`Option` for memory and `Option` for processes), and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. The plan of an adapter that boots its own guest, `SandboxPlan::adapter`, holds no launch settings and no export plan. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` arguments, with `--microvm-workload-identity` when it has launch settings, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. An adapter's plan renders the caller's tokens before those of its root, and refuses tokens that its root or OpenVMM's shares own. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx-rs sandbox` and `nvx-rs run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx-rs run` shares, and the managed exec environment; see [Sandbox input contract](#sandbox-input-contract). `distro`: the `DistroBinding` of a distro layer, an EROFS layer or a GPT disk, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `CheckedDistro::geometry` and `CheckedScratch::geometry` read, through the checked files, the `BlockGeometry` that OpenVMM presents for them: their length, a nonzero multiple of 512 bytes, logical blocks of 512 bytes, and physical blocks of the I/O block size of their filesystem on Linux, and of 4096 bytes on Windows. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | | `nvx::testing::fake_openvmm` | A fake OpenVMM that serves the control console and runs scripted workloads without a VM, for direct launches and for the managed and one-shot sandboxes of `nvx-rs sandbox`, and runs the VMs of templates' captures, which publish a snapshot, or fail, as a scenario asks | | `nvx::host::cli` | Feature `cli`: the `nvx-rs` executable; see [The nvx-rs CLI](#the-nvx-rs-cli) | | `nvx::guest` | The programs that run in Linux guests; see [Guest binaries](#guest-binaries) | @@ -340,6 +340,13 @@ own opens then keep the images, and what it inherited keeps the scratch image's claim. Staging, the checks, and the preparation of a launch read no content of the images. +The clone of a template takes its block devices by path instead, as the +template's capture does, since OpenVMM opens the files of a restore itself: +`ClonePlan::new` checks the clone's image and scratch image through their +bindings, then checks each at its path, and requires the geometry that the +template recorded for its own, which `CheckedDistro::geometry` and +`CheckedScratch::geometry` read through the checked files. + `nvx-rs sandbox provision` records the scratch image's binding under `scratch_binding` in `config.json`, at the image's resolved path, and `start` checks it after the distro binding, and launches OpenVMM through a prepared diff --git a/nvx-rs/src/host/sandbox/distro.rs b/nvx-rs/src/host/sandbox/distro.rs index d2d1f7442..0774bddc1 100644 --- a/nvx-rs/src/host/sandbox/distro.rs +++ b/nvx-rs/src/host/sandbox/distro.rs @@ -42,9 +42,10 @@ use serde::{Deserialize, Serialize}; use serde_json::Value; use sha2::{Digest, Sha256}; +use super::geometry::BlockGeometry; use super::layer::SandboxLayer; use super::mount::os_error; -use super::refused; +use super::{block_geometry, refused}; use crate::error::{Error, Result}; use crate::host::artifacts::DISTRO_MANIFEST_SUFFIX; use crate::host::network::policy::MAX_JSON_DEPTH; @@ -213,6 +214,29 @@ impl CheckedDistro { (self.format, &self.identity) } + /// Returns the block geometry that OpenVMM presents for the layer as the read-only `distro` + /// block device, read through the checked file, as OpenVMM reads it through its own open of + /// the layer: the layer's length, which must be a nonzero multiple of 512 bytes; logical + /// blocks of 512 bytes; and physical blocks of the I/O block size of the layer's filesystem, + /// its `st_blksize`, on Linux, of 4096 bytes on Windows, and of the logical block size on + /// other hosts, which must be a power of two of at least 512 bytes. + /// + /// It reads the file's metadata alone. A restore of a template compares the geometry of each + /// of its block devices with the one that the template recorded, so the image of a clone must + /// present the geometry of the template's `distro`, as + /// [`ClonePlan`](crate::host::template::ClonePlan) checks before the clone's launch. + /// + /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a length or a + /// physical block size that OpenVMM refuses, and with [`ErrorKind::Io`](crate::ErrorKind::Io) + /// if the metadata cannot be read. + pub fn geometry(&self) -> Result { + let description = match self.format { + DistroFormat::Erofs => "distro EROFS layer", + DistroFormat::Gpt => "distro GPT image", + }; + block_geometry(&self.file, &self.path, description) + } + /// Returns the checked layer, for a prepared launch to move. #[cfg(target_os = "linux")] pub(crate) fn file_mut(&mut self) -> &mut File { diff --git a/nvx-rs/src/host/sandbox/distro/tests.rs b/nvx-rs/src/host/sandbox/distro/tests.rs index e0a420fed..a57beeab2 100644 --- a/nvx-rs/src/host/sandbox/distro/tests.rs +++ b/nvx-rs/src/host/sandbox/distro/tests.rs @@ -271,6 +271,127 @@ fn the_staged_check_reads_no_content() { assert_eq!(verify_with(&layer, Access::READ).unwrap(), binding); } +/// Returns the physical block size that OpenVMM presents for the file at `path`: the I/O block +/// size of its filesystem on Linux, and 4096 bytes on Windows. +fn physical_block_size(path: &Path) -> u32 { + #[cfg(target_os = "linux")] + { + use std::os::unix::fs::MetadataExt; + + u32::try_from(fs::metadata(path).unwrap().blksize()).unwrap() + } + #[cfg(windows)] + { + let _ = path; + 4096 + } +} + +#[test] +fn checked_layers_present_the_geometry_that_openvmm_reads_through_them() { + let staged = Staged::new(false); + let image = staged.image(); + let resize = |length: u64| { + OpenOptions::new() + .write(true) + .open(&image) + .unwrap() + .set_len(length) + .unwrap(); + }; + let check = |layer: &SandboxLayer| { + import_with(layer, PolicyVerdict::ACCEPTED, METADATA_ONLY) + .unwrap() + .check_with(layer, DISTRO_POLICY_REVISION, METADATA_ONLY) + .unwrap() + }; + resize(8192); + let checked = check(&staged.layer()); + let geometry = BlockGeometry { + length: 8192, + logical_block_size: 512, + physical_block_size: physical_block_size(&image), + }; + // It reads the metadata of the checked file alone, which reads nothing. + assert_eq!(checked.geometry().unwrap(), geometry); + // The geometry is the checked file's, whatever later takes its path. + #[cfg(target_os = "linux")] + { + let replacement = staged.other("replacement.erofs"); + fs::write(&replacement, [0; 4096]).unwrap(); + fs::rename(&replacement, &image).unwrap(); + assert_eq!(checked.geometry().unwrap(), geometry); + } + drop(checked); + // OpenVMM attaches block devices of nonzero multiples of 512 bytes alone. + let gpt = SandboxLayer::gpt(LayerRole::Distro, image.to_str().unwrap(), UUID); + for (layer, description) in [ + (staged.layer(), "distro EROFS layer"), + (gpt, "distro GPT image"), + ] { + for length in [0, 8191] { + resize(length); + assert_eq!( + failure(check(&layer).geometry()), + ( + ErrorKind::Unsupported, + format!( + "the {description} {} of {length} bytes cannot be attached: OpenVMM \ + attaches block devices of nonzero multiples of 512 bytes", + shown(&image) + ) + ) + ); + } + } +} + +/// A layer on hugetlbfs, whose I/O block size is a huge page, presents physical blocks of 2 MiB. +#[cfg(target_os = "linux")] +#[test] +fn layers_present_the_block_size_of_their_filesystem() { + use nix::sys::memfd::{MFdFlags, memfd_create}; + + const HUGE_PAGE: u64 = 2 * 1024 * 1024; + let flags = MFdFlags::MFD_CLOEXEC | MFdFlags::MFD_HUGETLB | MFdFlags::MFD_HUGE_2MB; + let huge = match memfd_create("nvx-geometry-test", flags) { + Ok(descriptor) => File::from(descriptor), + Err(error) => { + eprintln!("this host offers no file on hugetlbfs: {error}"); + return; + } + }; + if let Err(error) = huge.set_len(HUGE_PAGE) { + eprintln!("this host cannot size a file on hugetlbfs: {error}"); + return; + } + let staged = Staged::new(false); + let image = staged.image(); + let open = |path: &Path| { + if path == image { + huge.try_clone() + } else { + Err(io::Error::from(io::ErrorKind::NotFound)) + } + }; + let access = Access { + open: &open, + seal: &FileSeal::of, + }; + let checked = import_with(&staged.layer(), PolicyVerdict::ACCEPTED, access) + .unwrap() + .check_with(&staged.layer(), DISTRO_POLICY_REVISION, access) + .unwrap(); + assert_eq!( + checked.geometry().unwrap(), + BlockGeometry { + length: HUGE_PAGE, + logical_block_size: 512, + physical_block_size: HUGE_PAGE as u32, + } + ); +} + /// Stages a layer, with a manifest if `manifest` is true, binds it by verification, applies /// `change`, and returns the failure of the check on a host where reads of content fail, with the /// staged layer. diff --git a/nvx-rs/src/host/sandbox/geometry.rs b/nvx-rs/src/host/sandbox/geometry.rs new file mode 100644 index 000000000..840c2597b --- /dev/null +++ b/nvx-rs/src/host/sandbox/geometry.rs @@ -0,0 +1,15 @@ +//! The block geometry of a sandbox block device, which the checks of its files read as OpenVMM +//! presents it, and which a template records for the devices of its capture. +//! [`template`](crate::host::template) re-exports what is public here. + +/// The length and block sizes of a sandbox block device. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct BlockGeometry { + /// The length that the device presents, in bytes: nonzero, and a multiple of the logical + /// block size. + pub length: u64, + /// The logical block size, in bytes: a power of two of at least 512. + pub logical_block_size: u32, + /// The physical block size, in bytes: a power of two of at least the logical block size. + pub physical_block_size: u32, +} diff --git a/nvx-rs/src/host/sandbox/mod.rs b/nvx-rs/src/host/sandbox/mod.rs index 5c98ac18e..facb54d53 100644 --- a/nvx-rs/src/host/sandbox/mod.rs +++ b/nvx-rs/src/host/sandbox/mod.rs @@ -31,6 +31,10 @@ //! layer and its manifest, and which a launch checks from metadata alone. //! - [`scratch`]: the binding of the scratch image, its identity, kind, and length, which a //! launch checks from metadata alone, and the claim of the image by one sandbox at a time. +//! - The block geometry that OpenVMM presents for each checked file, its length and its logical +//! and physical block sizes, which [`CheckedDistro::geometry`](distro::CheckedDistro::geometry) +//! and [`CheckedScratch::geometry`](scratch::CheckedScratch::geometry) read through the file, +//! as OpenVMM reads it. //! - [`PreparedLaunch`]: the launch of a plan with the files that the checks of its block devices //! returned, which hands the VMM those objects rather than paths that something could replace //! meanwhile: the OpenVMM arguments that name them, and the descriptors that the VMM inherits; @@ -49,6 +53,7 @@ //! the path quoted with Rust's debug formatting. pub mod distro; +pub(crate) mod geometry; pub mod layer; pub mod mount; pub mod options; @@ -56,10 +61,13 @@ pub mod plan; pub mod prepared; pub mod scratch; +use std::fs::File; use std::path::Path; use self::distro::{DistroBinding, DistroFormat}; +use self::geometry::BlockGeometry; pub use self::layer::{LAYER_ROLES, LayerRole, SandboxLayer}; +use self::mount::os_error; pub use self::mount::{MountAccess, MountKind, MountOwner, MountPolicy, PolicyKind, SandboxMount}; pub use self::plan::{ BlockRole, PLAN_VERSION, SandboxDistro, SandboxNetwork, SandboxPlan, SandboxUpper, @@ -189,6 +197,64 @@ pub(crate) fn reject_container_path(path: &Path, shown: &str) -> Result<()> { Ok(()) } +/// The logical block size that OpenVMM presents for a sandbox block device, in bytes, of which the +/// device's length is a nonzero multiple. +const LOGICAL_BLOCK_SIZE: u32 = 512; + +/// The physical block size that OpenVMM presents for a sandbox block device on Windows, in bytes. +#[cfg(windows)] +const WINDOWS_PHYSICAL_BLOCK_SIZE: u32 = 4096; + +/// Returns the block geometry that OpenVMM presents for a sandbox block device whose file `file` +/// holds open, which messages call `description` at `path`, read through `file` as OpenVMM reads +/// it through its own open of the file, as +/// [`CheckedDistro::geometry`](distro::CheckedDistro::geometry) describes. +/// +/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a length or a physical +/// block size that OpenVMM refuses, and with [`ErrorKind::Io`](crate::ErrorKind::Io) if the file's +/// metadata cannot be read. +pub(crate) fn block_geometry(file: &File, path: &Path, description: &str) -> Result { + let shown = path.display(); + let metadata = file + .metadata() + .map_err(|error| os_error(&error, Some(path)))?; + let length = metadata.len(); + if length == 0 || !length.is_multiple_of(u64::from(LOGICAL_BLOCK_SIZE)) { + return Err(refused(format!( + "the {description} {shown} of {length} bytes cannot be attached: OpenVMM attaches \ + block devices of nonzero multiples of {LOGICAL_BLOCK_SIZE} bytes" + ))); + } + #[cfg(target_os = "linux")] + let (reported, physical) = { + use std::os::unix::fs::MetadataExt; + + let size = metadata.blksize(); + (size, u32::try_from(size).ok()) + }; + #[cfg(windows)] + let (reported, physical) = ( + u64::from(WINDOWS_PHYSICAL_BLOCK_SIZE), + Some(WINDOWS_PHYSICAL_BLOCK_SIZE), + ); + #[cfg(not(any(target_os = "linux", windows)))] + let (reported, physical) = (u64::from(LOGICAL_BLOCK_SIZE), Some(LOGICAL_BLOCK_SIZE)); + match physical { + Some(physical) if physical >= LOGICAL_BLOCK_SIZE && physical.is_power_of_two() => { + Ok(BlockGeometry { + length, + logical_block_size: LOGICAL_BLOCK_SIZE, + physical_block_size: physical, + }) + } + _ => Err(refused(format!( + "the {description} {shown} cannot be attached: its filesystem's block size of \ + {reported} bytes is not a physical block size that OpenVMM presents, a power of two \ + from {LOGICAL_BLOCK_SIZE} bytes to 2 GiB" + ))), + } +} + /// The launch settings of a [`SandboxPlan`] or a [`SandboxLaunch`], besides its block devices and /// its shares, with the defaults of `nvx-rs sandbox`. #[derive(Clone, Debug, PartialEq, Eq)] diff --git a/nvx-rs/src/host/sandbox/scratch.rs b/nvx-rs/src/host/sandbox/scratch.rs index f568656a3..ccea75636 100644 --- a/nvx-rs/src/host/sandbox/scratch.rs +++ b/nvx-rs/src/host/sandbox/scratch.rs @@ -56,8 +56,9 @@ use std::time::{Duration, Instant}; use serde::{Deserialize, Serialize}; +use super::geometry::BlockGeometry; use super::mount::os_error; -use super::refused; +use super::{block_geometry, refused}; use crate::error::{Error, ErrorCode, Result}; use crate::host::paths; use crate::host::seal::{FileId, FileKind, FileSeal, SealScheme}; @@ -157,6 +158,25 @@ impl CheckedScratch { &self.path } + /// Returns the block geometry that OpenVMM presents for the image as the `scratch` block + /// device, read through the checked file, as + /// [`CheckedDistro::geometry`](super::distro::CheckedDistro::geometry) reads that of a distro + /// layer: the image's length, logical blocks of 512 bytes, and physical blocks of the I/O + /// block size of the image's filesystem on Linux, of 4096 bytes on Windows, and of the logical + /// block size on other hosts. On Windows, the checked file is the image's guard, which may + /// read the image's metadata. + /// + /// It reads the file's metadata alone, never the image's content. The scratch image of a + /// clone must present the geometry of its template's scratch, its length included, as + /// [`ClonePlan`](crate::host::template::ClonePlan) checks before the clone's launch. + /// + /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a length or a + /// physical block size that OpenVMM refuses, and with [`ErrorKind::Io`](crate::ErrorKind::Io) + /// if the metadata cannot be read. + pub fn geometry(&self) -> Result { + block_geometry(&self.file, &self.path, DESCRIPTION) + } + /// Returns the checked image, which the caller then holds with its claim: an owned descriptor /// on Linux, and on Windows an owned handle, the image's guard, which then holds the claim /// alone, since this closes the claim's handle. diff --git a/nvx-rs/src/host/sandbox/scratch/tests.rs b/nvx-rs/src/host/sandbox/scratch/tests.rs index 4078350f8..f7ce827d4 100644 --- a/nvx-rs/src/host/sandbox/scratch/tests.rs +++ b/nvx-rs/src/host/sandbox/scratch/tests.rs @@ -297,6 +297,68 @@ fn checks_claim_the_image_without_reading_it() { drop(staged.claim(&binding)); } +/// Returns the physical block size that OpenVMM presents for the file at `path`: the I/O block +/// size of its filesystem on Linux, and 4096 bytes on Windows. +fn physical_block_size(path: &Path) -> u32 { + #[cfg(target_os = "linux")] + { + use std::os::unix::fs::MetadataExt; + + u32::try_from(fs::metadata(path).unwrap().blksize()).unwrap() + } + #[cfg(windows)] + { + let _ = path; + 4096 + } +} + +#[test] +fn checked_images_present_the_geometry_that_openvmm_reads_through_them() { + let staged = Staged::new(); + let resize = |length: u64| { + OpenOptions::new() + .write(true) + .open(staged.image()) + .unwrap() + .set_len(length) + .unwrap(); + }; + resize(1 << 20); + let binding = staged.bind(); + let checked = staged.claim(&binding); + // It reads the metadata of what the check holds, which reads nothing, while the claim lasts. + assert_eq!( + checked.geometry().unwrap(), + BlockGeometry { + length: 1 << 20, + logical_block_size: 512, + physical_block_size: physical_block_size(&staged.image()), + } + ); + assert_eq!( + failure(staged.check(&binding, Instant::now())), + claimed_failure(&staged.image()) + ); + drop(checked); + // OpenVMM attaches block devices of nonzero multiples of 512 bytes alone. + for length in [0, CONTENT.len() as u64, (1 << 20) + 1] { + resize(length); + let binding = staged.bind(); + assert_eq!( + failure(staged.claim(&binding).geometry()), + ( + ErrorKind::Unsupported, + format!( + "the ext4 scratch image {} of {length} bytes cannot be attached: OpenVMM \ + attaches block devices of nonzero multiples of 512 bytes", + shown(&staged.image()) + ) + ) + ); + } +} + #[test] fn a_claim_lasts_while_the_vmm_holds_the_image() { let staged = Staged::new(); diff --git a/nvx-rs/src/host/store/lifecycle/tests.rs b/nvx-rs/src/host/store/lifecycle/tests.rs index f024866af..cdafeaaf7 100644 --- a/nvx-rs/src/host/store/lifecycle/tests.rs +++ b/nvx-rs/src/host/store/lifecycle/tests.rs @@ -10,7 +10,9 @@ use std::thread; use std::time::Duration; use super::*; -use crate::host::store::{CAPABILITY_NAME, CONSOLE_NAME, LOG_NAME, OUTCOME_NAME, SOCKET_NAME}; +use crate::host::store::{ + CAPABILITY_NAME, CLONE_SCRATCH_NAME, CONSOLE_NAME, LOG_NAME, OUTCOME_NAME, SOCKET_NAME, +}; use crate::host::test_support::released; /// The configuration record of these tests' caller. @@ -321,6 +323,7 @@ fn a_removal_removes_the_store_the_callers_files_and_the_directory() { CAPABILITY_NAME, SOCKET_NAME, CLONE_BINDING_NAME, + CLONE_SCRATCH_NAME, BOOT, ".runtime.json.12345.tmp", ".sandbox.lock.7.tmp", diff --git a/nvx-rs/src/host/store/mod.rs b/nvx-rs/src/host/store/mod.rs index 114f987ae..0bef346a1 100644 --- a/nvx-rs/src/host/store/mod.rs +++ b/nvx-rs/src/host/store/mod.rs @@ -15,6 +15,8 @@ //! outcome.json the VMM's outcome report of the latest launch //! .tmpXXXXXX the outcome report while the VMM writes it, which it then publishes //! clone.json the binding of a clone to the generation of a template that it leases +//! clone-scratch.raw the file of the scratch device of a clone whose upper is in RAM, a +//! sparse file that the clone's plan creates //! ... the caller's configuration record and other files, which it declares //! ``` //! @@ -238,7 +240,7 @@ use std::thread; use std::time::{Duration, Instant}; pub use self::caller::CallerFiles; -use self::caller::is_report_staging; +use self::caller::{is_report_staging, is_store_temporary}; use self::files::{io_error, remove_if_present}; pub use self::identity::{ ArtifactIdentity, IdentityCheck, IdentityPart, MAX_IDENTITY_LEN, RuntimeIdentity, @@ -283,11 +285,19 @@ pub const OUTCOME_NAME: &str = "outcome.json"; /// removes once its VM has exited. A [removal](self#removal) of the sandbox removes it first, and /// makes its absence durable before the sandbox goes. pub const CLONE_BINDING_NAME: &str = "clone.json"; +/// Name of the file of the `scratch` block device of a clone whose upper is in RAM, which the +/// clone's restore attaches in place of the template's scratch, and which the guest leaves +/// unused: a sparse file of the template's scratch length, which +/// [`template::ClonePlan::new`](crate::host::template::ClonePlan::new) creates under the lease, +/// replacing one that an earlier clone of the sandbox left, and which +/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM +/// has exited, as a [removal](self#removal) of the sandbox does. +pub const CLONE_SCRATCH_NAME: &str = "clone-scratch.raw"; /// Every file that the store keeps in a sandbox's directory. Replacing a record leaves a /// temporary file named `.NAME.NUMBER.tmp` beside it only if the replacement was interrupted, and /// a VMM that is killed while it writes its outcome report leaves the file in which it stages it; /// see [the report staging file](self#the-report-staging-file). -pub const FILE_NAMES: [&str; 9] = [ +pub const FILE_NAMES: [&str; 10] = [ LOCK_NAME, JOURNAL_NAME, RUNTIME_NAME, @@ -297,6 +307,7 @@ pub const FILE_NAMES: [&str; 9] = [ CONSOLE_NAME, OUTCOME_NAME, CLONE_BINDING_NAME, + CLONE_SCRATCH_NAME, ]; /// Interval at which a lock that another holder has is tried again: the sandbox's lock, by a @@ -489,6 +500,12 @@ impl SandboxStore { self.path(CLONE_BINDING_NAME) } + /// Returns the path of the file of the scratch device of a clone whose upper is in RAM, + /// [`CLONE_SCRATCH_NAME`]. + pub fn clone_scratch_path(&self) -> PathBuf { + self.path(CLONE_SCRATCH_NAME) + } + /// Reads the runtime record, which names the sandbox's VM from the commit of its launch until /// its runtime is cleared. pub fn runtime(&self) -> Result> { @@ -662,6 +679,32 @@ impl LockedStore { self.reserved = launch; } + /// Returns the launch of the sandbox's clone that the lease acquired under this lock reserves, + /// until [`begin_launch`](Self::begin_launch) begins it, or a release ends the lease. + pub(crate) fn reserved(&self) -> Option<&LaunchId> { + self.reserved.as_ref() + } + + /// Returns the paths at which the store reaches the entries of the sandbox's directory that + /// it owns, which its transitions replace and remove: its files, [`FILE_NAMES`], at its own + /// paths, whether or not they exist, which the file system resolves as it resolves every + /// name, without regard to case on Windows, say; and the temporary files of its records and + /// the files in which a VMM staged an outcome report, under the names by which the directory + /// lists them now, as a [removal](self#removal) finds them. + pub(crate) fn owned_entries(&self) -> io::Result> { + let mut owned: Vec = FILE_NAMES.iter().map(|name| self.path(name)).collect(); + for entry in std::fs::read_dir(self.dir())? { + let entry = entry?; + let name = entry.file_name(); + if let Some(name) = name.to_str() + && (is_store_temporary(name) || is_report_staging(name)) + { + owned.push(entry.path()); + } + } + Ok(owned) + } + /// Returns the caller's files that the lock knows: those that a [lifecycle /// lock](self#lifecycle-locks) was taken with, and none for [`SandboxStore::lock`]. pub fn files(&self) -> &CallerFiles { diff --git a/nvx-rs/src/host/store/regular.rs b/nvx-rs/src/host/store/regular.rs index 7a5404b00..771c18ab0 100644 --- a/nvx-rs/src/host/store/regular.rs +++ b/nvx-rs/src/host/store/regular.rs @@ -1,10 +1,12 @@ //! The files that the template modules read, and that a sandbox's store reads as its clone -//! binding: regular files that are not links, which they read within a bound. +//! binding: regular files that are not links, which they read within a bound, and the identity of +//! a file that a template module holds open. use std::fs::{File, OpenOptions}; use std::io::{self, Read}; use std::path::Path; +use crate::host::seal::FileSeal; use crate::host::{filesystem, platform}; /// Returns the error of a file that is a link, or not a regular file. @@ -80,3 +82,11 @@ pub(crate) fn read_at_most(file: File, limit: u64, what: &str) -> io::Result io::Result<(u64, u128)> { + let seal = FileSeal::of(file)?; + Ok((seal.device(), seal.file_id())) +} diff --git a/nvx-rs/src/host/supervisor/mod.rs b/nvx-rs/src/host/supervisor/mod.rs index 3e1781951..1b2bc240e 100644 --- a/nvx-rs/src/host/supervisor/mod.rs +++ b/nvx-rs/src/host/supervisor/mod.rs @@ -110,10 +110,13 @@ //! identity of the launch that the lease reserves, //! [`Lease::launch`](crate::host::template::Lease::launch), before any VMM of the clone //! exists. It returns the sandbox's lock. -//! 2. Under that lock, the caller claims the clone's scratch image, if it has one, as -//! [`ScratchBinding::check`](crate::host::sandbox::scratch::ScratchBinding::check) does, and -//! runs [`launch`] as the reserved launch, which [`LockedStore::begin_launch`] requires of a -//! sandbox that holds a lease, and begins once, under that lock alone, whatever its outcome. +//! 2. Under that lock, the caller plans the clone with +//! [`ClonePlan::new`](crate::host::template::ClonePlan::new), which checks the clone's image, +//! upper, and memory target against the generation, and claims the clone's scratch image, if +//! it has one, as +//! [`ScratchBinding::check`](crate::host::sandbox::scratch::ScratchBinding::check) does. It +//! then runs [`launch`] as the reserved launch, which [`LockedStore::begin_launch`] requires of +//! a sandbox that holds a lease, and begins once, under that lock alone, whatever its outcome. //! 3. Once the clone's VM has exited, as a stop, a termination, or a reattach verifies, //! [`Store::release`](crate::host::template::Store::release) ends the lease, under the //! sandbox's lock, taken through the lease's path, diff --git a/nvx-rs/src/host/template/clone.rs b/nvx-rs/src/host/template/clone.rs new file mode 100644 index 000000000..adc8ddda4 --- /dev/null +++ b/nvx-rs/src/host/template/clone.rs @@ -0,0 +1,783 @@ +//! The plan of a clone, [`ClonePlan`]: the generation of a template that a clone leased, with the +//! clone's own image, upper, and memory target, which the plan checks against the generation +//! before the clone's launch, so that a request that the template cannot serve fails before any +//! VMM starts. + +#[cfg(all(test, any(target_os = "linux", windows)))] +mod tests; + +use std::fmt; +use std::fs::{self, File}; +use std::io; +use std::path::{Path, PathBuf}; +use std::time::Instant; + +use super::key::TemplateKey; +use super::manifest::BlockGeometry; +use super::plan::MEMORY_BLOCK_SIZE; +use super::store::{Acquired, Generation, Lease, Store}; +use crate::contract; +use crate::error::{Error, ErrorKind, Result}; +use crate::host::filesystem; +use crate::host::instance::generate_capability; +use crate::host::paths; +use crate::host::platform; +use crate::host::sandbox::distro::{self, CheckedDistro, DistroBinding, DistroFormat}; +use crate::host::sandbox::plan::bound_image; +use crate::host::sandbox::scratch::{self, CheckedScratch, ScratchBinding}; +use crate::host::sandbox::{ + LayerRole, SandboxLayer, SandboxPlan, block_geometry, reject_container_path, reject_disk_path, +}; +use crate::host::store::LockedStore; +use crate::host::store::files::{create_new, remove_if_present}; +use crate::host::store::regular::{identity_of, open_regular, was_removed}; +use crate::protocol::CAPABILITY_LEN; + +/// Bytes in a mebibyte, the unit in which messages give the size of memory blocks. +const MIB: u64 = 1024 * 1024; + +/// How messages name the file of the `scratch` block device of a clone whose upper is in RAM. +const SPARSE_WHAT: &str = "scratch device of the clone"; + +/// What the clone of a sandbox restores of the generation of a template that it leased: the +/// adapter's plan of the clone's image and upper, the binding of the image, the binding of the +/// scratch image of an upper on one, and the memory target, which [`ClonePlan::new`] checks +/// against the generation. +#[derive(Debug, Clone)] +pub struct CloneRequest { + /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the clone's image, its + /// `distro`, over its upper: in RAM, a tmpfs, as + /// [`SandboxUpper::RAM`](crate::host::sandbox::SandboxUpper::RAM) plans it, or on the + /// caller's scratch image. Its network is the one of the template's NIC, which every restore + /// of the template attaches, or none for a template without a NIC. + pub adapter: SandboxPlan, + /// The binding of the image. + pub binding: DistroBinding, + /// The revision of the distro policy that the caller accepts, such as + /// [`DISTRO_POLICY_REVISION`](crate::host::sandbox::distro::DISTRO_POLICY_REVISION), which + /// [`DistroBinding::check`] requires of the binding's verdict. + pub policy_revision: u32, + /// The binding of the scratch image of an upper on one, or `None` for an upper in RAM. + pub scratch: Option, + /// The memory target, in bytes: the RAM of the clone's VM once restored, to which the restore + /// expands the template's RAM in whole blocks. + pub memory: u64, +} + +/// The upper of a clone's plan, with the file of the clone's `scratch` block device, which every +/// restore of a template attaches in place of the template's scratch. +#[derive(Debug)] +pub enum CloneUpper { + /// An upper in RAM, a tmpfs, which leaves the `scratch` block device unused: its file is a + /// sparse file of the template's scratch length, + /// [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME) in the sandbox's directory, + /// which the plan created, and which [`Store::release`] removes once the clone's VM has + /// exited, as a [removal](crate::host::store#removal) of the sandbox does. + Tmpfs { + /// The sparse file. + path: PathBuf, + }, + /// An upper on the caller's scratch image, which the plan checked and claimed: the claim + /// lasts as long as the checked image, as [`ScratchBinding::check`] describes. + Scratch(CheckedScratch), +} + +/// The plan of a clone: the generation of a template that the clone leased, with the clone's own +/// image, upper, and memory target, checked against the generation, and a fresh capability, which +/// [`new`](Self::new) makes under the lock of the sandbox that acquired the lease, before the +/// clone's launch. +/// +/// A restore of a template attaches the template's two block devices, its `distro` and its +/// `scratch`, with files of the clone's, which OpenVMM opens itself, by their paths, as +/// [Block devices by path](Store#block-devices-by-path) describes, and whose geometries it +/// compares with those that the template recorded. The plan checks each of the clone's inputs as +/// OpenVMM will find it, and holds what the checks opened: +/// +/// - The image, the clone's `distro`, must pass the check of its binding, +/// [`DistroBinding::check`], and present the geometry of the template's placeholder, as its +/// key records it: the placeholder's length, and the same logical and physical block sizes, +/// which [`CheckedDistro::geometry`] reads through the checked file. At its path, it must be a +/// regular file, not a link, nor on Windows any other reparse point, that is the file that the +/// check found, that belongs to the current user or to an administrator, and that no one else +/// may change. The plan holds the file that the check opened, for reading alone. +/// - Neither the image nor a scratch image may be a file that the sandbox's store owns, by any +/// path, through a link among its directories, say, or a hard link: the store's transitions +/// replace or remove its files, as a plan replaces the file of the scratch device of a clone in +/// RAM, [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME), a launch its log, and a +/// release the clone binding. The store creates nothing but regular files under its names, and +/// its control socket, so the plan also refuses a link or a directory under one of them, +/// through which the path of an image could pass. +/// - An upper in RAM, a tmpfs, leaves the `scratch` block device unused, which the restore +/// attaches all the same: the plan creates its file, +/// [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME) in the sandbox's directory, a +/// sparse file of the template's scratch length, private to the current user, in place of one +/// that an earlier clone of the sandbox left, and closes it. The filesystem of the sandbox's +/// directory must present the template's scratch geometry. +/// - An upper on the caller's scratch image must pass the check of its binding, +/// [`ScratchBinding::check`], which checks the image's identity, kind, and length and claims it +/// for the clone, and present the geometry of the template's scratch, its length included, +/// which [`CheckedScratch::geometry`] reads through the checked file. At its path, it must be a +/// regular file, not a link, nor on Windows any other reparse point, that is the file that the +/// check found, and that no one else may change. The plan holds the checked image, and with it +/// the claim. Nothing reads, formats, or copies the image. +/// - The memory target must be whole blocks of [`MEMORY_BLOCK_SIZE`], and of the template's own +/// memory blocks, from the template's RAM to its RAM capacity. +/// - The capability is fresh: the plan draws it for the clone's launch. +/// +/// The plan's checks read the metadata of the clone's files, never their content. +/// +/// ```no_run +/// use std::time::{Duration, Instant}; +/// +/// use nvx::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding}; +/// use nvx::host::sandbox::{SandboxDistro, SandboxPlan, SandboxUpper}; +/// use nvx::host::store::CallerFiles; +/// use nvx::host::template::{ +/// CloneRequest, ClonePlan, Generation, HostTime, MEMORY_BLOCK_SIZE, Store, +/// }; +/// +/// # fn clone( +/// # store: &Store, +/// # generation: &Generation, +/// # image: SandboxDistro, +/// # binding: DistroBinding, +/// # ) -> nvx::Result<()> { +/// let files = CallerFiles::new("config.json")?; +/// let mut acquired = store.acquire( +/// generation.generation(), +/// "/var/lib/example/sandboxes/clone", +/// &files, +/// &HostTime::now()?, +/// None, +/// )?; +/// let request = CloneRequest { +/// adapter: SandboxPlan::adapter(image, SandboxUpper::RAM), +/// binding, +/// policy_revision: DISTRO_POLICY_REVISION, +/// scratch: None, +/// memory: 8 * MEMORY_BLOCK_SIZE, +/// }; +/// let deadline = Instant::now() + Duration::from_secs(5); +/// let plan = ClonePlan::new(store, &mut acquired, &request, deadline)?; +/// // ... the clone's launch, under `acquired.locked`, as the launch that the lease reserves, of +/// // `plan.snapshot()` with `plan.image().path()` and `plan.scratch_path()` ... +/// # let _ = plan; +/// # Ok(()) +/// # } +/// ``` +pub struct ClonePlan { + /// The lease of the generation. + lease: Lease, + /// The leased generation, as the store's index lists it. + generation: Generation, + /// The generation's compatibility key. + key: TemplateKey, + /// The generation's snapshot, which the clone restores. + snapshot: PathBuf, + /// The clone's image, which the check of its binding opened. + image: CheckedDistro, + /// The clone's upper, with the file of its `scratch` block device. + upper: CloneUpper, + /// The memory target, in bytes. + memory: u64, + /// The capability of the clone's launch. + capability: [u8; CAPABILITY_LEN], +} + +/// How a plan reaches the clone's files: as the checks of their bindings reach them, and as it +/// creates the file of the `scratch` block device of an upper in RAM; or, in tests, through opens +/// that read no content, or that reach files on filesystems of other block sizes. +#[derive(Clone, Copy)] +struct Access<'a> { + /// How the check of the image's binding reaches the image. + distro: distro::Access<'a>, + /// How the check of the scratch image's binding reaches the image. + scratch: scratch::Access<'a>, + /// Creates the file of the `scratch` block device of an upper in RAM, open for writing. + create: &'a dyn Fn(&Path) -> io::Result, +} + +impl Access<'static> { + /// The access of every plan outside tests. + const CHECKED: Self = Self { + distro: distro::Access::GUARDED, + scratch: scratch::Access::WRITABLE, + create: &create_sparse, + }; +} + +impl ClonePlan { + /// Plans the clone of the sandbox of `acquired`, which leased a generation of `store`, as + /// `request` asks, waiting until `deadline` for a claim of its scratch image that another + /// launch holds to be released. + /// + /// The plan is made under the lock that acquired the lease, which `acquired` holds, before the + /// launch that the lease reserves, [`Lease::launch`], which the caller then runs under the + /// same lock: the sandbox records no VM and no launch, so no VM of an earlier clone holds a + /// file that the plan replaces. It takes `acquired` exclusively while it plans, since it may + /// create a file in the sandbox's directory, as the transitions of a sandbox's store take its + /// lock, and borrows nothing of it once it returns. It reads the generation's key, + /// [`Store::read_key`], without computing its identity again, and checks the generation's + /// snapshot, [`Store::snapshot_dir`], then checks the clone's files, as the [type](Self) + /// describes. + /// + /// Fails, and holds nothing, keeping no file that it created, checking in this order: + /// + /// - with [`ErrorKind::Unsupported`] for a request that no clone takes: a plan of the sandbox + /// profile, with workload settings and its shares; a plan whose image is not the format and + /// identity that the binding names, as for a binding whose identity is not a UUID in its + /// canonical text; an image or a scratch image whose path is relative, which OpenVMM would + /// resolve against the directory in which it runs, holds a comma or a semicolon, which + /// OpenVMM's block device options would split, or whose name ends in `.vhd`, `.vhdx`, + /// `.vmgs`, or `.iso`, which OpenVMM would not attach as a raw image; and an upper on a + /// scratch image without the image's binding, or one in RAM with a binding; + /// - with [`ErrorKind::Other`] for a lease of another store, and a lock that is not the one + /// that acquired the lease, before the launch that it reserves began; + /// - with [`ErrorKind::NotFound`] if the store no longer indexes the leased generation, with + /// [`ErrorKind::Io`] if it indexes it with another key than the lease names, and as + /// [`Store::index`], [`Store::read_key`], and [`Store::snapshot_dir`] fail; + /// - with [`ErrorKind::Unsupported`] for a request that the template cannot serve: a template + /// whose key names other contracts than this build speaks, the boot and control contracts + /// of the runtime that captured it included; a plan with a network for a template without a + /// NIC, or without one for a template with a NIC; and a memory target that is not whole + /// blocks of [`MEMORY_BLOCK_SIZE`] and of the template's memory blocks, from the template's + /// RAM to its RAM capacity; + /// - with [`ErrorKind::Unavailable`] for a link or a directory under a name that the sandbox's + /// store owns, and with [`ErrorKind::Io`] if the store's files cannot be listed or examined; + /// - as [`DistroBinding::check`] and [`CheckedDistro::geometry`] fail, with + /// [`ErrorKind::Unsupported`] for an image of another geometry than the template's + /// placeholder, its length included, with [`ErrorKind::Unavailable`] for an image that is + /// not, at its path, the regular file that the check found, of the current user or an + /// administrator, and that no one else may change, and with [`ErrorKind::Unsupported`] for + /// an image that is a file of the sandbox's store; + /// - for an upper in RAM, with [`ErrorKind::Unsupported`] for a sandbox whose path holds a + /// comma or a semicolon, or whose filesystem presents another geometry than the template's + /// scratch, and with [`ErrorKind::Io`] if the sparse file cannot be created, sized, or + /// checked; + /// - for an upper on a scratch image, as [`ScratchBinding::check`] and + /// [`CheckedScratch::geometry`] fail, with [`ErrorKind::Unsupported`] for an image of + /// another geometry than the template's scratch, its length included, with + /// [`ErrorKind::Unavailable`] for an image that is not, at its path, the regular file that + /// the check found, of the current user or an administrator, and that no one else may + /// change, and with [`ErrorKind::Unsupported`] for an image that is a file of the sandbox's + /// store. + pub fn new( + store: &Store, + acquired: &mut Acquired, + request: &CloneRequest, + deadline: Instant, + ) -> Result { + Self::new_with(store, acquired, request, deadline, Access::CHECKED) + } + + /// Plans a clone as [`new`](Self::new) does, reaching its files through `access`. + fn new_with( + store: &Store, + acquired: &mut Acquired, + request: &CloneRequest, + deadline: Instant, + access: Access<'_>, + ) -> Result { + check_request(request)?; + let Acquired { lease, locked } = acquired; + check_lease(store, lease, locked)?; + let generation = store + .index()? + .get(lease.generation()) + .cloned() + .ok_or_else(|| { + Error::new( + ErrorKind::NotFound, + format!( + "the template store {} holds no generation {}, which the lease {} names", + store.dir().display(), + lease.generation(), + lease.id() + ), + ) + })?; + if generation.key() != lease.key() { + return Err(Error::io(format!( + "the generation {} of the template store {} has the key {}, not the key {} that \ + the lease {} names", + generation.generation(), + store.dir().display(), + generation.key(), + lease.key(), + lease.id() + ))); + } + let key = store.read_key(&generation)?; + let snapshot = store.snapshot_dir(&generation)?; + check_template(request, &key, &generation)?; + let owned = StoreFiles::of(locked)?; + let image = check_image(request, &key, &owned, access)?; + let capability = generate_capability()?; + let upper = match (request.adapter.upper().scratch_path(), &request.scratch) { + (Some(path), Some(binding)) => CloneUpper::Scratch(check_scratch( + path, binding, &key, &owned, deadline, access, + )?), + _ => CloneUpper::Tmpfs { + path: create_scratch(locked, &key, access)?, + }, + }; + Ok(Self { + lease: lease.clone(), + generation, + key, + snapshot, + image, + upper, + memory: request.memory, + capability, + }) + } + + /// Returns the lease of the generation, which reserves the clone's launch, + /// [`Lease::launch`]. + pub fn lease(&self) -> &Lease { + &self.lease + } + + /// Returns the leased generation, as the store's index lists it. + pub fn generation(&self) -> &Generation { + &self.generation + } + + /// Returns the generation's compatibility key, as the store read it from the generation's + /// payload. + pub fn key(&self) -> &TemplateKey { + &self.key + } + + /// Returns the generation's snapshot, which the clone restores. + pub fn snapshot(&self) -> &Path { + &self.snapshot + } + + /// Returns the clone's image, which the check of its binding opened, for reading alone, and + /// which OpenVMM opens again at its path, [`CheckedDistro::path`], as the clone's `distro`. + pub fn image(&self) -> &CheckedDistro { + &self.image + } + + /// Returns the clone's upper, with the file of its `scratch` block device. + pub fn upper(&self) -> &CloneUpper { + &self.upper + } + + /// Returns the file of the clone's `scratch` block device, which OpenVMM opens at this path: + /// the sparse file [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME) in the + /// sandbox's directory for an upper in RAM, and the caller's scratch image otherwise. + pub fn scratch_path(&self) -> &Path { + match &self.upper { + CloneUpper::Tmpfs { path } => path, + CloneUpper::Scratch(checked) => checked.path(), + } + } + + /// Returns the memory target, in bytes. + pub fn memory(&self) -> u64 { + self.memory + } + + /// Returns the capability of the clone's launch, which the plan drew, and which the VMM + /// receives, as [`LockedStore::begin_launch`] stores it. + pub fn capability(&self) -> &[u8; CAPABILITY_LEN] { + &self.capability + } +} + +impl fmt::Debug for ClonePlan { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + // The capability authenticates control sessions, so it is never shown. + formatter + .debug_struct("ClonePlan") + .field("lease", &self.lease) + .field("generation", &self.generation) + .field("snapshot", &self.snapshot) + .field("image", &self.image) + .field("upper", &self.upper) + .field("memory", &self.memory) + .finish_non_exhaustive() + } +} + +/// Refuses a request that no clone takes, as [`ClonePlan::new`] describes, without reading or +/// opening anything. +fn check_request(request: &CloneRequest) -> Result<()> { + let adapter = &request.adapter; + if adapter.settings().is_some() { + return Err(Error::unsupported( + "a clone restores an adapter's plan, not a plan of the sandbox profile, whose \ + workload settings no template holds", + )); + } + let distro = adapter.distro(); + let binding = &request.binding; + if bound_image(binding)? != distro.image() { + return Err(Error::unsupported(format!( + "the image {} is not the {} layer with identity {} that its distro binding names", + distro.path().display(), + binding.format(), + binding.identity() + ))); + } + check_disk_path(distro.path(), "image")?; + match (adapter.upper().scratch_path(), &request.scratch) { + (Some(path), Some(_)) => check_disk_path(path, "scratch image"), + (None, None) => Ok(()), + (Some(path), None) => Err(Error::unsupported(format!( + "the upper of a clone on the scratch image {} needs the image's scratch binding", + path.display() + ))), + (None, Some(_)) => Err(Error::unsupported( + "the upper of a clone in RAM takes no scratch binding", + )), + } +} + +/// Refuses the path of the file of a clone's block device, which messages call `what`, that +/// OpenVMM would not open as the plan checks it: a relative one, which it would resolve against the +/// directory in which it runs, one with a comma or a semicolon, which its block device options +/// would split, and one whose name ends in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, which it would not +/// attach as a raw image. +fn check_disk_path(path: &Path, what: &str) -> Result<()> { + let shown = path.display().to_string(); + if !path.is_absolute() { + return Err(Error::unsupported(format!( + "the {what} of a clone must be absolute, since OpenVMM would resolve it against the \ + directory in which it runs: {shown}" + ))); + } + reject_disk_path(path, &shown)?; + reject_container_path(path, &shown) +} + +/// Refuses a lease of a store other than `store`, and a lock, `locked`, that is not the one that +/// acquired `lease`, before the launch that it reserves began. +fn check_lease(store: &Store, lease: &Lease, locked: &LockedStore) -> Result<()> { + if lease.store() != store.id() { + return Err(Error::other(format!( + "the lease {} is one of the template store {}, not of this one, {}", + lease.id(), + lease.store(), + store.id() + ))); + } + if locked.dir() != lease.sandbox() || locked.reserved() != Some(lease.launch()) { + return Err(Error::other(format!( + "a clone is planned under the lock that acquired its lease, before the launch that \ + the lease reserves begins, but the lock of the sandbox in {} reserves no launch {} \ + of the lease {}", + locked.dir().display(), + lease.launch(), + lease.id() + ))); + } + Ok(()) +} + +/// Refuses a request that the template of `generation`, whose key is `key`, cannot serve, as +/// [`ClonePlan::new`] describes, without reading or opening anything. +fn check_template( + request: &CloneRequest, + key: &TemplateKey, + generation: &Generation, +) -> Result<()> { + let name = generation.generation(); + // The key's contracts are those through which its clones are restored; the runtime's are + // those through which this build boots and drives the guest that the template holds. + let runtime = key.runtime(); + if !key.speaks_current_contracts() + || runtime.boot_contract != contract::BOOT_CONTRACT_VERSION + || runtime.control_contract != contract::CONTROL_CONTRACT_REVISION + || runtime.control_session != contract::CONTROL_SESSION_PROTOCOL_VERSION + { + return Err(Error::unsupported(format!( + "the template's generation {name} speaks other contracts than this build, which \ + restores none of its clones" + ))); + } + // A network that OpenVMM attaches no device for is no NIC, as a capture class's. + let attached = match request.adapter.network() { + None => false, + Some(network) => !network.openvmm_arguments()?.is_empty(), + }; + if attached != key.has_network() { + return Err(Error::unsupported(if attached { + format!( + "the adapter's plan of a clone attaches a network, but the template's generation \ + {name} has no NIC" + ) + } else { + format!( + "the template's generation {name} has a NIC, which its clones restore with the \ + network of the adapter's plan, but the plan has none" + ) + })); + } + let memory = request.memory; + let block = key.memory_block_size(); + if memory < key.memory() + || memory > key.memory_capacity() + || !memory.is_multiple_of(MEMORY_BLOCK_SIZE) + || !memory.is_multiple_of(block) + { + return Err(Error::unsupported(format!( + "a clone of the template's generation {name} cannot restore with {memory} bytes of \ + RAM: its memory target must be whole blocks of {} MiB, and of the template's memory \ + blocks of {block} bytes, from the template's {} bytes of RAM to its capacity of {} \ + bytes", + MEMORY_BLOCK_SIZE / MIB, + key.memory(), + key.memory_capacity() + ))); + } + Ok(()) +} + +/// Checks the image of `request` against its binding, through `access`, and then its geometry, +/// against the placeholder's that `key` records, the image at its path, and that it is none of +/// `owned`, the files of the sandbox's store, as [`ClonePlan`] describes, and returns the file +/// that the binding's check opened. +fn check_image( + request: &CloneRequest, + key: &TemplateKey, + owned: &StoreFiles, + access: Access<'_>, +) -> Result { + let path = request.adapter.distro().path(); + let shown = path.display(); + let refused = |problem: String| Error::unavailable(format!("the image {shown} {problem}")); + let text = path + .to_str() + .ok_or_else(|| refused("has a path that is not valid Unicode".to_owned()))?; + let binding = &request.binding; + let layer = match binding.format() { + DistroFormat::Erofs => SandboxLayer::new(LayerRole::Distro, text, binding.identity()), + DistroFormat::Gpt => SandboxLayer::gpt(LayerRole::Distro, text, binding.identity()), + }; + let checked = binding.check_with(&layer, request.policy_revision, access.distro)?; + check_geometry( + checked.geometry()?, + key.distro(), + &format!("the image {shown}"), + "placeholder", + )?; + let identity = identity_of(checked.file()) + .map_err(|error| refused(format!("cannot be identified: {error}")))?; + check_at_path(path, identity, refused)?; + owned.check(identity, "image", path)?; + Ok(checked) +} + +/// Checks and claims the scratch image at `path` against `binding`, through `access`, waiting +/// until `deadline` for a claim that another launch holds to be released, and then checks its +/// geometry, against the template's scratch that `key` records, the image at its path, and that +/// it is none of `owned`, the files of the sandbox's store, as [`ClonePlan`] describes, and +/// returns the image that the binding's check opened and claimed. +fn check_scratch( + path: &Path, + binding: &ScratchBinding, + key: &TemplateKey, + owned: &StoreFiles, + deadline: Instant, + access: Access<'_>, +) -> Result { + let shown = path.display(); + let checked = binding.check_with(path, deadline, access.scratch)?; + check_geometry( + checked.geometry()?, + key.scratch(), + &format!("the scratch image {shown}"), + "scratch", + )?; + let refused = + |problem: String| Error::unavailable(format!("the scratch image {shown} {problem}")); + let identity = identity_of(checked.file()) + .map_err(|error| refused(format!("cannot be identified: {error}")))?; + check_at_path(path, identity, refused)?; + owned.check(identity, "scratch image", path)?; + Ok(checked) +} + +/// The regular files that the store of a sandbox owns, with their identities, which no image or +/// scratch image of a clone may be. +struct StoreFiles(Vec<(PathBuf, (u64, u128))>); + +impl StoreFiles { + /// Returns the regular files that the store of the sandbox of `locked` owns now, as + /// [`LockedStore::owned_entries`] lists its entries: its files, among them the file of the + /// scratch device of a clone in RAM, which a plan replaces, its log, which a launch replaces, + /// and the clone binding, which a release removes, and its temporary files. + /// + /// The store creates nothing but regular files under its names, and its control socket, so a + /// link or a directory among its entries is refused: the path of an image could pass through + /// it as through a directory, and a transition that replaces or removes it would leave that + /// path naming nothing. Other kinds of files, such as a socket, are no image. + fn of(locked: &LockedStore) -> Result { + let failed = |error: io::Error| { + Error::io(format!( + "cannot list the files of the sandbox's store in {}", + locked.dir().display() + )) + .with_source(error) + }; + let mut files = Vec::new(); + for entry in locked.owned_entries().map_err(failed)? { + let shown = paths::normalized(&entry); + let absent = |error: &io::Error| { + error.kind() == io::ErrorKind::NotFound || was_removed(&entry, error) + }; + let failed = |error: io::Error| { + Error::io(format!( + "cannot examine {}, a file of the sandbox's store", + shown.display() + )) + .with_source(error) + }; + let metadata = match fs::symlink_metadata(&entry) { + Ok(metadata) => metadata, + Err(error) if absent(&error) => continue, + Err(error) => return Err(failed(error)), + }; + let kind = metadata.file_type(); + if kind.is_dir() || kind.is_symlink() || filesystem::is_reparse_point(&metadata) { + return Err(Error::unavailable(format!( + "{} is a link or a directory, which the sandbox's store never creates there", + shown.display() + ))); + } + if !kind.is_file() { + continue; + } + let file = match open_regular(&entry) { + Ok(file) => file, + Err(error) if absent(&error) => continue, + Err(error) => return Err(failed(error)), + }; + let identity = identity_of(&file).map_err(failed)?; + files.push((shown, identity)); + } + Ok(Self(files)) + } + + /// Refuses the clone's image or scratch image at `path`, which messages call `what`, and + /// whose identity is `identity`, if it is one of these files, by whatever path `path` names + /// it, as through a link among its directories, or a hard link. + fn check(&self, identity: (u64, u128), what: &str, path: &Path) -> Result<()> { + match self.0.iter().find(|(_, owned)| *owned == identity) { + None => Ok(()), + Some((shown, _)) => Err(Error::unsupported(format!( + "the {what} {} is {}, a file of the sandbox's store, which its transitions \ + replace or remove", + path.display(), + shown.display() + ))), + } + } +} + +/// Creates the file of the `scratch` block device of a clone whose upper is in RAM, in the +/// sandbox of `locked`, through `access`: a sparse file of the length of the template's scratch, +/// which `key` records, whose geometry must be the scratch's, and which is checked at its path as +/// the capture of a template checks its own, as [`ClonePlan`] describes. Returns its path, once +/// the file is closed. It removes the file if any step fails, its creation included, which can +/// fail once it made the file, as a file that cannot be made sparse does. +fn create_scratch(locked: &LockedStore, key: &TemplateKey, access: Access<'_>) -> Result { + let path = locked.clone_scratch_path(); + let shown = path.display(); + reject_disk_path(&path, &shown.to_string())?; + let failed = |error: io::Error| { + Error::io(format!("cannot create the {SPARSE_WHAT} {shown}")).with_source(error) + }; + let created = (|| { + let file = (access.create)(&path).map_err(&failed)?; + let scratch = key.scratch(); + file.set_len(scratch.length).map_err(&failed)?; + check_geometry( + block_geometry(&file, &path, SPARSE_WHAT)?, + scratch, + &format!("the {SPARSE_WHAT} {shown}, in the sandbox's filesystem,"), + "scratch", + )?; + let identity = identity_of(&file).map_err(&failed)?; + drop(file); + check_at_path(&path, identity, |problem| { + Error::io(format!("the {SPARSE_WHAT} {shown} {problem}")) + }) + })(); + match created { + Ok(()) => Ok(path), + Err(error) => { + let _ = remove_if_present(&path); + Err(error) + } + } +} + +/// Creates the file of the `scratch` block device of a clone whose upper is in RAM at `path`, a +/// sparse file private to the current user, open for writing, in place of one that an earlier +/// clone of the sandbox left. +fn create_sparse(path: &Path) -> io::Result { + match fs::remove_file(path) { + Err(error) if error.kind() != io::ErrorKind::NotFound => return Err(error), + _ => {} + } + let file = create_new(path)?; + platform::make_sparse(&file)?; + Ok(file) +} + +/// Refuses `found`, the geometry of the file of a clone's block device, which messages call +/// `what`, unless it is `expected`, the geometry that the template's `device` presented. +fn check_geometry( + found: BlockGeometry, + expected: BlockGeometry, + what: &str, + device: &str, +) -> Result<()> { + if found.length != expected.length { + return Err(Error::unsupported(format!( + "{what} is {} bytes long, not the {} bytes of the template's {device}", + found.length, expected.length + ))); + } + if found != expected { + return Err(Error::unsupported(format!( + "{what} presents logical blocks of {} bytes and physical blocks of {} bytes, but the \ + template's {device} presented {} and {} bytes", + found.logical_block_size, + found.physical_block_size, + expected.logical_block_size, + expected.physical_block_size + ))); + } + Ok(()) +} + +/// Checks the file at `path` as OpenVMM will open it, by its path: a regular file, not a link, +/// nor on Windows any other reparse point, whose identity is `expected`, the identity of the file +/// that a check found, and that no one but the current user, or an administrator, may change. +/// Fails with the error that `refused` makes of the problem that it found. +fn check_at_path( + path: &Path, + expected: (u64, u128), + refused: impl Fn(String) -> Error, +) -> Result<()> { + let file = open_regular(path).map_err(|error| refused(format!("cannot be used: {error}")))?; + let identity = + identity_of(&file).map_err(|error| refused(format!("cannot be identified: {error}")))?; + if identity != expected { + return Err(refused( + "changed while it was checked: its path names another file".to_owned(), + )); + } + if let Some(problem) = platform::foreign_writer_problem(&file) + .map_err(|error| refused(format!("cannot be checked: {error}")))? + { + return Err(refused(problem)); + } + Ok(()) +} diff --git a/nvx-rs/src/host/template/clone/tests.rs b/nvx-rs/src/host/template/clone/tests.rs new file mode 100644 index 000000000..ba2bca434 --- /dev/null +++ b/nvx-rs/src/host/template/clone/tests.rs @@ -0,0 +1,1469 @@ +//! Tests of the plans of clones: the checks of their images, uppers, and memory targets against +//! the generation that they lease, which read no content, the claims of their scratch images, and +//! the sparse scratch device of an upper in RAM. + +use std::io::Read; +use std::time::Duration; + +use super::*; +use crate::ErrorCode; +use crate::host::filesystem; +use crate::host::network::{Access as NetworkAccess, NetworkPolicy}; +use crate::host::paths; +use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, PolicyVerdict}; +use crate::host::sandbox::scratch::CLAIM_RELEASE_WAIT; +use crate::host::sandbox::{SandboxDistro, SandboxNetwork, SandboxSettings, SandboxUpper}; +use crate::host::seal::{FileSeal, open_metadata_only}; +use crate::host::store::{CLONE_SCRATCH_NAME, CallerFiles, LOG_NAME, SandboxStore}; +use crate::host::template::test_support::{MIB, ManifestSpec, class, network_class}; +use crate::host::template::{CaptureClass, HostTime}; +use crate::host::test_support::released; +use crate::protocol::root::RootImage; +use crate::uuid::Uuid; + +/// The UUID of the staged images. +const UUID: &str = "11111111-1111-1111-1111-111111111111"; +/// The configuration record of these tests' sandboxes. +const CONFIG: &str = "config.json"; +/// The length of the template's placeholder, and of the staged images: whole huge pages of 2 MiB, +/// as the length of a file on hugetlbfs is. +const IMAGE_LEN: u64 = 4 * MIB; +/// The length of the template's scratch, and of the staged scratch images: a huge page of 2 MiB. +const SCRATCH_LEN: u64 = 2 * MIB; +/// The template's RAM, in bytes, as [`ManifestSpec::template`] records it. +const MEMORY: u64 = 512 * MIB; +/// The template's RAM capacity, in bytes, as [`ManifestSpec::template`] records it. +const MEMORY_CAPACITY: u64 = 2048 * MIB; +/// Nanoseconds in a second. +const SECOND: u64 = 1_000_000_000; + +/// The access of a host that lets the plan open the clone's files and read their metadata, but +/// fails every read and mapping of their content through what the checks of their bindings +/// open: the image opened for its metadata alone, and the scratch image, on Linux, for writing +/// alone, and on Windows through its guard and the claim's handle, which never ask to read. +const NO_CONTENT: Access<'static> = Access { + distro: distro::Access { + open: &open_metadata_only, + seal: &FileSeal::of, + }, + scratch: WRITE_ONLY, + create: &create_sparse, +}; + +/// Opens a scratch image for writing alone, as [`NO_CONTENT`] does. +#[cfg(target_os = "linux")] +const WRITE_ONLY: scratch::Access<'static> = scratch::Access { + open: &open_write_only, +}; +/// Opens a scratch image for writing alone, as [`NO_CONTENT`] does. +#[cfg(windows)] +const WRITE_ONLY: scratch::Access<'static> = scratch::Access::WRITABLE; + +/// Opens `path` for writing alone, without waiting for a reader of a FIFO. +#[cfg(target_os = "linux")] +fn open_write_only(path: &Path) -> io::Result { + use std::os::unix::fs::OpenOptionsExt; + + fs::OpenOptions::new() + .write(true) + .custom_flags(nix::fcntl::OFlag::O_NONBLOCK.bits()) + .open(path) +} + +/// Creates the file `path` of `length` bytes, or truncates it to that length, which no one else +/// may change, whatever the umask, as staging callers keep their images. +fn stage(path: &Path, length: u64) { + let mut options = fs::OpenOptions::new(); + options.write(true).create(true).truncate(true); + #[cfg(unix)] + std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); + options.open(path).unwrap().set_len(length).unwrap(); +} + +/// Returns the physical block size that OpenVMM presents for the files of the directory `dir`: +/// the I/O block size of its filesystem on Linux, and 4096 bytes on Windows. +fn physical_block_size(dir: &Path) -> u32 { + #[cfg(target_os = "linux")] + { + use std::os::unix::fs::MetadataExt; + + let probe = dir.join("probe"); + fs::write(&probe, b"").unwrap(); + let size = fs::metadata(&probe).unwrap().blksize(); + fs::remove_file(&probe).unwrap(); + u32::try_from(size).unwrap() + } + #[cfg(windows)] + { + let _ = dir; + 4096 + } +} + +/// Returns a physical block size other than `physical` that a template may have recorded. +fn other_physical(physical: u32) -> u32 { + if physical > 512 { + physical / 2 + } else { + physical * 2 + } +} + +/// Returns the geometry of a block device of `length` bytes, with logical blocks of 512 bytes and +/// physical blocks of `physical` bytes. +fn geometry(length: u64, physical: u32) -> BlockGeometry { + BlockGeometry { + length, + logical_block_size: 512, + physical_block_size: physical, + } +} + +/// Gives the template of `class`, whose capture `spec` describes, the geometries `distro` and +/// `scratch`. +fn set_geometries( + class: &mut CaptureClass, + spec: &mut ManifestSpec, + distro: BlockGeometry, + scratch: BlockGeometry, +) { + class.distro = distro; + class.scratch = scratch; + for (block, geometry) in spec.contract().blocks.iter_mut().zip([distro, scratch]) { + block.length = geometry.length; + block.logical_block_size = geometry.logical_block_size; + block.physical_block_size = geometry.physical_block_size; + } +} + +/// Returns the files that these tests' callers keep in their sandboxes. +fn files() -> CallerFiles { + CallerFiles::new(CONFIG).unwrap() +} + +/// Returns `path` as messages show it. +fn shown(path: &Path) -> String { + paths::normalized(path).display().to_string() +} + +/// Returns the distro layer at `path`. +fn layer(path: &Path) -> SandboxLayer { + SandboxLayer::new(LayerRole::Distro, path.to_str().unwrap(), UUID) +} + +/// Returns the request of a clone of the image at `image`, which `binding` binds, over an upper in +/// RAM, with the template's RAM as its memory target. +fn tmpfs_request(image: &Path, binding: &DistroBinding) -> CloneRequest { + CloneRequest { + adapter: SandboxPlan::adapter( + SandboxDistro::new(image.to_str().unwrap(), bound_image(binding).unwrap()), + SandboxUpper::RAM, + ), + binding: binding.clone(), + policy_revision: DISTRO_POLICY_REVISION, + scratch: None, + memory: MEMORY, + } +} + +/// Returns the request of a clone of the image at `image`, which `binding` binds, over an upper on +/// the scratch image at `scratch`, which `scratch_binding` binds, with the template's RAM as its +/// memory target. +fn scratch_request( + image: &Path, + binding: &DistroBinding, + scratch: &Path, + scratch_binding: &ScratchBinding, +) -> CloneRequest { + let request = tmpfs_request(image, binding); + CloneRequest { + adapter: SandboxPlan::adapter( + request.adapter.distro().clone(), + SandboxUpper::scratch(scratch.to_str().unwrap()), + ), + scratch: Some(scratch_binding.clone()), + ..request + } +} + +/// Returns the request `request` with the network `network`. +fn with_network(request: CloneRequest, network: Option) -> CloneRequest { + CloneRequest { + adapter: request.adapter.clone().with_network(network), + ..request + } +} + +/// Returns the network of a guest at `10.0.0.2/24` under a policy whose default is `access`. +fn network(access: NetworkAccess) -> SandboxNetwork { + SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(access)).unwrap() +} + +/// Returns the kind and the message of the error of `result`. +fn failure(result: Result) -> (ErrorKind, String) { + let error = result.unwrap_err(); + (error.kind(), error.to_string()) +} + +/// Returns the time of the host that captured `generation`, during the boot of the capture, +/// `seconds` after it. +fn after(generation: &Generation, seconds: u64) -> HostTime { + let time = generation.time(); + HostTime { + host_id: time.host_id, + boot_id: time.boot_id, + clock: time.host_clock, + monotonic_ns: time.capture_monotonic_ns + seconds * SECOND, + utc_ns: time.capture_utc_ns + seconds * SECOND, + } +} + +/// A template store in a temporary directory, with a published generation whose `distro` and +/// scratch have the geometries of the files of the directory, and the directories of the staged +/// images, scratch images, and sandboxes. +struct Fixture { + /// The temporary directory. + directory: tempfile::TempDir, + /// The store. + store: Store, + /// The published generation. + generation: Generation, + /// The physical block size of the files of the directory. + physical: u32, +} + +impl Fixture { + /// Publishes a template whose `distro` is [`IMAGE_LEN`] bytes long and whose scratch is + /// [`SCRATCH_LEN`] bytes long, both with the physical block size of the files of the + /// directory. + fn new() -> Self { + Self::with(|_, _, _| {}) + } + + /// Publishes a template as [`new`](Self::new) does, of the class and manifest that `change` + /// changes, given the physical block size of the files of the directory. + fn with(change: impl FnOnce(&mut CaptureClass, &mut ManifestSpec, u32)) -> Self { + let directory = tempfile::tempdir().unwrap(); + let physical = physical_block_size(directory.path()); + let store = Store::create(directory.path().join("templates")).unwrap(); + let (mut class, mut spec) = (class(), ManifestSpec::template()); + set_geometries( + &mut class, + &mut spec, + geometry(IMAGE_LEN, physical), + geometry(SCRATCH_LEN, physical), + ); + change(&mut class, &mut spec, physical); + let generation = store.publish_spec_for_tests(&class, &spec); + // Staging callers keep images in directories that only their own account can write. + for name in ["images", "scratches", "sandboxes"] { + let dir = directory.path().join(name); + assert_eq!(platform::create_owner_only_dir(&dir).unwrap(), None); + } + Self { + directory, + store, + generation, + physical, + } + } + + /// Returns the path of the file `name` of the directory `kind`. + fn path(&self, kind: &str, name: &str) -> PathBuf { + self.directory.path().join(kind).join(name) + } + + /// Stages the image `name` of `length` bytes, and returns its path and its binding. + fn image(&self, name: &str, length: u64) -> (PathBuf, DistroBinding) { + let path = self.path("images", name); + stage(&path, length); + let binding = DistroBinding::import(&layer(&path), PolicyVerdict::ACCEPTED).unwrap(); + (path, binding) + } + + /// Stages the scratch image `name` of `length` bytes, and returns its path and its binding. + fn scratch(&self, name: &str, length: u64) -> (PathBuf, ScratchBinding) { + let path = self.path("scratches", name); + stage(&path, length); + let binding = ScratchBinding::bind(&path).unwrap(); + (path, binding) + } + + /// Provisions the sandbox `name`, and leases the generation to it, a second after its + /// capture. + fn acquire(&self, name: &str) -> Acquired { + let dir = self.path("sandboxes", name); + let locked = SandboxStore::lock_for_provision(&dir, &files(), None).unwrap(); + fs::write(dir.join(CONFIG), b"{}").unwrap(); + drop(locked); + self.store + .acquire( + self.generation.generation(), + filesystem::canonicalize(&dir).unwrap(), + &files(), + &after(&self.generation, 1), + None, + ) + .unwrap() + } + + /// Plans the clone of `acquired` as `request` asks, through `access`, waiting + /// [`CLAIM_RELEASE_WAIT`] at most for the claim of its scratch image. + fn plan( + &self, + acquired: &mut Acquired, + request: &CloneRequest, + access: Access<'_>, + ) -> Result { + ClonePlan::new_with( + &self.store, + acquired, + request, + Instant::now() + CLAIM_RELEASE_WAIT, + access, + ) + } +} + +/// Returns the path of the scratch device's file of a clone in RAM in the sandbox of `acquired`. +fn sparse_path(acquired: &Acquired) -> PathBuf { + acquired.locked.dir().join(CLONE_SCRATCH_NAME) +} + +/// Asserts that the sandbox of `acquired` holds no scratch device's file of a clone in RAM, and +/// that its lease still reserves its launch, as after a plan that failed. +fn assert_untouched(acquired: &Acquired) { + assert!(!sparse_path(acquired).exists()); + assert_eq!(acquired.locked.reserved(), Some(acquired.lease.launch())); +} + +/// Returns a file of `length` bytes on hugetlbfs, whose I/O block size is a huge page of 2 MiB, +/// or `None`, saying why, if the host offers none. +#[cfg(target_os = "linux")] +fn huge_file(length: u64) -> Option { + use nix::sys::memfd::{MFdFlags, memfd_create}; + + let flags = MFdFlags::MFD_CLOEXEC | MFdFlags::MFD_HUGETLB | MFdFlags::MFD_HUGE_2MB; + let file = match memfd_create("nvx-clone-test", flags) { + Ok(descriptor) => File::from(descriptor), + Err(error) => { + eprintln!("this host offers no file on hugetlbfs: {error}"); + return None; + } + }; + if let Err(error) = file.set_len(length) { + eprintln!("this host cannot size a file on hugetlbfs: {error}"); + return None; + } + Some(file) +} + +/// Links `link` to the file `target` with a symbolic link, and returns whether it could: Windows +/// lets only Developer Mode or a privilege create one. +fn symlink_file(target: &Path, link: &Path) -> bool { + #[cfg(unix)] + let linked = std::os::unix::fs::symlink(target, link); + #[cfg(windows)] + let linked = std::os::windows::fs::symlink_file(target, link); + created_link(linked) +} + +/// Links `link` to the directory `target` with a symbolic link, and returns whether it could, as +/// [`symlink_file`] does. +fn symlink_dir(target: &Path, link: &Path) -> bool { + #[cfg(unix)] + let linked = std::os::unix::fs::symlink(target, link); + #[cfg(windows)] + let linked = std::os::windows::fs::symlink_dir(target, link); + created_link(linked) +} + +/// Returns whether a symbolic link was created, as `linked` reports, and fails unless Windows +/// refused it. +fn created_link(linked: io::Result<()>) -> bool { + match linked { + Ok(()) => true, + Err(error) if cfg!(windows) => { + eprintln!("cannot create a symbolic link: {error}"); + false + } + Err(error) => panic!("cannot create a symbolic link: {error}"), + } +} + +#[test] +fn a_clone_in_ram_restores_its_lease_with_its_image_and_a_sparse_scratch_device() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + let store = &fixture.store; + assert_eq!(plan.lease(), &acquired.lease); + assert_eq!(plan.generation(), &fixture.generation); + assert_eq!(plan.key(), &store.read_key(&fixture.generation).unwrap()); + assert_eq!( + plan.snapshot(), + store.snapshot_dir(&fixture.generation).unwrap().as_path() + ); + assert_eq!(plan.memory(), MEMORY); + // The image is the file that its binding's check opened, which reads nothing. + assert_eq!(plan.image().path().to_str().unwrap(), shown(&image)); + assert_eq!( + plan.image().geometry().unwrap(), + geometry(IMAGE_LEN, fixture.physical) + ); + let mut byte = [0]; + assert!(plan.image().file().read(&mut byte).is_err()); + // The scratch device's file is a sparse file of the template's scratch length, private to the + // current user, in the sandbox's directory, which the plan closed. + let sparse = sparse_path(&acquired); + assert!(matches!(plan.upper(), CloneUpper::Tmpfs { path } if *path == sparse)); + assert_eq!(plan.scratch_path(), sparse.as_path()); + let metadata = fs::metadata(&sparse).unwrap(); + assert_eq!(metadata.len(), SCRATCH_LEN); + #[cfg(target_os = "linux")] + { + use std::os::unix::fs::MetadataExt; + + assert_eq!(metadata.mode() & 0o777, 0o600); + assert!( + metadata.blocks() * 512 < SCRATCH_LEN, + "{}", + metadata.blocks() + ); + } + #[cfg(windows)] + { + use std::os::windows::fs::MetadataExt; + + /// `FILE_ATTRIBUTE_SPARSE_FILE`. + const SPARSE: u32 = 0x200; + assert_ne!(metadata.file_attributes() & SPARSE, 0); + } + let opened = File::open(&sparse).unwrap(); + assert_eq!(platform::foreign_writer_problem(&opened).unwrap(), None); + drop(opened); + // The capability is fresh, and never shown. + let capability = *plan.capability(); + assert_ne!(capability, [0; CAPABILITY_LEN]); + let shown_plan = format!("{plan:?}"); + assert!(shown_plan.starts_with("ClonePlan {"), "{shown_plan}"); + assert!( + !shown_plan.contains(&format!("{capability:?}")), + "{shown_plan}" + ); + assert!(!shown_plan.contains("capability"), "{shown_plan}"); + // The plan borrows nothing of the acquisition, whose lease still reserves its launch. + assert_eq!(acquired.locked.reserved(), Some(acquired.lease.launch())); + drop(plan); + // The access of launches plans alike, with another capability. + let again = ClonePlan::new(store, &mut acquired, &request, Instant::now()).unwrap(); + assert_ne!(*again.capability(), capability); + assert_eq!(again.scratch_path(), sparse.as_path()); +} + +#[test] +fn the_scratch_device_of_a_clone_in_ram_is_fresh_and_goes_with_its_lease() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + // An earlier clone of the sandbox left its file, which the plan replaces. + let sparse = sparse_path(&acquired); + fs::write(&sparse, b"left by an earlier clone").unwrap(); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!(plan.scratch_path(), sparse.as_path()); + let mut content = vec![1; 64]; + File::open(&sparse) + .unwrap() + .read_exact(&mut content) + .unwrap(); + assert_eq!(content, [0; 64]); + drop(plan); + // The release of the lease, once the clone's VM has exited, removes it with the binding. + assert_eq!( + fixture.store.release(&mut acquired.locked).unwrap(), + Some(acquired.lease.clone()) + ); + assert!(!sparse.exists()); + assert!(!acquired.locked.binding_path().exists()); + // The removal of the sandbox removes one that remains, as one of the store's files. + fs::write(&sparse, b"").unwrap(); + let Acquired { locked, .. } = acquired; + assert!(locked.remove().unwrap().is_none()); + assert!(!sparse.exists()); +} + +#[test] +fn a_scratch_device_that_the_plan_cannot_initialize_goes() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + // A creation that makes the file, and then fails to make it sparse, as one on a volume that + // has no sparse files does. + let create = |path: &Path| -> io::Result { + drop(create_new(path)?); + Err(io::Error::other("the volume has no sparse files")) + }; + let access = Access { + create: &create, + ..NO_CONTENT + }; + let error = fixture.plan(&mut acquired, &request, access).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Io, "{error}"); + assert_eq!( + error.message(), + format!( + "cannot create the scratch device of the clone {}", + shown(&sparse_path(&acquired)) + ) + ); + assert_eq!( + std::error::Error::source(&error).unwrap().to_string(), + "the volume has no sparse files" + ); + assert_untouched(&acquired); + // The next plan creates it afresh. + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!( + fs::metadata(plan.scratch_path()).unwrap().len(), + SCRATCH_LEN + ); +} + +#[test] +fn files_of_the_sandboxs_store_are_refused_as_images() { + let fixture = Fixture::new(); + let mut acquired = fixture.acquire("sandbox"); + let sparse = sparse_path(&acquired); + let refusal = |what: &str, path: &Path, owned: &Path| { + ( + ErrorKind::Unsupported, + format!( + "the {what} {} is {}, a file of the sandbox's store, which its transitions \ + replace or remove", + shown(path), + shown(owned) + ), + ) + }; + let seal_of = |path: &Path| FileSeal::of(&File::open(path).unwrap()).unwrap(); + // An image that is the file of the scratch device of a clone in RAM, which a plan would + // replace, is refused before anything replaces it... + stage(&sparse, IMAGE_LEN); + let binding = DistroBinding::import(&layer(&sparse), PolicyVerdict::ACCEPTED).unwrap(); + let staged = seal_of(&sparse); + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&sparse, &binding), NO_CONTENT)), + refusal("image", &sparse, &sparse) + ); + assert_eq!(seal_of(&sparse), staged); + // ...by whatever path the request names it: through a hard link... + let hard = fixture.path("images", "hard.erofs"); + fs::hard_link(&sparse, &hard).unwrap(); + let hard_binding = DistroBinding::import(&layer(&hard), PolicyVerdict::ACCEPTED).unwrap(); + assert_eq!( + failure(fixture.plan( + &mut acquired, + &tmpfs_request(&hard, &hard_binding), + NO_CONTENT + )), + refusal("image", &hard, &sparse) + ); + // ...or through a link among its directories. + let alias = fixture.path("images", "alias"); + if symlink_dir(acquired.locked.dir(), &alias) { + let aliased = alias.join(CLONE_SCRATCH_NAME); + let aliased_binding = + DistroBinding::import(&layer(&aliased), PolicyVerdict::ACCEPTED).unwrap(); + assert_eq!( + failure(fixture.plan( + &mut acquired, + &tmpfs_request(&aliased, &aliased_binding), + NO_CONTENT + )), + refusal("image", &aliased, &sparse) + ); + } + // A release of the lease removes that file, so a clone on a scratch image takes it neither + // as its image, which is bound again, since the hard link changed its change time on + // Linux... + let binding = DistroBinding::import(&layer(&sparse), PolicyVerdict::ACCEPTED).unwrap(); + let staged = seal_of(&sparse); + let (image, image_binding) = fixture.image("distro.erofs", IMAGE_LEN); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let request = scratch_request(&sparse, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + refusal("image", &sparse, &sparse) + ); + assert_eq!(seal_of(&sparse), staged); + // ...nor as its scratch image, whose claim the refused plan releases. + fs::remove_file(&sparse).unwrap(); + stage(&sparse, SCRATCH_LEN); + let sparse_binding = ScratchBinding::bind(&sparse).unwrap(); + let request = scratch_request(&image, &image_binding, &sparse, &sparse_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + refusal("scratch image", &sparse, &sparse) + ); + assert_eq!(fs::metadata(&sparse).unwrap().len(), SCRATCH_LEN); + assert!(released(|| sparse_binding + .check(&sparse, Instant::now()) + .is_ok())); + fs::remove_file(&sparse).unwrap(); + // The store's other files are refused alike, such as its log, which a launch replaces, and + // the temporary files of its records. + for name in [LOG_NAME, ".runtime.json.1.tmp"] { + let owned = acquired.locked.dir().join(name); + stage(&owned, IMAGE_LEN); + let binding = DistroBinding::import(&layer(&owned), PolicyVerdict::ACCEPTED).unwrap(); + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&owned, &binding), NO_CONTENT)), + refusal("image", &owned, &owned) + ); + assert_eq!(fs::metadata(&owned).unwrap().len(), IMAGE_LEN); + fs::remove_file(&owned).unwrap(); + } + assert_untouched(&acquired); +} + +#[test] +fn links_and_directories_under_the_names_of_the_sandboxs_store_are_refused() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let sparse = sparse_path(&acquired); + let refusal = |owned: &Path| { + ( + ErrorKind::Unavailable, + format!( + "{} is a link or a directory, which the sandbox's store never creates there", + shown(owned) + ), + ) + }; + let request = tmpfs_request(&image, &binding); + // A directory, which a plan could not replace, nor a release or a launch remove. + for name in [CLONE_SCRATCH_NAME, LOG_NAME] { + let owned = acquired.locked.dir().join(name); + fs::create_dir(&owned).unwrap(); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + refusal(&owned) + ); + assert!(owned.is_dir()); + fs::remove_dir(&owned).unwrap(); + } + // A link to the directory of the images, through which the path of an image can pass, and + // which a plan would replace, leaving that path naming nothing. + if symlink_dir(image.parent().unwrap(), &sparse) { + let through = sparse.join("distro.erofs"); + let through_binding = + DistroBinding::import(&layer(&through), PolicyVerdict::ACCEPTED).unwrap(); + for request in [tmpfs_request(&through, &through_binding), request.clone()] { + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + refusal(&sparse) + ); + } + assert!( + fs::symlink_metadata(&sparse) + .unwrap() + .file_type() + .is_symlink() + ); + assert!(fs::metadata(&through).unwrap().is_file()); + #[cfg(unix)] + fs::remove_file(&sparse).unwrap(); + #[cfg(windows)] + fs::remove_dir(&sparse).unwrap(); + } + assert_untouched(&acquired); + // A control socket that a launch left is no image, and the launch removes it. + #[cfg(unix)] + let _socket = std::os::unix::net::UnixListener::bind( + acquired.locked.dir().join(crate::host::store::SOCKET_NAME), + ) + .unwrap(); + // The plan creates the file of the scratch device once nothing else is there. + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert!(fs::symlink_metadata(plan.scratch_path()).unwrap().is_file()); +} + +#[test] +fn names_that_the_file_system_takes_for_those_of_the_stores_files_are_the_stores() { + let fixture = Fixture::new(); + let (image, image_binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let dir = acquired.locked.dir().to_owned(); + let refusal = |what: &str, path: &Path, owned: &Path| { + ( + ErrorKind::Unsupported, + format!( + "the {what} {} is {}, a file of the sandbox's store, which its transitions \ + replace or remove", + shown(path), + shown(owned) + ), + ) + }; + // Whether the file system takes the paths `a` and `b` for one file. + let same_file = |a: &Path, b: &Path| { + let identity = |path: &Path| { + File::open(path) + .and_then(|file| FileSeal::of(&file)) + .map(|seal| (seal.device(), seal.file_id())) + .ok() + }; + identity(a).is_some() && identity(a) == identity(b) + }; + // An image under the name of one of the store's files in other letter cases is that file + // where the file system ignores case, as Windows' do, and another file where it does not. + for name in [CLONE_SCRATCH_NAME, LOG_NAME] { + let cased = dir.join(name.to_ascii_uppercase()); + stage(&cased, IMAGE_LEN); + let binding = DistroBinding::import(&layer(&cased), PolicyVerdict::ACCEPTED).unwrap(); + let owned = dir.join(name); + let same = same_file(&cased, &owned); + let result = fixture.plan(&mut acquired, &tmpfs_request(&cased, &binding), NO_CONTENT); + if same { + assert_eq!(failure(result), refusal("image", &cased, &owned)); + } else { + drop(result.unwrap()); + } + assert_eq!(fs::metadata(&cased).unwrap().len(), IMAGE_LEN); + fs::remove_file(&cased).unwrap(); + } + // So is a scratch image, which a release would remove as the store's file where the file + // system takes it for that file, and keeps where it does not. + let sparse = sparse_path(&acquired); + let _ = fs::remove_file(&sparse); + let cased = dir.join(CLONE_SCRATCH_NAME.to_ascii_uppercase()); + stage(&cased, SCRATCH_LEN); + let binding = ScratchBinding::bind(&cased).unwrap(); + let request = scratch_request(&image, &image_binding, &cased, &binding); + let same = same_file(&cased, &sparse); + let result = fixture.plan(&mut acquired, &request, NO_CONTENT); + if same { + assert_eq!(failure(result), refusal("scratch image", &cased, &sparse)); + } else { + drop(result.unwrap()); + assert_eq!( + fixture.store.release(&mut acquired.locked).unwrap(), + Some(acquired.lease.clone()) + ); + } + assert_eq!(fs::metadata(&cased).unwrap().len(), SCRATCH_LEN); +} + +#[test] +fn templates_that_a_runtime_of_other_contracts_captured_are_refused() { + // A store that a build of other boot or control contracts kept holds templates whose guests + // this build cannot drive. + type ClassChange = fn(&mut CaptureClass); + let changes: [(&str, ClassChange); 3] = [ + ("boot contract", |class| class.runtime.boot_contract += 1), + ("control contract", |class| { + class.runtime.control_contract = "other-revision".to_owned(); + }), + ("control session", |class| { + class.runtime.control_session += 1 + }), + ]; + for (name, change) in changes { + let fixture = Fixture::with(|class, _, _| change(class)); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the template's generation {} speaks other contracts than this build, which \ + restores none of its clones", + fixture.generation.generation() + ) + ), + "{name}" + ); + assert_untouched(&acquired); + } +} + +#[test] +fn a_clone_on_a_scratch_image_claims_it_and_reads_no_content() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + let CloneUpper::Scratch(checked) = plan.upper() else { + panic!("the clone's upper is not on its scratch image: {plan:?}"); + }; + assert_eq!(checked.path().to_str().unwrap(), shown(&scratch)); + assert_eq!(plan.scratch_path(), checked.path()); + assert_eq!( + checked.geometry().unwrap(), + geometry(SCRATCH_LEN, fixture.physical) + ); + // What the checks hold reads nothing, and the plan creates no scratch device of its own. + let mut byte = [0]; + assert!(checked.file().read(&mut byte).is_err()); + assert!(plan.image().file().read(&mut byte).is_err()); + assert!(!sparse_path(&acquired).exists()); + // The plan holds the claim: another check finds the image claimed. + let error = scratch_binding.check(&scratch, Instant::now()).unwrap_err(); + assert_eq!(error.code(), ErrorCode::ScratchClaimed, "{error}"); + drop(plan); + assert!(released(|| scratch_binding + .check(&scratch, Instant::now()) + .is_ok())); +} + +#[test] +fn a_scratch_image_serves_one_clone_at_a_time() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + let mut first = fixture.acquire("first"); + let mut second = fixture.acquire("second"); + let plan = fixture.plan(&mut first, &request, NO_CONTENT).unwrap(); + // The other clone's plan waits until its deadline for the claim, then fails, holding nothing. + let wait = Duration::from_millis(200); + let started = Instant::now(); + let error = ClonePlan::new_with( + &fixture.store, + &mut second, + &request, + started + wait, + NO_CONTENT, + ) + .unwrap_err(); + assert!(started.elapsed() >= wait); + assert_eq!(error.code(), ErrorCode::ScratchClaimed, "{error}"); + assert_eq!(error.kind(), ErrorKind::TimedOut, "{error}"); + assert_eq!( + error.message(), + format!( + "another sandbox claims the ext4 scratch image {}", + shown(&scratch) + ) + ); + assert_untouched(&second); + // Once the first plan is gone, the image serves the other clone. + drop(plan); + let plan = fixture.plan(&mut second, &request, NO_CONTENT).unwrap(); + assert!(matches!(plan.upper(), CloneUpper::Scratch(_))); +} + +#[test] +fn scratch_images_whose_identity_or_kind_changed_are_refused() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let changed = |path: &Path, field: &str| { + ( + ErrorKind::Unavailable, + format!( + "{} changed after its scratch binding was made: its {field} differs", + shown(path) + ), + ) + }; + // Another file in the image's place, of the same length, has another identity. + let (scratch, scratch_binding) = fixture.scratch("replaced.ext4", SCRATCH_LEN); + let copy = fixture.path("scratches", "copy.ext4"); + fs::copy(&scratch, ©).unwrap(); + fs::rename(©, &scratch).unwrap(); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + changed(&scratch, "identity") + ); + // So has a directory in its place. The image stays, under another name, so that the + // directory cannot take its inode number, which is its file ID on Linux. + let (scratch, scratch_binding) = fixture.scratch("directory.ext4", SCRATCH_LEN); + fs::rename(&scratch, fixture.path("scratches", "moved.ext4")).unwrap(); + fs::create_dir(&scratch).unwrap(); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + changed(&scratch, "identity") + ); + // A binding of another kind names another file. + let (scratch, scratch_binding) = fixture.scratch("kind.ext4", SCRATCH_LEN); + let mut value = serde_json::to_value(&scratch_binding).unwrap(); + value["kind"] = "directory".into(); + let scratch_binding: ScratchBinding = serde_json::from_value(value).unwrap(); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + changed(&scratch, "kind") + ); + assert_untouched(&acquired); +} + +#[test] +fn images_of_another_length_or_geometry_are_refused() { + let fixture = Fixture::new(); + let mut acquired = fixture.acquire("sandbox"); + let mut refusal = |length: u64| { + let (image, binding) = fixture.image("other.erofs", length); + let failed = + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), NO_CONTENT)); + fs::remove_file(&image).unwrap(); + (shown(&image), failed) + }; + // A clone's image has the placeholder's length. + for length in [IMAGE_LEN - 512, IMAGE_LEN + 512, 2 * IMAGE_LEN] { + let (image, failed) = refusal(length); + assert_eq!( + failed, + ( + ErrorKind::Unsupported, + format!( + "the image {image} is {length} bytes long, not the {IMAGE_LEN} bytes of the \ + template's placeholder" + ) + ) + ); + } + // OpenVMM attaches no block device of another length. + for length in [0, IMAGE_LEN + 1] { + let (image, failed) = refusal(length); + assert_eq!( + failed, + ( + ErrorKind::Unsupported, + format!( + "the distro EROFS layer {image} of {length} bytes cannot be attached: OpenVMM \ + attaches block devices of nonzero multiples of 512 bytes" + ) + ) + ); + } + assert_untouched(&acquired); + + // A template whose placeholder presented physical blocks of another size serves no image of + // the filesystem of this one. + let fixture = Fixture::with(|class, spec, physical| { + let scratch = class.scratch; + let distro = geometry(IMAGE_LEN, other_physical(physical)); + set_geometries(class, spec, distro, scratch); + }); + let mut acquired = fixture.acquire("sandbox"); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let physical = fixture.physical; + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the image {} presents logical blocks of 512 bytes and physical blocks of \ + {physical} bytes, but the template's placeholder presented 512 and {} bytes", + shown(&image), + other_physical(physical) + ) + ) + ); + assert_untouched(&acquired); +} + +#[test] +fn scratch_images_of_another_length_or_geometry_are_refused() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + for length in [SCRATCH_LEN - 512, SCRATCH_LEN + 512] { + let (scratch, scratch_binding) = fixture.scratch("other.ext4", length); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the scratch image {} is {length} bytes long, not the {SCRATCH_LEN} bytes of \ + the template's scratch", + shown(&scratch) + ) + ) + ); + // The refused plan holds no claim. + drop( + scratch_binding + .check(&scratch, Instant::now() + CLAIM_RELEASE_WAIT) + .unwrap(), + ); + fs::remove_file(&scratch).unwrap(); + } + let (scratch, scratch_binding) = fixture.scratch("unaligned.ext4", SCRATCH_LEN + 1); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the ext4 scratch image {} of {} bytes cannot be attached: OpenVMM attaches block \ + devices of nonzero multiples of 512 bytes", + shown(&scratch), + SCRATCH_LEN + 1 + ) + ) + ); + assert_untouched(&acquired); +} + +#[test] +fn sandboxes_whose_filesystem_presents_another_scratch_geometry_are_refused() { + // A template whose scratch presented physical blocks of another size than the filesystem of + // the sandboxes serves no clone in RAM there, nor on a scratch image of that filesystem. + let fixture = Fixture::with(|class, spec, physical| { + let distro = class.distro; + let scratch = geometry(SCRATCH_LEN, other_physical(physical)); + set_geometries(class, spec, distro, scratch); + }); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let (physical, other) = (fixture.physical, other_physical(fixture.physical)); + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the scratch device of the clone {}, in the sandbox's filesystem, presents \ + logical blocks of 512 bytes and physical blocks of {physical} bytes, but the \ + template's scratch presented 512 and {other} bytes", + shown(&sparse_path(&acquired)) + ) + ) + ); + // The refused plan removed the file that it created. + assert_untouched(&acquired); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the scratch image {} presents logical blocks of 512 bytes and physical blocks of \ + {physical} bytes, but the template's scratch presented 512 and {other} bytes", + shown(&scratch) + ) + ) + ); + assert_untouched(&acquired); +} + +/// Files on hugetlbfs, whose I/O block size is a huge page, present physical blocks of 2 MiB, +/// unlike the template's: an image, a scratch image, and the scratch device of a clone in RAM, +/// which each reach such a file. +#[cfg(target_os = "linux")] +#[test] +fn files_on_a_filesystem_of_another_block_size_are_refused() { + use crate::host::sandbox::distro::import_with; + use crate::host::sandbox::scratch::bind_with; + + const HUGE_PAGE: u64 = 2 * MIB; + + let fixture = Fixture::new(); + if u64::from(fixture.physical) == HUGE_PAGE { + eprintln!("the files of the test's directory present physical blocks of a huge page"); + return; + } + let Some(huge_image) = huge_file(IMAGE_LEN) else { + return; + }; + let mut acquired = fixture.acquire("sandbox"); + let presented = |what: String, device: &str| { + ( + ErrorKind::Unsupported, + format!( + "{what} presents logical blocks of 512 bytes and physical blocks of {HUGE_PAGE} \ + bytes, but the template's {device} presented 512 and {} bytes", + fixture.physical + ), + ) + }; + // The image: its binding and the check of the binding reach a file on hugetlbfs at its path. + let image = fixture.path("images", "huge.erofs"); + let open = |path: &Path| { + if path == image { + huge_image.try_clone() + } else { + Err(io::Error::from(io::ErrorKind::NotFound)) + } + }; + let huge_distro = distro::Access { + open: &open, + seal: &FileSeal::of, + }; + let binding = import_with(&layer(&image), PolicyVerdict::ACCEPTED, huge_distro).unwrap(); + let access = Access { + distro: huge_distro, + ..NO_CONTENT + }; + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), access)), + presented(format!("the image {}", shown(&image)), "placeholder") + ); + // The scratch image: its binding and its check reach a file on hugetlbfs at its path. + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let huge_scratch = huge_file(SCRATCH_LEN).unwrap(); + let scratch = fixture.path("scratches", "huge.ext4"); + let open = |_: &Path| huge_scratch.try_clone(); + let scratch_access = scratch::Access { open: &open }; + let scratch_binding = bind_with(&paths::normalized(&scratch), scratch_access).unwrap(); + let access = Access { + scratch: scratch_access, + ..NO_CONTENT + }; + let request = scratch_request(&image, &binding, &scratch, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, access)), + presented(format!("the scratch image {}", shown(&scratch)), "scratch") + ); + // The scratch device of a clone in RAM, which a sandbox on hugetlbfs creates there. + let huge_device = huge_file(SCRATCH_LEN).unwrap(); + let create = |_: &Path| huge_device.try_clone(); + let access = Access { + create: &create, + ..NO_CONTENT + }; + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&image, &binding), access)), + presented( + format!( + "the scratch device of the clone {}, in the sandbox's filesystem,", + shown(&sparse_path(&acquired)) + ), + "scratch" + ) + ); + assert_untouched(&acquired); +} + +#[test] +fn memory_targets_are_whole_blocks_from_the_templates_ram_to_its_capacity() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let block = MEMORY_BLOCK_SIZE; + let generation = fixture.generation.generation(); + for memory in [ + 0, + MEMORY - block, + MEMORY + 1, + MEMORY + block / 2, + MEMORY_CAPACITY - 4096, + MEMORY_CAPACITY + block, + u64::MAX, + ] { + let request = CloneRequest { + memory, + ..tmpfs_request(&image, &binding) + }; + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "a clone of the template's generation {generation} cannot restore with \ + {memory} bytes of RAM: its memory target must be whole blocks of 128 MiB, \ + and of the template's memory blocks of {block} bytes, from the template's \ + {MEMORY} bytes of RAM to its capacity of {MEMORY_CAPACITY} bytes" + ) + ) + ); + } + assert_untouched(&acquired); + for memory in [MEMORY, MEMORY + block, MEMORY_CAPACITY] { + let request = CloneRequest { + memory, + ..tmpfs_request(&image, &binding) + }; + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!(plan.memory(), memory); + } +} + +#[test] +fn requests_that_no_clone_takes_are_refused_before_anything_is_opened() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let (scratch, scratch_binding) = fixture.scratch("scratch.ext4", SCRATCH_LEN); + let mut acquired = fixture.acquire("sandbox"); + let mut refusal = |request: &CloneRequest| { + let (kind, message) = failure(fixture.plan(&mut acquired, request, NO_CONTENT)); + assert_eq!(kind, ErrorKind::Unsupported, "{message}"); + message + }; + let tmpfs = tmpfs_request(&image, &binding); + let distro = tmpfs.adapter.distro().clone(); + // A plan of the sandbox profile. + let profile = CloneRequest { + adapter: SandboxPlan::new( + distro.clone(), + SandboxUpper::RAM, + SandboxSettings::default(), + ) + .unwrap(), + ..tmpfs.clone() + }; + let message = refusal(&profile); + assert!( + message.starts_with("a clone restores an adapter's plan, not a plan of the sandbox"), + "{message}" + ); + // An image that is not the one that the binding names. + let other = Uuid::parse("22222222-2222-2222-2222-222222222222").unwrap(); + let mistaken = CloneRequest { + adapter: SandboxPlan::adapter( + SandboxDistro::new(image.to_str().unwrap(), RootImage::erofs(Some(other))), + SandboxUpper::RAM, + ), + ..tmpfs.clone() + }; + assert_eq!( + refusal(&mistaken), + format!( + "the image {} is not the erofs layer with identity {UUID} that its distro binding \ + names", + shown(&image) + ) + ); + // Paths that OpenVMM would not open as the plan checks them. + let at = |path: &str| CloneRequest { + adapter: SandboxPlan::adapter(SandboxDistro::new(path, distro.image()), SandboxUpper::RAM), + ..tmpfs.clone() + }; + let message = refusal(&at("distro.erofs")); + assert!( + message.starts_with("the image of a clone must be absolute"), + "{message}" + ); + let message = refusal(&at(&format!("{},x", image.to_str().unwrap()))); + assert!( + message.starts_with("disk paths containing commas or semicolons"), + "{message}" + ); + let message = refusal(&at(image.with_extension("vhdx").to_str().unwrap())); + assert!( + message.starts_with("disk paths ending in .vhdx"), + "{message}" + ); + let on_scratch = |path: &str| CloneRequest { + adapter: SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch(path)), + scratch: Some(scratch_binding.clone()), + ..tmpfs.clone() + }; + let message = refusal(&on_scratch("scratch.ext4")); + assert!( + message.starts_with("the scratch image of a clone must be absolute"), + "{message}" + ); + let message = refusal(&on_scratch(scratch.with_extension("iso").to_str().unwrap())); + assert!( + message.starts_with("disk paths ending in .iso"), + "{message}" + ); + // An upper on a scratch image needs its binding, and one in RAM takes none. + let unbound = CloneRequest { + scratch: None, + ..scratch_request(&image, &binding, &scratch, &scratch_binding) + }; + assert_eq!( + refusal(&unbound), + format!( + "the upper of a clone on the scratch image {} needs the image's scratch binding", + shown(&scratch) + ) + ); + let bound = CloneRequest { + scratch: Some(scratch_binding.clone()), + ..tmpfs.clone() + }; + assert_eq!( + refusal(&bound), + "the upper of a clone in RAM takes no scratch binding" + ); + // None of them opened anything: the scratch image is free, and the lease still reserves the + // clone's launch, which a plan of a request that a clone takes then serves. + drop(scratch_binding.check(&scratch, Instant::now()).unwrap()); + assert_untouched(&acquired); + drop(fixture.plan(&mut acquired, &tmpfs, NO_CONTENT).unwrap()); +} + +#[test] +fn networks_that_the_template_does_not_have_are_refused() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let generation = fixture.generation.generation(); + let attached = with_network(request.clone(), Some(network(NetworkAccess::Allow))); + assert_eq!( + failure(fixture.plan(&mut acquired, &attached, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the adapter's plan of a clone attaches a network, but the template's generation \ + {generation} has no NIC" + ) + ) + ); + assert_untouched(&acquired); + // A network that lets nothing through attaches no NIC. + let closed = with_network(request, Some(network(NetworkAccess::Deny))); + drop(fixture.plan(&mut acquired, &closed, NO_CONTENT).unwrap()); + + // A template with a NIC serves the clones that attach its network. + let fixture = Fixture::with(|class, spec, physical| { + *class = network_class(); + *spec = ManifestSpec::with_network(); + let (distro, scratch) = ( + geometry(IMAGE_LEN, physical), + geometry(SCRATCH_LEN, physical), + ); + set_geometries(class, spec, distro, scratch); + }); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let generation = fixture.generation.generation(); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unsupported, + format!( + "the template's generation {generation} has a NIC, which its clones restore with \ + the network of the adapter's plan, but the plan has none" + ) + ) + ); + assert_untouched(&acquired); + let attached = with_network(request, Some(network(NetworkAccess::Allow))); + drop(fixture.plan(&mut acquired, &attached, NO_CONTENT).unwrap()); +} + +#[test] +fn clones_are_planned_under_the_lock_that_acquired_their_lease_before_their_launch() { + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let request = tmpfs_request(&image, &binding); + // A lease of another store. + let other = Fixture::new(); + let mut elsewhere = other.acquire("sandbox"); + let (kind, message) = failure(fixture.plan(&mut elsewhere, &request, NO_CONTENT)); + assert_eq!(kind, ErrorKind::Other, "{message}"); + assert_eq!( + message, + format!( + "the lease {} is one of the template store {}, not of this one, {}", + elsewhere.lease.id(), + other.store.id(), + fixture.store.id() + ) + ); + assert_untouched(&elsewhere); + // Another lock of the sandbox, which reserves no launch, as one taken once the lock that + // acquired the lease is gone. + let Acquired { lease, locked } = fixture.acquire("sandbox"); + drop(locked); + let locked = SandboxStore::lock_existing(lease.sandbox(), &files(), None).unwrap(); + let mut relocked = Acquired { lease, locked }; + let (kind, message) = failure(fixture.plan(&mut relocked, &request, NO_CONTENT)); + assert_eq!(kind, ErrorKind::Other, "{message}"); + assert!( + message.starts_with("a clone is planned under the lock that acquired its lease"), + "{message}" + ); + assert!(!sparse_path(&relocked).exists()); + // The lock that acquired a lease, once the launch that the lease reserves has begun. + let mut acquired = fixture.acquire("begun"); + let key = fixture.store.read_key(&fixture.generation).unwrap(); + drop( + acquired + .locked + .begin_launch(acquired.lease.launch(), key.runtime(), &[7; CAPABILITY_LEN]) + .unwrap(), + ); + let (kind, message) = failure(fixture.plan(&mut acquired, &request, NO_CONTENT)); + assert_eq!(kind, ErrorKind::Other, "{message}"); + assert!(!sparse_path(&acquired).exists()); +} + +#[test] +fn files_that_their_paths_do_not_name_as_checked_are_refused() { + let fixture = Fixture::new(); + let mut acquired = fixture.acquire("sandbox"); + // An image whose path is a link, which OpenVMM's open does not follow. + let (target, _) = fixture.image("target.erofs", IMAGE_LEN); + let link = fixture.path("images", "link.erofs"); + if symlink_file(&target, &link) { + let binding = DistroBinding::import(&layer(&link), PolicyVerdict::ACCEPTED).unwrap(); + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&link, &binding), NO_CONTENT)), + ( + ErrorKind::Unavailable, + format!( + "the image {} cannot be used: it is a link, or not a regular file", + shown(&link) + ) + ) + ); + // A scratch image whose path is a link. + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let (target, _) = fixture.scratch("target.ext4", SCRATCH_LEN); + let link = fixture.path("scratches", "link.ext4"); + assert!(symlink_file(&target, &link)); + let scratch_binding = ScratchBinding::bind(&link).unwrap(); + let request = scratch_request(&image, &binding, &link, &scratch_binding); + assert_eq!( + failure(fixture.plan(&mut acquired, &request, NO_CONTENT)), + ( + ErrorKind::Unavailable, + format!( + "the scratch image {} cannot be used: it is a link, or not a regular file", + shown(&link) + ) + ) + ); + // The refused plan holds no claim of the image. + drop( + scratch_binding + .check(&link, Instant::now() + CLAIM_RELEASE_WAIT) + .unwrap(), + ); + } + // An image that others may change. + let path = fixture.path("images", "shared.erofs"); + File::create(&path).unwrap().set_len(IMAGE_LEN).unwrap(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + + fs::set_permissions(&path, fs::Permissions::from_mode(0o666)).unwrap(); + } + #[cfg(windows)] + platform::set_access(&path, "D:P(A;;FA;;;OW)(A;;FA;;;SY)(A;;FA;;;WD)").unwrap(); + let binding = DistroBinding::import(&layer(&path), PolicyVerdict::ACCEPTED).unwrap(); + let problem = if cfg!(windows) { + "allows other users to change it" + } else { + "is writable by other users (mode 666)" + }; + assert_eq!( + failure(fixture.plan(&mut acquired, &tmpfs_request(&path, &binding), NO_CONTENT)), + ( + ErrorKind::Unavailable, + format!("the image {} {problem}", shown(&path)) + ) + ); + assert_untouched(&acquired); +} diff --git a/nvx-rs/src/host/template/manifest.rs b/nvx-rs/src/host/template/manifest.rs index cd84c5b54..cbaf7f091 100644 --- a/nvx-rs/src/host/template/manifest.rs +++ b/nvx-rs/src/host/template/manifest.rs @@ -18,6 +18,7 @@ use super::wire::{Field, Message, WireError}; use crate::contract; use crate::error::{Error, ErrorCode, Result}; use crate::host::process::BootId; +pub use crate::host::sandbox::geometry::BlockGeometry; use crate::host::store::regular; use crate::hypervisor::Hypervisor; @@ -434,18 +435,6 @@ pub(super) struct Interrupts { pub(super) status_page_length: u64, } -/// The length and block sizes of a sandbox block device. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] -pub struct BlockGeometry { - /// The length that the device presents, in bytes: nonzero, and a multiple of the logical - /// block size. - pub length: u64, - /// The logical block size, in bytes: a power of two of at least 512. - pub logical_block_size: u32, - /// The physical block size, in bytes: a power of two of at least the logical block size. - pub physical_block_size: u32, -} - impl BlockGeometry { /// Explains why the geometry is invalid, if it is, as a phrase whose subject is a size, such /// as "length 1000 is not a nonzero multiple of its logical block size". diff --git a/nvx-rs/src/host/template/mod.rs b/nvx-rs/src/host/template/mod.rs index 4f762b4a2..f0c8ae886 100644 --- a/nvx-rs/src/host/template/mod.rs +++ b/nvx-rs/src/host/template/mod.rs @@ -31,6 +31,12 @@ //! sandbox, until [`Store::release`] ends the lease once the clone's VM has exited. A tombstone //! quarantines a generation, which takes no new lease, and [`Store::retire`] deletes it once its //! leases have drained. +//! - [`ClonePlan::new`] plans the clone of a leased generation, as a [`CloneRequest`] asks, under +//! the lock that acquired the lease, before the clone's launch: it checks the clone's image +//! against its binding, its upper, in RAM or on the caller's scratch image, which it claims, and +//! its memory target against the template, and refuses a request that the template cannot +//! serve, such as an image of another length or block geometry than the template's placeholder, +//! before any VMM starts. //! //! # Canonical encoding //! @@ -52,6 +58,7 @@ //! `tests/vectors/template-key-v1.txt` fix version 1 of a key's encoding. mod canonical; +mod clone; mod eligibility; mod key; mod manifest; @@ -61,6 +68,7 @@ mod store; mod test_support; mod wire; +pub use self::clone::{ClonePlan, CloneRequest, CloneUpper}; pub use self::eligibility::{Downtime, DowntimeSource, Eligibility, HostTime, MAX_DOWNTIME_NS}; pub use self::key::{CaptureClass, ClassIdentity, Invariants, KeyIdentity, TemplateKey}; pub use self::manifest::{ diff --git a/nvx-rs/src/host/template/store.rs b/nvx-rs/src/host/template/store.rs index 4cf5bec64..8dd9cf4dd 100644 --- a/nvx-rs/src/host/template/store.rs +++ b/nvx-rs/src/host/template/store.rs @@ -496,6 +496,18 @@ impl Store { tests::storage(1), ) } + + /// Publishes a capture of `class` whose manifest `spec` encodes, as + /// [`publish_for_tests`](Self::publish_for_tests) does, for the tests of the other modules of + /// templates, and returns it. + #[cfg(any(target_os = "linux", windows))] + pub(super) fn publish_spec_for_tests( + &self, + class: &super::CaptureClass, + spec: &super::test_support::ManifestSpec, + ) -> Generation { + tests::publish_with(self, class, spec, tests::storage(1)) + } } impl Store { diff --git a/nvx-rs/src/host/template/store/capture.rs b/nvx-rs/src/host/template/store/capture.rs index 3c1795501..204ecb438 100644 --- a/nvx-rs/src/host/template/store/capture.rs +++ b/nvx-rs/src/host/template/store/capture.rs @@ -26,9 +26,8 @@ use crate::host::instance::generate_capability; use crate::host::platform; use crate::host::sandbox::distro::{CheckedDistro, DistroBinding, DistroFormat}; use crate::host::sandbox::{LayerRole, SandboxLayer, SandboxPlan}; -use crate::host::seal::FileSeal; use crate::host::store::files::{create_new, remove_if_present, sync_dir, write_private}; -use crate::host::store::regular::{open_regular, read_at_most}; +use crate::host::store::regular::{identity_of, open_regular, read_at_most}; use crate::host::store::{ JOURNAL_NAME, LOCK_NAME, LOG_NAME, LaunchId, LockedStore, OUTCOME_NAME, Recovery, SOCKET_NAME, SandboxStore, @@ -1017,14 +1016,6 @@ fn create_scratch(path: &Path, length: u64) -> Result<(u64, u128)> { identity_of(&file).map_err(failed) } -/// Returns the identity of the file that `file` holds open, as its [`FileSeal`] records it: its -/// device or volume, and its file ID, the 128-bit one of `FILE_ID_INFO` on Windows, which a file -/// system such as ReFS keeps unique where the 64-bit file index is not. -fn identity_of(file: &fs::File) -> io::Result<(u64, u128)> { - let seal = FileSeal::of(file)?; - Ok((seal.device(), seal.file_id())) -} - /// Checks the files of the capture of `template`, with the inputs `capture`, before its VM /// starts: its transaction directory, its placeholder, and its scratch, which the capture /// created with the identity `scratch`. Returns the placeholder that the check of its binding diff --git a/nvx-rs/src/host/template/store/leases.rs b/nvx-rs/src/host/template/store/leases.rs index 64059689d..87672428b 100644 --- a/nvx-rs/src/host/template/store/leases.rs +++ b/nvx-rs/src/host/template/store/leases.rs @@ -319,10 +319,12 @@ impl Store { /// run. The sandbox's store clears those records only once it has verified that the VM's /// process exited, through its recorded [`ProcessRef`](crate::host::process::ProcessRef), as /// a stop or a [`reattach`](crate::host::supervisor::reattach) does. The release removes the - /// clone binding, and once its removal is durable, the lease's record in the store, so that an - /// interrupted release leaves a record that no binding names, which a - /// [retirement](Self::retire) removes, and never a binding without its record. It takes no - /// lock of the store. + /// file of the scratch device of a clone whose upper is in RAM, + /// [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME), which the clone's + /// [plan](crate::host::template::ClonePlan) created, then the clone binding, and once their + /// removal is durable, the lease's record in the store, so that an interrupted release leaves + /// a record that no binding names, which a [retirement](Self::retire) removes, and never a + /// binding without its record. It takes no lock of the store. /// /// The caller takes the sandbox's lock through the lease's path, as [`Lease::sandbox`] /// returns it, and as [`acquire`](Self::acquire) takes it, which has no links: a link in the @@ -361,6 +363,10 @@ impl Store { ))); } let recorded = self.check_records_dir(LEASES_NAME, LEASES_WHAT)?; + // The clone's VM has exited, and the sandbox launches nothing else under the lease, so no + // VM holds the scratch device's file of a RAM upper. The binding's removal makes the + // file's durable too, since both are entries of the sandbox's directory. + remove_if_present(&locked.clone_scratch_path())?; unbind(locked.dir(), sync)?; locked.reserve(None); if recorded { diff --git a/nvx-rs/src/host/template/store/leases/tests.rs b/nvx-rs/src/host/template/store/leases/tests.rs index 7996b2dd7..1c8c26d9e 100644 --- a/nvx-rs/src/host/template/store/leases/tests.rs +++ b/nvx-rs/src/host/template/store/leases/tests.rs @@ -17,7 +17,9 @@ use super::*; use crate::activation::TemplateReference; use crate::host::process::{Process, ProcessRef}; use crate::host::store::files::write_private; -use crate::host::store::{FORMAT, LaunchState, RUNTIME_NAME, Recovery, RuntimeIdentity}; +use crate::host::store::{ + CLONE_SCRATCH_NAME, FORMAT, LaunchState, RUNTIME_NAME, Recovery, RuntimeIdentity, +}; use crate::host::supervisor::{self, Reattached, StopOutcome, VmmCommand}; use crate::host::template::eligibility::MAX_DOWNTIME_NS; use crate::host::template::store::tests::{ @@ -784,19 +786,22 @@ fn a_release_makes_the_removal_of_its_binding_durable_before_it_removes_its_reco .dir() .join(LEASES_NAME) .join(format!("{}.json", lease.id())); - // Each sync notes its directory, and whether the binding and the record existed then. + // Each sync notes its directory, and whether the scratch device's file of a clone in RAM, + // the binding, and the record existed then. let synced = RefCell::new(Vec::new()); let sync = |dir: &Path| -> Result<()> { + let scratch = sandbox.join(CLONE_SCRATCH_NAME).exists(); let bound = sandbox.join(CLONE_BINDING_NAME).exists(); synced .borrow_mut() - .push((dir.to_owned(), bound, record.exists())); + .push((dir.to_owned(), scratch, bound, record.exists())); Ok(()) }; + fs::write(sandbox.join(CLONE_SCRATCH_NAME), b"").unwrap(); let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); assert_eq!(store.release_with(&mut locked, &sync).unwrap(), Some(lease)); drop(locked); - assert_eq!(*synced.borrow(), [(sandbox.clone(), false, true)]); + assert_eq!(*synced.borrow(), [(sandbox.clone(), false, false, true)]); assert!(!record.exists()); // A release that cannot make the binding's removal durable keeps the record, which a // retirement removes once no binding names it.