diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 0843d1ae0..600f8ad05 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -94,7 +94,7 @@ nvx-rs/ | `nvx::host::nvxs` | `HostConnection`: the host's role of the NVXS session over a platform transport, on which `nvx::host::session` and other host applications run their protocols. It attaches with the launch capability, waiting while the broker asks it to, sends application bytes in as many DATA records as they take, and returns each DATA payload from the guest, with deadlines; a receive that times out keeps a partial record | | `nvx::host::process` | Verified process references. A `ProcessRef` names one process by its ID, its start time, and the `BootId` of the host boot during which it started. `Process::open` checks a reference and holds its process through a pidfd on Linux or a process handle on Windows, whether or not the caller started it, so that waits sleep until the process exits and termination never reaches a process that reuses its ID. A reference from an earlier boot names an exited process, and a process whose identity this user cannot check, such as another user's, makes `Process::open` fail rather than report an exit. Linux hosts without pidfds poll the reference instead, and cannot terminate the process | | `nvx::host::supervisor` | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock, as a launch whose `LaunchId` the caller reserved beforehand, a fresh one for each launch, or that a clone's lease reserves: it writes the journal's marker, which names the launch, spawns OpenVMM with the claimed log, records its `ProcessRef` right after, and returns a `Starting` launch that the caller commits once the VMM serves its control endpoint, or abandons. A launch that fails never leaves an unrecorded VMM running. `VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that the VMM inherits under their numbers, besides its standard streams, and it inherits nothing else of the caller's: on Linux, they must be close-on-exec, a flag that only the VMM's own process clears; on Windows, they join the VMM's explicit handle list. `Vmm::stop` runs one graceful step of the caller's and waits for the VMM's exit event until a deadline, and takes a bounded forced step through the VMM's verified identity only on request. `Vmm::terminate` takes that forced step alone, within the caller's bound, for a VM whose guest the caller must not ask anything, such as one of another runtime identity. `reattach` recovers an interrupted launch, and holds the recorded VMM again after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate`. A clone's launch takes its locks in one order, the template store's, then the sandbox's, which guards its clone binding and launch journal, then the scratch image's claim, and holds none for the VM's lifetime; its lease lasts through crashes and reattaches, until a release ends it once a stop, a termination, or a reattach has verified the VM's exit | -| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; a network exactly when the template has a NIC; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and which nothing formats, copies, or reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the sandbox's directory, which the release removes; its export, if it has one, an `ExportPlan` that `filesystem::plan` made for the container root, which must export exactly one directory, since the template's dormant filesystem slot binds one directory and no file, and which must not overlap the template store, by the paths that the plan resolved; and a fresh capability. It builds the clone's activation record, `ClonePlan::activation`, which the clone's guest applies in place of the configuration that the template captured, from the request's `WorkloadView`, overlay mode, journal policy, and bound of a GPT disk's first partition, and from the checked inputs: a restore section with a fresh transaction ID of 16 random bytes and the lease's `TemplateReference`; the image, an EROFS layer or a GPT disk, as its binding names it, with its binding's identity and the filesystem UUID or disk GUID that the adapter's plan expects of it, and for a GPT disk the bound of its first partition, whose partition table the guest checks with `nvx::gpt` as at a cold launch; a tmpfs, or the scratch image's binding identity with the journal policy; the export, a directory, with its plan's identity, and its mappings, the plan's binds, from 1 to 4,096, or no export and no mappings, which leaves the slot dormant; the template NIC's policy identity, as its key records it, unpublished listeners, and the guest's proxy URL; and the template's processors and the memory target. The identity of a binding or of an export plan is the SHA-256 digest of a domain and of its serde form, never of a file's content. The image and the scratch device must present the `BlockGeometry` of the template's, which a restore compares, and be, at the paths by which OpenVMM opens them, the regular files that the checks found, which are not links or other reparse points, and which no one else may change. Neither the image nor a scratch image may be a file that the sandbox's store owns, such as `clone-scratch.raw`, its log, or its records, by any path, since the store's transitions replace or remove them, and a link or a directory under one of the store's names is refused. `ClonePlan::launch` then launches the clone as the launch that its lease reserves, once it has checked that the sandbox's clone binding records the lease, that the sandbox records no failure of the clone, which fails once under its lease, and, with `ExportPlan::verify`, that the objects of its export, if it has one, are still those that its plan found, and returns a `Restoring`: it writes the clone's activation record in its canonical encoding, `clone-activation.bin`, from which a caller resends it unchanged; it listens for the VMM's restore-ready event, at the socket `restore.sock` on Linux, and at a named pipe of a random name on Windows; and it runs `supervisor::launch` with OpenVMM's restore of the clone's devices, its export bound to the dormant slot, `--mount` with the directory, and its hidden, re-exposed, and writable paths and its owner, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, `--restore-memory` with the memory target, and `--restore-ready-path`, under the template's backend, CPU profile, and processors, with a boot console that discards its output, as cold boots have, and the control console, but no kernel, initramfs, command line, or guest address, which the snapshot fixes. On Linux, where OpenVMM requires a restore's control socket in the snapshot's parent, the clone's VMM listens in the generation's payload, at `.sock`, named after the launch that the lease reserves, whose path must be shorter than 108 bytes with or without the payload's links resolved; on Windows, it listens at a named pipe, as every launch does. The VMM inherits the claim of a scratch image, on Windows through the image's guard, once the launch has closed the claim's handle, which OpenVMM's open of the image would refuse. A payload holds the sockets of its clones only while they lease it: the VMM removes its socket as it exits, a stop or a recovery removes the one that an abrupt exit leaves, and the release makes the socket's absence durable before the lease ends, and removes the record and `restore.sock`, as it removes `clone-scratch.raw`. `Restoring::activate` then runs the clone's restore sequence through the adapter's framing of its control session, a `CloneControl`, by a restore-to-Ready deadline that runs from the launch, `RESTORE_TO_READY_TIMEOUT`, 60 seconds by default: it reads the restore-ready event as soon as it comes, refusing another process's connection and other bytes, since OpenVMM on Windows waits until the host has read it; only then connects to the clone's control endpoint, whose console bounds the host's authentication from the connection on; attaches with the launch's capability; sends the record, read again from `clone-activation.bin`, which must still be byte for byte the launch's, in the adapter's Activate, as the session's first request; and checks that Ready reports the record's configuration identity and transaction ID. It replaces a session that the adapter lost, as `CloneControl::lost` tells, with a new one that sends the same record, in at most `MAX_ACTIVATION_SESSIONS` sessions by the same deadline, and retries nothing else. `Activated::start` then sends the clone's mapping table, if it has an export, through `CloneControl::map`, in order, from paths in the exported directory, which name no child's index, with an empty source for the directory itself, and starts the sandbox once the guest applied every mapping, both by a Ready-to-Start deadline from the check of Ready, `READY_TO_START_TIMEOUT`, 30 seconds by default, never sending either again, and `Started::commit` commits the launch, which writes its runtime record. A failure, a partial or failed mapping table included, ends the launch as `Starting::abandon` does; a crash before the commit leaves the launch's journal, whose recovery ends the VMM, while the lease lasts until the release, and a crash after it leaves a VM that reattaches, and keeps its lease while it runs. A failure of a clone is a `CloneFailure`, whose `Evidence` attributes it to an `Outcome`, at a `Stage`, once the clone got as far as its `Progress`: the request, for a denied authorization, a stale binding, invalid targets, format, settings, Activate, or mappings, such as an export that no clone takes, or whose objects changed since its plan, or a mapping table that the guest refused, as `CloneControl::refused` tells, or an image, a scratch, or a root that the template cannot serve, and for a supplied file of the wrong identity, length, or geometry, which the host's staging takes instead once the caller proves it, `CloneFailure::with_staging_proven`; the host, for an endpoint that cannot be bound, resource exhaustion, a host that cannot restore the template, another runtime, or a failure of its own operations; unknown, for a lost status, a timeout, a reset, a forced kill, or a broken protocol; the template, for a malformed generation, or a defect of its capture or repair that a diagnosis attributes to it, the only outcome that quarantines the generation, through its tombstone, `CloneFailure::quarantine`; and the repair stage, for a status of the clone's repair alone. OpenVMM's outcome report gives the status with which the guest ended the clone's VM, which `status_evidence` attributes, and which prevails over what the host or the caller saw, but not over evidence of the template, which shows the provenance that a status alone leaves to establish: a status of the repair at any point, and one of the activation before the start was sent. A VMM that exits before its restore-ready event, as OpenVMM does once it cannot restore the snapshot, after it closed the event's endpoint, which the sequence then gives ten seconds to exit, leaves the line with which OpenVMM reported its fatal error in its log, which the failure's message keeps, and whose time ABI code, if it says that the host cannot present the template's CPU profile, TSC, or LAPIC timer, attributes the failure to the host. From the launch on, a failure ends the clone's VMM through its verified identity, and the sandbox keeps the clone's progress log, `clone-progress.log`, which records each step from the spawn to the commit, after a first line that names the launch and the boot of the host, a few lines that it appends without a sync, and the record of its failure, `clone-failure.json`, JSON in format `FAILURE_FORMAT`, which it writes before the teardown, with the teardown pending, so that a crash of the caller during the teardown leaves it for a reattach to finish, and again once the teardown ended, with the stage, the evidence and the status, the original outcome, the transaction, the generation, the progress, and the teardown: no VM, an exit or a termination that the host verified, or an unresolved one, which keeps the launch's journal or runtime record, and the lease; a release removes both, and a sandbox that boots cold writes neither. A step or a check of the caller's own that fails once the clone is ready, or once its sandbox started, fails the clone with the caller's evidence through `Activated::fail` or `Started::fail`, which end the launch and record the failure as the sequence's own failures do. `reattach_clone` takes a clone over after the caller of its launch restarted: it reattaches by the identity of its VMM's process, checks a committed clone in a new session, in which the clone answers the byte-equal Activate of `clone-activation.bin` with the Ready that it answered once, and ends a clone that another runtime identity launched, or that fails the check, through its verified identity; it records the failure of a launch that a crash interrupted, but not the journal that a commit may leave beside its runtime record, classifies a VM that exited by its outcome report, records a failure before it ends a VMM or clears the runtime record of one that exited, finishes a failure that a crash left pending as its teardown would have recorded it, keeping its transaction, resolves the teardown of an earlier failure with what the VMM left as it ended, its outcome report and OpenVMM's fatal error, attributes a status of the guest's as one that a workload may have given once the clone's progress log is lost, damaged, or was written during another boot of the host, whose crash may have lost its last lines, and reads nothing of the template store, so that a tombstone or a retirement of the generation leaves the clone its lease. `CloneRecords::read` reads what the sandbox keeps of a clone, for the operations that list, inspect, or remove clones. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | +| `nvx::host::template` | The metadata of templates, captures of a microVM from which independent clones are restored. `TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access both ways; `TemplateResources`, with every processor online at boot, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, so that the capture's whole command line holds tokens of OpenVMM's platform allowlist alone, and no tenant data, which the tests check with a copy of the allowlist's source; and the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the transaction directory, and `--snapshot-block-identity generation` with the capture's bound-block `StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link or other reparse point, and refuses another snapshot format version, an unknown, mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors and RAM, the geometries of the disks, the network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, and the backend, CPU profile and effective CPUID, topology, rates, resources, kernel command line, disks, and devices of its machine. A capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin the key's encoding and identity. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private directory with an identity of its own, a `StoreId`, the immutable payload of each generation, its key's canonical encoding beside the snapshot that OpenVMM published, and the versioned `Index` of them, which lists each generation with the identities of its key and capture class and its own metadata, such as its time contract. `Store::create` creates a store, or opens the one that its directory holds, crash-safely and once between concurrent creations, and refuses a directory with foreign entries; `Store::open` opens one. Its records are JSON in one format, replaced atomically, and records of another format fail rather than being migrated. Its directories must be private to the current user, and it refuses links and, on Windows, other reparse points among its directories and files. Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's publication ID, which the index assigns in the order of publication and never twice, and by which a clone's activation record names its template; the bound-block `StorageGeneration` of a capture, which belongs to the storage owner; the time lineage counter of the time contract; and the VM generation ID, which the store never records. `TimeContract::eligibility` decides, apart from compatibility, whether a clone may restore a capture at a host's time, a `HostTime`, which `HostTime::now` samples as OpenVMM does: within the boot of the capture, by the host's monotonic clock, and otherwise by UTC, the downtime must not be negative, and the interval by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 days. `Store::select` selects the latest eligible generation of a key that no tombstone quarantines. `Store::acquire` leases a generation to a clone in one transaction, under the store's lock and then the sandbox's: it checks the generation's eligibility, writes the lease's record in the store and the clone binding, `clone.json`, beside the sandbox's launch journal, and returns the lease with the sandbox's lock, the only one under which the caller launches the clone, once, as the launch that the lease reserves, `Lease::launch`. A lease never expires: `Store::release` ends it once the sandbox records no VM and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; a network exactly when the template has a NIC; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and which nothing formats, copies, or reads, or for an upper in RAM a fresh sparse file, `clone-scratch.raw` in the sandbox's directory, which the release removes; its export, if it has one, an `ExportPlan` that `filesystem::plan` made for the container root, which must export exactly one directory, since the template's dormant filesystem slot binds one directory and no file, and which must neither overlap the template store, by the paths that the plan and the store's directory resolve to, nor let the guest write in a directory through which the store's path leads, links included, since the host reaches the store, and OpenVMM its snapshots, by that path, nor be or hold the directory of the clone's sandbox, in which the host writes the clone's records, checks of paths that leave two rules to whoever names the export, as they leave the directories above the store: the export must not reach the store through an alias of it that an administrator configures, such as a bind mount or a mount point of the store, or of a directory above or inside it, a loopback share, or a hard link of a payload, nor lie inside a directory that a writable export of another sandbox or clone holds, since OpenVMM resolves an export's path again at each launch and restore, so that the directories above the export's directory must be ones that no guest may change, as those above a clone's block devices must; and a fresh capability. It builds the clone's activation record, `ClonePlan::activation`, which the clone's guest applies in place of the configuration that the template captured, from the request's `WorkloadView`, overlay mode, journal policy, and bound of a GPT disk's first partition, and from the checked inputs: a restore section with a fresh transaction ID of 16 random bytes and the lease's `TemplateReference`; the image, an EROFS layer or a GPT disk, as its binding names it, with its binding's identity and the filesystem UUID or disk GUID that the adapter's plan expects of it, and for a GPT disk the bound of its first partition, whose partition table the guest checks with `nvx::gpt` as at a cold launch; a tmpfs, or the scratch image's binding identity with the journal policy; the export, a directory, with its plan's identity, and its mappings, the plan's binds, from 1 to 4,096, or no export and no mappings, which leaves the slot dormant; the template NIC's policy identity, as its key records it, unpublished listeners, and the guest's proxy URL; and the template's processors and the memory target. The identity of a binding or of an export plan is the SHA-256 digest of a domain and of its serde form, never of a file's content. The image and the scratch device must present the `BlockGeometry` of the template's, which a restore compares, and be, at the paths by which OpenVMM opens them, the regular files that the checks found, which are not links or other reparse points, and which no one else may change. Neither the image nor a scratch image may be a file that the sandbox's store owns, such as `clone-scratch.raw`, its log, or its records, by any path, since the store's transitions replace or remove them, and a link or a directory under one of the store's names is refused. `ClonePlan::launch` then launches the clone as the launch that its lease reserves, once it has checked that the sandbox's clone binding records the lease, that the sandbox records no failure of the clone, which fails once under its lease, and, with `ExportPlan::verify`, that the objects of its export, if it has one, are still those that its plan found, and returns a `Restoring`: it writes the clone's activation record in its canonical encoding, `clone-activation.bin`, from which a caller resends it unchanged; it listens for the VMM's restore-ready event, at the socket `restore.sock` on Linux, and at a named pipe of a random name on Windows; and it runs `supervisor::launch` with OpenVMM's restore of the clone's devices, its export bound to the dormant slot, `--mount` with the directory, and its hidden, re-exposed, and writable paths and its owner, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, `--restore-memory` with the memory target, and `--restore-ready-path`, under the template's backend, CPU profile, and processors, with a boot console that discards its output, as cold boots have, and the control console, but no kernel, initramfs, command line, or guest address, which the snapshot fixes. On Linux, where OpenVMM requires a restore's control socket in the snapshot's parent, the clone's VMM listens in the generation's payload, at `.sock`, named after the launch that the lease reserves, whose path must be shorter than 108 bytes with or without the payload's links resolved; on Windows, it listens at a named pipe, as every launch does. The VMM inherits the claim of a scratch image, on Windows through the image's guard, once the launch has closed the claim's handle, which OpenVMM's open of the image would refuse. A payload holds the sockets of its clones only while they lease it: the VMM removes its socket as it exits, a stop or a recovery removes the one that an abrupt exit leaves, and the release makes the socket's absence durable before the lease ends, and removes the record and `restore.sock`, as it removes `clone-scratch.raw`. `Restoring::activate` then runs the clone's restore sequence through the adapter's framing of its control session, a `CloneControl`, by a restore-to-Ready deadline that runs from the launch, `RESTORE_TO_READY_TIMEOUT`, 60 seconds by default: it reads the restore-ready event as soon as it comes, refusing another process's connection and other bytes, since OpenVMM on Windows waits until the host has read it; only then connects to the clone's control endpoint, whose console bounds the host's authentication from the connection on; attaches with the launch's capability; sends the record, read again from `clone-activation.bin`, which must still be byte for byte the launch's, in the adapter's Activate, as the session's first request; and checks that Ready reports the record's configuration identity and transaction ID. It replaces a session that the adapter lost, as `CloneControl::lost` tells, with a new one that sends the same record, in at most `MAX_ACTIVATION_SESSIONS` sessions by the same deadline, and retries nothing else. `Activated::start` then sends the clone's mapping table, if it has an export, through `CloneControl::map`, in order, from paths in the exported directory, which name no child's index, with an empty source for the directory itself, and starts the sandbox once the guest applied every mapping, both by a Ready-to-Start deadline from the check of Ready, `READY_TO_START_TIMEOUT`, 30 seconds by default, never sending either again, and `Started::commit` commits the launch, which writes its runtime record. A failure, a partial or failed mapping table included, ends the launch as `Starting::abandon` does; a crash before the commit leaves the launch's journal, whose recovery ends the VMM, while the lease lasts until the release, and a crash after it leaves a VM that reattaches, and keeps its lease while it runs. A failure of a clone is a `CloneFailure`, whose `Evidence` attributes it to an `Outcome`, at a `Stage`, once the clone got as far as its `Progress`: the request, for a denied authorization, a stale binding, invalid targets, format, settings, Activate, or mappings, such as an export that no clone takes, or whose objects changed since its plan, or a mapping table that the guest refused, as `CloneControl::refused` tells, or an image, a scratch, or a root that the template cannot serve, and for a supplied file of the wrong identity, length, or geometry, which the host's staging takes instead once the caller proves it, `CloneFailure::with_staging_proven`; the host, for an endpoint that cannot be bound, resource exhaustion, a host that cannot restore the template, another runtime, or a failure of its own operations; unknown, for a lost status, a timeout, a reset, a forced kill, or a broken protocol; the template, for a malformed generation, or a defect of its capture or repair that a diagnosis attributes to it, the only outcome that quarantines the generation, through its tombstone, `CloneFailure::quarantine`; and the repair stage, for a status of the clone's repair alone. OpenVMM's outcome report gives the status with which the guest ended the clone's VM, which `status_evidence` attributes, and which prevails over what the host or the caller saw, but not over evidence of the template, which shows the provenance that a status alone leaves to establish: a status of the repair at any point, and one of the activation before the start was sent. A VMM that exits before its restore-ready event, as OpenVMM does once it cannot restore the snapshot, after it closed the event's endpoint, which the sequence then gives ten seconds to exit, leaves the line with which OpenVMM reported its fatal error in its log, which the failure's message keeps, and whose time ABI code, if it says that the host cannot present the template's CPU profile, TSC, or LAPIC timer, attributes the failure to the host. From the launch on, a failure ends the clone's VMM through its verified identity, and the sandbox keeps the clone's progress log, `clone-progress.log`, which records each step from the spawn to the commit, after a first line that names the launch and the boot of the host, a few lines that it appends without a sync, and the record of its failure, `clone-failure.json`, JSON in format `FAILURE_FORMAT`, which it writes before the teardown, with the teardown pending, so that a crash of the caller during the teardown leaves it for a reattach to finish, and again once the teardown ended, with the stage, the evidence and the status, the original outcome, the transaction, the generation, the progress, and the teardown: no VM, an exit or a termination that the host verified, or an unresolved one, which keeps the launch's journal or runtime record, and the lease; a release removes both, and a sandbox that boots cold writes neither. A step or a check of the caller's own that fails once the clone is ready, or once its sandbox started, fails the clone with the caller's evidence through `Activated::fail` or `Started::fail`, which end the launch and record the failure as the sequence's own failures do. `reattach_clone` takes a clone over after the caller of its launch restarted: it reattaches by the identity of its VMM's process, checks a committed clone in a new session, in which the clone answers the byte-equal Activate of `clone-activation.bin` with the Ready that it answered once, and ends a clone that another runtime identity launched, or that fails the check, through its verified identity; it records the failure of a launch that a crash interrupted, but not the journal that a commit may leave beside its runtime record, classifies a VM that exited by its outcome report, records a failure before it ends a VMM or clears the runtime record of one that exited, finishes a failure that a crash left pending as its teardown would have recorded it, keeping its transaction, resolves the teardown of an earlier failure with what the VMM left as it ended, its outcome report and OpenVMM's fatal error, attributes a status of the guest's as one that a workload may have given once the clone's progress log is lost, damaged, or was written during another boot of the host, whose crash may have lost its last lines, and reads nothing of the template store, so that a tombstone or a retirement of the generation leaves the clone its lease. `CloneRecords::read` reads what the sandbox keeps of a clone, for the operations that list, inspect, or remove clones. `Store::tombstone` quarantines a generation, which takes no new lease while its leases keep working, and `Store::retire` deletes it once its leases have drained, crash-safely, removing the records of leases that no clone binding names any more. `Store::create_template` serves a `Capture`, an adapter's plan for a capture class with the placeholder's binding and the runtime's artifacts, with a generation of its class that the store holds and that is eligible at the host's time, which it reads once it holds the class's slot, `Created::Existing`, or captures one, `Created::Captured`, while `Store::refresh_template` always captures, and leaves older generations and their leases untouched. A capture runs in its class's slot under `captures/`, a sandbox store whose lock serializes the class's captures, which comes before the store's lock, and which keeps the log and outcome report of the VMM of the class's latest capture: in a private transaction on the store's volume, which is also the sandbox store of its VM, since OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, it creates the dormant scratch, checks the placeholder through its binding, and both block devices at their paths, regular files that are not links or other reparse points, that belong to the user or an administrator, and that no one else may change, and launches the template's VM through `supervisor::launch` from the transaction, holding no handle that could write either file, since OpenVMM opens them itself by path, as it does a restore's. Once the VMM has exited, it checks the outcome report, the snapshot's manifest and artifacts, and the key, and publishes the transaction in two commits, a rename into `generations/` that replaces nothing, then the index. A capture that fails publishes nothing, and `Store::recover_captures` ends the captures that a crash interrupted, publishing those that were checked, and retiring the others | | `nvx::host::seal` | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it: the platform scheme, the device or volume serial number, the file ID, the kind, the length, the modification time, and on Linux the change time, with a versioned serde form. A seal detects accidental change, not tampering; see [Distro bindings](#distro-bindings) | | `nvx::host::session`, `platform`, `report` | The authenticated control session, which carries NVX's control protocol, `NVXC`, one frame per DATA record of a `HostConnection`; Linux and Windows primitives; and OpenVMM's outcome report | | `nvx::host::sandbox` | The launch profile of `nvx.py sandbox`, ported from the one that `scripts/nvx_tools/sandbox.py` held: `SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and resource limits, and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. The plan of an adapter that boots its own guest, `SandboxPlan::adapter`, holds no launch settings and no export plan. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` arguments, with `--microvm-workload-identity` when it has launch settings, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. An adapter's plan renders the caller's tokens before those of its root, and refuses tokens that its root or OpenVMM's shares own. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx.py sandbox` and `nvx.py run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx.py run` shares, and the managed exec environment; see [Python parity](#python-parity). `distro`: the `DistroBinding` of a distro layer, an EROFS layer or a GPT disk, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `CheckedDistro::geometry` and `CheckedScratch::geometry` read, through the checked files, the `BlockGeometry` that OpenVMM presents for them: their length, a nonzero multiple of 512 bytes, logical blocks of 512 bytes, and physical blocks of the I/O block size of their filesystem on Linux, and of 4096 bytes on Windows. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | diff --git a/nvx-rs/src/host/template/clone.rs b/nvx-rs/src/host/template/clone.rs index 2c11ab234..e645c7b02 100644 --- a/nvx-rs/src/host/template/clone.rs +++ b/nvx-rs/src/host/template/clone.rs @@ -26,10 +26,11 @@ pub use self::restore::{ RESTORE_TO_READY_TIMEOUT, RestoreDeadlines, Started, }; +use std::collections::BTreeSet; use std::fmt; use std::fs::{self, File}; use std::io; -use std::path::{Path, PathBuf}; +use std::path::{Component, Path, PathBuf}; use std::time::Instant; use self::export::DirectoryExport; @@ -63,6 +64,10 @@ use crate::protocol::{CAPABILITY_LEN, HostMap}; /// Bytes in a mebibyte, the unit in which messages give the size of memory blocks. const MIB: u64 = 1024 * 1024; +/// The most links that [`path_directories`] follows in a path, more than a Linux host, 40, or a +/// Windows host, 63, follows in one. +const MAX_PATH_LINKS: usize = 64; + /// How messages name the file of the `scratch` block device of a clone whose upper is in RAM. const SPARSE_WHAT: &str = "scratch device of the clone"; @@ -180,10 +185,26 @@ pub enum CloneUpper { /// directory: a template's filesystem slot is dormant, and a restore binds one directory to it, /// which is then the export's root, in which the sources of the plan's binds name their paths /// without the child's index, as the mapping table, [`table`](Self::table), sends them. The -/// directory may neither lie inside the template store's directory nor hold it, by the path -/// that the plan resolved, so that no guest reaches the payload of a generation through it. The -/// launch checks the plan's host objects again, with [`ExportPlan::verify`], before it restores -/// the clone. A request without an export leaves the slot dormant. +/// directory may neither lie inside the template store's directory nor hold it, by the paths +/// that the plan and the store's directory resolve to, so that no guest reaches the payload of +/// a generation through it, nor let the guest write in a directory through which the store's +/// path leads, links and `..` included, since the host reaches the store, and OpenVMM its +/// snapshots, by that path. Nor may it be, or hold, the directory of the clone's sandbox, which +/// the lease names by its resolved path, and in which the host writes the clone's records, +/// which the guest could otherwise change, or turn elsewhere, into a leased snapshot, say, +/// though a directory inside the sandbox's, among the caller's files, may be exported. These +/// checks compare paths, so an alias of the store that an administrator configures is outside +/// them, as the directories above the store are: a bind mount or a mount point of the store, +/// or of a directory above or inside it, a loopback share, or a hard link of a payload. +/// Whoever names an export must not name such an alias, nor a directory that holds one. The +/// launch checks the plan's host objects again, with [`ExportPlan::verify`], before it +/// restores the clone, but OpenVMM resolves an export's path again at each launch and restore, +/// so the directories above the export's directory must be ones that no guest may change, as +/// those above the clone's block devices must, which +/// [Block devices by path](Store#block-devices-by-path) describes: whoever names an export +/// must not name one inside a directory that a writable export of another sandbox or clone +/// holds, whose guest could replace the directory with a link, and turn the export elsewhere, +/// to a leased payload, say. A request without an export leaves the slot dormant. /// - The activation record, which the clone's guest applies in place of the configuration that /// the template captured, names the restore, with a fresh transaction ID, the clone's image, /// upper, export, and settings, the template's network, and the resources that the clone must @@ -338,9 +359,13 @@ impl ClonePlan { /// that acquired the lease, before the launch that it reserves began, /// [`Evidence::HostFailure`]; /// - with [`ErrorKind::Unsupported`] for an export whose directory, by the path that its plan - /// resolved, lies inside the template store's directory, [`Store::dir`], or holds it, which - /// the plan compares without reading anything, and so without resolving the store's own - /// path, [`Evidence::InvalidMappings`]; + /// resolved, lies inside the template store's directory, [`Store::dir`], by the path that + /// it resolves to, or holds it, or is or holds the directory of the clone's sandbox, + /// [`Lease::sandbox`], or that lets the guest write in a directory through which the store's + /// path leads, as the host resolves it, links and `..` included, which the guest could then + /// turn elsewhere, or keep the host from following, [`Evidence::InvalidMappings`], + /// and with [`ErrorKind::Io`] if the store's directory, or a directory through which its + /// path leads, cannot be resolved, the host's evidence; /// - with [`ErrorKind::NotFound`] if the store no longer indexes the leased generation, and as /// [`Store::index`] fails, the host's evidence; with [`ErrorKind::Io`] if it indexes it with /// another key than the lease names, [`Evidence::MalformedGeneration`]; and as @@ -447,8 +472,24 @@ impl ClonePlan { let Acquired { lease, locked } = acquired; check_lease(store, lease, locked).attribute(Evidence::HostFailure)?; if let Some(export) = &export { + // The plan resolved the export's paths, so the store's path is resolved as well, so + // that no other spelling of it hides an overlap, and so are the directories through + // which it leads, in which the guest must not write. The lease names the sandbox's + // directory, in which the host writes the clone's records, by its resolved path. + let dir = store.dir(); + let unresolved = |error: io::Error| { + Error::io(format!( + "cannot resolve the template store {}", + dir.display() + )) + .with_source(error) + }; + let resolved = filesystem::canonicalize(dir) + .map_err(unresolved) + .on_host()?; + let through = path_directories(dir).map_err(unresolved).on_host()?; export - .check_outside(store.dir()) + .check_outside(dir, &resolved, &through, lease.sandbox()) .attribute(Evidence::InvalidMappings)?; } let generation = store @@ -730,6 +771,60 @@ fn generation_fault(error: Error) -> Fault { Fault::new(Evidence::MalformedGeneration, error) } +/// Returns the directories through which `path` leads, as the host resolves it, each resolved as +/// [`filesystem::canonicalize`] resolves the paths of an export plan: for each name of the path, +/// `..` included, from the current directory's own for a relative path, the directory in which +/// the host looks it up, which the path before the name resolves to, and, for a name that is a +/// link, the directories through which the link's target leads from there, through at most +/// [`MAX_PATH_LINKS`] links in all. +/// +/// The host reaches the template store, and OpenVMM the snapshots in it, by the path that the +/// store names, so a guest that could write in one of these directories could replace a link or +/// a directory of the path, and turn the host's later operations on the store elsewhere, or +/// make the directory unsearchable, and keep the host from the store. +fn path_directories(path: &Path) -> io::Result> { + let mut directories = BTreeSet::new(); + let mut pending = vec![(PathBuf::new(), std::path::absolute(path)?)]; + let mut links = 0; + while let Some((start, rest)) = pending.pop() { + let mut prefix = start; + for component in rest.components() { + match component { + Component::Normal(_) => {} + Component::CurDir | Component::ParentDir => { + // Unix looks `.` and `..` up in the directory before them, as it does a name, + // so a guest that could change that directory could keep the host from + // searching it. Windows resolves them from a path's text, as + // `std::path::absolute` did. + directories.insert(filesystem::canonicalize(&prefix)?); + prefix.push(component); + continue; + } + Component::RootDir | Component::Prefix(_) => { + // The root and a prefix name no entry. + prefix.push(component); + continue; + } + } + let parent = filesystem::canonicalize(&prefix)?; + prefix.push(component); + if fs::symlink_metadata(&prefix)?.file_type().is_symlink() { + links += 1; + if links > MAX_PATH_LINKS { + return Err(io::Error::other(format!( + "{} leads through more than {MAX_PATH_LINKS} links", + path.display() + ))); + } + // A link's target that is absolute replaces the directory that it starts from. + pending.push((parent.clone(), fs::read_link(&prefix)?)); + } + directories.insert(parent); + } + } + Ok(directories.into_iter().collect()) +} + /// Refuses the path of the file of a clone's block device, which messages call `what`, that /// OpenVMM would not open as the plan checks it: a relative one, which it would resolve against the /// directory in which it runs, one with a comma or a semicolon, which its block device options diff --git a/nvx-rs/src/host/template/clone/export.rs b/nvx-rs/src/host/template/clone/export.rs index d0fc8eca9..47907124b 100644 --- a/nvx-rs/src/host/template/clone/export.rs +++ b/nvx-rs/src/host/template/clone/export.rs @@ -10,7 +10,7 @@ //! directory is then the export's root, which the clone's guest mounts, and the sources of the //! plan's binds name paths in it without the child's index, which only an aggregate's root lists. -use std::path::Path; +use std::path::{Path, PathBuf}; use crate::error::{Error, Result}; use crate::host::filesystem::{ExportPlan, GUEST_EXPORT, Owner, ReservedTargets}; @@ -57,22 +57,82 @@ impl DirectoryExport { } /// Refuses the export if its directory lies inside the directory of a template store, - /// `store`, or holds it: the guest could then reach the payload of a generation that clones + /// `resolved`, or holds it: the guest could then reach the payload of a generation that clones /// lease, and write it through a writable export, though nothing may change a leased payload - /// or its snapshot. + /// or its snapshot. Refuses it as well if it lets the guest write in one of `through`, the + /// directories through which the store's path, `store`, leads: the host reaches the store, + /// and OpenVMM the snapshots in it, by that path, so the guest could replace a link or a + /// directory of the path, and turn the path elsewhere, or make a directory of it + /// unsearchable, and keep the host from following it. /// - /// The check reads nothing: it compares the directory's path, as the plan resolved it, with - /// the store's, [`Store::dir`](crate::host::template::Store::dir), as the store names it, - /// which the check does not resolve, so it does not find an overlap that only the store's - /// resolved path shows, through a link, or on Windows another letter case or a short name. - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for such an export. - pub(super) fn check_outside(&self, store: &Path) -> Result<()> { + /// Refuses it too if its directory is, or holds, the directory of the clone's sandbox, + /// `sandbox`, which the lease names by its path with its links resolved: the host writes the + /// clone's records there, by their paths, so the guest could otherwise change them, or + /// replace the directory, or one above it, with a link, and turn the host's writes + /// elsewhere, into a leased snapshot, say. The directories above it are the only ones through + /// which that path leads, and only an export that holds it could let the guest change one. A + /// directory inside the sandbox's, among the caller's files, holds none of the host's records. + /// + /// The check reads nothing: it compares the directory's path, and the paths in which the + /// guest may write, as the plan resolved them, with `resolved`, `through`, and `sandbox`, + /// which the caller and the lease resolved as [`filesystem::canonicalize`] resolves the plan's + /// paths, so that no link, and on Windows no other letter case, short name, or verbatim + /// prefix, hides an overlap. Fails with + /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for such an export. + /// + /// Since the check compares the paths that the export and the store resolve to, an alias of + /// the store that an administrator configures is outside it, as the directories above the + /// store are: a bind mount or a mount point of the store, or of a directory above or inside + /// it, a loopback share, or a hard link of a payload. Whoever names an export must not name + /// such an alias, nor a directory that holds one. Nor does the check see what changes after + /// it: OpenVMM resolves an export's path again at each launch and restore, so the directories + /// above the export's directory must be ones that no guest may change, as those above the + /// clone's block devices must, which [Block devices by + /// path](crate::host::template::Store#block-devices-by-path) describes: whoever names an + /// export must not name one inside a directory that a writable export of another sandbox or + /// clone holds. + /// + /// [`filesystem::canonicalize`]: crate::host::filesystem::canonicalize + pub(super) fn check_outside( + &self, + store: &Path, + resolved: &Path, + through: &[PathBuf], + sandbox: &Path, + ) -> Result<()> { for child in &self.plan.children { - if child.root.starts_with(store) || store.starts_with(&child.root) { + if child.root.starts_with(resolved) || resolved.starts_with(&child.root) { return Err(Error::unsupported(format!( "the export of a clone cannot share {}, which overlaps the template store {}, \ whose leased payloads nothing may change", child.root.display(), + resolved.display() + ))); + } + if sandbox.starts_with(&child.root) { + return Err(Error::unsupported(format!( + "the export of a clone cannot share {}, which holds the directory of the \ + clone's sandbox {}, in which the host writes the clone's records", + child.root.display(), + sandbox.display() + ))); + } + // The guest may write anywhere in a writable directory, unless the plan limits its + // writes to some of the directory's paths. + let written: &[PathBuf] = match (child.writable, child.write.is_empty()) { + (false, _) => &[], + (true, true) => std::slice::from_ref(&child.root), + (true, false) => child.write.as_slice(), + }; + let directory = through + .iter() + .find(|directory| written.iter().any(|path| directory.starts_with(path))); + if let Some(directory) = directory { + return Err(Error::unsupported(format!( + "the export of a clone cannot let its guest write in {}, through which the \ + path of the template store {} leads, since the guest could then turn that \ + path elsewhere, or keep the host from following it", + directory.display(), store.display() ))); } diff --git a/nvx-rs/src/host/template/clone/tests.rs b/nvx-rs/src/host/template/clone/tests.rs index ab8422b3e..8a46246ca 100644 --- a/nvx-rs/src/host/template/clone/tests.rs +++ b/nvx-rs/src/host/template/clone/tests.rs @@ -275,6 +275,23 @@ fn export_plan( filesystem::plan(&request).unwrap().unwrap() } +/// Returns another spelling of the existing directory `directory`, which names it, but which +/// differs from the path that resolves it: through a directory in it and its `..` on Unix, where +/// an absolute path keeps its `..` components, and with the verbatim prefix on Windows. +fn alias(directory: &Path) -> PathBuf { + #[cfg(windows)] + { + let resolved = filesystem::canonicalize(directory).unwrap(); + PathBuf::from(format!(r"\\?\{}", resolved.display())) + } + #[cfg(not(windows))] + { + let inner = directory.join("alias"); + fs::create_dir(&inner).unwrap(); + inner.join("..") + } +} + /// Returns the mapping table of these tests' exports, [`Fixture::data_export`]: the exported /// directory itself, the export's root, which an empty source names, and the directory `work` in /// it, by its path there, without the index of a child. @@ -574,6 +591,29 @@ fn created_link(linked: io::Result<()>) -> bool { } } +/// Links `link` to the directory `target`: a symbolic link on Unix, and a junction on Windows, +/// which any user may create. +fn link_dir(target: &Path, link: &Path) { + #[cfg(unix)] + std::os::unix::fs::symlink(target, link).unwrap(); + #[cfg(windows)] + { + use std::os::windows::process::CommandExt; + use std::process::{Command, Stdio}; + + let status = Command::new("cmd") + .raw_arg(format!( + "/C mklink /J \"{}\" \"{}\"", + link.display(), + target.display() + )) + .stdout(Stdio::null()) + .status() + .unwrap(); + assert!(status.success(), "cannot link {}", link.display()); + } +} + #[test] fn a_clone_in_ram_restores_its_lease_with_its_image_and_a_sparse_scratch_device() { let fixture = Fixture::new(); @@ -2307,20 +2347,14 @@ fn exports_that_a_dormant_filesystem_slot_does_not_bind_are_refused_before_anyth #[test] fn exports_that_overlap_the_template_store_are_refused_before_anything_is_opened() { - // The plan compares the paths that the export's plan resolved with the store's, as the store - // names it, without reading anything: this store's path is resolved. - let fixture = Fixture::in_store( - |directory| { - filesystem::canonicalize(directory) - .unwrap() - .join("templates") - }, - |_, _, _| {}, - ); + // The plan compares the paths that the export's plan resolved with the store's, resolved as + // well: this store's path names its directory in another spelling than its resolved one. + let fixture = Fixture::in_store(|directory| alias(directory).join("templates"), |_, _, _| {}); let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); let mut acquired = fixture.acquire("sandbox"); let request = tmpfs_request(&image, &binding); - let store = fixture.store.dir().to_owned(); + let store = filesystem::canonicalize(fixture.store.dir()).unwrap(); + assert_ne!(store, fixture.store.dir()); let payload = fixture.store.payload_dir(fixture.generation.generation()); // The store's directory, the payload of a generation inside it, and a directory that holds // the store, which a guest could reach the payload through, and write it if writable. @@ -2352,6 +2386,172 @@ fn exports_that_overlap_the_template_store_are_refused_before_anything_is_opened drop(fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap()); } +#[test] +fn exports_that_let_the_guest_turn_the_template_store_s_path_are_refused() { + // The store's path leads through two links: `share/link`, to `hop/next`, which links to + // `private`, which holds the store. Neither directory of a link holds any part of the store, + // but a guest that could write in either could replace its link, and turn the path, by which + // the host reaches the store, and OpenVMM the snapshots in it, elsewhere. + let fixture = Fixture::in_store( + |directory| { + for name in ["share", "hop", "private"] { + fs::create_dir(directory.join(name)).unwrap(); + } + fs::create_dir(directory.join("share").join("work")).unwrap(); + let next = directory.join("hop").join("next"); + link_dir(&directory.join("private"), &next); + let link = directory.join("share").join("link"); + link_dir(&next, &link); + link.join("templates") + }, + |_, _, _| {}, + ); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let resolved = |name: &str| filesystem::canonicalize(&fixture.directory.path().join(name)); + let (share, hop) = (resolved("share").unwrap(), resolved("hop").unwrap()); + // The store's path resolves outside both directories, so no export of either overlaps it. + let store = filesystem::canonicalize(fixture.store.dir()).unwrap(); + assert_eq!(store, resolved("private").unwrap().join("templates")); + // A writable export of either directory. + for dir in [&share, &hop] { + let export = export_plan(&[(dir, "/data", false)], &[], &[]); + let request = with_export(request.clone(), &export); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT); + assert_eq!( + refused(plan, Evidence::InvalidMappings), + ( + ErrorKind::Unsupported, + format!( + "the export of a clone cannot let its guest write in {}, through which the \ + path of the template store {} leads, since the guest could then turn that \ + path elsewhere, or keep the host from following it", + dir.display(), + fixture.store.dir().display() + ) + ) + ); + assert_untouched(&acquired); + } + // A read-only export of the directory of a link, and one in which the guest writes only a + // directory beside the link, leave the link as it is. + let work = share.join("work"); + for mappings in [ + &[(share.as_path(), "/data", true)][..], + &[ + (share.as_path(), "/data", true), + (work.as_path(), "/work", false), + ], + ] { + let export = export_plan(mappings, &[], &[]); + let request = with_export(request.clone(), &export); + drop(fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap()); + } +} + +#[test] +fn exports_that_let_the_guest_block_the_template_store_s_path_are_refused() { + // The store's path leads through `traverse` and its `..`, which Unix looks up in `traverse`: + // a guest that could write there could make it unsearchable, and keep the host from the + // store. Windows resolves `..` from the path's text, and never looks in `traverse`. + let fixture = Fixture::in_store( + |directory| { + for name in ["traverse", "private"] { + fs::create_dir(directory.join(name)).unwrap(); + } + fs::create_dir(directory.join("traverse").join("work")).unwrap(); + directory + .join("traverse") + .join("..") + .join("private") + .join("templates") + }, + |_, _, _| {}, + ); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let traverse = filesystem::canonicalize(&fixture.directory.path().join("traverse")).unwrap(); + // A writable export of the directory. + let export = export_plan(&[(&traverse, "/data", false)], &[], &[]); + let writable = with_export(request.clone(), &export); + let plan = fixture.plan(&mut acquired, &writable, NO_CONTENT); + if cfg!(windows) { + drop(plan.unwrap()); + } else { + assert_eq!( + refused(plan, Evidence::InvalidMappings), + ( + ErrorKind::Unsupported, + format!( + "the export of a clone cannot let its guest write in {}, through which the \ + path of the template store {} leads, since the guest could then turn that \ + path elsewhere, or keep the host from following it", + traverse.display(), + fixture.store.dir().display() + ) + ) + ); + assert_untouched(&acquired); + } + // A read-only export of the directory, and one in which the guest writes only a directory + // in it, leave the directory as it is. + let work = traverse.join("work"); + for mappings in [ + &[(traverse.as_path(), "/data", true)][..], + &[ + (traverse.as_path(), "/data", true), + (work.as_path(), "/work", false), + ], + ] { + let export = export_plan(mappings, &[], &[]); + let request = with_export(request.clone(), &export); + drop(fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap()); + } +} + +#[test] +fn exports_that_hold_the_sandbox_s_directory_are_refused() { + // The host writes the clone's records in the sandbox's directory, by their paths, so a guest + // that could reach the directory could change them, or replace it with a link, and turn the + // host's writes elsewhere, into a leased snapshot, say. + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = tmpfs_request(&image, &binding); + let sandbox = acquired.lease.sandbox().to_owned(); + // The sandbox's directory, writable, and the directory that holds it, read-only. + for (dir, read_only) in [ + (sandbox.clone(), false), + (sandbox.parent().unwrap().to_owned(), true), + ] { + let export = export_plan(&[(&dir, "/data", read_only)], &[], &[]); + let request = with_export(request.clone(), &export); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT); + assert_eq!( + refused(plan, Evidence::InvalidMappings), + ( + ErrorKind::Unsupported, + format!( + "the export of a clone cannot share {}, which holds the directory of the \ + clone's sandbox {}, in which the host writes the clone's records", + dir.display(), + sandbox.display() + ) + ) + ); + assert_untouched(&acquired); + } + // A directory inside the sandbox's, among the caller's files, holds none of the host's + // records. + let shared = sandbox.join("shared"); + fs::create_dir(&shared).unwrap(); + let export = export_plan(&[(&shared, "/data", false)], &[], &[]); + let request = with_export(request, &export); + drop(fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap()); +} + #[test] fn the_export_of_a_clone_binds_its_directory_with_its_policy_and_owner() { let fixture = Fixture::new(); diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index b89e42156..58f658470 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -1239,7 +1239,8 @@ and its directories and files, and the paths that the guest could move inside the parts that it may write, still name what they named when the export was planned. Once the guest is ready, the launch sends the table, `LayeredLaunch::table`, in the [host's order](#the-hosts-order): each mapping -before those inside its target. +before those inside its target. A [clone's](#clones) start takes the same +shares, whose export NVX binds to the template's dormant filesystem slot. ### Networks @@ -1389,9 +1390,12 @@ library side, `clone`, holds what the command's options choose of the clone, template's placeholder, its upper, on an ext4 scratch image or in RAM, its memory target, the template's RAM unless given, the workload identity as which its executions run, its overlay mode, its scratch's journal policy, the bound -of a GPT disk's first partition, and the machine ID that its executions see. +of a GPT disk's first partition, the machine ID that its executions see, and +the host paths that it shares with its root, as a launch's +[shares](#shares). `Request::plan` plans the image over the upper as a launch plans a layered -root, and `Request::clone_request` returns NVX's `CloneRequest` of it, with the +root, `Request::export` plans the export of the shares as a launch plans it, +and `Request::clone_request` returns NVX's `CloneRequest` of both, with the bindings that the sandbox's store stages for the images, as `stage` stages them, so the start hashes nothing. The clone's activation record names the hostname `runtime-adapter`, which every execution sees, the image's own machine @@ -1404,7 +1408,9 @@ The guest of a GPT clone rereads the disk's partition table before it assembles the root, since its kernel created the partitions of the template's placeholder. -The command first reads the generation from the store's index, and its +The command first plans the export of the clone's shares, if it has any, and +refuses those that it cannot export, as a launch refuses them, then reads the +generation from the store's index, and its compatibility key, before it needs NVX's artifacts. The store then leases the generation to the sandbox, `Store::acquire`, at the host's time, which it refuses for a generation that a tombstone quarantines, or that is not eligible @@ -1423,19 +1429,37 @@ NVX's `ClonePlan` then checks the clone against the generation from the metadata of its files alone, under the sandbox's lock, which the lease holds: the bindings of its images, which no one else may change, the geometry of its image, which must be the placeholder's, its length included, and that of its -scratch image, which must be the template's scratch's, and its memory target, -whole blocks of 128 MiB, from the template's RAM to its capacity. A clone that +scratch image, which must be the template's scratch's, its memory target, +whole blocks of 128 MiB, from the template's RAM to its capacity, and its +export, if it has one: the template's filesystem slot, which its capture left +dormant, binds one directory, so the export must have exactly one, the +export's root, with any other shares inside it, and the directory may neither +lie inside the template store's directory nor hold it, so that no clone +reaches the payload of a generation. A clone that the template cannot serve fails there, before any VMM starts. The command then records the clone in the sandbox's store, as `clone::CloneRecord`, in the format of a launch's [record](#layered-roots), in the mode `clone`, with the template's kernel command line, the plan and the bindings, so that `probe` and `exec` read the plan back, no root configuration, and the template store and the generation, so -that a later stop finds the store whose lease it releases. OpenVMM restores the +that a later stop finds the store whose lease it releases. Once NVX checked +the export again, with `ExportPlan::verify`, OpenVMM restores the generation's snapshot in place, through NVX's supervisor, as the launch that the -lease reserves, and the restore sequence activates the clone through the +lease reserves, with the export's directory on the template's filesystem slot, +and the restore sequence activates the clone through the adapter's clone order, `host::CloneOrder`, which sends the clone's activation -record in an `Activate`, checks the guest agent's build in its `Ready`, and -sends `Start`, with no mapping table: the command maps nothing into a clone. +record in an `Activate`, checks the guest agent's build in its `Ready`, sends +the export's mapping table, which the record announces, in order, as a launch +sends its own, and then `Start`. The record names the export by the identity of +its plan, and the table holds its mappings, of the directory itself, from an +empty source, and of the paths inside it. A clone without shares has no +export: its restore leaves the slot dormant, its record announces no mapping, +and the sequence sends no table. If the guest fails a mapping, the sequence +sends no more of the table, nor `Start`, and the clone fails, once the clone +order asked the guest to stop, which tears down the root. The guest's +`MapFailed` does not tell the request's mappings from a failure of its own, so +the clone order takes it for neither, and NVX attributes the failure to what +the host saw of it: a broken protocol, unless the deadline passed or the +session was lost. A clone that fails, from its plan to its commit, fails with NVX's `CloneFailure`, which attributes the failure, by the evidence that the host saw @@ -1579,7 +1603,7 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from its record again, and releases the lease of a clone that ended, through the template store TEMPLATES, if given, in place of the one that the clone's record names. -- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)]` +- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)] [SHARES]` starts a [clone](#clones) of the generation G of the template store TEMPLATES in the sandbox store STORE, which it creates if needed, and whose `stage` staged its images, which no one else may change: the image PATH, an @@ -1594,8 +1618,12 @@ the directory that `NVX_ARTIFACTS_DIR` names, and the guest agent's build from partition, 64 KiB by default, as a launch's guest takes it. Every execution of the clone sees the hostname `runtime-adapter`, and the machine ID that `--machine-id` chooses: the image's own, `image`, by default, or one that the - guest generates once for the clone, `generated`. - Once the restore sequence started the clone's sandbox, it exchanges a `Ping` + guest generates once for the clone, `generated`. SHARES, as `launch` takes + them, share host paths with the clone's root: the start refuses those that it + cannot export before anything else, and NVX's plan those that the + template's filesystem slot cannot bind, as [Clones](#clones) describes. + Once the restore sequence sent the clone's mapping table, if it has one, and + started the clone's sandbox, it exchanges a `Ping` and the longest `Echo` with the guest agent, commits the launch, and exits, leaving the VM running, and the sandbox holding the lease. A clone that fails, such as one whose activation failed before it was ready, and whose VM @@ -2061,20 +2089,25 @@ Windows, without a hypervisor: - the command lines of `start` that the host program refuses, each with status 2: missing options, a generation that is not a positive number, a scratch's journal policy for an upper in RAM, a first-partition bound for an EROFS - image, invalid values, an option given twice, and the options of other - commands, and the options of `start` that other commands do not take, with a - clock fixture that is not a host's time too; + image, invalid values, an option given twice, a share's options without a + share, and the options of other commands, and the options of `start` that + other commands do not take, with a clock fixture that is not a host's time + too; - a start whose template store does not exist, or does not hold the - generation, and whose image the plan refuses, each with status 1 before it - needs NVX's artifacts or creates the sandbox's store; + generation, whose image the plan refuses, and whose shares it cannot export, + each with status 1 before it needs NVX's artifacts or creates the sandbox's + store, and the shares before it reads the template store; - the text form of a host's time, which `RUNTIME_ADAPTER_HOST_TIME` takes, written and read back on both clocks, and the texts that are none; - a clone's record: its JSON, with the plan that `layered::LaunchRecord` reads back, the template store and the generation that a stop reads, a launch's record that is no clone's, and records that a reader refuses; - NVX's request of a clone, with the staged bindings, the memory target, and - the guest's defaults or the request's settings, and the images that the - sandbox's store does not stage, which it refuses. + the guest's defaults or the request's settings, the images that the + sandbox's store does not stage, which it refuses, and the export of the + request's shares: none without shares, and with them, one directory, the + root of a read-only share, holding a read-write one, mapped from an empty + source and from its path. `tests/initramfs.rs` and `tests/runtime.rs` check, in the host-only configuration, on Linux and Windows: @@ -2424,7 +2457,9 @@ scratch template, which no one else may change either, or in RAM. The GPT disks of a template's placeholder and of the clones, which the test writes with `tests/common/gpt.rs`, as `tests/vm_layered.rs` writes its own, hold their first partition, then layers, empty but for one disk's regular file `etc`, at -partitions that differ from one disk to another: +partitions that differ from one disk to another. The host directories that +clones export lie in the template's directory, beside the template store, in a +directory that the starts authorize: | Test | Clones | Expects | | --- | --- | --- | @@ -2443,6 +2478,10 @@ partitions that differ from one disk to another: | `a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity` | In RAM, then a `reattach` with another guest agent's build | Status 8, `reattached=failed`, without a session, and the record of a failure of the host, at the reattach, by the evidence of the runtime's mismatch, once NVX terminated the VMM, which quarantines nothing; the lease released, with the records; then another clone that starts and stops | | `a_tombstone_leaves_the_running_clone_of_its_generation_to_its_lease` | Over a scratch image, then a tombstone of its generation, through NVX's store | A start that the tombstone refuses, without a lease; the running clone, which reattaches, probes, runs an execution, and stops, which releases its lease; and the generation's snapshot unchanged | | `the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store` | In RAM, in a sandbox that NVX's store leased a generation to, as a start that ended before it recorded the clone leaves it: first without a record of a clone, then with the record of an earlier clone, of another template | A stop that finds the clone `ended` at `planned`, and keeps the lease, whose template store no record names; a stop with another template store, which the release refuses; a `reattach` or a `stop` with `--template` naming the lease's own, which releases it; and a start and a stop in the sandbox after each | +| `a_clone_maps_its_export_beneath_its_root_before_its_sandbox_starts` | Over a scratch image, sharing one host directory: the directory itself, read-only, a read-write directory inside it, at its target there, a read-only file, and a read-write directory, with a file hidden | Started, with four mappings, and a record that names the export, a directory, by the identity of its plan, with four mappings; on Linux, the VMM's one `--mount`, of the directory, read-write; the export mounted once, read-write, `nosuid`, and `nodev`, below a `hostfs` of mode 0700 that root owns; the mappings below the root, and nothing else, in the table's order, each with its access, `nosuid`, and `nodev`, showing what the host holds but for the hidden file, with the missing targets created empty in the upper; the same from an execution as user ID 0; and writes through the writable mappings alone, which the host's directory holds, before and after the stop | +| `clones_at_once_see_only_their_own_exports` | Four at once: one with a read-write export of its own and a generated machine ID, two over the same read-only export, over a scratch image and in RAM, and one without an export | Each with an export started with one mapping, and a record that names it by the identity of its plan, the same for the clones of one export, and on Linux, one `--mount`, read-write or read-only, as the mapping needs; each export mounted read-write, even where its mapping is read-only; below the root, the mapping, beside the bind of a generated machine ID; each clone's own directory, with its access; without an export, no export in the record, no mapping, no `--mount` on Linux, no export mounted, and nothing below the root; the `entries` probe, whose helper writes the root's machine ID, failing with `EROFS` where the generated one is bound; the host's directories with what the writable export took alone; then each stop releases its lease | +| `a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts` | Sharing two host directories, and sharing a file alone | Status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid mappings, which quarantines nothing; the lease released, and the host's directories unchanged | +| `a_clone_whose_mapping_fails_ends_before_its_sandbox_starts` | Over a scratch image, sharing a host directory whose second mapping puts a directory onto a file of the image | Status 8, a `Ready`, but no start, and the record of a failure of an unknown cause, at the start, at the clone's `Ready`, by the evidence of the protocol, once the VM exited with the guest's success, or NVX terminated it, with the guest's report that the table failed after its first mapping; nothing quarantined, and the lease released, with the records; the host's directory unchanged | A test that fails keeps its template's directory, with the template store and the platform's image, as long as it keeps the directory of a clone that leases diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index 42e67ff4e..d45f1f340 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -98,37 +98,49 @@ //! - `runtime-adapter-host start STORE --template TEMPLATES --generation G --distro FORMAT:PATH //! --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity //! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] -//! [--machine-id (image | generated)]` starts a clone of the generation G of the template store -//! TEMPLATES, an `nvx::host::template::Store`, in the sandbox store STORE, which it creates if -//! needed; see `runtime_adapter::clone`. It reads the generation from the store's index, and its -//! compatibility key, whose RAM is the clone's memory target unless MIB mebibytes are given, -//! before it needs NVX's artifacts. The store then leases the generation to the sandbox, at the -//! host's time, with `Store::acquire`, which refuses a generation that a tombstone quarantines, -//! or that is not eligible then, and NVX's `ClonePlan` checks the clone against the generation, -//! from the metadata of its files alone: its image, the raw file PATH, an EROFS filesystem, -//! `erofs`, or a GPT disk, `gpt`, of which the guest expects the identity UUID, of the length of -//! the template's placeholder, whatever the placeholder's format, its upper, the ext4 scratch -//! image PATH, of the geometry of the template's scratch, or RAM, with the bindings that STORE -//! stages for the images, as `stage` stages them, and its memory target, in whole blocks, from -//! the template's RAM to its capacity. +//! [--machine-id (image | generated)] [SHARES]` starts a clone of the generation G of the +//! template store TEMPLATES, an `nvx::host::template::Store`, in the sandbox store STORE, which +//! it creates if needed; see `runtime_adapter::clone`. SHARES, as `launch` takes them, share +//! host paths with the clone's root: the command plans their export as a launch does, with +//! NVX's `filesystem::plan`, and refuses shares that it cannot export before anything else. It +//! then reads the generation from the store's index, and its compatibility key, whose RAM is the +//! clone's memory target unless MIB mebibytes are given, before it needs NVX's artifacts. The +//! store then leases the generation to the sandbox, at the host's time, with `Store::acquire`, +//! which refuses a generation that a tombstone quarantines, or that is not eligible then, and +//! NVX's `ClonePlan` checks the clone against the generation, from the metadata of its files +//! alone: its image, the raw file PATH, an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, of +//! which the guest expects the identity UUID, of the length of the template's placeholder, +//! whatever the placeholder's format, its upper, the ext4 scratch image PATH, of the geometry of +//! the template's scratch, or RAM, with the bindings that STORE stages for the images, as +//! `stage` stages them, its memory target, in whole blocks, from the template's RAM to its +//! capacity, and its export, if it has one, which must export exactly one directory, which +//! neither lies inside the template store's directory nor holds it: the export's root, which +//! the clone's restore binds to the template's dormant filesystem slot. //! A request that the template cannot serve fails there, before any VMM starts. The command //! records the clone in STORE, with the template store and the generation; see -//! `runtime_adapter::clone::CloneRecord`. OpenVMM then restores the generation's snapshot in -//! place, with the clone's devices, through NVX's supervisor, as the launch that the lease -//! reserves, and the restore sequence of `nvx::host::template` activates the clone through the -//! adapter's clone order, `runtime_adapter::host::CloneOrder`, with the clone's activation -//! record: the workload identity UID:GID, if given, as which every execution runs, whose account -//! the guest checks in the clone's root before it is ready, and the overlay mode MODE and the -//! scratch's journal policy POLICY, as a launch takes them, and for a GPT disk, the bound BYTES -//! of its first partition, 64 KiB unless given, as a launch's guest takes it by default. Every -//! execution of the clone sees the hostname `runtime-adapter`, and the machine ID that -//! `--machine-id` chooses: the image's own, `image`, by default, or one that the guest generates -//! once for the clone, `generated`, which a read-only bind holds at `/etc/machine-id`. The -//! guest refuses a first partition larger than BYTES, and an upper that is not fresh, whose -//! `upper` or `work` directory on the scratch image exists and is not empty, as an earlier -//! clone's would be, before the clone is ready. Once the clone is ready, the command starts its -//! sandbox, exchanges a `Ping` and the longest `Echo` with the guest agent, commits the launch, -//! and exits, leaving the VM running, and the sandbox holding the lease, which a stop releases. +//! `runtime_adapter::clone::CloneRecord`. Once NVX checked the export again, with +//! `ExportPlan::verify`, OpenVMM restores the generation's snapshot in place, with the clone's +//! devices, through NVX's supervisor, as the launch that the lease reserves, and the restore +//! sequence of `nvx::host::template` activates the clone through the adapter's clone order, +//! `runtime_adapter::host::CloneOrder`, with the clone's activation record: the workload +//! identity UID:GID, if given, as which every execution runs, whose account the guest checks in +//! the clone's root before it is ready, the overlay mode MODE and the scratch's journal policy +//! POLICY, as a launch takes them, for a GPT disk, the bound BYTES of its first partition, 64 +//! KiB unless given, as a launch's guest takes it by default, and the export, by the identity of +//! its plan, with the number of its mappings. Every execution of the clone sees the hostname +//! `runtime-adapter`, and the machine ID that `--machine-id` chooses: the image's own, `image`, +//! by default, or one that the guest generates once for the clone, `generated`, which a +//! read-only bind holds at `/etc/machine-id`. The guest refuses a first partition larger than +//! BYTES, and an upper that is not fresh, whose `upper` or `work` directory on the scratch image +//! exists and is not empty, as an earlier clone's would be, before the clone is ready. Once the +//! clone is ready, the sequence sends the export's mapping table, in order, as a launch sends +//! its own; a clone without shares has no export: its restore leaves the slot dormant, and the +//! sequence sends no table. If the guest fails a mapping, the sequence sends no more of the +//! table, nor `Start`, and ends the clone, whose guest the clone order asks to stop, which tears +//! down the root, and which then fails as below. Once every mapping applied, the sequence +//! starts the clone's sandbox, and the command exchanges a `Ping` and the longest `Echo` with +//! the guest agent, commits the launch, and exits, leaving the VM running, and the sandbox +//! holding the lease, which a stop releases. //! A clone that fails, from its plan to its commit, fails with NVX's //! `nvx::host::template::CloneFailure`, which attributes the failure by its evidence, and records //! it in STORE from the clone's launch on: the command prints the failure's record, releases the @@ -289,6 +301,7 @@ use std::time::{Duration, Instant}; use nvx::activation::{MachineIdMode, StoreGeneration}; use nvx::exec::{Cause, Key, Limits, Outcome, Status, Stream, Timers, Window}; use nvx::host::artifacts::{Artifacts, absolute}; +use nvx::host::filesystem::ExportPlan; use nvx::host::instance::generate_capability; use nvx::host::launch::{DEFAULT_MEMORY_MIB, LaunchSpec}; use nvx::host::nvxs::HostConnectionError; @@ -407,6 +420,8 @@ const USAGE: &str = concat!( " [--workload-identity UID:GID] [--overlay-mode MODE]", " [--scratch-journal POLICY]\n", " [--first-partition-bound BYTES] [--machine-id (image | generated)]\n", + " [--share HOST:TARGET[:ro]... [--deny PATH]... [--reserved TARGET]...", + " [--share-root DIR]...]\n", " runtime-adapter-host probe STORE PROBE... [--guest-build BUILD]\n", " runtime-adapter-host exec STORE [--guest-build BUILD] EXECUTION [';' EXECUTION]...\n", " EXECUTION: [--identity UID:GID [--groups GID,...] [--capabilities N,...]", @@ -805,17 +820,7 @@ fn parse(arguments: Vec) -> Result { } } "launch" => { - let shares = &options.shares; - let share_options = [ - ("--deny", !shares.denied.is_empty()), - ("--reserved", !shares.reserved.is_empty()), - ("--share-root", !shares.roots.is_empty()), - ]; - if shares.shares.is_empty() - && let Some((option, _)) = share_options.iter().find(|(_, given)| *given) - { - return Err(format!("{option} needs --share")); - } + check_shares(&options.shares)?; let layered = match (options.distro, options.identity, options.upper) { (Some(_), None, _) => return Err("--distro needs --distro-identity".to_owned()), (Some(_), Some(_), None) => return Err("--distro needs --upper".to_owned()), @@ -913,6 +918,7 @@ fn parse(arguments: Vec) -> Result { if format == ImageFormat::Erofs && options.first_partition_bound.is_some() { return Err("--first-partition-bound needs --distro gpt:PATH".to_owned()); } + check_shares(&options.shares)?; Command::Start { store, guest_build, @@ -930,6 +936,7 @@ fn parse(arguments: Vec) -> Result { scratch_journal: options.scratch_journal, first_partition_bound: options.first_partition_bound, machine_id: options.machine_id.unwrap_or(MachineIdMode::Image), + shares: options.shares, }, }), time: host_time_from_env()?, @@ -1052,7 +1059,7 @@ fn parse_option( ) })?); } - ("launch", "--share") => { + ("launch" | "start", "--share") => { let value = text(value)?; let share = Share::parse(&value).ok_or_else(|| { format!( @@ -1062,9 +1069,9 @@ fn parse_option( })?; options.shares.shares.push(share); } - ("launch", "--deny") => options.shares.denied.push(PathBuf::from(value)), - ("launch", "--reserved") => options.shares.reserved.push(text(value)?), - ("launch", "--share-root") => options.shares.roots.push(PathBuf::from(value)), + ("launch" | "start", "--deny") => options.shares.denied.push(PathBuf::from(value)), + ("launch" | "start", "--reserved") => options.shares.reserved.push(text(value)?), + ("launch" | "start", "--share-root") => options.shares.roots.push(PathBuf::from(value)), ("launch" | "create", "--network") if options.network.is_none() => { options.network = Some(PathBuf::from(value)); } @@ -1117,6 +1124,22 @@ fn parse_option( Ok(()) } +/// Says why the options of `shares`, of a launch or a start, are not a command's: `--deny`, +/// `--reserved`, or `--share-root` without a `--share`. +fn check_shares(shares: &Shares) -> Result<(), String> { + let options = [ + ("--deny", !shares.denied.is_empty()), + ("--reserved", !shares.reserved.is_empty()), + ("--share-root", !shares.roots.is_empty()), + ]; + if shares.shares.is_empty() + && let Some((option, _)) = options.iter().find(|(_, given)| *given) + { + return Err(format!("{option} needs --share")); + } + Ok(()) +} + /// Parses the arguments of `exec` after its sandbox store `store`: its executions, each ended by /// a lone `;` or the end of the arguments, with the options before them, or says why they are not /// the command's. @@ -1737,9 +1760,11 @@ fn start( clone: &CloneOptions, time: Option, ) -> Result { - // A request that the adapter's plan refuses, and a generation that the template store does not - // hold, fail the start before it needs NVX's artifacts or the sandbox's store. + // A request that the adapter's plan refuses, shares that it cannot export, and a generation + // that the template store does not hold, fail the start before it needs NVX's artifacts or + // the sandbox's store. let plan = clone.request.plan().map_err(|error| describe(&error))?; + let export = clone.request.export().map_err(|error| describe(&error))?; let templates = Store::open(&clone.templates).map_err(|error| describe(&error))?; let generation = templates .index() @@ -1785,6 +1810,7 @@ fn start( templates: &templates, request: &clone.request, plan, + export, key: &key, runtime: &runtime, guest_build, @@ -1819,6 +1845,8 @@ struct CloneLaunch<'a> { request: &'a runtime_adapter::clone::Request, /// The adapter's plan of the clone's image over its upper. plan: nvx::host::sandbox::SandboxPlan, + /// The export of the host paths that the clone shares with its root, if any. + export: Option, /// The leased generation's compatibility key. key: &'a TemplateKey, /// The runtime of NVX's artifacts. @@ -1829,8 +1857,9 @@ struct CloneLaunch<'a> { impl CloneLaunch<'_> { /// Plans, records, and launches the clone in the sandbox of `acquired`, which leased the - /// generation, activates it, starts its sandbox, exchanges a `Ping` and the longest `Echo` - /// with its guest agent, and commits the launch, leaving the clone running. + /// generation, activates it, maps its export's table, if it has one, starts its sandbox, + /// exchanges a `Ping` and the longest `Echo` with its guest agent, and commits the launch, + /// leaving the clone running. /// /// Fails with the clone's failure, as NVX's plan, launch, restore sequence, and commit return /// it, or as `Started::fail` records it for a guest agent that does not answer the command @@ -1842,6 +1871,7 @@ impl CloneLaunch<'_> { templates, request, plan, + export, key, runtime, guest_build, @@ -1851,7 +1881,7 @@ impl CloneLaunch<'_> { let bindings = staged.map_or_else(Bindings::default, |staged| staged.bindings(&plan)); let memory = request.memory.unwrap_or_else(|| key.memory()); let clone_request = request - .clone_request(plan.clone(), &bindings, memory) + .clone_request(plan.clone(), export, &bindings, memory) .map_err(|error| Failed::Command(binding_error(&error)))?; let clone_plan = ClonePlan::new( templates, @@ -1860,6 +1890,8 @@ impl CloneLaunch<'_> { Instant::now() + LOCK_TIMEOUT, ) .map_err(Failed::Clone)?; + // The mappings that the clone's record announces, which the restore sequence sends. + let mappings = clone_plan.activation().configuration().mappings; let record = CloneRecord { template: templates.dir(), generation: acquired.lease.generation(), @@ -1883,7 +1915,7 @@ impl CloneLaunch<'_> { println!("build={}", escape(guest_build)); let mut started = activated.start(&mut order).map_err(Failed::Clone)?; println!("table=applied"); - println!("mapped=0"); + println!("mapped={mappings}"); println!("sandbox=started"); let deadline = Instant::now() + SESSION_TIMEOUT; let client = started.session(); @@ -1927,10 +1959,11 @@ fn unanswered( } /// Returns the adapter's order of a clone, with the guest agent's build `build`, which the restore -/// sequence of a clone's start, and a reattach to it, take, the one place that makes it. A clone's -/// record announces no mapping table: the adapter maps nothing into a clone. +/// sequence of a clone's start, and a reattach to it, take, the one place that makes it. The +/// sequence sends through it the clone's mapping table, which the clone's record announces if the +/// clone has an export, and a reattach sends none. fn clone_order(build: &str) -> CloneOrder<'_> { - CloneOrder { build, table: &[] } + CloneOrder { build } } /// Ends a command whose clone, in the sandbox of `locked`, failed with `failure`, as NVX diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index f2d639dc7..e1312a754 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -4,13 +4,17 @@ //! A [`Request`] holds what the host program's options choose of a clone: its image, an EROFS //! layer or a GPT disk of the length of the template's placeholder, its upper, on an ext4 scratch //! image or in RAM, its memory target, the workload identity as which its executions run, its -//! overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, and the -//! machine ID that its executions see, the image's own or one that its guest generates. -//! [`Request::plan`] plans the image over the upper as a launch plans a layered root, and -//! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of it, with the +//! overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, the +//! machine ID that its executions see, the image's own or one that its guest generates, and the +//! host paths that it shares with its root, as a launch shares them, `layered::Shares`. +//! [`Request::plan`] plans the image over the upper as a launch plans a layered root, +//! [`Request::export`] plans the export of the shares as a launch plans it, and +//! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of both, with the //! bindings that the sandbox's store stages for its images, as `layered::Staged` records them: //! staging hashes, and a start never does. NVX's `ClonePlan` then checks the request against the -//! leased generation before any VMM starts, from the metadata of the clone's files alone. +//! leased generation before any VMM starts, from the metadata of the clone's files alone, and +//! takes only an export of one directory, which the clone's restore binds to the template's +//! dormant filesystem slot. //! //! A clone's [record](CloneRecord) names the template store and the generation that the clone //! leases, beside the plan and the bindings, as the record of a launch names what it launched, so @@ -30,6 +34,7 @@ use nvx::activation::{ Journal, MachineIdMode, OverlayMode as RecordOverlay, StoreGeneration, WorkloadIdentity, WorkloadView, }; +use nvx::host::filesystem::ExportPlan; use nvx::host::process::BootId; use nvx::host::sandbox::SandboxPlan; use nvx::host::sandbox::distro::DISTRO_POLICY_REVISION; @@ -39,7 +44,9 @@ use nvx::uuid::Uuid; use nvx::{Error, Result}; use serde_json::{Value, json}; -use crate::layered::{Bindings, RECORD_FORMAT, RECORD_NAME, invalid, plan, unstaged, write_record}; +use crate::layered::{ + Bindings, RECORD_FORMAT, RECORD_NAME, Shares, invalid, plan, unstaged, write_record, +}; use crate::root::{DEFAULT_FIRST_PARTITION_BOUND, JournalPolicy, OverlayMode}; /// The mode of a clone's [record](CloneRecord). @@ -83,6 +90,8 @@ pub struct Request { /// The machine ID that the clone's executions see: the image's own, as a launch's do, or one /// that the clone's guest generates. pub machine_id: MachineIdMode, + /// The host paths that the clone shares with its root, as a launch shares them, if any. + pub shares: Shares, } impl Request { @@ -102,9 +111,22 @@ impl Request { ) } - /// Returns NVX's request of the clone of `plan`, the [plan](Self::plan) of this request, with - /// the bindings that the sandbox's store stages for its images, `bindings`, as - /// `layered::Staged::bindings` finds them, and the memory target `memory`, in bytes. + /// Plans the export of the host paths that the clone shares with its root, as a launch plans + /// it, with `layered::Shares::plan`, or returns `None` for a clone that shares none, whose + /// restore leaves the template's filesystem slot dormant. + /// + /// Fails as `Shares::plan` does, such as for a share outside every host directory that the + /// adapter authorizes. The plan of an export that the template's dormant filesystem slot + /// cannot bind, of more than one directory or file, or of a file, is NVX's `ClonePlan` to + /// refuse. + pub fn export(&self) -> Result> { + self.shares.plan() + } + + /// Returns NVX's request of the clone of `plan`, the [plan](Self::plan) of this request, and + /// of `export`, the [export](Self::export) of its shares, if any, with the bindings that the + /// sandbox's store stages for its images, `bindings`, as `layered::Staged::bindings` finds + /// them, and the memory target `memory`, in bytes. /// /// The record of the clone's activation names the workload identity, if any, the /// [`CLONE_HOSTNAME`], the request's machine ID, the overlay's mode, and the scratch's @@ -118,6 +140,7 @@ impl Request { pub fn clone_request( &self, plan: SandboxPlan, + export: Option, bindings: &Bindings, memory: u64, ) -> Result { @@ -147,6 +170,7 @@ impl Request { binding, policy_revision: DISTRO_POLICY_REVISION, scratch, + export, memory, workload: WorkloadView { hostname: CLONE_HOSTNAME.as_bytes().to_vec(), diff --git a/tests/runtime-adapter/tests/clone_start.rs b/tests/runtime-adapter/tests/clone_start.rs index 4d26729ad..59185578c 100644 --- a/tests/runtime-adapter/tests/clone_start.rs +++ b/tests/runtime-adapter/tests/clone_start.rs @@ -1,9 +1,10 @@ //! The host program's `start` refuses options that do not make a start, and a clock fixture that is -//! not a host's time, as usage errors, and a generation that the template store does not hold, or -//! a template store that does not exist, before it needs NVX's artifacts or creates the sandbox's -//! store. A clone's record names its template store and its generation, which a stop reads back, -//! beside the plan that the other commands read, and a host's time keeps its text form, the form -//! of the clock fixture. The tests need no hypervisor. +//! not a host's time, as usage errors, and shares that it cannot export, a generation that the +//! template store does not hold, or a template store that does not exist, before it needs NVX's +//! artifacts or creates the sandbox's store. A clone's record names its template store and its +//! generation, which a stop reads back, beside the plan that the other commands read, a clone's +//! request carries the export of its shares, and a host's time keeps its text form, the form of +//! the clock fixture. The tests need no hypervisor. #![cfg(feature = "host")] use std::fs; @@ -19,7 +20,9 @@ use nvx::uuid::Uuid; use runtime_adapter::clone::{ CLONE_MODE, CloneRecord, HOST_TIME_VARIABLE, Request, host_time_text, parse_host_time, }; -use runtime_adapter::layered::{Bindings, LaunchRecord, RECORD_FORMAT, RECORD_NAME, Staged}; +use runtime_adapter::layered::{ + Bindings, LaunchRecord, RECORD_FORMAT, RECORD_NAME, Share, Shares, Staged, +}; use runtime_adapter::root::DEFAULT_FIRST_PARTITION_BOUND; use serde_json::{Value, json}; @@ -179,13 +182,38 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { vec!["stop", "store", "--memory", "512"], "unexpected option \"--memory\" of stop", ), + // A start takes the shares of a launch, with the same rules. + ( + with(&["--upper", "ram", "--deny", "secret"]), + "--deny needs --share", + ), + ( + with(&["--upper", "ram", "--reserved", "/opt/adapter"]), + "--reserved needs --share", + ), + ( + with(&["--upper", "ram", "--share-root", "sources"]), + "--share-root needs --share", + ), + ( + with(&["--upper", "ram", "--share", ":/b:ro"]), + "--share takes HOST:TARGET or HOST:TARGET:ro, with an absolute guest path TARGET, \ + not \":/b:ro\"", + ), + ( + with(&["--upper", "ram", "--share", "a"]), + "--share takes HOST:TARGET or HOST:TARGET:ro", + ), + ( + vec!["stop", "store", "--share", "a:/b"], + "unexpected option \"--share\" of stop", + ), ]; // Each option that a start does not take. - let others: [(&str, &str); 7] = [ + let others: [(&str, &str); 6] = [ ("--guest-agent", "agent"), ("--hypervisor", "whp"), ("--runtime-dir", "/run/a"), - ("--share", "a:/b"), ("--network", "network.json"), ("--cpu-profile", "auto"), ("--memory-capacity", "512"), @@ -369,6 +397,7 @@ fn a_clone_s_record_names_its_template_store_and_generation_beside_its_plan() { scratch_journal: None, first_partition_bound: None, machine_id: MachineIdMode::Image, + shares: Shares::default(), }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); @@ -458,17 +487,19 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { scratch_journal: None, first_partition_bound: None, machine_id: MachineIdMode::Image, + shares: Shares::default(), }; let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); let bindings = staged.bindings(&plan); let memory = 512 << 20; let clone = request - .clone_request(plan.clone(), &bindings, memory) + .clone_request(plan.clone(), None, &bindings, memory) .unwrap(); assert_eq!(clone.adapter, plan); assert_eq!(clone.binding, staged.distro.binding); assert_eq!(clone.scratch, Some(staged.scratch.unwrap().binding)); + assert_eq!(clone.export, None); assert_eq!(clone.memory, memory); assert_eq!( clone.policy_revision, @@ -491,7 +522,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { ..request.clone() }; let clone = named - .clone_request(plan.clone(), &bindings, memory) + .clone_request(plan.clone(), None, &bindings, memory) .unwrap(); assert_eq!( clone.workload.identity, @@ -518,7 +549,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { ), ] { let error = request - .clone_request(plan.clone(), &bindings, memory) + .clone_request(plan.clone(), None, &bindings, memory) .unwrap_err(); assert_eq!(error.kind(), nvx::ErrorKind::Unavailable, "{what}"); assert!( @@ -536,13 +567,144 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { }; let plan = in_ram.plan().unwrap(); let error = in_ram - .clone_request(plan.clone(), &Bindings::default(), memory) + .clone_request(plan.clone(), None, &Bindings::default(), memory) .unwrap_err(); assert_eq!(error.kind(), nvx::ErrorKind::Unavailable); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), None).unwrap(); let clone = in_ram - .clone_request(plan.clone(), &staged.bindings(&plan), memory) + .clone_request(plan.clone(), None, &staged.bindings(&plan), memory) .unwrap(); assert_eq!(clone.scratch, None); assert_eq!(clone.adapter, plan); + + // A request without shares plans no export. One with shares plans their export as a launch + // does, of one directory here, the root of the read-only share, which holds the other, and + // NVX's request carries the export that it is given. + assert_eq!(in_ram.export().unwrap(), None); + let sources = directory.join("sources"); + fs::create_dir_all(sources.join("out")).unwrap(); + let shared = Request { + shares: Shares { + shares: vec![ + Share { + host: sources.clone(), + target: "/srv/data".to_owned(), + read_only: true, + }, + Share { + host: sources.join("out"), + target: "/srv/out".to_owned(), + read_only: false, + }, + ], + roots: vec![directory.clone()], + ..Shares::default() + }, + ..in_ram + }; + let export = shared.export().unwrap().expect("the shares have an export"); + assert_eq!(export.children.len(), 1, "{export:?}"); + assert!(!export.children[0].file, "{export:?}"); + let binds = export + .binds + .iter() + .map(|bind| (bind.source.as_str(), bind.target.as_str(), bind.read_only)) + .collect::>(); + assert_eq!( + binds, + [("", "/srv/data", true), ("out", "/srv/out", false)], + "{export:?}" + ); + let clone = shared + .clone_request( + plan.clone(), + Some(export.clone()), + &staged.bindings(&plan), + memory, + ) + .unwrap(); + assert_eq!(clone.export, Some(export)); +} + +#[test] +fn a_start_refuses_shares_that_it_cannot_export_before_anything_else() { + // The start plans the export of its shares before it reads the template store, which does + // not exist, or needs NVX's artifacts, which the commands do not name, and before it creates + // the sandbox's store. + let directory = directory("start-unshared"); + let distro = image(directory.join("distro.erofs"), 4096); + let distro = format!("erofs:{}", distro.display()); + let sources = directory.join("sources"); + let outside = directory.join("outside"); + for path in [&sources, &outside] { + fs::create_dir(path).unwrap(); + } + let templates = directory.join("templates"); + let store = directory.join("store"); + let missing = directory.join("sources").join("missing"); + let inner = directory.join("sources").join("inner"); + fs::create_dir(&inner).unwrap(); + let (sources, outside) = (sources.to_str().unwrap(), outside.to_str().unwrap()); + let cases: [(Vec, String); 5] = [ + ( + vec![format!("{outside}:/srv/data")], + format!( + "--share entry {outside} lies outside the host directories that --share-root \ + authorizes" + ), + ), + ( + vec![format!("{sources}:/proc/data")], + "has the guest target /proc/data, which would cover the guest's /proc".to_owned(), + ), + ( + vec![ + format!("{sources}:/srv/data"), + "--share".to_owned(), + format!("{}:/srv/data/inner:ro", inner.display()), + ], + "where the host cannot keep it read-only".to_owned(), + ), + ( + vec![ + format!("{sources}:/srv/data"), + "--reserved".to_owned(), + "srv".to_owned(), + ], + "--reserved entry \"srv\" is not absolute".to_owned(), + ), + ( + vec![format!("{}:/srv/missing", missing.display())], + format!("--share entry {} is not accessible", missing.display()), + ), + ]; + for (shares, problem) in cases { + let mut arguments = vec![ + "start".to_owned(), + store.to_str().unwrap().to_owned(), + "--template".to_owned(), + templates.to_str().unwrap().to_owned(), + "--generation".to_owned(), + "1".to_owned(), + "--distro".to_owned(), + distro.clone(), + "--distro-identity".to_owned(), + IDENTITY.to_owned(), + "--upper".to_owned(), + "ram".to_owned(), + "--share-root".to_owned(), + sources.to_owned(), + "--share".to_owned(), + ]; + arguments.extend(shares); + let arguments = arguments.iter().map(String::as_str).collect::>(); + let output = host(&arguments, None); + assert_eq!(output.status.code(), Some(1), "{arguments:?}: {output:?}"); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains(&problem), "{arguments:?}: {stderr}"); + assert!(!stderr.contains("template store"), "{stderr}"); + assert!(!stderr.contains("usage:"), "{stderr}"); + assert!(!store.exists(), "the start created the sandbox's store"); + assert!(!templates.exists()); + } } diff --git a/tests/runtime-adapter/tests/vm_clone.rs b/tests/runtime-adapter/tests/vm_clone.rs index 871e63dbf..191c19546 100644 --- a/tests/runtime-adapter/tests/vm_clone.rs +++ b/tests/runtime-adapter/tests/vm_clone.rs @@ -41,6 +41,22 @@ //! - A clone whose generated machine ID the guest cannot bind, since its image holds a regular //! file `etc`, ends before it is ready, with the status of the workload's view, while a clone of //! the image that keeps the image's own machine ID, which needs no bind, starts. +//! - A clone maps its export beneath its root once it is ready, before its sandbox starts: one +//! host directory, the export's root, which NVX binds to the template's dormant filesystem slot, +//! and whose mappings, the directory's own, from an empty source, and those of paths inside it, +//! each keep their access, show what the host holds, but for a path that the start hides, and +//! take what the guest, and the clone's workload, write through the writable ones alone. The +//! clone's record names the export by the identity of its plan, with the number of its +//! mappings. +//! - Clones of one generation at once see only their own exports: each its own directory, or two +//! the same one, which their records name by the same identity, and a clone without an export +//! none, whose restore leaves the slot dormant. A clone mounts its export read-write, even where +//! every mapping is read-only, mount tables list the bind of a generated machine ID below the +//! root beside the mappings, and the bind holds even against the guest's root. +//! - A start refuses an export that the template's dormant filesystem slot cannot bind before any +//! VMM starts, as a failure of the request: one of two directories, and one of a file. +//! - A clone whose guest fails a mapping of its table ends before its sandbox starts, nothing of +//! it runs, and the host's directory stays as it was. //! - A clone runs its executions as its record's workload identity, once it is ready: the repair //! acknowledged the restore before the clone took its reset, the host's first message came //! after that, and the clone's Ready, its start, and its first execution came in that order. @@ -75,7 +91,7 @@ mod common; -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use std::env; use std::ffi::{OsStr, OsString}; use std::fs::{self, File}; @@ -86,9 +102,13 @@ use std::path::{Path, PathBuf}; use std::thread; use std::time::{Duration, Instant, SystemTime}; -use nvx::activation::{MachineIdMode, ReadyActivation, StoreGeneration}; +use nvx::activation::{ + ActivationRecord, Configuration, ExportKind, MachineIdMode, ReadyActivation, StoreGeneration, +}; use nvx::exec::{Key, Limits, Timers, Window}; use nvx::host::artifacts::Artifacts; +#[cfg(target_os = "linux")] +use nvx::host::filesystem::GUEST_EXPORT; use nvx::host::platform::Transport; use nvx::host::process::{BootId, Process}; use nvx::host::store::{CallerFiles, LOG_NAME, SandboxStore}; @@ -97,9 +117,9 @@ use nvx::host::template::{ Generation, HostTime, KEY_NAME, Lease, MAX_DOWNTIME_NS, MEMORY_BLOCK_SIZE, Outcome, Progress, RestoreDeadlines, SNAPSHOT_NAME, Stage, Store, Teardown, }; -use nvx::protocol::CAPABILITY_LEN; use nvx::protocol::root::ImageFormat; use nvx::protocol::status; +use nvx::protocol::{CAPABILITY_LEN, HostMap}; use nvx::uuid::Uuid; use runtime_adapter::GUEST_BUILD; use runtime_adapter::capture::DEFAULT_SCRATCH_LENGTH; @@ -107,7 +127,7 @@ use runtime_adapter::clone::{CLONE_HOSTNAME, HOST_TIME_VARIABLE, Request, host_t use runtime_adapter::exec::{HostMessage, Start}; use runtime_adapter::frame::Message; use runtime_adapter::host::Client; -use runtime_adapter::layered::Staged; +use runtime_adapter::layered::{Shares, Staged}; use runtime_adapter::root::DEFAULT_RUNTIME; use runtime_adapter::runtime::Runtime; @@ -162,6 +182,66 @@ fi echo "marker $(cat /marker 2>/dev/null || echo absent)" printf '%s' "$1" > /marker "#; +/// The directory of a template's sandbox that holds the host directories that the scenario's +/// clones export, the one host directory that their starts authorize. +const EXPORTS: &str = "exports"; +/// The directories of the host directory that a clone exports in full, with the +/// [`EXPORT_SHARES`]: `dir`, which holds the empty directory `inner`, and `out`. +const EXPORT_DIRECTORIES: [&str; 3] = ["dir", "dir/inner", "out"]; +/// The files of the host directory that a clone exports in full, each a path in it, with +/// components separated by `/`, and its contents. +const EXPORT_FILES: [(&str, &str); 4] = [ + ("dir/marker", "marker"), + ("dir/secret", "secret"), + ("file.txt", "file"), + ("out/marker", "marker"), +]; +/// The shares of a clone that exports a host directory in full, each a path in it, `.` for the +/// directory itself, and its target in the root, with `:ro` for a read-only share: the directory, +/// read-only, with the read-write directory `dir/inner` inside it, at the target that it has +/// there, the read-only file `file.txt`, and the read-write directory `out`. The plan exports the +/// directory alone, the export's root, whose mapping comes from an empty source, and the start +/// hides `dir/secret`. +const EXPORT_SHARES: [(&str, &str); 4] = [ + (".", "/srv/maps:ro"), + ("dir/inner", "/srv/maps/dir/inner"), + ("file.txt", "/srv/file.txt:ro"), + ("out", "/srv/out"), +]; +/// The `map` probe of the [`EXPORT_SHARES`]: each target, and the hidden file. +const MAP_EXPORT: &str = "map maps=srv/maps inner=srv/maps/dir/inner file=srv/file.txt \ + out=srv/out secret=srv/maps/dir/secret"; +/// The mappings of the [`EXPORT_SHARES`], in the order in which the guest applied them, which the +/// root's mount table keeps: by the depth of their targets, then by their targets. +const EXPORT_MOUNTS: &str = "/srv/file.txt,/srv/maps,/srv/out,/srv/maps/dir/inner"; +/// The program of the executions that show what a clone's workload finds of the +/// [`EXPORT_SHARES`], `/bin/sh -c EXPORTED sh`, which prints, a line each: the contents of +/// `/srv/maps/dir/marker` and of `/srv/file.txt`, `read PATH TEXT`, or `read PATH refused` for one +/// that cannot be read; whether the hidden `/srv/maps/dir/secret` could be read, `secret read`, +/// or not, `secret refused`; the names in `/srv/maps/dir`, each followed by a comma, `listing +/// NAMES`; and for each of `/srv/maps`, `/srv/maps/dir/inner`, and `/srv/out`, whether a write of +/// the new file `workload` in it succeeded, `write DIRECTORY written`, or not, `write DIRECTORY +/// refused`. +const EXPORTED: &str = r#"for path in /srv/maps/dir/marker /srv/file.txt; do + if text=$(cat "$path" 2>/dev/null); then + echo "read $path $text" + else + echo "read $path refused" + fi +done +if cat /srv/maps/dir/secret >/dev/null 2>&1; then echo "secret read"; else echo "secret refused"; fi +echo "listing $(ls /srv/maps/dir | tr '\n' ',')" +for directory in /srv/maps /srv/maps/dir/inner /srv/out; do + if { printf '%s' workload > "$directory/workload"; } 2>/dev/null; then + echo "write $directory written" + else + echo "write $directory refused" + fi +done +"#; +/// The `map` probe of the clones that share one host directory at `/srv/data`: the directory, and +/// its file `marker`. +const MAP_DATA: &str = "map data=srv/data marker=srv/data/marker"; /// The placeholder of the template of the scenarios of GPT disks: a disk of two layers, whose /// partitions start at multiples of 1 MiB. const PLACEHOLDER: Disk = Disk { @@ -354,6 +434,16 @@ impl Template { .store } + /// Returns the directory of the template's sandbox, which holds the template store, the + /// platform's directory, and the directory of the host directories that the scenario's clones + /// export, [`EXPORTS`]. + fn root(&self) -> &Path { + let sandbox = self.sandbox.as_ref(); + &sandbox + .expect("the sandbox lasts until the template drops") + .root + } + /// Returns the template store. fn store(&self) -> Store { Store::open(self.dir()).unwrap() @@ -572,6 +662,12 @@ impl Cloned { /// Checks that the start started the clone, as the [module documentation](self) describes, /// leasing the generation, and recorded its VM, which runs. fn assert_started(&self) { + self.assert_mapped(0); + } + + /// Checks that the start started the clone, as [`assert_started`](Self::assert_started) does, + /// once its guest applied `mapped` mappings of its export, none for a clone without one. + fn assert_mapped(&self, mapped: usize) { let started = &self.started; assert_eq!(started["generation"], GENERATION.to_string(), "{started:?}"); let lease = &started["lease"]; @@ -580,10 +676,11 @@ impl Cloned { "{started:?}" ); started["downtime"].parse::().unwrap(); + let mapped = mapped.to_string(); let facts = ["root", "build", "table", "mapped", "sandbox"].map(|key| &started[key]); assert_eq!( facts, - ["ready", GUEST_BUILD, "applied", "0", "started"], + ["ready", GUEST_BUILD, "applied", mapped.as_str(), "started"], "{started:?}" ); let record = self.sandbox.record().expect("the start recorded its VM"); @@ -705,6 +802,66 @@ impl Cloned { marker: value(marker, "marker"), } } + + /// Runs `script` with `/bin/sh -c` in the clone's root, as user ID 0, checks that it exited + /// with status 0, and wrote nothing to its standard error, and returns the lines that it + /// printed. + fn script(&mut self, script: &str) -> Vec { + let program = ["/bin/sh", "-c", script, "sh"]; + let ran = self.exec(&[&["--identity", "0:0", "--"][..], &program].concat()); + let facts = ["launch", "status", "stderr"].map(|key| &ran[&format!("exec.1.{key}")]); + assert_eq!(facts, ["started", "exit 0", ""], "{ran:?}"); + // The host program escapes each line feed, and each backslash, which none of the lines + // holds. + ran["exec.1.stdout"] + .strip_suffix("\\n") + .filter(|lines| !lines.contains("\\\\")) + .unwrap_or_else(|| panic!("{ran:?}")) + .split("\\n") + .map(str::to_owned) + .collect() + } + + /// Returns the configuration of the clone's activation record, which NVX keeps in the clone's + /// sandbox, in its canonical encoding, while the clone holds its lease. + fn configuration(&self) -> Configuration { + let store = SandboxStore::open(&self.sandbox.store).unwrap(); + let encoded = fs::read(store.clone_activation_path()).unwrap(); + let record = ActivationRecord::decode(&encoded).unwrap(); + record.configuration().clone() + } + + /// Checks the arguments with which NVX started the clone's VMM, on Linux, where the test, as + /// root, reads them: one `--mount`, which binds the directory of `export`, as the plan + /// resolved it, to the template's dormant filesystem slot, with the access `mode`, or none, + /// for a clone without an export, whose restore leaves the slot dormant. Elsewhere, NVX's own + /// tests of a clone's launch check the arguments, through a fake VMM. + fn assert_vmm_mount(&self, export: Option<(&Export, &str)>) { + #[cfg(target_os = "linux")] + { + let pid = &self.started["pid"]; + let command_line = fs::read(format!("/proc/{pid}/cmdline")).unwrap(); + let arguments = command_line + .split(|byte| *byte == 0) + .map(|argument| String::from_utf8_lossy(argument).into_owned()) + .collect::>(); + let mounts = arguments + .windows(2) + .filter(|pair| pair[0] == "--mount") + .map(|pair| pair[1].as_str()) + .collect::>(); + let expected = export + .map(|(export, mode)| { + let directory = fs::canonicalize(&export.root).unwrap(); + format!("{GUEST_EXPORT},{},{mode}", directory.display()) + }) + .into_iter() + .collect::>(); + assert_eq!(mounts, expected, "{arguments:?}"); + } + #[cfg(not(target_os = "linux"))] + let _ = export; + } } /// What an execution of [`VIEW`] found in a clone's root. @@ -722,6 +879,124 @@ struct View { marker: String, } +/// A host directory that a scenario's clones export, in the [`EXPORTS`] of the template's +/// sandbox, beside the template store, which neither holds the other. +struct Export { + /// The directory. + root: PathBuf, +} + +impl Export { + /// Writes the directory `name` in the [`EXPORTS`] of the sandbox of `template`, with `files`, + /// each a path in it, with components separated by `/`, and its contents, and the empty + /// `directories`, and lets anyone create entries in each of its directories, itself included, + /// so that only a mapping's access refuses what a clone's execution writes there. + /// + /// A Linux host shows the guest each host object's owner and mode, which the guest's kernel + /// checks: on an unprivileged host, the directories belong to the test's user, not to user + /// ID 0, as which an execution runs without the capabilities that override the check, so they + /// would refuse its writes. A Windows host shows the guest user ID 0 as each object's owner, so + /// there the directories keep their access. + fn write( + template: &Template, + name: &str, + files: &[(&str, &str)], + directories: &[&str], + ) -> Self { + let export = Self { + root: template.root().join(EXPORTS).join(name), + }; + fs::create_dir_all(&export.root).unwrap(); + for directory in directories { + fs::create_dir_all(export.path(directory)).unwrap(); + } + for (file, contents) in files { + let path = export.path(file); + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, contents).unwrap(); + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + + let mut pending = vec![export.root.clone()]; + while let Some(directory) = pending.pop() { + fs::set_permissions(&directory, fs::Permissions::from_mode(0o777)).unwrap(); + for entry in fs::read_dir(&directory).unwrap() { + let entry = entry.unwrap(); + if entry.file_type().unwrap().is_dir() { + pending.push(entry.path()); + } + } + } + } + export + } + + /// Returns the host path of `path`, a path in the directory, with components separated by + /// `/`, or `.` for the directory itself. + fn path(&self, path: &str) -> PathBuf { + path.split('/') + .filter(|name| *name != ".") + .fold(self.root.clone(), |host, name| host.join(name)) + } + + /// Returns the options of a start that shares each of `shares` with the clone's root, a path + /// in the directory, `.` for the directory itself, and its target, with `:ro` for a read-only + /// share, hides each of `denied`, paths in the directory, and authorizes the [`EXPORTS`]. + fn options(&self, shares: &[(&str, &str)], denied: &[&str]) -> Vec { + let mut options = Vec::new(); + for (path, target) in shares { + let share = format!("{}:{target}", self.path(path).display()); + options.extend(["--share".to_owned(), share]); + } + for path in denied { + options.extend(["--deny".to_owned(), self.path(path).display().to_string()]); + } + let exports = self.root.parent().unwrap(); + options.extend(["--share-root".to_owned(), exports.display().to_string()]); + options + } + + /// Returns what the directory holds, at any depth: each path in it, with components separated + /// by `/`, and a file's contents, or `None` for a directory. + fn tree(&self) -> BTreeMap> { + let mut found = BTreeMap::new(); + let mut pending = vec![(self.root.clone(), String::new())]; + while let Some((path, prefix)) = pending.pop() { + for entry in fs::read_dir(&path).unwrap() { + let entry = entry.unwrap(); + let name = format!("{prefix}{}", entry.file_name().to_str().unwrap()); + if entry.file_type().unwrap().is_dir() { + pending.push((entry.path(), format!("{name}/"))); + found.insert(name, None); + } else { + found.insert(name, Some(fs::read_to_string(entry.path()).unwrap())); + } + } + } + found + } +} + +/// Returns the tree of a directory that holds the directories `directories`, and `files`, each a +/// path in it, with components separated by `/`, and its contents, as [`Export::tree`] lists it. +fn tree_of(directories: &[&str], files: &[(&str, &str)]) -> BTreeMap> { + let mut tree = BTreeMap::new(); + for directory in directories { + tree.insert((*directory).to_owned(), None); + } + for (file, contents) in files { + tree.insert((*file).to_owned(), Some((*contents).to_owned())); + } + tree +} + +/// Returns the options of a mount, as the `map` probe gives them. +fn mount_options(options: &str) -> BTreeSet<&str> { + options.split(',').collect() +} + /// Returns the scratch image of a clone over one, in the clone's sandbox, `sandbox`, in a /// directory of the sandbox's own. fn scratch_image(sandbox: &Sandbox) -> PathBuf { @@ -1033,6 +1308,15 @@ impl CloneControl for Early { }) } + fn map( + &mut self, + _session: &mut Client, + _table: &[HostMap], + _deadline: Instant, + ) -> io::Result<()> { + unreachable!("a clone that is never ready gets no mapping table") + } + fn start(&mut self, _session: &mut Client, _deadline: Instant) -> io::Result<()> { Err(io::Error::other("the clone is never ready")) } @@ -1064,12 +1348,13 @@ fn refuses_early(template: &Template, early: Message) { scratch_journal: None, first_partition_bound: None, machine_id: MachineIdMode::Image, + shares: Shares::default(), }; let plan = request.plan().unwrap(); let staged = Staged::read(&sandbox.store).unwrap().unwrap(); let key = store.read_key(&template.generation()).unwrap(); let clone = request - .clone_request(plan.clone(), &staged.bindings(&plan), key.memory()) + .clone_request(plan.clone(), None, &staged.bindings(&plan), key.memory()) .unwrap(); let now = HostTime::now().unwrap(); let mut acquired = store @@ -2055,3 +2340,364 @@ fn start_in(sandbox: &mut Sandbox, template: &Template, templates: &Path) -> Pri ]; sandbox.run("start", &options, 0) } + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_clone_maps_its_export_beneath_its_root_before_its_sandbox_starts() { + let template = Template::create("cl-export"); + let export = Export::write(&template, "full", &EXPORT_FILES, &EXPORT_DIRECTORIES); + let shares = export.options(&EXPORT_SHARES, &["dir/secret"]); + let options = shares.iter().map(String::as_str).collect::>(); + let start = StartOf { + upper: Upper::Scratch, + options: &options, + ..StartOf::default() + }; + let mut clone = template.start("cl-export-1", start, 0); + clone.assert_mapped(EXPORT_SHARES.len()); + + // The clone's record names the export, a directory, by the identity of its plan, and the + // number of its mappings, which the restore sequence sent once the clone was ready. NVX bound + // the directory to the template's dormant filesystem slot, read-write, since some of the + // mappings are. + let configuration = clone.configuration(); + let recorded = configuration.export.expect("the record names the export"); + assert_eq!(recorded.kind, ExportKind::Directory); + assert_eq!(configuration.mappings, EXPORT_SHARES.len()); + clone.assert_vmm_mount(Some((&export, "rw"))); + + // The export, mounted once, read-write, without set-user-ID programs or devices, below a + // directory of the runtime's that only root may enter, and the table's mappings below the + // root, and nothing else, in their order, as a launch's: each with its access, `nosuid`, and + // `nodev`, the guest writing the writable ones alone, the one inside a read-only mapping + // included, each showing what the host holds, but the file that the start hides. The guest + // created the missing targets in the upper, empty, but none inside another mapping. + let facts = clone.probe(&[MAP_EXPORT]); + let fact = |key: &str| facts[&format!("map.{key}")].as_str(); + assert_eq!( + (fact("export.count"), fact("export.type")), + ("1", "virtiofs") + ); + let options = mount_options(fact("export.options")); + assert!( + options.is_superset(&BTreeSet::from(["rw", "nosuid", "nodev"])), + "{options:?}" + ); + assert_eq!((fact("hostfs.mode"), fact("hostfs.owner")), ("0700", "0:0")); + assert_eq!(fact("mounts"), EXPORT_MOUNTS); + for (name, access, kind, write, upper) in [ + ("maps", "ro", "directory", "refused", "directory"), + ("inner", "rw", "directory", "allowed", "absent"), + ("file", "ro", "file 4", "refused", "file 0"), + ("out", "rw", "directory", "allowed", "directory"), + ] { + let options = mount_options(fact(&format!("{name}.options"))); + for flag in [access, "nosuid", "nodev"] { + assert!(options.contains(flag), "{name}: {options:?}"); + } + let found = ["kind", "write", "upper"].map(|key| fact(&format!("{name}.{key}"))); + assert_eq!(found, [kind, write, upper], "{name}"); + } + assert_eq!(fact("maps.entries"), "dir,file.txt,out"); + assert_eq!(fact("inner.entries"), ""); + assert_eq!(fact("file.read"), "file"); + assert_eq!(fact("out.entries"), "marker"); + let secret = ["options", "kind", "upper"].map(|key| fact(&format!("secret.{key}"))); + assert_eq!(secret, ["none", "denied", "absent"]); + assert_eq!(fact("latched"), "no"); + + // The clone's workload finds the same, as user ID 0: the files that the mappings show, but + // not the hidden one, and the writable mappings alone take what it writes. + assert_eq!( + clone.script(EXPORTED), + [ + "read /srv/maps/dir/marker marker", + "read /srv/file.txt file", + "secret refused", + "listing inner,marker,", + "write /srv/maps refused", + "write /srv/maps/dir/inner written", + "write /srv/out written", + ] + ); + + // The host's directory holds what the guest wrote through the writable mappings, and nothing + // else changed, before the stop, which tears the root down, and after it. + let mut expected = tree_of(&EXPORT_DIRECTORIES, &EXPORT_FILES); + for (file, contents) in [ + ("dir/inner/written", "runtime-adapter"), + ("dir/inner/workload", "workload"), + ("out/written", "runtime-adapter"), + ("out/workload", "workload"), + ] { + expected.insert(file.to_owned(), Some(contents.to_owned())); + } + assert_eq!(export.tree(), expected); + clone.stop(); + assert_eq!(export.tree(), expected); + assert!(template.leases().is_empty()); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn clones_at_once_see_only_their_own_exports() { + let template = Template::create("cl-exports"); + let own = Export::write(&template, "own", &[("marker", "own")], &[]); + let common = Export::write(&template, "common", &[("marker", "common")], &[]); + let writable = own.options(&[(".", "/srv/data")], &[]); + let mut generated = writable.iter().map(String::as_str).collect::>(); + generated.extend(["--machine-id", "generated"]); + let read_only = common.options(&[(".", "/srv/data:ro")], &[]); + let read_only = read_only.iter().map(String::as_str).collect::>(); + // Four clones of one generation at once: one with a read-write export of its own and a + // generated machine ID, two over the same read-only export, over a scratch image and in RAM, + // and one without an export. + let starts = [ + ( + "cl-own", + StartOf { + options: &generated, + ..StartOf::default() + }, + ), + ( + "cl-common", + StartOf { + upper: Upper::Scratch, + options: &read_only, + ..StartOf::default() + }, + ), + ( + "cl-common-2", + StartOf { + options: &read_only, + ..StartOf::default() + }, + ), + ("cl-none", StartOf::default()), + ]; + let mut clones = starts.map(|(name, start)| template.start(name, start, 0)); + let [with_own, with_common, with_same, without] = &mut clones; + for clone in [&*with_own, &*with_common, &*with_same] { + clone.assert_mapped(1); + } + without.assert_started(); + + // Each record names its clone's export, a directory, by the identity of its plan, which the + // clones of the same export share, and one mapping; the clone without an export has none, and + // no mapping, and NVX restored it with no `--mount`, which leaves the template's filesystem + // slot dormant. NVX bound each directory to the slot with the access that its mapping needs. + let exported = |clone: &Cloned| { + let configuration = clone.configuration(); + let export = configuration + .export + .map(|export| (export.kind, export.plan)); + (export, configuration.mappings) + }; + let (own_export, own_mappings) = exported(with_own); + let (common_export, common_mappings) = exported(with_common); + let (own_export, common_export) = (own_export.unwrap(), common_export.unwrap()); + assert_eq!((own_export.0, own_mappings), (ExportKind::Directory, 1)); + assert_eq!( + (common_export.0, common_mappings), + (ExportKind::Directory, 1) + ); + assert_ne!(own_export.1, common_export.1); + assert_eq!(exported(with_same), (Some(common_export), 1)); + assert_eq!(exported(without), (None, 0)); + with_own.assert_vmm_mount(Some((&own, "rw"))); + with_common.assert_vmm_mount(Some((&common, "ro"))); + with_same.assert_vmm_mount(Some((&common, "ro"))); + without.assert_vmm_mount(None); + + // Each clone with an export mounted it read-write, even where every mapping of its table is + // read-only, as a cold launch's would not be, since OpenVMM keeps the export's access, and each + // mapping carries its own. Below the root, beside its mapping, the mount table shows the bind + // of a generated machine ID. Each shows its own directory, and takes what its access allows. + for (clone, marker, access, write, mounts) in [ + ( + &mut *with_own, + "own", + "rw", + "allowed", + "/etc/machine-id,/srv/data", + ), + (&mut *with_common, "common", "ro", "refused", "/srv/data"), + (&mut *with_same, "common", "ro", "refused", "/srv/data"), + ] { + let facts = clone.probe(&[MAP_DATA]); + let fact = |key: &str| facts[&format!("map.{key}")].as_str(); + assert_eq!(fact("export.count"), "1", "{facts:?}"); + let options = mount_options(fact("export.options")); + assert!( + options.is_superset(&BTreeSet::from(["rw", "nosuid", "nodev"])), + "{facts:?}" + ); + assert_eq!(fact("mounts"), mounts, "{facts:?}"); + assert!( + mount_options(fact("data.options")).contains(access), + "{facts:?}" + ); + let data = ["kind", "entries", "write"].map(|key| fact(&format!("data.{key}"))); + assert_eq!(data, ["directory", "marker", write], "{facts:?}"); + let size = format!("file {}", marker.len()); + let found = ["kind", "read"].map(|key| fact(&format!("marker.{key}"))); + assert_eq!(found, [size.as_str(), marker], "{facts:?}"); + } + // The clone without an export mounted none, and maps nothing. + let facts = without.probe(&[MAP_DATA]); + let fact = |key: &str| facts[&format!("map.{key}")].as_str(); + assert_eq!(fact("export.count"), "0", "{facts:?}"); + assert!(!facts.contains_key("map.export.type"), "{facts:?}"); + assert_eq!(fact("hostfs.mode"), "absent", "{facts:?}"); + assert_eq!(fact("mounts"), "", "{facts:?}"); + let found = ["data.options", "data.kind", "marker.kind"].map(fact); + assert_eq!(found, ["none", "absent", "absent"], "{facts:?}"); + + // The read-only bind of the generated machine ID holds even against the guest's root: the + // helper of the `entries` probe, which writes the root's own machine ID first, finds its file + // system read-only. + with_own + .sandbox + .run("probe", &[OsStr::new("entries")], FAILED); + let stderr = with_own.sandbox.stderr(); + assert!(stderr.contains("Read-only file system"), "{stderr}"); + + // The host's directories hold what the clone of the writable export wrote, and nothing else. + let written = [("marker", "own"), ("written", "runtime-adapter")]; + assert_eq!(own.tree(), tree_of(&[], &written)); + assert_eq!(common.tree(), tree_of(&[], &[("marker", "common")])); + for clone in &mut clones { + clone.stop(); + } + assert!(template.leases().is_empty()); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts() { + let template = Template::create("cl-unbound"); + let one = Export::write(&template, "one", &[("marker", "one")], &[]); + let two = Export::write(&template, "two", &[("marker", "two")], &[]); + // Two directories, neither inside the other, which the plan exports as two children of an + // aggregate export, and a file alone, which the plan exports as its one child. + let mut several = one.options(&[(".", "/srv/one")], &[]); + several.extend([ + "--share".to_owned(), + format!("{}:/srv/two", two.root.display()), + ]); + let file = one.options(&[("marker", "/srv/marker:ro")], &[]); + let slot = "the export of a clone binds one directory to its template's dormant filesystem \ + slot, but its export plan exports"; + for (name, options, problem) in [ + ( + "cl-several", + several, + format!("{slot} 2 directories and files"), + ), + ("cl-file", file, format!("{slot} the file ")), + ] { + let options = options.iter().map(String::as_str).collect::>(); + let start = StartOf { + options: &options, + ..StartOf::default() + }; + let refused = template.start(name, start, CLONE_FAILED); + refused.assert_refused(&problem); + let printed = &refused.started; + assert_eq!( + failure(printed), + [ + "request", + "invalid-mappings", + "plan", + "planned", + "none", + "no-vm" + ], + "{printed:?}" + ); + assert_eq!(printed["quarantined"], "no", "{printed:?}"); + } + assert!(template.leases().is_empty()); + assert_eq!(one.tree(), tree_of(&[], &[("marker", "one")])); + assert_eq!(two.tree(), tree_of(&[], &[("marker", "two")])); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_clone_whose_mapping_fails_ends_before_its_sandbox_starts() { + let template = Template::create("cl-mapfail"); + let export = Export::write(&template, "data", &[("out/marker", "marker")], &["out"]); + // The table's second mapping, after the directory's own, puts the directory `out` onto + // `/usr/lib/os-release`, a file of NVX's Ubuntu image, which the guest refuses once it + // applied the first. + let shares = export.options( + &[(".", "/srv/maps:ro"), ("out", "/usr/lib/os-release")], + &[], + ); + let options = shares.iter().map(String::as_str).collect::>(); + let start = StartOf { + upper: Upper::Scratch, + options: &options, + ..StartOf::default() + }; + let failed = template.start("cl-mapfail-1", start, CLONE_FAILED); + let printed = &failed.started; + + // The clone was ready, but the restore sequence sent no start once its guest failed the + // table: the failure is the start's, at the clone's Ready, by the evidence of the protocol, + // since the guest's refusal does not tell the request's mappings from a failure of its own. + // The clone order asked the guest to stop, which tears the root down, and NVX then ended the + // launch at once, which found the VM exited, with the guest's success, or terminated it, + // whether or not OpenVMM had reported the guest's success yet. + assert!(printed.contains_key("pid"), "{printed:?}"); + assert_eq!( + (printed["root"].as_str(), printed["build"].as_str()), + ("ready", GUEST_BUILD) + ); + for key in ["table", "mapped", "sandbox"] { + assert!(!printed.contains_key(key), "{printed:?}"); + } + let [outcome, evidence, stage, progress, status, teardown] = failure(printed); + assert_eq!( + [outcome, evidence, stage, progress], + ["unknown", "protocol", "start", "ready"], + "{printed:?}" + ); + assert!( + matches!( + (status, teardown), + ("0", "exited" | "terminated") | ("none", "terminated") + ), + "{printed:?}" + ); + let stderr = failed.sandbox.stderr(); + // The guest applied the directory's own mapping, the table's first, and refused the second: + // the guest's own report, which no other failure of the table prints. + assert!( + stderr.contains( + "the clone did not apply its mapping table: the mapping table failed after 1 \ + mapping: " + ), + "{stderr}" + ); + // Nothing of the clone runs: the failure quarantined nothing, and the start released the + // lease, with the records of the clone's launch. + assert_eq!(printed["quarantined"], "no", "{printed:?}"); + assert_eq!(printed["released"], printed["lease"], "{printed:?}"); + assert!(failed.sandbox.record().is_none()); + let store = SandboxStore::open(&failed.sandbox.store).unwrap(); + assert!(CloneRecords::read(&store).unwrap().is_none()); + assert!(template.leases().is_empty()); + // The guest wrote nothing to the host's directory. + assert_eq!( + export.tree(), + tree_of(&["out"], &[("out/marker", "marker")]) + ); + assert_payload(&template, false); +}