diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 043b84870..2fbd85d2d 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -574,15 +574,16 @@ template's capture, without I/O: the placeholder `distro`, which its over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's transaction directory, with the dormant filesystem slot, and no NIC or one under the template's policy, which allows neither ingress nor host-loopback access -both ways; `TemplateResources`, with every processor online at boot, and RAM and -a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line -`nvx_sandbox=1`, so that the capture's whole command line holds tokens of -OpenVMM's platform allowlist alone, and no tenant data, which the tests check -with a copy of the allowlist's source; and the capture's options, -`--snapshot-tier platform`, a `--snapshot-destination`, `SNAPSHOT_NAME` in the -transaction directory, and `--snapshot-block-identity generation` with the -capture's bound-block `StorageGeneration`, so that the capture hashes no block -device. +both ways; `TemplateResources`, with the processors, 1, 2, 4, or 8, and those +online at boot, as many or fewer, and RAM and a `--memory-capacity` in blocks of +`MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, with `maxcpus=` +and the processors online at boot when they are fewer, so that the capture's +whole command line holds tokens of OpenVMM's platform allowlist alone, and no +tenant data, which the tests check with a copy of the allowlist's source; and +the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, +`SNAPSHOT_NAME` in the transaction directory, and +`--snapshot-block-identity generation` with the capture's bound-block +`StorageGeneration`, so that the capture hashes no block device. `TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link @@ -593,8 +594,8 @@ identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the -processors and RAM, the geometries of the disks, the network, and the adapter's -other `Invariants`. +processors, those online at boot, and RAM, the geometries of the disks, the +network, and the adapter's other `Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: @@ -666,7 +667,9 @@ its key, which must speak the current contracts, the boot and control contracts of the runtime that captured it included, and its snapshot; the template's network exactly, as its key records it, or none for a template without a NIC, so that a template serves the clones of its network policy, at its guest's address, -which OpenVMM restores from the snapshot, alone; a memory target in whole blocks +which OpenVMM restores from the snapshot, alone; a processor target of 1, 2, 4, +or 8 from the template's processors online at boot to its capacity; a memory +target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the caller's scratch image, which the check of its `ScratchBinding` claims, and @@ -703,9 +706,9 @@ or the scratch image's binding identity with the journal policy; the export, a directory, with its plan's identity, and its mappings, the plan's binds, from 1 to 4,096, or no export and no mappings, which leaves the slot dormant; the template NIC's policy identity, as its key records it, unpublished listeners, -and the guest's proxy URL; and the template's processors and the memory target. -The identity of a binding or of an export plan is the SHA-256 digest of a domain -and of its serde form, never of a file's content. +and the guest's proxy URL; and the processor and memory targets. The identity of +a binding or of an export plan is the SHA-256 digest of a domain and of its +serde form, never of a file's content. The image must present the logical and physical block sizes of the template's placeholder and be no longer than it, since a restore presents every clone's @@ -734,7 +737,9 @@ key records it, unchanged, its export bound to the dormant slot, `--mount` with the directory, and its hidden, re-exposed, and writable paths and its owner, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, -`--restore-memory` with the memory target, and `--restore-ready-path`, under the +`--restore-memory` with the memory target, `--restore-processors` with the +processor target, for a template whose processors were not all online at boot, +and `--restore-ready-path`, under the template's backend, CPU profile, and processors, with a boot console that discards its output, as cold boots have, and the control console, but no kernel, initramfs, command line, or guest address, which the snapshot fixes. @@ -1207,17 +1212,19 @@ may fail in a clone whose restore it could not repair. Each failure is a `CaptureError`, with its `CaptureStage` and `CaptureCause`. `repair`, or a `Repair` that the caller may retry, repairs the clone of a -`Restore` next: it checks that the online CPUs are a prefix that meets the -restore's processor target, since it brings no CPU online; with a memory target, -it checks, before it adds any memory block, that the kernel's block size is 128 -MiB and that the restore's ranges continue the captured RAM in logical order, -with no gap, overlap, or range in the MMIO aperture, while the host bounds the -target by the snapshot's capacity, then adds and onlines the blocks through -sysfs, as `nvx-snapshot` does, and verifies them; it reseeds the kernel's random -number generator with the `Seed`, having drawn no randomness before, and drops -it; and it finishes the restore through `TimeAbi::restore_finish`, with the -acknowledgement when the host gates its input until then, which hands the -deferred time work to the time daemon that the caller supervises. +`Restore` next: it checks that the online CPUs are a prefix that the restore's +processor target does not undercut, brings the target's other CPUs online +through sysfs, and verifies them; with a memory target, it checks, before it +adds any memory block, that the kernel's block size is 128 MiB and that the +restore's ranges continue the captured RAM in logical order, with no gap, +overlap, or range in the MMIO aperture, while the host bounds the target by the +snapshot's capacity, then adds and onlines the blocks through sysfs, as +`nvx-snapshot` does, and verifies them; it reseeds the kernel's random number +generator with the `Seed`, having drawn no randomness before, and drops it; and +it finishes the restore through `TimeAbi::restore_finish`, with the CPUs that +the restore brought online and the acknowledgement when the host gates its input +until then, which hands the deferred time work to the time daemon that the +caller supervises. It returns the `Repaired` clone, with its generation ID, its online CPUs and memory, and when it acknowledged the restore, and never touches the fence. It diff --git a/nvx-rs/src/guest/clone.rs b/nvx-rs/src/guest/clone.rs index f3535d716..833917bab 100644 --- a/nvx-rs/src/guest/clone.rs +++ b/nvx-rs/src/guest/clone.rs @@ -51,8 +51,11 @@ //! runs one once. [`Repair::run`], in order: //! //! 1. checks that the restore holds its seed, and that its capture did not finish it; -//! 2. reads the online CPUs, which must be a prefix of the CPUs, and refuses a processor target -//! that they don't meet already: the repair brings no CPU online; +//! 2. reads the online CPUs, which must be a prefix of the CPUs, those that the clone captured +//! online. With a processor target, which may not be below them, it brings each CPU of the +//! target's prefix that is offline online, through sysfs, as `nvx-snapshot` does, and verifies +//! that the online CPUs are the target's prefix. Those are CPUs that the VM has had since its +//! boot, offline until then, so no device is plugged; //! 3. with a memory target, checks it before it adds any memory block: the kernel's memory block //! size must be 128 MiB, and the restore's ranges, aligned to it, must continue the RAM that //! the clone captured, in logical order: the first starts where the blocks that the kernel @@ -68,24 +71,25 @@ //! [`reseed::reseed`](crate::guest::reseed::reseed), then drops the seed, which zeroes its //! bytes. Nothing before draws randomness, from `getrandom` or from `/dev/urandom`; //! 5. finishes the restore through [`TimeAbi::restore_finish`], with the acknowledgement when -//! the host gates its input until then, which hands the deferred time work to the time -//! daemon: the restore checks, the stall detectors' grace period and release, and the -//! wall-clock discipline's fast cadence. The caller starts the daemon with -//! [`TimeAbi::start_daemon`] at boot, before the capture, and supervises it. A finish without -//! a running daemon fails with a violation of the time ABI's check C10, and in report-only -//! mode, it runs the work itself before it returns. The work may end after the clone is -//! ready, and its failures are fatal repair failures: the daemon reports them as violations -//! of the time ABI, which stop the VM, with the time ABI's status, unless the ABI is -//! report-only, since the daemon enforces it. +//! the host gates its input until then, and the CPUs that it brought online, which hands the +//! deferred time work to the time daemon: the restore checks, of the new CPUs too, the stall +//! detectors' grace period and release, and the wall-clock discipline's fast cadence. The +//! caller starts the daemon with [`TimeAbi::start_daemon`] at boot, before the capture, and +//! supervises it. A finish without a running daemon fails with a violation of the time ABI's +//! check C10, and in report-only mode, it runs the work itself before it returns. The work +//! may end after the clone is ready, and its failures are fatal repair failures: the daemon +//! reports them as violations of the time ABI, which stop the VM, with the time ABI's status, +//! unless the ABI is report-only, since the daemon enforces it. //! //! It returns the [`Repaired`] clone: its new VM generation ID, its number of online CPUs, the //! size of its online memory blocks, and when it acknowledged the restore, which an activation //! checks against the resources that it expects. The repair records how far it got, so that it -//! never adds a memory block twice, and runs the reseed and the finish at most once, whoever -//! retries it: a failure at or after the reseed is final. It never touches the fence. A failure is -//! a [`RepairError`], which names the [`RepairStage`] that failed, its [`RepairCause`], and the -//! repair's [`RepairProgress`], and selects the status with which the caller ends the VM, -//! [`RepairError::status`], which `nvx::protocol::status` documents. +//! never brings a CPU online twice nor adds a memory block twice, and runs the reseed and the +//! finish at most once, whoever retries it: a failure at or after the reseed is final. It never +//! touches the fence. A failure is a [`RepairError`], which names the [`RepairStage`] that +//! failed, its [`RepairCause`], and the repair's [`RepairProgress`], and selects the status with +//! which the caller ends the VM, [`RepairError::status`], which `nvx::protocol::status` +//! documents. //! //! # Strict mode //! @@ -256,14 +260,14 @@ //! reseed needs `CAP_SYS_ADMIN`. The capture reads the mount table of the caller's mount //! namespace, which must be the one where the guest mounts its devices, such as its initial //! namespace, and the repair reads the kernel's CPU and memory interfaces in sysfs, and writes -//! its memory interface. They mount nothing and start no thread, and they start no process and -//! collect no child's status, except a report-only finish without a running time daemon, which -//! runs the deferred time work itself, in a child that it waits for, as -//! [`TimeAbi::restore_finish`] tells. Apart from a violation of the time ABI in the capture's -//! preparation, which stops the VM unless the ABI is report-only, as [strict -//! mode](#strict-mode) tells, they never power the VM off, so a caller that is not PID 1 keeps -//! its own reaping, transport, and shutdown. The activation needs no privilege, and makes no -//! system call: of the clone, it changes only the caller's fence, which it opens. Its +//! the states of the CPUs that it brings online and its memory interface. They mount nothing and +//! start no thread, and they start no process and collect no child's status, except a +//! report-only finish without a running time daemon, which runs the deferred time work itself, +//! in a child that it waits for, as [`TimeAbi::restore_finish`] tells. Apart from a violation of +//! the time ABI in the capture's preparation, which stops the VM unless the ABI is report-only, +//! as [strict mode](#strict-mode) tells, they never power the VM off, so a caller that is not +//! PID 1 keeps its own reaping, transport, and shutdown. The activation needs no privilege, and +//! makes no system call: of the clone, it changes only the caller's fence, which it opens. Its //! application, [`apply`](fn@apply), needs the guest's root user, in the guest's initial mount //! namespace, as [`root::assemble`] does: for a GPT disk, it asks the kernel to reread the disk's //! partition table, which needs `CAP_SYS_ADMIN`; it mounts the root, reads the account databases diff --git a/nvx-rs/src/guest/clone/repair.rs b/nvx-rs/src/guest/clone/repair.rs index 8b32eb67d..ce2193660 100644 --- a/nvx-rs/src/guest/clone/repair.rs +++ b/nvx-rs/src/guest/clone/repair.rs @@ -20,7 +20,7 @@ use crate::guest::parse::{parse_decimal, parse_hexadecimal}; use crate::guest::reseed::{self, ReseedError, Seed}; use crate::guest::root::Paths; use crate::guest::sys; -use crate::guest::time::{MemoryRange, Restore, RestoreFinish, TimeAbi, TimeError}; +use crate::guest::time::{CpuSet, MemoryRange, Restore, RestoreFinish, TimeAbi, TimeError}; use crate::protocol::status; /// The memory block size of the snapshot ABI, 128 MiB, to which a restore's memory ranges are @@ -43,8 +43,15 @@ const PROBE_FILE: &str = "probe"; const BLOCK_PREFIX: &str = "memory"; /// The state of a memory block, such as `online` or `offline`, in the block's directory. const STATE_FILE: &str = "state"; +/// The directory of the kernel's CPUs, below sysfs. +const CPUS: &str = "devices/system/cpu"; /// The CPUs that are online, as a CPU list, below sysfs. const ONLINE_CPUS: &str = "devices/system/cpu/online"; +/// The name of a CPU's directory, which the CPU's number follows, in the directory of the CPUs. +const CPU_PREFIX: &str = "cpu"; +/// The state of a CPU, `1` online and `0` offline, in the CPU's directory, to which `1` is +/// written to bring the CPU online. +const CPU_ONLINE_FILE: &str = "online"; /// The state of a memory block that is online. const ONLINE: &[u8] = b"online"; @@ -58,16 +65,17 @@ pub fn repair(restore: Restore) -> Result { } /// The repair of a clone that a restore started, of the [`Restore`] that the clone's capture -/// returned, as the [module](super) describes: its memory target, its reseed, and its finish, -/// with the acknowledgement that the host awaits. +/// returned, as the [module](super) describes: its processor target, its memory target, its +/// reseed, and its finish, with the acknowledgement that the host awaits. /// /// [`run`](Self::run) records how far the repair got, so that the repair never adds a memory /// block twice, never reseeds twice, and never finishes the restore twice, whoever retries it: /// /// - Until the memory target is active, a failed attempt may be retried, and the next one -/// repeats the checks and adds or onlines only the memory blocks that are not online yet. The -/// first attempt derives the end of the RAM that the clone captured, before it adds any memory -/// block, and every later one keeps it. +/// repeats the checks, brings online only the CPUs of the processor target that are not online +/// yet, and adds or onlines only the memory blocks that are not online yet. The first attempt +/// records the CPUs that are online before it brings any online, and derives the end of the RAM +/// that the clone captured, before it adds any memory block, and every later one keeps both. /// - The reseed and the finish run at most once. A failure of either is final: every later call /// returns its error again. /// - Once the repair completed, every later call returns the same [`Repaired`]. @@ -84,6 +92,9 @@ pub struct Repair { seed: Option, /// Where the repair resumes. state: State, + /// The number of CPUs that were online before the repair brought any online, a prefix of the + /// CPUs, which its first attempt that reads them records. + captured_cpus: Option, /// The memory blocks that the repair added and onlined, across its attempts. activations: Activations, /// How far the repair got, which its errors report. @@ -112,6 +123,7 @@ impl Repair { restore, seed, state: State::Memory { captured: None }, + captured_cpus: None, activations: Activations::default(), progress: RepairProgress::default(), system, @@ -123,12 +135,14 @@ impl Repair { /// /// A failure names its [`RepairStage`], its [`RepairCause`], and the repair's /// [`RepairProgress`]. One before the reseed may be retried, and changes nothing that a retry - /// undoes: a refused restore or memory target adds no memory block, and a failed activation - /// leaves the blocks that it added and onlined, which a retry counts as done. The reseed and - /// the finish run at most once, so a failure of either, or of a finish that never returned, - /// such as after a panic, is final, and every later call returns that error again. A failed - /// finish may have acknowledged the restore already. Once the repair completed, every later - /// call returns the same [`Repaired`], without another step. + /// undoes: a refused restore or processor target brings no CPU online and adds no memory + /// block, a refused memory target adds no memory block, although the activation of the + /// processor target before it may have brought CPUs online, and a failed activation leaves + /// the CPUs that it brought online and the blocks that it added and onlined, which a retry + /// counts as done. The reseed and the finish run at most once, so a failure of either, or of a + /// finish that never returned, such as after a panic, is final, and every later call returns + /// that error again. A failed finish may have acknowledged the restore already. Once the + /// repair completed, every later call returns the same [`Repaired`], without another step. pub fn run(&mut self) -> Result { self.resume().map_err(|error| RepairError { progress: self.progress, @@ -160,8 +174,8 @@ impl Repair { } } - /// Checks the restore and the online CPUs, and activates the memory target: the steps before - /// the reseed, which a failure leaves to a retry. + /// Checks the restore, and activates the processor and memory targets: the steps before the + /// reseed, which a failure leaves to a retry. fn activate(&mut self) -> Result { if self.restore.finished() { return Err(RepairError::new( @@ -172,26 +186,58 @@ impl Repair { if self.seed.is_none() { return Err(RepairError::new(RepairStage::Restore, RepairCause::NoSeed)); } - let cpus = self - .check_processors() + let (cpus, new_cpus) = self + .activate_processors() .map_err(|cause| RepairError::new(RepairStage::Processors, cause))?; let memory = self .activate_memory() .map_err(|cause| RepairError::new(RepairStage::Memory, cause))?; - Ok(Resources { cpus, memory }) + Ok(Resources { + cpus, + new_cpus, + memory, + }) } - /// Returns the number of online CPUs, which must be a prefix of the CPUs, and must meet the - /// restore's processor target, if it has one, since the repair brings no CPU online. - fn check_processors(&mut self) -> Result { - let text = self.system.online_cpus().map_err(RepairCause::Io)?; - let online = online_prefix(&text).ok_or_else(|| RepairCause::OnlineCpus(lossy(&text)))?; - if let Some(target) = self.restore.processor_target() - && u32::from(target) != online - { + /// Activates the restore's processor target, if it has one, and returns the number of online + /// CPUs, which must be a prefix of the CPUs, with the CPUs that the repair brought online. + /// + /// A target below the CPUs that are online is refused, since the repair takes no CPU + /// offline. The repair brings online each CPU of the target's prefix that is offline, through + /// sysfs, as `nvx-snapshot` does, then verifies that the online CPUs are the target's prefix. + /// The CPUs that it brought online are those of the prefix beyond the ones that were online + /// before its first attempt, which the clone captured. + fn activate_processors(&mut self) -> Result<(u32, CpuSet), RepairCause> { + let online = self.online_cpus()?; + let captured = *self.captured_cpus.get_or_insert(online); + let Some(target) = self.restore.processor_target() else { + return Ok((online, CpuSet::new())); + }; + let count = u32::from(target); + if count < online { return Err(RepairCause::ProcessorTarget { target, online }); } - Ok(online) + for cpu in online..count { + self.system.online_cpu(cpu).map_err(RepairCause::Io)?; + } + let text = self.system.online_cpus().map_err(RepairCause::Io)?; + if online_prefix(&text) != Some(count) { + return Err(RepairCause::Activation { + target, + online: lossy(&text), + }); + } + // A processor target is at most 8 CPUs, which the time ABI covers. + let new_cpus = (captured..count).fold(CpuSet::new(), |cpus, cpu| { + cpus.with(cpu as usize).unwrap_or(cpus) + }); + Ok((count, new_cpus)) + } + + /// Returns the number of online CPUs, which must be a prefix of the CPUs. + fn online_cpus(&mut self) -> Result { + let text = self.system.online_cpus().map_err(RepairCause::Io)?; + online_prefix(&text).ok_or_else(|| RepairCause::OnlineCpus(lossy(&text))) } /// Activates the restore's memory target, if it has one, and returns the memory size of the @@ -272,10 +318,11 @@ impl Repair { } /// Finishes the restore, with the acknowledgement if the host gates its input until then, - /// once, and returns the repaired clone, which has `resources`. + /// once, and returns the repaired clone, which has `resources`. The finish records the CPUs + /// that the repair brought online, which the time ABI's restore checks check too. fn finish(&mut self, resources: Resources) -> Result { let acknowledge = self.restore.acknowledgement_required(); - let mut finish = RestoreFinish::new(); + let mut finish = RestoreFinish::new().new_cpus(resources.new_cpus); if acknowledge { finish = finish.acknowledge(); } @@ -338,6 +385,8 @@ enum State { struct Resources { /// The number of online CPUs. cpus: u32, + /// The CPUs that the repair brought online. + new_cpus: CpuSet, /// The memory size of the online memory blocks, in bytes. memory: u64, } @@ -477,7 +526,8 @@ impl fmt::Display for RepairProgress { pub enum RepairStage { /// Checking that the restore holds its seed, and that its capture did not finish it. Restore, - /// Checking the online CPUs against the restore's processor target. + /// Checking the online CPUs, bringing those of the restore's processor target online, and + /// verifying them. Processors, /// Checking the restore's memory target, adding and onlining its memory blocks, and /// verifying them. @@ -494,7 +544,7 @@ impl fmt::Display for RepairStage { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { formatter.write_str(match self { Self::Restore => "check the restore", - Self::Processors => "check the online CPUs", + Self::Processors => "activate the processor target", Self::Memory => "activate the memory target", Self::Reseed => "reseed the kernel's random number generator", Self::Finish => "finish the restore", @@ -547,14 +597,22 @@ pub enum RepairCause { Finished, /// The online CPUs, as the kernel lists them, are not a prefix of the CPUs, such as `0-3`. OnlineCpus(String), - /// The restore asks for another number of online CPUs, which needs CPUs brought online, - /// which the repair doesn't do. + /// The restore asks for fewer online CPUs than are online, which the repair doesn't take + /// offline. ProcessorTarget { /// The number of online CPUs that the restore asks for. target: u8, /// The number of online CPUs. online: u32, }, + /// The online CPUs are not the processor target's prefix of the CPUs once the repair brought + /// those of the target online. + Activation { + /// The number of online CPUs that the restore asks for. + target: u8, + /// The online CPUs, as the kernel lists them. + online: String, + }, /// The kernel's memory block size, which it writes in hexadecimal, is not 128 MiB, as the /// restore's memory target needs, or is malformed. BlockSize(String), @@ -605,6 +663,10 @@ impl fmt::Display for RepairCause { formatter, "the restore asks for {target} online CPUs, and {online} are online" ), + Self::Activation { target, online } => write!( + formatter, + "the online CPUs are {online} once the first {target} were brought online" + ), Self::BlockSize(size) => write!( formatter, "the memory block size is {size} in hexadecimal, not 128 MiB" @@ -855,6 +917,9 @@ trait RepairSystem: Send { /// Reads the CPUs that are online, as the kernel lists them. fn online_cpus(&mut self) -> io::Result>; + /// Brings the CPU `cpu` online. + fn online_cpu(&mut self, cpu: u32) -> io::Result<()>; + /// Reads the memory block size, which the kernel writes in hexadecimal. fn block_size(&mut self) -> io::Result>; @@ -903,6 +968,16 @@ impl RepairSystem for LiveSystem { read(&self.paths.sys.join(ONLINE_CPUS)) } + fn online_cpu(&mut self, cpu: u32) -> io::Result<()> { + let path = self + .paths + .sys + .join(CPUS) + .join(format!("{CPU_PREFIX}{cpu}")) + .join(CPU_ONLINE_FILE); + write(&path, b"1\n") + } + fn block_size(&mut self) -> io::Result> { read(&self.memory(BLOCK_SIZE_FILE)) } diff --git a/nvx-rs/src/guest/clone/repair/tests.rs b/nvx-rs/src/guest/clone/repair/tests.rs index ec3c3412e..e8a94aa15 100644 --- a/nvx-rs/src/guest/clone/repair/tests.rs +++ b/nvx-rs/src/guest/clone/repair/tests.rs @@ -30,6 +30,8 @@ const OFFLINE_STATE: &[u8] = b"offline\n"; enum Call { /// The online CPUs were read. OnlineCpus, + /// The CPU of this number was brought online. + OnlineCpu(u32), /// The memory block size was read. BlockSize, /// The memory blocks were listed. @@ -69,13 +71,18 @@ impl Calls { /// A system that records its calls, except its reads of the blocks' states, with CPUs, memory /// blocks, a reseed, and a time ABI that a test chooses: by default, two online CPUs, 128 MiB /// memory blocks, the four online blocks of a clone that captured 512 MiB of RAM, and a reseed -/// and a finish that succeed. It opens no file and draws no randomness: it keeps its blocks in -/// ordered maps, which draw no random keys, as hashed maps would. +/// and a finish that succeed. Bringing online the CPU that follows a prefix of online CPUs +/// extends the prefix. It opens no file and draws no randomness: it keeps its blocks in ordered +/// maps, which draw no random keys, as hashed maps would. struct FakeSystem { /// The log. calls: Calls, /// The online CPUs, as the kernel lists them. - cpus: &'static [u8], + cpus: Vec, + /// The error numbers with which the next onlining of each CPU fails, once. + cpu_failures: BTreeMap, + /// A CPU that the kernel leaves offline when it is brought online. + stuck_cpu: Option, /// The memory block size, as the kernel writes it. block_size: &'static [u8], /// The state of each memory block that the kernel has, by the block's number. @@ -110,7 +117,9 @@ impl FakeSystem { fn new(calls: &Calls) -> Self { Self { calls: calls.clone(), - cpus: b"0-1\n", + cpus: b"0-1\n".to_vec(), + cpu_failures: BTreeMap::new(), + stuck_cpu: None, block_size: b"8000000\n", blocks: (0..4).map(|block| (block, ONLINE_STATE)).collect(), add_failures: BTreeMap::new(), @@ -139,7 +148,18 @@ impl FakeSystem { impl RepairSystem for FakeSystem { fn online_cpus(&mut self) -> io::Result> { self.calls.push(Call::OnlineCpus); - Ok(self.cpus.to_vec()) + Ok(self.cpus.clone()) + } + + fn online_cpu(&mut self, cpu: u32) -> io::Result<()> { + self.calls.push(Call::OnlineCpu(cpu)); + if let Some(errno) = self.cpu_failures.remove(&cpu) { + return Err(errno.into()); + } + if self.stuck_cpu != Some(cpu) && online_prefix(&self.cpus) == Some(cpu) { + self.cpus = format!("0-{cpu}\n").into_bytes(); + } + Ok(()) } fn block_size(&mut self) -> io::Result> { @@ -267,6 +287,12 @@ fn changes(calls: Vec) -> Vec { .collect() } +/// Returns the set of `cpus`. +fn cpu_set(cpus: impl IntoIterator) -> CpuSet { + cpus.into_iter() + .fold(CpuSet::new(), |set, cpu| set.with(cpu).unwrap()) +} + #[test] fn a_clone_is_reseeded_then_acknowledged_once_and_reentry_returns_the_same_clone() { let calls = Calls::default(); @@ -322,6 +348,10 @@ impl RepairSystem for Deferring { self.system.online_cpus() } + fn online_cpu(&mut self, cpu: u32) -> io::Result<()> { + self.system.online_cpu(cpu) + } + fn block_size(&mut self) -> io::Result> { self.system.block_size() } @@ -699,11 +729,15 @@ fn each_failure_class_ends_the_clone_with_a_status_of_its_own() { )); // The processors. let mut unlisted = system(); - unlisted.cpus = b"1-3\n"; + unlisted.cpus = b"1-3\n".to_vec(); let error = failure(restore(), unlisted); cases.push(("online CPUs", error, status::REPAIR_PROCESSORS)); - let error = failure(restore().with_processor_target(4), system()); + let error = failure(restore().with_processor_target(1), system()); cases.push(("a processor target", error, status::REPAIR_PROCESSORS)); + let mut stuck_cpu = system(); + stuck_cpu.stuck_cpu = Some(3); + let error = failure(restore().with_processor_target(4), stuck_cpu); + cases.push(("an offline CPU", error, status::REPAIR_PROCESSORS)); // The memory target, which the repair refuses before it adds any memory block. let mut wide = system(); wide.block_size = b"40000000\n"; @@ -888,17 +922,20 @@ fn a_block_counts_once_the_repair_finds_it_added_or_online() { } /// Checks that the repair of a clone whose kernel lists the online CPUs `cpus`, and whose restore -/// has the processor target `target`, if any, counts `online` CPUs. -fn counts(cpus: &'static [u8], target: Option, online: u32) { +/// has the processor target `target`, if any, counts `online` CPUs, and finishes the restore +/// with `new_cpus`, the CPUs that it brought online. +fn counts(cpus: &'static [u8], target: Option, online: u32, new_cpus: CpuSet) { let calls = Calls::default(); let mut system = FakeSystem::new(&calls); - system.cpus = cpus; + system.cpus = cpus.to_vec(); let restore = match target { Some(target) => restore().with_processor_target(target), None => restore(), }; let repaired = repairing(restore, system).run().unwrap(); assert_eq!(repaired.cpus(), online, "{cpus:?}"); + let finish = RestoreFinish::new().acknowledge().new_cpus(new_cpus); + assert_eq!(calls.take().last(), Some(&Call::Finish(finish)), "{cpus:?}"); } /// Checks that the repair of a clone whose kernel lists the online CPUs `cpus`, and whose restore @@ -906,7 +943,7 @@ fn counts(cpus: &'static [u8], target: Option, online: u32) { fn refuses_cpus(cpus: &'static [u8], target: Option, cause: &RepairCause) { let calls = Calls::default(); let mut system = FakeSystem::new(&calls); - system.cpus = cpus; + system.cpus = cpus.to_vec(); let restore = match target { Some(target) => restore().with_processor_target(target), None => restore(), @@ -918,21 +955,153 @@ fn refuses_cpus(cpus: &'static [u8], target: Option, cause: &RepairCause) { } #[test] -fn the_online_cpus_must_be_a_prefix_that_meets_the_processor_target() { - counts(b"0\n", None, 1); - counts(b"0-3\n", None, 4); - counts(b"0-3\n", Some(4), 4); - // The repair brings no CPU online, nor takes one offline. +fn the_online_cpus_must_be_a_prefix_that_the_processor_target_does_not_undercut() { + counts(b"0\n", None, 1, CpuSet::new()); + counts(b"0-3\n", None, 4, CpuSet::new()); + counts(b"0-3\n", Some(4), 4, CpuSet::new()); + // The repair takes no CPU offline. let cause = |target, online| RepairCause::ProcessorTarget { target, online }; - refuses_cpus(b"0-3\n", Some(8), &cause(8, 4)); refuses_cpus(b"0-3\n", Some(2), &cause(2, 4)); + refuses_cpus(b"0-7\n", Some(1), &cause(1, 8)); let lists: [&'static [u8]; 4] = [b"0-1,3\n", b"1-3\n", b"0-\n", b"\n"]; for cpus in lists { let listed = String::from_utf8_lossy(cpus.strip_suffix(b"\n").unwrap()).into_owned(); - refuses_cpus(cpus, None, &RepairCause::OnlineCpus(listed)); + refuses_cpus(cpus, None, &RepairCause::OnlineCpus(listed.clone())); + refuses_cpus(cpus, Some(8), &RepairCause::OnlineCpus(listed)); } } +#[test] +fn a_processor_target_brings_the_offline_cpus_of_its_prefix_online_and_finishes_with_them() { + counts(b"0\n", Some(2), 2, cpu_set([1])); + counts(b"0-1\n", Some(8), 8, cpu_set(2..8)); + // The repair brings each CPU online in order, before the memory target, and verifies them. + let calls = Calls::default(); + let mut system = FakeSystem::new(&calls); + system.cpus = b"0\n".to_vec(); + let target = restore().with_processor_target(4); + let repaired = repairing(target, system).run().unwrap(); + assert_eq!(repaired.cpus(), 4); + let mut expected = vec![ + Call::OnlineCpus, + Call::OnlineCpu(1), + Call::OnlineCpu(2), + Call::OnlineCpu(3), + Call::OnlineCpus, + Call::BlockSize, + ]; + let mut finished = finishing(); + finished[3] = Call::Finish(RestoreFinish::new().acknowledge().new_cpus(cpu_set(1..4))); + expected.extend(finished); + assert_eq!(calls.take(), expected); +} + +#[test] +fn a_retry_brings_online_only_the_cpus_that_are_not_and_reports_every_new_one() { + let calls = Calls::default(); + let mut system = FakeSystem::new(&calls); + system.cpus = b"0\n".to_vec(); + system.cpu_failures.insert(2, Errno::EBUSY); + let mut repair = repairing(restore().with_processor_target(4), system); + // The first attempt brought CPU 1 online before CPU 2 failed. + let error = repair.run().unwrap_err(); + assert_eq!(error.stage(), RepairStage::Processors); + assert!(matches!( + error.cause(), + RepairCause::Io(error) if error.raw_os_error() == Some(Errno::EBUSY as i32) + )); + assert_eq!(error.status(), status::REPAIR_PROCESSORS); + assert_eq!( + changes(calls.take()), + [Call::OnlineCpu(1), Call::OnlineCpu(2)] + ); + // The retry brings CPUs 2 and 3 online, and the finish names the three that the repair + // brought online. + assert_eq!(repair.run().unwrap().cpus(), 4); + let calls = changes(calls.take()); + assert_eq!(calls[..2], [Call::OnlineCpu(2), Call::OnlineCpu(3)]); + let finish = RestoreFinish::new().acknowledge().new_cpus(cpu_set(1..4)); + assert_eq!(calls.last(), Some(&Call::Finish(finish))); +} + +#[test] +fn a_cpu_that_stays_offline_fails_the_activation_of_the_processor_target() { + let calls = Calls::default(); + let mut system = FakeSystem::new(&calls); + system.stuck_cpu = Some(2); + let error = failure(restore().with_processor_target(4), system); + assert_eq!(error.stage(), RepairStage::Processors); + assert_eq!( + error.to_string(), + "failed to activate the processor target: the online CPUs are 0-1 once the first 4 \ + were brought online" + ); + // The repair went no further. + assert_eq!( + calls.take(), + [ + Call::OnlineCpus, + Call::OnlineCpu(2), + Call::OnlineCpu(3), + Call::OnlineCpus + ] + ); +} + +#[test] +fn a_memory_target_that_fails_after_the_processor_target_leaves_its_cpus_online() { + // A refused memory target adds no memory block, but the processor target before it is + // active already, and stays so. + let calls = Calls::default(); + let mut system = FakeSystem::new(&calls); + system.cpus = b"0\n".to_vec(); + system.block_size = b"40000000\n"; + let targets = restore() + .with_processor_target(2) + .with_memory_target(&[(512 * MIB, 512 * MIB)]); + let error = repairing(targets, system).run().unwrap_err(); + assert_eq!(error.stage(), RepairStage::Memory); + assert_eq!(error.status(), status::REPAIR_MEMORY_TARGET); + assert_eq!( + calls.take(), + [ + Call::OnlineCpus, + Call::OnlineCpu(1), + Call::OnlineCpus, + Call::BlockSize + ] + ); + + // A retry after a failed activation of the memory target brings no CPU online again, and the + // finish names every CPU that the repair brought online. + let mut system = FakeSystem::new(&calls); + system.cpus = b"0\n".to_vec(); + system.add_failures.insert(512 * MIB, Errno::EIO); + let targets = restore() + .with_processor_target(4) + .with_memory_target(&[(512 * MIB, 128 * MIB)]); + let mut repair = repairing(targets, system); + let error = repair.run().unwrap_err(); + assert_eq!(error.status(), status::REPAIR_MEMORY); + assert_eq!( + changes(calls.take()), + [ + Call::OnlineCpu(1), + Call::OnlineCpu(2), + Call::OnlineCpu(3), + Call::MemoryBlocks, + Call::Add(512 * MIB) + ] + ); + let repaired = repair.run().unwrap(); + assert_eq!((repaired.cpus(), repaired.memory()), (4, 640 * MIB)); + let mut expected = activations([4]); + let mut finished = finishing(); + finished[3] = Call::Finish(RestoreFinish::new().acknowledge().new_cpus(cpu_set(1..4))); + expected.extend(finished); + assert_eq!(changes(calls.take()), expected); +} + #[test] fn a_restore_without_its_seed_or_that_its_capture_finished_is_refused() { let calls = Calls::default(); @@ -994,12 +1163,12 @@ fn errors_name_their_stage_and_their_cause() { RepairError::new( RepairStage::Processors, RepairCause::ProcessorTarget { - target: 8, + target: 2, online: 4, }, ), - "failed to check the online CPUs: the restore asks for 8 online CPUs, and 4 are \ - online", + "failed to activate the processor target: the restore asks for 2 online CPUs, and 4 \ + are online", ), ( RepairError::new(RepairStage::Restore, RepairCause::NoSeed), @@ -1081,7 +1250,9 @@ fn write_file(path: &Path, contents: &str) { fn the_live_system_reads_and_writes_the_kernel_s_cpu_and_memory_interfaces() { let root = tempfile::tempdir().unwrap(); let memory = root.path().join(MEMORY_BLOCKS); + let cpus = root.path().join(CPUS); write_file(&root.path().join(ONLINE_CPUS), "0-3\n"); + write_file(&cpus.join("cpu4/online"), ""); write_file(&memory.join(BLOCK_SIZE_FILE), "8000000\n"); write_file(&memory.join(PROBE_FILE), ""); write_file(&memory.join("memory0/state"), "online\n"); @@ -1100,6 +1271,15 @@ fn the_live_system_reads_and_writes_the_kernel_s_cpu_and_memory_interfaces() { abi: TimeAbi::report_only(), }; assert_eq!(system.online_cpus().unwrap(), b"0-3\n"); + // Bringing a CPU online writes its new state, as `nvx-snapshot` does; a CPU without a state + // file cannot be brought online, and the error names the file. + system.online_cpu(4).unwrap(); + assert_eq!(fs::read(cpus.join("cpu4/online")).unwrap(), b"1\n"); + let error = system.online_cpu(5).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::NotFound); + assert!(!cpus.join("cpu5/online").exists()); + let named = format!("{}: ", cpus.join("cpu5/online").display()); + assert!(error.to_string().starts_with(&named), "{error}"); assert_eq!(system.block_size().unwrap(), b"8000000\n"); assert_eq!(system.memory_blocks().unwrap(), [0, 2, 12]); assert_eq!(system.block_state(12).unwrap().unwrap(), b"offline\n"); diff --git a/nvx-rs/src/host/template/clone.rs b/nvx-rs/src/host/template/clone.rs index 542417467..0e4e3ea64 100644 --- a/nvx-rs/src/host/template/clone.rs +++ b/nvx-rs/src/host/template/clone.rs @@ -1,8 +1,8 @@ //! The plan of a clone, [`ClonePlan`]: the generation of a template that a clone leased, with the -//! clone's own image, upper, memory target, and export, which the plan checks against the -//! generation before the clone's launch, so that a request that the template cannot serve fails -//! before any VMM starts; and the clone's launch, [`ClonePlan::launch`], OpenVMM's restore of the -//! generation's snapshot with what the plan checked. +//! clone's own image, upper, processor and memory targets, and export, which the plan checks +//! against the generation before the clone's launch, so that a request that the template cannot +//! serve fails before any VMM starts; and the clone's launch, [`ClonePlan::launch`], OpenVMM's +//! restore of the generation's snapshot with what the plan checked. #[cfg(all(test, any(target_os = "linux", windows)))] mod tests; @@ -36,7 +36,7 @@ use std::time::Instant; use self::export::DirectoryExport; use self::failure::{Attribute, Fault, host_evidence}; use super::key::TemplateKey; -use super::manifest::BlockGeometry; +use super::manifest::{BlockGeometry, PROCESSOR_COUNTS}; use super::plan::MEMORY_BLOCK_SIZE; use super::store::{Acquired, Generation, Lease, Store}; use crate::activation::{ActivationRecord, Journal, OverlayMode, WorkloadView}; @@ -68,10 +68,11 @@ const MAX_PATH_LINKS: usize = 64; /// What the clone of a sandbox restores of the generation of a template that it leased: the /// adapter's plan of the clone's image and upper, the binding of the image, the binding of the /// scratch image of an upper on one, the export of the host paths that the clone shares with its -/// root, if any, and the memory target, which [`ClonePlan::new`] checks against the generation, -/// and the settings of the clone's activation that the template does not fix, which the clone's -/// [activation record](ClonePlan::activation) names: the workload's view of its VM, the overlay's -/// mode, the journal policy of a scratch image, and the bound of a GPT disk's first partition. +/// root, if any, and the processor and memory targets, which [`ClonePlan::new`] checks against the +/// generation, and the settings of the clone's activation that the template does not fix, which +/// the clone's [activation record](ClonePlan::activation) names: the workload's view of its VM, +/// the overlay's mode, the journal policy of a scratch image, and the bound of a GPT disk's first +/// partition. #[derive(Debug, Clone)] pub struct CloneRequest { /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the clone's image, its @@ -98,6 +99,10 @@ pub struct CloneRequest { /// dormant. The slot binds one directory, so the plan must export exactly one, and no file, /// as [`ClonePlan`] describes. pub export: Option, + /// The processor target: the processors online in the clone's VM once restored, a prefix of + /// the template's processor capacity, of which the restore brings online those beyond the + /// template's processors online at boot. + pub processors: u32, /// The memory target, in bytes: the RAM of the clone's VM once restored, to which the restore /// expands the template's RAM in whole blocks. pub memory: u64, @@ -187,6 +192,10 @@ pub enum CloneUpper { /// held open, and other names, but not a mount or a network share that exposes the store's /// directories elsewhere: a store's directories must not be exposed that way. The plan holds /// the checked image, and with it the claim. Nothing reads, formats, or copies the image. +/// - The processor target must be 1, 2, 4, or 8, from the template's processors online at boot to +/// its processor capacity, as its key records them, [`TemplateKey::boot_processors`] and +/// [`TemplateKey::processors`]. A template whose processors are all online at boot records none +/// online at boot, and serves the target of its capacity alone. /// - The memory target must be whole blocks of [`MEMORY_BLOCK_SIZE`], and of the template's own /// memory blocks, from the template's RAM to its RAM capacity. /// - The network of the adapter's plan must be the template's: lowered to OpenVMM's arguments, as @@ -265,6 +274,7 @@ pub enum CloneUpper { /// policy_revision: DISTRO_POLICY_REVISION, /// scratch: None, /// export: None, +/// processors: 2, /// memory: 8 * MEMORY_BLOCK_SIZE, /// workload: WorkloadView { /// hostname: b"clone".to_vec(), @@ -301,6 +311,8 @@ pub struct ClonePlan { image: CheckedDistro, /// The clone's upper, with the file of its `scratch` block device. upper: CloneUpper, + /// The processor target. + processors: u32, /// The memory target, in bytes. memory: u64, /// The clone's activation record. @@ -398,9 +410,10 @@ impl ClonePlan { /// [`TemplateKey::network_arguments`]: a network for a template without a NIC, none for a /// template with a NIC, or one with another guest network than the template's NIC, whose /// address OpenVMM restores from the snapshot, or another policy, - /// [`Evidence::InvalidSettings`]; and a memory target that is not whole blocks of - /// [`MEMORY_BLOCK_SIZE`] and of the template's memory blocks, from the template's RAM to its - /// RAM capacity, [`Evidence::InvalidTargets`]; + /// [`Evidence::InvalidSettings`]; and a processor target other than 1, 2, 4, or 8, or + /// outside the template's processors online at boot and its processor capacity, and a + /// memory target that is not whole blocks of [`MEMORY_BLOCK_SIZE`] and of the template's + /// memory blocks, from the template's RAM to its RAM capacity, [`Evidence::InvalidTargets`]; /// - with [`ErrorKind::Io`] for a template whose command line names the DNS server of its /// guest's resolver in another form than OpenVMM writes: more than once, without a NIC, or /// other than as one IP address in its canonical spelling, [`Evidence::MalformedGeneration`]; @@ -574,6 +587,7 @@ impl ClonePlan { snapshot, image, upper, + processors: request.processors, memory: request.memory, activation, capability, @@ -626,6 +640,11 @@ impl ClonePlan { } } + /// Returns the processor target: the processors online in the clone's VM once restored. + pub fn processors(&self) -> u32 { + self.processors + } + /// Returns the memory target, in bytes. pub fn memory(&self) -> u64 { self.memory @@ -669,8 +688,8 @@ impl ClonePlan { /// enforces there, as the template's key records it, the DNS server with which the template's /// guest configured its resolver at boot, if the template's /// [command line](TemplateKey::command_line) names one, unpublished listeners, and the URL of - /// the policy's proxy as the guest sees it, if any; and the template's processors, all online, - /// and the memory target, as the resources that the clone must have. + /// the policy's proxy as the guest sees it, if any; and the processor target, which the + /// restore brings online, and the memory target, as the resources that the clone must have. /// /// The identity of a binding, or of an export plan, is the SHA-256 digest of a domain of its /// kind and of its serde form, a few hundred bytes of metadata: the plan reads no content to @@ -696,6 +715,7 @@ impl fmt::Debug for ClonePlan { .field("snapshot", &self.snapshot) .field("image", &self.image) .field("upper", &self.upper) + .field("processors", &self.processors) .field("memory", &self.memory) .field("activation", &self.activation) .field("export", &self.export()) @@ -962,6 +982,24 @@ fn check_template( Error::unsupported(problem), )); } + let processors = request.processors; + let capacity = key.processors(); + // A template that records no processors online at boot has all of them online, and a restore + // brings none online. + let boot = match key.boot_processors() { + 0 => capacity, + boot => boot, + }; + if !PROCESSOR_COUNTS.contains(&processors) || processors < boot || processors > capacity { + return Err(Fault::new( + Evidence::InvalidTargets, + Error::unsupported(format!( + "a clone of the template's generation {name} cannot restore with a processor \ + target of {processors}: a target must be 1, 2, 4, or 8, from the template's \ + processors online at boot, {boot}, to its capacity, {capacity}" + )), + )); + } let memory = request.memory; let block = key.memory_block_size(); if memory < key.memory() diff --git a/nvx-rs/src/host/template/clone/launch.rs b/nvx-rs/src/host/template/clone/launch.rs index cc989906e..842c9f6cc 100644 --- a/nvx-rs/src/host/template/clone/launch.rs +++ b/nvx-rs/src/host/template/clone/launch.rs @@ -139,8 +139,11 @@ impl ClonePlan { /// mode, `rw` if a mapping writes it, and `ro` otherwise, then its hidden, re-exposed, and /// writable paths, and the caller owner, as its [plan](ClonePlan::export) holds them; without /// an export, the slot stays dormant. Then come `--single-process`, the template's backend, - /// CPU profile, and processors, `--restore-snapshot` with the directory of the generation's - /// snapshot, [`ClonePlan::snapshot`], `--restore-memory` with the memory target, + /// CPU profile, and processor capacity, `--restore-snapshot` with the directory of the + /// generation's snapshot, [`ClonePlan::snapshot`], for a template that records its processors + /// online at boot, [`TemplateKey::boot_processors`], `--restore-processors` with the processor + /// target, which brings online those of the target beyond them, `--restore-memory` with the + /// memory target, /// `--restore-ready-path` with the endpoint of the restore-ready event, a boot console that /// discards its output, as cold boots have, the control console at the launch's endpoint, /// which authenticates with the capability that the VMM reads from its standard input, and the @@ -272,6 +275,7 @@ impl ClonePlan { snapshot, image, upper, + processors, memory, activation, capability, @@ -348,6 +352,7 @@ impl ClonePlan { devices, key: &key, snapshot, + processors, memory, ready: &ready, report: locked.outcome_path(), @@ -638,6 +643,8 @@ struct Restore<'a> { key: &'a TemplateKey, /// The directory of the snapshot that the VMM restores. snapshot: PathBuf, + /// The processor target. + processors: u32, /// The memory target, in bytes. memory: u64, /// The endpoint of the restore-ready event. @@ -652,20 +659,27 @@ impl Restore<'_> { fn arguments(&self, endpoint: &str) -> Vec { let key = self.key; let profile = key.cpu_profile(); - let processors = key.processors().to_string(); + let capacity = key.processors().to_string(); + let processors = self.processors.to_string(); let memory = format!("{}M", self.memory / MIB); let listen = format!("listen={endpoint}"); - let mut arguments: Vec = self.devices.iter().map(OsString::from).collect(); - for argument in [ + let mut options = vec![ OsStr::new("--single-process"), OsStr::new("--hypervisor"), OsStr::new(key.hypervisor().as_str()), OsStr::new("--cpu-profile"), OsStr::new(&profile), OsStr::new("--processors"), - OsStr::new(&processors), + OsStr::new(&capacity), OsStr::new("--restore-snapshot"), self.snapshot.as_os_str(), + ]; + // A template that records no processors online at boot has all of them online, which + // OpenVMM restores without the option, and refuses with it. + if key.boot_processors() != 0 { + options.extend([OsStr::new("--restore-processors"), OsStr::new(&processors)]); + } + options.extend([ OsStr::new("--restore-memory"), OsStr::new(&memory), OsStr::new("--restore-ready-path"), @@ -677,9 +691,9 @@ impl Restore<'_> { OsStr::new("--microvm-control-auth-stdin"), OsStr::new("--microvm-report"), self.report.as_os_str(), - ] { - arguments.push(argument.to_owned()); - } + ]); + let mut arguments: Vec = self.devices.iter().map(OsString::from).collect(); + arguments.extend(options.into_iter().map(OsStr::to_owned)); arguments } } diff --git a/nvx-rs/src/host/template/clone/record.rs b/nvx-rs/src/host/template/clone/record.rs index 70f7c730e..2e218d7f6 100644 --- a/nvx-rs/src/host/template/clone/record.rs +++ b/nvx-rs/src/host/template/clone/record.rs @@ -60,8 +60,8 @@ const EXPORT_DOMAIN: &[u8] = b"nvx export plan\0"; /// command line names, [`TemplateKey::dns_server`], if it names one; unpublished listeners, /// since a template's policy allows neither ingress nor host-loopback access both ways; and the /// URL of the policy's proxy as the guest sees it, if it has one; -/// - the expected resources: the template's processors, all of which a template's VM keeps -/// online, and the request's memory target. +/// - the expected resources: the request's processor target, which the restore brings online, and +/// its memory target. /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a configuration that /// no record holds, as [`Configuration::check`] finds it, such as a GPT disk's first-partition @@ -126,7 +126,7 @@ pub(super) fn build( network, workload: request.workload.clone(), resources: ExpectedResources { - cpus: key.processors(), + cpus: request.processors, memory: request.memory, }, }; diff --git a/nvx-rs/src/host/template/clone/tests.rs b/nvx-rs/src/host/template/clone/tests.rs index 49d063618..b61a60197 100644 --- a/nvx-rs/src/host/template/clone/tests.rs +++ b/nvx-rs/src/host/template/clone/tests.rs @@ -1,7 +1,7 @@ -//! Tests of the plans of clones: the checks of their images, uppers, exports, and memory targets -//! against the generation that they lease, which read no content, the claims of their scratch -//! images, and the uppers in RAM, whose scratch devices their restores declare absent; in -//! [`launch`], of their launches; and in [`restore`], of their restore sequences. +//! Tests of the plans of clones: the checks of their images, uppers, exports, and processor and +//! memory targets against the generation that they lease, which read no content, the claims of +//! their scratch images, and the uppers in RAM, whose scratch devices their restores declare +//! absent; in [`launch`], of their launches; and in [`restore`], of their restore sequences. mod launch; mod restore; @@ -46,6 +46,8 @@ const CONFIG: &str = "config.json"; const IMAGE_LEN: u64 = 4 * MIB; /// The length of the template's scratch, and of the staged scratch images: a huge page of 2 MiB. const SCRATCH_LEN: u64 = 2 * MIB; +/// The template's processors, all online at boot, as [`ManifestSpec::template`] records them. +const PROCESSORS: u32 = 2; /// The template's RAM, in bytes, as [`ManifestSpec::template`] records it. const MEMORY: u64 = 512 * MIB; /// The template's RAM capacity, in bytes, as [`ManifestSpec::template`] records it. @@ -178,6 +180,17 @@ fn files() -> CallerFiles { CallerFiles::new(CONFIG).unwrap() } +/// Gives the template of `class`, whose capture `spec` describes, a capacity of `processors` +/// processors, of which `boot` are online at boot, as a capture records them. +fn set_processors(class: &mut CaptureClass, spec: &mut ManifestSpec, processors: u32, boot: u32) { + class.processors = processors; + class.boot_processors = boot; + spec.vp_count = processors; + let contract = spec.contract(); + contract.topology = (1, 1, processors, 1, (0..processors).collect()); + contract.boot_online_vp_count = boot; +} + /// Returns `path` as messages show it. fn shown(path: &Path) -> String { paths::normalized(path).display().to_string() @@ -194,8 +207,8 @@ fn gpt_layer(path: &Path) -> SandboxLayer { } /// Returns the request of a clone of the image at `image`, which `binding` binds, over an upper in -/// RAM, with the template's RAM as its memory target, and the settings of [`workload`], for -/// untrusted lowers, and with a journal policy, [`Journal::Forbidden`], that its record leaves +/// RAM, with the template's processors and RAM as its targets, and the settings of [`workload`], +/// for untrusted lowers, and with a journal policy, [`Journal::Forbidden`], that its record leaves /// out, and for a GPT disk, the bound [`FIRST_PARTITION_BOUND`]. fn tmpfs_request(image: &Path, binding: &DistroBinding) -> CloneRequest { CloneRequest { @@ -207,6 +220,7 @@ fn tmpfs_request(image: &Path, binding: &DistroBinding) -> CloneRequest { policy_revision: DISTRO_POLICY_REVISION, scratch: None, export: None, + processors: PROCESSORS, memory: MEMORY, workload: workload(), overlay: OverlayMode::UntrustedLowers, @@ -1774,6 +1788,72 @@ fn memory_targets_are_whole_blocks_from_the_templates_ram_to_its_capacity() { } } +#[test] +fn processor_targets_are_counts_from_the_templates_processors_online_at_boot_to_its_capacity() { + let refusal = |generation: &str, processors: u32, boot: u32, capacity: u32| { + ( + ErrorKind::Unsupported, + format!( + "a clone of the template's generation {generation} cannot restore with a \ + processor target of {processors}: a target must be 1, 2, 4, or 8, from the \ + template's processors online at boot, {boot}, to its capacity, {capacity}" + ), + ) + }; + // A template that boots with 2 of its 8 processors online serves each supported count from 2 + // to 8. + let fixture = Fixture::with(|class, spec, _| set_processors(class, spec, 8, 2)); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let generation = fixture.generation.generation().to_string(); + for processors in [0, 1, 3, 6, 16, u32::MAX] { + let request = CloneRequest { + processors, + ..tmpfs_request(&image, &binding) + }; + assert_eq!( + refused( + fixture.plan(&mut acquired, &request, NO_CONTENT), + Evidence::InvalidTargets + ), + refusal(&generation, processors, 2, 8) + ); + } + assert_untouched(&acquired); + for processors in [2, 4, 8] { + let request = CloneRequest { + processors, + ..tmpfs_request(&image, &binding) + }; + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + assert_eq!(plan.processors(), processors); + assert_eq!(plan.activation().configuration().resources.cpus, processors); + } + // A template whose processors are all online at boot records none online at boot, and serves + // its capacity alone. + let fixture = Fixture::new(); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let generation = fixture.generation.generation().to_string(); + for processors in [1, 4] { + let request = CloneRequest { + processors, + ..tmpfs_request(&image, &binding) + }; + assert_eq!( + refused( + fixture.plan(&mut acquired, &request, NO_CONTENT), + Evidence::InvalidTargets + ), + refusal(&generation, processors, PROCESSORS, PROCESSORS) + ); + } + let plan = fixture + .plan(&mut acquired, &tmpfs_request(&image, &binding), NO_CONTENT) + .unwrap(); + assert_eq!(plan.processors(), PROCESSORS); +} + #[test] fn requests_that_no_clone_takes_are_refused_before_anything_is_opened() { let fixture = Fixture::new(); @@ -2187,7 +2267,7 @@ fn the_activation_record_of_a_clone_in_ram_names_its_restore_image_and_settings( network: None, workload: workload(), resources: ExpectedResources { - cpus: plan.key().processors(), + cpus: PROCESSORS, memory: MEMORY, }, } diff --git a/nvx-rs/src/host/template/clone/tests/launch.rs b/nvx-rs/src/host/template/clone/tests/launch.rs index 3610ad1a5..50fe94aae 100644 --- a/nvx-rs/src/host/template/clone/tests/launch.rs +++ b/nvx-rs/src/host/template/clone/tests/launch.rs @@ -1048,6 +1048,59 @@ fn a_clone_of_a_template_with_a_nic_restores_its_policy_without_its_address() { assert!(!arguments.iter().any(|argument| argument == "--net")); } +#[test] +fn a_clone_of_a_template_with_processors_offline_at_boot_restores_its_processor_target() { + let fixture = Fixture::with(|class, spec, _| set_processors(class, spec, 4, 1)); + let (snapshot, _) = complete_snapshot(&fixture); + let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); + let mut acquired = fixture.acquire("sandbox"); + let request = CloneRequest { + processors: 2, + ..tmpfs_request(&image, &binding) + }; + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + let runtime = plan.key().runtime().clone(); + let mut vmm = VmmDir::new(); + let dir = vmm.path().to_owned(); + let made = RefCell::new(None); + let command = fake_vmm(&dir, &made); + let hooks = Hooks { + command: &command, + ..Hooks::LAUNCH + }; + let restoring = plan + .launch_with(&mut acquired, Path::new(OPENVMM), &runtime, hooks) + .unwrap(); + vmm.reference = Some(restoring.starting().process().reference().clone()); + // The VM keeps the template's processor capacity, of which the restore brings the target's + // prefix online. The clone in RAM declares the template's scratch absent among its devices, + // as any clone in RAM does. + let arguments = made.take().unwrap().arguments; + let at = arguments + .iter() + .position(|argument| argument == "--processors") + .unwrap(); + assert!( + arguments[..at] + .iter() + .any(|argument| argument == "--restore-absent-scratch"), + "{arguments:?}" + ); + assert_eq!( + arguments[at..at + 8], + [ + "--processors", + "4", + "--restore-snapshot", + snapshot.to_str().unwrap(), + "--restore-processors", + "2", + "--restore-memory", + "512M", + ] + ); +} + #[test] fn a_clone_on_a_scratch_image_passes_its_claim_to_its_vmm() { let fixture = Fixture::new(); diff --git a/nvx-rs/src/host/template/plan.rs b/nvx-rs/src/host/template/plan.rs index b3cbad1a9..6b3fd4c07 100644 --- a/nvx-rs/src/host/template/plan.rs +++ b/nvx-rs/src/host/template/plan.rs @@ -25,9 +25,14 @@ pub const MEMORY_BLOCK_SIZE: u64 = 128 * MIB; /// file of the template's scratch length, which the template does not keep. pub const SCRATCH_NAME: &str = "scratch.raw"; -/// The kernel command line of a template's VM. +/// The kernel command line of a template's VM, with which the guest's init hands the VM to its +/// sandbox control. const COMMAND_LINE: &str = "nvx_sandbox=1"; +/// The kernel command-line parameter that limits the processors that the guest brings online at +/// boot, and with which OpenVMM records them as the template's processors online at boot. +const BOOT_PROCESSORS: &str = "maxcpus"; + /// The snapshot tier of a template's capture: OpenVMM's clone point of the invariant /// configuration alone. const TIER: &str = "platform"; @@ -40,16 +45,22 @@ const BLOCK_IDENTITY: &str = "generation"; /// length is a multiple. const LOGICAL_BLOCK_SIZE: u64 = 512; -/// The resources of a template's VM: its processors, all online at boot, and its RAM, with the -/// capacity to which a restore may expand it. +/// The resources of a template's VM: its processors, of which a prefix is online at boot, and its +/// RAM, with the capacity to which a restore may expand it. /// -/// A template boots with every one of its processors online, so its clones restore with all of -/// them, and it serves clones of its processor count alone. [`TemplatePlan::new`] checks the -/// resources. +/// A template's VM has the processors of the largest of its clones, its processor capacity, of +/// which the first, those online at boot, are online when it is captured. A restore brings a +/// prefix of the others online, so that one template serves clones of each processor count from +/// those online at boot to its capacity. A template whose processors are all online at boot +/// serves clones of its capacity alone. [`TemplatePlan::new`] checks the resources. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub struct TemplateResources { - /// The processors, all online at boot: 1, 2, 4, or 8. + /// The processor capacity: 1, 2, 4, or 8. pub processors: u32, + /// The processors online at boot: 1, 2, 4, or 8, at most [`processors`](Self::processors). + /// Fewer than the capacity limit those that the guest brings online at boot, as the + /// [kernel command line](TemplatePlan::kernel_command_line) says. + pub boot_processors: u32, /// The RAM at boot, in bytes: a nonzero multiple of [`MEMORY_BLOCK_SIZE`]. pub memory: u64, /// The RAM capacity, in bytes, to which a restore may expand the RAM: a multiple of @@ -77,21 +88,26 @@ pub struct TemplateResources { /// template whose clones have no network. The policy allows neither ingress nor host-loopback /// access both ways, which its forwards need: through them, OpenVMM publishes the guest's /// listeners as live host attachments, which no snapshot holds. -/// - Its [`TemplateResources`]: its processors, all online at boot, and its RAM, with the capacity -/// to which a restore may expand it, OpenVMM's `--memory-capacity`. +/// - Its [`TemplateResources`]: its processor capacity, of which those online at boot are online +/// when it is captured, and its RAM, with the capacity to which a restore may expand it, +/// OpenVMM's `--memory-capacity`. /// /// # The kernel command line /// /// OpenVMM keeps the effective kernel command line of a template, which its /// [key](super::TemplateKey) holds and every clone restores, and refuses a capture whose command /// line holds a token outside the allowlist of its platform tier. The template's own command line -/// is `nvx_sandbox=1`, with which the guest's init hands the VM to its sandbox control. OpenVMM -/// writes the other tokens itself, each on the allowlist: the early console, console, reboot, and -/// panic settings, `nr_cpus=` with the processors, `nvx_snapshot_tier=platform`, the virtio -/// devices and the control tty that the guest discovers, and the NIC's addresses. The arguments -/// name no workload identity, lifecycle, share, or published listener, for which OpenVMM would -/// write other tokens. The command line therefore holds no image identity, hostname, share, -/// proxy, entrypoint, or other tenant data. +/// is `nvx_sandbox=1`, with which the guest's init hands the VM to its sandbox control, and for a +/// template that boots with fewer processors online than its capacity, `maxcpus=` with the +/// processors online at boot, which limits those that the guest brings online at boot, and which +/// OpenVMM records as the template's, so that each restore may bring a prefix of the others +/// online. OpenVMM writes the other tokens itself, each on the allowlist: the early console, +/// console, reboot, and panic settings, `nr_cpus=` with the processor capacity, +/// `nvx_snapshot_tier=platform`, the virtio devices and the control tty that the guest discovers, +/// and the NIC's addresses. The arguments name no workload identity, lifecycle, share, or +/// published listener, for which OpenVMM would write other tokens. The command line therefore +/// holds no image identity, hostname, share, proxy, entrypoint, or other tenant data, and every +/// clone of the template has the same. /// /// # The capture /// @@ -127,7 +143,8 @@ pub struct TemplateResources { /// # generation: StorageGeneration, /// # ) -> nvx::Result<()> { /// let resources = TemplateResources { -/// processors: 2, +/// processors: 4, +/// boot_processors: 1, /// memory: 4 * MEMORY_BLOCK_SIZE, /// memory_capacity: 16 * MEMORY_BLOCK_SIZE, /// }; @@ -151,6 +168,8 @@ pub struct TemplatePlan { scratch_length: u64, /// The processors and RAM. resources: TemplateResources, + /// The kernel command line of the capture's VM. + command_line: String, /// The capture's transaction directory, spelled with its components alone. transaction: PathBuf, /// The snapshot's destination in the transaction directory. @@ -180,7 +199,8 @@ impl TemplatePlan { /// which OpenVMM would not attach as a raw image; a placeholder whose binding records /// a length, or a scratch length, that is not a nonzero multiple of the 512 bytes of the /// logical blocks that OpenVMM presents; - /// processors other than 1, 2, 4, or 8; RAM or a RAM capacity that is not a nonzero multiple + /// processors other than 1, 2, 4, or 8, or processors online at boot other than 1, 2, 4, or + /// 8, or more than the processors; RAM or a RAM capacity that is not a nonzero multiple /// of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM; a network policy that allows ingress, /// or host-loopback access both ways; and a transaction directory that is not valid Unicode, /// that is relative, which OpenVMM would resolve against the directory in which it runs, or @@ -249,11 +269,22 @@ impl TemplatePlan { let destination = transaction.join(SNAPSHOT_NAME); let plan = SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch_at(scratch)) .with_network(adapter.network().cloned()); + // A template whose processors are all online at boot needs no limit, and records none, + // as OpenVMM records none without the parameter. + let command_line = if resources.boot_processors < resources.processors { + format!( + "{COMMAND_LINE} {BOOT_PROCESSORS}={}", + resources.boot_processors + ) + } else { + COMMAND_LINE.to_owned() + }; Ok(Self { plan, binding: binding.clone(), scratch_length, resources, + command_line, transaction, destination, generation, @@ -310,8 +341,10 @@ impl TemplatePlan { self.generation } - /// Returns the kernel command line of the capture's VM, `nvx_sandbox=1`, which the caller - /// passes as OpenVMM's `--cmdline`, alone. + /// Returns the kernel command line of the capture's VM, which the caller passes as OpenVMM's + /// `--cmdline`, alone: `nvx_sandbox=1`, then, for a template that boots with fewer processors + /// online than its capacity, `maxcpus=` with the processors online at boot, as + /// [The kernel command line](Self#the-kernel-command-line) describes. /// /// It takes the place of the command line of [`plan`](Self::plan), which /// [`SandboxPlan::kernel_command_line`] and a [`PreparedLaunch`] render: that one names the @@ -320,7 +353,7 @@ impl TemplatePlan { /// /// [`PreparedLaunch`]: crate::host::sandbox::PreparedLaunch pub fn kernel_command_line(&self) -> &str { - COMMAND_LINE + &self.command_line } /// Returns OpenVMM's arguments of the capture: those of its devices, as @@ -342,10 +375,11 @@ impl TemplatePlan { Ok(arguments) } - /// Returns OpenVMM's arguments of the capture besides those of its devices: `--processors`, - /// `--memory` and `--memory-capacity` in MiB, and the capture's options, - /// `--snapshot-destination`, `--snapshot-tier platform`, - /// `--snapshot-block-identity generation`, and `--snapshot-generation-id`. + /// Returns OpenVMM's arguments of the capture besides those of its devices: `--processors` + /// with the processor capacity, `--memory` and `--memory-capacity` in MiB, and the capture's + /// options, `--snapshot-destination`, `--snapshot-tier platform`, + /// `--snapshot-block-identity generation`, and `--snapshot-generation-id`. The processors + /// online at boot are the kernel command line's. /// /// A caller that renders the arguments of the devices itself adds these to them. pub fn capture_arguments(&self) -> Vec { @@ -353,6 +387,7 @@ impl TemplatePlan { processors, memory, memory_capacity, + .. } = self.resources; [ ("--processors", processors.to_string()), @@ -378,12 +413,13 @@ fn is_block_length(length: u64) -> bool { length != 0 && length.is_multiple_of(LOGICAL_BLOCK_SIZE) } -/// Refuses `resources` that no template's VM has: processors other than 1, 2, 4, or 8, and RAM or -/// a RAM capacity that is not a nonzero multiple of [`MEMORY_BLOCK_SIZE`], or a capacity below -/// the RAM. +/// Refuses `resources` that no template's VM has: processors other than 1, 2, 4, or 8, processors +/// online at boot other than 1, 2, 4, or 8, or more than the processors, and RAM or a RAM capacity +/// that is not a nonzero multiple of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM. fn check_resources(resources: &TemplateResources) -> Result<()> { let TemplateResources { processors, + boot_processors, memory, memory_capacity, } = *resources; @@ -392,6 +428,12 @@ fn check_resources(resources: &TemplateResources) -> Result<()> { "a template's VM has 1, 2, 4, or 8 processors, not {processors}" ))); } + if !PROCESSOR_COUNTS.contains(&boot_processors) || boot_processors > processors { + return Err(Error::unsupported(format!( + "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, not \ + {boot_processors}" + ))); + } if memory == 0 || !memory.is_multiple_of(MEMORY_BLOCK_SIZE) || !memory_capacity.is_multiple_of(MEMORY_BLOCK_SIZE) diff --git a/nvx-rs/src/host/template/plan/tests.rs b/nvx-rs/src/host/template/plan/tests.rs index 41ed23081..963b057c5 100644 --- a/nvx-rs/src/host/template/plan/tests.rs +++ b/nvx-rs/src/host/template/plan/tests.rs @@ -200,10 +200,12 @@ fn networks() -> Vec> { ] } -/// Returns the resources of `processors`, with 512 MiB of RAM and a capacity of 2 GiB. +/// Returns the resources of `processors`, all online at boot, with 512 MiB of RAM and a capacity +/// of 2 GiB. fn resources(processors: u32) -> TemplateResources { TemplateResources { processors, + boot_processors: processors, memory: 512 * MIB, memory_capacity: 2 * GIB, } @@ -271,20 +273,29 @@ fn template(adapter: SandboxPlan) -> TemplatePlan { } /// Returns the templates of every format of placeholder, of adapters' plans over a clone's scratch -/// and over RAM, of every network of [`networks`], and of every processor count. +/// and over RAM, of every network of [`networks`], and of every processor count, with every count +/// up to it online at boot. fn every_template() -> Vec { let mut templates = Vec::new(); for (format, image) in [("erofs", erofs()), ("gpt", gpt())] { for upper in [SandboxUpper::scratch(CLONE_SCRATCH), SandboxUpper::RAM] { for network in networks() { for processors in PROCESSOR_COUNTS { - let inputs = Inputs { - adapter: adapter_with(image, upper.clone(), network.clone()), - binding: binding_of(format, IDENTITY, GIB), - resources: resources(processors), - ..Inputs::default() - }; - templates.push(inputs.lower().unwrap()); + for boot_processors in PROCESSOR_COUNTS + .into_iter() + .filter(|boot| *boot <= processors) + { + let inputs = Inputs { + adapter: adapter_with(image, upper.clone(), network.clone()), + binding: binding_of(format, IDENTITY, GIB), + resources: TemplateResources { + boot_processors, + ..resources(processors) + }, + ..Inputs::default() + }; + templates.push(inputs.lower().unwrap()); + } } } } @@ -394,6 +405,7 @@ fn templates_render_their_devices_resources_and_capture_options() { let inputs = Inputs { resources: TemplateResources { processors: 8, + boot_processors: 8, memory: MEMORY_BLOCK_SIZE, memory_capacity: MEMORY_BLOCK_SIZE, }, @@ -449,6 +461,32 @@ fn templates_keep_the_placeholder_and_the_files_of_their_capture_in_its_transact ); } +#[test] +fn templates_limit_their_processors_online_at_boot_on_their_command_line() { + // A template whose processors are all online at boot sets no limit, and OpenVMM records none. + for processors in PROCESSOR_COUNTS { + let template = Inputs { + resources: resources(processors), + ..Inputs::default() + } + .lower() + .unwrap(); + assert_eq!(template.kernel_command_line(), "nvx_sandbox=1"); + } + // One that boots with fewer online limits them, and keeps its capacity. + let inputs = Inputs { + resources: TemplateResources { + boot_processors: 2, + ..resources(8) + }, + ..Inputs::default() + }; + let template = inputs.lower().unwrap(); + assert_eq!(template.kernel_command_line(), "nvx_sandbox=1 maxcpus=2"); + assert_eq!(value(&template.capture_arguments(), "--processors"), "8"); + assert_eq!(template.resources(), inputs.resources); +} + #[test] fn the_upper_of_an_adapters_plan_is_a_clones_and_no_part_of_its_template() { let over_scratch = template(adapter(None)); @@ -491,6 +529,7 @@ fn openvmms_platform_allowlist_admits_its_tokens_and_refuses_tenant_data() { "nvx_config=0xd0010000,65536", "nvx_snapshot_tier=platform", "nr_cpus=8", + "maxcpus=1", "nvx_control_tty=hvc2", "virtio_mmio.device=0x1000@0xd0001000:6", "virtnet_ip=10.0.0.2", @@ -757,6 +796,23 @@ fn templates_refuse_what_no_capture_takes() { format!("a template's VM has 1, 2, 4, or 8 processors, not {processors}") ); } + // Processors online at boot other than 1, 2, 4, and 8, or more than the processors. + for (processors, boot_processors) in [(2, 0), (8, 3), (2, 4), (8, 16), (8, u32::MAX)] { + let inputs = Inputs { + resources: TemplateResources { + boot_processors, + ..resources(processors) + }, + ..Inputs::default() + }; + assert_eq!( + refusal(inputs.lower()), + format!( + "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, \ + not {boot_processors}" + ) + ); + } // RAM and capacities that are not whole blocks, and capacities below the RAM. for (memory, memory_capacity) in [ (0, 0), @@ -769,6 +825,7 @@ fn templates_refuse_what_no_capture_takes() { let inputs = Inputs { resources: TemplateResources { processors: 2, + boot_processors: 2, memory, memory_capacity, }, @@ -906,12 +963,13 @@ fn templates_check_their_inputs_in_order() { scratch_length: 0, resources: TemplateResources { processors: 3, + boot_processors: 3, memory: 0, memory_capacity: 0, }, transaction: "templates".to_owned(), }; - let steps: [(&str, Fix); 11] = [ + let steps: [(&str, Fix); 12] = [ ("a template lowers an adapter's plan", |inputs| { inputs.adapter = adapter_at("placeholder.vhd", gpt(), SandboxUpper::RAM, listening()); }), @@ -941,6 +999,9 @@ fn templates_check_their_inputs_in_order() { ("processors, not 3", |inputs| { inputs.resources.processors = 2 }), + ("of its 2 processors online, not 3", |inputs| { + inputs.resources.boot_processors = 1 + }), ("needs nonzero RAM", |inputs| { inputs.resources = resources(2) }), diff --git a/nvx-rs/src/host/template/store/capture.rs b/nvx-rs/src/host/template/store/capture.rs index 11b6d95f1..7a95ee07d 100644 --- a/nvx-rs/src/host/template/store/capture.rs +++ b/nvx-rs/src/host/template/store/capture.rs @@ -91,16 +91,16 @@ const CAPTURES_WHAT: &str = "the captures of the template store"; /// network, the binding of the placeholder, and the artifacts that run its VM, which /// [`Store::create_template`] and [`Store::refresh_template`] capture. /// -/// The adapter's plan is lowered as [`TemplatePlan::new`] lowers it, with the processors and RAM -/// of the class, and a scratch of the class's scratch length; its upper is no part of the -/// template. The class and the plan must describe one VM: a class whose processors are all online -/// at boot, a placeholder whose binding records the class's `distro` length, and a network of the -/// plan that lowers to the OpenVMM arguments of the class's network. +/// The adapter's plan is lowered as [`TemplatePlan::new`] lowers it, with the processors, those +/// online at boot among them, and RAM of the class, and a scratch of the class's scratch length; +/// its upper is no part of the template. The class and the plan must describe one VM: a +/// placeholder whose binding records the class's `distro` length, and a network of the plan that +/// lowers to the OpenVMM arguments of the class's network. #[derive(Debug, Clone)] pub struct Capture { /// The class of the capture, which names its slot, and from which the capture takes its - /// backend, its CPU profile, its processors, its RAM, its scratch length, and the runtime - /// identity of its launch. + /// backend, its CPU profile, its processors and those online at boot, its RAM, its scratch + /// length, and the runtime identity of its launch. pub class: CaptureClass, /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the placeholder `distro` and /// the template's network. @@ -208,7 +208,7 @@ fn uninterrupted(_: Point) -> Result<()> { struct Inputs { /// The identity of the class. class: ClassIdentity, - /// The template's processors and RAM, from the class. + /// The template's processors, those online at boot among them, and RAM, from the class. resources: TemplateResources, /// The artifacts, as absolute paths. artifacts: Artifacts, @@ -218,21 +218,13 @@ impl Inputs { /// Checks the inputs of `capture`, without reading or opening anything, and returns what the /// capture takes of them. /// - /// Fails with [`ErrorKind::Unsupported`] as [`CaptureClass::identity`] fails, and for a class - /// whose processors are not all online at boot, a placeholder whose binding records another - /// length than the class's `distro`, and a plan whose network lowers to other arguments than - /// the class's; and with [`ErrorKind::Unavailable`] for an artifact whose path cannot be - /// resolved. + /// Fails with [`ErrorKind::Unsupported`] as [`CaptureClass::identity`] fails, and for a + /// placeholder whose binding records another length than the class's `distro`, and a plan + /// whose network lowers to other arguments than the class's; and with + /// [`ErrorKind::Unavailable`] for an artifact whose path cannot be resolved. fn check(capture: &Capture) -> Result { let class = &capture.class; let identity = class.identity()?; - if class.boot_processors != class.processors { - return Err(Error::unsupported(format!( - "a template boots with all of its processors online, but its class has {} \ - processors, {} online at boot", - class.processors, class.boot_processors - ))); - } let length = capture.binding.image().length(); if length != class.distro.length { return Err(Error::unsupported(format!( @@ -261,6 +253,7 @@ impl Inputs { class: identity, resources: TemplateResources { processors: class.processors, + boot_processors: class.boot_processors, memory: class.memory, memory_capacity: class.memory_capacity, }, diff --git a/nvx-rs/src/host/template/store/capture/tests.rs b/nvx-rs/src/host/template/store/capture/tests.rs index 9f224c341..40387f546 100644 --- a/nvx-rs/src/host/template/store/capture/tests.rs +++ b/nvx-rs/src/host/template/store/capture/tests.rs @@ -472,6 +472,47 @@ fn captures_publish_generations_that_creations_then_serve() { assert_eq!(bench.store.read_key(&first).unwrap(), key); } +#[test] +fn captures_limit_the_processors_that_their_class_keeps_offline_at_boot() { + let mut bench = Bench::new(); + bench.capture.class.processors = 4; + bench.capture.class.boot_processors = 1; + // OpenVMM records the processors online at boot that the command line limits. + let mut spec = ManifestSpec::template(); + spec.vp_count = 4; + let contract = spec.contract(); + contract.topology = (1, 1, 4, 1, vec![0, 1, 2, 3]); + contract.boot_online_vp_count = 1; + let emulation = Emulation { + manifest: Some(spec), + ..Emulation::capture() + }; + let arguments = RefCell::new(Vec::new()); + let run = |locked: &mut LockedStore, vm: &Vm<'_>| { + let command = vm.command("control", Path::new("outcome.json"), locked.dir())?; + arguments.replace(command.arguments); + emulation.run(locked, vm) + }; + let created = bench.serve_with(Some(&|| Ok(after(1))), &run, &uninterrupted, false); + let Created::Captured(generation) = created.unwrap() else { + panic!("the first creation of a class captures it"); + }; + // The VM has the class's processors, and boots with those that it keeps online. + let arguments: Vec = arguments + .take() + .into_iter() + .map(|argument| argument.into_string().unwrap()) + .collect(); + let value = |option: &str| { + let at = arguments.iter().position(|given| given == option).unwrap(); + arguments[at + 1].as_str() + }; + assert_eq!(value("--processors"), "4"); + assert_eq!(value("--cmdline"), "nvx_sandbox=1 maxcpus=1"); + let key = bench.store.read_key(&generation).unwrap(); + assert_eq!((key.processors(), key.boot_processors()), (4, 1)); +} + #[test] fn refreshes_leave_older_generations_and_their_leases_untouched() { let bench = Bench::new(); diff --git a/nvx-rs/src/protocol/status.rs b/nvx-rs/src/protocol/status.rs index b73444b1e..69a5d3aa6 100644 --- a/nvx-rs/src/protocol/status.rs +++ b/nvx-rs/src/protocol/status.rs @@ -35,8 +35,9 @@ pub const TIME_REPAIR: u8 = 195; /// The restore cannot be repaired: it holds no fresh entropy, as when its capture wrote the /// entropy to a file, or its capture finished it already. pub const REPAIR_RESTORE: u8 = 40; -/// The online CPUs cannot be read, are not a prefix of the CPUs, or do not meet the restore's -/// processor target, which repair does not activate. +/// The online CPUs cannot be read or are not a prefix of the CPUs, the restore's processor target +/// is below them, or the repair could not bring the target's CPUs online: a write of a CPU's +/// state failed, or the online CPUs are not the target's prefix once it brought them online. pub const REPAIR_PROCESSORS: u8 = 41; /// The restore's memory target is refused, before any memory block was added: the kernel's /// memory block size is not the snapshot ABI's, the restore lists ranges without a target, or @@ -96,7 +97,7 @@ pub fn describe(status: u8) -> Option<&'static str> { TIME_VIOLATION => "the time ABI's violation watcher observed a violation", TIME_REPAIR => "the time ABI's part of a restore's repair failed", REPAIR_RESTORE => "a clone's restore cannot be repaired", - REPAIR_PROCESSORS => "a clone's online CPUs do not meet its restore", + REPAIR_PROCESSORS => "a clone's online CPUs do not meet its restore's processor target", REPAIR_MEMORY_TARGET => "a clone's memory target is refused", REPAIR_MEMORY => "a clone's memory activation failed", REPAIR_RESEED => "a clone's reseed failed", diff --git a/nvx-rs/src/testing/fake_openvmm.rs b/nvx-rs/src/testing/fake_openvmm.rs index 32b03682e..52790eb19 100644 --- a/nvx-rs/src/testing/fake_openvmm.rs +++ b/nvx-rs/src/testing/fake_openvmm.rs @@ -101,7 +101,8 @@ //! share, `--snapshot-destination`, `--snapshot-tier platform`, //! `--snapshot-block-identity generation` with a nonzero `--snapshot-generation-id` of 32 //! hexadecimal digits, the options of a cold boot of a run with a `--memory-capacity` that is a -//! multiple of 128 MiB, `nvx_sandbox=1` alone on the kernel command line, `--virtio-console none`, +//! multiple of 128 MiB, `nvx_sandbox=1` on the kernel command line, alone or followed by a +//! canonical `maxcpus=` of 1, 2, 4, or 8 of its `--processors`, `--virtio-console none`, //! the control console with its capability on standard input, and an outcome report. On Unix, the //! control console's socket must be in the directory of the snapshot's destination, as above. The //! fake serves no control session of a capture, which the host opens none of. Like OpenVMM, it @@ -192,6 +193,10 @@ const LOGICAL_BLOCK_SIZE: u64 = 512; /// Name of the scenario of a template's capture, next to the kernel. const CAPTURE_SCENARIO_NAME: &str = "fake-openvmm-capture.json"; +/// The refusal of a capture whose kernel command line is not a template's. +const CAPTURE_COMMAND_LINE: &str = "a capture's kernel command line is nvx_sandbox=1, alone or \ + with a canonical maxcpus= of 1, 2, 4, or 8 of its processors"; + /// Prefix of the name of the file that backs the RAM of a capture's VM, beside the snapshot's /// destination, which the process ID of the VMM follows. const MEMORY_FILE_PREFIX: &str = ".openvmm-microvm-memory-"; @@ -1219,9 +1224,30 @@ fn parse(program: &str, arguments: &[String]) -> Result { ram_overlay_requested(&tokens)?, )?; if capture { - // A template's VM boots the invariant configuration alone, without a tenant. - if tokens != ["nvx_sandbox=1"] { - return Err("a capture's kernel command line is nvx_sandbox=1 alone".to_owned()); + // A template's VM boots the invariant configuration alone, without a tenant, with at + // most the limit of its processors online at boot, which OpenVMM records in the + // canonical form alone. + let processors = values + .get("--processors") + .and_then(|given| given.first()) + .and_then(|given| given.parse::().ok()); + let invariant = match tokens.as_slice() { + ["nvx_sandbox=1"] => true, + ["nvx_sandbox=1", limit] => limit + .strip_prefix("maxcpus=") + .and_then(|count| { + count + .parse::() + .ok() + .filter(|parsed| parsed.to_string() == count) + }) + .is_some_and(|count| { + [1, 2, 4, 8].contains(&count) && processors.is_some_and(|all| count <= all) + }), + _ => false, + }; + if !invariant { + return Err(CAPTURE_COMMAND_LINE.to_owned()); } let roles: BTreeSet<&str> = blocks.iter().map(|block| block.role.as_str()).collect(); if roles != BTreeSet::from(["distro", "scratch"]) { @@ -3028,6 +3054,15 @@ mod tests { assert!(options.children.is_empty()); } + #[test] + fn captures_may_limit_the_processors_online_at_boot() { + // The capture of `CAPTURE` has 2 processors. + for command_line in ["nvx_sandbox=1 maxcpus=1", "nvx_sandbox=1 maxcpus=2"] { + let (_temporary, options) = parse_capture(&capture_with("--cmdline", command_line)); + assert_eq!(options.err(), None, "{command_line}"); + } + } + #[cfg(unix)] #[test] fn captures_listen_in_the_directory_of_their_snapshot_destination() { @@ -3186,7 +3221,25 @@ mod tests { ), ( capture_with("--cmdline", "nvx_sandbox=1 nvx_hostname=tenant"), - "a capture's kernel command line is nvx_sandbox=1 alone".to_owned(), + CAPTURE_COMMAND_LINE.to_owned(), + ), + // OpenVMM records a limit of the processors online at boot in its canonical form + // alone, of a count that a microVM supports, and at most its processors. + ( + capture_with("--cmdline", "nvx_sandbox=1 maxcpus=01"), + CAPTURE_COMMAND_LINE.to_owned(), + ), + ( + capture_with("--cmdline", "nvx_sandbox=1 maxcpus=3"), + CAPTURE_COMMAND_LINE.to_owned(), + ), + ( + capture_with("--cmdline", "nvx_sandbox=1 maxcpus=4"), + CAPTURE_COMMAND_LINE.to_owned(), + ), + ( + capture_with("--cmdline", "nvx_sandbox=1 maxcpus=1 maxcpus=1"), + CAPTURE_COMMAND_LINE.to_owned(), ), ( capture_with("--snapshot-destination", "{dir}/transaction"), diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index 27d662109..0e7938782 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -229,6 +229,7 @@ the root, of normal components, and the other probes take no arguments: | `reads` | The sectors that the guest read from each of its disks, by the disk's statistics: `disks`, each as `NAME:SECTORS`, in 512-byte sectors, in the order of their names, separated by spaces | | `read-disks` | It reads each of the guest's disks from its start to its end, through the page cache: `disks`, each as `NAME:BYTES`, the bytes that it read, in the order of their names, separated by spaces | | `memory` | The guest's memory: the size of its memory blocks, `block-size`, how many are online, `blocks`, and their bytes, `online`, and the memory that the kernel manages, `total`, each in bytes | +| `processors` | The guest's CPUs, as the kernel lists them: those that are online, `online`, present, `present`, and possible, `possible`; and the CPUs that the restore of a clone brought online, as the time ABI's record of the restore names them, `restored`, `none` for none, or `absent` without a record | | `time` | The guest's time state, `/run/nvx/time/state`, which NVX's time daemon keeps: each line `KEY=VALUE` as a fact, with each `_` of its key as `-`, in order, leaving out a key that no fact may have, and a repeated one | | `first-partition` | The bytes of the first partition of a GPT disk that the assembly returned, without their trailing padding: how many, `length`, and the bytes, `bytes`, in hexadecimal, or `none` for both for an EROFS image | | `cgroups` | The file system at the cgroups' parent, `parent.type`, and its `parent.subtree-control`; whether each cgroup exists, `agent.exists` and `workload.exists`; `agent.memory.low`, `workload.memory.max`, and `workload.pids.max`; the number of processes in each, `agent.processes` and `workload.processes`; and `user-namespaces`, the guest's limit of user namespaces, or `unavailable` without them | @@ -777,11 +778,13 @@ worker of a template's VM admits no execution before the capture. The worker returns from the capture only once the capture ended without a snapshot, or failed, or in a clone, which a restore of the snapshot started. In a clone, the worker repairs the restore next, with `nvx::guest::clone::repair`, behind its -fence, which stays closed: the repair checks the online CPUs, activates the -restore's memory target, reseeds the kernel's random number generator with the -restore's fresh entropy, and finishes the restore, with the acknowledgement -that the host awaits, which hands the deferred time work, the restore checks -and the release of the stall detectors, to the time daemon. NVX's `/init` +fence, which stays closed: the repair brings the CPUs of the restore's +processor target online, activates the restore's memory target, reseeds the +kernel's random number generator with the restore's fresh entropy, and +finishes the restore, with the CPUs that the restore brought online and the +acknowledgement that the host awaits, which hands the deferred time work, the +restore checks, those of the new CPUs among them, and the release of the stall +detectors, to the time daemon. NVX's `/init` started the daemon in the template, with the time ABI's boot step, before the guest agent, so every clone holds it, and the guest agent's PID 1 collects its status. The repair takes the time ABI in strict mode, so that a violation of @@ -889,7 +892,7 @@ ends with one of these statuses, after a diagnostic on standard error: | 29 | A stage of the capture failed that this build of the worker does not know | | 30 | The capture ended without a snapshot, as without a destination, or once OpenVMM rejected it, and the template continued | | 40 | The clone's restore cannot be repaired: it holds no fresh entropy, or its capture finished it | -| 41 | The clone's online CPUs cannot be read, are not a prefix of the CPUs, or do not meet the restore's processor target | +| 41 | The clone's online CPUs cannot be read, are not a prefix of the CPUs, exceed the restore's processor target, or the CPUs of the target cannot be brought online | | 42 | The restore's memory target is refused, before any memory block is added | | 43 | The activation of the restore's memory target failed | | 44 | The reseed of the kernel's random number generator failed | @@ -1379,9 +1382,10 @@ command's options choose of the template, and `Request::capture` returns NVX's class's captures, and the template's compatibility key: the runtime identity of the adapter's VMs, with the guest agent's build, `runtime-adapter-guest/BUILD`, the backend and the CPU profile, the - processors, all online at boot, the RAM and its capacity, the geometries of - the placeholder and of the dormant scratch, as OpenVMM presents a sandbox - block device, which `capture::geometry` gives, and the network; + processors, those online at boot, 1 unless the command names more, the + fewest that the template's clones take, the RAM and its capacity, the + geometries of the placeholder and of the dormant scratch, as OpenVMM presents + a sandbox block device, which `capture::geometry` gives, and the network; - NVX's artifacts, with the initramfs that puts the guest agent in place of NVX's agents, which the command writes into a directory of its own in the system's temporary directory, private to the current user, as a launch @@ -1395,10 +1399,13 @@ OpenVMM requires. On Linux, the path of that socket, so NVX refuses a store whose path is too long for it. The capture boots the template's VM with the kernel command line -`nvx_sandbox=1`, to which OpenVMM adds the platform tier's tokens, -`nvx_snapshot_tier=platform` among them, so the worker of the guest agent -captures the VM in [template mode](#template-mode) before it assembles any root, -and the capture ends the VM. The store checks what the VM left, and publishes +`nvx_sandbox=1`, and for a template whose processors are not all online at +boot, `maxcpus=` with those that are, to which OpenVMM adds the platform tier's +tokens, `nvx_snapshot_tier=platform` among them, so the worker of the guest +agent captures the VM in [template mode](#template-mode) before it assembles +any root, and the capture ends the VM. OpenVMM records the processors online at +boot, from which each restore brings a clone's processor target online. The +store checks what the VM left, and publishes the snapshot as its next generation, with the record of its capture and its compatibility key, whose kernel command line holds the platform tier's tokens alone: neither the placeholder's identity nor a root's tokens. The template has @@ -1413,8 +1420,9 @@ build created is not supported: delete it and create a new one. There is no migration. NVX refuses a network whose policy no template takes, one that allows ingress or host-loopback access both ways, before any VM starts, and the -command refuses a class that no capture takes, such as one of 3 processors, -before it creates the store. A capture that fails leaves the store's +command refuses a class that no capture takes, such as one of 3 processors, or +of more online at boot than in all, before it creates the store. A capture that +fails leaves the store's generations as they were, and the command's diagnostic names the log of the VMM of the class's latest capture, when the class's slot keeps one. @@ -1482,6 +1490,7 @@ in a sandbox store, through NVX's public API, `nvx::host::template`. Its library side, `clone`, holds what the command's options choose of the clone, `clone::Request`: its image, an EROFS layer or a GPT disk no longer than the template's placeholder, its upper, on an ext4 scratch image or in RAM, its +processor target, the template's processors online at boot unless given, its memory target, the template's RAM unless given, the workload identity as which its executions run, its overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, the machine ID that its executions see, @@ -1516,11 +1525,14 @@ The command first reads the clone's network, if given, as a launch reads it, and refuses one that the plan refuses, then plans the export of the clone's shares, if it has any, and refuses those that it cannot export, as a launch refuses them, then reads the -generation from the store's index, and, unless the request names the clone's -memory target, its compatibility key, whose RAM the target is then, before it -needs NVX's artifacts. A key that cannot be read is left to NVX's plan, which -reads it again, and attributes the failure, to a malformed generation, which -quarantines it, or to the host, as it does for every clone. The store then +generation from the store's index, and, unless the request names both of the +clone's targets, its compatibility key, before it needs NVX's artifacts: the +processor target is then the key's processors online at boot, all of its +processors for a template that records none, unless the request names one, and +the memory target the key's RAM, unless the request names one. A key that +cannot be read is left to NVX's plan, which reads it again, and attributes the +failure, to a malformed generation, which quarantines it, or to the host, as it +does for every clone. The store then leases the generation to the sandbox, `Store::acquire`, at the host's time, which it refuses for a generation that a tombstone quarantines, or that is not eligible then, by the monotonic clock of the host's boot that captured the @@ -1557,10 +1569,12 @@ the bindings of its images, which no one else may change, the geometry of its image, which must have the placeholder's block sizes, and be no longer than the placeholder, whose length is the capacity that the restore presents, reading zeros past the image's end, and that of its scratch image, which must -be the template's scratch's, its length included, its memory target, -whole blocks of 128 MiB, from the template's RAM to its capacity, and its -export, if it has one: the template's filesystem slot, which its capture left -dormant, binds one directory, so the export must have exactly one, the +be the template's scratch's, its length included, its processor target, +1, 2, 4, or 8, from the template's processors online at boot to its capacity, +which the restore brings online, and the clone's repair verifies, its memory +target, whole blocks of 128 MiB, from the template's RAM to its capacity, and +its export, if it has one: the template's filesystem slot, which its capture +left dormant, binds one directory, so the export must have exactly one, the export's root, with any other shares inside it, and the directory may neither lie inside the template store's directory nor hold it, so that no clone reaches the payload of a generation. A clone that @@ -1764,7 +1778,7 @@ Each command of `runtime-adapter-host` but `stage`, `list`, `inspect`, and mapping, the launch sends no more of the table, nor `Start`, stops the guest, which tears down the root, waits for the VM to end, abandons the launch, and checks that the VM ended with a success and a complete teardown. -- `create STORE --guest-agent PATH --distro FORMAT:PATH --distro-identity UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` +- `create STORE --guest-agent PATH --distro FORMAT:PATH --distro-identity UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--boot-processors B] [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` serves a [template](#templates) of the guest agent PATH from the template store STORE, which it creates if needed: a generation of the template's class that the store holds, and that is eligible at the host's time, or else @@ -1772,8 +1786,9 @@ Each command of `runtime-adapter-host` but `stage`, `list`, `inspect`, and file PATH, an EROFS file system, `erofs`, or a GPT disk, `gpt`, with the identity UUID. The template's VM runs on the hypervisor NAME, as a launch's does, with the CPU profile PROFILE, `auto` by default, as OpenVMM's - `--cpu-profile` takes it, N processors, 1 by default, all online at boot, - MIB mebibytes of RAM, 256 by default, a RAM capacity of MIB mebibytes, its + `--cpu-profile` takes it, N processors, 1 by default, of which B are online + at boot, 1 by default, the fewest that its clones take, MIB mebibytes of + RAM, 256 by default, a RAM capacity of MIB mebibytes, its RAM by default, and a dormant scratch of BYTES bytes, 64 MiB by default. It has one NIC under the policy of the [network](#networks) FILE, unless the policy lets nothing through, and none without a network. The command reads @@ -1781,7 +1796,7 @@ Each command of `runtime-adapter-host` but `stage`, `list`, `inspect`, and before it creates the store, then prints the generation once the store has committed it, and [evicts](#eviction) the generations of the class that it supersedes. -- `refresh STORE --guest-agent PATH --distro FORMAT:PATH --distro-identity UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` +- `refresh STORE --guest-agent PATH --distro FORMAT:PATH --distro-identity UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--boot-processors B] [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` takes the options of `create`, and captures a new generation of the template's class in STORE, whether or not the store holds an eligible one, then prints it once the store has committed it, and evicts the generations of @@ -1810,16 +1825,18 @@ Each command of `runtime-adapter-host` but `stage`, `list`, `inspect`, and its record again, and releases the lease of a clone that ended, through the template store TEMPLATES, if given, in place of the one that the clone's record names. -- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES]` +- `start STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--processors N] [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES]` starts a [clone](#clones) of the generation G of the template store TEMPLATES in the sandbox store STORE, which it creates if needed, and whose `stage` staged its images, which no one else may change: the image PATH, an EROFS file system, `erofs`, or a GPT disk, `gpt`, of which the guest expects the identity UUID, the file system's UUID or the disk's GUID, no longer than the template's placeholder, whatever the placeholder's format, over the - ext4 scratch image PATH, or RAM, with a memory target of MIB mebibytes, the - template's RAM by default, the workload identity UID:GID, if given, as which - every execution runs, and whose account the guest checks before the clone is + ext4 scratch image PATH, or RAM, with a processor target of N processors, the + template's processors online at boot by default, a memory target of MIB + mebibytes, the template's RAM by default, the workload identity UID:GID, if + given, as which every execution runs, and whose account the guest checks + before the clone is ready, the overlay mode MODE and the scratch's journal policy POLICY, as a launch takes them, and for a GPT disk, the bound BYTES of its first partition, 64 KiB by default, as a launch's guest takes it. Every execution @@ -1885,7 +1902,7 @@ Each command of `runtime-adapter-host` but `stage`, `list`, `inspect`, and number. The guest refuses a key that it heard of before, and the epochs of a broker instance only advance, while a broker restored from a snapshot has a new instance ID, so no command repeats the key of another's execution. -- `run STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES] EXECUTION` +- `run STORE --template TEMPLATES --generation G --distro FORMAT:PATH --distro-identity UUID --upper (scratch:PATH | ram) [--processors N] [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES] EXECUTION` runs one EXECUTION, one of `exec`'s, in a [clone](#clones) of the generation G of the template store TEMPLATES, in the sandbox store STORE, which it creates if needed, then tears the clone down, as @@ -2414,9 +2431,9 @@ Windows, without a hypervisor: record of the clone takes, and the clone's record, from whose plan an execution finds that URL in each of the proxy's variables, after its own entries; and an open network, and none, with no proxy's settings; -- NVX's request of a clone, with the staged bindings, the memory target, and - the guest's defaults or the request's settings, the images that the - sandbox's store does not stage, which it refuses, and the export of the +- NVX's request of a clone, with the staged bindings, the processor and memory + targets, and the guest's defaults or the request's settings, the images that + the sandbox's store does not stage, which it refuses, and the export of the request's shares: none without shares, and with them, one directory, the root of a read-only share, holding a read-write one, mapped from an empty source and from its path. @@ -2509,10 +2526,11 @@ and Windows: request's network, over RAM, which no template keeps; the placeholder's binding, the policy revision, and the artifacts, with the initramfs that it names; and the class, with the runtime identity of the adapter's VMs and the - guest agent's build, the backend, the `auto` CPU profile, every processor - online at boot, the RAM and its capacity, the geometries that OpenVMM - presents, of 512-byte logical blocks, and the network, which names the class: - a class without it is another; + guest agent's build, the backend, the `auto` CPU profile, the processors and + those online at boot, which the request names, 1 by default, whatever the + capacity, and which name the class, the RAM and its capacity, the geometries + that OpenVMM presents, of 512-byte logical blocks, and the network, which + names the class: a class without it is another; - `runtime-adapter-host create` with a network that the plan refuses, or that it cannot read, which it refuses with status 1 before it needs NVX's artifacts or creates the store, and with one that the plan takes, or none, @@ -2755,7 +2773,8 @@ artifacts, and creates a template of it in a template store of its own: | Test | Template | Expects | | --- | --- | --- | -| `a_template_without_a_network_has_no_nic` | 2 processors, 256 MiB of RAM with a capacity of 512 MiB, a 32 MiB scratch, and no network | A captured template without a NIC | +| `a_template_without_a_network_has_no_nic` | 2 processors, 1 of them online at boot, 256 MiB of RAM with a capacity of 512 MiB, a 32 MiB scratch, and no network | A captured template without a NIC, whose key records its 1 processor online at boot, which `maxcpus=1` limits | +| `a_template_whose_processors_are_all_online_at_boot_limits_none` | 2 processors, both online at boot | A captured template whose key records no processors online at boot, without `maxcpus=` | | `a_network_that_lets_nothing_through_has_no_nic` | A network that denies egress, ingress, and host-loopback access, without a proxy | A captured template without a NIC | | `an_egress_network_has_its_nic` | Egress allowed | A captured template with a NIC | | `an_allow_rule_has_its_nic` | Egress denied but for one TCP rule to the gateway, with outbound host-loopback access | A captured template with a NIC | @@ -2763,24 +2782,25 @@ artifacts, and creates a template of it in a template store of its own: | `a_proxy_only_network_has_its_nic` | A proxy-only network | A captured template with a NIC | | `ingress_is_refused_before_any_vm_starts` | Ingress allowed | NVX's refusal, with status 1, a store without a generation, and no VMM's log or transaction in any slot | | `host_loopback_both_ways_is_refused_before_any_vm_starts` | Host-loopback access both ways | NVX's refusal, with status 1, a store without a generation, and no VMM's log or transaction in any slot | -| `a_class_that_no_capture_takes_is_refused_before_the_store` | 3 processors | NVX's refusal of the class, with status 1, before the store exists | +| `a_class_that_no_capture_takes_is_refused_before_the_store` | 3 processors, then 2 processors with 4 online at boot | NVX's refusal of each class, with status 1, before the store exists | | `a_refresh_captures_and_retires_the_generations_that_it_supersedes` | A creation, then two refreshes, the second while a clone leases the second generation, then a creation once the clone released it | `list` and `inspect` of the first generation; each refresh captures a new generation, although the store holds an eligible one, and its eviction retires the first generation, and leaves the leased second one, tombstoned, with its payload; the creation serves the third without a capture, and its eviction deletes the second | | `a_removal_reports_busy_while_leases_keep_the_generation` | A creation, two clones' leases of its generation, a removal, a second creation, then a removal after each release | `list` and `inspect` with the leases; the removal tombstones the generation, which keeps its payload and takes no new lease, and reports busy, with status 7 and its leases; the second creation captures a new generation, and leaves the first to its leases; a removal reports busy while a lease remains, and deletes the generation once none does; a second removal finds it retired, one of a generation that the store never published fails, and a generation that no lease keeps goes at once | | `a_refresh_refuses_a_class_that_no_capture_takes_before_the_store` | A refresh of 3 processors | NVX's refusal of the class, with status 1, before the store exists | -Unless a test says otherwise, a template has 1 processor, 256 MiB of RAM, its -capacity, and a 64 MiB scratch. A captured template is the store's one -generation, whose payload holds the record of its capture, its key, and its -snapshot, and whose key holds what the test asked for: the runtime identity of -the adapter's VMs with the guest agent's build, the backend, the processors, of -which a restore activates none, as all are online at boot, the RAM and its -capacity, the geometries of the placeholder and of the scratch, the NIC, and a -kernel command line of the platform tier's tokens alone, with the processors -and, exactly with a NIC, the NIC's addresses, which names neither the -placeholder's identity nor a root's tokens. Its class's slot then holds no -transaction and records no launch, and a second creation prints `existing`, -with the same generation, key, and class, without a capture: the slot's log of -the VMM stays as the capture left it. +Unless a test says otherwise, a template has 1 processor, online at boot, 256 +MiB of RAM, its capacity, and a 64 MiB scratch. A captured template is the +store's one generation, whose payload holds the record of its capture, its key, +and its snapshot, and whose key holds what the test asked for: the runtime +identity of the adapter's VMs with the guest agent's build, the backend, the +processors and those online at boot, none for a template whose processors are +all online at boot, the RAM and its capacity, the geometries of the placeholder +and of the scratch, the NIC, and a kernel command line of the platform tier's +tokens alone, with the processors, `maxcpus=` with those online at boot +exactly when they are fewer, and, exactly with a NIC, the NIC's addresses, +which names neither the placeholder's identity nor a root's tokens. Its class's +slot then holds no transaction and records no launch, and a second creation +prints `existing`, with the same generation, key, and class, without a capture: +the slot's log of the VMM stays as the capture left it. The scenarios of the operations lease generations to clones through NVX's store, as a clone's start does, but launch nothing, and release the leases as @@ -2815,6 +2835,8 @@ directory that the starts authorize: | Test | Clones | Expects | | --- | --- | --- | | `one_generation_serves_clones_of_each_upper_and_memory_target` | Two at once, over a scratch image at 256 MiB, and in RAM at 512 MiB with untrusted lowers, then one over a scratch image at 1 GiB | Each started, by the host's monotonic clock, with its own lease, which the store lists with the clone's sandbox, its memory target online, its upper, overlay mode, and image's contents, and the events of its activation in order; a stop releases each lease; the generation's snapshot unchanged, and its payload with nothing but the capture's files once the clones stopped | +| `one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity` | Of a template of 4 processors, 1 of them online at boot: in turn, one without a processor target, one of 2, and one of 4; then targets of 3 and 8 | Each started, with the CPUs of its target online, CPU 0 alone by default, of the 4 possible ones, and the CPUs beyond the first that the time ABI's record of the restore names as those that the restore brought online, none by default, and the events of its activation in order; then for each of the others, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid targets, which quarantines nothing; the lease released, and the generation's snapshot unchanged | +| `a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with` | Of a template of 4 processors, 2 of them online at boot: a target of 1, then in turn one without a processor target, and one of 4 | Status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid targets; then each started, with CPUs 0 and 1 online by default, which the restore brought none of online, and all 4 online for the target of 4, of which the restore brought CPUs 2 and 3 online | | `one_generation_serves_clones_of_either_format_over_either_upper` | Of a template whose placeholder is a GPT disk of two layers, longer than the EROFS image, four at once, each image shorter than the placeholder: the EROFS image over a scratch image and in RAM, a GPT disk of one layer, of the EROFS image's length, in RAM, and one of three layers, as short as its layout allows, over a scratch image, with a bound that its first partition meets exactly; then the disk of one layer, with a bound one byte smaller than its first partition | Each started, with its upper and the overlay's options; the EROFS image's root, without a first partition; each GPT disk's layers, ext4 file systems mounted read-only once each, from the partitions that its own table names, stacked in order, and the bytes of its own first partition; the image's read-only disk at the placeholder's length, and the scratch device at the template's scratch length, each of which the guest reads to its end; for a clone in RAM, no file of its sandbox as long as the template's scratch, and on Linux `--restore-absent-scratch` and no `scratch` block device among its VMM's arguments, which name one over a scratch image; then status 8, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of an unusable root, with the guest's status 54; nothing quarantined, and the lease released, with the records | | `a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts` | An EROFS image and a GPT disk 4 KiB longer, and an EROFS image 100 bytes longer, than the placeholder, a scratch image of twice the template's scratch, memory targets of 2 GiB and of 320 MiB, unstaged images, and a generation that the store does not hold, then the generation once its key is gone | For those that NVX's plan refuses, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of a length that the request supplied, of the image's geometry, which OpenVMM cannot attach, or of invalid targets, which quarantines nothing; for unstaged images, status 1, without a record; for the generation, status 10, with the record of a selection that names no template, which the store never published, without a lease; any lease released, and no VMM's log or record in the sandbox; for the generation without its key, status 8, and the record of a failure of the template, at the plan, by the evidence of a malformed generation, which quarantines the generation, so that the store refuses a later start's selection, with status 10, as `ineligible`, by the evidence of its tombstone, without a lease | | `a_clone_whose_activation_fails_ends_before_it_is_ready` | A scratch image of zeros, a journal that the record forbids, a workload identity whose account the image lacks, and, through NVX's restore sequence, a ping, a start, or the start of an execution before the `Activate` | Status 8, no `Ready`, and the record of a failure, at the activation, once the VM exited: of the request, by the evidence of an unusable root, with the guest's status 54, or of invalid settings, with 55, or, before the `Activate`, of an unknown cause, by the protocol's, with 51, which the sandbox records until the release; nothing quarantined, and the lease released, with the records | @@ -2888,7 +2910,7 @@ layered roots, captures of templates, and the starts and runs of their clones. | `template_operations` | `the_operations_open_a_template_store_and_create_none` | `list`, `inspect`, and `remove` of a directory that does not exist, or that holds no template store, fail with status 1, and create nothing | | `clone_start` | `a_clone_s_plan_request_and_record_carry_its_network` | A clone's network reaches the adapter's plan, NVX's request of the clone, and the clone's record, from which an execution finds the URL of the network's proxy | | `clone_start` | `a_clone_s_record_names_its_template_store_and_generation_beside_its_plan` | A clone's record names its template store and its generation, which a stop reads back, beside the plan that the other commands read | -| `clone_start` | `a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults` | NVX's request of a clone takes the staged bindings, the memory target, the guest's defaults or the request's settings, and the export of the request's shares, and refuses images that the store does not stage | +| `clone_start` | `a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults` | NVX's request of a clone takes the staged bindings, the processor and memory targets, the guest's defaults or the request's settings, and the export of the request's shares, and refuses images that the store does not stage | | `clone_start` | `a_host_s_time_keeps_its_text_form` | A host's time keeps the clock fixture's text form, on both clocks, and the texts that are none are refused | | `clone_start` | `a_run_refuses_a_network_or_shares_that_the_plan_refuses_before_it_stages_anything` | A run refuses a network that it cannot read, or that the plan refuses, and shares that it cannot export, with status 1 before it stages anything or creates the sandbox's store | | `clone_start` | `a_run_refuses_a_selection_that_names_no_template_once_its_preflight_staged_the_images` | A run refuses an image that the plan refuses with status 1 before it stages anything, and a selection that names no template with status 10 and the refusal's record once its preflight staged the images, without a lease | @@ -2992,13 +3014,14 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_layered` | `recovery_required_layer_fails_the_assembly` | A layer whose journal needs recovery is refused before any mount, whatever `noload` says, and the VM ends with 14 | | `vm_layered` | `root_profiles_enter_the_root` | Executions enter the root as PID 1 of their namespaces, with their standard streams alone, no inheritable or ambient capability, and no new privileges | | `vm_layered` | `unstaged_images_are_refused_before_spawn` | Images that the store does not stage are refused before the VM spawns, with nothing of a launch left, and boot once staged | -| `vm_template` | `a_class_that_no_capture_takes_is_refused_before_the_store` | `create` of a class that no capture takes, of 3 processors, fails with status 1 before the store exists | +| `vm_template` | `a_class_that_no_capture_takes_is_refused_before_the_store` | `create` of a class that no capture takes, of 3 processors, or of more online at boot than in all, fails with status 1 before the store exists | | `vm_template` | `a_network_that_lets_nothing_through_has_no_nic` | `create` of a network that lets nothing through captures a template without a NIC | | `vm_template` | `a_proxy_only_network_has_its_nic` | `create` of a proxy-only network captures a template with a NIC | | `vm_template` | `a_refresh_captures_and_retires_the_generations_that_it_supersedes` | `refresh` captures a new generation although the store holds an eligible one, and its eviction retires the one that it supersedes, or tombstones it while a clone leases it, as `list` and `inspect` report | | `vm_template` | `a_refresh_refuses_a_class_that_no_capture_takes_before_the_store` | `refresh` of a class that no capture takes fails with status 1 before the store exists | | `vm_template` | `a_removal_reports_busy_while_leases_keep_the_generation` | `remove` tombstones a leased generation, keeps its payload, and reports busy, with status 7 and its leases, until no lease remains, then deletes it | -| `vm_template` | `a_template_without_a_network_has_no_nic` | `create` captures a template without a NIC, whose key holds what it asked for, and serves it again without a capture | +| `vm_template` | `a_template_whose_processors_are_all_online_at_boot_limits_none` | `create` captures a template whose processors are all online at boot without `maxcpus=`, and whose key records none online at boot | +| `vm_template` | `a_template_without_a_network_has_no_nic` | `create` captures a template without a NIC, whose key holds what it asked for, its processor online at boot among it, and serves it again without a capture | | `vm_template` | `an_allow_rule_has_its_nic` | `create` of an allow rule captures a template with a NIC | | `vm_template` | `an_egress_network_has_its_nic` | `create` of a network with egress allowed captures a template with a NIC | | `vm_template` | `host_loopback_both_ways_is_refused_before_any_vm_starts` | `create` refuses host-loopback access both ways with status 1, before any VM starts, with no generation | @@ -3017,6 +3040,7 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_clone` | `a_clone_whose_view_cannot_be_set_ends_before_it_is_ready` | A generated machine ID that no bind can hold ends the clone before it is ready, with status 55, while the image's own needs no bind | | `vm_clone` | `a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released` | A clone whose VM exits once a stop found it running fails, with status 8 and the record of a failure by the evidence of its lost status, quarantines nothing, and releases its lease | | `vm_clone` | `a_generation_is_eligible_by_the_clock_of_its_capture` | The store checks eligibility by the host's monotonic clock on the boot of the capture, and by UTC after a reboot, and refuses an expired or an early generation with status 10, without a lease | +| `vm_clone` | `a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with` | NVX's plan refuses a processor target below the template's processors online at boot before any VMM starts, and the generation serves its clones from those up | | `vm_clone` | `a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity` | A host program of another runtime ends the clone through its verified identity, without a word to its guest, with status 8, and releases its lease | | `vm_clone` | `a_run_in_a_template_of_a_network_finds_its_policy_s_proxy` | A run's execution, in a clone of a proxy-only template, finds the proxy's URL in its environment, and a run of another network runs nothing, with status 8 | | `vm_clone` | `a_run_runs_one_execution_in_its_clone_then_tears_the_clone_down` | A run stages its images, starts a clone, runs one execution in its root, with the execution's environment and working directory, then stops the clone and releases its lease, whatever the execution did | @@ -3032,6 +3056,7 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_clone` | `clones_at_once_keep_their_own_uppers_and_machine_ids` | Clones at once keep their own uppers, and their own machine IDs, generated, distinct, and bound read-only, or the image's own | | `vm_clone` | `clones_at_once_see_only_their_own_exports` | Clones of one generation at once see only their own exports, and a clone without one none | | `vm_clone` | `each_command_reattaches_to_the_clone_from_a_new_process` | `reattach`, `probe`, and `exec` each reattach to the clone from a new process, in a later epoch, without a second activation | +| `vm_clone` | `one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity` | One generation serves clones of each supported processor count from the template's processors online at boot to its capacity, whose repair brings the target's CPUs online, and NVX's plan refuses an unsupported count and one beyond the capacity before any VMM starts | | `vm_clone` | `one_generation_serves_clones_of_each_upper_and_memory_target` | One generation serves clones over a scratch image and in RAM, at each memory target up to its capacity, each with its own lease, and leaves its snapshot unchanged | | `vm_clone` | `one_generation_serves_clones_of_either_format_over_either_upper` | A template of a GPT placeholder serves EROFS and GPT clones of other layouts, shorter than the placeholder, at its length, over either upper, those in RAM with no scratch file, and a first partition over its bound ends a clone with status 54 | | `vm_clone` | `the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store` | The lease of a start that ended before it recorded its clone is released only through its own template store, which `--template` names | diff --git a/tests/runtime-adapter/scenarios.txt b/tests/runtime-adapter/scenarios.txt index 28ec8f38e..7e036652f 100644 --- a/tests/runtime-adapter/scenarios.txt +++ b/tests/runtime-adapter/scenarios.txt @@ -133,6 +133,7 @@ vm_template a_proxy_only_network_has_its_nic vm_template a_refresh_captures_and_retires_the_generations_that_it_supersedes vm_template a_refresh_refuses_a_class_that_no_capture_takes_before_the_store vm_template a_removal_reports_busy_while_leases_keep_the_generation +vm_template a_template_whose_processors_are_all_online_at_boot_limits_none vm_template a_template_without_a_network_has_no_nic vm_template an_allow_rule_has_its_nic vm_template an_egress_network_has_its_nic @@ -153,6 +154,7 @@ vm_clone a_clone_whose_mapping_fails_ends_before_its_sandbox_starts vm_clone a_clone_whose_view_cannot_be_set_ends_before_it_is_ready vm_clone a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released vm_clone a_generation_is_eligible_by_the_clock_of_its_capture +vm_clone a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with vm_clone a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity vm_clone a_run_in_a_template_of_a_network_finds_its_policy_s_proxy vm_clone a_run_runs_one_execution_in_its_clone_then_tears_the_clone_down @@ -168,6 +170,7 @@ vm_clone a_tombstone_leaves_the_running_clone_of_its_generation_to_its_lease vm_clone clones_at_once_keep_their_own_uppers_and_machine_ids vm_clone clones_at_once_see_only_their_own_exports vm_clone each_command_reattaches_to_the_clone_from_a_new_process +vm_clone one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity vm_clone one_generation_serves_clones_of_each_upper_and_memory_target vm_clone one_generation_serves_clones_of_either_format_over_either_upper vm_clone the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index 288d30247..4a209e04f 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -69,16 +69,18 @@ //! nor `Start`, asks the guest agent to stop, which tears down the root, waits for the VM to //! end, abandons it, and checks that the VM ended with success and a complete teardown. //! - `runtime-adapter-host create STORE --guest-agent PATH --distro FORMAT:PATH --distro-identity -//! UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--memory MIB] -//! [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` serves a template of the -//! guest agent PATH from the template store STORE, an `nvx::host::template::Store`, which it -//! creates if needed: a generation of the template's class that the store holds, and that is -//! eligible at the host's time, or else a new one, which the store captures and publishes, with -//! `Store::create_template`. The options choose the class; see `runtime_adapter::capture`. The -//! placeholder is the raw file PATH, an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, with -//! the identity UUID, a trusted artifact of the platform, whose length is the capacity of every -//! clone's image. The template's VM runs on the hypervisor NAME, as a launch's does, with the CPU -//! profile PROFILE, `auto` by default, N processors, 1 by default, all online at boot, MIB +//! UUID [--hypervisor NAME] [--cpu-profile PROFILE] [--processors N] [--boot-processors B] +//! [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES] [--network FILE]` serves a +//! template of the guest agent PATH from the template store STORE, an +//! `nvx::host::template::Store`, which it creates if needed: a generation of the template's +//! class that the store holds, and that is eligible at the host's time, or else a new one, +//! which the store captures and publishes, with `Store::create_template`. The options choose +//! the class; see `runtime_adapter::capture`. The placeholder is the raw file PATH, an EROFS +//! filesystem, `erofs`, or a GPT disk, `gpt`, with the identity UUID, a trusted artifact of the +//! platform, whose length is the capacity of every clone's image. The template's VM runs on the +//! hypervisor NAME, as a launch's does, with the CPU profile PROFILE, `auto` by default, N +//! processors, 1 by default, of which B are online at boot, 1 by default, the fewest that its +//! clones take, since each clone's restore brings its own processor target online, up to N, MIB //! mebibytes of RAM, 256 by default, and a RAM capacity of MIB mebibytes, its RAM by default, to //! which a restore may expand it, and a dormant scratch of BYTES bytes, 64 MiB by default. It has //! one NIC under the policy of the network FILE, which the creation reads as a launch reads its @@ -120,9 +122,10 @@ //! launched and which it holds although it is not its child, attaches a new session to it, in //! the session's next epoch, and exchanges a `Ping`, leaving the VM running. //! - `runtime-adapter-host start STORE --template TEMPLATES --generation G --distro FORMAT:PATH -//! --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity -//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] -//! [--machine-id (image | generated)] [--network FILE] [SHARES]` starts a clone of the +//! --distro-identity UUID --upper (scratch:PATH | ram) [--processors N] [--memory MIB] +//! [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] +//! [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES]` +//! starts a clone of the //! generation G of the template store TEMPLATES, an `nvx::host::template::Store`, in the sandbox //! store STORE, which it creates if needed; see `runtime_adapter::clone`. The clone's VM has the //! network of the file FILE, if given, which the command reads first, as `launch` reads its @@ -131,20 +134,25 @@ //! that lets nothing through. SHARES, as `launch` takes them, share //! host paths with the clone's root: the command plans their export as a launch does, with //! NVX's `filesystem::plan`, and refuses shares that it cannot export before anything but its -//! network. It then reads the generation from the store's index, and, unless MIB mebibytes are -//! given, its compatibility key, whose RAM is then the clone's memory target, before it needs -//! NVX's artifacts. A key that cannot be read is left to NVX's plan, which reads it again, and -//! attributes the failure, to the generation or to the host, as it does for every clone. The -//! store then leases the generation to the sandbox, at the host's time, with `Store::acquire`, -//! which refuses a generation that a tombstone quarantines, or that is not eligible then, and -//! NVX's `ClonePlan` checks the clone against the generation, from the metadata of its files +//! network. It then reads the generation from the store's index, and, unless N and MIB +//! mebibytes are both given, its compatibility key, before it needs NVX's artifacts: the key's +//! processors online at boot, or all of its processors for a template that records none, are +//! then the clone's processor target unless N is given, and its RAM the clone's memory target +//! unless MIB mebibytes are given. A key that cannot be read is left to NVX's plan, which reads +//! it again, and attributes the failure, to the generation or to the host, as it does for every +//! clone. The store then leases the generation to the sandbox, at the host's time, with +//! `Store::acquire`, which refuses a generation that a tombstone quarantines, or that is not +//! eligible then, and NVX's `ClonePlan` checks the clone against the generation, from the +//! metadata of its files //! alone: its network, which must be the one that the template's key records, its image, the //! raw file PATH, an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, of //! which the guest expects the identity UUID, no longer than the template's placeholder, //! whatever the placeholder's format, and of its block sizes, its upper, the ext4 scratch image //! PATH, of the geometry of the template's scratch, or RAM, which needs no file, with the //! bindings that STORE stages for the images, as -//! `stage` stages them, its memory target, in whole blocks, from the template's RAM to its +//! `stage` stages them, its processor target, 1, 2, 4, or 8, from the template's processors +//! online at boot to its capacity, which the restore brings online, its memory target, in whole +//! blocks, from the template's RAM to its //! capacity, and its export, if it has one, which must export exactly one directory, which //! neither lies inside the template store's directory nor holds it: the export's root, which //! the clone's restore binds to the template's dormant filesystem slot. @@ -230,9 +238,10 @@ //! before, and the epochs of a broker instance only advance, while a broker restored from a //! snapshot has a new instance ID, so no command repeats the key of another's execution. //! - `runtime-adapter-host run STORE --template TEMPLATES --generation G --distro FORMAT:PATH -//! --distro-identity UUID --upper (scratch:PATH | ram) [--memory MIB] [--workload-identity -//! UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] [--first-partition-bound BYTES] -//! [--machine-id (image | generated)] [--network FILE] [SHARES] EXECUTION` runs one execution in +//! --distro-identity UUID --upper (scratch:PATH | ram) [--processors N] [--memory MIB] +//! [--workload-identity UID:GID] [--overlay-mode MODE] [--scratch-journal POLICY] +//! [--first-partition-bound BYTES] [--machine-id (image | generated)] [--network FILE] [SHARES] +//! EXECUTION` runs one execution in //! a clone of the generation G of the template store TEMPLATES, in the sandbox store STORE, //! which it creates if needed, then tears the clone down. The EXECUTION is one of `exec`'s, whose //! options may come among the start's, in any order, and whose PROGRAM and ARGs are every @@ -438,7 +447,8 @@ use nvx::uuid::Uuid; use nvx::{ErrorCode, ErrorKind, Hypervisor}; use runtime_adapter::GUEST_BUILD; use runtime_adapter::capture::{ - DEFAULT_CPU_PROFILE, DEFAULT_PROCESSORS, DEFAULT_SCRATCH_LENGTH, Request, + DEFAULT_BOOT_PROCESSORS, DEFAULT_CPU_PROFILE, DEFAULT_PROCESSORS, DEFAULT_SCRATCH_LENGTH, + Request, }; use runtime_adapter::clone::{CloneRecord, host_time_from_env}; use runtime_adapter::exec::{self, AccountMode, HostExecution, Start}; @@ -549,8 +559,9 @@ const USAGE: &str = concat!( " --distro-identity UUID\n", " [--hypervisor NAME] [--guest-build BUILD] [--cpu-profile PROFILE]", " [--processors N]\n", - " [--memory MIB] [--memory-capacity MIB] [--scratch-length BYTES]", - " [--network FILE]\n", + " [--boot-processors N] [--memory MIB] [--memory-capacity MIB]", + " [--scratch-length BYTES]\n", + " [--network FILE]\n", " runtime-adapter-host list STORE\n", " runtime-adapter-host (inspect | remove) STORE GENERATION\n", " runtime-adapter-host (reattach | stop) STORE [--guest-build BUILD]", @@ -558,8 +569,8 @@ const USAGE: &str = concat!( " runtime-adapter-host start STORE --template TEMPLATES --generation G", " --distro FORMAT:PATH\n", " --distro-identity UUID --upper (scratch:PATH | ram) [--guest-build BUILD]", - " [--memory MIB]\n", - " [--workload-identity UID:GID] [--overlay-mode MODE]", + " [--processors N]\n", + " [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE]", " [--scratch-journal POLICY]\n", " [--first-partition-bound BYTES] [--machine-id (image | generated)]", " [--network FILE]\n", @@ -568,8 +579,8 @@ const USAGE: &str = concat!( " runtime-adapter-host run STORE --template TEMPLATES --generation G", " --distro FORMAT:PATH\n", " --distro-identity UUID --upper (scratch:PATH | ram) [--guest-build BUILD]", - " [--memory MIB]\n", - " [--workload-identity UID:GID] [--overlay-mode MODE]", + " [--processors N]\n", + " [--memory MIB] [--workload-identity UID:GID] [--overlay-mode MODE]", " [--scratch-journal POLICY]\n", " [--first-partition-bound BYTES] [--machine-id (image | generated)]", " [--network FILE]\n", @@ -813,8 +824,10 @@ struct Template { identity: Uuid, /// The CPU profile. cpu_profile: String, - /// The processors, all online at boot. + /// The processors: the processor capacity. processors: u32, + /// The processors online at boot. + boot_processors: u32, /// The RAM, in bytes. memory: u64, /// The RAM capacity, in bytes. @@ -938,6 +951,8 @@ struct Options { cpu_profile: Option, /// `--processors`. processors: Option, + /// `--boot-processors`. + boot_processors: Option, /// `--memory`, in bytes. memory: Option, /// `--memory-capacity`, in bytes. @@ -1196,6 +1211,7 @@ fn parse(arguments: Vec) -> Result { .cpu_profile .unwrap_or_else(|| DEFAULT_CPU_PROFILE.to_owned()), processors: options.processors.unwrap_or(DEFAULT_PROCESSORS), + boot_processors: options.boot_processors.unwrap_or(DEFAULT_BOOT_PROCESSORS), memory, memory_capacity: options.memory_capacity.unwrap_or(memory), scratch_length: options.scratch_length.unwrap_or(DEFAULT_SCRATCH_LENGTH), @@ -1357,16 +1373,11 @@ fn parse_option( ("create", "--cpu-profile") if options.cpu_profile.is_none() => { options.cpu_profile = Some(text(value)?); } - ("create", "--processors") if options.processors.is_none() => { - let value = text(value)?; - let processors = value - .parse::() - .ok() - .filter(|processors| *processors > 0); - options.processors = - Some(processors.ok_or_else(|| { - format!("--processors takes a positive number, not {value:?}") - })?); + ("create" | "start", "--processors") if options.processors.is_none() => { + options.processors = Some(count(option, &text(value)?)?); + } + ("create", "--boot-processors") if options.boot_processors.is_none() => { + options.boot_processors = Some(count(option, &text(value)?)?); } ("create" | "start", "--memory") if options.memory.is_none() => { options.memory = Some(mebibytes(option, &text(value)?)?); @@ -1643,6 +1654,7 @@ fn clone_options(name: &str, options: Options) -> Result { format, identity, scratch, + processors: options.processors, memory: options.memory, workload_identity: options.workload_identity, overlay_mode: options.overlay_mode, @@ -1720,6 +1732,16 @@ fn mebibytes(option: &str, value: &str) -> Result { .ok_or_else(|| format!("{option} takes a positive number of MiB, not {value:?}")) } +/// Returns `value`, the value of `option`, a positive number of processors, or says why it is not +/// one. +fn count(option: &str, value: &str) -> Result { + value + .parse::() + .ok() + .filter(|count| *count > 0) + .ok_or_else(|| format!("{option} takes a positive number, not {value:?}")) +} + /// Stages the image `distro`, in `format`, of which the guest expects `identity`, and the ext4 /// scratch image `scratch`, if any, in the sandbox store `store`, which it creates if needed. fn stage( @@ -2085,6 +2107,7 @@ fn create( hypervisor, cpu_profile: template.cpu_profile.clone(), processors: template.processors, + boot_processors: template.boot_processors, memory: template.memory, memory_capacity: template.memory_capacity, scratch_length: template.scratch_length, @@ -2415,14 +2438,26 @@ fn start_clone( &error, ))); }; - // The clone's memory target is the request's, or else the template's RAM, which the - // generation's key gives. A key that cannot be read gives none: NVX's plan reads the key + // The clone's processor and memory targets are the request's, or else the template's, which + // the generation's key gives: its processors online at boot, all of them for a template that + // records none, and its RAM. A key that cannot be read gives none: NVX's plan reads the key // again, once the store leased the generation, and attributes the failure, to the generation - // or to the host, before it checks a memory target. - let memory = clone - .request + // or to the host, before it checks a target. + let targets = &clone.request; + let key = if targets.processors.is_none() || targets.memory.is_none() { + templates.read_key(&generation).ok() + } else { + None + }; + let processors = targets.processors.or_else(|| { + key.as_ref().map(|key| match key.boot_processors() { + 0 => key.processors(), + boot => boot, + }) + }); + let memory = targets .memory - .or_else(|| templates.read_key(&generation).ok().map(|key| key.memory())); + .or_else(|| key.as_ref().map(|key| key.memory())); let runtime = runtime(guest_build)?; runtime .artifacts @@ -2460,6 +2495,7 @@ fn start_clone( request: &request, plan, export, + processors, memory, runtime: &runtime, guest_build, @@ -2563,6 +2599,9 @@ struct CloneLaunch<'a> { plan: nvx::host::sandbox::SandboxPlan, /// The export of the host paths that the clone shares with its root, if any. export: Option, + /// The clone's processor target, or none if the request names none, and the generation's + /// key, whose processors online at boot it would be, cannot be read. + processors: Option, /// The clone's memory target, in bytes, or none if the request names none, and the /// generation's key, whose RAM it would be, cannot be read. memory: Option, @@ -2590,6 +2629,7 @@ impl CloneLaunch<'_> { request, plan, export, + processors, memory, runtime, guest_build, @@ -2597,10 +2637,17 @@ impl CloneLaunch<'_> { let staged = Staged::read(acquired.locked.dir()) .map_err(|error| Failed::Command(describe(&error)))?; let bindings = staged.map_or_else(Bindings::default, |staged| staged.bindings(&plan)); - // No template serves a memory target of 0, which stands for the one of a key that could - // not be read: the plan fails as it reads the key again, before it checks the target. + // No template serves a processor or memory target of 0, which stands for the one of a key + // that could not be read: the plan fails as it reads the key again, before it checks the + // targets. let clone_request = request - .clone_request(plan.clone(), export, &bindings, memory.unwrap_or(0)) + .clone_request( + plan.clone(), + export, + &bindings, + processors.unwrap_or(0), + memory.unwrap_or(0), + ) .map_err(|error| Failed::Command(binding_error(&error)))?; let clone_plan = ClonePlan::new( templates, diff --git a/tests/runtime-adapter/src/capture.rs b/tests/runtime-adapter/src/capture.rs index d3e69bfa3..22ffbb406 100644 --- a/tests/runtime-adapter/src/capture.rs +++ b/tests/runtime-adapter/src/capture.rs @@ -4,8 +4,8 @@ //! create` asks. //! //! A [`Request`] holds what the host program's options choose of a template: its placeholder, the -//! backend, the CPU profile, the processors, all online at boot, the RAM and its capacity, the -//! length of the dormant scratch, and the network, if any. [`Request::capture`] returns NVX's +//! backend, the CPU profile, the processors and those online at boot, the RAM and its capacity, +//! the length of the dormant scratch, and the network, if any. [`Request::capture`] returns NVX's //! `Capture` of it: the adapter's plan of the placeholder, which the host program plans as it //! plans an image, with the network, the placeholder's binding, which the host program stages as //! it stages an image, NVX's artifacts with the initramfs of the adapter's VMs, and the class of @@ -15,7 +15,9 @@ //! - the runtime identity of the adapter's VMs, which [`crate::runtime`] gives, and the guest //! agent's build, as `runtime-adapter-guest/BUILD`; //! - the backend and the CPU profile, as OpenVMM's `--hypervisor` and `--cpu-profile` take them; -//! - the processors, all online at boot, and the RAM and its capacity; +//! - the processors, the template's processor capacity, and those online at boot, the fewest that +//! its clones take, from which a restore brings each clone's processor target online, and the +//! RAM and its capacity; //! - the geometries of the placeholder and of the dormant scratch, as OpenVMM presents a sandbox //! block device, which [`geometry`] gives; //! - the network, which the adapter's plan renders too, and no other invariant setting. @@ -23,8 +25,9 @@ //! The capture's VM boots NVX's kernel with the initramfs of the adapter's VMs, which puts the //! guest agent in place of NVX's agents; see [`crate::initramfs`]. NVX gives it the kernel command //! line `nvx_sandbox=1`, with which NVX's `/init` hands it to the guest agent's PID 1 of a layered -//! root, and OpenVMM adds the platform tier's tokens, `nvx_snapshot_tier=platform` among them: the -//! worker then captures the VM before it assembles any root, and the capture ends the VM. +//! root, and for a template whose processors are not all online at boot, `maxcpus=` with those +//! that are, and OpenVMM adds the platform tier's tokens, `nvx_snapshot_tier=platform` among them: +//! the worker then captures the VM before it assembles any root, and the capture ends the VM. use std::path::{Path, PathBuf}; @@ -45,9 +48,14 @@ use crate::runtime::Runtime; /// selects without the option. pub const DEFAULT_CPU_PROFILE: &str = "auto"; -/// The processors of a template, unless the host program names another number. +/// The processors of a template, its processor capacity, unless the host program names another +/// number. pub const DEFAULT_PROCESSORS: u32 = 1; +/// The processors of a template that are online at boot, unless the host program names another +/// number: 1, with which the template serves clones of every processor count up to its capacity. +pub const DEFAULT_BOOT_PROCESSORS: u32 = 1; + /// The length of a template's dormant scratch, in bytes, unless the host program names another: /// 64 MiB, the length of the scratch template that NVX's CI makes with the guest artifacts. pub const DEFAULT_SCRATCH_LENGTH: u64 = 64 << 20; @@ -75,8 +83,11 @@ pub struct Request { pub hypervisor: Hypervisor, /// The CPU profile, as OpenVMM's `--cpu-profile` takes it: `auto`, `host`, or a profile ID. pub cpu_profile: String, - /// The processors, all online at boot. + /// The processors: the processor capacity, the most that a clone of the template takes. pub processors: u32, + /// The processors online at boot, at most [`processors`](Self::processors): the fewest that a + /// clone of the template takes. + pub boot_processors: u32, /// The RAM at boot, in bytes. pub memory: u64, /// The RAM capacity, in bytes, to which a restore may expand the RAM. @@ -126,7 +137,7 @@ impl Request { hypervisor: self.hypervisor, cpu_profile: self.cpu_profile.clone(), processors: self.processors, - boot_processors: self.processors, + boot_processors: self.boot_processors, memory: self.memory, memory_capacity: self.memory_capacity, distro: geometry(adapter.distro().path(), binding.image().length())?, diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index 073e0b74b..77fa06b10 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -3,11 +3,12 @@ //! //! A [`Request`] holds what the host program's options choose of a clone: its image, an EROFS //! layer or a GPT disk no longer than the template's placeholder, its upper, on an ext4 scratch -//! image or in RAM, its memory target, the workload identity as which its executions run, its -//! overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, the -//! machine ID that its executions see, the image's own or one that its guest generates, the host -//! paths that it shares with its root, as a launch shares them, `sharing::HostFsSharing`, and its -//! network, as a launch reads it, `layered::read_network`, which must be the template's. +//! image or in RAM, its processor and memory targets, the workload identity as which its +//! executions run, its overlay mode, its scratch's journal policy, the bound of a GPT disk's +//! first partition, the machine ID that its executions see, the image's own or one that its +//! guest generates, the host paths that it shares with its root, as a launch shares them, +//! `sharing::HostFsSharing`, and its network, as a launch reads it, `layered::read_network`, +//! which must be the template's. //! [`Request::plan`] plans the image over the upper, with the network, as a launch plans a layered //! root, [`Request::export`] plans the export of the shares as a launch plans it, and //! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of both, with the @@ -77,6 +78,8 @@ pub struct Request { pub identity: Uuid, /// The ext4 scratch image of the upper, or `None` for an upper in RAM. pub scratch: Option, + /// The processor target, or `None` for the template's processors online at boot. + pub processors: Option, /// The memory target, in bytes, or `None` for the template's RAM. pub memory: Option, /// The workload identity as which every execution runs, if any: its user ID and its group ID. @@ -135,7 +138,7 @@ impl Request { /// Returns NVX's request of the clone of `plan`, the [plan](Self::plan) of this request, and /// of `export`, the [export](Self::export) of its shares, if any, with the bindings that the /// sandbox's store stages for its images, `bindings`, as `staged::Staged::bindings` finds - /// them, and the memory target `memory`, in bytes. + /// them, the processor target `processors`, and the memory target `memory`, in bytes. /// /// The record of the clone's activation names the workload identity, if any, the /// [`CLONE_HOSTNAME`], the request's machine ID, the overlay's mode, and the scratch's @@ -151,6 +154,7 @@ impl Request { plan: SandboxPlan, export: Option, bindings: &Bindings, + processors: u32, memory: u64, ) -> Result { let binding = bindings @@ -180,6 +184,7 @@ impl Request { policy_revision: DISTRO_POLICY_REVISION, scratch, export, + processors, memory, workload: WorkloadView { hostname: CLONE_HOSTNAME.as_bytes().to_vec(), diff --git a/tests/runtime-adapter/src/template.rs b/tests/runtime-adapter/src/template.rs index 625ca1a0f..1f3203741 100644 --- a/tests/runtime-adapter/src/template.rs +++ b/tests/runtime-adapter/src/template.rs @@ -39,7 +39,7 @@ use nvx::protocol::cmdline::{self, Parameter}; /// `nvx_sandbox=1`, `nvx_config=0xd0010000,65536`, [`PLATFORM_TIER`], and the control tty's token, /// `nvx_control_tty=hvc2`, whole, and the tokens that start with `nr_cpus=`, `maxcpus=`, /// `virtio_mmio.device=`, `virtnet_ip=`, `virtnet_mask=`, `virtnet_gw=`, `virtnet_dns=`, -/// `virtnet_ip6=`, or `virtnet_gw6=`, whose values OpenVMM writes. +/// `virtnet_ip6=`, or `virtnet_gw6=`, whose values OpenVMM writes or checks. /// /// It is OpenVMM's own check, `platform_command_line_token_is_invariant` of /// `openvmm_helpers::snapshot::microvm`, compiled from the excerpts of OpenVMM's sources that diff --git a/tests/runtime-adapter/src/worker/probes.rs b/tests/runtime-adapter/src/worker/probes.rs index 427765da6..2d80baa07 100644 --- a/tests/runtime-adapter/src/worker/probes.rs +++ b/tests/runtime-adapter/src/worker/probes.rs @@ -12,6 +12,7 @@ //! | `reads` | The sectors that the guest read from each of its disks, by the disk's statistics: `disks`, each as `NAME:SECTORS`, in 512-byte sectors, in the order of their names, separated by spaces | //! | `read-disks` | It reads each of the guest's disks from its start to its end, through the page cache: `disks`, each as `NAME:BYTES`, the bytes that it read, in the order of their names, separated by spaces | //! | `memory` | The guest's memory: the size of its memory blocks, `block-size`, how many are online, `blocks`, and their bytes, `online`, and the memory that the kernel manages, `total`, each in bytes | +//! | `processors` | The guest's CPUs, as the kernel lists them: those that are online, `online`, present, `present`, and possible, `possible`; and the CPUs that the restore of a clone brought online, as the time ABI's record of the restore names them, `restored`, `none` for none, or `absent` without a record | //! | `time` | The guest's time state, [`TIME_STATE`], which NVX's time service keeps: each line `KEY=VALUE` as a fact, with each `_` of its key as `-`, in order, leaving out a key that no fact may have, and a repeated one | //! | `first-partition` | The bytes of the first partition of a GPT disk that the assembly returned, without their trailing padding: how many, `length`, and the bytes, `bytes`, in hexadecimal, or `none` for both for an EROFS image | //! | `cgroups` | The file system at the cgroups' parent, `parent.type`, and its `parent.subtree-control`; whether each cgroup exists, `agent.exists` and `workload.exists`; `agent.memory.low`, `workload.memory.max`, and `workload.pids.max`; the number of processes in each, `agent.processes` and `workload.processes`; and `user-namespaces`, the guest's limit of user namespaces, or `unavailable` without them | @@ -56,6 +57,11 @@ const FILL_SLACK: u64 = 16 << 20; const MEMORY_INFORMATION: &str = "/proc/meminfo"; /// The guest's memory blocks, of which the `memory` probe counts those online. const MEMORY_BLOCKS: &str = "/sys/devices/system/memory"; +/// The guest's CPUs, which the `processors` probe lists. +const CPUS: &str = "/sys/devices/system/cpu"; +/// The time ABI's record of the restore of a clone, whose `new_cpus=` line names the CPUs that the +/// restore brought online. +const TIME_RESTORE: &str = "/run/nvx/time-restore"; /// The guest's block devices. const BLOCK_DEVICES: &str = "/sys/block"; /// The directory of the guest's device nodes, through which the `read-disks` probe reads its disks. @@ -139,6 +145,7 @@ pub fn run(probe: &Probe, context: &Context<'_>, held: &mut Vec) -> Result "reads", "read-disks", "memory", + "processors", "time", "first-partition", "cgroups", @@ -164,6 +171,7 @@ pub fn run(probe: &Probe, context: &Context<'_>, held: &mut Vec) -> Result ("reads", _) => reads(), ("read-disks", _) => read_disks(), ("memory", _) => memory(), + ("processors", _) => Ok(processors()), ("time", _) => time(), ("first-partition", _) => Ok(first_partition(context)), ("cgroups", _) => cgroups(context), @@ -449,6 +457,34 @@ fn memory() -> io::Result { Ok(facts) } +/// Runs the `processors` probe. +fn processors() -> Facts { + let cpus = Path::new(CPUS); + let mut facts = Facts::new(); + for list in ["online", "present", "possible"] { + facts.push(list, read(&cpus.join(list))); + } + let restored = match fs::read_to_string(TIME_RESTORE) { + Ok(record) => restored_cpus(&record), + Err(error) if error.kind() == io::ErrorKind::NotFound => "absent".to_owned(), + Err(error) => format!("unreadable: {error}"), + }; + facts.push("restored", restored); + facts +} + +/// Returns the CPUs that the time ABI's record of a restore, `record`, names as the CPUs that the +/// restore brought online, after `new_cpus=`, or `unreadable: PROBLEM` for a record without them. +fn restored_cpus(record: &str) -> String { + record + .lines() + .find_map(|line| line.strip_prefix("new_cpus=")) + .map_or_else( + || "unreadable: the record names no new CPUs".to_owned(), + str::to_owned, + ) +} + /// Runs the `time` probe. fn time() -> io::Result { Ok(time_facts(&fs::read_to_string(TIME_STATE)?)) @@ -875,7 +911,7 @@ rw,lowerdir=/run/a\\040b/layers/distro,metacopy=on,xino=on run(&with_argument, &context, &mut held), Err("the first-partition probe takes no arguments".to_owned()) ); - for name in ["reads", "read-disks", "memory", "time"] { + for name in ["reads", "read-disks", "memory", "processors", "time"] { let with_argument = Probe::parse(&format!("{name} vda")).unwrap(); assert_eq!( run(&with_argument, &context, &mut held), @@ -978,4 +1014,14 @@ rw,lowerdir=/run/a\\040b/layers/distro,metacopy=on,xino=on assert_eq!(run(&probe, &context, &mut held), Ok(expected)); assert_eq!(hexadecimal(&[0x0f, 0xa0, 0x5c]), "0fa05c"); } + + #[test] + fn the_cpus_that_a_restore_brought_online_are_those_of_its_record() { + assert_eq!(restored_cpus("generation=3\nnew_cpus=1-3\n"), "1-3"); + assert_eq!(restored_cpus("generation=3\nnew_cpus=none\n"), "none"); + assert_eq!( + restored_cpus("generation=3\n"), + "unreadable: the record names no new CPUs" + ); + } } diff --git a/tests/runtime-adapter/tests/clone_start.rs b/tests/runtime-adapter/tests/clone_start.rs index 8d39b2b16..72d7bc9ec 100644 --- a/tests/runtime-adapter/tests/clone_start.rs +++ b/tests/runtime-adapter/tests/clone_start.rs @@ -132,6 +132,14 @@ fn the_host_program_refuses_options_that_do_not_make_a_start() { with(&["--upper", "ram", "--memory", "0"]), "--memory takes a positive number of MiB, not \"0\"", ), + ( + with(&["--upper", "ram", "--processors", "0"]), + "--processors takes a positive number, not \"0\"", + ), + ( + with(&["--upper", "ram", "--boot-processors", "1"]), + "unexpected option \"--boot-processors\" of start", + ), ( with(&["--upper", "ram", "--workload-identity", "0:1000"]), "--workload-identity takes UID:GID, two numbers from 1 to 4294967294", @@ -694,6 +702,7 @@ fn a_clone_s_record_names_its_template_store_and_generation_beside_its_plan() { format: ImageFormat::Erofs, identity: identity(), scratch: Some(scratch.clone()), + processors: None, memory: None, workload_identity: Some((1000, 1000)), overlay_mode: None, @@ -785,6 +794,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { format: ImageFormat::Erofs, identity: identity(), scratch: Some(scratch.clone()), + processors: None, memory: None, workload_identity: None, overlay_mode: None, @@ -797,15 +807,15 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { let plan = request.plan().unwrap(); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), Some(&scratch)).unwrap(); let bindings = staged.bindings(&plan); - let memory = 512 << 20; + let (processors, memory) = (2, 512 << 20); let clone = request - .clone_request(plan.clone(), None, &bindings, memory) + .clone_request(plan.clone(), None, &bindings, processors, memory) .unwrap(); assert_eq!(clone.adapter, plan); assert_eq!(clone.binding, staged.distro.binding); assert_eq!(clone.scratch, Some(staged.scratch.unwrap().binding)); assert_eq!(clone.export, None); - assert_eq!(clone.memory, memory); + assert_eq!((clone.processors, clone.memory), (processors, memory)); assert_eq!( clone.policy_revision, nvx::host::sandbox::distro::DISTRO_POLICY_REVISION @@ -827,7 +837,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { ..request.clone() }; let clone = named - .clone_request(plan.clone(), None, &bindings, memory) + .clone_request(plan.clone(), None, &bindings, processors, memory) .unwrap(); assert_eq!( clone.workload.identity, @@ -854,7 +864,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { ), ] { let error = request - .clone_request(plan.clone(), None, &bindings, memory) + .clone_request(plan.clone(), None, &bindings, processors, memory) .unwrap_err(); assert_eq!(error.kind(), nvx::ErrorKind::Unavailable, "{what}"); assert!( @@ -872,12 +882,18 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { }; let plan = in_ram.plan().unwrap(); let error = in_ram - .clone_request(plan.clone(), None, &Bindings::default(), memory) + .clone_request(plan.clone(), None, &Bindings::default(), processors, memory) .unwrap_err(); assert_eq!(error.kind(), nvx::ErrorKind::Unavailable); let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), None).unwrap(); let clone = in_ram - .clone_request(plan.clone(), None, &staged.bindings(&plan), memory) + .clone_request( + plan.clone(), + None, + &staged.bindings(&plan), + processors, + memory, + ) .unwrap(); assert_eq!(clone.scratch, None); assert_eq!(clone.adapter, plan); @@ -911,6 +927,7 @@ fn a_clone_s_request_takes_the_staged_bindings_and_the_guest_s_defaults() { plan.clone(), Some(export.clone()), &staged.bindings(&plan), + processors, memory, ) .unwrap(); @@ -1038,6 +1055,7 @@ fn a_clone_s_plan_request_and_record_carry_its_network() { format: ImageFormat::Erofs, identity: identity(), scratch: None, + processors: None, memory: None, workload_identity: None, overlay_mode: None, @@ -1058,7 +1076,7 @@ fn a_clone_s_plan_request_and_record_carry_its_network() { let staged = Staged::stage(&distro, ImageFormat::Erofs, identity(), None).unwrap(); let bindings = staged.bindings(&plan); let clone = request - .clone_request(plan.clone(), None, &bindings, 512 << 20) + .clone_request(plan.clone(), None, &bindings, 1, 512 << 20) .unwrap(); assert_eq!(clone.adapter, plan); assert_eq!( diff --git a/tests/runtime-adapter/tests/template_capture.rs b/tests/runtime-adapter/tests/template_capture.rs index c5ea68f5c..78845e3a8 100644 --- a/tests/runtime-adapter/tests/template_capture.rs +++ b/tests/runtime-adapter/tests/template_capture.rs @@ -16,8 +16,8 @@ use nvx::host::store::{ArtifactIdentity, RuntimeIdentity}; use nvx::protocol::root::{ImageFormat, RootImage}; use nvx::uuid::Uuid; use runtime_adapter::capture::{ - DEFAULT_CPU_PROFILE, DEFAULT_PROCESSORS, DEFAULT_SCRATCH_LENGTH, LOGICAL_BLOCK_SIZE, Request, - geometry, + DEFAULT_BOOT_PROCESSORS, DEFAULT_CPU_PROFILE, DEFAULT_PROCESSORS, DEFAULT_SCRATCH_LENGTH, + LOGICAL_BLOCK_SIZE, Request, geometry, }; use runtime_adapter::layered::read_network; use runtime_adapter::runtime::Runtime; @@ -83,6 +83,7 @@ fn template(distro: &Path, network: Option) -> Request { hypervisor: Hypervisor::Kvm, cpu_profile: DEFAULT_CPU_PROFILE.to_owned(), processors: DEFAULT_PROCESSORS, + boot_processors: DEFAULT_BOOT_PROCESSORS, memory: 256 * MIB, memory_capacity: 256 * MIB, scratch_length: DEFAULT_SCRATCH_LENGTH, @@ -152,7 +153,7 @@ fn a_capture_takes_its_class_and_inputs_from_the_request() { assert_eq!(capture.artifacts.initrd, initramfs); // The class: the runtime, the guest agent's build, the machine that the request asks for, - // with every processor online at boot, the geometries that OpenVMM presents, and the network. + // with its processors online at boot, the geometries that OpenVMM presents, and the network. let class = &capture.class; assert_eq!(class.runtime, runtime.identity); assert_eq!(class.guest_build.as_str(), "runtime-adapter-guest/build"); @@ -179,14 +180,15 @@ fn a_capture_takes_its_class_and_inputs_from_the_request() { assert!(class.invariants.is_empty()); let with_nic = class.identity().unwrap(); - // Every processor is online at boot, whatever the request's number, and the network names - // the class: one without a network is another. + // A template keeps the processors online at boot that the request names, 1 by default, the + // fewest that its clones take, whatever its capacity, and the network names the class: one + // without a network is another. let mut more = template(&distro, None); more.processors = 4; more.memory_capacity = 512 * MIB; let capture = more .capture( - staged.distro.binding, + staged.distro.binding.clone(), &runtime, "build", &initramfs, @@ -195,12 +197,28 @@ fn a_capture_takes_its_class_and_inputs_from_the_request() { .unwrap(); assert_eq!( (capture.class.processors, capture.class.boot_processors), - (4, 4) + (4, 1) ); assert_eq!(capture.class.memory_capacity, 512 * MIB); assert_eq!(capture.adapter.network(), None); assert_eq!(capture.class.network, None); assert_ne!(capture.class.identity().unwrap(), with_nic); + let fewer = capture.class.identity().unwrap(); + more.boot_processors = 2; + let capture = more + .capture( + staged.distro.binding, + &runtime, + "build", + &initramfs, + &directory, + ) + .unwrap(); + assert_eq!( + (capture.class.processors, capture.class.boot_processors), + (4, 2) + ); + assert_ne!(capture.class.identity().unwrap(), fewer); // OpenVMM presents logical blocks of 512 bytes. let presented = geometry(&distro, 4096).unwrap(); diff --git a/tests/runtime-adapter/tests/template_operations.rs b/tests/runtime-adapter/tests/template_operations.rs index d9903d7a9..e9c12c731 100644 --- a/tests/runtime-adapter/tests/template_operations.rs +++ b/tests/runtime-adapter/tests/template_operations.rs @@ -213,6 +213,10 @@ fn the_host_program_refuses_command_lines_of_the_operations() { with(&["--processors", "0"]), "--processors takes a positive number, not \"0\"", ), + ( + with(&["--boot-processors", "one"]), + "--boot-processors takes a positive number, not \"one\"", + ), ( with(&["--memory", "256M"]), "--memory takes a positive number of MiB, not \"256M\"", diff --git a/tests/runtime-adapter/tests/vm_clone.rs b/tests/runtime-adapter/tests/vm_clone.rs index cc03d6232..8af71080b 100644 --- a/tests/runtime-adapter/tests/vm_clone.rs +++ b/tests/runtime-adapter/tests/vm_clone.rs @@ -18,6 +18,13 @@ //! memory target from the template's RAM to its capacity, which the guest has online. The store //! leases the generation to each, a stop releases it, and the clones change nothing of the //! generation's snapshot, and leave nothing in its payload. +//! - One generation of a template of 4 processors, one of them online at boot, serves clones of +//! each processor count from that one to its capacity: a clone takes the template's processors +//! online at boot by default, and the restore of each other target brings the rest of its prefix +//! online, which the guest has online, and which the time ABI's record of the restore names. +//! NVX's plan refuses a count that a microVM cannot have, and one beyond the capacity, before +//! any VMM starts. The generation of a template that boots with 2 of its 4 processors online +//! refuses a clone of 1, and serves those of 2, its default, and 4. //! - One generation serves clones of either format over either upper, at once: the platform's //! EROFS image, and GPT disks of other layouts than the template's placeholder, a GPT disk too, //! and than each other's. The guest of a GPT clone rereads its disk's partition table before it @@ -730,11 +737,36 @@ impl Template { Self::capture(name, None, Some(network)) } + /// Creates the sandbox of the scenario `name`, stages the platform's copy of the image, and + /// captures the template of the image with a capacity of `processors` processors, of which + /// `boot` are online at boot, or the host program's default, as [`create`](Self::create) + /// does. + fn of_processors(name: &str, processors: u32, boot: Option) -> Self { + let processors = processors.to_string(); + let boot = boot.map(|boot| boot.to_string()); + let mut options = vec!["--processors", processors.as_str()]; + options.extend( + boot.iter() + .flat_map(|boot| ["--boot-processors", boot.as_str()]), + ); + Self::captured(name, None, None, &options) + } + /// Creates the sandbox of the scenario `name`, stages the platform's copy of the image, and /// captures the template of the GPT disk `placeholder`, if any, [`PLACEHOLDER_SLACK`] longer /// than the image, and of the image otherwise, with the network `network`, if any, with the /// guest at [`GUEST_NETWORK`]. fn capture(name: &str, placeholder: Option<&Disk>, network: Option) -> Self { + Self::captured(name, placeholder, network, &[]) + } + + /// Captures the template as [`capture`](Self::capture) does, with the options `extra` too. + fn captured( + name: &str, + placeholder: Option<&Disk>, + network: Option, + extra: &[&str], + ) -> Self { let mut sandbox = Sandbox::new(name); let fixtures = Fixtures::find(); let image = stage_platform(&sandbox.root.join("platform"), &fixtures); @@ -764,6 +796,7 @@ impl Template { if let Some(network) = &network { options.extend([OsStr::new("--network"), network.as_os_str()]); } + options.extend(extra.iter().map(OsStr::new)); let created = sandbox.create(&options, 0); assert_eq!(created["created"], "captured", "{created:?}"); assert_eq!(created["generation"], GENERATION.to_string(), "{created:?}"); @@ -1971,6 +2004,7 @@ fn refuses_early(template: &Template, early: Message) { format: ImageFormat::Erofs, identity: Uuid::parse(&template.identity).unwrap(), scratch: None, + processors: None, memory: None, workload_identity: None, overlay_mode: None, @@ -1984,7 +2018,13 @@ fn refuses_early(template: &Template, early: Message) { let staged = Staged::read(&sandbox.store).unwrap().unwrap(); let key = store.read_key(&template.generation()).unwrap(); let clone = request - .clone_request(plan.clone(), None, &staged.bindings(&plan), key.memory()) + .clone_request( + plan.clone(), + None, + &staged.bindings(&plan), + key.processors(), + key.memory(), + ) .unwrap(); let now = HostTime::now().unwrap(); let mut acquired = store @@ -2140,6 +2180,122 @@ fn one_generation_serves_clones_of_each_upper_and_memory_target() { assert_payload(&template, false); } +/// Checks that a clone's `processors` probe found the CPUs `online` online, of the possible CPUs +/// `possible`, and that the restore brought the CPUs `restored` online, `none` for none. +fn assert_processors(facts: &Printed, online: &str, possible: &str, restored: &str) { + let found = ["online", "possible", "restored"].map(|key| &facts[&format!("processors.{key}")]); + assert_eq!(found, [online, possible, restored], "{facts:?}"); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity() +{ + // A template of 4 processors, one of them online at boot, the host program's default. + let template = Template::of_processors("cl-cpus", 4, None); + let snapshot = template.payload().join(SNAPSHOT_NAME); + let captured = files_below(&snapshot); + // A clone takes the processors online at boot by default, which its restore brings none + // online of, and the restore of each other target brings the rest of its prefix online. + for (options, online, restored) in [ + (&[][..], "0", "none"), + (&["--processors", "2"][..], "0-1", "1"), + (&["--processors", "4"][..], "0-3", "1-3"), + ] { + let start = StartOf { + options, + ..StartOf::default() + }; + let mut cloned = template.start("cl-n", start, 0); + cloned.assert_started(); + let facts = cloned.probe(&["processors", "activation"]); + assert_processors(&facts, online, "0-3", restored); + assert_timeline(&facts); + cloned.stop(); + } + assert!(template.leases().is_empty()); + + // A count that a microVM cannot have, and one beyond the template's capacity, are refused + // before any VMM starts. + for target in ["3", "8"] { + let options = ["--processors", target]; + let start = StartOf { + options: &options, + ..StartOf::default() + }; + let refused = template.start("cl-n", start, CLONE_FAILED); + assert_eq!( + failure(&refused.started), + [ + "request", + "invalid-targets", + "plan", + "planned", + "none", + "no-vm" + ], + "{target}: {:?}", + refused.started + ); + assert_eq!(refused.started["quarantined"], "no"); + refused.assert_refused(&format!( + "cannot restore with a processor target of {target}: a target must be 1, 2, 4, or 8, \ + from the template's processors online at boot, 1, to its capacity, 4" + )); + assert!(template.leases().is_empty()); + } + assert_eq!(files_below(&snapshot), captured); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with() { + let template = Template::of_processors("cl-boot", 4, Some(2)); + let options = ["--processors", "1"]; + let start = StartOf { + options: &options, + ..StartOf::default() + }; + let refused = template.start("cl-m", start, CLONE_FAILED); + assert_eq!( + failure(&refused.started), + [ + "request", + "invalid-targets", + "plan", + "planned", + "none", + "no-vm" + ], + "{:?}", + refused.started + ); + refused.assert_refused( + "cannot restore with a processor target of 1: a target must be 1, 2, 4, or 8, from the \ + template's processors online at boot, 2, to its capacity, 4", + ); + assert!(template.leases().is_empty()); + + // Its clones take its 2 processors online at boot by default, and the restore of a larger + // target brings online the CPUs beyond them. + for (options, online, restored) in [ + (&[][..], "0-1", "none"), + (&["--processors", "4"][..], "0-3", "2-3"), + ] { + let start = StartOf { + options, + ..StartOf::default() + }; + let mut cloned = template.start("cl-m", start, 0); + cloned.assert_started(); + let facts = cloned.probe(&["processors"]); + assert_processors(&facts, online, "0-3", restored); + cloned.stop(); + } + assert!(template.leases().is_empty()); +} + #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn one_generation_serves_clones_of_either_format_over_either_upper() { diff --git a/tests/runtime-adapter/tests/vm_template.rs b/tests/runtime-adapter/tests/vm_template.rs index c002de776..c9f786c5a 100644 --- a/tests/runtime-adapter/tests/vm_template.rs +++ b/tests/runtime-adapter/tests/vm_template.rs @@ -7,10 +7,11 @@ //! of it, with a [`Network`], or without one, in a template store of its own. A template that the //! store captured is the store's one generation, whose payload holds the record of its capture, //! its key, and its snapshot, and whose key holds what the scenario asked for: the runtime of the -//! adapter's VMs, with the guest agent's build, the backend, the processors, all online at boot, -//! the RAM and its capacity, the geometries of the placeholder and of the dormant scratch, a NIC -//! exactly when the network lets something through, and a kernel command line of the platform -//! tier's tokens alone, which names neither the placeholder's identity nor a root. The slot of the +//! adapter's VMs, with the guest agent's build, the backend, the processors and those online at +//! boot, the RAM and its capacity, the geometries of the placeholder and of the dormant scratch, a +//! NIC exactly when the network lets something through, and a kernel command line of the platform +//! tier's tokens alone, with `maxcpus=` for a template whose processors are not all online at +//! boot, which names neither the placeholder's identity nor a root. The slot of the //! template's class then holds no transaction, and records no launch, and a second creation serves //! the generation without a capture. NVX refuses a network whose policy no template takes before //! any VM starts, and the host program refuses a class that no capture takes before it creates the @@ -151,15 +152,17 @@ impl Network { } } -/// A template that a scenario creates: its network, and the processors, the RAM, the RAM -/// capacity, and the length of the dormant scratch that the creation names, or else the host -/// program's defaults. +/// A template that a scenario creates: its network, and the processors, those online at boot, the +/// RAM, the RAM capacity, and the length of the dormant scratch that the creation names, or else +/// the host program's defaults. #[derive(Clone, Copy, Debug)] struct Template { /// The network. network: Network, - /// The processors, all online at boot, or 1 by default. + /// The processors, or 1 by default. processors: Option, + /// The processors online at boot, or 1 by default. + boot_processors: Option, /// The RAM, in bytes, or [`DEFAULT_MEMORY`] by default. memory: Option, /// The RAM capacity, in bytes, or the RAM by default. @@ -174,6 +177,7 @@ impl Template { Self { network, processors: None, + boot_processors: None, memory: None, memory_capacity: None, scratch_length: None, @@ -185,6 +189,11 @@ impl Template { self.processors.unwrap_or(1) } + /// Returns the processors of the template that are online at boot. + fn boot_processors(&self) -> u32 { + self.boot_processors.unwrap_or(1) + } + /// Returns the RAM of the template, in bytes. fn memory(&self) -> u64 { self.memory.unwrap_or(DEFAULT_MEMORY) @@ -231,8 +240,9 @@ impl Templates { } /// Returns the options of a creation or a refresh of `template`, which name its placeholder, - /// and the network that it writes to a file of the test's, if any, and the processors, the - /// RAM, its capacity, and the length of the scratch that the template names. + /// and the network that it writes to a file of the test's, if any, and the processors, those + /// online at boot, the RAM, its capacity, and the length of the scratch that the template + /// names. fn options(&self, template: &Template) -> Vec { let mut distro = OsString::from("erofs:"); distro.push(&self.placeholder); @@ -244,6 +254,7 @@ impl Templates { ]; let numbers = [ ("--processors", template.processors.map(u64::from)), + ("--boot-processors", template.boot_processors.map(u64::from)), ("--memory", template.memory.map(|memory| memory / MIB)), ( "--memory-capacity", @@ -378,8 +389,12 @@ fn assert_captured(templates: &Templates, created: &Printed, template: &Template assert_eq!(key.hypervisor(), Hypervisor::from_env_or_default().unwrap()); let processors = template.processors(); assert_eq!(key.processors(), processors); - // Every processor is online at boot, so a restore activates none. - assert_eq!(key.boot_processors(), 0); + // A template whose processors are all online at boot records none, and a restore brings + // none online; one that boots with fewer records them, and a restore may bring the others + // online. + let boot = template.boot_processors(); + let limited = boot < processors; + assert_eq!(key.boot_processors(), if limited { boot } else { 0 }); assert_eq!(key.memory(), template.memory()); assert_eq!(key.memory_capacity(), template.memory_capacity()); let placeholder = fs::metadata(&templates.placeholder).unwrap().len(); @@ -396,8 +411,9 @@ fn assert_captured(templates: &Templates, created: &Printed, template: &Template assert_eq!(key.has_network(), template.network.has_nic()); // The kernel command line, which every clone restores: the platform tier's tokens alone, - // which OpenVMM's capture admits, with the processors, and the NIC's addresses exactly when - // the template has a NIC, but neither the placeholder's identity nor a root's tokens. + // which OpenVMM's capture admits, with the processors, those online at boot when they are + // fewer, and the NIC's addresses exactly when the template has a NIC, but neither the + // placeholder's identity nor a root's tokens. let command_line = key.command_line(); let tokens: Vec> = cmdline::parameters(command_line.as_bytes()) .map(|parameter| token(¶meter)) @@ -419,6 +435,16 @@ fn assert_captured(templates: &Templates, created: &Printed, template: &Template "{expected} in {command_line}" ); } + let limits: Vec<&Vec> = tokens + .iter() + .filter(|token| token.starts_with(b"maxcpus=")) + .collect(); + let expected = format!("maxcpus={boot}"); + match limits.as_slice() { + [] => assert!(!limited, "{command_line}"), + [limit] => assert!(limited && **limit == expected.as_bytes(), "{command_line}"), + _ => panic!("{command_line} limits the processors online at boot more than once"), + } assert_eq!( tokens.iter().any(|token| token.starts_with(b"virtnet_ip=")), template.network.has_nic(), @@ -494,7 +520,8 @@ fn refuses(name: &str, template: Template, problem: &str) { #[test] #[ignore = "boots a VM; see the module documentation for what it needs"] fn a_template_without_a_network_has_no_nic() { - // Every processor is online at boot, and a restore may expand the RAM to its capacity. + // One of its processors is online at boot, the host program's default, of which a restore + // may bring the other online, and a restore may expand the RAM to its capacity. let template = Template { processors: Some(2), memory: Some(2 * MEMORY_BLOCK_SIZE), @@ -505,6 +532,17 @@ fn a_template_without_a_network_has_no_nic() { captures("tpl-none", template); } +#[test] +#[ignore = "boots a VM; see the module documentation for what it needs"] +fn a_template_whose_processors_are_all_online_at_boot_limits_none() { + let template = Template { + processors: Some(2), + boot_processors: Some(2), + ..Template::new(Network::None) + }; + captures("tpl-online", template); +} + #[test] #[ignore = "boots a VM; see the module documentation for what it needs"] fn a_network_that_lets_nothing_through_has_no_nic() { @@ -558,15 +596,19 @@ fn host_loopback_both_ways_is_refused_before_any_vm_starts() { #[test] #[ignore = "needs NVX's artifacts; see the module documentation"] fn a_class_that_no_capture_takes_is_refused_before_the_store() { - let mut templates = Templates::new("tpl-class"); - let template = Template { - processors: Some(3), - ..Template::new(Network::None) - }; - templates.create(&template, FAILED); - let stderr = templates.sandbox.stderr(); - assert!(stderr.contains("needs 1, 2, 4, or 8 of each"), "{stderr}"); - assert!(!templates.sandbox.store.exists()); + // Processors that a microVM cannot have, and more online at boot than in all. + for (processors, boot_processors) in [(3, None), (2, Some(4))] { + let mut templates = Templates::new("tpl-class"); + let template = Template { + processors: Some(processors), + boot_processors, + ..Template::new(Network::None) + }; + templates.create(&template, FAILED); + let stderr = templates.sandbox.stderr(); + assert!(stderr.contains("needs 1, 2, 4, or 8 of each"), "{stderr}"); + assert!(!templates.sandbox.store.exists()); + } } /// Returns the store generation `number`.