[Enrique Saurez (esaurez)]

Part of #849. Stacked on #925.

Chain: #951 → #954 → #957 → #959 → #960 → #955 → #958 → #962 → #963 → #925 → this PR, on dev; #879, #881, #885, #886, #877, #932, #884, #887, #933, #880, #936, #937, #939, #940, #941, #888, #949, and #950 have merged.

What and why

Until now, a template booted with all of its processors online (B = C), and its clones got no processor target, so each processor count needed a template of its own. OpenVMM can bring a contiguous prefix of a snapshot's processors online at restore: a capture with maxcpus=B records B as the boot-online count, and --restore-processors N restores with N online, where B <= N <= C. With this PR, one template serves each supported count from its B to its capacity C.

Host: nvx::host::template

• TemplateResources::boot_processors is B: 1, 2, 4, or 8, and at most the processors C. TemplatePlan::new refuses any other value before a VM starts.
• A template with B < C boots with the kernel command line nvx_sandbox=1 maxcpus=B. That is the canonical token that OpenVMM's platform tier admits and records as the snapshot's boot-online count, and the key keeps it (TemplateKey::boot_processors).
• A template with B = C renders no maxcpus= and records none, as before. The store's interim refusal of B < C is gone.
• CloneRequest::processors is the clone's processor target N. ClonePlan::new checks it against the key, together with the memory target, before any VMM starts. N must be 1, 2, 4, or 8, from B (C for a template that records none) up to C. Otherwise the plan fails at the request, with the evidence invalid-targets.
• The restore passes --restore-processors N only when the key records B. A template that records none restores at C, as before.
• The clone's activation record expects N CPUs, which the guest checks against its repair before it activates (nvx-rs: the restricted control state and Activate after a restore, in the guest library and the test adapter's framing #843).
• The fake OpenVMM's capture admits maxcpus=B as OpenVMM does.

Guest: nvx::guest::clone::repair

The repair's processor stage now activates the processor target, while host input is still gated and before the acknowledgement:

• It reads the online CPUs. They must form a prefix, and a target below them is refused, since the repair takes no CPU offline.
• It brings each offline CPU of the target's prefix online through /sys/devices/system/cpu/cpuK/online, as nvx-snapshot does.
• It checks that the online CPUs are exactly 0..N-1. Otherwise it fails with the new cause RepairCause::Activation and status 41, REPAIR_PROCESSORS.

These CPUs have been in the VM since its boot, so no device is plugged. A retry brings online only the CPUs that are still offline. The finish reports CPUs B to N-1 through RestoreFinish::new_cpus, so the time daemon's deferred restore checks cover them too.

Authentication timer

No change. The host keeps connecting after the restore-ready event.

Runtime adapter

• create and refresh take --boot-processors B, 1 by default. That is the fewest processors the template's clones take, so by default one template serves every count up to its --processors capacity. B belongs to the class and the key.
• start and run take --processors N. The default is the template's B, or C when the template records none. Without N or a memory target, the command reads the generation's key once, before the lease, as it does for the memory target; a key that cannot be read leaves both to NVX's plan, which attributes the failure.
• The guest agent has a new processors probe. It reports the online, present, and possible CPUs, and the CPUs that the restore brought online, read from the time ABI's record /run/nvx/time-restore.

Tests

nvx-rs unit tests:

• plan: every template of every B up to C. templates_limit_their_processors_online_at_boot_on_their_command_line renders maxcpus=B only when B < C. The refusals now cover B of 0, 3, and 16, and B above C.
• store::capture: captures_limit_the_processors_that_their_class_keeps_offline_at_boot captures 4 processors with 1 online at boot, which gives --cmdline "nvx_sandbox=1 maxcpus=1" and a key of (4, 1). It replaces the interim rule.
• clone: processor_targets_are_counts_from_the_templates_processors_online_at_boot_to_its_capacity. A template of 8 with 2 online at boot serves 2, 4, and 8, and refuses 0, 1, 3, 6, 16, and u32::MAX with InvalidTargets. A template that records none refuses 1 and 4, and serves only its capacity, 2.
• clone::launch: a_clone_of_a_template_with_processors_offline_at_boot_restores_its_processor_target restores a clone in RAM with --restore-processors, beside the --restore-absent-scratch of its devices. The exact arguments of the existing restore test still hold no such option for a template that records none.
• fake_openvmm: the capture admits a canonical maxcpus=B of at most --processors, and refuses maxcpus=01, maxcpus=3, a B above the processors, and a second token.
• guest::clone::repair, with four new tests:
  • a target below the online CPUs is refused;
  • the offline CPUs of the prefix come online and are reported as new CPUs;
  • a retry onlines only the CPUs still offline and reports every new one;
  • a CPU that stays offline fails the activation.

Runtime adapter, host-only:

• clone_start passes the processor target into NVX's request, and refuses --processors 0 and --boot-processors on start.
• template_capture checks the class's (C, B): (1, 1) by default, then (4, 1) and (4, 2), each with a class of its own.
• template_operations refuses --boot-processors one.

Runtime adapter VM scenarios (scenarios.txt, now 158):

• vm_template a_template_whose_processors_are_all_online_at_boot_limits_none: 2 of 2 online at boot, no maxcpus=, and a key that records none.
• vm_template a_template_without_a_network_has_no_nic now captures 2 processors with 1 online at boot. The capture assertions check maxcpus=B exactly when B < C, and the key's B, in every capture scenario.
• vm_template a_class_that_no_capture_takes_is_refused_before_the_store also refuses 2 processors with 4 online at boot.
• vm_clone one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity: one generation of 4 processors, 1 online at boot.
  • It is restored by default and at 2 and 4, with online CPUs 0, 0-1, and 0-3, new CPUs none, 1, and 1-3, and possible CPUs 0-3.
  • Targets of 3 and 8 are refused with status 8, at the plan, with invalid-targets and no VM.
• vm_clone a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with: a template of 4 with 2 online at boot.
  • It refuses a target of 1.
  • By default it restores with 0-1 online and no new CPUs; at 4 it restores with 0-3 online and new CPUs 2-3.

Guide changes (#591's numbering)

These changes are not made in #591; they are listed here for it:

• 2.1.1 Allowlisted Platform: the platform tier's kernel command line admits one canonical maxcpus=B token, with B in {1, 2, 4, 8} and at most the processor capacity C. A template's capture adds it only when B < C, and OpenVMM records it as the snapshot's boot-online count. A template whose processors are all online at boot carries no maxcpus= and records no boot-online count.
• 5.5.3 Processor Activation: each clone of one template chooses its own N. The host refuses an N outside {1, 2, 4, 8} or outside [B, C] before any VMM starts. A snapshot that records no B serves only N = C. The host passes --restore-processors N only for a snapshot that records B, and the clone's activation record expects N online processors.
• 5.5.7 Guest Repair, step 4: before the acknowledgement, the repair reads the online CPU list, which must be a prefix no longer than N. It onlines each offline CPU of 0..N-1 through /sys/devices/system/cpu/cpuK/online, then rereads the list and requires exactly 0..N-1. The finish reports CPUs B..N-1 as new CPUs, and the conformance checks that rerun after the acknowledgement (step 7) cover them. The guest library reports a failure as status 41.
• The guide at 950c1fe contains no interim rule (B = C, no target), so there is nothing to remove from it.

Validation

At this PR's head 02b92e2, the same two commits as the earlier head 2dd7ce6, restacked onto #925's head ce7932d: its tree, 4d6e9339, is the merge of 2dd7ce6 into dev at fbeb563, which holds #951, #954 and #957, plus #955's fix that treats a path whose links loop as a changed object. That merge, without the fix, passed the documentation gates of nvx-rs and of the test adapter, Clippy, validate-nvx on Linux and on Windows, and the Markdown link check; the fix has its own tests in #955. The rest of this section describes 2dd7ce6, on #925's earlier head 1c77b97.

At 2dd7ce6 (tree 16977fde), on #925 1c77b97:

• Gates:
  • Windows:
    • check-nvx-rs's Windows steps:
      • fmt;
      • Clippy with all features, --lib, and --lib --no-default-features;
      • the tests with all features and with the defaults, and the library without default features;
      • the bundled-artifacts test with a synthetic release;
      • rustdoc with -D warnings;
      • MSRV 1.89;
      • the runtime adapter's protocol-only and host-only configurations.
    • check-quality's ruff, pyright, ruff format, Markdown link, and PowerShell syntax checks.
    • validate-nvx's Windows steps: the 1425 script tests, the host tests, and the CLI help. The adversarial tests ran on Linux, and in CI's NVX CLI / Windows job.
    • test-runtime-adapter --check-only, which names 158 scenarios.
  • Linux: the same, plus:
    • the macOS check;
    • the guest binaries' check, build, and tests;
    • the runtime adapter's guest-only configuration for musl;
    • check-quality's shellcheck, shfmt, and Specula checks;
    • the 84 adversarial tests;
    • the root-only guest::clone, guest::root, and guest::mappings tests (13), as root.
• VM suites: the runtime adapter's 158 scenarios of 11 targets passed on WHP and on MSHV.
  • They used OpenVMM 832d5bb4 from release v0.1.0-dev.2d14547b669f, and the initramfs and guest agent built from this head.
  • Per target: vm_clone 33, vm_layered 43, vm_template 13, vm_restart 3, vm_steps 3, and the start and run scenarios.
• CI: run 40852461399, dispatched on this head, passed on attempt 2: 36 jobs succeeded and 6 were skipped.
  • Attempt 1 failed one scenario on KVM: vm_clone a_clone_refuses_a_scratch_image_whose_upper_an_earlier_clone_used.
  • That's a known flake, which passed on rerun. On MSHV it also passed 20 times alone, and in 5 runs of all of vm_clone.

[Copilot]

🔵 Needs a closer look

CPU activation and deferred time checks still need the pending Linux and cross-backend VM validation on the final integration base, followed by human review.

1 open issue

• Avoid leaving CPUs online when memory target activation is refused · New

What changed in this PR

Implements per-clone CPU targets for #849, allowing one NVX template to serve supported processor counts from its boot-online count to its capacity.

Changes:

• Adds boot-online CPU configuration, clone target validation, and restore arguments.
• Extends guest repair to activate CPUs and report them for deferred time checks.
• Updates runtime-adapter options, probes, tests, and documentation.

File	Description
tests/​runtime-adapter/​tests/​vm_template.rs	Tests boot-online counts and capture limits.
tests/​runtime-adapter/​tests/​vm_clone.rs	Tests multiple CPU targets from one generation.
tests/​runtime-adapter/​tests/​template_operations.rs	Tests invalid boot-processor input.
tests/​runtime-adapter/​tests/​template_capture.rs	Checks CPU counts in capture classes.
tests/​runtime-adapter/​tests/​clone_start.rs	Tests processor options and request propagation.
tests/​runtime-adapter/​src/​worker/​probes.rs	Adds CPU-state and restored-CPU probes.
tests/​runtime-adapter/​src/​template.rs	Updates allowlist documentation and tests.
tests/​runtime-adapter/​src/​clone.rs	Carries clone processor targets.
tests/​runtime-adapter/​src/​capture.rs	Adds boot-processor configuration and defaults.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-host.rs	Parses CPU options and selects clone defaults.
tests/​runtime-adapter/​scenarios.txt	Registers three new VM scenarios.
tests/​runtime-adapter/​README.md	Documents CPU options, repair, and scenarios.
nvx-rs/​tests/​vectors/​README.md	Updates the documented OpenVMM source revision.
nvx-rs/​tests/​vectors/​openvmm/​platform_command_line_token_is_invariant.rs	Adds maxcpus= to the copied allowlist.
nvx-rs/​src/​testing/​fake_openvmm.rs	Validates canonical capture CPU limits.
nvx-rs/​src/​protocol/​status.rs	Clarifies processor-repair status descriptions.
nvx-rs/​src/​host/​template/​store/​capture/​tests.rs	Tests capture with CPUs offline at boot.
nvx-rs/​src/​host/​template/​store/​capture.rs	Allows and propagates boot-online CPU counts.
nvx-rs/​src/​host/​template/​plan/​tests.rs	Tests CPU combinations, rendering, and refusals.
nvx-rs/​src/​host/​template/​plan.rs	Validates boot counts and renders maxcpus=.
nvx-rs/​src/​host/​template/​clone/​tests/​launch.rs	Tests processor-target restore arguments.
nvx-rs/​src/​host/​template/​clone/​tests.rs	Tests target bounds and activation resources.
nvx-rs/​src/​host/​template/​clone/​record.rs	Records the requested CPU count.
nvx-rs/​src/​host/​template/​clone/​launch.rs	Conditionally passes --restore-processors.
nvx-rs/​src/​host/​template/​clone.rs	Validates and retains clone CPU targets.
nvx-rs/​src/​guest/​clone/​repair/​tests.rs	Tests CPU activation, retries, and failures.
nvx-rs/​src/​guest/​clone/​repair.rs	Activates CPUs and reports newly online CPUs.
nvx-rs/​src/​guest/​clone.rs	Documents processor repair and time checks.
nvx-rs/​README.md	Updates host and guest API descriptions.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

CPU restore spans host, guest, and OpenVMM contracts, with Linux gates, cross-backend VM validation, and final review after the integration rebase still pending.

No open issues 🎉

1 resolved since last review

• Avoid leaving CPUs online when memory target activation is refused

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues were identified for the per-clone CPU target changes.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

CPU activation changes the pre-acknowledgement restore path and deferred time checks, while validation of the new VM scenarios remains pending on KVM, MSHV, and WHP.

No open issues 🎉

_{Review effort: Balanced}