Repository navigation
nvx-rs: boot templates with an adapter's kernel tuning parameters, such as mitigations=off #995
Description
Follow-up to the snapshot/restore workstream (#833).
Goal. An adapter can boot its templates with the kernel tuning parameters that its cold boots use, so that its clones run the same kernel configuration as its cold sandboxes. The motivating case is mitigations=off: Linux applies it once, at boot, so a clone has no other way to get it.
Today (dev c3f15cc8; OpenVMM 832d5bb4, NVX's pin):
- A template's kernel command line is NVX's own:
TemplatePlan::kernel_command_linereturnsnvx_sandbox=1(nvx-rs/src/host/template/plan.rs:29,:322; nvx-rs: serve clones of each processor count from one template #926 addsmaxcpus=), which the capture passes as--cmdline, alone (nvx-rs/src/host/template/store/capture.rs:315-316).CaptureClasshas no field for other parameters (nvx-rs/src/host/template/key.rs:387-419). - OpenVMM checks a platform snapshot's manifest when it publishes the capture, and again at every restore (
validate_manifest_contents,openvmm/openvmm_helpers/src/snapshot/format.rs:141, called frompublish.rs:219andrestore.rs:200). Every token of the effective command line must passplatform_command_line_token_is_invariant(openvmm/openvmm_helpers/src/snapshot/microvm.rs:2401-2437): OpenVMM's own console, reboot and panic tokens,nvx_sandbox=1,nvx_config=,nvx_snapshot_tier=platform,nr_cpus=,maxcpus=, the control tty,virtio_mmio.device=, andvirtnet_*. Any other token is refused as "tenant or unsupported configuration". - So an adapter whose cold boots pass, for example,
no_timer_check mitigations=off quiet loglevel=0gets clones that run with the kernel's default CPU vulnerability mitigations, and that log at the default console level. Its clones and its cold sandboxes then differ in the performance of workloads heavy in system calls, and a comparison of their start latencies measures the mitigations along with the restore.
Proposal:
-
OpenVMM (
feature/microvm/dev): admit a closed set of guest kernel tuning parameters on a platform snapshot's command line. Like the tokens admitted today, each is fixed when the template's kernel boots, is the same for every clone, and carries no tenant data:mitigations=off,mitigations=auto, andmitigations=auto,nosmt;no_timer_check;quiet, andloglevel=0tologlevel=7.
Each may appear once. Everything else stays refused, including the clock parameters that the time ABI refuses (
CLOCK_TOKENS,vmm_core/virt/src/time_abi/mod.rs:359). -
nvx-rs: a
CaptureClassfield that holds the template's tuning parameters, whichCaptureClass::checkchecks against the same set, so that a refusal comes before any VM boots.TemplatePlanappends them to its kernel command line. The class's canonical encoding and identity hold them, under a new encodingVERSION. The key already holds the effective command line, so templates that differ only in their parameters are different templates. -
A PR here moves NVX's pin to the OpenVMM change, as microvm: at a platform restore, read zeros past a shorter image and from an absent fresh scratch #856's did.
quiet and loglevel= matter less than mitigations=, since a guest can set its console log level at run time, before its capture. They are on the list so that an adapter's templates can boot as its cold sandboxes do.
Acceptance:
- OpenVMM unit tests: each admitted parameter is accepted at publish and at restore; a duplicate,
mitigations=with another value,loglevel=8, and an unknown token are refused. - nvx-rs unit tests: the plan renders the parameters after its own tokens;
CaptureClass::checkrefuses a parameter outside the set; two classes that differ only in their parameters have different identities. - The test adapter captures a template with
mitigations=off, and restores a clone whose/proc/cmdlineholds it, on KVM, MSHV, and WHP.
Not in scope: the command lines of cold launches, and parameters that carry per-sandbox data, which a clone's activation record carries instead.
Activity