Skip to content

nvx-rs: boot templates with an adapter's kernel tuning parameters, such as mitigations=off #995

Open

Description

Follow-up to the snapshot/restore workstream (#833).

Goal. An adapter can boot its templates with the kernel tuning parameters that its cold boots use, so that its clones run the same kernel configuration as its cold sandboxes. The motivating case is mitigations=off: Linux applies it once, at boot, so a clone has no other way to get it.

Today (dev c3f15cc8; OpenVMM 832d5bb4, NVX's pin):

  • A template's kernel command line is NVX's own: TemplatePlan::kernel_command_line returns nvx_sandbox=1 (nvx-rs/src/host/template/plan.rs:29, :322; nvx-rs: serve clones of each processor count from one template #926 adds maxcpus=), which the capture passes as --cmdline, alone (nvx-rs/src/host/template/store/capture.rs:315-316). CaptureClass has no field for other parameters (nvx-rs/src/host/template/key.rs:387-419).
  • OpenVMM checks a platform snapshot's manifest when it publishes the capture, and again at every restore (validate_manifest_contents, openvmm/openvmm_helpers/src/snapshot/format.rs:141, called from publish.rs:219 and restore.rs:200). Every token of the effective command line must pass platform_command_line_token_is_invariant (openvmm/openvmm_helpers/src/snapshot/microvm.rs:2401-2437): OpenVMM's own console, reboot and panic tokens, nvx_sandbox=1, nvx_config=, nvx_snapshot_tier=platform, nr_cpus=, maxcpus=, the control tty, virtio_mmio.device=, and virtnet_*. Any other token is refused as "tenant or unsupported configuration".
  • So an adapter whose cold boots pass, for example, no_timer_check mitigations=off quiet loglevel=0 gets clones that run with the kernel's default CPU vulnerability mitigations, and that log at the default console level. Its clones and its cold sandboxes then differ in the performance of workloads heavy in system calls, and a comparison of their start latencies measures the mitigations along with the restore.

Proposal:

  1. OpenVMM (feature/microvm/dev): admit a closed set of guest kernel tuning parameters on a platform snapshot's command line. Like the tokens admitted today, each is fixed when the template's kernel boots, is the same for every clone, and carries no tenant data:

    • mitigations=off, mitigations=auto, and mitigations=auto,nosmt;
    • no_timer_check;
    • quiet, and loglevel=0 to loglevel=7.

    Each may appear once. Everything else stays refused, including the clock parameters that the time ABI refuses (CLOCK_TOKENS, vmm_core/virt/src/time_abi/mod.rs:359).

  2. nvx-rs: a CaptureClass field that holds the template's tuning parameters, which CaptureClass::check checks against the same set, so that a refusal comes before any VM boots. TemplatePlan appends them to its kernel command line. The class's canonical encoding and identity hold them, under a new encoding VERSION. The key already holds the effective command line, so templates that differ only in their parameters are different templates.

  3. A PR here moves NVX's pin to the OpenVMM change, as microvm: at a platform restore, read zeros past a shorter image and from an absent fresh scratch #856's did.

quiet and loglevel= matter less than mitigations=, since a guest can set its console log level at run time, before its capture. They are on the list so that an adapter's templates can boot as its cold sandboxes do.

Acceptance:

  • OpenVMM unit tests: each admitted parameter is accepted at publish and at restore; a duplicate, mitigations= with another value, loglevel=8, and an unknown token are refused.
  • nvx-rs unit tests: the plan renders the parameters after its own tokens; CaptureClass::check refuses a parameter outside the set; two classes that differ only in their parameters have different identities.
  • The test adapter captures a template with mitigations=off, and restores a clone whose /proc/cmdline holds it, on KVM, MSHV, and WHP.

Not in scope: the command lines of cold launches, and parameters that carry per-sandbox data, which a clone's activation record carries instead.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions