Skip to content

Snapshot/restore workstream, stage 2: the OpenVMM changes #835

Closed
3 of 3 issues completed
Closed

Description

Parent: #833. Needs: nothing else in this workstream; each change proceeds in OpenVMM without waiting for stage 1.

Goal. The four OpenVMM changes that platform clones need, each in its own issue. Each reaches NVX through a PR that moves NVX's OpenVMM pin and adds VMM-level snapshot scenarios, as #801 does for #796. Everything else that clones need already works in today's OpenVMM (below).

Change Issue NVX pin PR Enables
Zero fill: past a shorter image, and for an absent scratch #856 Comes with the change #848
CPU targets for platform snapshots, and the authentication timer during a gated restore #796 #801, stacked on #797 #849
A new network policy at restore, for deny-all templates #536 Comes with the change #850
An aggregate export on the dormant filesystem slot at restore #540 Comes with the change #857

Zero fill, expected behavior (#856):

  • distro:
    • A platform restore presents the template's distro capacity, the placeholder's size, before any vCPU runs.
    • Reads inside the image go to the image. Reads past its end return zeros, including a request that crosses the end. Nothing resizes, pads or writes the image file.
    • An image longer than the capacity, or with different logical or physical block sizes, is refused.
  • Scratch:
    • A platform restore may declare its fresh scratch absent, explicitly. OpenVMM then attaches the scratch device with no file, at the captured length and block sizes, taken from the manifest.
    • Reads return zeros, and writes fail with an I/O error. The guest negotiated the device as writable at the template's boot, and restore compares the access, so the device stays writable: its writes are refused, never dropped or kept in host memory. A tmpfs clone never writes to it.
    • Only a fresh scratch, the platform tier's, may be absent. A scratch file, when given, keeps the exact captured length.

Today a disk's size is its file length (openvmm/openvmm_entry/src/storage_builder/microvm.rs:247), and every snapshot-capable disk needs a nonempty raw file (:190-197, :248-251). Restore compares every disk field exactly, including the access, the length and the block sizes (openvmm/openvmm_helpers/src/snapshot/microvm.rs:1633-1647), and a fresh-scratch restore requires a scratch file (openvmm/openvmm_entry/src/microvm/restore.rs:369-374). On Linux the physical block size is the backing filesystem's (storage_builder/microvm.rs:256), so images, the placeholder and scratch files must live on filesystems with the same block size. A GPT image on the larger presented disk passes nvx::gpt, which only requires the backup header to lie inside the disk (nvx-rs/src/gpt/mod.rs:591-593).

CPU targets (#796, #801): OpenVMM already brings a CPU prefix online at restore (maxcpus=B at capture, then --restore-processors N), but the platform allowlist refuses maxcpus=. #801 admits one canonical maxcpus= token, and suspends the authentication deadline while host input is gated.

Network policy (#536): restore requires the captured policy today (openvmm/openvmm_helpers/src/snapshot/microvm.rs:579-606), and every snapshot refuses ingress, explicit host loopback and forwards (openvmm/openvmm_entry/src/cli_args/microvm.rs:731-739, :1225-1247). #536 lets a clone of a template captured with a complete deny-all policy get its own policy, installed before any vCPU runs. A template NIC is needed only for clones with network; clones without network come from templates without a NIC.

Aggregate export (#540): at restore, a dormant filesystem slot accepts only one --mount directory (openvmm/openvmm_entry/src/microvm/filesystem.rs:983-995), although OpenVMM builds aggregates for cold launches, and restores one that a snapshot captured (openvmm/openvmm_entry/src/microvm/config.rs:570-628, filesystem.rs:997-1027). #540 lets a platform restore attach an aggregate of directories and files to the dormant slot, with its hidden, re-exposed and writable paths and its owner mode, validated as at a cold launch. The guest learns of the export from the activation record, so OpenVMM exposes no layout, and the saved command line doesn't change.

Used as it is (pinned OpenVMM 79a501b6):

Need Today's OpenVMM
The clone's image in place of the placeholder A platform snapshot treats every read-only disk as unbound (snapshot/microvm.rs:1636-1645, :2265-2274)
A scratch file per scratch clone A platform snapshot's scratch is fresh: restore requires one, accepts any file of the captured length, and never reads or hashes it
One directory export per clone, until #540 A dormant filesystem slot accepts one --mount directory at restore (openvmm/openvmm_entry/src/microvm/filesystem.rs:983-995)
Memory targets --memory-capacity at capture and --restore-memory at restore, for every tier
No hashing --snapshot-block-identity generation --snapshot-generation-id at capture; platform restores never hash
The clone's control session Restore queues a RESET for the session that the guest attached before capture (vm/devices/virtio/virtio_console/src/control_session_broker.rs:744-771)
When to connect --restore-ready-path is written after the guest acknowledges repair and host input is released

Acceptance. Each change is merged in OpenVMM with its VMM tests, and NVX's pin includes it.

Activity

  1. changed the title [-]Platform clones, stage 2: the OpenVMM changes[/-] [+]Snapshot/restore workstream, stage 2: the OpenVMM changes[/+] on Oct 24, 2026
  2. ppenna commented on Oct 29, 2026

    @ppenna
    Contributor

    All four changes are merged in OpenVMM with their tests, and NVX's pin includes them, so this meets its acceptance.

    Change OpenVMM NVX pin PR VMM scenario
    Zero fill (#856) microsoft/openvmm#4749 (a12d9167) #901 platform-zero-fill
    CPU targets (#796) microsoft/openvmm#4735 (2b681564) #801 platform-restore-processors
    Network policy (#536) microsoft/openvmm#4767 (57d0a18b) #931 platform-network-policy
    Aggregate export (#540) microsoft/openvmm#4775 (832d5bb4) #948 platform-aggregate-export
    • Each OpenVMM PR adds unit tests for its behavior, and none of the 21 later commits on feature/microvm/dev reverts one.
    • dev pins OpenVMM c32c57d1, the tip of feature/microvm/dev, which descends from all four heads. All four pin PRs are in dev.
    • The four scenarios are sharded for kvm, mshv and whp. They passed on all three in openvmm: verify pinned CPU profiles at partition creation #1016's run at c32c57d1, and in the last green dev run, at bd2b27a2.

    The stage 3 consumers (#848, #849, #850, #857) are tracked in #836. #942 extends #536 to snapshots without a tier and belongs to #800.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions