Skip to content

[DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot - #4775

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
feature/microvm/devfrom
copilot/microvm-platform-aggregate-export
Oct 28, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
feature/microvm/devfrom
copilot/microvm-platform-aggregate-export

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Lets the restore of a microVM platform snapshot whose filesystem slot is dormant attach an aggregate of host directories and files, with its access policy and owner, validated as at a cold boot, before any vCPU runs, for microsoft/nvx#540. This is a new feature of the platform tier rather than a fixup.

Based on feature/microvm/dev. This branch is two commits directly on b92daa350, the head of feature/microvm/dev and the revision that NVX's dev pins.

The device takes a dormant slot's state into an aggregate (057a2eaa6)

A capture that leaves the slot dormant saves a dormant device state, without an attachment, a policy, objects, or an initialized FUSE session, and the device restores it into any attached device, since it restores only the session. microvm_dormant_state_restores_into_aggregate_attachment checks it for the device of an aggregate of a read-write directory and a read-only file, and that the device then saves the aggregate, with its children's kinds and modes, rather than the dormant slot. Only a test changes.

A platform restore attaches an aggregate to a dormant slot (832d5bb48)

  • Options. At the restore of a platform snapshot whose slot is dormant, --mount-aggregate with its --mount-child options, and their --mount-deny, --mount-allow, --mount-write, and --mount-owner policy, attach an aggregate to the slot.
  • Validation. effective_microvm_filesystem builds it with a cold boot's code, microvm_filesystem_from_request, so the restore checks the children's kinds, modes, names, overlaps, and limits, the policy paths, and the owner as a cold boot does, and refuses what a cold boot refuses, before any vCPU runs.
  • Nothing added. The slot's device keeps the stable ID, tag, MMIO range, interrupt, features, and queues that the snapshot recorded, and the guest keeps its captured command line, which names no filesystem: OpenVMM exposes no layout, and the guest learns of the export from its host. The device takes the dormant slot's state as it does for a --mount, and the restore contract still records the dormant slot.
  • Tiers. prepare_restore admits the aggregate only for a platform snapshot, which is captured before any tenant input; a later tier carries a workload that ran without a filesystem. The dormant slot of a workload-start, instance-checkpoint, or untiered snapshot still binds only a --mount, a snapshot that predates the dormant slot binds nothing, and a snapshot that captured a filesystem still requires exactly that filesystem.
  • Unchanged. --mount, and a restore without an attachment, behave as before. The management API, which describes only a single directory, does not change.

Tests

  • virtiofs::microvm::device: microvm_dormant_state_restores_into_aggregate_attachment.
  • microvm::filesystem:
    • dormant_slot_restore_attaches_an_aggregate_as_a_cold_boot_does: directory and file children, read-only and read-write, with denied, allowed, and writable paths, resolve to a cold boot's configuration, roots, and identities.
    • dormant_slot_restore_refuses_the_aggregates_that_a_cold_boot_refuses: overlapping directories, a file inside a directory child, a directory or file of the wrong kind, a repeated name, a file child's policy path, a path outside every child, and a relative path, each with a cold boot's error.
    • dormant_slot_restore_applies_the_owner_of_a_cold_boot: --mount-owner caller resolves, or is refused, as at a cold boot.
    • restore_attaches_an_aggregate_only_to_a_dormant_slot, which replaces restore_attaches_an_aggregate_only_where_the_snapshot_did: a snapshot without the dormant slot, and one of a single directory, refuse an aggregate.
    • The tests of captured attachments, such as aggregate_restore_requires_the_same_children and aggregate_restore_requires_the_same_files, and of --mount on a dormant or legacy slot are unchanged.
  • microvm::restore: only_a_platform_restore_attaches_an_aggregate_to_a_dormant_slot, for every tier.

Validation

  • At this head, on Windows: cargo clippy --all-targets and cargo doc --no-deps for virtiofs and openvmm_entry; cargo nextest run --profile agent for both (339 passed); and cargo xtask fmt --fix (clean).
  • Bare metal, with openvmm: let a platform restore attach an aggregate to a dormant filesystem slot nvx#948, which pins this branch and adds the platform-aggregate-export microVM scenario, on Linux/KVM, Linux/MSHV, and Windows/WHP hosts: the scenario, which restores one platform snapshot with an aggregate of directories and files, read-only and read-write, with denied, allowed, and writable paths, passes in 3.4, 4.1, and 9.1 s, and NVX's default microVM suite passes 41/41 on each. The clone's guest finds the aggregate's children with their kinds, modes, and policies, and the restores whose aggregates a cold boot refuses fail before the guest runs with a cold boot's errors, as caller ownership does on Windows, and an untiered snapshot's dormant slot refuses an aggregate. With the unpatched d2338ea22 on MSHV, the capture succeeds and the clone's restore fails at a dormant-slot snapshot can attach only a --mount on restore. Those runs were at d7892ffa8, these two commits on d2338ea22. [DO NOT REVIEW] disk_blob: bound HTTP blob requests and retry transient failures #4773 has since moved feature/microvm/dev to b92daa350 with three commits that change only disk_blob, and git range-diff shows that rebasing onto it left both patches unchanged. With this head, the scenario passes again on the same hosts, in 3.4, 4.3, and 8.8 s, as filesystem-snapshot does, and in openvmm: let a platform restore attach an aggregate to a dormant filesystem slot nvx#948's CI on all three backends, and with the unpatched b92daa350 on KVM, the clone's restore fails at the same refusal.
  • Not run locally: the VMM tests. The management RPC refuses aggregates, so no VMM test reaches this path; NVX's scenario covers it through the CLI.

@github-actions github-actions Bot added the unsafe Related to unsafe code label Oct 28, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Unsafe Code Detected

This PR modifies files containing unsafe Rust code. Extra scrutiny is required during review.

For more on why we check whole files, instead of just diffs, check out the Rustonomicon

@github-actions github-actions Bot added the Guide label Oct 28, 2026
@ppenna
Pedro Henrique Penna (ppenna) removed the request for review from a team October 28, 2026 00:33
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the copilot/microvm-platform-aggregate-export branch from ab39754 to d7892ff Compare October 28, 2026 00:38
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Pin microsoft/openvmm#4775 at d7892ffa857d4b01aef0ee309969dda3e95fee44,
two commits on d2338ea22, the previous pin, which is the head of
feature/microvm/dev.

A platform snapshot captured without a filesystem keeps the fixed
virtio-fs slot dormant, and its restore could bind only one directory
to it with --mount, although a cold boot attaches several host
directories and files as the children of one aggregate. A template
could therefore give its clones only part of what cold boots export.

The first commit tests that the device takes a dormant slot's state,
whose FUSE session the guest never initialized, into the device of an
aggregate, which then saves the aggregate. The second lets the restore
of a platform snapshot whose slot is dormant attach an aggregate with
--mount-aggregate and its --mount-child options, and their
--mount-deny, --mount-allow, --mount-write, and --mount-owner policy.
The restore builds it with a cold boot's code, so it validates the
children, the policy, and the owner as a cold boot does, and refuses
what a cold boot refuses, before any vCPU runs. The slot's device and
the captured command line, which names no filesystem, do not change,
so OpenVMM exposes no layout and the guest learns of the export from
its host. The dormant slot of a workload-start, instance-checkpoint,
or untiered snapshot still binds only a --mount, a snapshot that
predates the dormant slot binds nothing, and a snapshot that captured
a filesystem still requires exactly that filesystem.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Record the two commits that the previous commit pins as positions 350
and 351 of the microVM chain, and move the NVX pin to d7892ffa8.

Position 350 tests that the virtio-fs device takes the state of a
dormant filesystem slot into the device of an aggregate. Position 351
lets a platform restore bind an aggregate to the slot that the
snapshot left dormant, with its children, policy paths, and owner,
validated as at a cold boot. They add behavior rather than fix a
mapped commit, and position 351 changes the dormant-slot restore of
PR18's position 113, which position 293 limited to one --mount when it
added aggregates, so they stay unassigned, like positions 346-349.
Note in the table's remarks that assigning them means revisiting that
work. The pin is now 194 commits past the mapped tip.

Positions 348 and 349 are no longer only a proposal:
microsoft/openvmm#4769 merged them, so position 349, d2338ea22, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4775 as the proposal of positions 350 and 351.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Pin microsoft/openvmm#4775 at d7892ffa857d4b01aef0ee309969dda3e95fee44,
two commits on d2338ea22, the previous pin, which is the head of
feature/microvm/dev.

A platform snapshot captured without a filesystem keeps the fixed
virtio-fs slot dormant, and its restore could bind only one directory
to it with --mount, although a cold boot attaches several host
directories and files as the children of one aggregate. A template
could therefore give its clones only part of what cold boots export.

The first commit tests that the device takes a dormant slot's state,
whose FUSE session the guest never initialized, into the device of an
aggregate, which then saves the aggregate. The second lets the restore
of a platform snapshot whose slot is dormant attach an aggregate with
--mount-aggregate and its --mount-child options, and their
--mount-deny, --mount-allow, --mount-write, and --mount-owner policy.
The restore builds it with a cold boot's code, so it validates the
children, the policy, and the owner as a cold boot does, and refuses
what a cold boot refuses, before any vCPU runs. The slot's device and
the captured command line, which names no filesystem, do not change,
so OpenVMM exposes no layout and the guest learns of the export from
its host. The dormant slot of a workload-start, instance-checkpoint,
or untiered snapshot still binds only a --mount, a snapshot that
predates the dormant slot binds nothing, and a snapshot that captured
a filesystem still requires exactly that filesystem.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Record the two commits that the previous commit pins as positions 350
and 351 of the microVM chain, and move the NVX pin to d7892ffa8.

Position 350 tests that the virtio-fs device takes the state of a
dormant filesystem slot into the device of an aggregate. Position 351
lets a platform restore bind an aggregate to the slot that the
snapshot left dormant, with its children, policy paths, and owner,
validated as at a cold boot. They add behavior rather than fix a
mapped commit, and position 351 changes the dormant-slot restore of
PR18's position 113, which position 293 limited to one --mount when it
added aggregates, so they stay unassigned, like positions 346-349.
Note in the table's remarks that assigning them means revisiting that
work. The pin is now 194 commits past the mapped tip.

Positions 348 and 349 are no longer only a proposal:
microsoft/openvmm#4769 merged them, so position 349, d2338ea22, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4775 as the proposal of positions 350 and 351.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A capture that leaves the microVM filesystem slot dormant saves a
dormant device state, without an attachment, a policy, objects, or an
initialized FUSE session. A restore may bind a host filesystem to the
slot: the device takes the dormant state into any attached device,
since it restores only the session, which the guest never initialized.
Only the device of a single directory has been tested that way,
though.

Test that the dormant state of the slot restores into the device of an
aggregate of a read-write directory and a read-only file, and that the
device then saves the aggregate, with its children's kinds and modes,
rather than the dormant slot. The entry point does not let a restore
bind an aggregate to a dormant slot yet; the next change does.

See microsoft/nvx#540.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A platform snapshot that was captured without a filesystem keeps the
fixed virtio-fs slot dormant, and its restore could bind only one
directory to it, with --mount, although a cold boot attaches several
host directories and files as the children of one aggregate, and the
snapshot of an aggregate restores it. A template could therefore give
its clones only part of what cold boots export.

Let the restore of a platform snapshot whose slot is dormant attach an
aggregate: --mount-aggregate with its --mount-child options, and their
--mount-deny, --mount-allow, --mount-write, and --mount-owner policy.
The restore builds it with a cold boot's code, so it validates the
children's kinds, modes, names, overlaps, and limits, the policy paths,
and the owner as a cold boot does, and refuses what a cold boot
refuses, before any vCPU runs. Nothing is added to the machine: the
slot's device keeps the stable ID, tag, MMIO range, interrupt,
features, and queues that the snapshot recorded, and the guest keeps
its captured command line, which names no filesystem, so OpenVMM
exposes no layout, and the guest learns of the export from its host.
The device takes the dormant slot's state, whose FUSE session the guest
never initialized, as it does for a --mount, and the restore contract
still records the dormant slot.

Only the platform tier changes, which prepare_restore checks: a
platform snapshot is captured before any tenant input, while a later
tier carries a workload that ran without a filesystem. The dormant slot
of a workload-start, instance-checkpoint, or untiered snapshot still
binds only a --mount, a snapshot that predates the dormant slot binds
nothing, and a snapshot that captured a filesystem still requires
exactly that filesystem. The management API, which describes only a
single directory, does not change.

Unit tests check that a dormant-slot restore resolves an aggregate of
directory and file children, with mixed modes and denied, allowed, and
writable paths, to the configuration, roots, and identities of a cold
boot; that it refuses overlapping children, children of the wrong
kind, a repeated name, and policy paths that no child takes, with a
cold boot's errors; that it applies --mount-owner as a cold boot does;
that a snapshot without the dormant slot, or of a single directory,
refuses an aggregate; and the tier rule for every tier. The CLI
reference, the virtio-fs reference, the snapshot guide, and the help
of --mount-aggregate and --mount-child describe the rule.

See microsoft/nvx#540.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the copilot/microvm-platform-aggregate-export branch from d7892ff to 832d5bb Compare October 28, 2026 20:27
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Pin microsoft/openvmm#4775 at 832d5bb48dadd2cc5221f2c8b8905e6b80d13d53,
two commits on b92daa350, the previous pin, which is the head of
feature/microvm/dev.

A platform snapshot captured without a filesystem keeps the fixed
virtio-fs slot dormant, and its restore could bind only one directory
to it with --mount, although a cold boot attaches several host
directories and files as the children of one aggregate. A template
could therefore give its clones only part of what cold boots export.

The first commit tests that the device takes a dormant slot's state,
whose FUSE session the guest never initialized, into the device of an
aggregate, which then saves the aggregate. The second lets the restore
of a platform snapshot whose slot is dormant attach an aggregate with
--mount-aggregate and its --mount-child options, and their
--mount-deny, --mount-allow, --mount-write, and --mount-owner policy.
The restore builds it with a cold boot's code, so it validates the
children, the policy, and the owner as a cold boot does, and refuses
what a cold boot refuses, before any vCPU runs. The slot's device and
the captured command line, which names no filesystem, do not change,
so OpenVMM exposes no layout and the guest learns of the export from
its host. The dormant slot of a workload-start, instance-checkpoint,
or untiered snapshot still binds only a --mount, a snapshot that
predates the dormant slot binds nothing, and a snapshot that captured
a filesystem still requires exactly that filesystem.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Record the two commits that the previous commit pins as positions 353
and 354 of the microVM chain, and move the NVX pin to 832d5bb48.

Position 353 tests that the virtio-fs device takes the state of a
dormant filesystem slot into the device of an aggregate. Position 354
lets a platform restore bind an aggregate to the slot that the
snapshot left dormant, with its children, policy paths, and owner,
validated as at a cold boot. They add behavior rather than fix a
mapped commit, and position 354 changes the dormant-slot restore of
PR18's position 113, which position 293 limited to one --mount when it
added aggregates, so they stay unassigned, like positions 346-352.
Note in the table's remarks that assigning them means revisiting that
work. The pin is now 197 commits past the mapped tip.

Positions 350-352 are no longer only a proposal:
microsoft/openvmm#4773 merged them, so position 352, b92daa350, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4775 as the proposal of positions 353 and 354.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) merged commit 832d5bb into feature/microvm/dev Oct 28, 2026
74 checks passed
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Pin microsoft/openvmm#4775 at 832d5bb48dadd2cc5221f2c8b8905e6b80d13d53,
two commits on b92daa350, the previous pin, which is the head of
feature/microvm/dev.

A platform snapshot captured without a filesystem keeps the fixed
virtio-fs slot dormant, and its restore could bind only one directory
to it with --mount, although a cold boot attaches several host
directories and files as the children of one aggregate. A template
could therefore give its clones only part of what cold boots export.

The first commit tests that the device takes a dormant slot's state,
whose FUSE session the guest never initialized, into the device of an
aggregate, which then saves the aggregate. The second lets the restore
of a platform snapshot whose slot is dormant attach an aggregate with
--mount-aggregate and its --mount-child options, and their
--mount-deny, --mount-allow, --mount-write, and --mount-owner policy.
The restore builds it with a cold boot's code, so it validates the
children, the policy, and the owner as a cold boot does, and refuses
what a cold boot refuses, before any vCPU runs. The slot's device and
the captured command line, which names no filesystem, do not change,
so OpenVMM exposes no layout and the guest learns of the export from
its host. The dormant slot of a workload-start, instance-checkpoint,
or untiered snapshot still binds only a --mount, a snapshot that
predates the dormant slot binds nothing, and a snapshot that captured
a filesystem still requires exactly that filesystem.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 28, 2026
Record the two commits that the previous commit pins as positions 353
and 354 of the microVM chain, and move the NVX pin to 832d5bb48.

Position 353 tests that the virtio-fs device takes the state of a
dormant filesystem slot into the device of an aggregate. Position 354
lets a platform restore bind an aggregate to the slot that the
snapshot left dormant, with its children, policy paths, and owner,
validated as at a cold boot. They add behavior rather than fix a
mapped commit, and position 354 changes the dormant-slot restore of
PR18's position 113, which position 293 limited to one --mount when it
added aggregates, so they stay unassigned, like positions 346-352.
Note in the table's remarks that assigning them means revisiting that
work. The pin is now 197 commits past the mapped tip.

Positions 350-352 are no longer only a proposal:
microsoft/openvmm#4773 merged them, so position 352, b92daa350, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4775 as the proposal of positions 353 and 354.

Part of #540

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit to microsoft/nvx that referenced this pull request Oct 29, 2026
Record the five commits that the previous commit pins as positions
355-359 of the microVM chain, and move the NVX pin to bd2b27a2e.

Positions 355-359 restrict the DMA of the microVM virtio devices to the
RAM ranges of the e820 map: guest memory views, the virtio queue's
whole-buffer check, the MMIO transport's restriction and needs-reset
report, the loader's e820 RAM ranges, and the microVM worker that
applies them. Position 355 changes only guestmem, which no other commit
of the chain changes, so it can go upstream on its own, as positions
350-352 can, and position 356 adds only the queue's check on top of it.
Positions 357-359 build on the fork's shared interrupt-status transport,
MP-table loader, and microVM worker; say so in the table's remarks. The
pin is now 202 commits past the mapped tip.

Positions 353 and 354 are no longer only a proposal:
microsoft/openvmm#4775 merged them, so position 354, 832d5bb48, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4794 as the proposal of positions 355-359.

Refs #554

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Guide unsafe Related to unsafe code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant