Skip to content

nvx-rs: plan the clones of templates, and check their block geometry - #888

Open
Enrique Saurez (esaurez) wants to merge 39 commits into
devfrom
esaurez/844-pr1-clone-plan-geometry-20261024
Open

Enrique Saurez (esaurez) wants to merge 39 commits into
devfrom
esaurez/844-pr1-clone-plan-geometry-20261024

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #887; this PR retargets when its parent merges.

Chain: #879 → #881 → #885 → #886 → #884 → #887 → this PR, on dev; #866, #882, #883, #876, #871, #873, and #874 have merged.

Part of #844 (PR 1 of 5: the clone plan and block geometry).

What and why

A clone restores the generation of a template that it leased, with its own image and upper. This PR adds the plan that checks those inputs against the generation before any VMM starts, from metadata alone, so that a request that the template cannot serve fails before the launch. It launches nothing: the clone's launch and restore come in PR 2.

ClonePlan

ClonePlan::new(&Store, &mut Acquired, &CloneRequest, deadline) -> Result<ClonePlan> runs under the sandbox's lock that Store::acquire returned, before the launch that the lease reserves. It takes Acquired exclusively while it plans, since it may create a file in the sandbox's directory, as the store's transitions under its lock do, and borrows nothing of it once it returns, so the launch can take acquired.locked afterwards. A CloneRequest holds the adapter's SandboxPlan (SandboxPlan::adapter: the clone's image over an upper in RAM or on the caller's scratch image, and its network), the image's DistroBinding, the distro policy revision that the caller accepts, the ScratchBinding of a scratch image, and the memory target. The plan checks, in this order:

  1. The request: no sandbox-profile settings; an image of the format and identity that its binding names; absolute paths that OpenVMM takes as raw images, without , or ;, and not named .vhd, .vhdx, .vmgs, or .iso; and a scratch binding exactly for an upper on a scratch image.
  2. The lease: one of this store, under the lock that acquired it, whose reserved launch has not begun (LockedStore::reserved).
  3. The generation: still indexed, with the lease's key; its key (Store::read_key, without computing its identity again) and its snapshot (Store::snapshot_dir).
  4. The template: it speaks this build's contracts, those through which its clones are restored and the boot and control contracts of the runtime that captured it; the plan has a network exactly when the template has a NIC; and the memory target is whole blocks of 128 MiB (MEMORY_BLOCK_SIZE), and of the template's memory blocks, from the template's RAM to its RAM capacity.
  5. The sandbox's store: a link or a directory under one of the names that it owns. Then the image: DistroBinding::check, then the geometry of the template's placeholder, its length included, then the file at its path, which must be none of the store's files (below), by any path.
  6. A fresh capability, which the plan draws for the clone's launch.
  7. The upper. On a scratch image: ScratchBinding::check, which claims the image for the clone, then the geometry of the template's scratch, its length included, then the file at its path, which must be none of the store's files either. Nothing formats, copies, or reads the image. In RAM: the scratch block device, which every restore attaches, gets a fresh sparse file of the template's scratch length, clone-scratch.raw in the sandbox's directory, whose geometry on the sandbox's filesystem must be the template scratch's.

The check at a path is the one that #887's captures make, since OpenVMM opens a restore's block devices itself, by path, as #887's Store docs describe in "Block devices by path": a regular file, not a link or other reparse point, the file that the binding's check opened, of the current user or an administrator, and that no one else may change. It compares the files by the device and file ID of their seals, which on Windows is the 128-bit file ID of FILE_ID_INFO, unique on ReFS too. The plan holds the checked image, and for a scratch image the checked image with its claim. It fails holding nothing, and removes a sparse file that it began to create, even when the creation itself failed after it made the file.

Block geometry

CheckedDistro::geometry() and CheckedScratch::geometry(), on what the bindings' checks return, give the BlockGeometry that OpenVMM presents for a checked file, the type that the template's key records for its devices. They read the file's metadata through the checked handle, never its content: its length, which must be a nonzero multiple of 512 bytes; logical blocks of 512 bytes; and physical blocks of the I/O block size of its filesystem (st_blksize) on Linux, and of 4096 bytes on Windows, which must be a power of two of at least 512 bytes. A restore compares the geometry of each block device with the template's, so a clone's image has the placeholder's length, the capacity of every clone's image, in this stage.

The sparse scratch device's lifetime

clone-scratch.raw is a file of the sandbox's store, store::CLONE_SCRATCH_NAME in FILE_NAMES, as clone.json is: callers cannot declare it, LockedStore::remove removes it, and Store::release now removes it before the clone binding, once the clone's VM has exited, so that the binding's durable removal covers both. A plan replaces a file that an earlier clone of the sandbox left.

The store's transitions replace or remove each of its files: a plan clone-scratch.raw, a launch the log, the capability, the journal, the socket, the outcome report, and the console log, a release the clone binding, and a removal them all. So that none of them ever replaces or removes a caller's image, the plan lists the entries that the store owns, through LockedStore::owned_entries(), which names the store's files by the store's own paths, so that the file system resolves them as it does for the store's transitions, without regard to case on Windows, say, and refuses an image or a scratch image that is one of them, comparing the identities of the files, which catches every path to it, through a link among its directories or a hard link. Since the store creates nothing but regular files under its names, and its control socket, the plan also refuses a link or a directory under one of them, through which the path of an image could pass.

Endpoints

The plan draws the clone's capability. The control endpoint is the one that the clone's launch chooses, and the boot console stays none, as for cold boots (--virtio-console none), so the plan holds no endpoint.

Not in this PR

Validation

  • Windows: every step of check-nvx-rs and check-aci-edge-sandboxes that runs on Windows (fmt, clippy with all features, without features, and for each feature, tests with all and default features and without features, artifact bundling, docs, MSRV 1.89, the runtime adapter's protocol-only and host-only configurations, agent bundling), the markdown link check, nvx.py verify, and the OpenVMM vectors test.
  • Linux (x86_64): the same, plus the macOS target check, the guest binaries' checks, the runtime adapter's guest-only configuration, and the host tests of the clone plan, the platform, the supervisor, and the stores as root.
  • Each commit builds with clippy -D warnings on Linux and Windows, and passes the template, sandbox, and store tests on Windows.
  • New tests, on Linux and Windows: 18 tests of the clone plan, among them files on hugetlbfs, whose physical blocks are 2 MiB, on Linux; a creation of the sparse file that fails once it made the file; images and scratch images that are files of the sandbox's store, such as clone-scratch.raw, by its path, a hard link, or a linked directory, its log, or a record's temporary file; directories, or a link to one, under the store's names, and a leftover control socket, which passes; names of the store's files in other letter cases, which are the store's files where the file system ignores case; and templates whose runtime has other boot or control contracts. Also: the geometry of checked distro layers and scratch images, and its refusals; a release that removes the sparse file with the binding, durably; and a removal of a sandbox with the sparse file.
  • No VM: this PR launches nothing.
  • Restacked onto nvx-rs: capture templates into their store, and publish them #887's new head, whose base ports host paths to std::path: the clone plan now takes &Path where it took the removed PurePath, and shows paths as the base's block-device checks do, with no other change.
  • Left to CI: the full matrix, once this PR targets dev; a stacked PR runs none.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Scratch creation can replace a checked distro image, and compatibility validation omits the runtime’s boot and control contracts.

2 open issues
What changed in this PR

Adds metadata-only clone planning to NVX’s template infrastructure, implementing the first stage of #844 without launching a VM.

Changes:

  • Introduces clone requests and plans that validate leases, images, uppers, memory targets, and template compatibility.
  • Exposes checked block geometry and manages sparse scratch files for RAM-backed uppers.
  • Adds validation and cleanup tests, plus API and lifecycle documentation.
File Description
nvx-rs/​src/​host/​template/​store/​leases/​tests.rs Tests durable scratch and binding removal.
nvx-rs/​src/​host/​template/​store/​leases.rs Removes clone scratch files during lease release.
nvx-rs/​src/​host/​template/​store.rs Adds a configurable publication helper for tests.
nvx-rs/​src/​host/​template/​mod.rs Exports and documents clone planning.
nvx-rs/​src/​host/​template/​clone/​tests.rs Tests validation, scratch claims, and cleanup.
nvx-rs/​src/​host/​template/​clone.rs Implements clone planning and input checks.
nvx-rs/​src/​host/​supervisor/​mod.rs Documents planning before reserved launches.
nvx-rs/​src/​host/​store/​mod.rs Registers clone scratch files and exposes reservations internally.
nvx-rs/​src/​host/​store/​lifecycle/​tests.rs Tests scratch cleanup during sandbox removal.
nvx-rs/​src/​host/​sandbox/​scratch/​tests.rs Tests checked scratch geometry.
nvx-rs/​src/​host/​sandbox/​scratch.rs Exposes checked scratch geometry.
nvx-rs/​src/​host/​sandbox/​plan.rs Adds internal normalized scratch-path access.
nvx-rs/​src/​host/​sandbox/​mod.rs Implements shared block-geometry inspection.
nvx-rs/​src/​host/​sandbox/​distro/​tests.rs Tests distro geometry and filesystem block sizes.
nvx-rs/​src/​host/​sandbox/​distro.rs Exposes checked distro geometry.
nvx-rs/​README.md Documents clone planning and path-based device checks.

Review effort: Balanced

Comment thread nvx-rs/src/host/template/clone.rs
Comment thread nvx-rs/src/host/template/clone.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Preflight still accepts block-image paths that the subsequent launch replaces, risking restore failure and image loss.

1 open issue
2 resolved since last review

Review effort: Balanced

Comment thread nvx-rs/src/host/template/clone.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Windows filename casing can bypass ownership checks and allow lease release to delete a caller-supplied scratch image.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread nvx-rs/src/host/store/mod.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform file-identity checks, scratch-claim lifetimes, and durable lease cleanup warrant final human review despite no identified blocking findings.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform file identity, exclusive scratch claims, and crash-durable cleanup require human review because mistakes could replace or remove caller-owned images.

No open issues 🎉

Review effort: Balanced

Enrique Saurez and others added 6 commits October 26, 2026 17:42
The runtime adapter's layered launches had no network. `launch --network
FILE` now reads a network in the serde form of a plan's network, which
the plan checks as it reads it, and puts it on the adapter's plan, which
renders it through NVX's lowering of a policy, after its block devices.
The launch's record keeps it in the plan. A launch reads its network
before anything else, so it refuses one that the plan refuses before it
needs NVX's artifacts or creates the store.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
The root configuration gains an optional listeners field, after the
workload identity, which says whether the guest's listeners are
published. Executions, and the account check of a workload identity,
enter the root with that publication, so that while listeners are
published the entry refuses CAP_NET_BIND_SERVICE and CAP_NET_RAW at
stage 0. A layered launch names the publication of its network as
OpenVMM publishes listeners: with ingress or host-loopback access
allowed, and none without a network.

The host gives an execution's start the URL of a proxy-only network's
proxy in the proxy variables of its environment, with
layered::with_proxy, from the plan that LaunchRecord::read_plan reads
back from the store. The network device, its addresses, and the
announcer of the guest's listeners stay with NVX's /init.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
The host program's exec reads the plan that the launch recorded, and
gives each execution the URL of the proxy of the VM's network, if any,
in its environment with layered::with_proxy, after its --env entries,
unless they name the proxy variables themselves.

The VM scenarios can now give their VM a network, which the launch reads
from a file that the scenario writes. In a proxy-only network, the
executions find the proxy's URL in their environment, and root may hold
CAP_NET_BIND_SERVICE and CAP_NET_RAW. A network with ingress allowed
publishes the guest's listeners, so the guest refuses both at stage 0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
The proxy's variables lengthen an execution's start, so a start that the
framing carried when exec parsed its command line may no longer fit once
exec gives it the proxy of the VM's network. exec found out only as it
sent that start, after it had sent the starts before it, and then failed
without collecting their outcomes.

exec now gives every start the proxy's variables, and checks each as it
checked them without the proxy, before it attaches; it runs none of the
executions if one no longer fits. A start takes its key once exec has
attached, and a key's encoding has the same size whatever its value.

On Linux, whose command lines carry a start as long as the framing
allows, the proxy-only VM scenario checks that exec refuses such a batch
before it attaches, that none of its executions ran, and that the VM
still runs the next command's executions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
The layered scenarios check what each kind of network lets the guest's
executions reach, as NVX's proxy-only scenario probes its network: a
network that denies everything, which attaches no network device; an
allow rule; a proxy-only network, whose other destinations never
arrive; outbound host-loopback access, which publishes no listener of
the guest; and host-loopback access in both directions, through which
the host reaches a listener of the guest, at its port or through a
forward.

Before the launch, the test binds the host's side of the network, whose
ports the network names, each drawn at random from the dynamic range: a
test proxy on host loopback, and the sockets of the probes'
destinations. A scenario whose network stops traffic first boots a VM
of an open network over the same sockets, the positive control, in
which every probe arrives and the host reaches the guest's listener.
Each VM with a network has its network devices checked, and that
OpenVMM asks /init to publish the guest's listeners exactly where the
root configuration says that they are published.

The scenarios' guest moves to 192.0.2.2/24, in a range reserved for
documentation, so that no host's own address is on its network, and the
proxy-only network's proxy is the test proxy, at a port of its own.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
The VM scenarios run at the same time. The host's side of a network
draws the ports of a listener of the guest and of a forward, then
releases them, since OpenVMM binds them only once it runs, and again for
the scenario's VM after the positive control's. Another scenario could
draw the same ports in between, and one of the two VMs would then fail
to bind them.

Each host's side now holds the ports that it drew, in a set that the
scenarios share, until its scenario ends, and draws none that another
holds. A scenario that ends confirmed that its VMs ended; one that fails
keeps its ports for good, since one of its VMs might still run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a7c15634-094a-4a66-af59-25a58e732f4b
Enrique Saurez and others added 14 commits October 26, 2026 21:06
Address the review of the leases of the template store:

- Tombstones are read and checked wherever the store lists them, so that a
  tombstone of another store or of another format, or a link, fails a
  listing and a selection, as it fails an acquisition, rather than
  quarantining a generation. A retirement removes a tombstone only once it
  has checked it.
- A retirement makes the absence of a clone binding durable in the
  directory that holds the sandbox's entry, besides the sandbox's own
  directory: a sandbox that was removed and provisioned again at its path
  is a new directory, whose entries never held the binding.
- The first use of a store's directory of leases or of tombstones in a
  process also syncs the store's parent, so that no binding names a record
  of a store whose own entry a crash of the host could lose.
- On Windows, a record that a transaction in another process removes as
  this one opens or removes it is gone, rather than an error: a file system
  without POSIX deletion refuses every open of a removed file while other
  handles, such as a scanner's, hold it. A retirement that raced releases
  failed so, rarely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
On Windows, a refused read or removal of a template store's record
counted as a removal by another transaction if an open without access
rights was refused too. Win32 opens ask for SYNCHRONIZE and
FILE_READ_ATTRIBUTES even then, so a file whose access control list
grants neither was refused both ways, and taken for a removed one: a
retirement skipped the record of a live lease, or the binding that
names it, and deleted the generation, and a tombstone that refused
access quarantined nothing.

platform::is_deleted now opens the file with NtOpenFile, relative to
its directory, whose status tells STATUS_DELETE_PENDING from
STATUS_ACCESS_DENIED, which Win32 reports alike, and a refused file
counts as removed only if its deletion is pending, or it is gone.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A lease named its sandbox by the absolute path through which it was
leased. That path could run through a link, or, on Unix, through a
directory that a `..` component passes through. Once that link or
directory went, the path named nothing, though the sandbox and its
clone binding remained, so a retirement took the lease for unbound,
and could delete its generation while the clone ran. And a link that
changed while the sandbox was leased could have redirected the
binding to another sandbox.

acquire now resolves the links of the sandbox's path before it takes
the sandbox's lock, and the lease, the binding, and the lock that it
returns all name the sandbox by that path. The tests that build paths
through links now move a sandbox's directory and link its old path
after the lease.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A release took the sandbox's lock through any path. If that path ran
through a link, and the link changed after the release checked that
no VM runs, the release could remove another sandbox's binding, whose
lock it does not hold, so a retirement could then delete the
generation that the other sandbox's running clone uses.

A release now refuses, before it removes anything, a lock that does
not name the sandbox by its lease's path. That path has its links
resolved, and acquire takes the lock through it too.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A removal of a sandbox removed its clone binding with its other files
without syncing the directory. A retirement that later found the
binding gone synced only the directories that still existed. If the
sandbox's parent had been removed and created again, that sync missed
the directory that held the old entry, so a crash of the host could
bring the binding back after the retirement had deleted the generation
it names.

A removal now removes the binding first and syncs the sandbox's
directory before anything else goes. The configuration record still
goes last, so a failed sync leaves a provisioned sandbox that can be
removed again. The sync also covers a release that a crash interrupted
before its own sync. This costs one more directory sync per removal on
Unix; on Windows syncing a directory is a no-op.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A removal checked that the store recorded no VM and no launch, and
then removed the sandbox's clone binding, through the sandbox's path.
A lock taken through a path with a link in it stays on the directory
that the link named then. If the link changed between the checks and
the unlink, the removal could remove the binding of another sandbox,
whose VM may run, and a retirement could then delete the generation
that its clone still needs.

A removal now opens the directory once. Through that handle, it checks
that the directory holds the lock file whose lock the removal holds,
and refuses otherwise. It then checks that no VM and no launch is
recorded, removes the binding, and syncs the directory. Unix uses
fstatat, unlinkat and fsync; Windows uses native opens relative to the
directory, as is_deleted already did. The rest of the removal still
goes through the path, under the store's trust model.

A removal also removed the binding twice: first, and again with the
store's other files. On a Windows file system without POSIX deletion,
a reader that holds the binding open leaves the first removal pending,
so the second failed with PermissionDenied and aborted the removal.
The binding is now removed once, and one whose deletion is already
pending counts as removed. Such a binding lingers like a lock file
until its readers close it, in the removal's scan and in
Lingering::wait. Where the file system supports POSIX deletion, the
binding's name goes at once.

The lease tests also retry the recoveries of closed launches, as the
supervisor seam's tests do: a process that a concurrent test forks
holds a copy of the closed log until it executes its program.

Per removal, this adds a check of the lock file's identity, two stat
calls, and on Windows one open of the directory; Unix already opened
the directory to sync it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
…e access list

The Windows test of `was_removed` gives a file an access control list
that allows no one anything, and asserted that `platform::file_identity`,
whose open has no access rights but backup semantics, fails with a denied
access. On GitHub's hosted Windows runners that open succeeds, most likely
because the runner's elevated process holds the backup privilege, which
lets an open with backup semantics bypass the list. The test failed there
every time, at that setup assertion, before any of its assertions about
removals.

Check the open's error only where the open fails. Every assertion about
which files were removed stays: a file that the list refuses is not
removed, whether or not the open bypasses the list, and a file whose
deletion is pending, or that is gone, is, but only a denied access tells
so. No other test assumes that an open with backup semantics of a file
that allows no one anything fails.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
…rder

A clone's lease now reserves the launch of its clone. The acquisition draws
the launch's identity, records it in the lease's record and in the clone
binding before any VMM of the clone exists, and returns it as
Lease::launch. The supervisor takes the launch's identity from its caller:
supervisor::launch and LockedStore::begin_launch take a LaunchId, which
callers that launch no clone draw with LaunchId::generate, now public.

A lease allows one launch of its clone, under the lock that acquired it:
begin_launch refuses, with sandbox-leased, any other launch of a sandbox
that holds a lease, a launch under another lock, and a second launch, until
a release ends the lease. After a crash before the launch, a release ends
the lease at once, and another acquisition checks the generation's
eligibility again.

A stop or a termination clears the runtime record only if it is still the
record of its own VM, and not another VMM's under a launch identity that
its caller used again.

The supervisor's documentation describes the launch of a clone, its
crashes, and one lock order: the template store's lock, then the sandbox's
lifecycle lock, which guards its clone binding and launch journal, then the
scratch image's claim. No lock lasts for a VM's lifetime.

Tests crash a clone's orchestrator, in a process of its own, at each step
of an acquisition and of a release, and around the spawn of the clone's
VMM, and check what each evidence of a VM's exit means for its lease: an
earlier boot, a reused process ID, a process that cannot be examined, and a
forced teardown that fails.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
Four lease tests began a launch, dropped it, and asserted that the
next recovery ends it. On Linux, a process that a concurrent test
forks holds a copy of the closed log until it executes its program.
If that outlasts recovery's own wait, the recovery is indeterminate
and the test fails although nothing is wrong.

The tests now retry the recovery until it ends the launch, through
test_support::released, as the sandbox store's tests do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: bc335726-838c-4538-a3c4-d6791aba5101
A template is a capture of a microVM that consumed only its invariant
configuration, for clones that each bind their own image, upper, and
export when they are restored. Before a store captures one, the host
needs the exact VM and options of that capture: OpenVMM refuses a
platform-tier capture whose kernel command line holds a token outside its
platform allowlist, and a template must hold no tenant data.

Add nvx::host::template::TemplatePlan, which lowers an adapter's
SandboxPlan without I/O into:

- the placeholder distro, bound through its DistroBinding, whose length
  is the capacity of every clone's image, at an absolute path that
  OpenVMM's block device options hold whole, and that it attaches as a
  raw image, over a dormant scratch device, a sparse file in the capture's
  transaction directory;
- the dormant filesystem slot, and no NIC or one fixed NIC under the
  template's policy, which may allow neither ingress nor host-loopback
  access both ways, since OpenVMM publishes the guest's listeners through
  them as live host attachments, which no snapshot holds;
- TemplateResources: every processor online at boot, and RAM with a
  --memory-capacity, in blocks of 128 MiB;
- the kernel command line nvx_sandbox=1, the one token that the host
  adds to those that OpenVMM writes itself;
- the capture's options: --snapshot-tier platform, a
  --snapshot-destination in the transaction directory, under the name of
  a generation's snapshot in the template store, and
  --snapshot-block-identity generation with the capture's
  StorageGeneration, the bound-block storage generation, so that the
  capture hashes no block device.

The tests check each token of a template's command line with OpenVMM's
own allowlist, compiled from verbatim excerpts of its source in
tests/vectors/openvmm/, which the ignored test of
tests/openvmm_vectors.rs compares with the OpenVMM submodule.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 32eea669-a52c-4a53-87fa-521c498537e5
Store::create_template serves a capture class with a generation that the
store holds and that is eligible at the host's time, which it reads once
it holds the class's slot, or captures one, and
Store::refresh_template always captures; Store::recover_captures ends the
captures that a crash interrupted.

A capture runs in its class's slot, a sandbox store under captures/ whose
lock serializes the class's captures. In a private transaction there, it
writes the capture's record, creates the dormant scratch, lowers the
adapter's plan into a TemplatePlan with a fresh storage generation, and
launches the template's VM through supervisor::launch, as a launch of its
own. OpenVMM opens snapshot-capable block devices itself, by path, without
following links and, on Windows, sharing only reads and deletion, so the
capture hands it paths: it checks the transaction, the placeholder through
its binding and at its path, and the scratch at its path, and holds no
handle that could write either file as the VM starts. Once the VMM has
exited, the capture checks the outcome report, the snapshot's manifest and
artifacts, and the key, then publishes the transaction in two commits: a
rename into generations/ that replaces nothing, then the index, whose
commit also syncs the store's own entry in its parent. Each capture, and
each recovery, first drops the generations whose payloads a crash lost,
then indexes a payload whose publication a crash interrupted, once it
has made the rename durable; a creation makes the store durable before
it serves a generation that the store holds. Older generations and their
leases stay untouched.

Store::snapshot_dir accepts the snapshot directories that OpenVMM
creates, whose own access control lists on Windows pass no entries on to
the files created in them, while every directory that the host creates
keeps its check.

The supervisor's lock order gains a capture slot's lock, which comes
before the template store's.

The fake OpenVMM runs the VMs of captures as a scenario asks, and records
whether a handle could write their block devices; the template-capture
tests run captures through the supervisor against it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 32eea669-a52c-4a53-87fa-521c498537e5
OpenVMM requires the control socket of a VM that captures a snapshot, on
Linux, in the directory of the snapshot's destination. It refused every
capture, whose VM listened in the slot of its class while the destination
is in the capture's transaction.

The capture now launches its VM from a sandbox store in the transaction
itself, whose journal records the VMM until the VM has ended. The host then
moves the VMM's log and outcome report to the slot, which keeps those of
the class's latest capture, closes the transaction's lock, and removes its
file before it checks and publishes the transaction. Recovery ends a launch
that a transaction records before it publishes or retires the transaction.

The supervisor's lock order now names the transaction's lock, which a
capture or its recovery takes under its slot's lock, and releases before
it takes the store's lock to publish.

The fake OpenVMM refuses a capture whose control socket is elsewhere, as
OpenVMM does.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 32eea669-a52c-4a53-87fa-521c498537e5
OpenVMM refuses, on Unix, a control console unless its socket's
directory is a directory, not a link, of its user with exactly the mode
0700, and, if the VM restores or captures a snapshot, the canonical
directory that holds the snapshot. The fake OpenVMM checked the second
rule for captures alone.

It now checks both rules, with OpenVMM's messages, for every control
console that it serves. As in OpenVMM, a snapshot that the VM restores
takes precedence over a capture's destination, so a restore that the
fake accepts must listen beside its snapshot too. The fake also refuses
a control console that names no endpoint after listen=.

The tests of captures' command lines use private directories. New tests
refuse other modes, a link, a missing directory, and an empty listener,
and check the rule for a restored snapshot.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 32eea669-a52c-4a53-87fa-521c498537e5
A capture checks, just before its VM starts, that the placeholder at its
path is the file that its binding's check opened, and that the scratch at
its path is the file that the capture created. It compared the identities
that platform::opened_file_identity returns, which on Windows are the
volume serial number and the 64-bit file index of
BY_HANDLE_FILE_INFORMATION. That index is not guaranteed unique on ReFS,
which Dev Drives use, so another file at either path could pass as the
one that the capture checked.

Compare the identities that the files' seals record instead: the device
or volume and the file ID, which on Windows is the 128-bit ID of
FILE_ID_INFO, unique on ReFS too. Linux keeps st_dev and st_ino, and
other hosts still cannot identify files.

A test renames another file of the scratch's length over the scratch
before the VM starts, which the capture refuses, and checks that the
scratch's check compares the whole of a 128-bit file ID.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 32eea669-a52c-4a53-87fa-521c498537e5
Enrique Saurez and others added 4 commits October 27, 2026 09:39
ClonePlan checks a clone against the generation that it leased, before
its launch and under the lock that acquired the lease: the template's
contracts, its NIC, a memory target in whole blocks from the template's
RAM to its capacity, and the clone's image and upper. The image and the
scratch device must present the block geometry that the template
recorded, which CheckedDistro::geometry and CheckedScratch::geometry
read through the checked files as OpenVMM presents it, and be, at the
paths by which OpenVMM opens them, the files that the checks found.

An upper in RAM gets a fresh sparse scratch device, clone-scratch.raw,
a file of the sandbox's store that a release of the lease removes. A
scratch image is claimed, and never formatted, copied, or read.

Part of #844.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a424884c-018a-45f5-a7fe-c57548a873f0
…ch as an image

A clone plan now refuses a template that a runtime of other boot or
control contracts captured, since this build cannot drive its guest.
It also refuses an image or a scratch image that is the sandbox's
clone-scratch.raw, by any path, since a plan replaces that file and a
release of the lease removes it. The plan compares the files'
identities, which catches a link among the directories of the path and
a hard link. Since the store creates nothing but a regular file there,
the plan also refuses a link or another kind of file at that path,
through which the path of an image could pass.

Part of #844.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a424884c-018a-45f5-a7fe-c57548a873f0
A clone plan refused an image or a scratch image that is the sandbox's
clone-scratch.raw. Every file that the sandbox's store owns is at the
same risk: a launch replaces the log, the capability, and the journal,
and removes the socket, the outcome report, and the console log, a
release removes the clone binding, and a removal removes them all. The
plan now lists the entries that the store owns, its files, the
temporary files of its records, and the staged outcome reports, through
LockedStore::owned_entries. It refuses an image or a scratch image that
is any of them, by any path, and a link or a directory under any of
their names.

Part of #844.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a424884c-018a-45f5-a7fe-c57548a873f0
LockedStore::owned_entries matched the directory's listed names against
the store's file names, which a file system that ignores case, as
Windows' do, may list in other letter cases: an image under such a name
passed a clone's plan, though it is the store's file, which a release
removes. It now returns the store's files at the store's own paths,
which the file system resolves as it does for the store's transitions,
and the temporary files under their listed names, as a removal finds
them.

Part of #844.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a424884c-018a-45f5-a7fe-c57548a873f0

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform file identity, scratch-claim lifetimes, and crash-durable cleanup warrant human review alongside the pending CI matrix.

No open issues 🎉

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/841-pr2-capture-publication-20261024 branch from 0a25211 to 0b23113 Compare October 27, 2026 19:53
Base automatically changed from esaurez/841-pr2-capture-publication-20261024 to dev October 28, 2026 01:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants