Repository navigation
tests: capture a template's VM in the runtime adapter's guest - #932
Merged
Pedro Henrique Penna (ppenna) merged 2 commits intoOct 27, 2026
Merged
Conversation
A platform template's snapshot must hold no root, so that every clone that a restore starts attaches its own. The runtime adapter's guest now has a template mode: on x86_64, its worker reads the kernel command line before anything else, and in a VM whose command line names the platform snapshot tier, it captures the VM before it assembles any root, through nvx::guest::clone. - template::boot splits the command line as the kernel does, through nvx::protocol::cmdline::parameters. A command line without nvx_snapshot_tier=platform boots cold, as before. With it, every parameter must be on the platform tier's allowlist, the tokens that OpenVMM's capture of a platform snapshot allows, and the command line may hold neither a double quote nor `--`. - worker::capture attaches the control session with GuestConnection::attach, waits until the tty sent the attachment, and runs the capture's preparation and request behind the worker's fence of workload admission, which a close acquires. It reads nothing from the session, and serves no application session, before the capture. - A refused command line, each failed stage of the capture, a capture that ended without a snapshot, and a restored clone, which the worker does not repair yet, end the VM with statuses 20 to 31, which the README tabulates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2da708b6-0bda-4804-b01b-936cdc5f054a
A split at ASCII whitespace, as OpenVMM's checks of a platform snapshot's command line make, keeps a vertical tab or the byte 0xa0 within a token, but the kernel splits the command line there. Template mode reads the command line as the kernel does, so it refuses a parameter that either splits off a platform token's value, and finds a `--` that a vertical tab separates. Pin both in its unit tests, and name them in the README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2da708b6-0bda-4804-b01b-936cdc5f054a
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The privileged capture path couples control-session state, guest shutdown, and time-ABI behavior, warranting human review supported by cross-backend VM validation.
No open issues 🎉
What changed in this PR
This PR adds template mode to the runtime adapter’s guest, completing the guest-side integration for #840. It captures a platform VM before root assembly so restored clones can attach their own roots.
Changes:
- Detects template mode from kernel parameters and enforces the platform command-line allowlist.
- Attaches and drains the control session before preparing and requesting capture, without serving application sessions.
- Documents and tests capture ordering, admission fencing, and exit statuses, including temporary shutdown of restored clones pending #842.
| File | Description |
|---|---|
tests/runtime-adapter/src/worker/capture.rs |
Implements capture preparation, session attachment, fencing, statuses, and unit tests. |
tests/runtime-adapter/src/worker.rs |
Exposes the x86_64 capture module and documents template behavior. |
tests/runtime-adapter/src/template.rs |
Defines template detection, the allowlist, shared statuses, and parser tests. |
tests/runtime-adapter/src/lib.rs |
Exports the template module in every configuration. |
tests/runtime-adapter/src/bin/runtime-adapter-guest.rs |
Routes template workers to capture before normal root handling. |
tests/runtime-adapter/README.md |
Documents template flow, failure statuses, and test coverage. |
Review effort: Balanced
Base automatically changed from
esaurez/840-pr1-capture-preparation-20261024
to
dev
October 27, 2026 19:44
This was referenced Oct 27, 2026
Pedro Henrique Penna (ppenna)
deleted the
esaurez/840-pr2-template-mode-20261024
branch
October 27, 2026 21:16
This was referenced Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A platform template's snapshot must hold no root, so that every clone that a restore starts attaches its own image, upper, and export. #877 adds
nvx::guest::clone, through which a guest init or agent prepares its guest for a capture before its root's assembly. This gives the runtime adapter's guest agent a template mode that uses it: in a VM whose kernel command line names the platform snapshot tier, the worker captures the VM before it assembles any root, and runs no application session before the capture.Part of #840: the second of its two pull requests. #841 runs this mode in VMs, and #842 repairs a restored clone.
Chain: #886 → #877 → this PR → #884 → #887 → #841 PR 3, on
dev; #879, #881 and #885 have merged.What changed
The command line (
tests/runtime-adapter/src/template.rs, in every configuration).template::bootreads a kernel command line, such as/proc/cmdline, as the kernel splits it, withnvx::protocol::cmdline::parameters. One in which the kernel finds no parameternvx_snapshot_tier=platformboots cold, whatever else it holds. With it, the VM is a template's, andbootrefuses, with aCommandLineError:--, after which the kernel hands the rest of the command line to init;template::is_platform_token.The allowlist is the one that OpenVMM's capture of a platform snapshot enforces, in
platform_command_line_token_is_invariantat the pinned OpenVMM:earlycon=xe9,console=hvc0,console=hvc1,reboot=t,panic=-1,nvx_sandbox=1,nvx_config=0xd0010000,65536,nvx_snapshot_tier=platform, andnvx_control_tty=hvc2whole, and any value ofnr_cpus=,maxcpus=,virtio_mmio.device=,virtnet_ip=,virtnet_mask=,virtnet_gw=,virtnet_dns=,virtnet_ip6=, andvirtnet_gw6=. It compares names exactly, as OpenVMM does.The capture (
tests/runtime-adapter/src/worker/capture.rs, in the guest-only configuration on x86_64).worker::capture::capture_on_terminal:GuestConnection::attach, and waits until the tty has sent the attachment, so that the snapshot holds the guest attached. It reads nothing from the session, and serves no application session, before the capture;CapturePreparation, for the worker's fence,Admission, and the template'sdevices(): thedistroand scratch block devices, at the MMIO windows from which the worker assembles a root, and OpenVMM's host filesystem device, taggedmicrovm;CaptureBarrier::capture.AdmissionimplementsAdmissionFencewith #877's acquire contract:closefails withResourceBusywhile the fence is closed, whoever closed it, and leaves it closed.The worker (
tests/runtime-adapter/src/bin/runtime-adapter-guest.rs). On x86_64,runtime-adapter-guest workerreads/proc/cmdlinebefore it opens its tty. A VM that boots cold goes on as before. A template's VM runs the capture, after which the worker, and with it the VM, ends with one of these statuses, after a diagnostic on standard error:--, or the worker cannot read itA committed capture ends the VM before the worker returns. The README's "Template mode" section describes the flow and the statuses, and the module documentation the API.
Design notes
bootdecides on the parameters that the kernel sees, so a tier that an unclosed quote or a--hides from the kernel boots cold, as the kernel does. OpenVMM splits the command line at ASCII whitespace, which leaves out the vertical tab and the byte 0xa0, at which the kernel splits too:virtnet_dns=10.0.2.3, a vertical tab, andquietpass OpenVMM's check as one token, but the kernel seesquiet, which template mode refuses. The guest fails closed there, and its tests pin both separators.template::CAPTURE_STATUS_BASEplus the number of the stage that failed, in the order in whichCaptureStagedeclares them. None collides with the root's stages, 11 to 16, the session's 1, or the time ABI's own 193 to 195.Validation
At
f28fdca, the head, on #877 at060eefc.Each of this PR's commits passes, on Linux x86_64 and Windows x86_64, nvx-rs's formatting, Clippy configurations, and host modules' layering test, and the runtime adapter's formatting and Clippy in each of its configurations, with template mode's tests: those of
templateeverywhere, and those ofworker::capturein the musl guest-only configuration on Linux.At the head, on Linux x86_64 (toolchain 1.95.0, MSRV 1.89): check-nvx-rs's runtime adapter steps, in the protocol-only configuration, with 39 library tests passing, the host-only one, and the musl guest-only one, with 61 library tests passing, the 6 of
templateand the 5 ofworker::captureamong them, each with its MSRV check; and nvx-rs'scargo clippy --liband host modules' layering test.On Windows x86_64:
nvx.py verify; nvx-rs's formatting, Clippy with all features and of the library, its tests with all features, and the layering test; and the runtime adapter's protocol-only and host-only configurations, with their MSRV checks.The runtime adapter's VM suites at the head, whose VMs boot cold as before, with the artifacts of NVX's development release of
devat60e6aab, whose guest, kernel, and OpenVMM inputs this PR shares, and its OpenVMM at the pinnede68871c:vm_layered43,vm_restart3, andvm_steps3 passing, on Linux with MSHV, with OpenVMM's host CPU profile, and on Windows with WHP.A CI run dispatched at the head passed every job but one: on Linux with MSHV, the runtime adapter's
outbound_host_loopback_publishes_no_listenerscenario, which this PR doesn't change, saw its guest listener exit at once with status 1, rather than wait out its 5 seconds. That is consistent with a collision of the listener's port, which the host draws from 49152 to 65535, with a guest port that the scenario's earlier outbound probes left in TIME_WAIT, which fails the listener's bind. The same job passed in #877's run, and the scenario passed with KVM and WHP in this run, and with MSHV and WHP above.An earlier final review of the change found no issues, as did a final review of the rebase, its conflict resolutions, and the new
maxcpus=token, and Copilot's review.