Skip to content

runtime-adapter: activate a clone through Activate in the framing - #941

Open
Enrique Saurez (esaurez) wants to merge 6 commits into
esaurez/843-pr2-root-workload-activation-20261024from
esaurez/843-pr3-adapter-activate-20261024
Open

Enrique Saurez (esaurez) wants to merge 6 commits into
esaurez/843-pr2-root-workload-activation-20261024from
esaurez/843-pr3-adapter-activate-20261024

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #940; this PR retargets when its parent merges.
Chain: #933 → #880 → #936 → #937 → #939 → #940 → this PR, on dev; #879, #881, #885, #886, #877, #932, #884, and #887 have merged.

Part of #843 (PR 3 of 3: Activate in the test adapter's framing).

What and why

With #937, the runtime adapter's guest repairs each clone that a restore of its template's snapshot starts, then ends it at once with status 31, since nothing activates a clone yet. With #939's Activation and #940's apply, the guest now continues past the repair to Ready, in the adapter's own framing.

The framing

Kind Message Body
27 Activate: the host's activation of a clone, which the guest answers with Ready the clone's activation record, as nvx::activation::ActivationRecord::encode writes it, at most nvx::activation::MAX_RECORD_BYTES (frame::MAX_ACTIVATE_BODY)
  • A clone's Ready (kind 7) reports its activation after the guest agent's build, as nvx::activation::ReadyActivation::encode writes it: 1, then the transaction ID of the record's restore. root::Ready::with_activation adds it, and root::Ready::activation returns it, so a host checks Ready::decode(body)?.activation() == Some(ReadyActivation::of(&record)).
  • A cold boot's Ready reports no activation, and root::Ready::decode refuses bytes after the build that are not an activation.
  • The worker of a VM that boots cold refuses an Activate, before and after Ready, as it refuses any message that it does not take in its state.
  • The framing refuses the header of an Activate that announces more than frame::MAX_ACTIVATE_BODY bytes before it reads any byte of the body. A clone's activation takes that refusal as an Activate whose record has that many bytes, which fails it with 52 once the session is restricted, whether the clone is ready or not.

A clone's session

  1. Once the worker repaired a clone, it resumes, on the same tty, the session that the template attached before its capture, without attaching again. It acknowledges the reset that the restore queued, which ends the activation's transport bootstrap.
  2. It takes one Activate. The activation decodes and checks the record, and the worker applies it with nvx::guest::clone::apply, with the inputs of the root's assembly that the template's own configuration gives (worker::capture::root_template), and the activation's deadline, past which the application waits for no device. Once it took what the session received while the application ran, it completes the activation, which opens the fence of workload admission, and answers Ready, to the Activate and to each byte-equal retry that came meanwhile, which joins the application.
  3. It then serves the host's order as after a cold boot: the mapping table, the start, the executions, the probes, and the stop. The record's workload identity, if it has one, becomes the sandbox's identity, as a root configuration's does, so that executions name none. Without one, each start names its own.

Until the clone is ready, the session takes nothing but that Activate and its byte-equal retries: anything else that came before Ready, while the application ran included, a reset among them, fails the activation, so a host waits for Ready before it sends the mapping table or the start. A byte-equal retry gets the same Ready without a second application, also after a later reset. The activation's deadline, worker::activation::ACTIVATION_TIMEOUT, is 120 seconds from the repair's acknowledgement of the restore.

How a clone ends

worker::activation::failure_status selects the status of an activation's failure, with nvx::protocol::status's constants:

Status Constant When
50 ACTIVATION_DEADLINE The deadline passed before the clone was ready, the application's waits for the devices included
51 ACTIVATION_PROTOCOL Another reset, or another message, before Ready, while the application runs included, or an Activate of another transaction or record
52 ACTIVATION_RECORD The record does not decode, breaks a limit, or is a cold boot's, or the Activate's header announces more bytes than a record has
53 ACTIVATION_RESOURCES The record expects other resources than the repair found
54 ACTIVATION_ROOT The root's assembly failed, or the image is a GPT disk
55 ACTIVATION_WORKLOAD The image holds no account of the workload identity, or not its group
58 ACTIVATION_ORDER The worker's own order misused the activation
  • 56 and 57 stay unused until nvx-rs: activate a clone's root for GPT images and either upper, with its export, network settings and workload view #845.
  • A broker that closes the stream before the clone is ready ends it with 1, the status of the host's session (worker::Worked::Abandoned).
  • After a failed activation, or such a close, the worker drops its executions, then waits for its stream, and then its tty, to take what it wrote, for at most worker::UNACTIVATED_DRAIN, a second each. It keeps the status whatever the stream does, so a stream that takes nothing, or fails, neither keeps the clone from ending nor changes its status.
  • A ready clone ends as the worker of a VM that boots cold does.

Changes to #937's template path

  • The template completes its session's handshake before the capture. The worker now reads the reset with which the broker answers its attachment, acknowledges it, and drains the tty, before it captures, so that the snapshot holds the session active. A template that never read its reset would leave it partly sent to the clone: the restore's broker finishes a record in progress, then queues its own reset, so the clone would see two resets and fail its activation with 51.
  • The template bootstraps the guest for every clone's root. Before the attachment, it bootstraps with its own root configuration, the default one, so that the snapshot holds the cgroups that clones use. A bootstrap that fails ends the template with 13.
  • Status 31 is retired. A repaired clone no longer ends at once; it ends as its session does.

Changes to #939's and #940's APIs

The review's fixes, below, change two APIs of this PR's parents, which stay as they are:

  • nvx::guest::clone::apply(record, template, deadline) takes the activation's deadline, Option<Instant>. The assembly waits for each device as long as the template's timeout allows, but never past the deadline: it checks the time before each poll, and sleeps no longer than until the deadline, at which it polls a last time if the deadline comes first, so that the application ends by then, past which Activation::complete fails the activation as expired. nvx::guest::root::assemble keeps its behavior.
  • nvx::guest::clone::ActivationInput::TooLong(length) is an Activate whose header announced length bytes, more than a record has, which the adapter's framing refused unread. It fails the activation as a record that long does, with ActivationError::TooLong, or, in the transport bootstrap, as a request.
  • The Activation example passes the activation what the session received while the application ran before it completes the activation, and answers each retry that joined the application with Ready.

Review fixes

  • 9965bf1: an Activate too long to read fails a clone with 52, rather than ending its session with 1.
  • a810ac4, 2e90508, and d912f46: the application's waits for the devices end by the activation's deadline, at which the last poll happens.
  • d22525c: the worker takes what the session received while the application ran before it completes the activation, so that a reset or a request that came before Ready fails the activation rather than reaching the ready clone.

Elsewhere

  • nvx::guest::clone::Activation::with_resources builds an activation from the resources and the instant of the acknowledgement that a repair reports. The session tests use it, since they run without a restore. Activation::new calls it.
  • The documentation of nvx::protocol::status::ACTIVATION_PROTOCOL now covers an Activate that changes the record after Ready.
  • In the framing's tests, the unknown kind moves from 27 to 28.

Validation

This PR was rebased onto #937's new head, 5683164, without conflicts and with its patches unchanged. At its new head, ea69821, with the OpenVMM submodule at its gitlink, d2338ea2:

Linux x86_64:

  • nvx-rs: cargo fmt --check; Clippy with -D warnings, --all-targets --all-features, --lib, and --lib --no-default-features; cargo test --all-features (1800 passed), cargo test (877 passed), and the --lib --no-default-features build and test (260 passed); tests/host_modules.rs (22 passed); the bundled-artifacts test (19 passed); rustdoc with -D warnings; the MSRV (1.89) check; the aarch64-apple-darwin check.
  • Guest binaries: nvx.py check-guest-binaries (musl Clippy and tests, 982 passed), build-guest-binaries, test-guest-binaries (33 passed), check-static-elf.
  • tests/runtime-adapter: fmt, Clippy, build, tests, and MSRV in its protocol-only (47 passed), host-only (99 passed), and musl guest (136 passed) configurations.
  • scripts/check_markdown_links.py.
  • As root, with standard input from /dev/null: the privileged capture test, the strict cold boot that stops at its violation, and the tests of guest::clone and guest::reseed (75 passed); the runtime adapter's library and guest-session tests (128 passed).

Windows (x86_64-pc-windows-msvc): nvx-rs fmt, Clippy (all features, --lib, no default features, each feature alone, and all features for the musl target), tests (all features 1118 passed, default 896, no default features 260), tests/host_modules.rs, rustdoc, MSRV; tests/runtime-adapter fmt, Clippy, build, tests (protocol-only 47 passed, host 100), and MSRV in its protocol-only and host configurations, and Clippy and MSRV in its musl guest configuration; the Markdown links. The adapter's musl rustdoc with -D warnings, which CI doesn't run, fails with the same five broken links in exec.rs and worker/mapping.rs as at #937's head; this PR doesn't touch them.

The runtime adapter's VM suites, in the CI run below, with OpenVMM at d2338ea2 and the guest agent built from ea69821: 58 of 58 passed on KVM, on MSHV, and on WHP (vm_layered 43, vm_template 9, vm_restart 3, vm_steps 3). No suite restores a clone yet, so the activation runs in no VM here.

Per commit, on Linux x86_64 and on Windows: fmt and Clippy with -D warnings for nvx-rs in CI's three configurations and --lib, tests/host_modules.rs, and the runtime adapter's fmt and Clippy in its protocol-only, host-only, and musl guest configurations; on Linux, also the aarch64-apple-darwin check.

CI: ci.yml run 40619378275, dispatched on ea69821, since this chain's PRs don't target dev and get no PR CI, passed: 36 jobs succeeded, among them nvx-rs and the CLI on Linux and Windows, the guest artifacts, OpenVMM's binaries, unit tests, and vmm-tests, and the NVX microVM tests, with the runtime adapter's VM suites, and the platform jobs on KVM, MSHV, and WHP; the 6 jobs that only pull requests or dev run were skipped.

Tests added:

  • The framing: Activate's kind, body, and limit, and a kind for every code from 1 to 27.
  • Ready: a clone's activation after the build, for builds of any length, and the bytes after a build that are not an activation, which Ready::decode refuses.
  • The template: the worker acknowledges the broker's reset before the capture, and reads nothing more, and no capture runs without the session's handshake.
  • The activation's statuses: each class of failure ends the clone with its own status, the deadline covers the wait for both devices, and a clone assembles its root with the template's configuration.
  • nvx-rs, for the review's fixes: an Activate too long to read fails as a request in the transport bootstrap, and from then on as a record that long, ready or not; an assembly waits for its devices until its deadline at the latest, also when a poll or a sleep overruns it, and finds a device that appears in the last, shorter, interval before the deadline; and the application hands the assembly the activation's deadline.
  • The session (tests/guest_session.rs): the cold worker refuses Activate; and a clone:
    • answers the restore's reset and activates with the record of its Activate, whose retries get the same Ready;
    • takes nothing but its Activate before it is ready;
    • fails finally with the status of each class of failure;
    • ends with the deadline's status if the host does not activate it in time;
    • still ends once its activation failed, or its broker closed or is gone, when its stream takes nothing;
    • fails on an Activate that is not a retry once it is ready;
    • fails with 52 on an Activate longer than a record, before it takes the body, whether it is ready or not;
    • passes its application the activation's deadline, and ends with the deadline's status once an application that would outlast it reaches it, with no Ready;
    • takes what came during its application before it is ready: a reset or a request fails it, and a byte-equal retry joins it, so that both get Ready after one application;
    • runs its executions as its record's workload identity.

Left to #844's PR 5, as the issue says: the VM tests on KVM, MSHV, and WHP, of no host input before the acknowledgement and no workload before Ready.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Oversized Activate headers produce the wrong session status, and synchronous application/device waits can exceed the activation deadline; both need fixes and regression tests.

2 open issues
What changed in this PR

This PR completes the guest-side activation framing for #843, connecting NVX’s clone activation library to the runtime adapter. Repaired clones can now accept Activate, report activation in Ready, and serve workloads instead of exiting immediately.

Changes:

  • Adds Activate framing and activation metadata to Ready.
  • Resumes captured sessions after repair, applies clone configuration, and handles deadlines, retries, and failure shutdown.
  • Updates template preparation, documentation, and framing/session tests.
File Description
tests/​runtime-adapter/​tests/​host_session.rs Moves the unknown-kind test to 28.
tests/​runtime-adapter/​tests/​guest_session.rs Adds clone activation and session lifecycle tests.
tests/​runtime-adapter/​src/​worker/​connection.rs Supports resumed sessions and bounded output draining.
tests/​runtime-adapter/​src/​worker/​capture.rs Completes the template handshake and serves repaired clones.
tests/​runtime-adapter/​src/​worker/​activation.rs Implements activation handling and failure-status selection.
tests/​runtime-adapter/​src/​worker.rs Integrates clone phases, deadlines, and shutdown handling.
tests/​runtime-adapter/​src/​template.rs Retires status 31 and updates lifecycle documentation.
tests/​runtime-adapter/​src/​root.rs Encodes and decodes activation metadata in Ready.
tests/​runtime-adapter/​src/​lib.rs Documents the clone activation flow.
tests/​runtime-adapter/​src/​guest.rs Adds time-limited terminal draining.
tests/​runtime-adapter/​src/​frame.rs Adds message kind 27 and its body limit.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-guest.rs Wires clone serving into the guest worker.
tests/​runtime-adapter/​README.md Documents framing, lifecycle, statuses, and tests.
nvx-rs/​src/​protocol/​status.rs Clarifies protocol-failure documentation after Ready.
nvx-rs/​src/​guest/​clone/​activation/​tests.rs Uses the public resource-based constructor.
nvx-rs/​src/​guest/​clone/​activation/​apply/​tests.rs Updates the test adapter’s activation construction.
nvx-rs/​src/​guest/​clone/​activation.rs Exposes Activation::with_resources.
nvx-rs/​README.md Documents resource-based activation construction.

Review effort: Balanced

Comment thread tests/runtime-adapter/src/worker.rs
Comment thread tests/runtime-adapter/src/worker/activation.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Deadline-limited device polling drops the final partial interval, potentially rejecting a device before the deadline and reporting ACTIVATION_ROOT instead of ACTIVATION_DEADLINE.

1 open issue
2 resolved since last review

Review effort: Balanced

Comment thread nvx-rs/src/guest/root/block.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The NVXS snapshot/restore and tty lifecycle interactions require human review and real-VM validation, since socket-based fakes cannot establish their runtime ordering.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Enrique Saurez and others added 6 commits October 27, 2026 18:57
A clone that a restore of a template's snapshot starts now continues
past its repair to Ready, in the adapter's own framing:

- The template's worker bootstraps the guest for every clone's root
  before its capture, with the template's own root configuration, the
  default one: its cgroups and their resource values, which the
  snapshot holds. A bootstrap that fails ends the template with 13.
- Before the capture, the worker also takes the reset with which the
  broker answers its attachment, and acknowledges it, so that the
  snapshot holds the session active. A clone then reads no reset but
  the one that the restore queues: the restore's broker would finish a
  reset that the template's broker began to send, and a second reset
  would fail the clone's activation.
- Once it repaired a clone, the worker resumes that session on the same
  tty, without attaching again, acknowledges the reset that the restore
  queued, and runs the clone's activation,
  nvx::guest::clone::Activation, whose deadline ends 120 seconds after
  the repair's acknowledgement.
- The framing has a new message, Activate, of kind 27, whose body is a
  clone's activation record, at most nvx::activation::MAX_RECORD_BYTES.
  The worker applies the accepted record with nvx::guest::clone::apply,
  with the inputs of the root's assembly that the template's
  configuration gives, and answers Ready, which now reports the
  activation after the build, as ReadyActivation encodes it.
- The record's workload identity, if it has one, becomes the sandbox's
  identity, as a root configuration's does, so that executions name
  none. The worker then serves the host's order as after a cold boot.

Until the clone is ready, its session takes nothing but the Activate.
Each failure of the activation ends the clone with the status of its
class, 50 to 58, which worker::activation::failure_status selects, and
a broker that closes the stream before the clone is ready ends it with
status 1. Either way, the worker waits for its stream, and then its
tty, to take what it wrote for at most a second each, and keeps that
status whatever the stream does, so that a stream that takes nothing,
or fails, neither keeps the clone from ending nor changes its status.
A retry of the same Activate gets the same Ready. The template's
status 31, which ended a repaired clone at once, is gone, and a VM that
boots cold refuses an Activate. Unknown kinds in the framing's tests
move from 27 to 28.

nvx-rs gains Activation::with_resources, which builds an activation
from the resources and the acknowledgement's instant that a repair
reports, for the adapter's session tests, which run without a restore.
The documentation of status 51 now covers an Activate that changes the
record once the clone is ready.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925
An Activate whose header announces more than MAX_ACTIVATE_BODY bytes, the
most that a record has, failed the framing, which ended a clone's session
with status 1, although nvx::protocol::status gives a record that breaks a
limit ACTIVATION_RECORD (52).

- nvx::guest::clone::ActivationInput gets TooLong(length): an Activate whose
  record the adapter's framing refused unread. The activation takes it as an
  Activate whose record has that many bytes, which ActivationRecord::decode
  refuses with ActivationError::TooLong: in the transport bootstrap, as any
  Activate, with ActivationCause::Request, and from then on with
  ActivationCause::Record, whether the clone is ready or not, after the
  deadline's check, as every input.
- The test adapter's worker hands such a refusal of a clone's framing, which
  still comes before the worker takes any byte of the body, to the clone's
  activation, and ends with the failure's status, through the bounded drain
  of UNACTIVATED_DRAIN, as for any failed activation. The worker of a VM that
  boots cold still ends its session on it.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925
A clone's adapter applies the accepted record synchronously, and the root's
assembly waited for each of its devices as long as the template's timeout
allows, up to 600 s each in the test adapter, while the activation's deadline,
120 s there, passed: Activation::complete then refused the late application,
but the clone failed only once the waits ended.

- nvx::guest::clone::apply takes the activation's deadline, until which the
  assembly waits for each device at the latest, through the crate's
  root::assemble_until; root::assemble and root::Assembly are unchanged. A
  device that has not appeared by then fails the assembly as at its timeout,
  so the application ends by the deadline, past which complete fails the
  activation as expired, whatever the result.
- The activation's example passes activation.deadline(), and its agent queues
  its acknowledgement and Ready, so that a host that reads nothing holds up
  neither the agent nor the deadline.
- The test adapter's RootOperations::apply takes the deadline, which a clone's
  worker passes from its activation.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925
An assembly with a deadline capped each device's wait at the time left
when the wait began, then polled a fixed number of times, so a slow poll
or a sleep that overran could carry it past the deadline. The polling
now takes the deadline itself: it sleeps no longer than until the
deadline, and polls no more once the deadline passed, however long a
poll or a sleep took.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925
A clone's worker completed the activation as soon as the application of
the Activate's record returned, and answered Ready before it took what
the host had sent meanwhile: a reset or a request that came before
Ready was taken once the clone was ready, as a later session's reset or
as a request in the clone's order. The worker now keeps the
application's result until its session took everything that it
received, which it passes the activation first: a retry of the
Activate, byte for byte, joins the application, and Ready answers it
too, and anything else, a reset included, fails the activation, as
before the application. A host therefore waits for Ready before it
sends the mapping table or the start.

The example of nvx's Activation does the same, and the tests' fake
root holds a clone's application at its gate, so that the host's input
comes while the application runs.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925
When a clone's deadline came before the end of an assembly's timeout,
the polling for a device counted the polls that whole intervals before
the deadline allowed, and dropped the last part of an interval: with
less than an interval left, it looked once, and failed the assembly,
before the deadline, with the root's status. The polling now rounds the
count up when the deadline comes first, so that it looks once more, at
the deadline, after a shorter sleep, and a device that is still missing
then fails the activation with the deadline's status.

Part of #843

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8a2f342b-b46d-4a26-a11e-a996e3c1e925

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants