[Enrique Saurez (esaurez)]

Chain: #951 → #954 → #957 → #959 → #960 → #955 → #958 → #962 → #963 → #925 → #926 → this PR, on dev; #879, #881, #885, #886, #877, #932, #884, #887, #933, #880, #936, #937, #939, #940, #941, #888, #949, and #950 have merged.

Stacked on #926.

Fixes #850.

What and why

Until now, a template's NIC carried the policy of its clones:

• The template's key held that policy, as the network's lowering to OpenVMM's arguments, and every restore passed it unchanged. A host therefore kept one template per network policy.
• The capture refused ingress and host-loopback access both ways, which publish the guest's listeners. No clone could have ingress, host-loopback access to its listeners, or forwards.

OpenVMM lets a platform restore install a new policy on a NIC captured under the complete deny-all policy (#536, in dev's OpenVMM pin). With this PR:

• A template's NIC is always captured under that policy.
• Each clone's restore installs the clone's own policy, including ingress, both-way host loopback, and forwards.
• One template serves the clones of every policy at its guest's address.
• A clone whose policy publishes its listeners publishes them as a cold boot does.

It is one PR because a clone that publishes its listeners is correct only together with the announcer that publishes them.

Host: nvx::host::template

• Capture:

  • SandboxNetwork::template_arguments lowers a network to the NIC that its template captures: --net at the guest's address, under the complete deny-all policy (--network-profile portable --network-egress deny --network-ingress deny --host-loopback deny), whatever the network's own policy.
  • A network that lets nothing through still attaches no NIC, so its template has none.
  • The capture no longer refuses ingress or both-way host loopback.

• Class and key:

  • The class takes only the NIC that the template captures. Two networks at one guest's address whose policies both attach a NIC therefore share one class.
  • The key keeps, in its machine, the NIC's addresses and the identity of the deny-all policy that OpenVMM enforced at the capture. It no longer holds the network's lowering: version 1's eighth part, which held it, is gone. Version 1 has not been released, so there is no migration.
  • A key that holds such a part after its invariants is refused as malformed, so a store of such keys is never misread.
  • The vectors follow: template-key-v1.txt changes in place.

• Clone plan: ClonePlan::new requires the clone's network to be at the guest's address of the template's NIC, under a policy of its own. Before any VMM starts, it refuses, with the failure request, invalid-settings, plan:

  • a network at another address;
  • no network, or one that lets nothing through, for a template with a NIC;
  • a network with a NIC for a template without one.

• Restore: the restore's device arguments replace --net and the guest's address with --restore-network-policy, followed by the options of the clone's policy. The clone's policy therefore replaces the template's deny-all policy on the NIC before any restored processor runs.

• Activation record: the record names the clone's own network:

  • the identity of its policy (nvx-rs: a canonical activation record for platform clones #838's activation::Identity);
  • the DNS server that OpenVMM names to the guest under that policy (network::dns_server, OpenVMM's rule: the first gateway whose DNS service the egress policy lets the guest reach);
  • whether the policy publishes the guest's listeners: ingress allowed, or host-loopback access both ways;
  • the proxy, if any.

  Under the deny-all policy, OpenVMM writes neither virtnet_dns= nor virtnet_ingress= on the template's command line. A clone's guest therefore learns its DNS server and publication from the record alone.

• Busy forwards:

  • A clone's VMM may exit before its restore-ready event with OpenVMM's cause failed to create socket for port forward (Tcp 127.0.0.1:PORT). The clone then fails as the host's: outcome host, evidence forward-bind, at the restore.
  • The match runs only after such an exit, so a successful start pays nothing: no pre-bind and no added system call per forward.
  • The excerpt nvx-rs/tests/vectors/openvmm/port_forward_socket_creation.rs pins the text, and openvmm_vectors checks it against the pinned OpenVMM.

Guest: nvx::guest::clone and nvx::guest::ingress

• clone::apply starts the announcer of the workload's listeners when the record publishes them (ingress::Announcer):
  • As root, the library creates an IPv4 UDP socket and binds it to the highest free privileged port, from 1023 down to 512.
  • It spawns /sbin/nvx-ingress inherit GATEWAY with that socket as its standard input, and waits only until the launch report says that the program runs.
  • Nothing before Ready waits on a timer or polls the child, and clones without publication pay nothing.
• The gateway is the address of OpenVMM's virtnet_gw= token, which protocol::cmdline::gateway reads as strictly as virtnet_dns=.
• nvx-ingress inherit GATEWAY is a new mode. It takes its socket from its standard input and refuses anything but an IPv4 datagram socket bound to a privileged port. The cold-boot form, nvx-ingress GATEWAY, is unchanged.
• Every start failure is synchronous and comes before Ready: no gateway, the socket, the bind, or the spawn. Each is an ApplyError::Network (NetworkError::Gateway or NetworkError::Announcer), status 57.
• Applied keeps the announcer and its pidfd. The teardown stops it first, as a drop does.

Runtime adapter

• The guest's template mode reads the gateway from the kernel command line and refuses one that OpenVMM would not render (status 20). Its root activation passes the gateway to nvx::guest::clone::apply.
• The worker polls the announcer's pidfd in its loop. An announcer that ends, at any time, ends the clone with status 57: "the announcer of the workload's listeners ended: …".
• A clone's executions enter the root with its record's publication. While its listeners are published, Entry::check therefore refuses an execution whose identity holds NET_BIND_SERVICE or NET_RAW.
• create takes a network of any policy, and the creations of networks with a NIC at one guest's address share one template. start takes a network at the template's guest's address, under any policy. The README describes templates and clones of every policy, the announcer, and statuses 20 and 57.

Tests

nvx-rs unit tests:

• network:
  • templates_attach_their_nic_under_the_complete_deny_all_policy;
  • the_dns_server_is_the_first_gateway_whose_dns_service_the_policy_reaches.
• template::plan: every capture now covers policies with ingress, both-way host loopback, and a forward, each attaching its NIC under the deny-all policy.
• template::key: a_key_that_holds_its_network_s_lowering_after_its_invariants_is_refused. It replaces the tests of a key that holds its network or names a DNS server.
• template::clone:
  • templates_serve_the_networks_of_every_policy_at_their_nics_address_alone;
  • the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_own_network;
  • a_clone_of_a_template_with_a_nic_restores_its_own_policy_in_place_of_the_templates;
  • openvmm_s_fatal_error_is_evidence_of_a_forward_bind_when_it_names_a_forward_it_cannot_bind;
  • the_cause_that_names_a_forward_that_openvmm_cannot_bind_follows_the_fatal_line.
• guest::ingress:
  • the announcer: it runs with its socket until stopped, polls readable once it ends and keeps its status, ends when dropped, does not start when its program does not run, and names each start error's cause;
  • an_inherited_socket_is_taken_only_at_a_privileged_source_port;
  • guest::sys::socket: an_inherited_descriptor_is_taken_only_as_an_ipv4_datagram_socket.
• guest::clone::activation::apply: a_network_that_publishes_the_workload_s_listeners_starts_their_announcer_to_its_gateway. The order, teardown, and failure tests now include the announcer: it starts after the resolver file and before the binds, and a start failure, or a missing gateway, fails the activation with the network's error.
• protocol::cmdline: the_gateway_is_one_ipv4_address_in_one_token_as_openvmm_renders_it.
• guest_tools: ingress_inherits_only_an_ipv4_datagram_socket_bound_to_a_privileged_source_port.
• openvmm_vectors: the forward's excerpt at the pinned OpenVMM.

Runtime adapter, without VMs:

• template: template_mode_refuses_a_gateway_that_openvmm_would_not_render.
• guest_session:
  • a_clone_s_executions_enter_the_root_with_the_publication_of_its_record_s_network;
  • a_clone_whose_announcer_ends_ends_with_the_network_s_status.

Runtime adapter VM scenarios (scenarios.txt, now 154):

• vm_template the_networks_of_every_policy_at_one_guest_s_address_share_one_template:

  • The creation of a network with a forward captures a template whose NIC is under the deny-all policy.
  • The creations of the seven networks with a NIC then find the same class, key, and generation, and no VMM runs.
  • It replaces the interim rule's six rows: an_egress_network_has_its_nic, an_allow_rule_has_its_nic, outbound_host_loopback_has_its_nic, a_proxy_only_network_has_its_nic, ingress_is_refused_before_any_vm_starts, and host_loopback_both_ways_is_refused_before_any_vm_starts.

• vm_clone one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address replaces a_template_serves_the_clones_of_its_own_network_policy_alone. One template serves clones of seven policies: egress, an allow rule, outbound host loopback, a proxy, ingress, both-way host loopback, and a forward. For each clone, it checks:

  • the record's DNS server, proxy, and publication;
  • the VMM's --restore-network-policy arguments;
  • which host services the clone reaches at its gateway;
  • that the host reaches a guest listener at host loopback exactly under both-way host loopback, and through the forward;
  • that an execution holding NET_BIND_SERVICE or NET_RAW fails to launch exactly while the listeners are published.

  The records' policy identities all differ. Before any VMM starts, the plan refuses a network at another address, one that lets nothing through, and none.

• vm_clone a_forward_whose_port_another_socket_holds_fails_the_clone_as_the_host_s:

  • A host socket holds the forward's port, and the clone fails with host, forward-bind, restore, spawned, none, exited. It quarantines nothing, and the lease is released.
  • Once the port is free, the template serves the same network.

• The DNS and proxy scenarios now restore clones of other policies than their templates':

  • a proxy-only template serves an egress clone, which binds its policy's DNS server;
  • an egress template serves a proxy-only clone, whose executions find its proxy.

Replaced tests

The interim rule of one template per network policy and the key's network part go, so these base tests and rows go, each replaced:

• host::network: an_attached_lowering_attaches_the_device_of_a_policy_that_lets_nothing_through → templates_attach_their_nic_under_the_complete_deny_all_policy.
• template::key: keys_read_back_the_network_of_each_policy and a_key_holds_the_network_that_its_class_lowered, with the network checks of other_encodings_of_a_key_are_refused → a_key_that_holds_its_network_s_lowering_after_its_invariants_is_refused and the version-1 vectors.
• template::key: a_key_names_the_dns_server_that_its_command_line_names → network::the_dns_server_is_the_first_gateway_whose_dns_service_the_policy_reaches.
• template::clone:
  • networks_that_the_template_does_not_have_are_refused → templates_serve_the_networks_of_every_policy_at_their_nics_address_alone;
  • the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_policy_and_dns_server → the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_own_network;
  • a_clone_of_a_template_with_a_nic_restores_its_policy_without_its_address → a_clone_of_a_template_with_a_nic_restores_its_own_policy_in_place_of_the_templates.
• vm_template: an_egress_network_has_its_nic, an_allow_rule_has_its_nic, outbound_host_loopback_has_its_nic, a_proxy_only_network_has_its_nic, ingress_is_refused_before_any_vm_starts, and host_loopback_both_ways_is_refused_before_any_vm_starts → the_networks_of_every_policy_at_one_guest_s_address_share_one_template.
• vm_clone: a_template_serves_the_clones_of_its_own_network_policy_alone → one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address.

Guide changes (#591's numbering)

At 950c1fe, the guide already describes deny-all templates and new policies at restore. These refinements are not made in #591; they are listed here for it:

• 2.1.4 Restore-Time Configuration:
  • A host captures every template with a NIC under the complete deny-all policy, at its network's guest's address, so each such snapshot admits a new policy.
  • The restore names that policy with --restore-network-policy and its options, in place of --net and the address, which the snapshot fixes.
  • A clone's network at another address than the template's NIC is refused before any VMM starts.
• 3.5.4 Host-Local Access and Ingress:
  • In a platform clone, the guest agents start the announcer during activation, before readiness. As root, they bind the privileged source port and pass that socket to the announcer (nvx-ingress inherit GATEWAY). A cold boot is unchanged.
  • An announcer that cannot start fails the activation, and one that ends later ends the clone, both with status 57.
• 3.5.6 Bounded State, Snapshots, and Restore:
  • A template's key records the NIC's identity and the deny-all policy's digest, never a clone's policy. One template therefore serves the clones of every policy at its guest's address.
  • A template whose network lets nothing through has no NIC and serves only clones without one.
  • A forward whose host port another socket holds fails the restore before the clone is ready, as a host fault (forward-bind).
• 5.5.9 Platform-Clone Activation:
  • The record's ingress mode is whether the listeners are published: ingress allowed, or host-loopback access both ways.
  • Step 3's network settings include starting the announcer when the listeners are published.
  • The record's workload identity holds no capabilities. While the listeners are published, an execution whose own identity holds NET_BIND_SERVICE or NET_RAW is refused.
• The guide at 950c1fe holds no interim rule (one template per network policy), so there is nothing to remove from it.

Not in this PR

• OpenVMM: give a port forward that cannot be bound at restore a stable error code #972: a stable OpenVMM error code for a forward that cannot be bound at restore, so that NVX matches the code instead of the text.

Validation

At this PR's head dba01f0, the five commits of the earlier head 341dce9, restacked onto #926's head f87f1f1, which sits on #963 rebased onto dev at f8469b8. One commit needed a resolution: #1015 rewrote the module docs of nvx-rs/src/protocol/cmdline.rs for the tokens that the guest's init reads, and this PR adds the IPv4 gateway and virtnet_gw= to the same lines; the rebase keeps both. With that resolution, this PR and the one above it, merged into dev at 8f3a840, passed CI (run 40938392497). 341dce9 held the same five commits as the earlier head 1954de6; the rest of this section describes 1954de6, on #926's earlier head 2dd7ce6.

At the head, 1954de6, on #926's 2dd7ce6. The rebase adapted this PR's announcer tests to their new base: they pass its activation deadline as None, and send Start only after Ready, as its activation requires.

Windows, at each commit:

• nvx-rs: fmt; clippy with all features, and on the lib with and without default features; tests with all features; openvmm_vectors, ignored tests included, against the pin, 832d5bb4.
• Runtime adapter: fmt; clippy and tests in the protocol-only and host configurations; guest clippy for x86_64-unknown-linux-musl; the check of its scenario list.
• The time ABI tests, the README's module rows, and Markdown links.
• At the head, also:
  • nvx-rs: tests with default features and on the lib without them; docs with -D warnings; MSRV 1.89.
  • The adapter's MSRV in the protocol-only and host configurations.
  • The Python unit tests of validate-nvx, and the adversarial and host-connection tests.

Linux, on an MSHV host, at each commit:

• nvx-rs:
  • fmt;
  • clippy with all features, and on the lib without default features;
  • tests with all features, with default features, and on the lib without default features;
  • openvmm_vectors, ignored tests included;
  • docs with -D warnings;
  • the tests that need root.
• Runtime adapter: fmt; clippy and tests in the protocol-only, host, and musl guest configurations.
• At the head, also:
  • nvx-rs: MSRV and the macOS check.
  • The guest binaries: check, build, and tests.
  • The adapter's MSRV in each configuration.
  • The Python unit tests of validate-nvx, and the adversarial, host-connection, and Specula tests.

The adapter's VM suites at the head:

• They used the OpenVMM, kernel, and Ubuntu image of development release v0.1.0-dev.2d14547b669f: OpenVMM 832d5bb4, the pin. No commit after 2d14547, on dev or in this chain, changes the kernel's inputs or the image's.
• The guest initramfs and agent were built from the head, and both hosts used the same files.
• MSHV, as root: all 154 scenarios of 11 targets pass.
• WHP: all 154 scenarios of 11 targets pass.

CI: one manual full run of ci.yml on this branch at 1954de6, run 40866679824.

• 36 jobs passed, KVM, MSHV, and WHP included. Each backend's Guests and sandboxes job ran all 154 adapter scenarios.
• 6 were skipped by design: the performance gate and baseline, the development release, the nvx crate's packaging and publication, and the guest-artifact reuse rehearsal.

Review:

• Before the restack:
  • gpt-6-astra, three rounds. The first two found text that said an announcer that ended leaves the workload's listeners unpublished. OpenVMM's gateway withdraws a publication only when a later announcement omits it, so the clone must end, with status 57. Fix-ups, since squashed, corrected that text. The third round was clean.
  • Copilot: three rounds, with no comments.
• The restack:
  • gpt-6-astra: two rounds on its delta (the conflict resolutions, the ports to the new base, and the test changes above), both clean.
  • Copilot at 1954de6: no comments.

[Copilot]

🔵 Needs a closer look

The changes cross the host/guest isolation boundary and add a privileged process, requiring human confirmation of fail-closed restoration and lifecycle behavior across KVM, MSHV, and WHP, whose suite remains pending.

No open issues 🎉

What changed in this PR

This PR enables NVX clones to share a deny-all network template while applying their own policies at restore, addressing #850 using the OpenVMM support from #931. It also adds guest listener announcement and supervision for policies that publish listeners.

Changes:

• Captures network templates under complete deny-all policies and validates clones by NIC address rather than policy.
• Applies each clone’s network policy, DNS, proxy, and listener-publication settings, with announcer lifecycle management.
• Expands tests and vectors for shared templates, inherited sockets, and forward-bind failure attribution.

File	Description
tests/​runtime-adapter/​tests/​vm_template.rs	Tests template sharing across network policies and deny-all capture.
tests/​runtime-adapter/​src/​worker/​capture.rs	Passes the template gateway into clone activation.
tests/​runtime-adapter/​src/​worker/​activation.rs	Applies record publication settings and covers network failures.
tests/​runtime-adapter/​src/​worker.rs	Supervises the announcer and fails clones when it exits.
tests/​runtime-adapter/​src/​template.rs	Parses and validates template gateway tokens.
tests/​runtime-adapter/​src/​clone.rs	Documents per-clone policies at the template’s address.
tests/​runtime-adapter/​src/​capture.rs	Documents policy-independent capture classes.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-host.rs	Updates template and clone CLI documentation.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-guest.rs	Forwards the parsed gateway to template capture.
tests/​runtime-adapter/​scenarios.txt	Registers shared-policy and busy-forward scenarios.
nvx-rs/​tests/​vectors/​template-key-v1.txt	Updates the network class vector for deny-all capture.
nvx-rs/​tests/​vectors/​README.md	Documents revised key vectors and the forward-error excerpt.
nvx-rs/​tests/​vectors/​openvmm/​port_forward_socket_creation.rs	Pins OpenVMM’s socket-creation error text.
nvx-rs/​tests/​openvmm_vectors.rs	Checks the new excerpt against OpenVMM.
nvx-rs/​tests/​guest_tools.rs	Tests inherited-socket validation and ingress usage errors.
nvx-rs/​src/​protocol/​status.rs	Documents announcer failures under network status 57.
nvx-rs/​src/​protocol/​cmdline/​tests.rs	Tests strict gateway parsing.
nvx-rs/​src/​protocol/​cmdline.rs	Adds gateway parsing through shared address-token logic.
nvx-rs/​src/​host/​template/​store/​tests.rs	Updates unsupported key-version coverage.
nvx-rs/​src/​host/​template/​store/​capture.rs	Compares capture networks by their template NIC arguments.
nvx-rs/​src/​host/​template/​plan/​tests.rs	Tests deny-all captures across supported policies.
nvx-rs/​src/​host/​template/​plan.rs	Separates template NIC arguments and removes policy restrictions.
nvx-rs/​src/​host/​template/​mod.rs	Updates compatibility and encoding documentation.
nvx-rs/​src/​host/​template/​key.rs	Removes clone policy arguments from keys and exposes NIC addressing.
nvx-rs/​src/​host/​template/​clone/​tests/​restore.rs	Tests forward-bind failure attribution during restore.
nvx-rs/​src/​host/​template/​clone/​tests/​launch.rs	Tests replacement-policy arguments and simulates forward failures.
nvx-rs/​src/​host/​template/​clone/​tests.rs	Tests address compatibility and clone-specific activation records.
nvx-rs/​src/​host/​template/​clone/​records.rs	Retains forward-bind causes from OpenVMM’s fatal error chain.
nvx-rs/​src/​host/​template/​clone/​record.rs	Builds network activation settings from the clone’s policy.
nvx-rs/​src/​host/​template/​clone/​launch.rs	Emits --restore-network-policy for networked clones.
nvx-rs/​src/​host/​template/​clone/​failure.rs	Attributes forward socket-creation failures to the host.
nvx-rs/​src/​host/​template/​clone.rs	Validates NIC compatibility and retains each clone’s network.
nvx-rs/​src/​host/​sandbox/​plan.rs	Adds template NIC lowering and policy-based DNS selection.
nvx-rs/​src/​host/​network/​mod.rs	Adds deny-all capture arguments, publication detection, and DNS selection.
nvx-rs/​src/​guest/​sys/​socket.rs	Validates inherited IPv4 datagram sockets and exposes bound ports.
nvx-rs/​src/​guest/​mod.rs	Documents the guest announcer API.
nvx-rs/​src/​guest/​ingress/​sender.rs	Supports sending announcements through an inherited socket.
nvx-rs/​src/​guest/​ingress/​mod.rs	Adds inherit-mode dispatch and shared privileged-port binding.
nvx-rs/​src/​guest/​ingress/​announcer.rs	Implements announcer startup, supervision handles, and cleanup.
nvx-rs/​src/​guest/​clone.rs	Documents announcer activation, ownership, and teardown.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Changes to host/guest network isolation and privileged listener publication require maintainer confirmation of restore-policy enforcement and cross-backend VM validation, which was unavailable in the source-only review.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Cross-layer network isolation and privileged listener publication require maintainer confirmation of OpenVMM’s pre-vCPU enforcement contract and KVM/MSHV/WHP integration evidence.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The changes cross host/VMM/guest isolation and privileged listener-publication boundaries, requiring maintainer confirmation that the pending KVM, MSHV, and WHP VM scenarios pass on the exact PR head.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Human review is needed to confirm that per-clone policy replacement preserves isolation across supported backends when OpenVMM’s pre-vCPU enforcement depends on privileged guest announcements.

No open issues 🎉

_{Review effort: Balanced}