Repository navigation
Track sandbox network option bugs: provision accepts egress rules without --network-egress and loopback forwards without --host-loopback allow #756
Description
Tracking issue for confirmed bugs in how nvx.py sandbox and nvx-rs sandbox check their network options. Found against origin/dev at 2817f4eb; every link below is pinned to that commit.
1. provision records network options that OpenVMM refuses, so every start fails
Severity: medium. A sandbox provisioned this way can never start: every start fails with only OpenVMM exited during startup with status 2, and recovery takes deprovision and a new provision. No data loss and no security impact.
Where
runandprovisiontake their egress rules fromresolve_network_egress_rules, called for both operations. It requires--network-egressonly with--network-egress-policy-file(L489-L493) and returns--network-egress-allowand--network-egress-denyunchecked (L477-L482).SandboxOptions::checkpairs only--netwith--network-profile. Nothing checks--host-loopback-forwardagainst--host-loopback.extend_network_argumentspasses the options through,provisionrecords them inconfig.json, andextend_recorded_network_argumentsreplays them at everystart.- The library's own lowering already refuses forwards without host-loopback access (
host/network/mod.rsL304-L308), and the repository's fake OpenVMM mirrors both OpenVMM checks (fake_openvmm.rsL559-L569), but the sandbox CLI applies neither.
Trigger
sandbox provision (or sandbox run) with either:
--network-egress-allowor--network-egress-denybut no--network-egress, or--host-loopback-forwardwithout--host-loopback allow.
Expected
doc/usage.md documents both requirements for sandbox: --network-egress-allow and --network-egress-deny each "Requires explicit --network-egress", and forwards need --host-loopback allow. The pinned OpenVMM enforces both (microsoft/openvmm@9f04d05c cli_args/microvm.rs L1236-L1240 and L1259-L1264). NVX should refuse both combinations before provision writes any state and before run launches, as it already does for --network-egress-policy-file, and as the fix for #733 did for resource limits that the guest cannot apply.
Actual
provision exits 0 and records the invalid options. Every later start launches OpenVMM, which exits with status 2. The cause appears only in openvmm.log. sandbox run --dry-run with the same options also succeeds and prints an OpenVMM command that OpenVMM refuses.
Reproduction
Windows, nvx-rs built from 2817f4eb (cargo test --no-run --all-features). ROOT is a scratch checkout layout holding SOURCE-MANIFEST.json, placeholder build/vmlinux, build/initramfs.cpio.gz, and image files, with the repository's nvx-fake-openvmm.exe, which mirrors OpenVMM's checks, installed as openvmm/target/release/openvmm.exe. nvx.py sandbox forwards the same options to nvx-rs unchanged.
> nvx-rs --root ROOT sandbox provision --state-dir ROOT\state --layer distro,ROOT\img\distro.erofs,11111111-1111-1111-1111-111111111111 --scratch ROOT\img\scratch.ext4 --net 10.0.0.2/24 --network-profile portable --network-egress-allow 192.0.2.1:tcp:443
provision exit=0
config.json: "network_egress": null, "network_egress_allow": ["192.0.2.1:tcp:443"]
> nvx-rs --root ROOT sandbox start --state-dir ROOT\state --timeout 10
error: OpenVMM exited during startup with status 2
start exit=1
openvmm.log: openvmm: --network-egress is required with egress rules
> nvx-rs --root ROOT sandbox start --state-dir ROOT\state --timeout 10
error: OpenVMM exited during startup with status 2
start exit=1
> nvx-rs --root ROOT sandbox provision --state-dir ROOT\state2 --layer ... --scratch ... --net 10.0.0.2/24 --network-profile portable --host-loopback-forward tcp:18080:8080
provision exit=0
> nvx-rs --root ROOT sandbox start --state-dir ROOT\state2 --timeout 10
error: OpenVMM exited during startup with status 2
start exit=1
openvmm.log: openvmm: --host-loopback-forward requires explicit --host-loopback allow
Suggested fix
For run and provision, refuse non-empty --network-egress-allow or --network-egress-deny without --network-egress, using OpenVMM's message, in resolve_network_egress_rules next to the policy-file check. Also refuse --host-loopback-forward unless --host-loopback allow is given. Validating through host::network::openvmm_arguments would let the CLI and the library share one set of checks. Add nvx-rs/tests/cli.rs cases that provision with each combination and assert the refusal and that no state directory was written, and a run --dry-run case.
Explored
- Sandbox plan persistence (
host/sandbox/plan/form.rs, e4214a0): 28 Windows path forms (UNC,\\?\, drive-relative, root-relative, trailing separators) round-trip for the distro, scratch, and share paths; the UUID, limits, and unknown-field rules hold. nvx::uuid(0a7ac27) and--layerUUID normalization, which ports Python'suuid.UUID.- The release lookup retry of release: retry uploads that gh reports as "release not found" #716 in
scripts/nvx_tools/development_release.py. - CI change classification (
scripts/nvx_tools/ci.py, 9ad0468): no-renames diff, fallbacks, output checks. The guest-binary source identity and its dep-info guard (ci: key the guest images and VMM tests on the guest binaries' sources #710). - Windows-specific host code: sandbox store ACL privacy checks, file seals, process liveness and start times, CreateProcess argument quoting, the sandbox lifecycle lock's error classes.
- The
nvx.pytonvx-rsoption seam (_SandboxOption, the argparse emulation inhost/cli/args.rs), the Pythonint,float,list2cmdline, andshlexemulations, the timeout and memory options. scripts/check_markdown_links.pyanchors and links, compared against GitHub's own renderer on 29 headings and 18 link forms. No difference.- The host control session and NVXS connection, and
aci_edge_sandboxes.
Checked, not filed
sandbox startdeletes the previousoutcome.jsonbefore it checksconfig.json, the distro binding, or the artifacts (lifecycle.rsL480-L482), so a start refused before launch loses the last VM-level report. Reproducible, but inherited from the Python_start, consistent with the nvx-rs README's "report of the last start", and low impact.gh release viewreports transient draft-lookup failures asrelease not found(release: retry development release uploads that gh reports as "release not found" while the draft exists #686), which the observations aftercreateanduploadindevelopment_release.pytreat as final. Already tracked by release: address the development release by its ID instead of its tag #688.HostConnection::sendends the connection whenwrite_alltimes out, even if nothing was written (nvxs.rsL259-L271), whileHostConnectionError::TimedOutsays that such a send leaves it usable. Ending is required, sincewrite_allcannot report a partial write and the session already advanced its sequence, so only the wording is off.ControlSessionand the CLISessionstay open after a reply with a mismatched request ID. Their docs promise to end only after stream failures, and each CLI command exits on its first error.--network-proxyvalues are not checked by the CLI either. Unconfirmed: not reproduced against OpenVMM's checks.
Validation limits
- Reproduced on Windows x64 with
nvx-rsbuilt from2817f4eband the repository's fake OpenVMM. The real OpenVMM was not built or run in this session; its source at the pinned revision has the identical checks linked above. - Linux (KVM or MSHV) was not run. The defect is in platform-independent CLI option handling.
Activity
akhil9tiet commentedon Oct 22, 2026
I'd like to work on this — please assign it to me.
ppenna commentedon Oct 29, 2026
Fixed by #753 (b4d76e32a, nvx-rs: lower the sandbox network options through the typed policy), which merged on 2026-10-23, after this issue was filed. That PR didn't reference this issue, which is why #781 was closed as superseded.
sandbox runandsandbox provisionnow read their network options into one typed policy. They check it before anything launches and beforeprovisionrecords anything:sandbox_networkrefuses--network-egress-allowor--network-egress-denywithout--network-egress.openvmm_argumentsrefuses forwards without--host-loopback allow. The CLI and the library share this lowering, as the issue suggested.startchecks the recorded typed network again. It refuses aconfig.jsonthat recorded the raw options, as releases before nvx-rs: lower sandbox networks through one typed policy with a proxy-only mode #753 did, withsandbox configuration has no network policy; deprovision the sandbox and provision it again.network_policies_that_the_planner_refuses_fail_before_launchcovers both combinations forrun --dry-runandprovision, and asserts that no state directory is written.Verification with the real OpenVMM
I ran the same script on three test VMs:
nvx-kvm-test(linux-kvm),nvx-mshv-test(linux-mshv), andnvx-whp-test(windows-whp). It usednvx-rsbuilt fromdevat d89d370 and from this issue's 2817f4e. OpenVMM came from the development releases v0.1.0-dev.8afc615d86b4 (KVM) and v0.1.0-dev.320666c6c096 (MSHV, WHP), both built from OpenVMM bd2b27a2. Its network option checks are identical to those of the current pin, c32c57d1, which changed only CPU profile code. All three platforms gave the same results:nvx-rssandbox provision,sandbox run --dry-run, andsandbox run, with--network-egress-allow 192.0.2.1:tcp:443(orprovisionwith--network-egress-deny 198.51.100.0/24), without--network-egresserror[unsupported]: --network-egress is required with --network-egress-allow or --network-egress-deny. No state directory and no OpenVMM process.--host-loopback-forward tcp:18080:8080and either no--host-loopbackor--host-loopback denyerror[unsupported]: host-loopback forwards need host-loopback access allowed. No state directory and no OpenVMM process.provision,start,exec,stop, anddeprovision, once with--network-egress deny --network-egress-allow 192.0.2.1:tcp:443and once with--host-loopback allow --host-loopback-forward tcp:18080:8080--network-egress-allow 192.0.2.1/32:tcp:443,--host-loopback allow --host-loopback-forward tcp:18080:8080), the workload seeseth0, and OpenVMM listens on127.0.0.1:18080for the forward.provisionwith each combination, thenstarttwiceprovisionexits 0 and records the options. Both starts fail withOpenVMM exited during startup: vmm-failure status 1, andopenvmm.logholds OpenVMM'sfatal error: --network-egress is required with --network-egress-allow or --network-egress-denyorfatal error: --host-loopback-forward requires explicit --host-loopback allow.So with the real OpenVMM the bug reproduces at 2817f4e, where OpenVMM exits with status 1 rather than the fake's 2, and is fixed at d89d370. At the current tip, 29c6854, the checks are unchanged: #978 refactored the lowering but kept the forward check. The sandbox network CLI tests above also pass there on Windows.
Outside the scope of this issue,
nvx-rs runstill passes these options to OpenVMM as given, as its dry-run goldens expect. OpenVMM refuses them at once and prints its message on the terminal, and nothing is recorded.Closing as fixed by #753.