Repository navigation
tests: run the runtime adapter's executions in the root, under their identity - #868
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
Privileged namespace entry and credential switching warrant human review because the added execution tests bypass actual root entry.
No open issues 🎉
What changed in this PR
This PR advances #805 by running runtime-adapter executions in the assembled root under a host-selected identity, building on the preceding cgroup work.
Changes:
- Extends execution starts with identity, working-directory, and environment fields.
- Adds sandbox-wide identity validation and connects execution spawning to
nvx::guest::entry. - Adds encoding and fake-root tests, and documents the execution contract; real-root VM coverage remains planned for PR 3.
| File | Description |
|---|---|
tests/runtime-adapter/tests/guest_session.rs |
Tests identity selection, entry inputs, refusals, working directories, and environments using fake roots. |
tests/runtime-adapter/src/worker/executions/tests.rs |
Adapts execution tests to the entry-operations interface. |
tests/runtime-adapter/src/worker/executions.rs |
Prepares identities and passes entry, directory, and environment settings to spawning. |
tests/runtime-adapter/src/worker.rs |
Adds entry operations, hostname handling, and sandbox identity validation. |
tests/runtime-adapter/src/root.rs |
Adds workload identity configuration and the identity failure stage. |
tests/runtime-adapter/src/exec/tests.rs |
Tests the extended start encoding and malformed-input refusals. |
tests/runtime-adapter/src/exec.rs |
Defines and encodes execution identity, directory, and environment fields. |
tests/runtime-adapter/README.md |
Documents the new configuration, execution behavior, and test scope. |
Review effort: Balanced
|
On the review's note that real root entry deserves a closer look: the in-process tests here can't enter a root, since that needs privileges, so #870 (PR 3 of #805, stacked on this one) adds VM scenarios that enter real roots through this PR's worker:
They run in that branch's tree, which doesn't change the worker. On WHP all 25 layered scenarios pass, and on MSHV all the new ones do; CI will run them on KVM, MSHV, and WHP once the stack reaches |
a40048e to
20d5cdc
Compare
916f3d0 to
e487ea4
Compare
20d5cdc to
5a071a5
Compare
5a071a5 to
ec3af7d
Compare
e487ea4 to
654b1da
Compare
ec3af7d to
a8cc6e2
Compare
654b1da to
ca51780
Compare
ca51780 to
2f3935d
Compare
The worker ran each program in a process group of its own, which it killed before it collected the program's status. Nothing proved that a descendant that left the group was gone, so every outcome was fenced, and the worker could admit a new execution while such a descendant still ran. Each execution now gets a cgroup of its own below the workload cgroup, which the worker prepares with ExecutionCgroup::prepare, and which the program joins as it spawns. The execution settles through that cgroup alone, so the other executions keep running, and an execution whose program ends on its own is reusable. An execution keeps its place among the worker's limit until its cgroup is empty and removed. One that does not settle keeps its place, and its cgroup, until the host stops the worker, which ends and removes every cgroup that is left. The root's operations name the cgroups that the bootstrap prepared. A fake root names none where the in-process tests cannot create cgroups, and the programs keep their process groups there, as before. Where the tests can, each fake root has a workload cgroup of its own, and new tests check a descendant in a session of its own, which the settlement ends, and an execution that does not settle, which keeps its place until the stop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
A program that the worker spawned, but whose execution it could not have, may still be joining its cgroup, as its launch has yet to report, and removing the cgroup could then hide the program once it ended, which ExecutionCgroup::remove forbids. Such a start's cgroup now keeps its place, as that of an execution that did not settle, and a stop removes the executions' cgroups only once the worker collected the status of every program that it forgot. The holder in the test of an execution that does not settle now runs in a process group of its own, which its drop kills, so that its child no longer outlives a test that fails before the child joins the cgroup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
Once the in-process tests create cgroups, each fake root holds a workload cgroup of the test's own, which ends what is left in it and removes the cgroups of executions when it drops. Two tests discarded the fake root that a stop returns, so that it dropped before their checks: that the stop removed the cgroup of an execution that did not settle, and that it left no process of a program's group running. Those checks then held whatever the worker did. The tests now keep the fake root until their checks are done. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
…identity The worker ran each program in its own root and working directory, as its own identity, with its own environment. A start now names the program's identity, working directory and environment, and the root configuration may name a workload identity for the whole sandbox, whose account the worker checks in the assembled root before it answers: a missing account fails the new identity stage, with status 16. Exactly one of them names each execution's identity, as the activation record's optional identity will, so a start that breaks that rule is refused at stage 0. Each program then enters the assembled root through nvx::guest::entry, in new namespaces, with the hostname that the kernel command line configures, as NVX's init agent reads it, the image's own machine ID, no time status, and published listeners. The entry checks the identity, whose refusals the worker reports at stage 0, and reads its account. The fake root of the in-process tests prepares each entry in a directory of the tests' own, with account databases, and spawns the program without entering it, which needs privileges. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
…name The worker took the executions' hostname from the kernel command line's nvx_hostname= parameter. That parameter belongs to NVX's own boot contract, not to the runtime adapter: an adapter's launch need not carry it, and the adapter configures everything else through its root configuration. The executions now take nvx::guest::entry's default hostname, nvx-sandbox, which the worker's Sandbox::new names in one place, so that a sandbox's own hostname can later be set there. The worker no longer reads the parameter, and its layout stage no longer refuses a long one. No root-configuration field, frame field, or host option names a hostname. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
The worker exits with 16 once the root's identity stage fails, as its stages' statuses now run from 11 for the configuration to 16 for the identity, but the guest agent's usage still gave them as 11 to 15. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a7ad408a-da3b-45c4-aa02-5d25300b1c90
2f3935d to
6fca426
Compare
Stacked on #862 (PR 1 of #805), which is stacked on #878 (PR 8 of #822), whose other PRs have merged into
dev: #863 → #867 → #869 → #865 → #864 → #872 → #875. This PR retargets todevonce #878 and #862 merge.Part of #805: PR 2 of 3, executions in the assembled root, under their identity.
What changed
The worker ran each program in its own root and working directory, as its own identity, with its own environment.
ExecStart(exec::Start) gains, after the arguments: an optional identity,exec::Identity, with the UID, the GID, the supplementary groups, the capabilities (au64, bit N for capability N) and the account mode,exec::AccountMode(existing or numeric); an optional working directory, an absolute path in the root (otherwise/); and the program's whole environment,NAME=VALUEentries. A body that breaks these rules (a flag or account mode other than 0 or 1, a relative working directory, an entry that is notNAME=VALUE, a NUL byte) does not decode, as before for the arguments.RootConfig::workload_identity, the linesworkload-uidandworkload-gid, which follow runtime adapter: run cold sandboxes end to end on KVM, MSHV and WHP #822'smapping-count, the last of its 14 fields: both or neither, each between 1 and 4,294,967,294.CONFIG_VERSIONstays 1. Once the root is assembled and the host export mounted, the worker checks its account there, in existing-account mode, before it answers the configuration withReady; it runs executions only once the host started the sandbox, as runtime adapter: run cold sandboxes end to end on KVM, MSHV and WHP #822's order says. A missing account fails the new root stageidentity, code 6, with status 16.EINVAL. This mirrors the optional identity of the activation record (nvx-rs: a canonical activation record and configuration identity for platform clones #838).nvx::guest::entry, as aworker::Entrancedescribes it: new mount, PID and UTS namespaces, with the root moved over/; the entry's default hostname,nvx-sandbox, which the worker'sSandbox::newnames in one place, as nothing configures a hostname (no root-configuration field, frame field, host option, or kernel command-line parameter); the image's own machine ID and no time status, soEntry::bind_runtimebinds nothing; and published listeners, the entry's default, until runtime adapter: run cold sandboxes end to end on KVM, MSHV and WHP #822's PR 10 says whether listeners are published.Entry::preparechecks each identity and reads its account before the worker prepares the execution's cgroup. Its refusals are stage-0 failures:EPERMfor a capability outside the allowlist, any capability of a UID other than 0, andCAP_NET_BIND_SERVICEandCAP_NET_RAWwhile listeners are published;ENOENTfor a missing passwd or group entry; the database's error if it cannot be read;EINVALotherwise. Refusals take no place amongworker::MAX_EXECUTIONS.worker::EntryOperations(prepareandspawn), a supertrait ofRootOperations, lets the in-process tests' fake root prepare each entry in a directory of the tests' own, with account databases, and spawn the program without entering a root, which needs privileges.Tests
u32::MAX, one without the other); the identity stage's code, name and status.tests/guest_session.rs, with a fake root:EINVAL), a capability outside the allowlist, a capability of a UID other than 0,CAP_NET_BIND_SERVICEandCAP_NET_RAW(EPERM), a missing passwd entry and a missing group entry (ENOENT), another primary group and an invalid UID (EINVAL); after them, the worker still runs its limit of executions.The existing execution tests now run in a sandbox with a workload identity, and #822's tests of the worker's order and mapping table pass as they are.
Validation
This branch is rebased onto #862's current head (a8cc6e2), which is on #878's current head (47f8bcd), the maintainer's re-commit of #878 on
devafter #875, #907, #908 and #911 merged; the OpenVMM gitlink stays 2b681564. The rebase had no conflicts, andgit range-diffshows the three commits' patches unchanged (8cb6359 → 65ef123, 526c263 → 5c8682f, 654b1da → ca51780). The third commit corrects the guest agent's usage, which still gave the worker's statuses for a failed stage of the root as 11 to 15, without the identity stage's 16.check-nvx-rsin its three configurations, with rustfmt, Clippy with warnings denied, the build, the tests, andcargo +1.89 check(the MSRV): protocol-only, host-only, and the guest-onlyx86_64-unknown-linux-muslconfiguration; andscripts/check_markdown_links.py;guest_sessiontests; then 10 runs ofguest_sessionas root, with cgroups, and 10 unprivileged, with process groups, and 3 runs of the library's tests, all passing, with no cgroup left behind;host_session, all passing;v0.1.0-dev.c2ace076c60aartifacts (OpenVMM 2b681564): on WHP, locally, all 35 layered scenarios, the 3 restart tests and the 3 step tests pass; on MSHV, on a Linux test host, as root, with OpenVMM's--cpu-profile host, under a short temporary directory, which keeps the VMs' control sockets' paths withinSUN_LEN, so do all of them. KVM is left to CI.Command::enter.