diff --git a/doc/run.md b/doc/run.md index 40c98bf61..40604d597 100644 --- a/doc/run.md +++ b/doc/run.md @@ -900,9 +900,10 @@ the export, before the overlay is unmounted or the VM powers off, unless the guest stops answering, or `start` is interrupted, and OpenVMM is terminated first. Each target must be an absolute, canonical path; `/`, `/etc`, `/run`, and `/run/nvx`, and -the `/etc/machine-id`, `/proc`, `/sys`, `/dev`, `/.nvx-agent`, and -`/run/nvx/time` trees, are reserved for the container runtime, which replaces -`/etc/machine-id` with the workload's machine ID, creates `/run` and +the `/etc/machine-id`, `/etc/resolv.conf`, `/proc`, `/sys`, `/dev`, +`/.nvx-agent`, and `/run/nvx/time` trees, are reserved for the container +runtime, which replaces `/etc/machine-id` with the workload's machine ID, keeps +`/etc/resolv.conf` for the guest's resolver file, creates `/run` and `/run/nvx`, and binds its time state over `/run/nvx/time`. The guest refuses a target whose path crosses a symbolic link in a container layer, a target that a container layer holds as another kind of file than the diff --git a/doc/usage.md b/doc/usage.md index f170785a7..e93790fea 100644 --- a/doc/usage.md +++ b/doc/usage.md @@ -292,7 +292,7 @@ do, and their names cannot be abbreviated either. | `--environment-file PATH` | omitted | Read the exact managed `exec` environment from a strict UTF-8 JSON array of `KEY=VALUE` strings, limited to 1 MiB of input. Invalid JSON, non-finite numbers, and unpaired surrogate escapes are refused. An empty array requests an empty environment. This option is mutually exclusive with `--environment`; omitting both preserves guest defaults. | | `--inherit-default-environment` | off | Layer the managed `exec` environment from `--environment` or `--environment-file` over the guest default environment instead of replacing it; an entry replaces the default variable of the same name. Without either option the workload already receives the defaults, so the flag has no effect. | | `--hypervisor {auto,whp,kvm,mshv}` | `auto` | Select the host hypervisor. | -| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share a host directory, or a single regular host file without its directory, at the absolute target inside the container rootfs for `run` or `provision`; defaults to `ro`. The mode follows the last comma, so `HOST_PATH` may contain commas when the mode is given. Repeat to attach more shares, each with its own target and mode, for example a read-write workspace and a read-only tool cache, as the children of an aggregate that the guest binds at their targets; a shared file always uses the aggregate, and so does every share of a managed sandbox. Targets and host paths must not overlap. A one-shot `run` describes its shares in kernel command-line tokens that count toward the sandbox's 1024-byte budget; a managed sandbox announces only their number and sends their mapping table once its guest agent is ready, for at most 256 shares. An `rw` share accepts guest-created symbolic links, which the host never follows. `/`, `/etc`, `/run`, and `/run/nvx`, and the `/etc/machine-id`, `/proc`, `/sys`, `/dev`, `/.nvx-agent`, and `/run/nvx/time` trees, are reserved. | +| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share a host directory, or a single regular host file without its directory, at the absolute target inside the container rootfs for `run` or `provision`; defaults to `ro`. The mode follows the last comma, so `HOST_PATH` may contain commas when the mode is given. Repeat to attach more shares, each with its own target and mode, for example a read-write workspace and a read-only tool cache, as the children of an aggregate that the guest binds at their targets; a shared file always uses the aggregate, and so does every share of a managed sandbox. Targets and host paths must not overlap. A one-shot `run` describes its shares in kernel command-line tokens that count toward the sandbox's 1024-byte budget; a managed sandbox announces only their number and sends their mapping table once its guest agent is ready, for at most 256 shares. An `rw` share accepts guest-created symbolic links, which the host never follows. `/`, `/etc`, `/run`, and `/run/nvx`, and the `/etc/machine-id`, `/etc/resolv.conf`, `/proc`, `/sys`, `/dev`, `/.nvx-agent`, and `/run/nvx/time` trees, are reserved. | | `--mount-deny HOST_PATH` | none | Hide one existing file or directory inside a `--mount` host directory; relative paths are resolved inside it. It may name the directory itself when `--mount-allow` paths inside it remain. With several shares, it applies to the `--mount` before it, which must share a directory. Repeat to deny multiple paths. | | `--mount-allow HOST_PATH` | none | Expose one existing file or directory inside a `--mount-deny` path of the same share again, for example a readable `mcp-payloads` directory inside hidden logs. The hidden directories on the way list only the entries that lead to it, and the workload cannot modify them. Relative paths are resolved inside the share; with several shares, it applies to the `--mount` before it. Repeat to allow multiple paths. See [Access policy](run.md#access-policy). | | `--mount-write HOST_PATH` | none | Make one existing file or directory one of the only writable parts of an `rw` share; the workload's writes anywhere else in the share fail with `EROFS`. Relative paths are resolved inside the share; with several shares, it applies to the `--mount` before it, which must be `rw`. Repeat to declare multiple writable paths. `provision` persists the denied, allowed, and writable paths. | diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 354e61784..9e9dd2f82 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -69,7 +69,7 @@ nvx-rs/ | Module | Contents | | --- | --- | | `nvx::protocol` | The codec of the control console: `NVXS` outer records, `NVXC` application frames, the `EXEC` payload that the host encodes and the guest agent decodes, the `MAPS` payloads of the host mapping table, `GuestFeatures`, and the `nvx_maps=` and `nvx_workload_account=create` tokens. The [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/nvxc-v1.txt`, and OpenVMM's vectors and cases of the `NVXS` session pin every record, frame, and payload independently of the codec, which the host, the guest agent, and the fake OpenVMM share | -| `nvx::protocol::cmdline` | The kernel command line as Linux 6.18 reads it, for the readers that need the kernel's view of it: `parameters` splits it into `Parameter`s as `parse_args` and `next_arg` do, at the bytes that `is_space` accepts, with double quotes, up to a NUL or `--`, and `Parameter::is_named` compares names as `parameq` does, with dashes and underscores alike. `VirtioMmioDevice::parse` reads a `virtio_mmio.device` value, `SIZE@BASE:IRQ[:ID]`, as the virtio-mmio driver reads it, for a device of any type, and `virtio_mmio_devices` reads every such parameter of a command line | +| `nvx::protocol::cmdline` | The kernel command line as Linux 6.18 reads it, for the readers that need the kernel's view of it: `parameters` splits it into `Parameter`s as `parse_args` and `next_arg` do, at the bytes that `is_space` accepts, with double quotes, up to a NUL or `--`, and `Parameter::is_named` compares names as `parameq` does, with dashes and underscores alike. `VirtioMmioDevice::parse` reads a `virtio_mmio.device` value, `SIZE@BASE:IRQ[:ID]`, as the virtio-mmio driver reads it, for a device of any type, and `virtio_mmio_devices` reads every such parameter of a command line. `dns_server` reads the guest's DNS server from OpenVMM's `virtnet_dns=` token, `DNS_TOKEN`, strictly, as OpenVMM renders and reads it: in a command line split at ASCII whitespace, at most one token, with one IPv4 or IPv6 address in the form that OpenVMM writes, and `write_resolver` writes the resolver file that names it, `nameserver ADDRESS`, as NVX's init writes the initramfs's `/etc/resolv.conf` | | `nvx::protocol::export` | The boot tokens of the host export, the aggregate virtio-fs device from which a guest maps host paths, which OpenVMM appends to the kernel command line: `virtfs_dir=` the export's guest directory, `EXPORT_DIRECTORY`, `/run/nvx/hostfs/root`, where the host's `nvx::host::filesystem` plan exports it, `virtfs_tag=` the device's tag, `virtfs_mode=` `ro` or `rw`, an `ExportMode`, and `virtfs_aggregate=1`. `HostExport::parse` reads them as the guest's init splits the command line: all four tokens or none, each once and in its form, with the checks of NVX's shell init, and no `nvx_share=` token, which binds a live share. It returns the `HostExport`, the device's tag and mode, which `nvx::guest::root::Root::mount_export` mounts, or `None`. `tests/fixtures/export-tokens.json` holds its cases | | `nvx::protocol::nvxs` | The NVXS session, independent of `NVXC` and of any other application protocol: `RecordType`, `Record`, `encode_record`, and `RecordParser`, which parses records incrementally across partial reads with buffers bounded by one record; `HostSession` and `GuestSession`, the state machines of the host's and the guest's roles, which validate every record's type, identity, and sequence number, and the guest's receive window and credits, without I/O. An application message may span DATA records, and the application's own framing reassembles it | | `nvx::protocol::root` | The boot tokens of a layered root, which the host renders and guests parse by the same rules: a `RootLayout` is a `RootImage` on the `distro` block device, an EROFS filesystem or a GPT disk with the `nvx::uuid::Uuid` that the guest expects of it, if any, over a `RootUpper`, the ext4 scratch disk or RAM. `RootLayout::tokens` renders `nvx_layer=distro,0xd0003000,FORMAT[:IDENTITY]` and `nvx_scratch=0xd0006000,ext4` or `nvx_overlay_upper=ramfs`, and `RootLayout::parse` reads them from a kernel command line, and refuses malformed, repeated, and conflicting tokens and the earlier form without a format, with no fallback that probes the devices. It reads the virtio-mmio devices of the command line with `nvx::protocol::cmdline`, to refuse a RAM upper beside one at the scratch base. `tests/fixtures/root-tokens.json` holds its cases, which the tests of the guest's shell init replay too | @@ -100,7 +100,7 @@ nvx-rs/ | `nvx::guest::nvxs` | `GuestConnection`: the guest's role of the NVXS session over a byte stream, such as the control console, on which NVX's managed agent and other guest agents run their sessions. It never reads past the record in progress, so a caller that polls the stream between records sees every record that it has not returned | | `nvx::guest::terminal` | `Terminal`: the terminal on which a guest agent serves its session, such as the VM's control console, as a byte stream. It opens the terminal, waiting while the kernel sets it up during boot, puts it in raw mode, so that it passes the session's bytes as they are, and drains it, so that it sends what the agent wrote before the agent ends the VM. NVX's managed agent and other guest agents serve their sessions on it | | `nvx::guest::spawn` | `Command`: starts a program in a new process for any guest init or agent, with its arguments, environment, a `PATH` in which it looks for a program named without a `/`, root and working directories, standard streams, inherited descriptors, cgroup, process group or new session, a `Barrier` at which the program waits until the caller releases it, a reset that gives every signal its default disposition and blocks none, and an `entry::Prepared` execution to enter, without NVX's control protocol, managed agent, or workload identity. Its `LaunchReport` tells whether the program started, or which `LaunchStage` failed and why. NVX's managed agent is one caller | -| `nvx::guest::entry` | `Entry`: the execution that a program enters, for any guest init or agent, from typed inputs: a root, which is a mount, moved over `/` in new mount, PID, and UTS namespaces, where the program is PID 1 and leads a new session; a hostname; a fresh procfs, a read-only sysfs, and a `/dev` of its own with only the allowlisted `DeviceNode`s, a devpts, and a `/dev/shm`, every other mount `nodev`; the `MachineId`, the image's own or one generated by `MachineId::generate`, and the time status, which `Entry::bind_runtime` binds read-only into the root once, through `mappings::bind`, for every later execution; and the `Identity`: user and group IDs, supplementary groups, and `Capabilities` from an allowlist, which only user ID 0 may hold, without `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW` while the execution's listeners are `Publication::Published`, taken with `no_new_privs`. Its program starts with every signal's default disposition and none blocked. For user ID 0, the risky procfs and sysfs entries are read-only or masked. `Entry::prepare` checks it, reads the existing `Account` of the IDs from the root's account databases unless `AccountMode::Numeric` takes them as they are, and `spawn::Command::enter` runs it, with each step a `LaunchStage`. It reads no kernel command line or boot token and creates no account | +| `nvx::guest::entry` | `Entry`: the execution that a program enters, for any guest init or agent, from typed inputs: a root, which is a mount, moved over `/` in new mount, PID, and UTS namespaces, where the program is PID 1 and leads a new session; a hostname; a fresh procfs, a read-only sysfs, and a `/dev` of its own with only the allowlisted `DeviceNode`s, a devpts, and a `/dev/shm`, every other mount `nodev`; the `MachineId`, the image's own or one generated by `MachineId::generate`, the time status, and the resolver file that `Entry::resolver` names, if any, which `Entry::bind_runtime` binds read-only into the root once, through `mappings::bind`'s walk and attach, for every later execution, and returns as `RuntimeBinds`, which `RuntimeBinds::remove` unmounts in reverse order before the root's teardown, and of which a failed bind removes those made before it, with an `Error::Rollback` that names any that stays; and the `Identity`: user and group IDs, supplementary groups, and `Capabilities` from an allowlist, which only user ID 0 may hold, without `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW` while the execution's listeners are `Publication::Published`, taken with `no_new_privs`. Its program starts with every signal's default disposition and none blocked. For user ID 0, the risky procfs and sysfs entries are read-only or masked. `Entry::prepare` checks it, reads the existing `Account` of the IDs from the root's account databases unless `AccountMode::Numeric` takes them as they are, and `spawn::Command::enter` runs it, with each step a `LaunchStage`. It reads no kernel command line or boot token and creates no account | | `nvx::guest::process` | `Child`: the status of one child that the caller names, by its pidfd or its process ID. It waits for that child alone, so another child's status stays for the caller; a caller that collects every status itself, such as a PID 1 that adopts orphans, hands each one to its `Child` with `observe`. Statuses are `std::process::ExitStatus`, real-time signals included | | `nvx::guest::cgroup` | `Layout`: the agent's cgroup and the workload cgroup, which the caller names, and which may not hold the agent's. `ExecutionCgroup`: the cgroup of one execution, the workload cgroup itself or a child of it, which it prepares with the caller's resource values, places processes in, checks for occupancy through `pids.current`, kills with `cgroup.kill`, and settles, over the execution's cgroup or the whole workload subtree, as the caller's `Scope` chooses. It holds its cgroups' directories open, and the process's removals and preparations of cgroups take turns, so that it never reaches another cgroup that takes one of their paths. Settling calls the caller's status collector between its checks, and collects no status itself | | `nvx::guest::exec` | `Execution`: drives one program that the caller spawned with the program's ends of its `Pipes`, or that the caller started itself and adopts with its own ends of the program's pipes and a `Child` that names it, without blocking, from the caller's own loop or reactor, and reports each step as an `Event`: the launch report, which an adopted program has at once, the output of each stream, kill failures, and the `Outcome`. It reads a stream only as far as the caller's `grant` allows, so a stalled sink stalls its own execution alone, writes the program's standard input without `SIGPIPE`, and ends the program on a cancellation, a timeout, or a disconnection, with `SIGKILL` through the program's pidfd to the program and, if the caller asks, its process group, while no collector has taken its status, and, through the opened directory of a cgroup that the caller names, to that cgroup. It closes the program's input once the program has ended. It waits for its program alone, by a pidfd, or takes the status from the caller's collector with `observe`; with `Options::caller_reaps`, that collector is the program's only one. NVX's managed agent is one caller | diff --git a/nvx-rs/src/guest/entry.rs b/nvx-rs/src/guest/entry.rs index 677010ae7..dc3f06bbf 100644 --- a/nvx-rs/src/guest/entry.rs +++ b/nvx-rs/src/guest/entry.rs @@ -8,32 +8,42 @@ //! from an allowlist of 13, and the [`AccountMode`]; //! - the hostname, `nvx-sandbox` unless the adapter names another; //! - the [`MachineId`] mode, the [`Publication`] state, the source of the time status, if any, -//! and the [`DeviceNode`]s of `/dev`. +//! the resolver file, if any, and the [`DeviceNode`]s of `/dev`. //! //! [`Entry::bind_runtime`] binds the runtime's files into the root once, for every execution -//! that enters it later. [`Entry::prepare`] checks the inputs for one execution, and reads and -//! checks the account in existing-account mode; [`Command::enter`] then has the child enter that -//! execution, in launch stages of [`spawn`](super::spawn), in the order of [`LaunchStage`]. -//! Nothing here reads the kernel command line or a boot token, and nothing creates an account. +//! that enters it later, and returns the [`RuntimeBinds`], which a caller removes before it tears +//! the root down in the same mount namespace. [`Entry::prepare`] checks the inputs for one +//! execution, and reads and checks the account in existing-account mode; [`Command::enter`] then +//! has the child enter that execution, in launch stages of [`spawn`](super::spawn), in the order +//! of [`LaunchStage`]. Nothing here reads the kernel command line or a boot token, and nothing +//! creates an account. //! //! # The runtime's binds //! //! [`Entry::bind_runtime`] binds the file of a [generated](MachineId::Generated) machine ID over -//! the root's `/etc/machine-id`, and the time status's directory at the root's `/run/nvx/time`, -//! through [`mappings::bind`]: each is read-only, `nosuid`, and `nodev` before anything sees it, -//! and the walk creates what the root lacks only on the root's own mount, through no symbolic -//! link, and refuses a target of another kind than its source. A symbolic link at -//! `/etc/machine-id` itself, which no bind can cover, is replaced with an empty file first. In -//! [image](MachineId::Image) mode, the image's own `/etc/machine-id` is left alone. The time -//! status's directory must hold only what any execution may use, since one of user ID 0 can -//! open every file in it that user ID 0 owns, and hold its lock: the guest's time tools keep -//! their locks and private records outside it. +//! the root's `/etc/machine-id`, the time status's directory at the root's `/run/nvx/time`, and +//! a [resolver](Entry::resolver) file, which names the guest's DNS servers, over the root's +//! `/etc/resolv.conf`, through the walk and attach of [`mappings::bind`]: each is read-only, +//! `nosuid`, and `nodev` before anything sees it, and the walk creates what the root lacks only +//! on the root's own mount, through no symbolic link, and refuses a target of another kind than +//! its source. A symbolic link at `/etc/machine-id` or `/etc/resolv.conf` itself, which no bind +//! can cover, is replaced with an empty file first. In [image](MachineId::Image) mode, the +//! image's own `/etc/machine-id` is left alone, and without a resolver file, so is the image's +//! own `/etc/resolv.conf`. The time status's directory must hold only what any execution may +//! use, since one of user ID 0 can open every file in it that user ID 0 owns, and hold its lock: +//! the guest's time tools keep their locks and private records outside it. //! //! The binds are made in the caller's mount namespace, of which each execution's starts as a //! copy, so every execution that enters the root later sees them, and sees their sources as they //! change, such as a machine ID that the guest rewrites in place after a restore. They last as -//! long as that namespace: a caller whose root is torn down in its own namespace after it, such -//! as an agent that an init outlives, binds them in a mount namespace of its own. +//! long as that namespace, unless [`RuntimeBinds::remove`] removes them, in reverse order. A +//! root's teardown detaches no busy mount, so a caller that tears the root down in the namespace +//! where it bound them removes them first. A caller whose root is torn down after it ends, such +//! as an agent that an init outlives, may instead bind them in a mount namespace of its own, +//! which ends with it, and drop them. A bind that fails has the call remove the binds that it +//! made before it, and an [`Error::Rollback`] names any of them that stays mounted. A bind that a +//! rename moved while the walk ran, and that the attach failed to detach, fails the call, and +//! stays mounted wherever it went, as [`mappings::bind`] describes. //! //! # The execution //! @@ -82,15 +92,20 @@ //! //! use nvx::guest::entry::{Entry, Identity, MachineId}; //! use nvx::guest::spawn::Command; +//! use nvx::protocol::cmdline; //! //! # fn main() -> Result<(), Box> { //! let machine_id = MachineId::generate("/run/nvx/workload-machine-id")?; -//! let entry = Entry::new("/run/nvx/rootfs", Identity::new(65534, 65534)) +//! let mut entry = Entry::new("/run/nvx/rootfs", Identity::new(65534, 65534)) //! .hostname("sandbox") //! .machine_id(machine_id) //! .time_status("/run/nvx/time"); +//! // The guest's init wrote its resolver file from the DNS server that the command line names. +//! if cmdline::dns_server(&std::fs::read("/proc/cmdline")?)?.is_some() { +//! entry = entry.resolver("/etc/resolv.conf"); +//! } //! // Once, before the first execution. -//! entry.bind_runtime()?; +//! let binds = entry.bind_runtime()?; //! let prepared = entry.prepare()?; //! let home = prepared.account().map(|account| account.home().to_vec()); //! let mut spawned = Command::new(c"/bin/true".to_owned(), vec![c"true".to_owned()]) @@ -100,6 +115,8 @@ //! let barrier = spawned.take_barrier().expect("the command has a barrier"); //! // Place the child, by `spawned.id()`, in its cgroup; then: //! barrier.release()?; +//! // Once the executions have ended, before the root's teardown in this mount namespace: +//! binds.remove()?; //! # Ok(()) //! # } //! ``` @@ -109,11 +126,16 @@ use std::io::{self, Read}; use std::os::unix::ffi::OsStrExt; use std::path::{Path, PathBuf}; -use super::mappings::{self, Access, Kind, Mapping, MappingError, SourceRoot, Step, TargetRoot}; +use super::mappings::{ + self, Access, Kernel, Kind, Mapped, Mapping, MappingError, SourceRoot, Step, SystemKernel, + TargetRoot, TeardownError, +}; use super::parse::parse_decimal; #[cfg(doc)] use super::spawn::{Command, LaunchStage}; use super::sys::{self, EntryChild, Errno, FileDescriptor, Mode, OpenFlags}; +#[cfg(doc)] +use crate::protocol::cmdline::write_resolver; /// The hostname that an entry gives its execution unless the adapter names another. pub const DEFAULT_HOSTNAME: &str = "nvx-sandbox"; @@ -135,6 +157,8 @@ const MACHINE_ID_MODE: u32 = 0o444; const MACHINE_ID_TARGET: &str = "/etc/machine-id"; /// Where [`Entry::bind_runtime`] binds the time status in the root. const TIME_STATUS_TARGET: &str = "/run/nvx/time"; +/// Where [`Entry::bind_runtime`] binds the resolver file in the root. +const RESOLVER_TARGET: &str = "/etc/resolv.conf"; /// The names of the Linux capabilities, by number. const CAPABILITY_NAMES: [&str; 41] = [ @@ -553,6 +577,8 @@ pub struct Entry { publication: Publication, /// The directory of the time status, if the executions see one. time_status: Option, + /// The resolver file, if the executions see one. + resolver: Option, /// The device nodes of `/dev`, as [`DeviceNode::bit`]s. devices: u8, } @@ -560,7 +586,7 @@ pub struct Entry { impl Entry { /// Describes executions that enter the mount at `root`, an absolute path, as `identity`, /// with the hostname `nvx-sandbox`, the image's machine ID, published listeners, no time - /// status, and every [`DeviceNode`]. + /// status, the image's resolver file, and every [`DeviceNode`]. pub fn new(root: impl Into, identity: Identity) -> Self { Self { root: root.into(), @@ -569,6 +595,7 @@ impl Entry { machine_id: MachineId::Image, publication: Publication::Published, time_status: None, + resolver: None, devices: DeviceNode::ALL .iter() .fold(0, |devices, node| devices | node.bit()), @@ -606,6 +633,19 @@ impl Entry { self } + /// Has the executions see the regular file `source`, a resolver file in the format of + /// `/etc/resolv.conf` that names the guest's DNS servers, read-only at `/etc/resolv.conf`, + /// once [`bind_runtime`](Self::bind_runtime) has bound it. Without one, they see the image's + /// own `/etc/resolv.conf`. + /// + /// The file may be the one that the guest's init wrote from the `virtnet_dns=` token, or one + /// that [`write_resolver`] wrote. Every execution must be able to read it. The bind shows the + /// file as it is, so what its owner rewrites in place reaches every execution. + pub fn resolver(mut self, source: impl Into) -> Self { + self.resolver = Some(source.into()); + self + } + /// Gives the executions' `/dev` exactly the device nodes `devices`. pub fn devices(mut self, devices: impl IntoIterator) -> Self { self.devices = devices @@ -706,51 +746,143 @@ impl Entry { } /// Binds the runtime's files into the root, in the caller's mount namespace, for every - /// execution that enters the root later, as the [module](self) describes: the file of a - /// [generated](MachineId::Generated) machine ID over `/etc/machine-id`, then the - /// [time status](Self::time_status) at `/run/nvx/time`. An entry with neither binds nothing. + /// execution that enters the root later, as the [module](self) describes, and returns the + /// binds that it made: the file of a [generated](MachineId::Generated) machine ID over + /// `/etc/machine-id`, then the [time status](Self::time_status) at `/run/nvx/time`, then the + /// [resolver](Self::resolver) file over `/etc/resolv.conf`. An entry with none of them binds + /// nothing. + /// + /// The binds last as long as the caller's mount namespace, unless [`RuntimeBinds::remove`] + /// removes them, as a caller that tears the root down in that namespace does first. If a bind + /// fails, the call removes the binds that it made before it, in reverse order, and returns + /// the failure, or, if any of them stays mounted, an [`Error::Rollback`] that names them too. /// /// Each call binds again, over the earlier binds, so an adapter calls it once, before its /// first execution. It needs the privilege to mount, and Linux 6.8 or later. - pub fn bind_runtime(&self) -> Result<(), Error> { + pub fn bind_runtime(&self) -> Result { + self.bind_runtime_with(&mut SystemKernel) + } + + /// Implements [`bind_runtime`](Self::bind_runtime) with `kernel`. + pub(crate) fn bind_runtime_with(&self, kernel: &mut dyn Kernel) -> Result { self.check()?; - let machine_id = match &self.machine_id { - MachineId::Image => None, - MachineId::Generated(path) => Some(path), - }; - if machine_id.is_none() && self.time_status.is_none() { - return Ok(()); + if self.machine_id == MachineId::Image + && self.time_status.is_none() + && self.resolver.is_none() + { + return Ok(RuntimeBinds { + root: None, + binds: Vec::new(), + }); } - let root = TargetRoot::open(&self.root).map_err(Error::Root)?; - if let Some(path) = machine_id { - let bind = || bind_runtime_file(&root, path, MACHINE_ID_TARGET, Kind::File); - match bind() { - // No bind covers a link, so an empty file, which the bind creates, replaces it. - Err(MappingError::Link { path: link }) if link == Path::new(MACHINE_ID_TARGET) => { - remove_machine_id_link(&self.root) - .map_err(|_| Error::MachineId(MappingError::Link { path: link }))?; - bind() - } - bound => bound, - } - .map_err(Error::MachineId)?; + let root = TargetRoot::open_with(kernel, &self.root).map_err(Error::Root)?; + let mut binds = Vec::new(); + if let Err(error) = self.bind_each(kernel, &root, &mut binds) { + // The caller cannot remove the binds before the failure, which a teardown in this + // namespace would trip over, so it learns of any that stays. + return Err(match mappings::unbind_with(kernel, &root, &binds) { + Ok(()) => error, + Err(teardown) => Error::Rollback { + error: Box::new(error), + teardown, + }, + }); + } + Ok(RuntimeBinds { + root: Some(root), + binds, + }) + } + + /// Makes the binds of [`bind_runtime`](Self::bind_runtime) beneath `root`, the root's mount, + /// in order, with `kernel`, and adds each to `binds`, until one fails. + fn bind_each( + &self, + kernel: &mut dyn Kernel, + root: &TargetRoot, + binds: &mut Vec, + ) -> Result<(), Error> { + if let MachineId::Generated(source) = &self.machine_id { + let bound = bind_file(kernel, root, &self.root, source, MACHINE_ID_TARGET); + binds.push(bound.map_err(Error::MachineId)?); } if let Some(source) = &self.time_status { - bind_runtime_file(&root, source, TIME_STATUS_TARGET, Kind::Directory) - .map_err(Error::TimeStatus)?; + let bound = + bind_runtime_file(kernel, root, source, TIME_STATUS_TARGET, Kind::Directory); + binds.push(bound.map_err(Error::TimeStatus)?); + } + if let Some(source) = &self.resolver { + let bound = bind_file(kernel, root, &self.root, source, RESOLVER_TARGET); + binds.push(bound.map_err(Error::Resolver)?); } Ok(()) } } +/// The runtime's binds that [`Entry::bind_runtime`] made in the caller's mount namespace, in the +/// order in which it made them. +/// +/// A caller that tears the root down in that namespace [removes](Self::remove) them first: a +/// teardown detaches no busy mount, and the root's mount is busy while they are on it. A caller +/// whose namespace ends before the teardown, such as an agent that an init outlives, may drop +/// them instead, which leaves them mounted. Until they are removed or dropped, they hold a +/// descriptor of the root, which keeps its mount busy too. +#[derive(Debug)] +pub struct RuntimeBinds { + /// The root beneath which the binds were made, unless there are none. + root: Option, + /// The binds, in the order in which they were made. + binds: Vec, +} + +impl RuntimeBinds { + /// Removes the binds, in reverse order: unmounts each that is still the topmost mount at its + /// target, by its name in its parent directory, without detaching a busy one, which needs + /// procfs at `/proc`. It attempts each, whatever failed before, and reports every one that it + /// failed to unmount, which stays mounted. + pub fn remove(self) -> Result<(), TeardownError> { + self.remove_with(&mut SystemKernel) + } + + /// Implements [`remove`](Self::remove) with `kernel`. + pub(crate) fn remove_with(self, kernel: &mut dyn Kernel) -> Result<(), TeardownError> { + match &self.root { + Some(root) => mappings::unbind_with(kernel, root, &self.binds), + None => Ok(()), + } + } +} + +/// Binds the regular file at `source` read-only over `target`, a file of the root's `/etc`, +/// beneath `root`, the mount at `path`, with `kernel`, and returns the bind. A symbolic link at +/// `target` itself, which no bind can cover, is first replaced with an empty file, which the bind +/// creates. +fn bind_file( + kernel: &mut dyn Kernel, + root: &TargetRoot, + path: &Path, + source: &Path, + target: &str, +) -> Result { + match bind_runtime_file(kernel, root, source, target, Kind::File) { + Err(MappingError::Link { path: link }) if link == Path::new(target) => { + remove_link(path, target).map_err(|_| MappingError::Link { path: link })?; + bind_runtime_file(kernel, root, source, target, Kind::File) + } + bound => bound, + } +} + /// Binds the file or directory at `path`, which must be of `kind`, read-only at `target` beneath -/// `root`, through [`mappings::bind`] from a source root at `path`'s parent directory. +/// `root`, with `kernel`, through the walk and attach of [`mappings::bind`] from a source root at +/// `path`'s parent directory, and returns the bind. fn bind_runtime_file( + kernel: &mut dyn Kernel, root: &TargetRoot, path: &Path, target: &str, kind: Kind, -) -> Result<(), MappingError> { +) -> Result { let source_error = |error: i32| MappingError::System { step: Step::Source, error, @@ -763,7 +895,7 @@ fn bind_runtime_file( let mapping = Mapping::new(name, target, Access::ReadOnly) .map_err(|_| source_error(Errno::EINVAL as i32))? .with_kind(kind); - mappings::bind(root, &source, &mapping) + mappings::bind_with(kernel, root, &source, &mapping) } /// Returns the directory that holds the runtime file at `path`, the current directory for a @@ -778,9 +910,11 @@ fn runtime_source(path: &Path) -> Option<(&Path, &std::ffi::OsStr)> { } } -/// Removes the symbolic link at `/etc/machine-id` beneath `root`, reached through no other link -/// and on the root's own mount. -fn remove_machine_id_link(root: &Path) -> Result<(), Errno> { +/// Removes the symbolic link at `target`, a file of `/etc` such as `/etc/machine-id`, beneath +/// `root`, reached through no other link and on the root's own mount. +fn remove_link(root: &Path, target: &str) -> Result<(), Errno> { + let name = target.strip_prefix("/etc/").ok_or(Errno::EINVAL)?; + let name = sys::c_string(name.as_bytes())?; let root = sys::path_c_string(root)?; let directory = sys::open( &root, @@ -796,7 +930,7 @@ fn remove_machine_id_link(root: &Path) -> Result<(), Errno> { )?; let status = nix::sys::stat::fstatat( etc.borrowed(), - c"machine-id", + name.as_c_str(), nix::fcntl::AtFlags::AT_SYMLINK_NOFOLLOW, )?; if status.st_mode & nix::libc::S_IFMT != nix::libc::S_IFLNK { @@ -804,7 +938,7 @@ fn remove_machine_id_link(root: &Path) -> Result<(), Errno> { } nix::unistd::unlinkat( etc.borrowed(), - c"machine-id", + name.as_c_str(), nix::unistd::UnlinkatFlags::NoRemoveDir, ) } @@ -976,6 +1110,20 @@ pub enum Error { /// [`Entry::bind_runtime`] cannot bind the time status's directory at the root's /// `/run/nvx/time`. TimeStatus(MappingError), + /// [`Entry::bind_runtime`] cannot bind the resolver file over the root's `/etc/resolv.conf`: + /// it is missing or not a regular file, the target cannot hold it, or the caller cannot + /// mount. + Resolver(MappingError), + /// [`Entry::bind_runtime`] failed to make a bind, and to remove some of the binds that it made + /// before it, which stay mounted, so that a teardown of the root in the caller's mount + /// namespace fails. + Rollback { + /// Why the bind failed: an [`Error::MachineId`], [`Error::TimeStatus`], or + /// [`Error::Resolver`]. + error: Box, + /// The binds made before it that stay mounted. + teardown: TeardownError, + }, } impl fmt::Display for Error { @@ -1000,6 +1148,11 @@ impl fmt::Display for Error { Self::Account(error) => write!(formatter, "the account is unavailable: {error}"), Self::MachineId(error) => write!(formatter, "cannot bind the machine ID: {error}"), Self::TimeStatus(error) => write!(formatter, "cannot bind the time status: {error}"), + Self::Resolver(error) => write!(formatter, "cannot bind the resolver file: {error}"), + Self::Rollback { error, teardown } => write!( + formatter, + "{error}; the binds made before it stay mounted: {teardown}" + ), } } } @@ -1008,8 +1161,9 @@ impl std::error::Error for Error { fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { match self { Self::Root(error) => Some(error), - Self::MachineId(error) | Self::TimeStatus(error) => Some(error), + Self::MachineId(error) | Self::TimeStatus(error) | Self::Resolver(error) => Some(error), Self::Account(error) => Some(error), + Self::Rollback { error, .. } => Some(error.as_ref()), _ => None, } } diff --git a/nvx-rs/src/guest/entry/tests.rs b/nvx-rs/src/guest/entry/tests.rs index 0abb3d7a0..a46cf8c07 100644 --- a/nvx-rs/src/guest/entry/tests.rs +++ b/nvx-rs/src/guest/entry/tests.rs @@ -2,8 +2,10 @@ //! //! The launch tests run each entry without privileges: the child starts without new namespaces //! and skips the system calls of every entry stage, but fails the one that a test names, so that -//! each stage reports its own failure. The tests whose names start with `privileged_` run the -//! entry for real, and only where the test process may mount, as root does. +//! each stage reports its own failure. The runtime binds' tests run on the root operations' fake +//! kernel, which simulates mounts by their unique IDs. The tests whose names start with +//! `privileged_` run the entry, and the runtime's binds, for real, and only where the test +//! process may mount, as root does. use std::ffi::CString; use std::io::Read; @@ -11,8 +13,10 @@ use std::os::unix::ffi::OsStrExt; use std::os::unix::fs::PermissionsExt; use super::*; +use crate::guest::root::{Call, FakeKernel, Operation, attach_calls}; use crate::guest::spawn::{Command, LaunchFailure, LaunchStage}; use crate::guest::sys::{ChildStatus, Simulation}; +use crate::protocol::cmdline::write_resolver; /// The passwd database of the tests' roots. const PASSWD_DATABASE: &[u8] = b"root:x:0:0:root:/root:/bin/sh\n\ @@ -526,8 +530,11 @@ fn a_generated_machine_id_has_mode_0444_whatever_the_umask() { fn runtime_binds_need_a_mount_root_unless_there_are_none() { let root = root_with(None, None); let identity = || Identity::new(1, 1).account(AccountMode::Numeric); - // The image's machine ID and no time status bind nothing, so any root does. - Entry::new(root.path(), identity()).bind_runtime().unwrap(); + // The image's machine ID and resolver file, and no time status, bind nothing, so any root + // does, and there is nothing to remove. + let binds = Entry::new(root.path(), identity()).bind_runtime().unwrap(); + assert!(binds.root.is_none() && binds.binds.is_empty()); + binds.remove().unwrap(); let machine_id = MachineId::generate(root.path().join("machine-id")).unwrap(); let result = Entry::new(root.path(), identity()) .machine_id(machine_id) @@ -540,6 +547,10 @@ fn runtime_binds_need_a_mount_root_unless_there_are_none() { .time_status(root.path().join("etc")) .bind_runtime(); assert!(matches!(result, Err(Error::Root(_))), "{result:?}"); + let result = Entry::new(root.path(), identity()) + .resolver(root.path().join("machine-id")) + .bind_runtime(); + assert!(matches!(result, Err(Error::Root(_))), "{result:?}"); // The inputs are checked first. let result = Entry::new(root.path(), identity()) .hostname("") @@ -548,6 +559,228 @@ fn runtime_binds_need_a_mount_root_unless_there_are_none() { assert!(matches!(result, Err(Error::Hostname)), "{result:?}"); } +/// A root, `root`, on a mount of the fake kernel's, with an empty `etc`, and the sources of the +/// runtime's files, in `sources`: a generated machine ID, `machine-id`, the time status's +/// directory, `time`, and a resolver file that names `10.0.0.1`, `resolv.conf`, below a +/// temporary directory. +struct RuntimeFixture { + /// The temporary directory, which the fixture holds only so that dropping it removes it. + _directory: tempfile::TempDir, + /// The root. + root: PathBuf, + /// The directory of the sources. + sources: PathBuf, + /// The fake kernel. + kernel: FakeKernel, +} + +impl RuntimeFixture { + fn new() -> Self { + let directory = tempfile::tempdir().unwrap(); + let base = directory.path().canonicalize().unwrap(); + let root = base.join("root"); + let sources = base.join("sources"); + std::fs::create_dir_all(root.join("etc")).unwrap(); + std::fs::create_dir_all(sources.join("time")).unwrap(); + MachineId::generate(sources.join("machine-id")).unwrap(); + let resolver = std::fs::File::create(sources.join("resolv.conf")).unwrap(); + write_resolver(resolver, "10.0.0.1".parse().unwrap()).unwrap(); + let kernel = FakeKernel::default(); + kernel.add_mount(&root); + Self { + _directory: directory, + root, + sources, + kernel, + } + } + + /// Returns the entry of the root, of user ID 0 in numeric-account mode, with every runtime + /// file of the sources. + fn entry(&self) -> Entry { + Entry::new( + &self.root, + Identity::new(0, 0).account(AccountMode::Numeric), + ) + .machine_id(MachineId::Generated(self.sources.join("machine-id"))) + .time_status(self.sources.join("time")) + .resolver(self.sources.join("resolv.conf")) + } + + /// Returns the mount points of the runtime's targets in the root, in the order of the binds. + fn points(&self) -> [PathBuf; 3] { + ["etc/machine-id", "run/nvx/time", "etc/resolv.conf"].map(|target| self.root.join(target)) + } + + /// Returns the mount points as the fake kernel records them, in reverse order. + fn removals(&self, points: &[PathBuf]) -> Vec { + points + .iter() + .rev() + .map(|point| point.display().to_string()) + .collect() + } +} + +#[test] +fn runtime_binds_are_made_in_order_and_removed_in_reverse() { + let fixture = RuntimeFixture::new(); + // The image's resolver file is a link, as a resolver daemon's image has, which no bind + // covers; it gives way to an empty file, and its target is left alone. + let outside = fixture.sources.join("stub-resolv.conf"); + std::fs::write(&outside, b"nameserver 127.0.0.53\n").unwrap(); + std::os::unix::fs::symlink(&outside, fixture.root.join("etc/resolv.conf")).unwrap(); + let binds = fixture + .entry() + .bind_runtime_with(&mut fixture.kernel.clone()) + .unwrap(); + let points = fixture.points(); + let expected: Vec = ["machine-id", "time", "resolv.conf"] + .iter() + .zip(&points) + .flat_map(|(source, point)| { + attach_calls(&fixture.sources.join(source), point, Access::ReadOnly) + }) + .collect(); + assert_eq!(fixture.kernel.calls(), expected); + for (bind, point) in binds.binds.iter().zip(&points) { + assert_eq!(fixture.root.join(bind.target()), *point); + assert!(fixture.kernel.mount_at(point).is_some(), "{point:?}"); + } + let replaced = std::fs::symlink_metadata(fixture.root.join("etc/resolv.conf")).unwrap(); + assert!(replaced.is_file() && replaced.len() == 0); + assert_eq!(replaced.permissions().mode() & 0o7777, 0o644); + assert_eq!(std::fs::read(&outside).unwrap(), b"nameserver 127.0.0.53\n"); + // Their removal takes them off, last first. + binds.remove_with(&mut fixture.kernel.clone()).unwrap(); + assert_eq!(fixture.kernel.unmounts(), fixture.removals(&points)); + assert!( + points + .iter() + .all(|point| fixture.kernel.mount_at(point).is_none()) + ); +} + +#[test] +fn a_failed_runtime_bind_removes_the_binds_before_it() { + // A missing resolver file, after the machine ID and the time status. + let fixture = RuntimeFixture::new(); + let points = fixture.points(); + let result = fixture + .entry() + .resolver(fixture.sources.join("missing")) + .bind_runtime_with(&mut fixture.kernel.clone()); + assert!( + matches!( + &result, + Err(Error::Resolver(MappingError::System { step: Step::Source, error })) + if *error == Errno::ENOENT as i32 + ), + "{result:?}" + ); + assert_eq!(fixture.kernel.unmounts(), fixture.removals(&points[..2])); + assert!( + points + .iter() + .all(|point| fixture.kernel.mount_at(point).is_none()) + ); + // A target of another kind for the time status, after the machine ID. + let fixture = RuntimeFixture::new(); + let points = fixture.points(); + std::fs::create_dir_all(fixture.root.join("run/nvx")).unwrap(); + std::fs::write(fixture.root.join("run/nvx/time"), b"").unwrap(); + let result = fixture + .entry() + .bind_runtime_with(&mut fixture.kernel.clone()); + assert!( + matches!(result, Err(Error::TimeStatus(MappingError::TargetKind))), + "{result:?}" + ); + assert_eq!(fixture.kernel.unmounts(), fixture.removals(&points[..1])); + assert_eq!(fixture.kernel.mount_at(&points[0]), None); + // A failure of the first bind leaves nothing to remove. + let fixture = RuntimeFixture::new(); + std::fs::create_dir(fixture.root.join("etc/machine-id")).unwrap(); + let result = fixture + .entry() + .bind_runtime_with(&mut fixture.kernel.clone()); + assert!( + matches!(result, Err(Error::MachineId(MappingError::TargetKind))), + "{result:?}" + ); + assert!(fixture.kernel.calls().is_empty()); +} + +#[test] +fn a_failed_runtime_bind_names_the_binds_before_it_that_stay() { + // A missing resolver file, after the machine ID and the time status, whose bind is busy. + let fixture = RuntimeFixture::new(); + let points = fixture.points(); + fixture + .kernel + .fail(Operation::Unmount, &points[1], Errno::EBUSY); + let error = fixture + .entry() + .resolver(fixture.sources.join("missing")) + .bind_runtime_with(&mut fixture.kernel.clone()) + .unwrap_err(); + let Error::Rollback { + error: failure, + teardown, + } = &error + else { + panic!("{error:?}"); + }; + assert!( + matches!( + failure.as_ref(), + Error::Resolver(MappingError::System { step: Step::Source, error: code }) + if *code == Errno::ENOENT as i32 + ), + "{failure:?}" + ); + let stayed: Vec<&Path> = teardown + .failures() + .iter() + .map(|stayed| stayed.target()) + .collect(); + assert_eq!(stayed, [Path::new("/run/nvx/time")]); + assert_eq!( + error.to_string(), + format!("{failure}; the binds made before it stay mounted: {teardown}") + ); + assert!(std::error::Error::source(&error).is_some()); + // The rollback attempts each bind, so the machine ID's is removed all the same. + assert_eq!(fixture.kernel.unmounts(), fixture.removals(&points[..2])); + assert!(fixture.kernel.mount_at(&points[1]).is_some()); + assert!(fixture.kernel.mount_at(&points[0]).is_none()); +} + +#[test] +fn a_runtime_bind_that_stays_is_reported() { + let fixture = RuntimeFixture::new(); + let binds = fixture + .entry() + .bind_runtime_with(&mut fixture.kernel.clone()) + .unwrap(); + let points = fixture.points(); + fixture + .kernel + .fail(Operation::Unmount, &points[2], Errno::EBUSY); + let error = binds.remove_with(&mut fixture.kernel.clone()).unwrap_err(); + assert_eq!( + error.to_string(), + format!( + "failed to unmount the mapping at /etc/resolv.conf: failed to unmount the mapping: {}", + io::Error::from_raw_os_error(Errno::EBUSY as i32) + ) + ); + // The others are removed all the same. + assert_eq!(fixture.kernel.unmounts(), fixture.removals(&points)); + assert!(fixture.kernel.mount_at(&points[2]).is_some()); + assert!(fixture.kernel.mount_at(&points[0]).is_none()); +} + #[test] fn runtime_sources_are_the_directories_beside_their_files() { use std::ffi::OsStr; @@ -571,23 +804,31 @@ fn runtime_sources_are_the_directories_beside_their_files() { } #[test] -fn only_a_link_at_the_machine_id_is_removed() { +fn only_a_link_at_a_runtime_file_of_etc_is_removed() { + for target in [MACHINE_ID_TARGET, RESOLVER_TARGET] { + let root = root_with(None, None); + let link = root.path().join(target.trim_start_matches('/')); + let outside = tempfile::NamedTempFile::new().unwrap(); + std::os::unix::fs::symlink(outside.path(), &link).unwrap(); + remove_link(root.path(), target).unwrap(); + assert!(std::fs::symlink_metadata(&link).is_err()); + assert!(outside.path().exists()); + assert_eq!(remove_link(root.path(), target), Err(Errno::ENOENT)); + std::fs::write(&link, b"image\n").unwrap(); + assert_eq!(remove_link(root.path(), target), Err(Errno::EEXIST)); + assert_eq!(std::fs::read(&link).unwrap(), b"image\n"); + // A link on the way to it is not followed. + let linked = tempfile::tempdir().unwrap(); + std::os::unix::fs::symlink(root.path().join("etc"), linked.path().join("etc")).unwrap(); + assert_eq!(remove_link(linked.path(), target), Err(Errno::ELOOP)); + assert_eq!(std::fs::read(&link).unwrap(), b"image\n"); + } + // Only a file of `/etc` is a runtime file there. let root = root_with(None, None); - let target = root.path().join("etc/machine-id"); - let outside = tempfile::NamedTempFile::new().unwrap(); - std::os::unix::fs::symlink(outside.path(), &target).unwrap(); - remove_machine_id_link(root.path()).unwrap(); - assert!(std::fs::symlink_metadata(&target).is_err()); - assert!(outside.path().exists()); - assert_eq!(remove_machine_id_link(root.path()), Err(Errno::ENOENT)); - std::fs::write(&target, b"image\n").unwrap(); - assert_eq!(remove_machine_id_link(root.path()), Err(Errno::EEXIST)); - assert_eq!(std::fs::read(&target).unwrap(), b"image\n"); - // A link on the way to it is not followed. - let linked = tempfile::tempdir().unwrap(); - std::os::unix::fs::symlink(root.path().join("etc"), linked.path().join("etc")).unwrap(); - assert_eq!(remove_machine_id_link(linked.path()), Err(Errno::ELOOP)); - assert_eq!(std::fs::read(&target).unwrap(), b"image\n"); + assert_eq!( + remove_link(root.path(), TIME_STATUS_TARGET), + Err(Errno::EINVAL) + ); } #[test] @@ -828,6 +1069,7 @@ for set in CapInh CapPrm CapEff CapBnd CapAmb NoNewPrivs; do echo "$set=$(field echo "pid=$$" echo "hostname=$(cat /proc/sys/kernel/hostname)" echo "machine-id=$(cat /etc/machine-id)" +echo "resolver=$(cat /etc/resolv.conf)" echo "time=$(cat /run/nvx/time/state)" echo "dev=$(cd /dev && echo *)" echo "pts=$(ls -A /dev/pts)" @@ -980,6 +1222,7 @@ fn check_view(output: &str, machine_id: &str) { assert_eq!(value(output, "CapAmb"), "0000000000000000", "{output}"); assert_eq!(value(output, "hostname"), "entry-test", "{output}"); assert_eq!(value(output, "machine-id"), machine_id, "{output}"); + assert_eq!(value(output, "resolver"), "nameserver 10.0.0.1", "{output}"); assert_eq!(value(output, "time"), "time-state", "{output}"); assert_eq!( value(output, "dev"), @@ -1003,6 +1246,7 @@ fn check_view(output: &str, machine_id: &str) { ("/dev/pts", &["nosuid", "noexec"]), ("/dev/shm", &["nosuid", "nodev"]), ("/etc/machine-id", &["ro", "nosuid", "nodev"]), + ("/etc/resolv.conf", &["ro", "nosuid", "nodev"]), ("/run/nvx/time", &["ro", "nosuid", "nodev"]), ] { let options = mount_options(output, point); @@ -1025,9 +1269,21 @@ fn check_view(output: &str, machine_id: &str) { } } +/// Returns the mount points below `path`, not `path` itself, in the caller's mount namespace. +fn mounts_below(path: &Path) -> Vec { + let table = std::fs::read_to_string("/proc/self/mountinfo").unwrap(); + table + .lines() + .filter_map(|line| line.split(' ').nth(4)) + .map(PathBuf::from) + .filter(|point| point.starts_with(path) && point != path) + .collect() +} + /// Checks that [`Entry::bind_runtime`] refuses what it cannot bind into `root`, the privileged /// entries' mounted root, from the sources in `base`, and leaves the root as it was. fn check_runtime_binds(base: &Path, root: &Path) { + let mounts = mounts_below(root); let entry = || Entry::new(root, Identity::new(0, 0).account(AccountMode::Numeric)); let generated = |path: PathBuf| { entry() @@ -1053,6 +1309,11 @@ fn check_runtime_binds(base: &Path, root: &Path) { matches!(result, Err(Error::TimeStatus(MappingError::SourceKind))), "{result:?}" ); + let result = entry().resolver(base.join("time")).bind_runtime(); + assert!( + matches!(result, Err(Error::Resolver(MappingError::SourceKind))), + "{result:?}" + ); // A target that the image holds as another kind, or below a link, is refused. std::fs::create_dir(root.join("etc/machine-id")).unwrap(); let result = generated(base.join("machine-id")); @@ -1069,6 +1330,22 @@ fn check_runtime_binds(base: &Path, root: &Path) { "{result:?}" ); std::fs::remove_file(root.join("run")).unwrap(); + // A failure after the first binds removes them. + std::fs::create_dir(root.join("etc/resolv.conf")).unwrap(); + let result = entry() + .machine_id(MachineId::Generated(base.join("machine-id"))) + .time_status(base.join("time")) + .resolver(base.join("resolv.conf")) + .bind_runtime(); + assert!( + matches!(result, Err(Error::Resolver(MappingError::TargetKind))), + "{result:?}" + ); + std::fs::remove_dir(root.join("etc/resolv.conf")).unwrap(); + assert_eq!(mounts_below(root), mounts); + // What the walk created stays, but for the machine ID's file, which the caller's test puts a + // link in place of. + std::fs::remove_file(root.join("etc/machine-id")).unwrap(); } /// Checks that an execution of `entry`, whose root is the privileged entries' mounted `root`, @@ -1195,26 +1472,38 @@ fn privileged_entries_see_their_own_view_and_identity() { let id = std::fs::read_to_string(base.path().join("machine-id")).unwrap(); std::fs::create_dir(base.path().join("time")).unwrap(); std::fs::write(base.path().join("time/state"), "time-state\n").unwrap(); + let resolver = std::fs::File::create(base.path().join("resolv.conf")).unwrap(); + write_resolver(resolver, "10.0.0.1".parse().unwrap()).unwrap(); let entry = |identity: Identity| { Entry::new(&root, identity) .hostname("entry-test") .machine_id(machine_id.clone()) .time_status(base.path().join("time")) + .resolver(base.path().join("resolv.conf")) }; check_runtime_binds(base.path(), &root); - // The image's link at the machine ID, which no bind covers, gives way to an empty - // file, and its target is left alone. + // The image's links at the machine ID and at the resolver file, which no bind + // covers, give way to empty files, and their targets are left alone. std::fs::write(base.path().join("outside"), "outside\n").unwrap(); std::os::unix::fs::symlink( base.path().join("outside"), root.join("etc/machine-id"), ) .unwrap(); + std::os::unix::fs::symlink( + base.path().join("outside"), + root.join("etc/resolv.conf"), + ) + .unwrap(); entry(Identity::new(65534, 65534)).bind_runtime().unwrap(); assert_eq!( std::fs::read_to_string(root.join("etc/machine-id")).unwrap(), id ); + assert_eq!( + std::fs::read_to_string(root.join("etc/resolv.conf")).unwrap(), + "nameserver 10.0.0.1\n" + ); assert_eq!( std::fs::read_to_string(base.path().join("outside")).unwrap(), "outside\n" @@ -1288,3 +1577,86 @@ fn privileged_entries_see_their_own_view_and_identity() { } } } + +#[test] +fn privileged_runtime_binds_are_removed_before_a_teardown() { + if crate::guest::managed_agent::isolate_process_test( + "NVX_ENTRY_TEARDOWN_TEST_CHILD", + "guest::entry::tests::privileged_runtime_binds_are_removed_before_a_teardown", + ) { + return; + } + if !sys::holds_capability(sys::CAP_SYS_ADMIN).unwrap() { + return; + } + // As for the entries' own test, a child of the harness's process makes the calls. + match sys::fork_isolated_test().unwrap() { + sys::ForkResult::Child => { + let outcome = std::panic::catch_unwind(|| { + use nix::sys::statvfs::{FsFlags, statvfs}; + + sys::enter_private_mount_namespace().unwrap(); + let directory = tempfile::tempdir().unwrap(); + let base = directory.path().canonicalize().unwrap(); + // A root that is a mount of its own, as a layered root's overlay is, whose + // resolver file is a link, as a resolver daemon's image has. + let root = base.join("root"); + std::fs::create_dir(&root).unwrap(); + let point = sys::path_c_string(&root).unwrap(); + sys::mount( + Some(c"tmpfs"), + &point, + Some(c"tmpfs"), + sys::MountFlags::empty(), + Some(c"mode=0755"), + ) + .unwrap(); + std::fs::create_dir(root.join("etc")).unwrap(); + std::os::unix::fs::symlink( + "../run/resolver/resolv.conf", + root.join("etc/resolv.conf"), + ) + .unwrap(); + let machine_id = MachineId::generate(base.join("machine-id")).unwrap(); + std::fs::create_dir(base.join("time")).unwrap(); + let resolver = std::fs::File::create(base.join("resolv.conf")).unwrap(); + write_resolver(resolver, "fd00::a00:1".parse().unwrap()).unwrap(); + let binds = Entry::new(&root, Identity::new(0, 0).account(AccountMode::Numeric)) + .machine_id(machine_id) + .time_status(base.join("time")) + .resolver(base.join("resolv.conf")) + .bind_runtime() + .unwrap(); + // Each bind is read-only, `nosuid`, and `nodev`. + let targets = ["etc/machine-id", "run/nvx/time", "etc/resolv.conf"] + .map(|target| root.join(target)); + assert_eq!(mounts_below(&root), targets); + for target in &targets { + let flags = statvfs(target).unwrap().flags(); + assert!( + flags.contains(FsFlags::ST_RDONLY | FsFlags::ST_NOSUID | FsFlags::ST_NODEV), + "{}: {flags:?}", + target.display() + ); + } + assert_eq!( + std::fs::read_to_string(root.join("etc/resolv.conf")).unwrap(), + "nameserver fd00::a00:1\n" + ); + // A teardown that finds them fails, since it detaches no busy mount. + assert_eq!(sys::unmount(&point), Err(Errno::EBUSY)); + binds.remove().unwrap(); + assert!(mounts_below(&root).is_empty()); + // The image's link gave way to an empty file, which stays. + let replaced = std::fs::symlink_metadata(root.join("etc/resolv.conf")).unwrap(); + assert!(replaced.is_file() && replaced.len() == 0); + // Once they are removed, the teardown succeeds. + sys::unmount(&point).unwrap(); + }); + sys::exit_immediately(u8::from(outcome.is_err())); + } + sys::ForkResult::Parent(child) => { + assert_eq!(sys::wait(child), Some(ChildStatus::Exited(0))); + } + } +} diff --git a/nvx-rs/src/guest/managed_agent/dispatch.rs b/nvx-rs/src/guest/managed_agent/dispatch.rs index 4f406d6e0..467a7856a 100644 --- a/nvx-rs/src/guest/managed_agent/dispatch.rs +++ b/nvx-rs/src/guest/managed_agent/dispatch.rs @@ -736,6 +736,7 @@ mod tests { b"/dev/shm", b"/.nvx-agent", b"/etc/machine-id", + b"/etc/resolv.conf", b"/run/nvx/time/state", b"/run/nvx", b"/run", diff --git a/nvx-rs/src/guest/managed_agent/seam.rs b/nvx-rs/src/guest/managed_agent/seam.rs index 013ce0485..39c5ce521 100644 --- a/nvx-rs/src/guest/managed_agent/seam.rs +++ b/nvx-rs/src/guest/managed_agent/seam.rs @@ -192,13 +192,24 @@ impl System for RealSystem { fn bind_runtime(&mut self, entry: &Entry) -> Result<(), Errno> { sys::enter_private_mount_namespace()?; - entry.bind_runtime().map_err(|error| match error { - entry::Error::Root(error) => sys::errno_of(&error), - entry::Error::MachineId(error) | entry::Error::TimeStatus(error) => { - Errno::from_raw(error.raw_os_error()) - } - _ => Errno::EINVAL, - }) + // The binds end with the agent's mount namespace, before the init agent unmounts the + // root, so the agent leaves them mounted. + entry + .bind_runtime() + .map(drop) + .map_err(|error| runtime_errno(&error)) + } +} + +/// Returns the errno of a failure of [`Entry::bind_runtime`], that of the bind that failed. +fn runtime_errno(error: &entry::Error) -> Errno { + match error { + entry::Error::Root(error) => sys::errno_of(error), + entry::Error::MachineId(error) + | entry::Error::TimeStatus(error) + | entry::Error::Resolver(error) => Errno::from_raw(error.raw_os_error()), + entry::Error::Rollback { error, .. } => runtime_errno(error), + _ => Errno::EINVAL, } } diff --git a/nvx-rs/src/guest/mappings.rs b/nvx-rs/src/guest/mappings.rs index 10e597019..2490a0e44 100644 --- a/nvx-rs/src/guest/mappings.rs +++ b/nvx-rs/src/guest/mappings.rs @@ -4,8 +4,9 @@ //! export; a target below a target root, the tree that the mappings go into; and the target's //! [`Access`]. A [`Mapper`] applies mappings beneath one [`TargetRoot`], in order, from a //! [`SourceRoot`], tracks them, and unmounts them at its [`teardown`](Mapper::teardown). [`bind`] -//! applies one runtime bind, such as a generated machine ID or the runtime's time status, through -//! the same walk and attach, without the checks of a host mapping's target and without tracking. +//! applies one runtime bind, such as a generated machine ID, a resolver file, or the runtime's time +//! status, through the same walk and attach, without the checks of a host mapping's target and +//! without tracking. //! //! # Walk //! @@ -364,8 +365,8 @@ impl Reserved { /// Returns the targets that NVX's container runtime reserves in a container root, which /// [`protocol::share`](crate::protocol::share) lists for the host and the guest: the trees - /// `/proc`, `/sys`, `/dev`, `/.nvx-agent`, `/etc/machine-id`, and `/run/nvx/time`, and the - /// paths `/etc`, `/run`, and `/run/nvx`. + /// `/proc`, `/sys`, `/dev`, `/.nvx-agent`, `/etc/machine-id`, `/etc/resolv.conf`, and + /// `/run/nvx/time`, and the paths `/etc`, `/run`, and `/run/nvx`. pub fn container_root() -> Self { Self::new(RESERVED_MOUNT_TARGETS, RESERVED_EXACT_MOUNT_TARGETS) .expect("the container runtime's reserved targets are canonical") @@ -728,15 +729,24 @@ impl fmt::Display for TeardownError { impl std::error::Error for TeardownError {} -/// A mapping that a mapper attached and tracks. +/// A mapping that a mapper attached and tracks, or a runtime bind that [`bind_with`] made, which +/// [`unbind_with`] unmounts. #[derive(Clone, Debug, PartialEq, Eq)] -struct Mapped { +pub(crate) struct Mapped { /// The target, relative to the root. target: PathBuf, /// The unique ID of the mapping's mount. mount: u64, } +#[cfg(test)] +impl Mapped { + /// Returns the target, relative to the root. + pub(crate) fn target(&self) -> &Path { + &self.target + } +} + /// A mapping that a rename moved away from its target while the walk ran, which the attach /// failed to detach, and which stays mounted wherever it went. #[derive(Debug)] @@ -959,16 +969,16 @@ impl Mapper { /// So does a bind that a rename moved and that it failed to detach, wherever it went, which it /// reports as [`MappingError::Moved`] with `mounted`. It needs the privilege to mount. pub fn bind(root: &TargetRoot, source: &SourceRoot, mapping: &Mapping) -> Result<(), MappingError> { - bind_with(&mut SystemKernel, root, source, mapping) + bind_with(&mut SystemKernel, root, source, mapping).map(drop) } -/// Implements [`bind`] with `kernel`. +/// Implements [`bind`] with `kernel`, and returns the bind, which [`unbind_with`] unmounts. pub(crate) fn bind_with( kernel: &mut dyn Kernel, root: &TargetRoot, source: &SourceRoot, mapping: &Mapping, -) -> Result<(), MappingError> { +) -> Result { attach( kernel, root, @@ -978,10 +988,36 @@ pub(crate) fn bind_with( mapping.access(), mapping.kind(), ) - .map(drop) + .map(|mount| Mapped { + target: mapping.target().to_path_buf(), + mount, + }) .map_err(|failure| failure.error) } +/// Unmounts `binds`, which [`bind_with`] made beneath `root`, in reverse order, with `kernel`, as a +/// mapper's teardown unmounts its mappings: each that is the topmost mount at its target, by its +/// name in its parent directory, without detaching a busy one. It attempts each, whatever failed +/// before, and reports every one that it failed to unmount, which stays mounted. +pub(crate) fn unbind_with( + kernel: &mut dyn Kernel, + root: &TargetRoot, + binds: &[Mapped], +) -> Result<(), TeardownError> { + let mut failures = Vec::new(); + for bind in binds.iter().rev() { + if let Err(error) = unmount(kernel, root, bind) { + let target = absolute(&bind.target); + failures.push(UnmountFailure { target, error }); + } + } + if failures.is_empty() { + Ok(()) + } else { + Err(TeardownError { failures }) + } +} + /// What a mapping attaches: the object at a path beneath a source root, whose mount the attach /// copies, or a mount that it moves. #[derive(Clone, Copy, Debug)] diff --git a/nvx-rs/src/guest/mappings/privileged.rs b/nvx-rs/src/guest/mappings/privileged.rs index ec98dc688..92851c3d3 100644 --- a/nvx-rs/src/guest/mappings/privileged.rs +++ b/nvx-rs/src/guest/mappings/privileged.rs @@ -362,7 +362,13 @@ fn kinds_nested_targets_and_reserved_targets_are_checked() { "{target}" ); } - for target in ["/etc", "/etc/machine-id", "/proc/x", "/dev"] { + for target in [ + "/etc", + "/etc/machine-id", + "/etc/resolv.conf", + "/proc/x", + "/dev", + ] { assert_eq!( scene.refusal(Reserved::container_root(), "dir", target), MappingError::Reserved, diff --git a/nvx-rs/src/guest/mappings/tests.rs b/nvx-rs/src/guest/mappings/tests.rs index b4ec03a91..0b05ab275 100644 --- a/nvx-rs/src/guest/mappings/tests.rs +++ b/nvx-rs/src/guest/mappings/tests.rs @@ -511,6 +511,7 @@ fn reserved_targets_and_their_ancestors_are_refused() { "/.nvx-agent/tool", "/etc", "/etc/machine-id", + "/etc/resolv.conf", "/run", "/run/nvx", "/run/nvx/time/x", @@ -524,6 +525,7 @@ fn reserved_targets_and_their_ancestors_are_refused() { for target in [ "/etc/app", "/etc/machine-ids", + "/etc/resolvconf", "/procs", "/run/app", "/workspace", @@ -927,6 +929,82 @@ fn a_runtime_bind_skips_the_checks_of_a_host_mapping_but_not_the_walk() { assert_eq!(mapper.targets().len(), 0); } +#[test] +fn runtime_binds_unbind_in_reverse_order_and_report_each_failure() { + let fixture = Fixture::new(); + let root = fixture.root(); + let source = fixture.source(); + let bind_all = || { + [ + ("file.txt", "/etc/machine-id"), + ("dir", "/run/nvx/time"), + ("file.txt", "/etc/resolv.conf"), + ] + .map(|(source_path, target)| { + let mapping = mapping(source_path, target, Access::ReadOnly); + bind_with(&mut fixture.kernel(), &root, &source, &mapping).unwrap() + }) + }; + let points = [ + "root/etc/resolv.conf", + "root/run/nvx/time", + "root/etc/machine-id", + ] + .map(|point| fixture.path(point)); + // Each bind is the mount at its target, which the unbind takes off, last first. + let binds = bind_all(); + assert_eq!( + binds.iter().map(Mapped::target).collect::>(), + ["etc/machine-id", "run/nvx/time", "etc/resolv.conf"].map(Path::new) + ); + for (bind, point) in binds.iter().rev().zip(&points) { + assert_eq!(fixture.kernel.mount_at(point), Some(bind.mount)); + } + unbind_with(&mut fixture.kernel(), &root, &binds).unwrap(); + assert_eq!( + fixture.kernel.unmounts(), + points.clone().map(|point| point.display().to_string()) + ); + assert!( + points + .iter() + .all(|point| fixture.kernel.mount_at(point).is_none()) + ); + // Each is attempted, whatever failed before, and one that is busy, or that another mount + // covers, stays and is reported. + let binds = bind_all(); + fixture + .kernel + .fail(Operation::Unmount, &points[0], Errno::EBUSY); + let covering = fixture.kernel.add_mount(&points[1]); + let unmounts = fixture.kernel.unmounts().len(); + let error = unbind_with(&mut fixture.kernel(), &root, &binds).unwrap_err(); + assert_eq!( + error.failures(), + [ + UnmountFailure { + target: "/etc/resolv.conf".into(), + error: failed(Step::Unmount, Errno::EBUSY), + }, + UnmountFailure { + target: "/run/nvx/time".into(), + error: MappingError::Moved { mounted: true }, + }, + ] + ); + assert_eq!( + fixture.kernel.unmounts()[unmounts..], + [&points[0], &points[2]].map(|point| point.display().to_string()) + ); + assert_eq!(fixture.kernel.mount_at(&points[0]), Some(binds[2].mount)); + assert_eq!(fixture.kernel.mount_at(&points[1]), Some(covering)); + assert_eq!(fixture.kernel.mount_at(&points[2]), None); + // No binds, nothing to unbind. + let calls = fixture.kernel.calls().len(); + unbind_with(&mut fixture.kernel(), &root, &[]).unwrap(); + assert_eq!(fixture.kernel.calls().len(), calls); +} + #[test] fn a_staged_mount_moves_onto_its_target_with_its_restrictions() { let fixture = Fixture::new(); diff --git a/nvx-rs/src/guest/root.rs b/nvx-rs/src/guest/root.rs index d62c54a9e..ae049ea49 100644 --- a/nvx-rs/src/guest/root.rs +++ b/nvx-rs/src/guest/root.rs @@ -81,8 +81,8 @@ //! - holds no `\`, `=`, `,`, or `"`; //! - is not where a container runtime mounts or binds in the root, as //! [`protocol::share`](crate::protocol::share) lists them for the host and the guest: `/proc`, -//! `/sys`, `/dev`, `/.nvx-agent`, `/etc/machine-id`, `/run/nvx/time`, or below them, nor `/etc`, -//! `/run`, or `/run/nvx`, below which it mounts; +//! `/sys`, `/dev`, `/.nvx-agent`, `/etc/machine-id`, `/etc/resolv.conf`, `/run/nvx/time`, or +//! below them, nor `/etc`, `/run`, or `/run/nvx`, below which it mounts; //! - and is not another share's or mapping's target, nor inside one, nor holds one. //! //! The root's [`Mapper`] then attaches the shares beneath the overlay's descriptor, as diff --git a/nvx-rs/src/guest/root/share.rs b/nvx-rs/src/guest/root/share.rs index 54229e8c8..363e62422 100644 --- a/nvx-rs/src/guest/root/share.rs +++ b/nvx-rs/src/guest/root/share.rs @@ -688,6 +688,7 @@ mod tests { "/.hidden", "/etc/app", "/etc/machine-ids", + "/etc/resolv.conf.d", "/run/app", "/run/nvx/state", "/run/nvx/timer", @@ -736,6 +737,8 @@ mod tests { "/etc", "/etc/machine-id", "/etc/machine-id/x", + "/etc/resolv.conf", + "/etc/resolv.conf/x", "/run", "/run/nvx", "/run/nvx/time", diff --git a/nvx-rs/src/host/filesystem.rs b/nvx-rs/src/host/filesystem.rs index 8088c237f..daab0f329 100644 --- a/nvx-rs/src/host/filesystem.rs +++ b/nvx-rs/src/host/filesystem.rs @@ -3139,6 +3139,10 @@ mod tests { &[at(&work, "/proc/self", true)], "would cover the guest's /proc", ); + refused( + &[at(&config, "/etc/resolv.conf", true)], + "has the guest target /etc/resolv.conf, which would cover the guest's /etc/resolv.conf", + ); // The same object read-only and read-write at two targets would let the guest write it. refused( &[at(&work, "/a", false), at(&work, "/b", true)], @@ -3181,6 +3185,7 @@ mod tests { ("/.nvx-agent/tools", "/.nvx-agent"), ("/etc", "/etc"), ("/etc/machine-id", "/etc/machine-id"), + ("/etc/resolv.conf", "/etc/resolv.conf"), ("/run", "/run"), // The ancestors of the runtime's trees are reserved, as exact paths here. ("/run/nvx", "/run/nvx"), @@ -3190,6 +3195,7 @@ mod tests { } for target in [ "/etc/app", + "/etc/resolvconf", "/run/secrets", "/run/nvx/other", "/usr/local", diff --git a/nvx-rs/src/host/sandbox/mount/tests.rs b/nvx-rs/src/host/sandbox/mount/tests.rs index b5922d75b..c376b64a6 100644 --- a/nvx-rs/src/host/sandbox/mount/tests.rs +++ b/nvx-rs/src/host/sandbox/mount/tests.rs @@ -548,6 +548,9 @@ fn mount_target_validation_rejects_unsafe_and_reserved_targets() { ("/dev/shm", "/dev"), ("/.nvx-agent", "/.nvx-agent"), ("/.nvx-agent/tools", "/.nvx-agent"), + ("/etc/machine-id", "/etc/machine-id"), + ("/etc/resolv.conf", "/etc/resolv.conf"), + ("/etc/resolv.conf/x", "/etc/resolv.conf"), ] { assert_eq!( message(validate_mount_target(target)), @@ -577,6 +580,7 @@ fn mount_target_validation_rejects_unsafe_and_reserved_targets() { "/workspace", "/procfs", "/etc/app", + "/etc/resolvconf", "/opt/hostedtoolcache", "/run/secrets", "/run/nvx/other", diff --git a/nvx-rs/src/host/sandbox/plan/tests.rs b/nvx-rs/src/host/sandbox/plan/tests.rs index 6cdf283e6..c88c88ffd 100644 --- a/nvx-rs/src/host/sandbox/plan/tests.rs +++ b/nvx-rs/src/host/sandbox/plan/tests.rs @@ -527,6 +527,10 @@ fn plans_check_their_export_plan_for_the_container_root() { |export| export.binds[1].target = "/proc/self".to_owned(), "its guest target /proc/self would cover the guest's /proc", ), + ( + |export| export.binds[1].target = "/etc/resolv.conf".to_owned(), + "its guest target /etc/resolv.conf would cover the guest's /etc/resolv.conf", + ), ( |export| export.binds[0].target = "workspace".to_owned(), "its guest target \"workspace\" is not absolute", diff --git a/nvx-rs/src/host/sandbox/specification.rs b/nvx-rs/src/host/sandbox/specification.rs index 05e91c205..6617f88e2 100644 --- a/nvx-rs/src/host/sandbox/specification.rs +++ b/nvx-rs/src/host/sandbox/specification.rs @@ -630,7 +630,15 @@ proptest! { #[test] fn reserved_mount_targets_are_refused(suffix in "[a-z][a-z0-9-]{0,20}") { - for tree in ["/proc", "/sys", "/dev", "/.nvx-agent", "/etc/machine-id", "/run/nvx/time"] { + for tree in [ + "/proc", + "/sys", + "/dev", + "/.nvx-agent", + "/etc/machine-id", + "/etc/resolv.conf", + "/run/nvx/time", + ] { refused(validate_mount_target(tree)); refused(validate_mount_target(&format!("{tree}/{suffix}"))); let sibling = format!("{tree}-{suffix}"); diff --git a/nvx-rs/src/host/sandbox/tests.rs b/nvx-rs/src/host/sandbox/tests.rs index 399e385f7..6151d4cb7 100644 --- a/nvx-rs/src/host/sandbox/tests.rs +++ b/nvx-rs/src/host/sandbox/tests.rs @@ -736,11 +736,14 @@ fn launch_contract_attaches_a_file_as_an_aggregate_child() { message(MountKind::from_name("socket")), "unsupported sandbox mount kind \"socket\"; choose directory or file" ); - // The runtime replaces /etc/machine-id with a file, and binds its time state over - // /run/nvx/time, so nothing at or below them may be shared. + // The runtime replaces /etc/machine-id with a file, binds the guest's resolver file over + // /etc/resolv.conf, and binds its time state over /run/nvx/time, so nothing at or below them + // may be shared. for target in [ "/etc/machine-id", "/etc/machine-id/child", + "/etc/resolv.conf", + "/etc/resolv.conf/child", "/run/nvx/time", "/run/nvx/time/state", ] { diff --git a/nvx-rs/src/protocol/cmdline.rs b/nvx-rs/src/protocol/cmdline.rs index fdc60128d..330d4c5ab 100644 --- a/nvx-rs/src/protocol/cmdline.rs +++ b/nvx-rs/src/protocol/cmdline.rs @@ -1,5 +1,6 @@ //! The kernel command line as Linux reads it: the parameters into which the kernel splits it, and -//! the virtio-mmio devices that its `virtio_mmio.device` parameters describe. +//! the virtio-mmio devices that its `virtio_mmio.device` parameters describe; and the guest's DNS +//! server, which OpenVMM names in it. //! //! A reader that splits a command line by rules of its own may find parameters that the kernel //! never sees, or miss ones that it applies: the kernel splits at more bytes than ASCII @@ -14,14 +15,31 @@ //! virtio-mmio driver's `vm_cmdline_set` reads it, for a device of any type, and //! [`virtio_mmio_devices`] reads those of every such parameter of a command line. //! +//! The kernel leaves OpenVMM's `virtnet_dns=` token to the guest, which reads it from the whole +//! command line, as OpenVMM checks it: +//! +//! - [`dns_server`] reads the DNS server that the token names, strictly, as OpenVMM renders and +//! reads it: at most one token, with one IPv4 or IPv6 address. +//! - [`write_resolver`] writes the resolver file that names that server, as NVX's init writes the +//! initramfs's `/etc/resolv.conf` from the token. +//! //! Like the rest of [`protocol`](super), it needs nothing outside the standard library, so the //! host, the guest, and test doubles share it. +use std::io; use std::iter::FusedIterator; +use std::net::IpAddr; + +use super::{ProtocolError, violation}; /// The name of the kernel parameter that adds a virtio-mmio device. pub const VIRTIO_MMIO_DEVICE: &str = "virtio_mmio.device"; +/// Prefix of the token through which OpenVMM names the guest's DNS server, `virtnet_dns=`: one of +/// the gateway's addresses, IPv4 or IPv6, which it adds to a microVM's command line when the +/// network policy lets the guest reach the gateway's DNS service. +pub const DNS_TOKEN: &str = "virtnet_dns="; + /// Returns whether the kernel takes `byte` for whitespace, as its `isspace` does: the space, tab, /// newline, vertical tab, form feed, carriage return, and Latin-1's no-break space, 0xa0. pub const fn is_space(byte: u8) -> bool { @@ -212,6 +230,52 @@ pub fn virtio_mmio_devices(command_line: &[u8]) -> impl Iterator Result, ProtocolError> { + let mut server = None; + for token in command_line.split(u8::is_ascii_whitespace) { + let Some(value) = token.strip_prefix(DNS_TOKEN.as_bytes()) else { + continue; + }; + if server.is_some() { + return Err(violation("the virtnet_dns kernel parameter is repeated")); + } + let address = std::str::from_utf8(value).ok().and_then(|text| { + text.parse::() + .ok() + .filter(|address| address.to_string() == text) + }); + let Some(address) = address else { + return Err(violation(format!( + "the virtnet_dns kernel parameter must name one IPv4 or IPv6 address as OpenVMM \ + renders it, not '{}'", + value.escape_ascii() + ))); + }; + server = Some(address); + } + Ok(server) +} + +/// Writes to `writer` a resolver file, in the format of `/etc/resolv.conf`, that names `server` as +/// the guest's one DNS server: `nameserver ADDRESS` and a newline, with the address in its +/// canonical text, which [`dns_server`] reads, as NVX's init writes the initramfs's +/// `/etc/resolv.conf` from the `virtnet_dns=` token. +pub fn write_resolver(mut writer: impl io::Write, server: IpAddr) -> io::Result<()> { + writer.write_all(format!("nameserver {server}\n").as_bytes()) +} + /// Reads an integer conversion of the kernel's `sscanf` at the start of `text`, `%i` with `base` /// 0, or `%u` or `%d` with `base` 10: after any whitespace and, if `signed`, an optional minus /// sign, the integer that [`integer`] reads, which must start with a decimal digit and which the diff --git a/nvx-rs/src/protocol/cmdline/tests.rs b/nvx-rs/src/protocol/cmdline/tests.rs index 0ecf56373..bd5402de4 100644 --- a/nvx-rs/src/protocol/cmdline/tests.rs +++ b/nvx-rs/src/protocol/cmdline/tests.rs @@ -1,12 +1,21 @@ -//! Tests of the kernel command line as Linux reads it. The expectations follow Linux 6.18: -//! `lib/cmdline.c`, `kernel/params.c`, `lib/vsprintf.c`, `lib/kstrtox.c`, and -//! `drivers/virtio/virtio_mmio.c`. +//! Tests of the kernel command line as Linux reads it, and of the DNS server that OpenVMM names in +//! it. The expectations follow Linux 6.18: `lib/cmdline.c`, `kernel/params.c`, `lib/vsprintf.c`, +//! `lib/kstrtox.c`, and `drivers/virtio/virtio_mmio.c`; and, for the DNS server, OpenVMM's +//! rendering and reading of its `virtnet_dns=` token, `openvmm_defs::microvm`'s +//! `command_line_fragment_with` and `microvm_command_line_dns_server`, whose vectors they take. use super::*; /// The MMIO base of the scratch block device of a layered root. const SCRATCH: u64 = 0xd000_6000; +/// The network tokens that OpenVMM renders for the guest `10.0.0.2/24`, with the IPv6 identity +/// that it derives from it, before and after the place of a `virtnet_dns=` token. +const NETWORK: (&str, &str) = ( + "virtnet_ip=10.0.0.2 virtnet_mask=255.255.255.0 virtnet_gw=10.0.0.1", + "virtnet_ip6=fd00::a00:2/120 virtnet_gw6=fd00::a00:1", +); + /// Checks that the kernel splits `command_line` into the parameters `expected`. fn assert_parameters(command_line: &[u8], expected: &[(&str, Option<&str>)]) { let expected: Vec<_> = expected @@ -228,3 +237,136 @@ fn devices_come_from_the_parameters_that_the_kernel_reads() { ); assert_eq!(virtio_mmio_devices(b"").count(), 0); } + +#[test] +fn the_dns_server_is_read_as_openvmm_renders_it() { + let ipv4: IpAddr = "10.0.0.1".parse().unwrap(); + let ipv6: IpAddr = "fd00::a00:1".parse().unwrap(); + // OpenVMM names either gateway, in its canonical text, or none, among its network tokens, and + // the kernel appends a newline to `/proc/cmdline`. + let (before, after) = NETWORK; + for (dns, expected) in [ + (" virtnet_dns=10.0.0.1", Some(ipv4)), + (" virtnet_dns=fd00::a00:1", Some(ipv6)), + ("", None), + ] { + let command_line = format!("console=hvc0 {before}{dns} {after}\n"); + assert_eq!( + dns_server(command_line.as_bytes()), + Ok(expected), + "{command_line}" + ); + } + // A token is one that starts with `virtnet_dns=` between ASCII whitespace, without quoting, + // as OpenVMM splits the command line, and past a `--`, where the kernel stops. + for command_line in [ + &b"virtnet_dns=10.0.0.1"[..], + b"a\tvirtnet_dns=10.0.0.1\nb", + b"a\x0cvirtnet_dns=10.0.0.1\rb", + b"quiet -- virtnet_dns=10.0.0.1", + b"\"a virtnet_dns=10.0.0.1", + ] { + assert_eq!( + dns_server(command_line), + Ok(Some(ipv4)), + "{}", + command_line.escape_ascii() + ); + } + for command_line in [ + &b""[..], + b"virtnet_dns", + b"virtnet_dns =10.0.0.1", + b"virtnet-dns=10.0.0.1", + b"VIRTNET_DNS=10.0.0.1", + b"xvirtnet_dns=10.0.0.1", + b"\"virtnet_dns=10.0.0.1\"", + b"a=virtnet_dns=10.0.0.1", + ] { + assert_eq!( + dns_server(command_line), + Ok(None), + "{}", + command_line.escape_ascii() + ); + } +} + +#[test] +fn the_dns_server_is_one_address_in_one_token() { + // OpenVMM's refusals of the token itself, and the spellings that it never renders: its reader + // refuses each of them too. A vertical tab or a no-break space ends a parameter for the kernel, + // but not a token for OpenVMM. + for command_line in [ + &b"virtnet_dns=fd00:0::a00:1"[..], + b"virtnet_dns=010.0.0.1", + b"virtnet_dns=10.0.0.1,fd00::a00:1", + b"virtnet_dns=10.0.0.1 virtnet_dns=fd00::a00:1", + b"virtnet_dns=10.0.0.1 virtnet_dns=10.0.0.1", + b"virtnet_dns=10.0.0.1 virtnet_dns=", + b"virtnet_dns=bad virtnet_dns=10.0.0.1", + b"virtnet_dns=", + b"virtnet_dns= 10.0.0.1", + b"virtnet_dns=FD00::A00:1", + b"virtnet_dns=fd00::0a00:1", + b"virtnet_dns=10.0.0.1:53", + b"virtnet_dns=[fd00::a00:1]", + b"virtnet_dns=fd00::a00:1%eth0", + b"virtnet_dns=10.0.0.1/24", + b"virtnet_dns=10.0.0", + b"virtnet_dns=\"10.0.0.1\"", + b"virtnet_dns=10.0.0.1\x0b", + b"virtnet_dns=10.0.0.1\xa0", + b"virtnet_dns=gateway", + ] { + assert!( + dns_server(command_line).is_err(), + "{}", + command_line.escape_ascii() + ); + } + assert_eq!( + dns_server(b"virtnet_dns=10.0.0.1 virtnet_dns=fd00::a00:1") + .unwrap_err() + .message(), + "the virtnet_dns kernel parameter is repeated" + ); + assert_eq!( + dns_server(b"quiet virtnet_dns=010.0.0.1\xa0") + .unwrap_err() + .message(), + "the virtnet_dns kernel parameter must name one IPv4 or IPv6 address as OpenVMM renders \ + it, not '010.0.0.1\\xa0'" + ); +} + +#[test] +fn a_resolver_file_names_the_server_as_the_init_writes_it() { + for (server, file) in [ + ("10.0.0.1", "nameserver 10.0.0.1\n"), + ("fd00::a00:1", "nameserver fd00::a00:1\n"), + // An address in another spelling is written in its canonical text. + ("fd00:0:0::0a00:1", "nameserver fd00::a00:1\n"), + ] { + let server: IpAddr = server.parse().unwrap(); + let mut written = Vec::new(); + write_resolver(&mut written, server).unwrap(); + assert_eq!(String::from_utf8(written).unwrap(), file); + // The text is the one that the reader takes from OpenVMM's token. + let command_line = format!("{DNS_TOKEN}{}", file["nameserver ".len()..].trim_end()); + assert_eq!(dns_server(command_line.as_bytes()), Ok(Some(server))); + } + // The writer's failure is the call's. + struct Full; + impl io::Write for Full { + fn write(&mut self, _: &[u8]) -> io::Result { + Err(io::Error::other("the disk is full")) + } + + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } + } + let error = write_resolver(Full, "10.0.0.1".parse().unwrap()).unwrap_err(); + assert_eq!(error.to_string(), "the disk is full"); +} diff --git a/nvx-rs/src/protocol/mod.rs b/nvx-rs/src/protocol/mod.rs index 9b5c03245..2710670eb 100644 --- a/nvx-rs/src/protocol/mod.rs +++ b/nvx-rs/src/protocol/mod.rs @@ -17,7 +17,8 @@ //! such as [`MAPS_TOKEN`]. //! - [`cmdline`]: the kernel command line as Linux reads it, for the readers that need the //! kernel's view of it: the parameters into which the kernel splits it, and the virtio-mmio -//! devices, of any type, that its `virtio_mmio.device` parameters describe. +//! devices, of any type, that its `virtio_mmio.device` parameters describe; and the guest's DNS +//! server, which OpenVMM's `virtnet_dns=` token names, with the resolver file that names it. //! - [`root`]: the boot tokens of a layered root, which name its image's format and identity and //! its upper, with the host's renderer and the guest's parser. //! - [`export`]: the boot tokens of the host export, the aggregate virtio-fs device from which a diff --git a/nvx-rs/src/protocol/share.rs b/nvx-rs/src/protocol/share.rs index 48612e4f1..825b59e5b 100644 --- a/nvx-rs/src/protocol/share.rs +++ b/nvx-rs/src/protocol/share.rs @@ -8,13 +8,16 @@ /// The guest trees that the container runtime owns, which a share may neither be in nor contain: /// it mounts procfs, sysfs, and a private `/dev` over them, stages its tools under /// `/.nvx-agent`, binds the workload machine ID at `/etc/machine-id`, which it first replaces -/// with a regular file, and binds its time ABI state over `/run/nvx/time`. -pub const RESERVED_MOUNT_TARGETS: [&str; 6] = [ +/// with a regular file, binds the guest's resolver file at `/etc/resolv.conf`, which it first +/// replaces with a regular file if it is a symbolic link, and binds its time ABI state over +/// `/run/nvx/time`. +pub const RESERVED_MOUNT_TARGETS: [&str; 7] = [ "/proc", "/sys", "/dev", "/.nvx-agent", "/etc/machine-id", + "/etc/resolv.conf", "/run/nvx/time", ]; @@ -72,6 +75,7 @@ mod tests { "/dev", "/.nvx-agent", "/etc/machine-id", + "/etc/resolv.conf", "/run/nvx/time" ] ); @@ -89,6 +93,8 @@ mod tests { ("/.nvx-agent/tool", "/.nvx-agent"), ("/etc/machine-id", "/etc/machine-id"), ("/etc/machine-id/x", "/etc/machine-id"), + ("/etc/resolv.conf", "/etc/resolv.conf"), + ("/etc/resolv.conf/x", "/etc/resolv.conf"), ("/run/nvx/time", "/run/nvx/time"), ("/run/nvx/time/state", "/run/nvx/time"), ] { @@ -112,6 +118,8 @@ mod tests { "/.nvx-agents", "/etc/app", "/etc/machine-ids", + "/etc/resolv.conf.d", + "/etc/resolvconf", "/run/app", "/run/nvx/state", "/run/nvx/timer",