From 91d234b73cd56a19b20203ab56749d664b006e90 Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Sat, 24 Oct 2026 00:16:14 -0700 Subject: [PATCH 1/2] nvx-rs: mount the host export as a source of a root's mappings A guest that maps host paths into a layered root from the host's mapping table needs the host export, the aggregate virtio-fs device that OpenVMM attaches for those paths, mounted outside the root. NVX's shell init mounts it for its own agent, but a guest built on nvx::guest had no way to mount it as the source of Root::map. - nvx::protocol::export: HostExport::parse, a strict parser of the virtfs_dir=, virtfs_tag=, virtfs_mode=, and virtfs_aggregate=1 tokens that OpenVMM appends for the export: all four or none, each once and in its form, with the checks of the shell init's mount_host_export, and no nvx_share= token. EXPORT_DIRECTORY is the export's guest directory, which nvx::host::filesystem::GUEST_EXPORT now names, so the host's plan and the guest's parser share it. tests/fixtures/export-tokens.json holds the parser's cases. - Root::mount_export mounts the export nosuid,nodev, read-only or read-write, at hostfs/root in the runtime directory, below a directory that only root may enter, owns the mount, unmounts it after the mappings at teardown, and returns its SourceRoot for Root::map. A root mounts one export; a failure latches the root, and an assembly refuses a runtime directory where an export is mounted already. Part of #822. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3d31a6be-a0f8-4952-8667-0ebdc440d08d --- nvx-rs/README.md | 3 +- nvx-rs/src/guest/root.rs | 108 +++++++--- nvx-rs/src/guest/root/share.rs | 158 +++++++++++++- nvx-rs/src/guest/root/tests.rs | 260 ++++++++++++++++++++++- nvx-rs/src/host/filesystem.rs | 8 +- nvx-rs/src/protocol/export.rs | 232 ++++++++++++++++++++ nvx-rs/src/protocol/export/tests.rs | 207 ++++++++++++++++++ nvx-rs/src/protocol/mod.rs | 3 + nvx-rs/tests/fixtures/export-tokens.json | 61 ++++++ 9 files changed, 998 insertions(+), 42 deletions(-) create mode 100644 nvx-rs/src/protocol/export.rs create mode 100644 nvx-rs/src/protocol/export/tests.rs create mode 100644 nvx-rs/tests/fixtures/export-tokens.json diff --git a/nvx-rs/README.md b/nvx-rs/README.md index e5b49919a..7af6f9b99 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -67,6 +67,7 @@ nvx-rs/ | --- | --- | | `nvx::protocol` | The codec of the control console: `NVXS` outer records, `NVXC` application frames, the `EXEC` payload that the host encodes and the guest agent decodes, the `MAPS` payloads of the host mapping table, `GuestFeatures`, and the `nvx_maps=` and `nvx_workload_account=create` tokens. The [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/nvxc-v1.txt`, and OpenVMM's vectors and cases of the `NVXS` session pin every record, frame, and payload independently of the codec, which the host, the guest agent, and the fake OpenVMM share | | `nvx::protocol::cmdline` | The kernel command line as Linux 6.18 reads it, for the readers that need the kernel's view of it: `parameters` splits it into `Parameter`s as `parse_args` and `next_arg` do, at the bytes that `is_space` accepts, with double quotes, up to a NUL or `--`, and `Parameter::is_named` compares names as `parameq` does, with dashes and underscores alike. `VirtioMmioDevice::parse` reads a `virtio_mmio.device` value, `SIZE@BASE:IRQ[:ID]`, as the virtio-mmio driver reads it, for a device of any type, and `virtio_mmio_devices` reads every such parameter of a command line | +| `nvx::protocol::export` | The boot tokens of the host export, the aggregate virtio-fs device from which a guest maps host paths, which OpenVMM appends to the kernel command line: `virtfs_dir=` the export's guest directory, `EXPORT_DIRECTORY`, `/run/nvx/hostfs/root`, where the host's `nvx::host::filesystem` plan exports it, `virtfs_tag=` the device's tag, `virtfs_mode=` `ro` or `rw`, an `ExportMode`, and `virtfs_aggregate=1`. `HostExport::parse` reads them as the guest's init splits the command line: all four tokens or none, each once and in its form, with the checks of NVX's shell init, and no `nvx_share=` token, which binds a live share. It returns the `HostExport`, the device's tag and mode, which `nvx::guest::root::Root::mount_export` mounts, or `None`. `tests/fixtures/export-tokens.json` holds its cases | | `nvx::protocol::nvxs` | The NVXS session, independent of `NVXC` and of any other application protocol: `RecordType`, `Record`, `encode_record`, and `RecordParser`, which parses records incrementally across partial reads with buffers bounded by one record; `HostSession` and `GuestSession`, the state machines of the host's and the guest's roles, which validate every record's type, identity, and sequence number, and the guest's receive window and credits, without I/O. An application message may span DATA records, and the application's own framing reassembles it | | `nvx::protocol::root` | The boot tokens of a layered root, which the host renders and guests parse by the same rules: a `RootLayout` is a `RootImage` on the `distro` block device, an EROFS filesystem or a GPT disk with the `nvx::uuid::Uuid` that the guest expects of it, if any, over a `RootUpper`, the ext4 scratch disk or RAM. `RootLayout::tokens` renders `nvx_layer=distro,0xd0003000,FORMAT[:IDENTITY]` and `nvx_scratch=0xd0006000,ext4` or `nvx_overlay_upper=ramfs`, and `RootLayout::parse` reads them from a kernel command line, and refuses malformed, repeated, and conflicting tokens and the earlier form without a format, with no fallback that probes the devices. It reads the virtio-mmio devices of the command line with `nvx::protocol::cmdline`, to refuse a RAM upper beside one at the scratch base. `tests/fixtures/root-tokens.json` holds its cases, which the tests of the guest's shell init replay too | | `nvx::protocol::share` | The guest targets that a sandbox's live shares may not take, which the host's `sandbox::mount` and the guest's `root` both refuse: `RESERVED_MOUNT_TARGETS`, the trees where the container runtime mounts or binds, `RESERVED_EXACT_MOUNT_TARGETS`, the directories that it creates, `RESERVED_TARGET_CHARACTERS`, and `reservation`, which tells why a target is reserved | @@ -103,7 +104,7 @@ nvx-rs/ | `nvx::guest::exec::binding` | `Binding`: an `nvx::exec::Runner` bound to an `Execution`, the glue that a guest adapter would otherwise write. The runner's read allowances become grants, its discards discards, the input that it accepted writes and then a close, its terminations a cancellation, a disconnection, or a termination, and its settlement a kill of the execution's cgroup and checks until no process occupies it; without a cgroup, nothing proves that the program's descendants are gone, so the execution is fenced. It feeds the execution's events back to the runner, statuses that a caller who reaps hands over with `observe` included, and never blocks: the caller polls its descriptors and its input writer until its deadline, and each call takes a bounded number of steps, so that no execution holds the caller's loop | | `nvx::guest::time` | x86_64: `TimeAbi`, the steps of NVX time ABI v1 as typed calls in the caller's process, for any guest init or agent: the cold-boot time step and, separately, the start of the time daemon in a child process that the caller supervises, the VM generation ID, a `CaptureRequest` that repairs a `Restore`, and a `RestoreFinish`. A call takes the permission for OpenVMM's ports on the calling thread for each port transaction alone, and fails with `TimeError::PortRelease` if the thread may still hold some of it, which a process that it started would inherit, and from which `nvx::guest::spawn` starts none. No call waits for a child that it did not start. `nvx-time` runs its steps through them | | `nvx::guest::reseed` | `Seed::read` and `reseed`: a restore's fresh entropy, which starts with its `GenerationId`, and the reseed of the kernel's random number generator with it, in the caller's process. `nvx-reseed` runs them | -| `nvx::guest::root` | The operations of a layered root, for a guest init or agent, whether or not it is PID 1. `bootstrap`: the agent's and the workload's sibling cgroups of a `Layout`, with every process of their parent moved into the agent's, the agent's `memory.low`, and the workload cgroup prepared through `ExecutionCgroup::prepare`, with its `memory.max` and `pids.max`, and user namespaces kept unavailable. `assemble`: the `Assembly`'s block devices, found by their MMIO windows, checked against their roles' access modes and their expected sizes, and flushed, and the `Root` that it mounts, at a runtime directory, a canonical path that no symbolic link leads through, that it holds exclusively until its teardown, where nothing is mounted already, once it has checked everything: the read-only layers of the `Image` that the caller names, which it never probes: an EROFS image, whose expected `nvx::uuid::Uuid` it checks in its superblock, or a GPT disk whose partition table it reads and checks with `nvx::guest::gpt::read_disk`, against the caller's `nvx::gpt::Expected` values, whose first partition it reads, up to the caller's bound, as opaque bytes that it never mounts, and whose later partitions, at least one, hold ext4 layers that it mounts `ro,nosuid,nodev` with `errors=remount-ro`, so that an error that the kernel finds in a layer fails the operation that found it, rather than panic the guest, as the layer's superblock may ask, from the block devices that the kernel created when it found the disk, once their superblocks show that no journal needs a recovery, which their read-only device can't take, partition 2's at the overlay's base and the last partition's at its top, under an overlay mounted `nosuid,nodev` with the one option set of the caller's `OverlayMode`, `metacopy=on,xino=on` or, for lowers that may carry untrusted overlay xattrs, `metacopy=off,redirect_dir=nofollow`, with no fallback, whose upper and work directories are on the `Upper`: an ext4 scratch mounted `nosuid,nodev`, whose superblock it checks for a journal that the caller's `Journal` policy forbids, or a tmpfs mounted `nosuid,nodev` at the kernel's default size. It refuses overlay options longer than `MOUNT_OPTIONS_LIMIT`, the most that a mount passes, which, below `/run/nvx`, fit up to 195 layers of a GPT disk, and an overlay that the kernel's mount table shows read-only or without the mode's options in effect, since the kernel mounts one so, rather than fail, when its upper can't hold its work directory or its extended attributes. The `Root` reports the first partition's bytes, its `Lower` layers, its `UpperKind`, and its `OverlayMode`. `Root::mount_share` and `Root::mount_aggregate`: virtio-fs live shares at targets in the root that it checks, against the targets that `protocol::share` reserves among others, with one aggregate per root, which it attaches through its `mappings::Mapper` beneath the overlay, with NVX's container root's reserved targets: a share's device mounted out of the root, then moved onto its target, and a copy of each child's entry of the aggregate. `Root::map`: a caller's host mappings, through the same mapper. A share, a child, or a mapping that fails latches the root. `Root::teardown`: the shares and mappings in reverse order, then an aggregate's own mount, the overlay, the layers from the top down, and the upper, with every failed unmount reported. Each failure is a typed `Error`. The operations collect no child's status, change neither the caller's root nor its credentials, start no async runtime, and never power the VM off | +| `nvx::guest::root` | The operations of a layered root, for a guest init or agent, whether or not it is PID 1. `bootstrap`: the agent's and the workload's sibling cgroups of a `Layout`, with every process of their parent moved into the agent's, the agent's `memory.low`, and the workload cgroup prepared through `ExecutionCgroup::prepare`, with its `memory.max` and `pids.max`, and user namespaces kept unavailable. `assemble`: the `Assembly`'s block devices, found by their MMIO windows, checked against their roles' access modes and their expected sizes, and flushed, and the `Root` that it mounts, at a runtime directory, a canonical path that no symbolic link leads through, that it holds exclusively until its teardown, where nothing is mounted already, once it has checked everything: the read-only layers of the `Image` that the caller names, which it never probes: an EROFS image, whose expected `nvx::uuid::Uuid` it checks in its superblock, or a GPT disk whose partition table it reads and checks with `nvx::guest::gpt::read_disk`, against the caller's `nvx::gpt::Expected` values, whose first partition it reads, up to the caller's bound, as opaque bytes that it never mounts, and whose later partitions, at least one, hold ext4 layers that it mounts `ro,nosuid,nodev` with `errors=remount-ro`, so that an error that the kernel finds in a layer fails the operation that found it, rather than panic the guest, as the layer's superblock may ask, from the block devices that the kernel created when it found the disk, once their superblocks show that no journal needs a recovery, which their read-only device can't take, partition 2's at the overlay's base and the last partition's at its top, under an overlay mounted `nosuid,nodev` with the one option set of the caller's `OverlayMode`, `metacopy=on,xino=on` or, for lowers that may carry untrusted overlay xattrs, `metacopy=off,redirect_dir=nofollow`, with no fallback, whose upper and work directories are on the `Upper`: an ext4 scratch mounted `nosuid,nodev`, whose superblock it checks for a journal that the caller's `Journal` policy forbids, or a tmpfs mounted `nosuid,nodev` at the kernel's default size. It refuses overlay options longer than `MOUNT_OPTIONS_LIMIT`, the most that a mount passes, which, below `/run/nvx`, fit up to 195 layers of a GPT disk, and an overlay that the kernel's mount table shows read-only or without the mode's options in effect, since the kernel mounts one so, rather than fail, when its upper can't hold its work directory or its extended attributes. The `Root` reports the first partition's bytes, its `Lower` layers, its `UpperKind`, and its `OverlayMode`. `Root::mount_share` and `Root::mount_aggregate`: virtio-fs live shares at targets in the root that it checks, against the targets that `protocol::share` reserves among others, with one aggregate per root, which it attaches through its `mappings::Mapper` beneath the overlay, with NVX's container root's reserved targets: a share's device mounted out of the root, then moved onto its target, and a copy of each child's entry of the aggregate. `Root::mount_export`: the host export, the aggregate device of `protocol::export`, one per root, mounted `nosuid,nodev` with the access of its `ExportMode` at `hostfs/root` in the runtime directory, out of the root, below a directory that only root may enter, whose root it returns as the `SourceRoot` of the caller's mappings. `Root::map`: a caller's host mappings, through the same mapper. A share, a child, the export, or a mapping that fails latches the root. `Root::teardown`: the shares and mappings in reverse order, then the mounts of an aggregate and of the export, the later first, the overlay, the layers from the top down, and the upper, with every failed unmount reported. Each failure is a typed `Error`. The operations collect no child's status, change neither the caller's root nor its credentials, start no async runtime, and never power the VM off | | `nvx::guest::gpt` | `read_disk`: the partition table of a GPT disk, read and checked with `nvx::gpt`'s `Disk::read` through the disk's block device, which the caller opened, at the logical block size and size that `BLKSSZGET` and `BLKGETSIZE64` report, with the caller's `Expected` values, such as the disk GUID that a boot parameter names. It only reads the device, at explicit offsets, so it never moves the file's position. It neither compares the partitions that the kernel created with the table nor asks the kernel to reread the table. Each failure is a typed `DeviceError` | GPT lower layers use `noload,errors=remount-ro`: the kernel neither loads nor diff --git a/nvx-rs/src/guest/root.rs b/nvx-rs/src/guest/root.rs index 3f47f9c12..a0296b680 100644 --- a/nvx-rs/src/guest/root.rs +++ b/nvx-rs/src/guest/root.rs @@ -1,5 +1,6 @@ //! Operations that bootstrap a guest for a layered root, assemble the root, mount live shares and -//! mappings in it, and tear it down, for a guest init or agent, whether or not it is PID 1. +//! mappings in it, and the host export from which mappings come, and tear it down, for a guest +//! init or agent, whether or not it is PID 1. //! //! - [`bootstrap`] prepares the guest once: it creates the agent's and the workload's sibling //! cgroups of a [`Layout`](super::cgroup::Layout), moves every process of their parent into the @@ -16,12 +17,14 @@ //! mounted: the first partition's bytes, the [`Lower`] layers, the [`UpperKind`], and the //! [`OverlayMode`]. //! - [`Root::mount_share`] and [`Root::mount_aggregate`] mount the VMM's live shares at targets in -//! the root, and [`Root::map`] applies a caller's host mappings there, through the root's -//! [`Mapper`], as [live shares and mappings](#live-shares-and-mappings) describes. +//! the root, [`Root::mount_export`] mounts the host export out of the root, and [`Root::map`] +//! applies a caller's host mappings in the root, from the export or another source, through the +//! root's [`Mapper`], as [live shares and mappings](#live-shares-and-mappings) describes. //! - [`Root::teardown`] unmounts, in order, the live shares and mappings in reverse order, the -//! mount of an aggregate live share, the overlay, the layers from the top down, and the upper. -//! It attempts each, whatever failed before, and reports every unmount that failed. A root -//! that is dropped tears down the same way, but ignores failures. +//! mounts of an aggregate live share and of the host export, the later first, the overlay, the +//! layers from the top down, and the upper. It attempts each, whatever failed before, and +//! reports every unmount that failed. A root that is dropped tears down the same way, but +//! ignores failures. //! //! A failure is an [`Error`] that names what failed. The operations need the guest's root user, //! in the guest's initial mount and cgroup namespaces. They never collect a child's status, @@ -40,19 +43,21 @@ //! - `rootfs`, the overlay, which is the root; //! - `shares`, an aggregate live share, whose root lists every child, out of the workload's //! reach; +//! - `hostfs/root`, the host export, whose root lists every exported host directory and file, out +//! of the workload's reach, in `hostfs`, which only root may enter; //! - and `staging`, where a live share is mounted, out of the workload's reach, before it moves //! to its target. //! //! A root mounts and unmounts by these paths, so they must lead to the same directories whatever //! the root has mounted. An assembly refuses a runtime directory whose path is not canonical, as //! a live share's target must be, or leads through a symbolic link, and a root mounts nothing -//! where `layers` or the mount point is a symbolic link, since a `..` component or a link could -//! lead through a directory that one of the root's mounts hides later. A root also holds the -//! exclusive lock of its runtime directory, from the start of its assembly until it is torn down: -//! an assembly refuses a runtime directory that another root holds, in this process or another, -//! and one where a file system, or a bind mount of a directory, is mounted already at any of these -//! mount points but `staging`, whether or not the new root mounts there, such as an upper of -//! either kind, or a layer of either image, that a failed teardown left; a live share refuses a +//! where `layers`, `hostfs`, or the mount point is a symbolic link, since a `..` component or a +//! link could lead through a directory that one of the root's mounts hides later. A root also +//! holds the exclusive lock of its runtime directory, from the start of its assembly until it is +//! torn down: an assembly refuses a runtime directory that another root holds, in this process or +//! another, and one where a file system, or a bind mount of a directory, is mounted already at any +//! of these mount points but `staging`, whether or not the new root mounts there, such as an upper +//! of either kind, or a layer of either image, that a failed teardown left; a live share refuses a //! `staging` where one is mounted already. The lock is advisory: it keeps out the roots of these //! operations, not other mounts. //! @@ -93,11 +98,19 @@ //! [`Root::map`] applies a caller's host [`Mapping`](super::mappings::Mapping)s beneath the //! overlay the same way, from the caller's [`SourceRoot`](super::mappings::SourceRoot). //! +//! The host export is the device of an aggregate whose children the root never binds itself: +//! they are the sources of the caller's mappings, such as the mapping table of NVX's managed +//! lifecycle, whose boot tokens [`protocol::export`](crate::protocol::export) reads. +//! [`Root::mount_export`] mounts it at `hostfs/root`, out of the root, with the access of its +//! [`ExportMode`](crate::protocol::export::ExportMode), `nosuid`, and `nodev`, and returns its +//! root, the [`SourceRoot`](super::mappings::SourceRoot) from which [`Root::map`] applies the +//! caller's mappings. A root has one export at most. +//! //! The mounts are made in the caller's mount namespace, which must be recursively private, as a -//! guest's initial namespace is. A share, a child, or a mapping that fails once the root has -//! checked its request latches the root: the root refuses every later share and mapping, so that -//! its caller starts no workload on a partial set, and the mounts before it stay until the -//! teardown, which unmounts them first. +//! guest's initial namespace is. A share, a child, the export, or a mapping that fails once the +//! root has checked its request latches the root: the root refuses every later share, export, and +//! mapping, so that its caller starts no workload on a partial set, and the mounts before it stay +//! until the teardown, which unmounts them first. //! //! ```no_run //! use std::path::Path; @@ -105,8 +118,9 @@ //! //! use nvx::gpt::Expected; //! use nvx::guest::cgroup::Layout; -//! use nvx::guest::mappings::Access; +//! use nvx::guest::mappings::{Access, Mapping}; //! use nvx::guest::root::{self, Assembly, Device, Image, OverlayMode, Resources, Upper}; +//! use nvx::protocol::export::HostExport; //! use nvx::protocol::root::DISTRO_MMIO_BASE; //! //! # fn main() -> Result<(), Box> { @@ -131,6 +145,12 @@ //! let layers = root.lowers().len(); //! println!("the first partition holds {} bytes over {layers} layers", configuration.len()); //! root.mount_share("workspace", Path::new("/workspace"), Access::ReadWrite)?; +//! // The host export that the kernel command line names, if any, is the source of host mappings. +//! if let Some(export) = HostExport::parse(&std::fs::read("/proc/cmdline")?)? { +//! let source = root.mount_export(&export.tag, export.mode.into())?; +//! root.map(&source, &[Mapping::new("0/src", "/src", Access::ReadOnly)?])?; +//! // Dropping the export's root here lets the teardown unmount the export. +//! } //! // Run the workload in `bootstrap.workload()`, in the root at `root.path()`; then: //! root.teardown()?; //! # Ok(()) @@ -177,6 +197,15 @@ const ROOT_DIRECTORY: &str = "rootfs"; /// The mount point of an aggregate live share, below the runtime directory and outside the root, /// which exists once a root has mounted an aggregate. const AGGREGATE_DIRECTORY: &str = "shares"; +/// The directory of the host export's mount point, below the runtime directory and outside the +/// root, which only root may enter, and which exists once a root has mounted an export. +const HOSTFS_DIRECTORY: &str = "hostfs"; +/// The mount point of the host export, below the runtime directory: `root` in +/// [`HOSTFS_DIRECTORY`], which exists once a root has mounted an export. +const EXPORT_DIRECTORY: &str = "hostfs/root"; +/// The mode of the [`HOSTFS_DIRECTORY`] that a root creates, whatever the process's umask, as +/// NVX's shell init creates it under `umask 077`: only root may enter it. +const HOSTFS_DIRECTORY_MODE: u32 = 0o700; /// The mount point of a live share before it moves to its target, below the runtime directory /// and outside the root, which exists once a root has mounted a share. const STAGING_DIRECTORY: &str = "staging"; @@ -420,9 +449,10 @@ impl OverlayMode { /// `tmpfs`, and of the root, `rootfs`, a component at a time, and refuses a component that /// exists as anything but a directory, a symbolic link included; takes the runtime directory's /// exclusive lock, which the root holds from then on, and refuses the runtime directory if -/// another root holds it, or if one of these, the other upper's, any entry of `layers`, or -/// `shares`, where a root mounts an aggregate live share, is a mount point already, whether of -/// a file system or of a bind mount of a directory; +/// another root holds it, or if one of these, the other upper's, any entry of `layers`, +/// `shares`, where a root mounts an aggregate live share, or `hostfs/root`, where it mounts the +/// host export, is a mount point already, whether of a file system or of a bind mount of a +/// directory; /// 4. finds the distro device, polling for it every 50 ms until `timeout` passes, checks that it /// is read-only and, if the caller expects a size, of that size, and flushes its buffers; /// 5. for an EROFS image, checks the expected UUID, if any, in its superblock, and reads nothing @@ -536,7 +566,7 @@ impl Lower { } /// The handle of an assembled layered root, which owns its mounts, those of the assembly and -/// those of its live shares and mappings, and reports what the assembly mounted. +/// those of its live shares, host export, and mappings, and reports what the assembly mounted. /// /// [`teardown`](Self::teardown) unmounts them and reports each failure. Dropping the root /// unmounts them the same way, and ignores failures. Until then, the root holds the lock of its @@ -560,15 +590,17 @@ pub struct Root { overlay_mode: OverlayMode, /// The bytes of a GPT disk's first partition, without their trailing padding. first_partition: Option>, - /// The mounts of live shares that the root unmounts by path, and their mount points, in mount - /// order: an aggregate's, and a share's that stays at `staging` because its unmount failed. + /// The mounts of live shares and of the host export that the root unmounts by path, and their + /// mount points, in mount order: an aggregate's, the export's, and a share's that stays at + /// `staging` because its unmount failed. shares: Vec<(Mount, PathBuf)>, /// The mapper of the live shares and mappings in the root, once the root has mounted one. mapper: Option, /// What each of the mapper's mappings mounts, in the mapper's order, then what the one that /// a rename moved and that the mapper failed to detach mounts, if any. mapped: Vec, - /// Whether a share or a mapping failed, after which the root refuses every later one. + /// Whether a share, the export, or a mapping failed, after which the root refuses every later + /// one. latched: bool, /// The exclusive lock of the runtime directory, from the start of the assembly; dropping the /// root releases it after the unmounts. @@ -586,8 +618,8 @@ impl Root { &self.runtime } - /// Returns whether a live share or a mapping failed, after which the root refuses every later - /// one, so that its caller starts no workload in it. + /// Returns whether a live share, the host export, or a mapping failed, after which the root + /// refuses every later one, so that its caller starts no workload in it. pub fn is_latched(&self) -> bool { self.latched } @@ -619,11 +651,13 @@ impl Root { /// Unmounts the root's mounts, in order: a live share or a mapping that a rename moved and /// that its mapper failed to detach, which the mapper detaches, the live shares and mappings - /// in reverse order, the mount of an aggregate live share, the overlay, the layers from the - /// top down, and the upper. + /// in reverse order, the mounts of an aggregate live share and of the host export, the later + /// first, the overlay, the layers from the top down, and the upper. /// /// Each unmount is attempted, whatever failed before, without detaching a busy mount but the /// moved one, and every one that failed is reported, with the mount that it leaves in place. + /// A [`SourceRoot`](super::mappings::SourceRoot) of the export that the caller still holds + /// keeps the export busy. pub fn teardown(mut self) -> Result<(), TeardownError> { self.unmount() } @@ -667,13 +701,13 @@ impl Root { for point in &points { self.check_unoccupied(point, false)?; } - // A root whose teardown failed may have left mounted an upper of the other kind, or an - // aggregate share, which this root mounts only later. + // A root whose teardown failed may have left mounted an upper of the other kind, an + // aggregate share, or the host export, which this root mounts only later. let other_upper = match assembly.upper.kind() { UpperKind::Scratch => TMPFS_DIRECTORY, UpperKind::Tmpfs => SCRATCH_DIRECTORY, }; - for point in [other_upper, AGGREGATE_DIRECTORY] { + for point in [other_upper, AGGREGATE_DIRECTORY, EXPORT_DIRECTORY] { self.check_unoccupied(&runtime.join(point), true)?; } // Or a layer of either image, at any mount point below `layers`: an EROFS image's, or a @@ -1243,6 +1277,11 @@ pub enum Mount { /// The aggregate's virtio-fs tag. tag: String, }, + /// The host export, at `hostfs/root` in the runtime directory. + Export { + /// The export's virtio-fs tag. + tag: String, + }, /// A child of an aggregate live share, bound at its target in the root. Child { /// The child's name in the aggregate. @@ -1269,6 +1308,7 @@ impl fmt::Display for Mount { write!(formatter, "the live share {tag} at {}", target.display()) } Self::Aggregate { tag } => write!(formatter, "the aggregate live share {tag}"), + Self::Export { tag } => write!(formatter, "the host export {tag}"), Self::Child { name, target } => { write!( formatter, @@ -1461,14 +1501,14 @@ pub enum Error { /// A live share, a child of an aggregate live share, or a mapping was refused or failed to /// attach at its target in the root, once the root had checked its request; nothing of it /// stays mounted, unless the error is a moved mapping that stays mounted. The root refuses - /// every later share and mapping, with [`MappingError::Latched`]. + /// every later share, export, and mapping, with [`MappingError::Latched`]. Mapping { /// The mount that was refused or failed. mount: Mount, /// Why. error: MappingError, }, - /// A live share was refused before anything was mounted for it. + /// A live share, or the host export, was refused before anything was mounted for it. Share(ShareRefusal), } diff --git a/nvx-rs/src/guest/root/share.rs b/nvx-rs/src/guest/root/share.rs index b2c6e3d76..4607fd4ba 100644 --- a/nvx-rs/src/guest/root/share.rs +++ b/nvx-rs/src/guest/root/share.rs @@ -1,18 +1,25 @@ //! Live shares, virtio-fs devices that the VMM exports from the host, and a caller's mappings, //! mounted at targets in a layered root through the root's mapper, after the checks that the -//! [module](super#live-shares-and-mappings) lists. +//! [module](super#live-shares-and-mappings) lists, and the host export, the device from which a +//! caller's mappings come. use std::fmt; +use std::fs; use std::io; use std::os::unix::ffi::OsStrExt; +use std::os::unix::fs::{DirBuilderExt, PermissionsExt}; use std::path::{Path, PathBuf}; -use super::{AGGREGATE_DIRECTORY, Error, Mount, Root, STAGING_DIRECTORY}; +use super::{ + AGGREGATE_DIRECTORY, EXPORT_DIRECTORY, Error, HOSTFS_DIRECTORY, HOSTFS_DIRECTORY_MODE, Mount, + Root, STAGING_DIRECTORY, +}; use crate::guest::mappings::{ self, Access, Kernel as MappingKernel, Kind, Mapper, Mapping, MappingError, Reserved, SourceRoot, TargetRoot, }; use crate::guest::sys::{self, Errno, FileDescriptor, Mode, OpenFlags}; +use crate::protocol::export::ExportMode; use crate::protocol::share::{self, RESERVED_TARGET_CHARACTERS}; /// The file system type of a live share. @@ -52,7 +59,7 @@ pub struct ShareChild { pub kind: ShareKind, } -/// Why a live share was refused before anything was mounted for it. +/// Why a live share, or the host export, was refused before anything was mounted for it. #[derive(Clone, Debug, PartialEq, Eq, Hash)] #[non_exhaustive] pub enum ShareRefusal { @@ -86,6 +93,8 @@ pub enum ShareRefusal { /// The root owns an aggregate already, whose children may or may not all be bound: it mounts /// every aggregate at the same place. SecondAggregate, + /// The root owns the host export already: it mounts every export at the same place. + SecondExport, } impl fmt::Display for ShareRefusal { @@ -126,6 +135,18 @@ impl fmt::Display for ShareRefusal { Self::SecondAggregate => { formatter.write_str("the root has an aggregate live share already") } + Self::SecondExport => formatter.write_str("the root has a host export already"), + } + } +} + +impl From for Access { + /// Returns the access of the guest's mount of a host export of `mode`, which + /// [`Root::mount_export`] takes: read-only for `ro`, and read-write for `rw`. + fn from(mode: ExportMode) -> Self { + match mode { + ExportMode::ReadOnly => Self::ReadOnly, + ExportMode::ReadWrite => Self::ReadWrite, } } } @@ -227,6 +248,46 @@ impl Root { mounted } + /// Mounts the host export, the aggregate virtio-fs device `tag` from which the caller maps host + /// paths into the root with [`map`](Self::map), and returns the export's root, the + /// [`SourceRoot`] of those mappings. A guest reads the export's tag and [`ExportMode`] from + /// its kernel command line with + /// [`HostExport::parse`](crate::protocol::export::HostExport::parse), and mounts it with the + /// mode's access. + /// + /// The device is mounted with `access`, `nosuid`, and `nodev` at `hostfs/root` in the runtime + /// directory, outside the root, where the workload cannot list the export's children: below + /// `hostfs`, a directory that the root creates with mode 0700, whatever the umask, so that only + /// root may enter it, unless a directory is there already, which keeps its mode. Neither may be + /// a symbolic link. The tag is checked first, as a live share's is; nothing is created or + /// mounted for a refused one. + /// + /// The mount belongs to the root, which unmounts it when it is torn down, after the live + /// shares and mappings, those mapped from the export included. The returned [`SourceRoot`] + /// holds a descriptor of the export's root, which keeps the mount busy: drop it before the + /// teardown, which otherwise fails to unmount the export, and leaves it mounted. + /// + /// A root has one export at most, since every export is mounted at the same place: once the + /// root has mounted one, another is refused. A failure once the tag passed its check latches + /// the root; an export that is mounted by then stays for the teardown. + pub fn mount_export(&mut self, tag: &str, access: Access) -> Result { + let export = Mount::Export { + tag: tag.to_owned(), + }; + self.check_latch(&export)?; + if self + .shares + .iter() + .any(|(mount, _)| matches!(mount, Mount::Export { .. })) + { + return Err(Error::Share(ShareRefusal::SecondExport)); + } + check_tag(tag)?; + let mounted = self.mount_export_at(&export, tag, access); + self.latched |= mounted.is_err(); + mounted + } + /// Applies `mappings` beneath the root, in order, from `source`, such as the guest's mount of /// the host's export, through the root's mapper, as the /// [module](super#live-shares-and-mappings) describes, and owns them: the teardown unmounts @@ -355,6 +416,39 @@ impl Root { Ok(()) } + /// Mounts the export's device `tag` at `hostfs/root` and opens its root, as + /// [`mount_export`](Self::mount_export) describes. + fn mount_export_at( + &mut self, + export: &Mount, + tag: &str, + access: Access, + ) -> Result { + let hostfs = self.runtime.join(HOSTFS_DIRECTORY); + create_private_directory(&hostfs).map_err(|error| Error::MountPoint { + path: hostfs, + error, + })?; + let directory = self.runtime.join(EXPORT_DIRECTORY); + super::create_directory(&directory).map_err(|error| Error::MountPoint { + path: directory.clone(), + error, + })?; + self.mount_at( + export, + Some(tag.as_bytes()), + &directory, + Some(VIRTIOFS), + access.flags(), + None, + )?; + self.shares.push((export.clone(), directory.clone())); + SourceRoot::open(&directory).map_err(|error| Error::MountPoint { + path: directory, + error, + }) + } + /// Applies `mappings` from `source` through the root's mapper, as [`map`](Self::map) /// describes. fn apply(&mut self, source: &SourceRoot, mappings: &[Mapping]) -> Result<(), Error> { @@ -472,6 +566,27 @@ fn open_mount_point(path: &Path) -> Result { sys::open(&sys::path_c_string(path)?, flags, Mode::empty()) } +/// Creates the directory `path` with [`HOSTFS_DIRECTORY_MODE`], whatever the umask, unless a +/// directory, and not a symbolic link, is there already, which keeps its mode. It is created with +/// no permission beyond that mode at any time. +fn create_private_directory(path: &Path) -> io::Result<()> { + match fs::DirBuilder::new() + .mode(HOSTFS_DIRECTORY_MODE) + .create(path) + { + // The umask may have withheld some of the mode's permissions. + Ok(()) => fs::set_permissions(path, fs::Permissions::from_mode(HOSTFS_DIRECTORY_MODE)), + Err(error) if error.kind() == io::ErrorKind::AlreadyExists => { + if fs::symlink_metadata(path)?.is_dir() { + Ok(()) + } else { + Err(Errno::ENOTDIR.into()) + } + } + Err(error) => Err(error), + } +} + /// Returns `error` as the error of a teardown's failed unmount: the call's own for a failed /// system call. pub(super) fn io_error(error: &MappingError) -> io::Error { @@ -625,4 +740,41 @@ mod tests { assert!(check_name(name).is_err(), "{name}"); } } + + #[test] + fn the_export_s_directory_is_created_private_whatever_the_umask() { + for umask in [0o000, 0o022, 0o077, 0o777] { + // The test's own directory is created with the test's umask, in which this thread + // may still create entries. + let directory = tempfile::tempdir().unwrap(); + let hostfs = directory.path().join("hostfs"); + let mode = |path: &Path| fs::metadata(path).unwrap().permissions().mode() & 0o7777; + sys::with_thread_umask(umask, || create_private_directory(&hostfs)).unwrap(); + assert_eq!(mode(&hostfs), HOSTFS_DIRECTORY_MODE, "{umask:o}"); + // A directory that is there already keeps its mode. + fs::set_permissions(&hostfs, fs::Permissions::from_mode(0o750)).unwrap(); + sys::with_thread_umask(umask, || create_private_directory(&hostfs)).unwrap(); + assert_eq!(mode(&hostfs), 0o750, "{umask:o}"); + } + // Anything else there is refused, a symbolic link to a directory included. + let directory = tempfile::tempdir().unwrap(); + let file = directory.path().join("file"); + fs::write(&file, "").unwrap(); + let link = directory.path().join("link"); + std::os::unix::fs::symlink(directory.path(), &link).unwrap(); + for path in [file, link] { + let error = create_private_directory(&path).unwrap_err(); + assert_eq!( + error.raw_os_error(), + Some(Errno::ENOTDIR as i32), + "{path:?}" + ); + } + } + + #[test] + fn an_export_s_mode_is_the_access_of_its_mount() { + assert_eq!(Access::from(ExportMode::ReadOnly), Access::ReadOnly); + assert_eq!(Access::from(ExportMode::ReadWrite), Access::ReadWrite); + } } diff --git a/nvx-rs/src/guest/root/tests.rs b/nvx-rs/src/guest/root/tests.rs index a262f068e..04d31b97b 100644 --- a/nvx-rs/src/guest/root/tests.rs +++ b/nvx-rs/src/guest/root/tests.rs @@ -13,7 +13,7 @@ use super::kernel::{Call, FakeKernel, Operation, Paths, super_options}; use super::*; use crate::gpt::ExpectedPartition; use crate::guest::cgroup::Layout; -use crate::guest::mappings::{Access, SourceRoot}; +use crate::guest::mappings::{Access, Mapping, SourceRoot}; use crate::guest::sys::MountAttributes; /// The base address of the distro device's MMIO window. @@ -960,6 +960,39 @@ fn a_runtime_directory_where_a_file_system_is_mounted_is_refused() { ); assert_eq!(guest.kernel.mounts().len(), mounts); + // So does a host export whose unmount alone failed, which is mounted outside the overlay too. + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + drop(root.mount_export("microvm", Access::ReadOnly).unwrap()); + let mount_point = guest.path("run/nvx/hostfs/root"); + guest + .kernel + .fail(Operation::Unmount, mount_point.clone(), Errno::EBUSY); + let error = root.teardown().unwrap_err(); + let failed: Vec<(&Mount, &Path)> = error + .failures() + .iter() + .map(|failure| (failure.mount(), failure.path())) + .collect(); + let mount = Mount::Export { + tag: "microvm".into(), + }; + assert_eq!(failed, [(&mount, mount_point.as_path())]); + assert_eq!( + error.to_string(), + format!( + "failed to unmount the host export microvm: {}", + os_error(Errno::EBUSY) + ) + ); + let mounts = guest.kernel.mounts().len(); + let error = guest.assemble(&guest.assembly()).unwrap_err(); + assert!( + matches!(error.error(), Error::Occupied { path } if *path == mount_point), + "{error:?}" + ); + assert_eq!(guest.kernel.mounts().len(), mounts); + // So does an upper that a failed unmount left, whichever upper the next root mounts. for (first, second) in [ (scratch_upper(Journal::Allowed), Upper::Tmpfs), @@ -3018,6 +3051,231 @@ fn mappings_apply_beneath_the_overlay_in_order_and_go_first_at_the_teardown() { ); } +#[test] +fn a_root_in_run_nvx_mounts_the_host_export_where_its_boot_token_names_it() { + assert_eq!( + Path::new("/run/nvx").join(EXPORT_DIRECTORY), + Path::new(crate::protocol::export::EXPORT_DIRECTORY) + ); + assert_eq!( + Path::new(EXPORT_DIRECTORY).parent(), + Some(Path::new(HOSTFS_DIRECTORY)) + ); +} + +#[test] +fn the_host_export_is_mounted_out_of_the_root_and_unmounted_after_its_mappings() { + let restricted = MountFlags::MS_NOSUID | MountFlags::MS_NODEV; + for (access, flags) in [ + (Access::ReadOnly, restricted | MountFlags::MS_RDONLY), + (Access::ReadWrite, restricted), + ] { + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + let assembled = guest.kernel.calls().len(); + let source = root.mount_export("microvm", access).unwrap(); + // The export's device alone is mounted, restricted, out of the root, below a directory + // that only root may enter. + let mount_point = guest.path("run/nvx/hostfs/root"); + assert_eq!( + guest.kernel.calls()[assembled..], + [mount_call( + Some("microvm"), + &guest.text("run/nvx/hostfs/root"), + Some("virtiofs"), + flags, + None, + )] + ); + assert!(mount_point.is_dir()); + let mode = fs::metadata(guest.path("run/nvx/hostfs")).unwrap().mode() & 0o7777; + assert_eq!(mode, 0o700, "{access:?}"); + assert_eq!( + fs::read_dir(guest.path("run/nvx/rootfs")).unwrap().count(), + 0 + ); + assert!(!root.is_latched()); + + // Its root is the source of the caller's mappings: a directory child, `0`, and a file + // child, `1`, of the aggregate. + fs::create_dir_all(mount_point.join("0/src")).unwrap(); + fs::write(mount_point.join("1"), b"notes").unwrap(); + let mapped = guest.kernel.calls().len(); + root.map( + &source, + &[ + Mapping::new("0/src", "/workspace", Access::ReadWrite).unwrap(), + Mapping::new("1", "/home/user/notes", Access::ReadOnly).unwrap(), + ], + ) + .unwrap(); + let mut expected = attach_calls( + &mount_point.join("0/src"), + &guest.path("run/nvx/rootfs/workspace"), + Access::ReadWrite, + ); + expected.extend(attach_calls( + &mount_point.join("1"), + &guest.path("run/nvx/rootfs/home/user/notes"), + Access::ReadOnly, + )); + assert_eq!(guest.kernel.calls()[mapped..], expected); + assert_eq!(targets(&root), ["/workspace", "/home/user/notes"]); + + // A root mounts one export, since every export is mounted at the same place; refusing + // another latches nothing. + let mounts = guest.kernel.mounts().len(); + let error = root.mount_export("other", access).unwrap_err(); + assert!( + matches!(error, Error::Share(ShareRefusal::SecondExport)), + "{error:?}" + ); + assert_eq!(error.to_string(), "the root has a host export already"); + assert_eq!(guest.kernel.mounts().len(), mounts); + assert!(!root.is_latched()); + + // An aggregate live share, mounted later, is unmounted before the export. + export(&guest, &[("a", ShareKind::Directory)]); + let rw = Access::ReadWrite; + root.mount_aggregate( + "all", + rw, + &[child("a", "/data/a", rw, ShareKind::Directory)], + ) + .unwrap(); + + // The teardown unmounts the live shares and mappings, those from the export included, + // then the aggregate and the export, the later first, then the assembly's mounts. + drop(source); + root.teardown().unwrap(); + assert_eq!( + guest.kernel.unmounts(), + [ + guest.text("run/nvx/rootfs/data/a"), + guest.text("run/nvx/rootfs/home/user/notes"), + guest.text("run/nvx/rootfs/workspace"), + guest.text("run/nvx/shares"), + guest.text("run/nvx/hostfs/root"), + guest.text("run/nvx/rootfs"), + guest.text("run/nvx/layers/distro"), + guest.text("run/nvx/scratch"), + ] + ); + } +} + +#[test] +fn a_refused_tag_creates_and_mounts_nothing_and_a_failed_export_latches_the_root() { + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + let calls = guest.kernel.calls(); + for tag in ["", "a b", "a.b", "a/b", "a,b", "a=b", "micro\u{e9}vm"] { + let error = root.mount_export(tag, Access::ReadOnly).unwrap_err(); + assert!( + matches!(&error, Error::Share(ShareRefusal::Tag(refused)) if refused == tag), + "{error:?}" + ); + assert_eq!( + error.to_string(), + format!("live share tag is invalid: {tag}") + ); + } + assert_eq!(guest.kernel.calls(), calls); + assert!(!guest.path("run/nvx/hostfs").exists()); + // The refusals of requests latch nothing. + assert!(!root.is_latched()); + + // A device that fails to mount latches the root, and leaves nothing to unmount. + let mount_point = guest.path("run/nvx/hostfs/root"); + guest + .kernel + .fail(Operation::Mount, mount_point.clone(), Errno::ENODEV); + let error = root.mount_export("microvm", Access::ReadOnly).unwrap_err(); + assert!( + matches!( + &error, + Error::Mount { mount: Mount::Export { tag }, error } + if tag == "microvm" && error.raw_os_error() == Some(Errno::ENODEV as i32) + ), + "{error:?}" + ); + assert_eq!( + error.to_string(), + format!( + "failed to mount the host export microvm: {}", + os_error(Errno::ENODEV) + ) + ); + assert!(root.is_latched()); + + // A latched root refuses the export, as it refuses every share and mapping. + guest.kernel.restore(Operation::Mount, &mount_point); + let mounts = guest.kernel.mounts().len(); + let error = root.mount_export("microvm", Access::ReadOnly).unwrap_err(); + assert!( + matches!( + &error, + Error::Mapping { mount: Mount::Export { tag }, error: MappingError::Latched } + if tag == "microvm" + ), + "{error:?}" + ); + assert_eq!( + error.to_string(), + "failed to mount the host export microvm: an earlier mapping failed" + ); + assert_eq!(guest.kernel.mounts().len(), mounts); + root.teardown().unwrap(); + assert_eq!( + guest.kernel.unmounts(), + [ + guest.text("run/nvx/rootfs"), + guest.text("run/nvx/layers/distro"), + guest.text("run/nvx/scratch"), + ] + ); +} + +#[test] +fn the_host_export_is_mounted_through_no_symbolic_link() { + for link in ["run/nvx/hostfs", "run/nvx/hostfs/root"] { + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + fs::create_dir(guest.path("elsewhere")).unwrap(); + fs::create_dir_all(guest.path(link).parent().unwrap()).unwrap(); + std::os::unix::fs::symlink(guest.path("elsewhere"), guest.path(link)).unwrap(); + let mounts = guest.kernel.mounts().len(); + let error = root.mount_export("microvm", Access::ReadOnly).unwrap_err(); + assert!( + matches!( + &error, + Error::MountPoint { path, error } + if *path == guest.path(link) + && error.raw_os_error() == Some(Errno::ENOTDIR as i32) + ), + "{link}: {error:?}" + ); + assert_eq!(guest.kernel.mounts().len(), mounts, "{link}"); + assert_eq!( + fs::read_dir(guest.path("elsewhere")).unwrap().count(), + 0, + "{link}" + ); + assert!(root.is_latched(), "{link}"); + root.teardown().unwrap(); + } + + // A directory that is there already keeps its mode. + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + let hostfs = guest.path("run/nvx/hostfs"); + fs::create_dir(&hostfs).unwrap(); + fs::set_permissions(&hostfs, fs::Permissions::from_mode(0o750)).unwrap(); + drop(root.mount_export("microvm", Access::ReadOnly).unwrap()); + assert_eq!(fs::metadata(&hostfs).unwrap().mode() & 0o7777, 0o750); + root.teardown().unwrap(); +} + #[test] fn a_dropped_root_is_torn_down_once() { let guest = Guest::new(); diff --git a/nvx-rs/src/host/filesystem.rs b/nvx-rs/src/host/filesystem.rs index 1e35d4a0f..8088c237f 100644 --- a/nvx-rs/src/host/filesystem.rs +++ b/nvx-rs/src/host/filesystem.rs @@ -42,12 +42,14 @@ use serde::{Deserialize, Serialize}; use super::platform; use crate::error::{Error, Result}; +use crate::protocol::export::EXPORT_DIRECTORY; use crate::protocol::share::{RESERVED_EXACT_MOUNT_TARGETS, RESERVED_MOUNT_TARGETS}; use crate::protocol::{HostMap, MAPS_TOKEN, MAX_HOST_MAPPINGS}; -/// Guest directory where the guest's init mounts the export; only the guest's root can enter -/// its parent. -pub const GUEST_EXPORT: &str = "/run/nvx/hostfs/root"; +/// Guest directory where the guest's init mounts the export, which the export's `virtfs_dir=` +/// boot token names, and guests check: [`EXPORT_DIRECTORY`]. Only the guest's root can enter its +/// parent. +pub const GUEST_EXPORT: &str = EXPORT_DIRECTORY; /// Largest number of host directories and files that OpenVMM exports through one aggregate. pub const MAX_CHILDREN: usize = 256; diff --git a/nvx-rs/src/protocol/export.rs b/nvx-rs/src/protocol/export.rs new file mode 100644 index 000000000..2a5da53ef --- /dev/null +++ b/nvx-rs/src/protocol/export.rs @@ -0,0 +1,232 @@ +//! The boot tokens of the host export: the kernel command-line tokens through which OpenVMM tells +//! the guest of the aggregate virtio-fs export from which the guest maps host paths, and the +//! guest's strict parser of them. +//! +//! A host that maps host paths into the guest through a mapping table, as NVX's managed lifecycle +//! does, exports them through OpenVMM's one virtio-fs device as an aggregate: a read-only root that +//! lists one child per exported host directory or file, which the guest mounts out of its +//! workloads' reach, and from which it maps each entry of the table at its target. The host's plan +//! exports the aggregate at [`EXPORT_DIRECTORY`], and OpenVMM appends four tokens for it to the +//! kernel command line: +//! +//! - `virtfs_dir=/run/nvx/hostfs/root`, the guest directory of the export, [`EXPORT_DIRECTORY`]; +//! - `virtfs_tag=TAG`, the virtio-fs tag of the export's device, a nonempty name of letters, +//! digits, `_`, and `-`; +//! - `virtfs_mode=ro`, or `virtfs_mode=rw` when a child is read-write, the [`ExportMode`] of the +//! guest's mount; +//! - and `virtfs_aggregate=1`, which tells the aggregate's root, which lists the children, from a +//! single shared directory. +//! +//! Guests read them with [`HostExport::parse`], which accepts exactly these forms, all four tokens +//! or none, each once. It applies the checks with which NVX's shell init mounts the export of a +//! managed sandbox, and refuses `nvx_share=` tokens as that init does: NVX's one-shot lifecycle +//! binds the children of an aggregate as live shares with them, while a guest that mounts the +//! export maps host paths from its mapping table alone. +//! +//! Like the rest of [`protocol`](super), it needs nothing outside the standard library, so the +//! host, the guest, and test doubles share it. + +use std::fmt; + +use super::{ProtocolError, violation}; + +/// The guest directory of the host export, the value of its `virtfs_dir=` token: the directory +/// where the host's plan exports the aggregate, and where NVX's guests mount it. +pub const EXPORT_DIRECTORY: &str = "/run/nvx/hostfs/root"; + +/// The name of the kernel parameter that names the export's guest directory. +const DIRECTORY: &str = "virtfs_dir"; +/// The name of the kernel parameter that names the virtio-fs tag of the export's device. +const TAG: &str = "virtfs_tag"; +/// The name of the kernel parameter that names the mode of the guest's mount of the export. +const MODE: &str = "virtfs_mode"; +/// The name of the kernel parameter that marks the export as an aggregate. +const AGGREGATE: &str = "virtfs_aggregate"; +/// The value of the kernel parameter that marks the export as an aggregate. +const AGGREGATE_VALUE: &str = "1"; +/// The name of the kernel parameter with which NVX's one-shot lifecycle binds a child of an +/// aggregate as a live share. +const SHARE: &str = "nvx_share"; +/// The names of the export's kernel parameters, in the order in which OpenVMM appends them. +const PARAMETERS: [&str; 4] = [DIRECTORY, TAG, MODE, AGGREGATE]; + +/// The mode in which the guest mounts the host export, which its `virtfs_mode=` token names. +/// +/// OpenVMM limits each child's writes to that child's writable parts, so the mode only lets the +/// guest's mount carry the read-write children: it is never the boundary of a child's access. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum ExportMode { + /// `ro`: every child of the export is read-only. + ReadOnly, + /// `rw`: a child of the export is read-write. + ReadWrite, +} + +impl ExportMode { + /// Every mode. + const ALL: [Self; 2] = [Self::ReadOnly, Self::ReadWrite]; + + /// Returns the mode's value in the token: `ro` or `rw`. + pub const fn as_str(self) -> &'static str { + match self { + Self::ReadOnly => "ro", + Self::ReadWrite => "rw", + } + } + + /// Returns the mode that `value` names in the token, if any. + fn from_value(value: &[u8]) -> Option { + Self::ALL + .into_iter() + .find(|mode| mode.as_str().as_bytes() == value) + } +} + +impl fmt::Display for ExportMode { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str(self.as_str()) + } +} + +/// The host export that the boot tokens of a kernel command line describe: the device that the +/// guest mounts, and how. The export's guest directory is always [`EXPORT_DIRECTORY`]. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct HostExport { + /// The virtio-fs tag of the export's device: a nonempty name of letters, digits, `_`, and `-`. + pub tag: String, + /// The mode in which the guest mounts the export. + pub mode: ExportMode, +} + +impl HostExport { + /// Reads the host export from the boot tokens of a kernel command line, such as the guest's + /// `/proc/cmdline`, or returns `None` if the command line has none of them, as that of a guest + /// that the host maps no paths into has none. + /// + /// The command line splits at spaces, tabs, and newlines, without quoting, as the guest's init + /// splits it, and its tokens may come in any order. Each of the export's tokens must take its + /// form of the [module](self), with a value and at most once, and the command line must hold + /// all four or none: `virtfs_dir=` [`EXPORT_DIRECTORY`], which a single shared directory or the + /// aggregate of NVX's one-shot live shares names another way, `virtfs_tag=` a nonempty name of + /// letters, digits, `_`, and `-`, `virtfs_mode=` `ro` or `rw`, and `virtfs_aggregate=1`. An + /// `nvx_share=` token is refused too, whether or not the export's tokens are there: it binds a + /// child of an aggregate as a live share, which the guest of an export never does. Other tokens + /// are ignored. + /// + /// Fails with a [`ProtocolError`] that names the first token that breaks these rules, or the + /// tokens that an incomplete export lacks. + pub fn parse(command_line: &[u8]) -> Result, ProtocolError> { + let mut directory = false; + let mut tag = None; + let mut mode = None; + let mut aggregate = false; + for token in command_line.split(|byte| matches!(byte, b' ' | b'\t' | b'\n')) { + let (name, value) = match token.iter().position(|&byte| byte == b'=') { + Some(equals) => (&token[..equals], Some(&token[equals + 1..])), + None => (token, None), + }; + if name == SHARE.as_bytes() && value.is_some() { + return Err(violation(format!( + "the {SHARE} kernel parameter binds a live share, but a guest that mounts the \ + host export maps host paths from its mapping table alone" + ))); + } + let Some(parameter) = PARAMETERS + .into_iter() + .find(|parameter| parameter.as_bytes() == name) + else { + continue; + }; + let Some(value) = value else { + return Err(violation(format!( + "the {parameter} kernel parameter has no value" + ))); + }; + let seen = match parameter { + DIRECTORY => directory, + TAG => tag.is_some(), + MODE => mode.is_some(), + _ => aggregate, + }; + if seen { + return Err(violation(format!( + "the {parameter} kernel parameter is repeated" + ))); + } + match parameter { + DIRECTORY => { + if value != EXPORT_DIRECTORY.as_bytes() { + return Err(violation(format!( + "the {DIRECTORY} kernel parameter must be {EXPORT_DIRECTORY}, not '{}'", + value.escape_ascii() + ))); + } + directory = true; + } + TAG => tag = Some(parse_tag(value)?), + MODE => { + mode = Some(ExportMode::from_value(value).ok_or_else(|| { + violation(format!( + "the {MODE} kernel parameter must be ro or rw, not '{}'", + value.escape_ascii() + )) + })?); + } + _ => { + if value != AGGREGATE_VALUE.as_bytes() { + return Err(violation(format!( + "the {AGGREGATE} kernel parameter must be {AGGREGATE_VALUE}, not '{}'", + value.escape_ascii() + ))); + } + aggregate = true; + } + } + } + match (directory, tag, mode, aggregate) { + (true, Some(tag), Some(mode), true) => Ok(Some(Self { tag, mode })), + (false, None, None, false) => Ok(None), + (directory, tag, mode, aggregate) => { + let present = [directory, tag.is_some(), mode.is_some(), aggregate]; + let missing: Vec<&str> = PARAMETERS + .into_iter() + .zip(present) + .filter_map(|(parameter, present)| (!present).then_some(parameter)) + .collect(); + Err(violation(format!( + "the host export is incomplete: the command line has no {} kernel parameter", + alternatives(&missing) + ))) + } + } + } +} + +/// Parses the value of the tag's token, a nonempty name of letters, digits, `_`, and `-`. +fn parse_tag(value: &[u8]) -> Result { + if value.is_empty() + || !value + .iter() + .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-')) + { + return Err(violation(format!( + "the {TAG} kernel parameter must be a tag of letters, digits, _, and -, not '{}'", + value.escape_ascii() + ))); + } + // The bytes are ASCII. + Ok(value.iter().copied().map(char::from).collect()) +} + +/// Returns `names` as alternatives in a sentence: `a`, `a or b`, or `a, b, or c`. +fn alternatives(names: &[&str]) -> String { + match names { + [] => String::new(), + [name] => (*name).to_owned(), + [first, second] => format!("{first} or {second}"), + [rest @ .., last] => format!("{}, or {last}", rest.join(", ")), + } +} + +#[cfg(test)] +mod tests; diff --git a/nvx-rs/src/protocol/export/tests.rs b/nvx-rs/src/protocol/export/tests.rs new file mode 100644 index 000000000..398aa79a7 --- /dev/null +++ b/nvx-rs/src/protocol/export/tests.rs @@ -0,0 +1,207 @@ +//! Tests of the boot tokens of the host export. `tests/fixtures/export-tokens.json` holds the +//! command lines that every reader of the tokens must agree on. + +use serde_json::Value; + +use super::*; + +/// The command lines that every reader of the export's tokens agrees on. +const FIXTURE: &str = include_str!("../../../tests/fixtures/export-tokens.json"); + +fn parse(command_line: &str) -> Result, String> { + HostExport::parse(command_line.as_bytes()).map_err(ProtocolError::into_message) +} + +fn export(tag: &str, mode: ExportMode) -> HostExport { + HostExport { + tag: tag.to_owned(), + mode, + } +} + +/// Returns the export of a case of the fixture, or `None` for its `null`. +fn recorded_export(value: &Value) -> Option { + if value.is_null() { + return None; + } + let mode = match value["mode"].as_str().unwrap() { + "ro" => ExportMode::ReadOnly, + "rw" => ExportMode::ReadWrite, + other => panic!("the fixture names the unknown mode {other}"), + }; + Some(export(value["tag"].as_str().unwrap(), mode)) +} + +/// Returns the tokens that OpenVMM appends for an aggregate export with `tag` and `mode`. +fn tokens(tag: &str, mode: ExportMode) -> String { + format!("virtfs_dir={EXPORT_DIRECTORY} virtfs_tag={tag} virtfs_mode={mode} virtfs_aggregate=1") +} + +#[test] +fn wire_values_keep_their_literals() { + assert_eq!(EXPORT_DIRECTORY, "/run/nvx/hostfs/root"); + assert_eq!( + PARAMETERS, + [ + "virtfs_dir", + "virtfs_tag", + "virtfs_mode", + "virtfs_aggregate" + ] + ); + assert_eq!(AGGREGATE_VALUE, "1"); + assert_eq!(SHARE, "nvx_share"); + assert_eq!(ExportMode::ReadOnly.as_str(), "ro"); + assert_eq!(ExportMode::ReadWrite.to_string(), "rw"); + for mode in ExportMode::ALL { + assert_eq!(ExportMode::from_value(mode.as_str().as_bytes()), Some(mode)); + } + for value in ["", "RO", "Rw", "ro ", "r", "rwx", "ro,rw"] { + assert_eq!(ExportMode::from_value(value.as_bytes()), None, "{value}"); + } +} + +#[test] +fn every_export_reads_back_from_its_tokens_among_others() { + for mode in ExportMode::ALL { + for tag in ["microvm", "fs0", "A-b_9"] { + let tokens = tokens(tag, mode); + for command_line in [ + tokens.clone(), + format!("quiet nvx_sandbox=1 {tokens}\tnvx_maps=3\nnvx_entrypoint=/bin/sh"), + tokens.split(' ').rev().collect::>().join("\n"), + ] { + assert_eq!( + parse(&command_line), + Ok(Some(export(tag, mode))), + "{command_line}" + ); + } + } + } +} + +#[test] +fn every_reader_parses_the_recorded_command_lines_alike() { + let fixture: Value = serde_json::from_str(FIXTURE).unwrap(); + let cases = fixture["cases"].as_array().unwrap(); + let mut names = std::collections::HashSet::new(); + for case in cases { + let name = case["name"].as_str().unwrap(); + assert!(names.insert(name), "the fixture repeats the case {name}"); + let command_line = case["command_line"].as_str().unwrap(); + let actual = parse(command_line); + match case.get("error") { + Some(error) => assert_eq!(actual, Err(error.as_str().unwrap().to_owned()), "{name}"), + None => { + let expected = recorded_export(&case["export"]); + assert_eq!(actual, Ok(expected.clone()), "{name}"); + // What OpenVMM appends for the export reads back as the same export. + if let Some(expected) = expected { + assert_eq!( + parse(&tokens(&expected.tag, expected.mode)), + Ok(Some(expected)), + "{name}" + ); + } + } + } + } + // The corpus holds an export of each mode, no export, and a refusal of each token's form, of + // a repeated token, of an incomplete export, and of a live share. + for mode in ["ro", "rw"] { + assert!( + cases.iter().any(|case| case["export"]["mode"] == mode), + "the fixture lacks an export of mode {mode}" + ); + } + assert!( + cases + .iter() + .any(|case| case.get("error").is_none() && case["export"].is_null()) + ); + let errors: Vec<&str> = cases + .iter() + .filter_map(|case| case["error"].as_str()) + .collect(); + for parameter in PARAMETERS { + for refusal in ["has no value", "is repeated", "must be"] { + let expected = format!("the {parameter} kernel parameter {refusal}"); + assert!( + errors.iter().any(|error| error.starts_with(&expected)), + "the fixture lacks the refusal {expected}" + ); + } + assert!( + errors.iter().any(|error| { + error.starts_with("the host export is incomplete") && error.contains(parameter) + }), + "the fixture lacks an export without {parameter}" + ); + } + assert!( + errors + .iter() + .any(|error| error.starts_with("the nvx_share kernel parameter")) + ); +} + +#[test] +fn parsing_reports_bytes_that_are_not_text() { + assert_eq!( + parse(&tokens("micro\u{e9}vm", ExportMode::ReadWrite)), + Err( + "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not \ + 'micro\\xc3\\xa9vm'" + .to_owned() + ) + ); + assert_eq!( + HostExport::parse(b"virtfs_dir=/run/nvx/hostfs/\xff") + .unwrap_err() + .message(), + "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/hostfs/\\xff'" + ); + assert_eq!( + HostExport::parse(b"virtfs_mode=r\x00") + .unwrap_err() + .message(), + "the virtfs_mode kernel parameter must be ro or rw, not 'r\\x00'" + ); + // Other tokens may hold any bytes. + let mut command_line = b"\xff\xfe=\x00 nvx_share\xff=1 ".to_vec(); + command_line.extend_from_slice(tokens("microvm", ExportMode::ReadOnly).as_bytes()); + assert_eq!( + HostExport::parse(&command_line).unwrap(), + Some(export("microvm", ExportMode::ReadOnly)) + ); +} + +#[test] +fn an_incomplete_export_names_what_it_lacks_in_order() { + assert_eq!(alternatives(&["a"]), "a"); + assert_eq!(alternatives(&["a", "b"]), "a or b"); + assert_eq!(alternatives(&["a", "b", "c"]), "a, b, or c"); + // Each nonempty proper subset of the tokens is incomplete, and names the others in the order + // in which OpenVMM appends them. + let all = tokens("microvm", ExportMode::ReadWrite); + let all: Vec<&str> = all.split(' ').collect(); + for subset in 1..(1 << all.len()) - 1 { + let present: Vec<&str> = (0..all.len()) + .filter(|index| subset & (1 << index) != 0) + .map(|index| all[index]) + .collect(); + let missing: Vec<&str> = (0..all.len()) + .filter(|index| subset & (1 << index) == 0) + .map(|index| PARAMETERS[index]) + .collect(); + assert_eq!( + parse(&present.join(" ")), + Err(format!( + "the host export is incomplete: the command line has no {} kernel parameter", + alternatives(&missing) + )), + "{present:?}" + ); + } +} diff --git a/nvx-rs/src/protocol/mod.rs b/nvx-rs/src/protocol/mod.rs index 2c48b3f95..9b5c03245 100644 --- a/nvx-rs/src/protocol/mod.rs +++ b/nvx-rs/src/protocol/mod.rs @@ -20,6 +20,8 @@ //! devices, of any type, that its `virtio_mmio.device` parameters describe. //! - [`root`]: the boot tokens of a layered root, which name its image's format and identity and //! its upper, with the host's renderer and the guest's parser. +//! - [`export`]: the boot tokens of the host export, the aggregate virtio-fs device from which a +//! guest maps host paths, and the guest's strict parser of them. //! - [`share`]: the guest targets that a sandbox's live shares may not take, which the host and //! the guest both refuse. //! - [`nvxs`]: the NVXS session without I/O: validated records, incremental parsing, and the @@ -37,6 +39,7 @@ mod app; mod boot; pub mod cmdline; mod exec; +pub mod export; mod features; mod maps; pub mod nvxs; diff --git a/nvx-rs/tests/fixtures/export-tokens.json b/nvx-rs/tests/fixtures/export-tokens.json new file mode 100644 index 000000000..ca865c0bf --- /dev/null +++ b/nvx-rs/tests/fixtures/export-tokens.json @@ -0,0 +1,61 @@ +{ + "description": "Kernel command lines and the host export that nvx::protocol::export parses from each, null for none, or the error with which it refuses the command line. The tests of nvx-rs replay them.", + "cases": [ + {"name": "a read-write export", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "export": {"tag": "microvm", "mode": "rw"}}, + {"name": "a read-only export", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1", "export": {"tag": "microvm", "mode": "ro"}}, + {"name": "the tokens of a managed sandbox and of the VMM", "command_line": "quiet nvx_sandbox=1 nvx_layer=distro,0xd0003000,erofs:0123abcd-4567-89ef-0123-456789abcdef nvx_scratch=0xd0006000,ext4 nvx_entrypoint=/bin/sh nvx_hostname=nvx-sandbox nr_cpus=1 nvx_workload_uid=65534 nvx_workload_gid=65534 nvx_lifecycle=managed nvx_maps=2 virtio_mmio.device=0x1000@0xd0003000:4 virtio_mmio.device=0x1000@0xd0006000:11 virtio_mmio.device=0x1000@0xd0007000:3 nvx_control_tty=hvc2 virtio_mmio.device=0x1000@0xd0001000:6 virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "export": {"tag": "microvm", "mode": "rw"}}, + {"name": "the tokens of an adapter's layered root and of the VMM", "command_line": "console=hvc0 nvx_layer=distro,0xd0003000,gpt:fedcba98-7654-3210-fedc-ba9876543210 nvx_overlay_upper=ramfs virtio_mmio.device=0x1000@0xd0003000:4 virtio_mmio.device=0x1000@0xd0001000:6 virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1", "export": {"tag": "microvm", "mode": "ro"}}, + {"name": "the tokens in reverse order", "command_line": "virtfs_aggregate=1 virtfs_mode=ro virtfs_tag=microvm virtfs_dir=/run/nvx/hostfs/root", "export": {"tag": "microvm", "mode": "ro"}}, + {"name": "tabs and newlines between tokens", "command_line": "quiet\tvirtfs_dir=/run/nvx/hostfs/root\nvirtfs_tag=microvm virtfs_mode=rw\tvirtfs_aggregate=1\n", "export": {"tag": "microvm", "mode": "rw"}}, + {"name": "a tag of every kind of byte that a tag may hold", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=AZaz09_- virtfs_mode=rw virtfs_aggregate=1", "export": {"tag": "AZaz09_-", "mode": "rw"}}, + {"name": "a tag of one byte", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=t virtfs_mode=rw virtfs_aggregate=1", "export": {"tag": "t", "mode": "rw"}}, + {"name": "a parameter named nvx_share without a value, which binds nothing", "command_line": "nvx_share virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "export": {"tag": "microvm", "mode": "rw"}}, + {"name": "parameters whose names only begin like the export's", "command_line": "virtfs_dirs=/x virtfs_tags=t virtfs_modes=rw virtfs_aggregates=1 virtfs=1 virtfs_aggregate_count=2 xvirtfs_dir=/run/nvx/hostfs/root nvx_shares=0,/workspace,rw", "export": null}, + {"name": "no export, as for a guest that the host maps no paths into", "command_line": "quiet nvx_sandbox=1 nvx_entrypoint=/bin/sh", "export": null}, + {"name": "an empty command line", "command_line": "", "export": null}, + {"name": "separators alone", "command_line": " \t\n ", "export": null}, + {"name": "a single shared directory", "command_line": "virtfs_dir=/workspace virtfs_tag=microvm virtfs_mode=rw", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/workspace'"}, + {"name": "the aggregate of a one-shot sandbox's live shares", "command_line": "virtfs_dir=/run/nvx/shares virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/shares'"}, + {"name": "the export's parent", "command_line": "virtfs_dir=/run/nvx/hostfs virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/hostfs'"}, + {"name": "a directory below the export", "command_line": "virtfs_dir=/run/nvx/hostfs/root/0 virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/hostfs/root/0'"}, + {"name": "the directory with a trailing slash", "command_line": "virtfs_dir=/run/nvx/hostfs/root/ virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/hostfs/root/'"}, + {"name": "the directory through a dot component", "command_line": "virtfs_dir=/run/nvx/./hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not '/run/nvx/./hostfs/root'"}, + {"name": "a relative directory", "command_line": "virtfs_dir=run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not 'run/nvx/hostfs/root'"}, + {"name": "an empty directory", "command_line": "virtfs_dir= virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter must be /run/nvx/hostfs/root, not ''"}, + {"name": "a directory without a value", "command_line": "virtfs_dir virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter has no value"}, + {"name": "a tag without a value", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter has no value"}, + {"name": "a mode without a value", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter has no value"}, + {"name": "the aggregate's token without its value", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate", "error": "the virtfs_aggregate kernel parameter has no value"}, + {"name": "an empty tag", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag= virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not ''"}, + {"name": "a tag with a dot", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=a.b virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not 'a.b'"}, + {"name": "a tag with a slash", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=bad/tag virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not 'bad/tag'"}, + {"name": "a tag with a comma", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=a,b virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not 'a,b'"}, + {"name": "a tag with an equals sign", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=a=b virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not 'a=b'"}, + {"name": "a tag with a quote", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=\"microvm\" virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not '\\\"microvm\\\"'"}, + {"name": "another mode", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rx virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter must be ro or rw, not 'rx'"}, + {"name": "a mode in capitals", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=RW virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter must be ro or rw, not 'RW'"}, + {"name": "both modes", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro,rw virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter must be ro or rw, not 'ro,rw'"}, + {"name": "an empty mode", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode= virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter must be ro or rw, not ''"}, + {"name": "an aggregate token that says no", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=0", "error": "the virtfs_aggregate kernel parameter must be 1, not '0'"}, + {"name": "an aggregate token in words", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=yes", "error": "the virtfs_aggregate kernel parameter must be 1, not 'yes'"}, + {"name": "an aggregate token with a leading zero", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=01", "error": "the virtfs_aggregate kernel parameter must be 1, not '01'"}, + {"name": "an empty aggregate token", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=", "error": "the virtfs_aggregate kernel parameter must be 1, not ''"}, + {"name": "two directories", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter is repeated"}, + {"name": "two tags", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_tag=other virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_tag kernel parameter is repeated"}, + {"name": "two modes", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_mode=rw virtfs_aggregate=1", "error": "the virtfs_mode kernel parameter is repeated"}, + {"name": "two aggregate tokens", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1 virtfs_aggregate=1", "error": "the virtfs_aggregate kernel parameter is repeated"}, + {"name": "a second export", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1 virtfs_dir=/run/nvx/hostfs/root virtfs_tag=other virtfs_mode=ro virtfs_aggregate=1", "error": "the virtfs_dir kernel parameter is repeated"}, + {"name": "no aggregate token", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw", "error": "the host export is incomplete: the command line has no virtfs_aggregate kernel parameter"}, + {"name": "no tag", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_mode=rw virtfs_aggregate=1", "error": "the host export is incomplete: the command line has no virtfs_tag kernel parameter"}, + {"name": "no mode", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_aggregate=1", "error": "the host export is incomplete: the command line has no virtfs_mode kernel parameter"}, + {"name": "no directory", "command_line": "virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the host export is incomplete: the command line has no virtfs_dir kernel parameter"}, + {"name": "the tag and the mode alone", "command_line": "virtfs_tag=microvm virtfs_mode=rw", "error": "the host export is incomplete: the command line has no virtfs_dir or virtfs_aggregate kernel parameter"}, + {"name": "the directory alone", "command_line": "quiet virtfs_dir=/run/nvx/hostfs/root", "error": "the host export is incomplete: the command line has no virtfs_tag, virtfs_mode, or virtfs_aggregate kernel parameter"}, + {"name": "the aggregate token alone", "command_line": "virtfs_aggregate=1", "error": "the host export is incomplete: the command line has no virtfs_dir, virtfs_tag, or virtfs_mode kernel parameter"}, + {"name": "an invalid token of an incomplete export", "command_line": "virtfs_tag=a.b", "error": "the virtfs_tag kernel parameter must be a tag of letters, digits, _, and -, not 'a.b'"}, + {"name": "a live share beside the export", "command_line": "virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1 nvx_share=0,/workspace,rw", "error": "the nvx_share kernel parameter binds a live share, but a guest that mounts the host export maps host paths from its mapping table alone"}, + {"name": "a live share before the export", "command_line": "nvx_share=0,/workspace,rw virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1", "error": "the nvx_share kernel parameter binds a live share, but a guest that mounts the host export maps host paths from its mapping table alone"}, + {"name": "a live share without an export", "command_line": "quiet nvx_share=0,/workspace,rw", "error": "the nvx_share kernel parameter binds a live share, but a guest that mounts the host export maps host paths from its mapping table alone"}, + {"name": "an empty live share", "command_line": "nvx_share=", "error": "the nvx_share kernel parameter binds a live share, but a guest that mounts the host export maps host paths from its mapping table alone"} + ] +} From b68497613398f4984fe9c86f2f7d1a32840f3177 Mon Sep 17 00:00:00 2001 From: Enrique Saurez Date: Sat, 24 Oct 2026 00:37:50 -0700 Subject: [PATCH 2/2] nvx-rs: refuse a host export directory that is not private Root::mount_export mounts the host export below hostfs in the runtime directory, a directory that only root may enter, so that a workload reaches host paths only through their mappings. A hostfs that was there already kept its owner and mode, so one that another user owned, or that its group or others could enter, would expose the export's children directly. Accept a hostfs that is there already only if it is private to the process's effective user: owned by that user, with no permission for its group or others, as the one that an earlier root leaves. Refuse any other, and leave it unchanged. NVX's shell init refuses any hostfs that is there already, since it mounts the export once per boot, while a root may be assembled again in a runtime directory where an earlier root left hostfs. Part of #822 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3d31a6be-a0f8-4952-8667-0ebdc440d08d --- nvx-rs/src/guest/root/share.rs | 78 ++++++++++++++++++++++++++++------ nvx-rs/src/guest/root/tests.rs | 66 +++++++++++++++++++++++++--- 2 files changed, 126 insertions(+), 18 deletions(-) diff --git a/nvx-rs/src/guest/root/share.rs b/nvx-rs/src/guest/root/share.rs index 4607fd4ba..5f94689ab 100644 --- a/nvx-rs/src/guest/root/share.rs +++ b/nvx-rs/src/guest/root/share.rs @@ -7,7 +7,7 @@ use std::fmt; use std::fs; use std::io; use std::os::unix::ffi::OsStrExt; -use std::os::unix::fs::{DirBuilderExt, PermissionsExt}; +use std::os::unix::fs::{DirBuilderExt, MetadataExt, PermissionsExt}; use std::path::{Path, PathBuf}; use super::{ @@ -258,9 +258,11 @@ impl Root { /// The device is mounted with `access`, `nosuid`, and `nodev` at `hostfs/root` in the runtime /// directory, outside the root, where the workload cannot list the export's children: below /// `hostfs`, a directory that the root creates with mode 0700, whatever the umask, so that only - /// root may enter it, unless a directory is there already, which keeps its mode. Neither may be - /// a symbolic link. The tag is checked first, as a live share's is; nothing is created or - /// mounted for a refused one. + /// root may enter it. A `hostfs` that is there already, such as one that an earlier root left, + /// must be as private: owned by the process's effective user, root in a guest, with no + /// permission for its group or others. Another is refused, and keeps its owner and mode. + /// Neither `hostfs` nor `hostfs/root` may be a symbolic link. The tag is checked first, as a + /// live share's is; nothing is created or mounted for a refused one. /// /// The mount belongs to the root, which unmounts it when it is torn down, after the live /// shares and mappings, those mapped from the export included. The returned [`SourceRoot`] @@ -567,8 +569,9 @@ fn open_mount_point(path: &Path) -> Result { } /// Creates the directory `path` with [`HOSTFS_DIRECTORY_MODE`], whatever the umask, unless a -/// directory, and not a symbolic link, is there already, which keeps its mode. It is created with -/// no permission beyond that mode at any time. +/// directory, and not a symbolic link, is there already, which must be private to the process's +/// effective user, as [`check_private`] requires; another is refused, and stays unchanged. It is +/// created with no permission beyond that mode at any time. fn create_private_directory(path: &Path) -> io::Result<()> { match fs::DirBuilder::new() .mode(HOSTFS_DIRECTORY_MODE) @@ -577,16 +580,36 @@ fn create_private_directory(path: &Path) -> io::Result<()> { // The umask may have withheld some of the mode's permissions. Ok(()) => fs::set_permissions(path, fs::Permissions::from_mode(HOSTFS_DIRECTORY_MODE)), Err(error) if error.kind() == io::ErrorKind::AlreadyExists => { - if fs::symlink_metadata(path)?.is_dir() { - Ok(()) - } else { - Err(Errno::ENOTDIR.into()) + let metadata = fs::symlink_metadata(path)?; + if !metadata.is_dir() { + return Err(Errno::ENOTDIR.into()); } + check_private( + metadata.uid(), + metadata.mode(), + nix::unistd::geteuid().as_raw(), + ) } Err(error) => Err(error), } } +/// Refuses a directory that the user `owner` owns with `mode` unless it is private to `user`: +/// unless `user` owns it, and it grants its group and others no permission. +fn check_private(owner: u32, mode: u32, user: u32) -> io::Result<()> { + if owner == user && mode & (Mode::S_IRWXG | Mode::S_IRWXO).bits() == 0 { + return Ok(()); + } + Err(io::Error::new( + io::ErrorKind::PermissionDenied, + format!( + "the directory there already is not private to user {user}: user {owner} owns it \ + with mode {:04o}", + mode & 0o7777 + ), + )) +} + /// Returns `error` as the error of a teardown's failed unmount: the call's own for a failed /// system call. pub(super) fn io_error(error: &MappingError) -> io::Error { @@ -751,9 +774,14 @@ mod tests { let mode = |path: &Path| fs::metadata(path).unwrap().permissions().mode() & 0o7777; sys::with_thread_umask(umask, || create_private_directory(&hostfs)).unwrap(); assert_eq!(mode(&hostfs), HOSTFS_DIRECTORY_MODE, "{umask:o}"); - // A directory that is there already keeps its mode. - fs::set_permissions(&hostfs, fs::Permissions::from_mode(0o750)).unwrap(); + // A private directory that is there already, such as this one, serves as it is. sys::with_thread_umask(umask, || create_private_directory(&hostfs)).unwrap(); + assert_eq!(mode(&hostfs), HOSTFS_DIRECTORY_MODE, "{umask:o}"); + // One that its group or others may enter is refused, and keeps its mode. + fs::set_permissions(&hostfs, fs::Permissions::from_mode(0o750)).unwrap(); + let error = + sys::with_thread_umask(umask, || create_private_directory(&hostfs)).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{umask:o}"); assert_eq!(mode(&hostfs), 0o750, "{umask:o}"); } // Anything else there is refused, a symbolic link to a directory included. @@ -772,6 +800,32 @@ mod tests { } } + #[test] + fn a_directory_is_private_to_its_owner_when_its_group_and_others_have_no_permission() { + // The owner's permissions, the file type, and the set-ID and sticky bits grant no one + // else access. + for mode in [0o700, 0o500, 0o000, 0o40700, 0o2700, 0o1700] { + check_private(0, mode, 0).unwrap(); + check_private(1000, mode, 1000).unwrap(); + } + for mode in [0o710, 0o701, 0o750, 0o705, 0o755, 0o777, 0o40711] { + let error = check_private(0, mode, 0).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{mode:o}"); + } + assert_eq!( + check_private(0, 0o40750, 0).unwrap_err().to_string(), + "the directory there already is not private to user 0: user 0 owns it with mode 0750" + ); + // Another user's directory is refused, however private. + let error = check_private(1000, 0o40700, 0).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied); + assert_eq!( + error.to_string(), + "the directory there already is not private to user 0: user 1000 owns it with mode \ + 0700" + ); + } + #[test] fn an_export_s_mode_is_the_access_of_its_mount() { assert_eq!(Access::from(ExportMode::ReadOnly), Access::ReadOnly); diff --git a/nvx-rs/src/guest/root/tests.rs b/nvx-rs/src/guest/root/tests.rs index 04d31b97b..03f70ebb8 100644 --- a/nvx-rs/src/guest/root/tests.rs +++ b/nvx-rs/src/guest/root/tests.rs @@ -3242,7 +3242,12 @@ fn the_host_export_is_mounted_through_no_symbolic_link() { let guest = Guest::new(); let mut root = guest.assemble(&guest.assembly()).unwrap(); fs::create_dir(guest.path("elsewhere")).unwrap(); - fs::create_dir_all(guest.path(link).parent().unwrap()).unwrap(); + let parent = guest.path(link).parent().unwrap().to_owned(); + if !parent.exists() { + // A private `hostfs`, as an earlier root leaves it. + fs::create_dir(&parent).unwrap(); + fs::set_permissions(&parent, fs::Permissions::from_mode(0o700)).unwrap(); + } std::os::unix::fs::symlink(guest.path("elsewhere"), guest.path(link)).unwrap(); let mounts = guest.kernel.mounts().len(); let error = root.mount_export("microvm", Access::ReadOnly).unwrap_err(); @@ -3264,16 +3269,65 @@ fn the_host_export_is_mounted_through_no_symbolic_link() { assert!(root.is_latched(), "{link}"); root.teardown().unwrap(); } +} + +#[test] +fn a_hostfs_directory_that_is_there_already_must_be_private() { + // One that its group or others may enter is refused, keeps its mode, and latches the root. + let user = nix::unistd::geteuid().as_raw(); + for mode in [0o750, 0o705, 0o711] { + let guest = Guest::new(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + let hostfs = guest.path("run/nvx/hostfs"); + fs::create_dir(&hostfs).unwrap(); + fs::set_permissions(&hostfs, fs::Permissions::from_mode(mode)).unwrap(); + let mounts = guest.kernel.mounts().len(); + let error = root.mount_export("microvm", Access::ReadOnly).unwrap_err(); + assert!( + matches!( + &error, + Error::MountPoint { path, error } + if *path == hostfs && error.kind() == io::ErrorKind::PermissionDenied + ), + "{mode:o}: {error:?}" + ); + assert_eq!( + error.to_string(), + format!( + "failed to prepare {}: the directory there already is not private to user \ + {user}: user {user} owns it with mode {mode:04o}", + hostfs.display() + ) + ); + assert_eq!(guest.kernel.mounts().len(), mounts, "{mode:o}"); + assert!(!guest.path("run/nvx/hostfs/root").exists(), "{mode:o}"); + assert_eq!(fs::metadata(&hostfs).unwrap().mode() & 0o7777, mode); + assert!(root.is_latched(), "{mode:o}"); + root.teardown().unwrap(); + } - // A directory that is there already keeps its mode. + // One that only its owner, the process's effective user, may enter serves, such as the one + // that an earlier root left in the runtime directory. let guest = Guest::new(); let mut root = guest.assemble(&guest.assembly()).unwrap(); - let hostfs = guest.path("run/nvx/hostfs"); - fs::create_dir(&hostfs).unwrap(); - fs::set_permissions(&hostfs, fs::Permissions::from_mode(0o750)).unwrap(); drop(root.mount_export("microvm", Access::ReadOnly).unwrap()); - assert_eq!(fs::metadata(&hostfs).unwrap().mode() & 0o7777, 0o750); root.teardown().unwrap(); + let mut root = guest.assemble(&guest.assembly()).unwrap(); + drop(root.mount_export("microvm", Access::ReadWrite).unwrap()); + let mode = fs::metadata(guest.path("run/nvx/hostfs")).unwrap().mode() & 0o7777; + assert_eq!(mode, 0o700); + assert!(!root.is_latched()); + root.teardown().unwrap(); + let unmounts = [ + guest.text("run/nvx/hostfs/root"), + guest.text("run/nvx/rootfs"), + guest.text("run/nvx/layers/distro"), + guest.text("run/nvx/scratch"), + ]; + assert_eq!( + guest.kernel.unmounts(), + [unmounts.clone(), unmounts].concat() + ); } #[test]