Add sandbox and networking test suites - #6
Draft
dmcgowan wants to merge 6 commits into
Draft
Conversation
Add SandboxSuite covering container-in-sandbox lifecycle, and CRI and
network-namespace variants, plus supporting helpers for creating member
containers within a sandbox, real network namespace manipulation, and
stress-testing sandbox creation/teardown under load.
Fix two sandbox tests that still referenced /bin/connect, which was
removed when NetworkSuite was redesigned to use nc(1)/host(1) instead
of bespoke connect/udpconnect/resolve commands:
- ContainerOutboundTCP now uses /bin/host, since Task.Exec (used to run
processes inside an already-running member container) has no stdin
plumbing and nc's TCP mode requires stdin.
- ContainerTrafficScopedToNetworkSandbox now uses nc(1) in TCP mode over
a new opt-in stdin FIFO (withSandboxCtrStdin / writeContainerStdin),
mirroring the pattern already used by NetworkSuite's legacy-path
tests. This test is root-gated and was never caught by CI.
Add a MemberContainersShareNetwork test verifying that member containers
of the same sandbox share a network stack, the way containers in a
Kubernetes pod share the pod's network namespace. This needs an
in-guest TCP listener that terminates on its own after one exchange,
which nc(1)'s existing stream modes can't provide, so add a
purpose-built "echosrv <port>" testbin command that binds tcp4
explicitly, accepts one connection, echoes one read verbatim, and
exits.
Assert that SandboxStatus.State uses the sandbox API's own interoperable
state vocabulary (SANDBOX_READY / SANDBOX_NOTREADY) rather than an
ad hoc string like "ready": the base runtime/sandbox/v1 proto documents
State as a free-form string, but an unconstrained string is not a usable
contract, since any caller branching on readiness must match against a
fixed vocabulary. containerd's CRI layer is one caller that maps these
exact names onto the CRI v1 PodSandboxState enum. Adds
sandboxStateReady/sandboxStateNotReady constants and upgrades two
"status after stop" checks from soft t.Logf notes to real assertions.
Add MemberContainerHostVolume, verifying that a member container's
"bind" mount referencing a host directory is a real, live share: a file
updated on the host after the container has already started must become
visible inside it.
Add MemberContainersSharePID and MemberContainersShareIPC, verifying
that member containers of a sandbox can share a PID namespace
(cross-container process visibility via /proc) and an IPC namespace (a
SysV shared memory segment created by one container is visible to
another via a well-known key) when a host path is set on the
corresponding namespace entry of their OCI spec. Adds pidscan,
shmwrite, and shmread testbin commands, and a withSandboxCtrNamespace /
withHostPathNamespace helper pair.
Reframe every test and helper comment that justified a behavior by
appeal to CRI or Kubernetes semantics ("the CRI layer requires...",
"the way containers in a Kubernetes pod...", "containerd's
WithPodNamespaces does...") to state the contract in terms of the shim
API itself: what the caller sends (an OCI spec field, a
CreateSandboxRequest parameter) and what the shim must do in response.
CRI and Kubernetes now appear only as parenthetical "e.g." examples of
one consumer of the contract, never as the reason the contract exists,
so the suite specifies shim behavior for any current or future caller.
Also renames sandbox_suite_cri.go and sandbox_suite_cri_linux.go to
sandbox_suite_member.go and sandbox_suite_member_linux.go, and drops a
stray implementation-specific detail ("the libkrun FFI thread") that
had named a particular shim's internals in a supposedly
implementation-neutral comment.
Updates the README's Tests table throughout to document all of the
above, including renaming the "CRI/Kubernetes workload conformance
tests" section to "Member-container workload contracts."
Skip the new "sandbox" feature in the runc CI profiles:
containerd-shim-runc-v2 does not implement the runtime.sandbox.v1.Sandbox
service at all, so every SandboxSuite test fails against it with
Unimplemented. Add "sandbox" to the skip list in both runc.json and
runc-rootless.json, matching how other shim-specific gaps (oom on
rootless, transfer/uds/layers everywhere runc lacks them) are already
handled.
Signed-off-by: Derek McGowan <derek@mcg.dev>
dmcgowan
force-pushed
the
sandbox-tests
branch
from
August 10, 2026 22:35
b4ed32d to
4c60b76
Compare
Skip the "net" feature in both runc CI profiles. NetworkSuite is wired into TestShim here for the first time, and a bare containerd-shim-runc-v2 container has no network setup at all (no CNI, no veth, no port mapping) in either profile: it gets either the host's own network namespace (root profile) or a fresh, empty one with only a down loopback (rootless profile, since createOCISpec adds an isolated network namespace whenever running non-root). In both cases nc(1) has no way to reach the host, so every test in this suite fails against runc — TCP fails fast, but UDP hangs indefinitely (nc -u's recvfrom has no deadline), timing out the whole 2-minute test run. Skipping "net" restores runc's prior behavior (before this commit, NetworkSuite was never invoked at all) and keeps network coverage on shims with real container networking. Making NetworkSuite work against bare runc is left for a follow-up focused on that specifically. Signed-off-by: Derek McGowan <derek@mcg.dev>
Add MemberContainersSharePIDKillScopedToOwnContainer, verifying Task.Kill/Pids stay scoped to their own container despite a shared PID namespace. Add MemberContainersShareDevShm and MemberContainersDevShmNotSharedWithoutIPC, verifying /dev/shm sharing follows the IPC-sharing signal rather than the mount's shape. Adds shmmapwrite/shmmapread testbin commands (mmap(MAP_SHARED) based, unlike the existing SysV shmwrite/shmread) to back the new /dev/shm tests. Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Derek McGowan <derek@mcg.dev>
Verify that the kernel OOM-killing one member container (a memory-limited container running memhog, as in the standalone OOM test) does not affect a sibling member container or the sandbox itself. This is the OOM-specific counterpart to ContainerLifecycleIndependence, which only covers a peer's graceful exit. Adds withSandboxCtrOCIOpts, a generic passthrough for low-level OCI spec opts (e.g. withMemoryLimit) on sandbox member containers, reused by the OOM test to apply a memory limit. Signed-off-by: Derek McGowan <derek@mcg.dev>
Add MemberContainersShareUTS and its converse MemberContainersUTSNotSharedWithoutUTSPath, mirroring the existing PID/IPC namespace sharing tests: a hostname change made via the standard 'hostname <name>' command by one member container must be visible to a peer sharing its UTS namespace, and must not be visible to one that isn't. Adds a single 'hostname' testbin command matching the standard utility's CLI exactly (bare 'hostname' prints the kernel-reported name; 'hostname <name>' sets it, exiting silently on success and with a 'hostname: ...' stderr message otherwise), rather than two custom-named commands, consistent with how the other testbin commands (cat, date, echo, ls, host, nc) mirror their standard counterparts. Because 'hostname <name>' exits immediately rather than staying running, verifying a peer sees the change also proves the shared namespace outlives the container that set it, not merely that a still-running setter's own namespace is visible. sethostname(2) requires CAP_SYS_ADMIN, which shimtest's base container spec does not grant, so this adds withCapabilities (a CreateOCISpec opt to request specific Linux capabilities) and requests it explicitly on the hostname-setting container. This is a property of the requested container, not the host process running the test suite, and needs no host-level privilege — confirmed empirically against runc, which grants a container zero capabilities unless the spec asks for them, root or not. Since the base container spec's silence on capabilities means standalone (non-sandboxed) containers built the same way inherit the *caller's* UTS namespace when none is requested, this was verified in an isolated 'unshare --uts' namespace rather than through a live container, to avoid any risk of changing a real host's hostname. Whether a shim honors the capability request is a separate contract from namespace sharing; a fixed capability probe container runs first and the dependent tests are skipped, not failed, if it doesn't. Adds a containerOutputSnapshot sandbox helper to read a completed container's full captured stdout without needing to wait for a fixed substring, since the hostname a test observes isn't known until runtime. Signed-off-by: Derek McGowan <derek@mcg.dev>
dmcgowan
force-pushed
the
sandbox-tests
branch
from
August 11, 2026 06:06
97b443a to
1a5ee4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add SandboxSuite covering container-in-sandbox lifecycle, and CRI and network-namespace variants, plus supporting helpers for creating member containers within a sandbox, real network namespace manipulation, and stress-testing sandbox creation/teardown under load.
Fix two sandbox tests that still referenced /bin/connect, which was removed when NetworkSuite was redesigned to use nc(1)/host(1) instead of bespoke connect/udpconnect/resolve commands:
Add a MemberContainersShareNetwork test verifying that member containers of the same sandbox share a network stack, the way containers in a Kubernetes pod share the pod's network namespace. This needs an in-guest TCP listener that terminates on its own after one exchange, which nc(1)'s existing stream modes can't provide, so add a purpose-built "echosrv " testbin command that binds tcp4 explicitly, accepts one connection, echoes one read verbatim, and exits.
Assert that SandboxStatus.State uses the sandbox API's own interoperable state vocabulary (SANDBOX_READY / SANDBOX_NOTREADY) rather than an ad hoc string like "ready": the base runtime/sandbox/v1 proto documents State as a free-form string, but an unconstrained string is not a usable contract, since any caller branching on readiness must match against a fixed vocabulary. containerd's CRI layer is one caller that maps these exact names onto the CRI v1 PodSandboxState enum. Adds sandboxStateReady/sandboxStateNotReady constants and upgrades two "status after stop" checks from soft t.Logf notes to real assertions.
Add MemberContainerHostVolume, verifying that a member container's "bind" mount referencing a host directory is a real, live share: a file updated on the host after the container has already started must become visible inside it.
Add MemberContainersSharePID and MemberContainersShareIPC, verifying that member containers of a sandbox can share a PID namespace (cross-container process visibility via /proc) and an IPC namespace (a SysV shared memory segment created by one container is visible to another via a well-known key) when a host path is set on the corresponding namespace entry of their OCI spec. Adds pidscan, shmwrite, and shmread testbin commands, and a withSandboxCtrNamespace / withHostPathNamespace helper pair.
Reframe every test and helper comment that justified a behavior by appeal to CRI or Kubernetes semantics ("the CRI layer requires...", "the way containers in a Kubernetes pod...", "containerd's WithPodNamespaces does...") to state the contract in terms of the shim API itself: what the caller sends (an OCI spec field, a CreateSandboxRequest parameter) and what the shim must do in response. CRI and Kubernetes now appear only as parenthetical "e.g." examples of one consumer of the contract, never as the reason the contract exists, so the suite specifies shim behavior for any current or future caller. Also renames sandbox_suite_cri.go and sandbox_suite_cri_linux.go to sandbox_suite_member.go and sandbox_suite_member_linux.go, and drops a stray implementation-specific detail ("the libkrun FFI thread") that had named a particular shim's internals in a supposedly implementation-neutral comment.
Updates the README's Tests table throughout to document all of the above, including renaming the "CRI/Kubernetes workload conformance tests" section to "Member-container workload contracts."