Skip to content

Add sandbox and networking test suites - #6

Draft
dmcgowan wants to merge 6 commits into
mainfrom
sandbox-tests
Draft

Add sandbox and networking test suites#6
dmcgowan wants to merge 6 commits into
mainfrom
sandbox-tests

Conversation

@dmcgowan

Copy link
Copy Markdown
Member

Add SandboxSuite covering container-in-sandbox lifecycle, and CRI and network-namespace variants, plus supporting helpers for creating member containers within a sandbox, real network namespace manipulation, and stress-testing sandbox creation/teardown under load.

Fix two sandbox tests that still referenced /bin/connect, which was removed when NetworkSuite was redesigned to use nc(1)/host(1) instead of bespoke connect/udpconnect/resolve commands:

  • ContainerOutboundTCP now uses /bin/host, since Task.Exec (used to run processes inside an already-running member container) has no stdin plumbing and nc's TCP mode requires stdin.
  • ContainerTrafficScopedToNetworkSandbox now uses nc(1) in TCP mode over a new opt-in stdin FIFO (withSandboxCtrStdin / writeContainerStdin), mirroring the pattern already used by NetworkSuite's legacy-path tests. This test is root-gated and was never caught by CI.

Add a MemberContainersShareNetwork test verifying that member containers of the same sandbox share a network stack, the way containers in a Kubernetes pod share the pod's network namespace. This needs an in-guest TCP listener that terminates on its own after one exchange, which nc(1)'s existing stream modes can't provide, so add a purpose-built "echosrv " testbin command that binds tcp4 explicitly, accepts one connection, echoes one read verbatim, and exits.

Assert that SandboxStatus.State uses the sandbox API's own interoperable state vocabulary (SANDBOX_READY / SANDBOX_NOTREADY) rather than an ad hoc string like "ready": the base runtime/sandbox/v1 proto documents State as a free-form string, but an unconstrained string is not a usable contract, since any caller branching on readiness must match against a fixed vocabulary. containerd's CRI layer is one caller that maps these exact names onto the CRI v1 PodSandboxState enum. Adds sandboxStateReady/sandboxStateNotReady constants and upgrades two "status after stop" checks from soft t.Logf notes to real assertions.

Add MemberContainerHostVolume, verifying that a member container's "bind" mount referencing a host directory is a real, live share: a file updated on the host after the container has already started must become visible inside it.

Add MemberContainersSharePID and MemberContainersShareIPC, verifying that member containers of a sandbox can share a PID namespace (cross-container process visibility via /proc) and an IPC namespace (a SysV shared memory segment created by one container is visible to another via a well-known key) when a host path is set on the corresponding namespace entry of their OCI spec. Adds pidscan, shmwrite, and shmread testbin commands, and a withSandboxCtrNamespace / withHostPathNamespace helper pair.

Reframe every test and helper comment that justified a behavior by appeal to CRI or Kubernetes semantics ("the CRI layer requires...", "the way containers in a Kubernetes pod...", "containerd's WithPodNamespaces does...") to state the contract in terms of the shim API itself: what the caller sends (an OCI spec field, a CreateSandboxRequest parameter) and what the shim must do in response. CRI and Kubernetes now appear only as parenthetical "e.g." examples of one consumer of the contract, never as the reason the contract exists, so the suite specifies shim behavior for any current or future caller. Also renames sandbox_suite_cri.go and sandbox_suite_cri_linux.go to sandbox_suite_member.go and sandbox_suite_member_linux.go, and drops a stray implementation-specific detail ("the libkrun FFI thread") that had named a particular shim's internals in a supposedly implementation-neutral comment.

Updates the README's Tests table throughout to document all of the above, including renaming the "CRI/Kubernetes workload conformance tests" section to "Member-container workload contracts."

Add SandboxSuite covering container-in-sandbox lifecycle, and CRI and
network-namespace variants, plus supporting helpers for creating member
containers within a sandbox, real network namespace manipulation, and
stress-testing sandbox creation/teardown under load.

Fix two sandbox tests that still referenced /bin/connect, which was
removed when NetworkSuite was redesigned to use nc(1)/host(1) instead
of bespoke connect/udpconnect/resolve commands:

- ContainerOutboundTCP now uses /bin/host, since Task.Exec (used to run
  processes inside an already-running member container) has no stdin
  plumbing and nc's TCP mode requires stdin.
- ContainerTrafficScopedToNetworkSandbox now uses nc(1) in TCP mode over
  a new opt-in stdin FIFO (withSandboxCtrStdin / writeContainerStdin),
  mirroring the pattern already used by NetworkSuite's legacy-path
  tests. This test is root-gated and was never caught by CI.

Add a MemberContainersShareNetwork test verifying that member containers
of the same sandbox share a network stack, the way containers in a
Kubernetes pod share the pod's network namespace. This needs an
in-guest TCP listener that terminates on its own after one exchange,
which nc(1)'s existing stream modes can't provide, so add a
purpose-built "echosrv <port>" testbin command that binds tcp4
explicitly, accepts one connection, echoes one read verbatim, and
exits.

Assert that SandboxStatus.State uses the sandbox API's own interoperable
state vocabulary (SANDBOX_READY / SANDBOX_NOTREADY) rather than an
ad hoc string like "ready": the base runtime/sandbox/v1 proto documents
State as a free-form string, but an unconstrained string is not a usable
contract, since any caller branching on readiness must match against a
fixed vocabulary. containerd's CRI layer is one caller that maps these
exact names onto the CRI v1 PodSandboxState enum. Adds
sandboxStateReady/sandboxStateNotReady constants and upgrades two
"status after stop" checks from soft t.Logf notes to real assertions.

Add MemberContainerHostVolume, verifying that a member container's
"bind" mount referencing a host directory is a real, live share: a file
updated on the host after the container has already started must become
visible inside it.

Add MemberContainersSharePID and MemberContainersShareIPC, verifying
that member containers of a sandbox can share a PID namespace
(cross-container process visibility via /proc) and an IPC namespace (a
SysV shared memory segment created by one container is visible to
another via a well-known key) when a host path is set on the
corresponding namespace entry of their OCI spec. Adds pidscan,
shmwrite, and shmread testbin commands, and a withSandboxCtrNamespace /
withHostPathNamespace helper pair.

Reframe every test and helper comment that justified a behavior by
appeal to CRI or Kubernetes semantics ("the CRI layer requires...",
"the way containers in a Kubernetes pod...", "containerd's
WithPodNamespaces does...") to state the contract in terms of the shim
API itself: what the caller sends (an OCI spec field, a
CreateSandboxRequest parameter) and what the shim must do in response.
CRI and Kubernetes now appear only as parenthetical "e.g." examples of
one consumer of the contract, never as the reason the contract exists,
so the suite specifies shim behavior for any current or future caller.
Also renames sandbox_suite_cri.go and sandbox_suite_cri_linux.go to
sandbox_suite_member.go and sandbox_suite_member_linux.go, and drops a
stray implementation-specific detail ("the libkrun FFI thread") that
had named a particular shim's internals in a supposedly
implementation-neutral comment.

Updates the README's Tests table throughout to document all of the
above, including renaming the "CRI/Kubernetes workload conformance
tests" section to "Member-container workload contracts."

Skip the new "sandbox" feature in the runc CI profiles:
containerd-shim-runc-v2 does not implement the runtime.sandbox.v1.Sandbox
service at all, so every SandboxSuite test fails against it with
Unimplemented. Add "sandbox" to the skip list in both runc.json and
runc-rootless.json, matching how other shim-specific gaps (oom on
rootless, transfer/uds/layers everywhere runc lacks them) are already
handled.

Signed-off-by: Derek McGowan <derek@mcg.dev>
Skip the "net" feature in both runc CI profiles. NetworkSuite is
wired into TestShim here for the first time, and a bare
containerd-shim-runc-v2 container has no network setup at all (no
CNI, no veth, no port mapping) in either profile: it gets either the
host's own network namespace (root profile) or a fresh, empty one
with only a down loopback (rootless profile, since createOCISpec
adds an isolated network namespace whenever running non-root). In
both cases nc(1) has no way to reach the host, so every test in this
suite fails against runc — TCP fails fast, but UDP hangs
indefinitely (nc -u's recvfrom has no deadline), timing out the
whole 2-minute test run. Skipping "net" restores runc's prior
behavior (before this commit, NetworkSuite was never invoked at
all) and keeps network coverage on shims with real container
networking. Making NetworkSuite work against bare runc is left for a
follow-up focused on that specifically.

Signed-off-by: Derek McGowan <derek@mcg.dev>
Add MemberContainersSharePIDKillScopedToOwnContainer, verifying
Task.Kill/Pids stay scoped to their own container despite a shared
PID namespace. Add MemberContainersShareDevShm and
MemberContainersDevShmNotSharedWithoutIPC, verifying /dev/shm sharing
follows the IPC-sharing signal rather than the mount's shape.

Adds shmmapwrite/shmmapread testbin commands (mmap(MAP_SHARED)
based, unlike the existing SysV shmwrite/shmread) to back the new
/dev/shm tests.

Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Derek McGowan <derek@mcg.dev>
Verify that the kernel OOM-killing one member container (a
memory-limited container running memhog, as in the standalone OOM
test) does not affect a sibling member container or the sandbox
itself. This is the OOM-specific counterpart to
ContainerLifecycleIndependence, which only covers a peer's graceful
exit.

Adds withSandboxCtrOCIOpts, a generic passthrough for low-level OCI
spec opts (e.g. withMemoryLimit) on sandbox member containers, reused
by the OOM test to apply a memory limit.

Signed-off-by: Derek McGowan <derek@mcg.dev>
Add MemberContainersShareUTS and its converse
MemberContainersUTSNotSharedWithoutUTSPath, mirroring the existing
PID/IPC namespace sharing tests: a hostname change made via the
standard 'hostname <name>' command by one member container must be
visible to a peer sharing its UTS namespace, and must not be visible
to one that isn't.

Adds a single 'hostname' testbin command matching the standard
utility's CLI exactly (bare 'hostname' prints the kernel-reported
name; 'hostname <name>' sets it, exiting silently on success and
with a 'hostname: ...' stderr message otherwise), rather than two
custom-named commands, consistent with how the other testbin
commands (cat, date, echo, ls, host, nc) mirror their standard
counterparts. Because 'hostname <name>' exits immediately rather
than staying running, verifying a peer sees the change also proves
the shared namespace outlives the container that set it, not merely
that a still-running setter's own namespace is visible.

sethostname(2) requires CAP_SYS_ADMIN, which shimtest's base
container spec does not grant, so this adds withCapabilities (a
CreateOCISpec opt to request specific Linux capabilities) and
requests it explicitly on the hostname-setting container. This is a
property of the requested container, not the host process running
the test suite, and needs no host-level privilege — confirmed
empirically against runc, which grants a container zero capabilities
unless the spec asks for them, root or not. Since the base container
spec's silence on capabilities means standalone (non-sandboxed)
containers built the same way inherit the *caller's* UTS namespace
when none is requested, this was verified in an isolated
'unshare --uts' namespace rather than through a live container, to
avoid any risk of changing a real host's hostname.

Whether a shim honors the capability request is a separate contract
from namespace sharing; a fixed capability probe container runs
first and the dependent tests are skipped, not failed, if it
doesn't.

Adds a containerOutputSnapshot sandbox helper to read a completed
container's full captured stdout without needing to wait for a
fixed substring, since the hostname a test observes isn't known
until runtime.

Signed-off-by: Derek McGowan <derek@mcg.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant