Skip to content

Commit e2f1387

Browse files
docs: correct the skip matrix in the agent-context files (#42)
The table claimed a missing Docker daemon degrades to a silent skip. TestCreateToRunDocker runs in a container runtime context with no tooling guard, so it fails loudly instead, and the files now say so along with a warning not to quiet it with a skip guard. It also omitted nix. CI has no nix and the matrix skips any absent backend, so TestPythonFastAPILifecycle_Matrix/nix is the one skip in a green CI run and the nix path is covered nowhere — which the files have to state, since they direct attention at nix elsewhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 7026548 commit e2f1387

2 files changed

Lines changed: 28 additions & 15 deletions

File tree

‎.claude/skills/lifecycle-test-triage/SKILL.md‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,20 @@ go test -v ./... 2>&1 | grep -E '^\s*--- SKIP'
2121
| Missing | Skips |
2222
| --- | --- |
2323
| `uv` | the real uvicorn lifecycle |
24-
| `docker` | destroy-under-docker |
2524
| `syft`, `docker`, `uv` | the image SBOM inventory |
2625
| `python3` | the active-environment test |
27-
28-
CI installs `uv` and pins `syft` v1.48.0. If a skip covers the surface you
29-
changed, install the tool and re-run — do not report the green.
26+
| `nix` | the lifecycle matrix's nix backend |
27+
| a stopped daemon | the matrix's docker backend, and destroy-under-docker |
28+
29+
CI installs `uv` and pins `syft` v1.48.0, but **has no `nix`** — so
30+
`TestPythonFastAPILifecycle_Matrix/nix` is the one skip in a green CI run and
31+
the nix path is covered nowhere. If a skip covers the surface you changed,
32+
install the tool and re-run — do not report the green.
33+
34+
A missing Docker daemon is the exception: `TestCreateToRunDocker` has no
35+
tooling guard and fails rather than skips. That red is correct. Do not add a
36+
skip guard to make it green — that converts the loudest signal in the suite
37+
into the silent pass this whole section exists to prevent.
3038

3139
## 2. `panic: test timed out after 10m0s`
3240

‎AGENTS.md‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -81,17 +81,22 @@ matters. The lock order is documented on the struct fields — follow it.
8181

8282
**Nothing is mocked.** The suite drives a real uvicorn through `uv`, real
8383
containers through the Docker daemon, and a real `syft` scan of an image built
84-
from the recipe this agent emits. Tests skip rather than fake when a tool is
85-
absent, so a green run on a bare machine has proved much less than it looks:
86-
87-
| Missing | Silently skips |
88-
| --- | --- |
89-
| `uv` | the real uvicorn lifecycle |
90-
| `docker` | destroy-under-docker |
91-
| `syft` (+ `docker`, `uv`) | the image SBOM inventory |
92-
93-
CI installs `uv` and pins `syft` v1.48.0 in `setup-run` for exactly that
94-
reason. Get them locally too, or say which halves you did not exercise.
84+
from the recipe this agent emits. Where a tool is absent most of these tests
85+
skip rather than fake, so a green run proves less than it looks — `uv`, `syft`,
86+
`python3` and `nix` all degrade silently. A missing Docker daemon does not:
87+
`TestCreateToRunDocker` runs in a container runtime context with no tooling
88+
guard, so it fails loudly. Never quiet that failure with a skip guard; a loud
89+
red is the correct outcome and the skip would buy a false green.
90+
91+
**CI has no `nix`, so the nix backend is covered nowhere.**
92+
`testmatrix.ForEachEnvironment` skips any backend missing from the host, and
93+
this repo does not pass `Only(...)` to make absence fatal — so
94+
`TestPythonFastAPILifecycle_Matrix/nix` is the one skip in a green CI run.
95+
Changing `nixflake.go` or `nix/flake.nix` is unverified by CI: exercise it
96+
locally with `nix` installed, and say in the PR that you did.
97+
98+
CI installs `uv` and pins `syft` v1.48.0 in `setup-run` so the other halves do
99+
run. The full skip matrix is in the `lifecycle-test-triage` skill.
95100

96101
The suite is slow because it is real — CI finishes inside `go test`'s default
97102
10-minute timeout, but a contended local Docker daemon pushes the container

0 commit comments

Comments
 (0)