From 49ed6e2aecf07970980651f56ab83f3465a7bf7a Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Thu, 8 Oct 2026 17:29:21 -0400 Subject: [PATCH 01/18] fix(run): build recipe images for local container runtimes --- docs/development-loop.md | 49 ++- go.mod | 2 +- go.sum | 2 + pkg/conformance/matrix.json | 2 +- pkg/orchestration/build_plan.go | 51 ++- pkg/orchestration/builder.go | 7 + pkg/orchestration/local_runtime_image.go | 152 +++++++++ pkg/orchestration/local_runtime_image_test.go | 293 ++++++++++++++++++ pkg/orchestration/runner.go | 5 + .../runner_accepted_network_mappings_test.go | 3 + 10 files changed, 545 insertions(+), 21 deletions(-) create mode 100644 pkg/orchestration/local_runtime_image.go create mode 100644 pkg/orchestration/local_runtime_image_test.go diff --git a/docs/development-loop.md b/docs/development-loop.md index ead80f8ec..2ff93bac2 100644 --- a/docs/development-loop.md +++ b/docs/development-loop.md @@ -13,7 +13,7 @@ loop that changes the CLI itself, see [development.md](development.md). ## The cycle ``` -edit ──▶ codefly run service local, no image build seconds +edit ──▶ codefly run service local, builds runtime recipes when needed └▶ codefly deploy gitops render builds + pushes images minutes (kept when nothing changed) codefly deploy secrets seeds the environment store one store read per key codefly deploy gitops plan the publication diff seconds @@ -23,7 +23,7 @@ edit ──▶ codefly run service local, no image build seconds | Step | Verb | What it does | Cost | |---|---|---|---| -| Run it locally | `codefly run service ` | Starts the service and its dependency graph on this machine, injecting connection strings as env vars. No image is built — a docker-backed dependency runs from an image it already has. | Seconds. This is the inner loop; stay in it as long as you can. | +| Run it locally | `codefly run service ` | Starts the service and its dependency graph on this machine, injecting connection strings as env vars. Container services with agent-emitted recipes build and load their runtime image before Init; unchanged inputs reuse the loaded image. Stock-image services retain their own image selection. | Seconds when images are already built; the first recipe build can take minutes. | | Render | `codefly deploy gitops render --env ` | Resolves the module, builds and pushes every service image, writes the promotable tree under `deployments/modules//` and records `.codefly-render.json`. | Minutes — the image builds. Images whose inputs did not change are kept, so a configuration-only edit costs the render and no build. | | Seed secrets | `codefly deploy secrets --env ` | Reads the ExternalSecrets the render projected and resolves each remote property to a source — kept, derived, propagated, generated or required — then writes the store. Values are never printed. | One backend read per remote key, eight at a time: seconds for a small environment, longer for a large one. Run it after a render that added a service or a secret key. | | Plan | `codefly deploy gitops plan --env ` | Shows the exact publication diff — what publishing would change in the GitOps repository — and writes nothing. | Seconds. | @@ -39,6 +39,47 @@ after the images are built. Every flag of every verb is in --- +## Local container runtime images + +After selecting the container backend, the CLI loads and initializes the agent's +builder and asks it for its recipe when the manifest names no runtime image. +A single-image recipe is built using the +same verified buildx execution path as deploy, with `--load` for the local +Docker engine's architecture and no registry push. Native and Nix runtimes do +not enter this path. An agent that emits no recipe, or declares artifact builds +unsupported, retains its existing runtime image selection. + +The running agent must advertise `runtime-init-image/v1` before the CLI builds +and supplies a recipe image through `InitRequest.runtime_image` (field 10). +Bindings alone do not prove adoption. An older recipe-producing agent without +the capability is refused before image execution. This handoff follows +[handbook proposal #260](https://github.com/obin-ai/handbook/pull/260) and +[Core #748](https://github.com/codefly-dev/core/issues/748). + +The image tag is `codefly-local/runtime:sha256-<64 hex digits>`. The hash uses +the existing image-input identity: verified recipe files, Docker's effective +build context, target and build arguments, selected platform, Go module +manifests and resolved base-image digests. The destination image is normalized +in the build invocation before hashing, avoiding a circular hash; the original +agent plan digest, including its metadata, remains an input. Temporary paths +and builder names are normalized by the existing cache. +There is no timestamp, checkout path or `latest` in the result. + +The next run emits and verifies the recipe again, then reuses the cached image +only if its inputs match and Docker still holds the recorded image ID. A source, +recipe, argument or base-image change changes the tag. Inputs that cannot be +identified, or a context that changes during the build, fail the local handoff. +Network downloads inside a Dockerfile's `RUN` remain subject to the same +limitation as the build cache described below. + +Generated recipes and their image references stay under +`.codefly/runtime-build///`; reuse records stay under +`.codefly/build-cache/`. Nothing is written into `spec.runtime-image` or the +authored service tree. The agent retains responsibility for manifest precedence +and reference validation, including refusing an invalid explicit manifest value +even when the request supplies a valid image. Remote runtime initialization does +not receive a local image. + ## What is expensive, and what is kept The image build is the only step measured in minutes. A build reuses the image @@ -58,7 +99,9 @@ So: - **Reuse is verified, not assumed.** A pushed image is reused only if the registry still serves the recorded manifest digest, a loaded one only if the daemon still holds the recorded image ID. Anything the key cannot account for — - an unresolvable base image, an unwalkable context — builds. + an unresolvable base image, an unwalkable context — builds on the deployment + path. A local runtime handoff refuses those inputs because it cannot derive + an identifiable tag for them. The full rules, the exclusions, and what no digest over inputs can bind are in [commands.md → Keeping an image whose inputs did not change](commands.md#keeping-an-image-whose-inputs-did-not-change). diff --git a/go.mod b/go.mod index f0c6bdc32..032e5a009 100644 --- a/go.mod +++ b/go.mod @@ -11,7 +11,7 @@ require ( github.com/asottile/dockerfile v3.1.0+incompatible github.com/blang/semver v3.5.1+incompatible github.com/briandowns/spinner v1.23.2 - github.com/codefly-dev/core v0.15.0 + github.com/codefly-dev/core v0.16.1-0.20261008211642-b7f31f8d2c35 github.com/codefly-dev/golor v0.1.3 github.com/codefly-dev/llm v0.1.7 github.com/codefly-dev/sdk-go/workcontext v0.0.0-20261007181327-535084e1872d diff --git a/go.sum b/go.sum index 09fd52144..38d4aed17 100644 --- a/go.sum +++ b/go.sum @@ -175,6 +175,8 @@ github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb h1:EDmT6Q9Zs+SbUo github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb/go.mod h1:ZjrT6AXHbDs86ZSdt/osfBi5qfexBrKUdONk989Wnk4= github.com/codefly-dev/core v0.15.0 h1:sAadTEXELUIDYvYjOYvesCG2tRMFoHKPLfE1X12HNh8= github.com/codefly-dev/core v0.15.0/go.mod h1:rp4YZkCJEOufliWQAgK3sv8YlO7Xcva7+cN7hIlAR0I= +github.com/codefly-dev/core v0.16.1-0.20261008211642-b7f31f8d2c35 h1:MkIoEctXtwXYmbT+jLWGAuzv2z+/1QJIDcBOii8z+oM= +github.com/codefly-dev/core v0.16.1-0.20261008211642-b7f31f8d2c35/go.mod h1:rp4YZkCJEOufliWQAgK3sv8YlO7Xcva7+cN7hIlAR0I= github.com/codefly-dev/golor v0.1.3 h1:xmo+ceyJFRYZdvpWE2fNd0jeaadp/Ibm1BnganiGKOc= github.com/codefly-dev/golor v0.1.3/go.mod h1:sl/u/K1l7J0Pr3xyVZp8fOJYQItKKst1No9JqgzLLoY= github.com/codefly-dev/gortk v0.2.0 h1:7bOlS5valYz2zil+fZctQNcPCYBcPj86abcw9N8h1hQ= diff --git a/pkg/conformance/matrix.json b/pkg/conformance/matrix.json index f7e7fd998..922203003 100644 --- a/pkg/conformance/matrix.json +++ b/pkg/conformance/matrix.json @@ -1,7 +1,7 @@ { "schema_version": 1, "cli": "source", - "core": "v0.15.0", + "core": "v0.16.1", "rows": [ { "id": "linux-amd64-source", diff --git a/pkg/orchestration/build_plan.go b/pkg/orchestration/build_plan.go index edc19d8be..9d9cd1e3f 100644 --- a/pkg/orchestration/build_plan.go +++ b/pkg/orchestration/build_plan.go @@ -153,6 +153,9 @@ func (b *Builder) buildRecipe( shouldPush bool, goModuleProxies map[string]string, ) error { + if err := b.prepareLocalRuntimeRecipe(recipe); err != nil { + return err + } if shouldPush { targeted, err := b.targetEnvironmentPlatforms(recipe) if err != nil { @@ -206,6 +209,9 @@ func (b *Builder) buildRecipe( return err } identity, identityErr := imageBuildIdentity(ctx, recipe, plan, scope, dockerfile, probeBuilder, identityArgs, private.Proxies) + if err = b.nameLocalRuntimeImage(recipe, identity, identityErr); err != nil { + return err + } if identityErr != nil { // Not a build failure: an input this cannot account for means the image // is rebuilt, which is the behaviour without a cache at all. It is said @@ -215,22 +221,8 @@ func (b *Builder) buildRecipe( wool.Field("recipe", recipe.GetName()), wool.ErrField(identityErr)) } imageCache := b.imageBuildCache() - if identity.Key != "" && !b.world.RebuildImages { - entry, found := imageCache.lookup(identity.Key, b.instance.Unique(), shouldPush) - switch { - case found && imageStillExists(ctx, entry, probeBuilder): - w.Info("no image input changed; keeping the built image", - wool.Field("image", entry.Image), wool.Field("digest", entry.Digest), - wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) - b.adoptCachedImage(recipe, entry, captureDigest, resolveEvidence) - return nil - case found: - w.Debug("an image was built from these inputs but no longer exists; building", - wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) - default: - w.Debug("no image has been built from these inputs; building", - wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) - } + if b.reuseBuiltImage(ctx, w, recipe, identity, imageCache, captureDigest, resolveEvidence) { + return nil } // A caller-provided builder (e.g. a native amd64 buildkit) is authoritative: @@ -282,10 +274,37 @@ func (b *Builder) buildRecipe( if err != nil { return err } + if err = b.verifyLocalRuntimeBuild(ctx, w, recipe, scope, identity); err != nil { + return err + } b.recordImageForReuse(ctx, w, recipe, scope, identity, imageCache, shouldPush, pushedDigest, loadedImageID, recordable) return nil } +func (b *Builder) reuseBuiltImage(ctx context.Context, w *wool.Wool, recipe *builderv0.DockerBuildRecipe, + identity imageIdentity, cache *imageBuildCache, captureDigest, resolveEvidence bool, +) bool { + if identity.Key == "" || b.world.RebuildImages { + return false + } + entry, found := cache.lookup(identity.Key, b.instance.Unique(), b.world.Push) + if !found { + w.Debug("no image has been built from these inputs; building", + wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) + return false + } + if (b.localRuntime && entry.Image != recipe.GetImage()) || !imageStillExists(ctx, entry, b.world.BuildxBuilder) { + w.Debug("the recorded image cannot be reused; building", + wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) + return false + } + w.Info("no image input changed; keeping the built image", + wool.Field("image", entry.Image), wool.Field("digest", entry.Digest), + wool.Field("recipe", recipe.GetName()), wool.Field("identity", identity.Key)) + b.adoptCachedImage(recipe, entry, captureDigest, resolveEvidence) + return true +} + // resolvePushedDigest reads the immutable manifest digest of a pushed build from // the metadata file, when one was staged. // diff --git a/pkg/orchestration/builder.go b/pkg/orchestration/builder.go index a20b97e36..3f1f6300a 100644 --- a/pkg/orchestration/builder.go +++ b/pkg/orchestration/builder.go @@ -42,6 +42,10 @@ type Builder struct { imageDigest string + // localRuntime builds a recipe for this machine's container engine, keeping + // generated files out of authored services and naming the image by its inputs. + localRuntime bool + imageEvidence []*builderv0.ImageSBOM resolvedImages []coresbom.ResolvedImage @@ -275,6 +279,9 @@ func (b *Builder) buildRecipeRoot() (string, error) { if b.instance.Identity != nil { module, service = b.instance.Identity.Module, b.instance.Identity.Name } + if b.localRuntime { + return localRuntimeRecipeRoot(workspaceDir, module, service) + } return buildRecipeRoot(workspaceDir, module, service, b.instance.Service.Dir()) } diff --git a/pkg/orchestration/local_runtime_image.go b/pkg/orchestration/local_runtime_image.go new file mode 100644 index 000000000..09d69b3ca --- /dev/null +++ b/pkg/orchestration/local_runtime_image.go @@ -0,0 +1,152 @@ +package orchestration + +import ( + "context" + "fmt" + "path/filepath" + "strings" + + "github.com/codefly-dev/core/agents/contract" + agentv0 "github.com/codefly-dev/core/generated/go/codefly/services/agent/v0" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + "github.com/codefly-dev/core/resources" + "github.com/codefly-dev/core/wool" +) + +// localRuntimeImage runs before Runtime.Init, after the flow selected a backend. +// Stock-image agents may have a builder but emit no recipe. They keep their +// own image selection; no agent-name or release-version table is involved. +func (runner *Runner) localRuntimeImage(ctx context.Context) (string, error) { + if runner.remoteEnvironment != nil || runner.runtimeContext != resources.RuntimeContextContainer { + return "", nil + } + // An explicit manifest choice wins under the handoff contract. Leave its + // validation to the agent, including refusal of malformed/latest values. + if image, ok := runner.instance.Service.Spec["runtime-image"].(string); ok && image != "" { + return "", nil + } + if err := runner.instance.CheckCapabilities(agentv0.Capability_BUILDER); err != nil { + return "", nil + } + if advertised, supported := ValidationOperationSupport(runner.instance.Info, ValidationArtifactBuild); advertised && !supported { + return "", nil + } + if runner.instance.Builder == nil { + if err := runner.instance.LoadBuilder(ctx); err != nil { + return "", err + } + } + // Runtime.Load already published the endpoints. Loading the builder must + // not overwrite those runtime declarations in shared state. + runner.instance.Builder.Workspace = runner.world.Workspace + if _, err := runner.instance.Builder.Load(ctx); err != nil { + return "", err + } + b, err := NewBuilder(ctx, runner.instance, runner.world) + if err != nil { + return "", err + } + if _, err = b.Init(ctx); err != nil { + return "", err + } + return b.buildLocalRuntimeImage(ctx) +} + +// buildLocalRuntimeImage executes an agent's recipe through the same verified +// buildx path as deploy. Load and Init of the builder must have completed first. +// An empty result means the builder emits no recipe (for example a stock-image +// service); that agent continues to own its existing runtime image selection. +func (b *Builder) buildLocalRuntimeImage(ctx context.Context) (string, error) { + if b.world.Push { + return "", fmt.Errorf("local runtime images must be loaded into the container engine, not pushed") + } + b.localRuntime = true + if _, err := b.Plan(ctx); err != nil { + return "", err + } + planned := b.planned + b.planned = nil + plan := planned.response.GetResult().GetDockerBuildPlan() + if plan == nil { + return "", nil + } + if len(plan.GetRecipes()) != 1 { + return "", fmt.Errorf("local runtime for %s requires exactly one image recipe, got %d", b.instance.Unique(), len(plan.GetRecipes())) + } + if err := b.instance.RequireAgentCapabilities(contract.RuntimeInitImage); err != nil { + return "", fmt.Errorf("cannot supply a local runtime image to %s: %w", b.instance.Unique(), err) + } + if err := b.buildFromPlan(ctx, planned.outputDir, plan, planned.proxies); err != nil { + return "", err + } + if err := recordBuildRecipe(ctx, b.instance.Service, planned.recipeRoot, plan); err != nil { + return "", err + } + return plan.GetRecipes()[0].GetImage(), nil +} + +func localRuntimeRecipeRoot(workspaceDir, module, service string) (string, error) { + if workspaceDir == "" { + return "", fmt.Errorf("a local runtime recipe requires a workspace directory") + } + for _, name := range []string{module, service} { + if name == "" || name == "." || name == ".." || filepath.Base(name) != name { + return "", fmt.Errorf("invalid module/service identity for a local runtime recipe: %q", name) + } + } + return filepath.Join(workspaceDir, ".codefly", "runtime-build", module, service), nil +} + +// prepareLocalRuntimeRecipe runs after plan verification. The destination is +// normalized before hashing so the resulting tag is not its own input. The +// original verified plan digest (including agent metadata) remains bound. +func (b *Builder) prepareLocalRuntimeRecipe(recipe *builderv0.DockerBuildRecipe) error { + if !b.localRuntime { + return nil + } + host := b.world.hostArchitecture + if host == nil { + host = dockerEngineArchitecture + } + arch, err := host() + if err != nil { + return err + } + platform := "linux/" + arch + if !recipeBuildsPlatform(recipe.GetPlatforms(), platform) { + return fmt.Errorf("local container engine requires %s, which recipe %s does not build (it builds %v)", platform, recipe.GetName(), recipe.GetPlatforms()) + } + recipe.Platforms = []string{platform} + recipe.Image = "codefly-local/runtime:recipe-inputs" + return nil +} + +func (b *Builder) nameLocalRuntimeImage(recipe *builderv0.DockerBuildRecipe, identity imageIdentity, identityErr error) error { + if !b.localRuntime { + return nil + } + if identityErr != nil { + return fmt.Errorf("cannot identify local runtime image inputs: %w", identityErr) + } + if identity.Key == "" { + return fmt.Errorf("cannot identify local runtime image inputs for %s", recipe.GetName()) + } + recipe.Image = "codefly-local/runtime:sha256-" + strings.TrimPrefix(identity.Key, "sha256:") + return nil +} + +// Unlike a deployment's mutable version tag, the local tag claims particular +// inputs. A build whose context changed cannot be handed to Runtime.Init under +// that claim, even if it succeeded. The next invocation will re-plan and build. +func (b *Builder) verifyLocalRuntimeBuild(ctx context.Context, w *wool.Wool, recipe *builderv0.DockerBuildRecipe, scope imageBuildScope, identity imageIdentity) error { + if !b.localRuntime { + return nil + } + if !b.contextHeldStill(ctx, w, recipe, scope, identity) { + return fmt.Errorf("local runtime build inputs changed while building %s; retry the run", recipe.GetImage()) + } + if _, err := inspectLocalImageID(ctx, recipe.GetImage()); err != nil { + return fmt.Errorf("local runtime image was not loaded: %w", err) + } + return nil +} diff --git a/pkg/orchestration/local_runtime_image_test.go b/pkg/orchestration/local_runtime_image_test.go new file mode 100644 index 000000000..9a8cb08a2 --- /dev/null +++ b/pkg/orchestration/local_runtime_image_test.go @@ -0,0 +1,293 @@ +package orchestration + +import ( + "context" + "os" + "os/exec" + "path/filepath" + "testing" + "time" + + "github.com/codefly-dev/core/agents/contract" + agentservices "github.com/codefly-dev/core/agents/services" + agentv0 "github.com/codefly-dev/core/generated/go/codefly/services/agent/v0" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + "github.com/codefly-dev/core/resources" + "github.com/codefly-dev/core/services" + "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" +) + +func localRuntimeBuilder(t *testing.T, harness *imageCacheHarness) *Builder { + t.Helper() + workspace := &resources.Workspace{Name: "platform"} + workspace.WithDir(harness.workspaceDir) + service := &resources.Service{ + Name: "api", Version: "1.0.0", + Agent: &resources.Agent{Publisher: "codefly.dev", Name: "test-peer", Version: "0.1.0"}, + } + service.WithDir(harness.serviceDir) + instance := &services.Instance{Service: service, Identity: &resources.ServiceIdentity{Name: "api", Module: "saas"}} + declaration := contract.Current() + declaration.Capabilities = append(declaration.Capabilities, contract.RuntimeInitImage) + instance.Info = &agentv0.AgentInformation{Contract: declaration} + instance.Builder = &services.BuilderInstance{Instance: instance, Builder: harness.client} + b, err := NewBuilder(t.Context(), instance, &World{ + Workspace: workspace, BuildxBuilder: "stub", RebuildImages: harness.rebuild, + hostArchitecture: func() (string, error) { return "arm64", nil }, + }) + require.NoError(t, err) + return b +} + +type runtimeImageBuilderPeer struct { + imageCachePeer + stock bool +} + +func (*runtimeImageBuilderPeer) Load(context.Context, *builderv0.LoadRequest) (*builderv0.LoadResponse, error) { + return &builderv0.LoadResponse{State: &builderv0.LoadStatus{State: builderv0.LoadStatus_READY}}, nil +} + +func (*runtimeImageBuilderPeer) Init(context.Context, *builderv0.InitRequest) (*builderv0.InitResponse, error) { + return &builderv0.InitResponse{State: &builderv0.InitStatus{State: builderv0.InitStatus_SUCCESS}}, nil +} + +func (peer *runtimeImageBuilderPeer) Build(ctx context.Context, req *builderv0.BuildRequest) (*builderv0.BuildResponse, error) { + if peer.stock { + return &builderv0.BuildResponse{ + State: &builderv0.BuildStatus{State: builderv0.BuildStatus_SUCCESS}, + BuildxBuilder: req.GetBuildContext().GetDockerBuildContext().GetBuildxBuilder(), + }, nil + } + return peer.imageCachePeer.Build(ctx, req) +} + +func runtimeImageRunner(t *testing.T, peer *runtimeImageBuilderPeer) (*Runner, *dockerStub) { + t.Helper() + stub := installDockerStub(t, true) + runner, world := gatewayRunner(t, startRealAgent(t, realAgentInputs)) + root := t.TempDir() + world.Workspace.WithDir(root) + serviceDir := filepath.Join(root, "modules/web/services/gateway") + require.NoError(t, os.MkdirAll(filepath.Join(serviceDir, "code"), 0o755)) + require.NoError(t, os.WriteFile(filepath.Join(serviceDir, "code/payload"), []byte("runtime input\n"), 0o644)) + runner.instance.Service.WithDir(serviceDir) + runner.instance.Module.WithDir(filepath.Join(root, "modules/web")) + world.hostArchitecture = func() (string, error) { return "arm64", nil } + world.BuildxBuilder = "stub" + runner.runtimeContext = resources.RuntimeContextContainer + runner.containerRecoveryIdentity = "test-runtime-image-scope" + runner.instance.ContainerRecoveryScope = runner.containerRecoveryIdentity + declaration := contract.Current() + declaration.Capabilities = append(declaration.Capabilities, contract.RuntimeInitImage) + runner.instance.Info = &agentv0.AgentInformation{Contract: declaration} + runner.instance.Capabilities = []*agentv0.Capability{{Type: agentv0.Capability_BUILDER}} + server := grpc.NewServer() + builderv0.RegisterBuilderServer(server, peer) + conn, err := grpc.NewClient(serveGRPC(t, server), grpc.WithTransportCredentials(insecure.NewCredentials())) + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + runner.instance.Builder = &services.BuilderInstance{ + Instance: runner.instance, + Builder: &agentservices.BuilderAgent{BuilderClient: builderv0.NewBuilderClient(conn)}, + } + return runner, stub +} + +// Exercise the real Runner.Init, builder gRPC lifecycle and buildx boundary, +// then read the image the runtime agent received across a process boundary. +func TestRunnerBuildsAndHandsOffLocalRuntimeImage(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + _, err := runner.Init(t.Context()) + require.NoError(t, err) + first := exposedValues(t.Context(), t, runner.world)["runtime-image"] + require.Regexp(t, `^codefly-local/runtime:sha256-[0-9a-f]{64}$`, first) + require.Equal(t, 1, stub.calls(t, "build")) + _, err = runner.Init(t.Context()) + require.NoError(t, err) + require.Equal(t, first, exposedValues(t.Context(), t, runner.world)["runtime-image"]) + require.Equal(t, 1, stub.calls(t, "build")) +} + +func TestRunnerRefusesImageHandoffWithoutLiveCapability(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + runner.instance.Info.Contract = contract.Current() + _, err := runner.Init(t.Context()) + require.ErrorContains(t, err, contract.RuntimeInitImage) + require.Zero(t, stub.calls(t, "build")) +} + +func TestRunnerStockImageNeedsNoHandoffCapability(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{stock: true}) + runner.instance.Info.Contract = contract.Current() + _, err := runner.Init(t.Context()) + require.NoError(t, err) + require.NotContains(t, exposedValues(t.Context(), t, runner.world), "runtime-image") + require.Zero(t, stub.calls(t, "build")) +} + +func TestRunnerNativeRuntimeDoesNotBuildImage(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + runner.runtimeContext = resources.RuntimeContextNative + _, err := runner.Init(t.Context()) + require.NoError(t, err) + require.NotContains(t, exposedValues(t.Context(), t, runner.world), "runtime-image") + require.Zero(t, stub.calls(t, "build")) +} + +func TestRunnerLeavesExplicitImageValidationToAgent(t *testing.T) { + for _, image := range []string{"example.test/runtime:v1", "example.test/runtime:latest", "malformed image"} { + t.Run(image, func(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + runner.instance.Service.Spec = map[string]any{"runtime-image": image} + runner.instance.Info.Contract = contract.Current() + _, err := runner.Init(t.Context()) + require.NoError(t, err) // this test peer echoes; real agents own validation + require.NotContains(t, exposedValues(t.Context(), t, runner.world), "runtime-image") + require.Zero(t, stub.calls(t, "build")) + }) + } +} + +// Opt-in host-boundary integration: real buildx builds/loads an offline scratch +// image, and Runtime.Init receives the reference through the gRPC process peer. +// This deliberately does not stand in for the platform-obin composition boot. +func TestLocalRuntimeImageDockerHandoff(t *testing.T) { + if os.Getenv("CODEFLY_TEST_RUNTIME_IMAGE") != "1" { + t.Skip("set CODEFLY_TEST_RUNTIME_IMAGE=1 to exercise real buildx and the image handoff") + } + docker, err := exec.LookPath("docker") + require.NoError(t, err) + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + runner.world.BuildxBuilder = "" + runner.world.hostArchitecture = nil + // Count actual build invocations while forwarding every command to Docker. + t.Setenv("CODEFLY_TEST_REAL_DOCKER", docker) + t.Setenv("CODEFLY_TEST_DOCKER_CALLS", filepath.Join(stub.dir, "calls")) + require.NoError(t, os.WriteFile(filepath.Join(stub.dir, "docker"), []byte(`#!/bin/sh +if [ "$1 $2" = "buildx build" ]; then printf 'build\n' >> "$CODEFLY_TEST_DOCKER_CALLS"; fi +exec "$CODEFLY_TEST_REAL_DOCKER" "$@" +`), 0o755)) + // Give this fixture its own input-derived tag, so cleanup touches no image + // another invocation could be using. + require.NoError(t, os.WriteFile(filepath.Join(runner.instance.Service.Dir(), "code/payload"), []byte(t.TempDir()), 0o644)) + ctx, cancel := context.WithTimeout(t.Context(), 2*time.Minute) + defer cancel() + _, err = runner.Init(ctx) + require.NoError(t, err) + image := exposedValues(ctx, t, runner.world)["runtime-image"] + require.NotEmpty(t, image) + t.Cleanup(func() { + cleanup, stop := context.WithTimeout(context.Background(), 30*time.Second) + defer stop() + output, removeErr := exec.CommandContext(cleanup, docker, "image", "rm", image).CombinedOutput() + require.NoError(t, removeErr, "%s", output) + }) + imageID, err := inspectLocalImageID(ctx, image) + require.NoError(t, err) + require.Regexp(t, `^sha256:[0-9a-f]{64}$`, imageID) + require.Equal(t, 1, stub.calls(t, "build")) + _, err = runner.Init(ctx) + require.NoError(t, err) + require.Equal(t, image, exposedValues(ctx, t, runner.world)["runtime-image"]) + require.Equal(t, 1, stub.calls(t, "build"), "a second Init must reuse the loaded real image") +} + +func TestLocalRuntimeImageReusesRecipeAndInvalidatesSourceChanges(t *testing.T) { + h := newImageCacheHarness(t, false, true) + build := func() string { + image, err := localRuntimeBuilder(t, h).buildLocalRuntimeImage(t.Context()) + require.NoError(t, err) + require.Regexp(t, `^codefly-local/runtime:sha256-[0-9a-f]{64}$`, image) + return image + } + first := build() + require.Equal(t, 1, h.docker.calls(t, "build")) + require.Equal(t, first, build()) + require.Equal(t, 1, h.docker.calls(t, "build"), "unchanged inputs must not invoke buildx again") + h.write(t, filepath.Join(h.serviceDir, "notes.md"), "ignored edit\n") + require.Equal(t, first, build()) + require.Equal(t, 1, h.docker.calls(t, "build")) + h.write(t, filepath.Join(h.serviceDir, "code", "main.go"), "package main\nfunc main() { println(42) }\n") + second := build() + require.NotEqual(t, first, second) + require.Equal(t, 2, h.docker.calls(t, "build")) + require.Equal(t, second, build()) + require.Equal(t, 2, h.docker.calls(t, "build")) + // A forced build retains the recipe-derived name, never a clock-based tag. + h.rebuild = true + require.Equal(t, second, build()) + require.Equal(t, 3, h.docker.calls(t, "build")) + require.NoDirExists(t, filepath.Join(h.serviceDir, "builder")) + require.NoDirExists(t, filepath.Join(h.serviceDir, "build-recipes")) + manifest := filepath.Join(h.workspaceDir, ".codefly/runtime-build/saas/api/build-recipes/0.1.0/recipe.codefly.json") + require.Contains(t, h.read(t, manifest), second) + require.Contains(t, h.read(t, manifest), "linux/arm64") +} + +func TestLocalRuntimeImageIdentityDoesNotIncludeCheckoutPath(t *testing.T) { + h := newImageCacheHarness(t, false, true) + first, err := localRuntimeBuilder(t, h).buildLocalRuntimeImage(t.Context()) + require.NoError(t, err) + other := newImageCacheHarness(t, false, true) + second, err := localRuntimeBuilder(t, other).buildLocalRuntimeImage(t.Context()) + require.NoError(t, err) + require.Equal(t, first, second) +} + +func TestLocalRuntimeImageTagBindsBuildArgumentsAndRecipeBytes(t *testing.T) { + h := newImageCacheHarness(t, false, true) + build := func(argument, dockerfile string) string { + b := localRuntimeBuilder(t, h) + b.localRuntime = true + _, err := b.Plan(t.Context()) + require.NoError(t, err) + plan := b.planned.response.GetResult().GetDockerBuildPlan() + plan.Recipes[0].BuildArgs = map[string]string{"MODE": argument} + if dockerfile != "" { + h.write(t, filepath.Join(b.planned.outputDir, "Dockerfile"), dockerfile) + } + plan, err = agentservices.BuildEmittedDockerBuildPlan(b.planned.outputDir, plan.Recipes, []string{"Dockerfile", "dockerignore"}) + require.NoError(t, err) + b.planned.response.Result = &builderv0.BuildResult{Kind: &builderv0.BuildResult_DockerBuildPlan{DockerBuildPlan: plan}} + image, err := b.buildLocalRuntimeImage(t.Context()) + require.NoError(t, err) + return image + } + first := build("one", "") + require.Equal(t, first, build("one", "")) + second := build("two", "") + require.NotEqual(t, first, second) + third := build("two", "FROM scratch\nCOPY code /changed\n") + require.NotEqual(t, second, third) + require.Equal(t, third, build("two", "FROM scratch\nCOPY code /changed\n")) + require.Equal(t, 3, h.docker.calls(t, "build")) +} + +func TestLocalRuntimeImageRefusesChangedContextDuringBuild(t *testing.T) { + h := newImageCacheHarness(t, false, true) + h.docker.editDuringBuild(t, filepath.Join(h.serviceDir, "code/main.go"), "changed during build\n") + image, err := localRuntimeBuilder(t, h).buildLocalRuntimeImage(t.Context()) + require.ErrorContains(t, err, "inputs changed while building") + require.Empty(t, image) + entries, err := os.ReadDir(filepath.Join(h.workspaceDir, imageCacheDir)) + if !os.IsNotExist(err) { + require.NoError(t, err) + require.Empty(t, entries) + } +} + +func TestLocalRuntimeImageRejectsPushAndUnsupportedPlatform(t *testing.T) { + h := newImageCacheHarness(t, false, true) + b := localRuntimeBuilder(t, h) + b.world.Push = true + _, err := b.buildLocalRuntimeImage(t.Context()) + require.ErrorContains(t, err, "must be loaded") + b.world.Push = false + b.world.hostArchitecture = func() (string, error) { return "riscv64", nil } + _, err = b.buildLocalRuntimeImage(t.Context()) + require.ErrorContains(t, err, "requires linux/riscv64") + require.Zero(t, h.docker.calls(t, "build")) +} diff --git a/pkg/orchestration/runner.go b/pkg/orchestration/runner.go index e4f14a3c9..96c0d9148 100644 --- a/pkg/orchestration/runner.go +++ b/pkg/orchestration/runner.go @@ -271,6 +271,10 @@ func (runner *Runner) Init(ctx context.Context) (*OutputProperty, error) { if err := runner.checkInitialPortAvailability(ctx); err != nil { return nil, w.Wrapf(err, "cannot initialize %s", runner.instance.Unique()) } + runtimeImage, err := runner.localRuntimeImage(ctx) + if err != nil { + return nil, w.Wrapf(err, "cannot prepare local runtime image for %s", runner.instance.Unique()) + } // Configuration reads can block on a stalled provider (e.g. a dependency // service that failed to export its config). Bound each read with a @@ -296,6 +300,7 @@ func (runner *Runner) Init(ctx context.Context) (*OutputProperty, error) { // every dependency and never the service the suite is about. Start still // carries both, and the agent takes the first non-empty. req := &runtimev0.InitRequest{ + RuntimeImage: runtimeImage, RuntimeContext: runtimeContext, ProposedNetworkMappings: networkMappings, DependenciesEndpoints: dependenciesEndpoints, diff --git a/pkg/orchestration/runner_accepted_network_mappings_test.go b/pkg/orchestration/runner_accepted_network_mappings_test.go index bdbe7ed2f..a484e6a05 100644 --- a/pkg/orchestration/runner_accepted_network_mappings_test.go +++ b/pkg/orchestration/runner_accepted_network_mappings_test.go @@ -169,6 +169,9 @@ func exportedConfigurations(mode string, req *runtimev0.InitRequest) []*basev0.C values := []*basev0.ConfigurationValue{{Key: "session", Value: "exposed"}} if mode == realAgentInputs { values = []*basev0.ConfigurationValue{{Key: "fixture", Value: req.GetFixture()}} + if req.GetRuntimeImage() != "" { + values = append(values, &basev0.ConfigurationValue{Key: "runtime-image", Value: req.GetRuntimeImage()}) + } for key, value := range req.GetOverrides() { values = append(values, &basev0.ConfigurationValue{Key: key, Value: value}) } From 17b3a0ce66be4a2f4de65495bb412db0a96515af Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:56:04 -0400 Subject: [PATCH 02/18] fix(run): hand agents a verified canonical local image digest --- docs/development-loop.md | 7 ++- pkg/orchestration/image_cache_test.go | 4 ++ pkg/orchestration/local_runtime_image.go | 33 +++++++++++-- pkg/orchestration/local_runtime_image_test.go | 46 ++++++++++++++++--- .../runner_accepted_network_mappings_test.go | 11 +++++ 5 files changed, 90 insertions(+), 11 deletions(-) diff --git a/docs/development-loop.md b/docs/development-loop.md index 317eab6d4..9d6890df3 100644 --- a/docs/development-loop.md +++ b/docs/development-loop.md @@ -65,8 +65,11 @@ agent plan digest, including its metadata, remains an input. Temporary paths and builder names are normalized by the existing cache. There is no timestamp, checkout path or `latest` in the result. The tag identifies inputs for reuse; `Runtime.Init` receives the loaded -Docker image's immutable `sha256:…` ID, so retagging cannot change the image -the agent starts. +image's canonical `repository@sha256:…` reference, verified against its Docker +image ID, so retagging cannot change the image the agent starts. The engine +must expose and resolve a repository digest for locally loaded images (as +Docker's containerd image store does); an unavailable or mismatched reference +refuses Init without falling back to a tag or pushing to a registry. The next run emits and verifies the recipe again, then reuses the cached image only if its inputs match and Docker still holds the recorded image ID. A source, diff --git a/pkg/orchestration/image_cache_test.go b/pkg/orchestration/image_cache_test.go index 15b9eaca7..6cbb5d617 100644 --- a/pkg/orchestration/image_cache_test.go +++ b/pkg/orchestration/image_cache_test.go @@ -65,6 +65,10 @@ case "$1 $2" in ;; "image inspect") record inspect + if [ "$4" = '{{json .RepoDigests}}' ]; then + printf '["codefly-local/runtime@sha256:%064d"]\n' 7 + exit 0 + fi printf 'sha256:%064d\n' 7 exit 0 ;; diff --git a/pkg/orchestration/local_runtime_image.go b/pkg/orchestration/local_runtime_image.go index d8940b5cc..0a466e7f0 100644 --- a/pkg/orchestration/local_runtime_image.go +++ b/pkg/orchestration/local_runtime_image.go @@ -6,6 +6,8 @@ import ( "path/filepath" "strings" + "github.com/distribution/reference" + "github.com/codefly-dev/core/agents/contract" agentv0 "github.com/codefly-dev/core/generated/go/codefly/services/agent/v0" builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" @@ -55,9 +57,34 @@ func (runner *Runner) localRuntimeImage(ctx context.Context) (string, error) { if err != nil || image == "" { return "", err } - // The input-derived tag is for discovery and reuse. Init receives Docker's - // immutable image ID, so a later retag cannot change what the agent starts. - return inspectLocalImageID(ctx, image) + return localRuntimeImageReference(ctx, image) +} + +// Init's image contract uses a canonical repository reference, not a bare +// Docker image ID. Bind the daemon's manifest reference to the loaded image; +// neither retagging nor an unavailable digest may turn this into a pull. +func localRuntimeImageReference(ctx context.Context, image string) (string, error) { + id, err := inspectLocalImageID(ctx, image) + if err != nil { + return "", err + } + digest, err := inspectImageDigest(ctx, image) + if err != nil { + return "", fmt.Errorf("local runtime requires a container engine exposing a loaded repository digest: %w", err) + } + named, err := reference.ParseNormalizedNamed(image) + if err != nil { + return "", err + } + immutable := reference.TrimNamed(named).String() + "@" + digest + loaded, err := inspectLocalImageID(ctx, immutable) + if err != nil { + return "", err + } + if loaded != id { + return "", fmt.Errorf("local runtime reference %s does not identify the built image %s", immutable, id) + } + return immutable, nil } // buildLocalRuntimeImage executes an agent's recipe through the same verified diff --git a/pkg/orchestration/local_runtime_image_test.go b/pkg/orchestration/local_runtime_image_test.go index accd805bd..aee7410c5 100644 --- a/pkg/orchestration/local_runtime_image_test.go +++ b/pkg/orchestration/local_runtime_image_test.go @@ -6,6 +6,7 @@ import ( "os" "os/exec" "path/filepath" + "strings" "testing" "time" @@ -109,7 +110,7 @@ func TestRunnerBuildsAndHandsOffLocalRuntimeImage(t *testing.T) { _, err := runner.Init(t.Context()) require.NoError(t, err) first := exposedValues(t.Context(), t, runner.world)["runtime-image"] - require.Regexp(t, `^sha256:[0-9a-f]{64}$`, first) + require.Regexp(t, `^docker.io/codefly-local/runtime@sha256:[0-9a-f]{64}$`, first) require.Equal(t, 1, stub.calls(t, "build")) _, err = runner.Init(t.Context()) require.NoError(t, err) @@ -117,6 +118,33 @@ func TestRunnerBuildsAndHandsOffLocalRuntimeImage(t *testing.T) { require.Equal(t, 1, stub.calls(t, "build")) } +func TestLocalRuntimeImageReferenceRefusesUnavailableOrDifferentImage(t *testing.T) { + for _, test := range []struct{ name, digests, resolved, message string }{ + {"missing digest", "[]", "7", "exposing a loaded repository digest"}, + {"different image", fmt.Sprintf(`["codefly-local/runtime@sha256:%064d"]`, 7), "8", "does not identify the built image"}, + } { + t.Run(test.name, func(t *testing.T) { + dir := t.TempDir() + script := `#!/bin/sh +case "$4" in + '{{json .RepoDigests}}') printf '%s\n' '` + test.digests + `' ;; + '{{.Id}}') + case "$5" in + *@*) printf 'sha256:%064d\n' ` + test.resolved + ` ;; + *) printf 'sha256:%064d\n' 7 ;; + esac ;; + *) exit 1 ;; +esac +` + require.NoError(t, os.WriteFile(filepath.Join(dir, "docker"), []byte(script), 0o755)) + t.Setenv("PATH", strings.Join([]string{dir, os.Getenv("PATH")}, string(os.PathListSeparator))) + image, err := localRuntimeImageReference(t.Context(), "codefly-local/runtime:test") + require.ErrorContains(t, err, test.message) + require.Empty(t, image) + }) + } +} + func TestRunnerRefusesImageHandoffWithoutLiveCapability(t *testing.T) { runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) runner.instance.Info.Contract = contract.Current() @@ -207,27 +235,33 @@ exec "$CODEFLY_TEST_REAL_DOCKER" "$@" require.NoError(t, err) image := exposedValues(ctx, t, runner.world)["runtime-image"] require.NotEmpty(t, image) + require.Regexp(t, `^docker.io/codefly-local/runtime@sha256:[0-9a-f]{64}$`, image) + imageID, err := inspectLocalImageID(ctx, image) + require.NoError(t, err) t.Cleanup(func() { cleanup, stop := context.WithTimeout(context.Background(), 30*time.Second) defer stop() - output, removeErr := exec.CommandContext(cleanup, docker, "image", "rm", image).CombinedOutput() + output, removeErr := exec.CommandContext(cleanup, docker, "image", "rm", imageID).CombinedOutput() require.NoError(t, removeErr, "%s", output) }) - imageID, err := inspectLocalImageID(ctx, image) - require.NoError(t, err) require.Regexp(t, `^sha256:[0-9a-f]{64}$`, imageID) - require.Equal(t, imageID, image, "Init must receive the immutable loaded image ID") require.Equal(t, 1, stub.calls(t, "build")) _, err = runner.Init(ctx) require.NoError(t, err) require.Equal(t, image, exposedValues(ctx, t, runner.world)["runtime-image"]) require.Equal(t, 1, stub.calls(t, "build"), "a second Init must reuse the loaded real image") - output, err := exec.CommandContext(ctx, docker, "image", "rm", image).CombinedOutput() + output, err := exec.CommandContext(ctx, docker, "image", "rm", imageID).CombinedOutput() require.NoError(t, err, "%s", output) _, err = runner.Init(ctx) require.NoError(t, err) + image = exposedValues(ctx, t, runner.world)["runtime-image"] + imageID, err = inspectLocalImageID(ctx, image) + require.NoError(t, err) require.Equal(t, 2, stub.calls(t, "build"), "a lost image must be rebuilt") + _, err = runner.Init(ctx) + require.NoError(t, err) require.Equal(t, image, exposedValues(ctx, t, runner.world)["runtime-image"]) + require.Equal(t, 2, stub.calls(t, "build"), "the replacement image must also be reusable") } func TestLocalRuntimeImageReusesRecipeAndInvalidatesSourceChanges(t *testing.T) { diff --git a/pkg/orchestration/runner_accepted_network_mappings_test.go b/pkg/orchestration/runner_accepted_network_mappings_test.go index a484e6a05..85ea0a518 100644 --- a/pkg/orchestration/runner_accepted_network_mappings_test.go +++ b/pkg/orchestration/runner_accepted_network_mappings_test.go @@ -14,6 +14,8 @@ import ( "testing" "time" + "github.com/distribution/reference" + agentservices "github.com/codefly-dev/core/agents/services" "github.com/codefly-dev/core/architecture" "github.com/codefly-dev/core/configurations" @@ -129,6 +131,15 @@ func proposedPorts(proposed []*basev0.NetworkMapping) map[uint16]bool { } func (agent *realAgent) Init(_ context.Context, req *runtimev0.InitRequest) (*runtimev0.InitResponse, error) { + if image := req.GetRuntimeImage(); image != "" { + parsed, err := reference.ParseNamed(image) + if err != nil { + return nil, fmt.Errorf("invalid runtime image reference: %w", err) + } + if _, ok := parsed.(reference.Canonical); !ok { + return nil, fmt.Errorf("runtime image reference must carry an immutable digest") + } + } mappings := req.GetProposedNetworkMappings() if agent.mode == realAgentLegacy { // A pre-contract agent answers READY without naming any mapping. From b4aa8e2bce57e93b9f52a85575f7eb3c7630cad2 Mon Sep 17 00:00:00 2001 From: Gael Date: Tue, 27 Oct 2026 11:07:40 -0400 Subject: [PATCH 03/18] fix(composition): route foreign methods to their selected service --- cmd/generate/permissions.go | 9 ++-- cmd/generate/permissions_test.go | 45 ++++++++++++++++ docs/cli-reference.md | 5 +- docs/commands.md | 12 ++++- pkg/gitops/method_policies.go | 47 +++++++++++++--- pkg/gitops/method_policies_test.go | 68 ++++++++++++++++++++++++ pkg/orchestration/hop_configurations.go | 45 ++++++++++------ pkg/orchestration/hop_rpc_routes_test.go | 41 ++++++++++++++ 8 files changed, 240 insertions(+), 32 deletions(-) diff --git a/cmd/generate/permissions.go b/cmd/generate/permissions.go index b3b2bb885..9146282d7 100644 --- a/cmd/generate/permissions.go +++ b/cmd/generate/permissions.go @@ -36,8 +36,9 @@ By default each Go service gets its own table in its own Go module: //pkg/gen/permissions, the tree the go-grpc layout generates its bindings into (source-dir is the Go agent's setting, "code" unless the service declares another). A service's table holds the methods its -published contracts serve plus the module contract's foreign methods, which -name no service. What an endpoint serves is what the contract catalog records +published contracts serve plus foreign methods selected for that service. +Unselected foreign methods appear in every Go service's table for compatibility. +What an endpoint serves is what the contract catalog records for it (the declared surface, or every service of its package); a method a published descriptor declares that no endpoint serves, such as a protocol other modules serve, is left out with a note and needs no policy. The TypeScript file @@ -119,8 +120,8 @@ type permissionDestination struct { // permissionDestinations places the module's method policies, and names the // published methods no endpoint serves, which no table holds. An explicit // output holds the whole module's table. Otherwise each Go service gets the -// methods its own published contracts serve, plus the foreign methods the -// module contract declares without naming a service, in the generated tree +// methods its own published contracts serve and foreign methods selected for it, +// plus legacy foreign methods with no service selector, in the generated tree // of its own Go module, so the service imports them with no flag. A Python // service that publishes policies is skipped and named in skipped: nothing in // it reads a table, because the hop in front of it decides every call. diff --git a/cmd/generate/permissions_test.go b/cmd/generate/permissions_test.go index a0604774c..9e920496b 100644 --- a/cmd/generate/permissions_test.go +++ b/cmd/generate/permissions_test.go @@ -154,6 +154,51 @@ services: return module } +func TestPermissionsForeignServiceSelectorsReachGeneratedTables(t *testing.T) { + module := permissionsModule(t) + path := filepath.Join(module.Dir(), "module.contract.codefly.yaml") + contract, err := os.ReadFile(path) + require.NoError(t, err) + contract = append(contract, []byte(` - method: /foreign.Api/Read + service: api + requires: + - resource_kind: ledger.entry + actions: [read] + - method: /foreign.Admin/Approve + service: admin + requires: + - resource_kind: ledger.entry + actions: [approve] +`)...) + require.NoError(t, os.WriteFile(path, contract, 0o644)) + module, err = resources.LoadModuleFromDir(t.Context(), module.Dir()) + require.NoError(t, err) + require.NoError(t, generatePermissions(t.Context(), module, "", false)) + require.NoError(t, generatePermissions(t.Context(), module, "", true)) + union := t.TempDir() + require.NoError(t, generatePermissions(t.Context(), module, union, false)) + for _, file := range []string{"permissions_gen.go", "permissions_gen.ts"} { + read := func(root string) string { + data, err := os.ReadFile(filepath.Join(root, file)) + require.NoError(t, err) + return string(data) + } + api := read(filepath.Join(module.Dir(), "services/api/code/pkg/gen/permissions")) + admin := read(filepath.Join(module.Dir(), "services/admin/go/pkg/gen/permissions")) + all := read(union) + require.Contains(t, api, "/foreign.Api/Read") + require.NotContains(t, api, "/foreign.Admin/Approve") + require.Contains(t, admin, "/foreign.Admin/Approve") + require.NotContains(t, admin, "/foreign.Api/Read") + for _, table := range []string{api, admin, all} { + require.Contains(t, table, "/otlp.TraceService/Export") + } + for _, method := range []string{"/foreign.Api/Read", "/foreign.Admin/Approve", "/ledger.ApiService/Read", "/ledger.AdminService/Approve", "/ledger.EngineService/Recompute"} { + require.Contains(t, all, method) + } + } +} + func TestPermissionsDefaultLandsInEachGoServiceModule(t *testing.T) { module := permissionsModule(t) require.NoError(t, generatePermissions(t.Context(), module, "", false)) diff --git a/docs/cli-reference.md b/docs/cli-reference.md index b8e410f9e..004a7a42b 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -2829,8 +2829,9 @@ By default each Go service gets its own table in its own Go module: //pkg/gen/permissions, the tree the go-grpc layout generates its bindings into (source-dir is the Go agent's setting, "code" unless the service declares another). A service's table holds the methods its -published contracts serve plus the module contract's foreign methods, which -name no service. What an endpoint serves is what the contract catalog records +published contracts serve plus foreign methods selected for that service. +Unselected foreign methods appear in every Go service's table for compatibility. +What an endpoint serves is what the contract catalog records for it (the declared surface, or every service of its package); a method a published descriptor declares that no endpoint serves, such as a protocol other modules serve, is left out with a note and needs no policy. The TypeScript file diff --git a/docs/commands.md b/docs/commands.md index 16b023bc4..08c2e661e 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4625,8 +4625,10 @@ module: `//pkg/gen/permissions`, where `source-dir` is the Go agent's setting (`code` unless the service declares another; its first segment is where `go.mod` lives) and `pkg/gen` is the tree the go-grpc layout generates its bindings into. Each service's table holds the methods its own -published contracts serve, plus the module contract's `foreign_methods`, which -name no service and so appear in every Go service's table. What an endpoint +published contracts serve, plus the module contract's `foreign_methods` selected +for that `service`. Foreign methods without a service selector retain the legacy +behavior of appearing in every Go service's table. Their authority requirements +remain in the module-wide policy union. What an endpoint serves is what the contract catalog (`contracts/api/catalog.codefly.json`) records for it: the module's declared surface (`contracts.codefly.yaml`, see [Generate contracts](#generate-contracts)), or every service of the endpoint's @@ -4664,3 +4666,9 @@ Published package dependencies do not require local source trees. Permission generation is available through the CLI; no separate MCP generator was added. MCP run/test use the same composition preflight as their CLI paths. + +A module with multiple published protobuf services selects the serving service +on each `foreign_methods` entry with `service:`. The hop routes it only to that +service's published gRPC/Connect listeners. An absent selector requires exactly +one such service; unknown or non-protobuf selections are refused. This does not +change a method's authority policy or authorize shadowing a published method. diff --git a/pkg/gitops/method_policies.go b/pkg/gitops/method_policies.go index 889a82f36..32b2fa085 100644 --- a/pkg/gitops/method_policies.go +++ b/pkg/gitops/method_policies.go @@ -160,7 +160,8 @@ func (served *ServedMethodPolicies) Table(keep func(service string) bool) ([]*co // ServedMethodPolicies is a module's method policies by where they are // served: each service's published contracts, and the module contract's -// foreign methods, which name no service. +// foreign methods. Explicitly selected foreign methods join ByService; +// unselected entries remain in Foreign for existing consumers. type ServedMethodPolicies struct { // Package is the package's ID, which a refusal names. Package string @@ -198,13 +199,16 @@ type ServedMethodPolicies struct { // package's other services and its imports', so the rest is Unserved. A // package with no catalog falls back to that same default. func ServiceMethodPolicies(dir string) (served ServedMethodPolicies, ok bool, err error) { - foreign, err := foreignMethods(dir) + foreign, selected, err := foreignMethods(dir) if err != nil { return served, false, err } - served.Foreign = foreign served.ByService = map[string][]*compositionv1.MethodPolicyV1{} if _, statErr := os.Stat(filepath.Join(dir, composition.PackageManifestFileName)); os.IsNotExist(statErr) { + if len(selected) > 0 { + return served, false, fmt.Errorf("foreign method service selection requires a package manifest") + } + served.Foreign = foreign return served, len(foreign) > 0, nil } manifest, err := composition.LoadPackageManifest(dir) @@ -267,6 +271,31 @@ func ServiceMethodPolicies(dir string) (served ServedMethodPolicies, ok bool, er return served, false, fmt.Errorf("module package %s declares foreign method %s, which its published contracts already serve", manifest.ID, m.GetMethod()) } servedAnywhere[m.GetMethod()] = true + if service := selected[m.GetMethod()]; service != "" { + published := false + for _, candidate := range manifest.Services { + if candidate.Name != service { + continue + } + for _, contract := range candidate.APIContracts { + if contract.Kind == composition.APIContractKindProtobuf { + published = true + } + } + } + if !published { + return served, false, fmt.Errorf("foreign method %s selects service %q, which publishes no protobuf contract", m.GetMethod(), service) + } + served.ByService[service] = append(served.ByService[service], m) + if !slices.Contains(served.Services, service) { + served.Services = append(served.Services, service) + } + } else { + served.Foreign = append(served.Foreign, m) + } + } + for _, methods := range served.ByService { + sort.Slice(methods, func(i, j int) bool { return methods[i].GetMethod() < methods[j].GetMethod() }) } // A method one endpoint serves and another's descriptor merely carries is // served. @@ -287,23 +316,27 @@ func declaresNothing(m *compositionv1.MethodPolicyV1) bool { // foreignMethods reads the methods the module at dir serves from a protocol // it does not own, from its module contract, each with what a call requires. // A module that publishes no contract declares none. -func foreignMethods(dir string) ([]*compositionv1.MethodPolicyV1, error) { +func foreignMethods(dir string) ([]*compositionv1.MethodPolicyV1, map[string]string, error) { contract, err := modulecontract.Load(dir) if errors.Is(err, os.ErrNotExist) { - return nil, nil + return nil, nil, nil } if err != nil { - return nil, err + return nil, nil, err } var out []*compositionv1.MethodPolicyV1 + selected := map[string]string{} for _, method := range contract.ForeignMethods { m := &compositionv1.MethodPolicyV1{Method: method.Method} + if method.Service != "" { + selected[method.Method] = method.Service + } for _, required := range method.Requires { m.Requires = append(m.Requires, &authorityv1.WorkScopeV1{ResourceKind: required.ResourceKind, Actions: required.Actions}) } out = append(out, m) } - return out, nil + return out, selected, nil } // ScopeCeilings reads the module's own resource kinds with the actions declared diff --git a/pkg/gitops/method_policies_test.go b/pkg/gitops/method_policies_test.go index ff02ae924..a19dd1769 100644 --- a/pkg/gitops/method_policies_test.go +++ b/pkg/gitops/method_policies_test.go @@ -469,3 +469,71 @@ func TestAGatewayRouteCarriesThePolicyOfTheProtoMethodItRenders(t *testing.T) { t.Fatalf("a document declaring the proto's own policy: %v", err) } } + +func TestSelectedForeignPolicyOnlyJoinsItsServiceTable(t *testing.T) { + dir := policyPackage(t, false) + raw := []byte(`schema: codefly/module-contract/v1 +principal: console +namespaces: [] +queues: [] +scope_ceilings: [{resource_kind: console.spans, actions: [export]}] +bindings: [] +foreign_methods: + - service: api + method: /foreign.v1.Service/Export + requires: [{resource_kind: console.spans, actions: [export]}] +`) + file := filepath.Join(dir, modulecontract.FileName) + if err := os.WriteFile(file, raw, 0600); err != nil { + t.Fatal(err) + } + selected, ok, err := ServiceMethodPolicies(dir) + if err != nil || !ok { + t.Fatalf("selected: %v %v", ok, err) + } + if len(selected.Foreign) != 0 { + t.Fatal("selected method leaked into every service table") + } + found := false + for _, m := range selected.ByService["api"] { + if m.GetMethod() == "/foreign.v1.Service/Export" { + found = true + } + } + if !found { + t.Fatal("selected service lost foreign method") + } + filtered, err := selected.Table(func(service string) bool { return service != "api" }) + if err != nil { + t.Fatal(err) + } + for _, method := range filtered { + if method.GetMethod() == "/foreign.v1.Service/Export" { + t.Fatal("selected foreign method escaped service filtering") + } + } + legacy := strings.Replace(string(raw), " - service: api\n method:", " - method:", 1) + if err := os.WriteFile(file, []byte(legacy), 0600); err != nil { + t.Fatal(err) + } + previous, ok, err := ServiceMethodPolicies(dir) + if err != nil || !ok { + t.Fatalf("legacy: %v %v", ok, err) + } + selectedAll, err := selected.All() + if err != nil { + t.Fatal(err) + } + previousAll, err := previous.All() + if err != nil { + t.Fatal(err) + } + if len(selectedAll) != len(previousAll) { + t.Fatal("selection changed module policy set") + } + for i, m := range selectedAll { + if !proto.Equal(m, previousAll[i]) { + t.Fatal("selection changed authority") + } + } +} diff --git a/pkg/orchestration/hop_configurations.go b/pkg/orchestration/hop_configurations.go index 36a9e6b13..3573b99b0 100644 --- a/pkg/orchestration/hop_configurations.go +++ b/pkg/orchestration/hop_configurations.go @@ -222,8 +222,8 @@ func publishedHopEndpoints(ctx context.Context, module *resources.Module) ([]rou // from a protocol the module does not own, so no published contract of its // declares them — to the listeners that serve them, so the routes hold the // same method set as the policy table the authority receives -// (gitops.ServiceMethodPolicies delivers them). A foreign method names no -// service: it is served by the module's one service publishing protobuf over +// (gitops.ServiceMethodPolicies delivers them). A foreign method may select its service. Without a selection it is served +// by the module's one service publishing protobuf over // gRPC or Connect, on each of those listeners, which the hop treats as one // destination. A module where none or several services do is refused by name // rather than guessed, as is a foreign method a published contract already @@ -239,28 +239,39 @@ func routeForeignMethods(endpoints []routing.RouteEndpoint, foreign []modulecont services = append(services, endpoints[i].Service) } } - if len(services) != 1 { - methods := make([]string, 0, len(foreign)) - for _, method := range foreign { - methods = append(methods, method.Method) + + for _, method := range foreign { + service := method.Service + if service == "" { + if len(services) != 1 { + methods := make([]string, 0, len(foreign)) + for _, m := range foreign { + methods = append(methods, m.Method) + } + slices.Sort(services) + return fmt.Errorf("its module contract declares foreign methods %s, which name no service, and %d services publish gRPC or Connect contracts %v: the hop cannot tell which one serves them", + strings.Join(methods, ", "), len(services), services) + } + service = services[0] } - slices.Sort(services) - return fmt.Errorf("its module contract declares foreign methods %s, which name no service, and %d services publish gRPC or Connect contracts %v: the hop cannot tell which one serves them", - strings.Join(methods, ", "), len(services), services) - } - for i := range endpoints { - endpoint := &endpoints[i] - if !rpc(endpoint) { - continue + if !slices.Contains(services, service) { + return fmt.Errorf("foreign method %s selects service %q, which publishes no gRPC or Connect contract", method.Method, service) } - for _, method := range foreign { + // A declared service does not authorize shadowing an owned method anywhere. + for _, endpoint := range endpoints { if slices.Contains(endpoint.Methods, method.Method) { return fmt.Errorf("foreign method %s is also declared by the contract %s/%s publishes", method.Method, endpoint.Service, endpoint.Endpoint) } - endpoint.Methods = append(endpoint.Methods, method.Method) } - slices.Sort(endpoint.Methods) + for i := range endpoints { + endpoint := &endpoints[i] + if rpc(endpoint) && endpoint.Service == service { + endpoint.Methods = append(endpoint.Methods, method.Method) + slices.Sort(endpoint.Methods) + } + } } + return nil } diff --git a/pkg/orchestration/hop_rpc_routes_test.go b/pkg/orchestration/hop_rpc_routes_test.go index 87e252815..c0eac854a 100644 --- a/pkg/orchestration/hop_rpc_routes_test.go +++ b/pkg/orchestration/hop_rpc_routes_test.go @@ -4,6 +4,8 @@ import ( "context" "encoding/json" "fmt" + modulecontract "github.com/codefly-dev/core/contracts/module" + "os" "path/filepath" "strings" "testing" @@ -146,3 +148,42 @@ func TestHopRoutesRefuseForeignMethodsTheyCannotPlace(t *testing.T) { _, err = hopRoutesOf(t, rpcHopWorkspace(t, []string{"chat", "host"}, nil)) require.NoError(t, err) } + +func TestForeignMethodSelectsOnlyItsPublishedService(t *testing.T) { + method := "/foreign.v1.Service/Read" + endpoints := []routing.RouteEndpoint{ + {Service: "chat", Endpoint: "grpc", Transport: "grpc"}, + {Service: "chat", Endpoint: "connect", Transport: "connect"}, + {Service: "worker", Endpoint: "grpc", Transport: "grpc"}, + {Service: "web", Endpoint: "http", Transport: "http"}, + } + foreign := []modulecontract.ForeignMethod{{Service: "chat", Method: method}} + require.NoError(t, routeForeignMethods(endpoints, foreign)) + require.Equal(t, []string{method}, endpoints[0].Methods) + require.Equal(t, []string{method}, endpoints[1].Methods) + require.Empty(t, endpoints[2].Methods) + require.ErrorContains(t, routeForeignMethods(endpoints, foreign), "also declared") + foreign[0].Service = "missing" + require.ErrorContains(t, routeForeignMethods(endpoints, foreign), "publishes no gRPC or Connect contract") + foreign[0].Service = "web" + require.ErrorContains(t, routeForeignMethods(endpoints, foreign), "publishes no gRPC or Connect contract") +} + +func TestHopRoutesReadForeignServiceSelectionsFromContract(t *testing.T) { + t.Setenv(resources.CodeflyHomeEnv, filepath.Join(t.TempDir(), "home")) + workspace := rpcHopWorkspace(t, []string{"chat", "worker"}, []string{"/foreign.v1.Service/Read", "/foreign.v1.Service/Execute"}) + file := filepath.Join(workspace.Dir(), "modules", "owner", modulecontract.FileName) + raw, err := os.ReadFile(file) + require.NoError(t, err) + body := strings.Replace(string(raw), " - method: /foreign.v1.Service/Read", " - service: chat\n method: /foreign.v1.Service/Read", 1) + body = strings.Replace(body, " - method: /foreign.v1.Service/Execute", " - service: worker\n method: /foreign.v1.Service/Execute", 1) + require.NoError(t, os.WriteFile(file, []byte(body), 0600)) + routes, err := hopRoutesOf(t, workspace) + require.NoError(t, err) + for method, service := range map[string]string{"/foreign.v1.Service/Read": "chat", "/foreign.v1.Service/Execute": "worker"} { + destination, _, err := routes.Resolve("owner", "grpc", method) + require.NoError(t, err) + require.Equal(t, service, destination.Service) + } + +} From 59cd78e49e6dc9d05b44467496f5067be60838d8 Mon Sep 17 00:00:00 2001 From: Gael Date: Tue, 27 Oct 2026 14:46:25 -0400 Subject: [PATCH 04/18] build(deps): select Core's merged foreign-method service selector MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit codefly-dev/core#856 merged as b71ee8cd, so the `service:` field this branch consumes is available from an upstream module revision. Replace the pre-merge pin, which left every compiling job failing on `method.Service undefined (type module.ForeignMethod has no field or method Service)`. Verified with GOWORK=off: `go build ./...` and `go vet ./...` clean across the module, and the three packages this branch changes pass `go test -count=1` — pkg/gitops, pkg/orchestration and cmd/generate. Co-Authored-By: Claude Opus 5 --- go.mod | 2 +- go.sum | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/go.mod b/go.mod index 513ccffeb..8305e01e1 100644 --- a/go.mod +++ b/go.mod @@ -16,7 +16,7 @@ require ( github.com/codefly-dev/authority/libraries/delivery/go v0.0.0-20261026023425-000d44495d93 github.com/codefly-dev/authority/libraries/fixture-composition/go v0.0.0-20261027171336-5178f01696e3 github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad7224855 - github.com/codefly-dev/core v0.19.7-0.20261026194550-a8599bd3991a + github.com/codefly-dev/core v0.19.7-0.20261027183450-b71ee8cd6ed0 github.com/codefly-dev/golor v0.1.3 github.com/codefly-dev/llm v0.1.7 github.com/codefly-dev/sdk-go/workcontext v0.1.1-0.20261018052022-70dc7a546458 diff --git a/go.sum b/go.sum index 47505b108..888cbe95b 100644 --- a/go.sum +++ b/go.sum @@ -117,6 +117,8 @@ github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-00 github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-000d44495d93/go.mod h1:1rmjrVyr0h7zOuwOHS9i+BoeS9dr3XNvtupLZFwY6As= github.com/codefly-dev/core v0.19.7-0.20261026194550-a8599bd3991a h1:ar8ERpN9n9ZL29LZ63mqBZNLPZ7OpcnQ1+D1g89DbPY= github.com/codefly-dev/core v0.19.7-0.20261026194550-a8599bd3991a/go.mod h1:1ina3kP+YSAL4tlZvnEYd4P8bFGnJECfkP86ojeeF1A= +github.com/codefly-dev/core v0.19.7-0.20261027183450-b71ee8cd6ed0 h1:AH7fMtK5YrQV065aUS/9WibFEEpQiYTaxlBF/iKPTsw= +github.com/codefly-dev/core v0.19.7-0.20261027183450-b71ee8cd6ed0/go.mod h1:1ina3kP+YSAL4tlZvnEYd4P8bFGnJECfkP86ojeeF1A= github.com/codefly-dev/golor v0.1.3 h1:xmo+ceyJFRYZdvpWE2fNd0jeaadp/Ibm1BnganiGKOc= github.com/codefly-dev/golor v0.1.3/go.mod h1:sl/u/K1l7J0Pr3xyVZp8fOJYQItKKst1No9JqgzLLoY= github.com/codefly-dev/gortk v0.2.0 h1:7bOlS5valYz2zil+fZctQNcPCYBcPj86abcw9N8h1hQ= From 9e9e36a82e5f335d446d42b6794f8f0f41753ed2 Mon Sep 17 00:00:00 2001 From: Gael Date: Tue, 27 Oct 2026 15:32:40 -0400 Subject: [PATCH 05/18] style(composition): lift the foreign-method selector check out of the policy walk golangci-lint v2.14.0 refused the selector work in three places, all of them new in this branch: pkg/gitops/method_policies.go:276 rangeValCopy, 136 bytes (gocritic) pkg/orchestration/hop_configurations.go:284 rangeValCopy, 152 bytes (gocritic) pkg/gitops/method_policies.go:201 cyclomatic complexity 32 > 30 (gocyclo) Extracting publishesProtobuf answers two of them at once: the manifest walk stops copying a Service per iteration, and ServiceMethodPolicies sheds the branches that pushed it over the complexity ceiling. The hop's shadow check indexes its endpoints rather than copying a RouteEndpoint per iteration. No behaviour changes: same predicates, same refusal messages, same order. Verified with GOWORK=off: `golangci-lint run` reports 0 issues across pkg/gitops, pkg/orchestration and cmd/generate on the same linter version CI pins, `go build ./...` is clean, and all three packages pass `go test -count=1`. Co-Authored-By: Claude Opus 5 --- pkg/gitops/method_policies.go | 31 +++++++++++++++---------- pkg/orchestration/hop_configurations.go | 6 ++--- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/pkg/gitops/method_policies.go b/pkg/gitops/method_policies.go index 32b2fa085..ab0d8335b 100644 --- a/pkg/gitops/method_policies.go +++ b/pkg/gitops/method_policies.go @@ -272,18 +272,7 @@ func ServiceMethodPolicies(dir string) (served ServedMethodPolicies, ok bool, er } servedAnywhere[m.GetMethod()] = true if service := selected[m.GetMethod()]; service != "" { - published := false - for _, candidate := range manifest.Services { - if candidate.Name != service { - continue - } - for _, contract := range candidate.APIContracts { - if contract.Kind == composition.APIContractKindProtobuf { - published = true - } - } - } - if !published { + if !publishesProtobuf(manifest, service) { return served, false, fmt.Errorf("foreign method %s selects service %q, which publishes no protobuf contract", m.GetMethod(), service) } served.ByService[service] = append(served.ByService[service], m) @@ -313,6 +302,24 @@ func declaresNothing(m *compositionv1.MethodPolicyV1) bool { return !m.GetAnyCheckedCaller() && len(m.GetRequires()) == 0 } +// publishesProtobuf reports whether the manifest's service of that name +// publishes a protobuf contract, which is what a foreign method's service +// selector must name: the selector chooses where a method is served, so a +// service serving no protobuf cannot be the answer. +func publishesProtobuf(manifest *composition.PackageManifest, service string) bool { + for i := range manifest.Services { + if manifest.Services[i].Name != service { + continue + } + for _, contract := range manifest.Services[i].APIContracts { + if contract.Kind == composition.APIContractKindProtobuf { + return true + } + } + } + return false +} + // foreignMethods reads the methods the module at dir serves from a protocol // it does not own, from its module contract, each with what a call requires. // A module that publishes no contract declares none. diff --git a/pkg/orchestration/hop_configurations.go b/pkg/orchestration/hop_configurations.go index 9d3da4ad7..799311368 100644 --- a/pkg/orchestration/hop_configurations.go +++ b/pkg/orchestration/hop_configurations.go @@ -281,9 +281,9 @@ func routeForeignMethods(endpoints []routing.RouteEndpoint, foreign []modulecont return fmt.Errorf("foreign method %s selects service %q, which publishes no gRPC or Connect contract", method.Method, service) } // A declared service does not authorize shadowing an owned method anywhere. - for _, endpoint := range endpoints { - if slices.Contains(endpoint.Methods, method.Method) { - return fmt.Errorf("foreign method %s is also declared by the contract %s/%s publishes", method.Method, endpoint.Service, endpoint.Endpoint) + for i := range endpoints { + if slices.Contains(endpoints[i].Methods, method.Method) { + return fmt.Errorf("foreign method %s is also declared by the contract %s/%s publishes", method.Method, endpoints[i].Service, endpoints[i].Endpoint) } } for i := range endpoints { From a035440e0fd00dc8c0c20d2438d353e8135512a0 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 27 Oct 2026 19:59:43 -0400 Subject: [PATCH 06/18] fix(run): say which image the local runtime was handed The run's log names the canonical reference Init receives and the tag it was loaded under, so a local run's handoff is readable without --output-env. Co-Authored-By: Claude Fable 5.1 --- pkg/orchestration/local_runtime_image.go | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/pkg/orchestration/local_runtime_image.go b/pkg/orchestration/local_runtime_image.go index 4d8276757..5b846aee8 100644 --- a/pkg/orchestration/local_runtime_image.go +++ b/pkg/orchestration/local_runtime_image.go @@ -58,7 +58,13 @@ func (runner *Runner) localRuntimeImage(ctx context.Context) (string, error) { if err != nil || image == "" { return "", err } - return localRuntimeImageReference(ctx, image) + immutable, err := localRuntimeImageReference(ctx, image) + if err != nil { + return "", err + } + wool.Get(ctx).In("Runner.localRuntimeImage", wool.ThisField(runner.instance)).Info( + "handing the agent the image this machine built", wool.Field("runtime_image", immutable), wool.Field("loaded_as", image)) + return immutable, nil } // Init's image contract uses a canonical repository reference, not a bare From c1ee0e8b29c6aefe180a36d2d695bffb35bb3955 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 27 Oct 2026 22:09:08 -0400 Subject: [PATCH 07/18] feat(recover): consume Core exact-generation process recovery receipts --- cmd/recover.go | 109 ++++++++++++++++++++ cmd/recover_integration_test.go | 145 ++++++++++++++++++++++++++ cmd/recover_test.go | 177 ++++++++++++++++++++++++++++++++ cmd/root.go | 1 + docs/cli-reference.md | 39 +++++++ docs/commands.md | 37 +++++++ docs/orchestration.md | 9 ++ 7 files changed, 517 insertions(+) create mode 100644 cmd/recover.go create mode 100644 cmd/recover_integration_test.go create mode 100644 cmd/recover_test.go diff --git a/cmd/recover.go b/cmd/recover.go new file mode 100644 index 000000000..ce25d5250 --- /dev/null +++ b/cmd/recover.go @@ -0,0 +1,109 @@ +package cmd + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "strconv" + + "github.com/codefly-dev/cli/cmd/common" + "github.com/codefly-dev/core/runners/base" + "github.com/spf13/cobra" +) + +var RecoverCmd = newRecoverCommand(base.PreviewProcessGroupRecovery, base.RecoverProcessGroup) + +// Only selection and reporting live here. Core owns receipt decoding, +// authentication, generation checks, signal delivery and conditional removal. +func newRecoverCommand( + preview func(context.Context, int) (*base.ProcessGroupRecoveryExpectation, error), + recoverGroup func(context.Context, base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error), +) *cobra.Command { + parent := &cobra.Command{ + Use: "recover", Short: "Recover an explicitly selected retained Codefly resource", + } + var previewOnly bool + var expectationPath string + group := &cobra.Command{ + Use: "process-group PGID", + Short: "Preview or recover one authenticated orphan process-group generation", + Long: `Select one retained process group explicitly, independently of workspace-scoped stop. +Use --preview to authenticate it without signaling and print Core's JSON receipt. +Review and save the complete receipt, then use --expect FILE with the same PGID. +Core revalidates the registration, absent original owner and process births before +effects. Changed, missing or ambiguous evidence refuses recovery. Workspace +membership is not inferred. Containers, caches and other groups are not selected. +Recovery succeeds only when the group is dead and its exact record was removed. +An error can report partial termination; inspect the result before any retry.`, + Example: ` codefly recover process-group 12345 --preview > recovery.json + codefly recover process-group 12345 --expect recovery.json`, + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + pgid, err := strconv.Atoi(args[0]) + if err != nil || pgid <= 1 { + return errors.New("PGID must be an integer greater than 1") + } + if cmd.Flags().Changed("expect") && expectationPath == "" { + return errors.New("--expect requires a receipt file") + } + if previewOnly == (expectationPath != "") { + return errors.New("select exactly one of --preview or --expect FILE") + } + ctx, done := common.NewContext() + defer done() + ctx, stop := common.SignalContext(ctx) + defer stop() + if previewOnly { + expected, previewErr := preview(ctx, pgid) + if previewErr != nil { + return previewErr + } + if expected == nil || expected.PGID != pgid { + return errors.New("core returned no matching recovery preview") + } + return json.NewEncoder(cmd.OutOrStdout()).Encode(expected) + } + file, err := os.Open(expectationPath) + if err != nil { + return fmt.Errorf("read recovery receipt: %w", err) + } + defer file.Close() + // Bound allocation for operator input; Core enforces its own format + // size limit and is the sole decoder of the expectation. + const maxReceiptBytes = 1024 * 1024 + data, err := io.ReadAll(io.LimitReader(file, maxReceiptBytes+1)) + if err != nil { + return fmt.Errorf("read recovery receipt: %w", err) + } + if len(data) > maxReceiptBytes { + return errors.New("recovery receipt exceeds 1 MiB") + } + expected, err := base.ParseProcessGroupRecoveryExpectation(data) + if err != nil { + return fmt.Errorf("invalid recovery receipt: %w", err) + } + if expected.PGID != pgid { + return errors.New("receipt PGID does not match the explicitly selected PGID") + } + result, recoveryErr := recoverGroup(ctx, expected) + var reportErr error + if result != nil { + reportErr = json.NewEncoder(cmd.OutOrStdout()).Encode(result) + } + if recoveryErr != nil { + return errors.Join(recoveryErr, reportErr) + } + if result == nil || !result.GroupDead || !result.RecordRemoved { + return errors.Join(errors.New("recovery incomplete: group death and exact record removal were not both confirmed"), reportErr) + } + return reportErr + }, + } + group.Flags().BoolVar(&previewOnly, "preview", false, "Authenticate one generation and print its receipt without signaling") + group.Flags().StringVar(&expectationPath, "expect", "", "Recover only the generation in this reviewed Core JSON receipt") + parent.AddCommand(group) + return parent +} diff --git a/cmd/recover_integration_test.go b/cmd/recover_integration_test.go new file mode 100644 index 000000000..968750bbb --- /dev/null +++ b/cmd/recover_integration_test.go @@ -0,0 +1,145 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "os" + "os/exec" + "path/filepath" + "reflect" + "strconv" + "testing" + + "github.com/codefly-dev/core/runners/base" +) + +// Use the existing registered-owner subprocess fixture. No registry format or +// compatibility interpretation lives in this CLI test; all observations and +// effects go through Core's public API in a private home. +func recoveryCommandOrphan(t *testing.T) int { + t.Helper() + pidFile := filepath.Join(t.TempDir(), "pid") + owner := exec.Command(os.Args[0], "-test.run=^TestStopIdentityFallbackOwner$") + owner.Env = append(os.Environ(), "CODEFLY_TEST_STOP_IDENTITY_CHILD_PID="+pidFile, + "CODEFLY_TEST_STOP_IDENTITY_CHILD_BINARY="+os.Args[0]) + if out, err := owner.CombinedOutput(); err != nil { + t.Fatalf("start isolated owner: %v: %s", err, out) + } + data, err := os.ReadFile(pidFile) + if err != nil { + t.Fatal(err) + } + pgid, err := strconv.Atoi(string(data)) + if err != nil { + t.Fatal(err) + } + group, err := base.LookupProcessGroup(pgid) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + if err := group.Terminate(context.Background()); err != nil { + t.Errorf("fixture cleanup: %v", err) + } + if err := group.RemoveIfDead(); err != nil { + t.Errorf("fixture record cleanup: %v", err) + } + }) + return pgid +} + +func TestRecoveryCommandRealOwnerBoundary(t *testing.T) { + t.Setenv("HOME", t.TempDir()) + t.Setenv("CODEFLY_HOME", t.TempDir()) + target, sibling := recoveryCommandOrphan(t), recoveryCommandOrphan(t) + foreign := exec.Command(os.Args[0], "-test.run=^TestPsHelper$") + foreign.Env = append(os.Environ(), psHelperEnv+"=1") + if err := foreign.Start(); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = foreign.Process.Kill(); _ = foreign.Wait() }) + foreignBefore, err := base.InspectProcess(foreign.Process.Pid) + if err != nil { + t.Fatal(err) + } + siblingBefore, err := base.PreviewProcessGroupRecovery(t.Context(), sibling) + if err != nil { + t.Fatal(err) + } + run := func(args ...string) ([]byte, error) { + var out bytes.Buffer + command := newRecoverCommand(base.PreviewProcessGroupRecovery, base.RecoverProcessGroup) + command.SetOut(&out) + command.SetErr(&bytes.Buffer{}) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetArgs(append([]string{"process-group"}, args...)) + err := command.Execute() + return out.Bytes(), err + } + pgid := strconv.Itoa(target) + raw, err := run(pgid, "--preview") + if err != nil { + t.Fatal(err) + } + expected, err := base.ParseProcessGroupRecoveryExpectation(raw) + if err != nil { + t.Fatal(err) + } + // Preview is observational; repeat produces the same exact receipt. + again, err := base.PreviewProcessGroupRecovery(t.Context(), target) + if err != nil || again == nil || !reflect.DeepEqual(expected, *again) { + t.Fatalf("preview changed target: %v", err) + } + path := filepath.Join(t.TempDir(), "receipt.json") + if err := os.WriteFile(path, raw, 0o600); err != nil { + t.Fatal(err) + } + if _, err := run(strconv.Itoa(sibling), "--expect", path); err == nil { + t.Fatal("target mismatch succeeded") + } + changed := expected + changed.Members = append([]base.ProcessIdentity(nil), expected.Members...) + changed.Members[0].StartID++ + bad, err := json.Marshal(changed) + if err != nil { + t.Fatal(err) + } + badPath := filepath.Join(t.TempDir(), "changed.json") + if err := os.WriteFile(badPath, bad, 0o600); err != nil { + t.Fatal(err) + } + if _, err := run(pgid, "--expect", badPath); err == nil { + t.Fatal("changed birth succeeded") + } + again, err = base.PreviewProcessGroupRecovery(t.Context(), target) + if err != nil || again == nil || !reflect.DeepEqual(expected, *again) { + t.Fatalf("refused operation changed target: %v", err) + } + out, err := run(pgid, "--expect", path) + if err != nil { + t.Fatalf("recover: %v, result %s", err, out) + } + var result base.ProcessGroupRecoveryResult + if err := json.Unmarshal(out, &result); err != nil || !result.GroupDead || !result.RecordRemoved { + t.Fatalf("recovery postconditions: %s, %v", out, err) + } + if _, err := base.LookupProcessGroup(target); !errors.Is(err, base.ErrProcessGroupNotRegistered) { + t.Fatalf("target registration survived: %v", err) + } + for _, member := range expected.Members { + if _, err := base.InspectProcess(member.PID); !errors.Is(err, base.ErrProcessNotFound) { + t.Fatalf("target member %d survived: %v", member.PID, err) + } + } + siblingAfter, err := base.PreviewProcessGroupRecovery(t.Context(), sibling) + if err != nil || !reflect.DeepEqual(siblingBefore, siblingAfter) { + t.Fatalf("sibling evidence changed: %v", err) + } + foreignAfter, err := base.InspectProcess(foreign.Process.Pid) + if err != nil || foreignBefore != foreignAfter { + t.Fatalf("unregistered control changed: %v", err) + } + t.Logf("isolated target %d recovered; registered sibling %d and unregistered control %d preserved", target, sibling, foreign.Process.Pid) +} diff --git a/cmd/recover_test.go b/cmd/recover_test.go new file mode 100644 index 000000000..6f7721fcf --- /dev/null +++ b/cmd/recover_test.go @@ -0,0 +1,177 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/codefly-dev/core/runners/base" +) + +func recoveryReceipt(t *testing.T) (base.ProcessGroupRecoveryExpectation, []byte) { + t.Helper() + member := base.ProcessIdentity{PID: 12345, PGID: 12345, BootID: "test-boot", StartID: 101, Executable: "/fixture/service"} + expected := base.ProcessGroupRecoveryExpectation{ + Namespace: "authenticated-v2", RecordSHA256: strings.Repeat("a", 64), PGID: member.PGID, + Leader: member, Owner: base.ProcessIdentity{PID: 12344, BootID: "test-boot", StartID: 100, Executable: "/fixture/owner"}, + Members: []base.ProcessIdentity{member}, + } + data, err := json.Marshal(expected) + if err != nil { + t.Fatal(err) + } + return expected, data +} + +func TestRecoveryPreviewOnlyEmitsOwnerReceipt(t *testing.T) { + expected, _ := recoveryReceipt(t) + var out bytes.Buffer + command := newRecoverCommand(func(_ context.Context, pgid int) (*base.ProcessGroupRecoveryExpectation, error) { + if pgid != expected.PGID { + t.Fatalf("preview selected %d", pgid) + } + return &expected, nil + }, func(context.Context, base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error) { + t.Fatal("preview invoked effects") + return nil, nil + }) + command.SetOut(&out) + command.SetArgs([]string{"process-group", "12345", "--preview"}) + if err := command.Execute(); err != nil { + t.Fatal(err) + } + actual, err := base.ParseProcessGroupRecoveryExpectation(out.Bytes()) + if err != nil || !reflect.DeepEqual(actual, expected) { + t.Fatalf("receipt changed: %+v, %v", actual, err) + } +} + +func TestRecoveryRefusesBeforeOwnerEffects(t *testing.T) { + _, valid := recoveryReceipt(t) + for _, tc := range []struct { + name string + args []string + data string + }{ + {"no selection", nil, ""}, + {"extra selection", []string{"12345", "12346", "--preview"}, ""}, + {"invalid number", []string{"invalid", "--preview"}, ""}, + {"reserved number", []string{"1", "--preview"}, ""}, + {"no mode", []string{"12345"}, ""}, + {"both modes", []string{"12345", "--preview", "--expect", "RECEIPT"}, string(valid)}, + {"empty receipt path", []string{"12345", "--preview", "--expect="}, ""}, + {"missing file", []string{"12345", "--expect", "MISSING"}, ""}, + {"wrong target", []string{"12346", "--expect", "RECEIPT"}, string(valid)}, + {"empty", nil, ""}, + {"malformed", nil, "{"}, + {"incomplete", nil, "{}"}, + {"null", nil, "null"}, + {"trailing", nil, string(valid) + " {}"}, + {"unknown field", nil, strings.Replace(string(valid), "{", `{"unknown":true,`, 1)}, + {"duplicate field", nil, strings.Replace(string(valid), "{", `{"pgid":12345,`, 1)}, + {"oversized", nil, strings.Repeat(" ", 1024*1024+1)}, + } { + t.Run(tc.name, func(t *testing.T) { + path := filepath.Join(t.TempDir(), "receipt.json") + if err := os.WriteFile(path, []byte(tc.data), 0o600); err != nil { + t.Fatal(err) + } + args := tc.args + if args == nil && tc.name != "no selection" { + args = []string{"12345", "--expect", "RECEIPT"} + } + for i := range args { + if args[i] == "RECEIPT" { + args[i] = path + } else if args[i] == "MISSING" { + args[i] = filepath.Join(t.TempDir(), "missing") + } + } + command := newRecoverCommand(func(context.Context, int) (*base.ProcessGroupRecoveryExpectation, error) { + t.Fatal("invalid selection reached preview") + return nil, nil + }, func(context.Context, base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error) { + t.Fatal("invalid selection reached recovery") + return nil, nil + }) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs(append([]string{"process-group"}, args...)) + if err := command.Execute(); err == nil { + t.Fatal("invalid selection succeeded") + } + }) + } +} + +func TestRecoveryReportsOwnerPostconditionsAndErrors(t *testing.T) { + expected, data := recoveryReceipt(t) + path := filepath.Join(t.TempDir(), "receipt.json") + if err := os.WriteFile(path, data, 0o600); err != nil { + t.Fatal(err) + } + ownerErr := errors.New("owner refused or could not complete recovery") + for _, tc := range []struct { + name string + result *base.ProcessGroupRecoveryResult + err error + wantSuccess bool + }{ + {"complete", &base.ProcessGroupRecoveryResult{GroupDead: true, RecordRemoved: true}, nil, true}, + {"dead record retained", &base.ProcessGroupRecoveryResult{GroupDead: true}, nil, false}, + {"live record removed", &base.ProcessGroupRecoveryResult{RecordRemoved: true}, nil, false}, + {"no proof", &base.ProcessGroupRecoveryResult{}, nil, false}, + {"nil result", nil, nil, false}, + {"refused", &base.ProcessGroupRecoveryResult{}, ownerErr, false}, + {"partial failure", &base.ProcessGroupRecoveryResult{GroupDead: true}, ownerErr, false}, + {"postcondition error", &base.ProcessGroupRecoveryResult{GroupDead: true, RecordRemoved: true}, ownerErr, false}, + } { + t.Run(tc.name, func(t *testing.T) { + var out bytes.Buffer + calls := 0 + command := newRecoverCommand(func(context.Context, int) (*base.ProcessGroupRecoveryExpectation, error) { + t.Fatal("execution must not replace the reviewed receipt with a fresh preview") + return nil, nil + }, func(_ context.Context, actual base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error) { + calls++ + if !reflect.DeepEqual(actual, expected) { + t.Fatalf("owner received changed evidence: %+v", actual) + } + return tc.result, tc.err + }) + command.SetOut(&out) + command.SetErr(&bytes.Buffer{}) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetArgs([]string{"process-group", "12345", "--expect", path}) + err := command.Execute() + if calls != 1 || (err == nil) != tc.wantSuccess || (tc.err != nil && !errors.Is(err, tc.err)) { + t.Fatalf("calls=%d error=%v", calls, err) + } + if tc.result != nil { + var actual base.ProcessGroupRecoveryResult + if err := json.Unmarshal(out.Bytes(), &actual); err != nil || actual != *tc.result { + t.Fatalf("lost partial result: %q, %v", out.String(), err) + } + } + }) + } +} + +func TestRecoveryCommandRegisteredAndHelp(t *testing.T) { + command, _, err := RootCmd.Find([]string{"recover", "process-group"}) + if err != nil || command.Name() != "process-group" { + t.Fatalf("command missing: %v", err) + } + var out bytes.Buffer + command.SetOut(&out) + t.Cleanup(func() { command.SetOut(nil) }) + if err := command.Help(); err != nil || !strings.Contains(out.String(), "--expect") { + t.Fatalf("help missing receipt workflow: %v, %s", err, out.String()) + } +} diff --git a/cmd/root.go b/cmd/root.go index e4d0d9136..54abc39cb 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -261,6 +261,7 @@ func init() { // Stop a running stack (reap processes + orphaned groups, keep containers). RootCmd.AddCommand(StopCmd) + RootCmd.AddCommand(RecoverCmd) // Generate client code RootCmd.AddCommand(GenerateCmd) diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 3cc88a6f1..dc64c2995 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -73,6 +73,7 @@ Subcommands: - [`codefly provider`](#codefly-provider) - [`codefly ps`](#codefly-ps) - [`codefly publish`](#codefly-publish) +- [`codefly recover`](#codefly-recover) - [`codefly replay`](#codefly-replay) - [`codefly run`](#codefly-run) - [`codefly sbom`](#codefly-sbom) @@ -4201,6 +4202,44 @@ Flags: --dry-run show what would happen without modifying anything ``` +## `codefly recover` + +Recover an explicitly selected retained Codefly resource + +``` +codefly recover +``` + +Subcommands: + +- [`codefly recover process-group`](#codefly-recover-process-group) + +## `codefly recover process-group` + +Preview or recover one authenticated orphan process-group generation + +``` +Select one retained process group explicitly, independently of workspace-scoped stop. +Use --preview to authenticate it without signaling and print Core's JSON receipt. +Review and save the complete receipt, then use --expect FILE with the same PGID. +Core revalidates the registration, absent original owner and process births before +effects. Changed, missing or ambiguous evidence refuses recovery. Workspace +membership is not inferred. Containers, caches and other groups are not selected. +Recovery succeeds only when the group is dead and its exact record was removed. +An error can report partial termination; inspect the result before any retry. +``` + +``` +codefly recover process-group PGID +``` + +Flags: + +``` + --expect string Recover only the generation in this reviewed Core JSON receipt + --preview Authenticate one generation and print its receipt without signaling +``` + ## `codefly replay` Re-run operations recorded in a Codefly action track diff --git a/docs/commands.md b/docs/commands.md index 066f4dbb8..0198e51fa 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4401,6 +4401,43 @@ registration and owner-death evidence; a name or environment marker alone never permits signaling. Missing or ambiguous proof leaves the process running and returns an error. Registry failures prevent fallback cleanup from bypassing refusal. +### `codefly recover process-group` + +Recover one explicitly selected orphan generation when ordinary workspace-scoped +stop cannot admit it, including a retained legacy Darwin boot representation. +This is an operator operation, separate from `stop`, `clear` and startup reaping; +it does not establish missing workspace attribution or widen their selection. + +```bash +codefly recover process-group 12345 --preview > recovery.json +# Review the complete Core receipt before executing it. +codefly recover process-group 12345 --expect recovery.json +``` + +Use the same user home for both commands: Core owns the process registry under +the user's home independently of the CLI plugin directory. `--preview` +authenticates without signaling and prints the owner's JSON receipt: +namespace, registration SHA256, PGID, recorded leader/owner identities and observed +member identities. Save it unchanged. The CLI requires one PGID and exactly one +mode; execution uses Core's strict receipt parser and rejects a different PGID. +There is no name/port selection, `--all`, inferred workspace or automatic retry. + +Core owns format compatibility and revalidates unchanged registration evidence, +original-owner absence, supported boot identity and initial member births before +effects. Ambiguous namespaces or changed/missing evidence refuse. Later escalation +can reach authenticated descendants of that registered generation; the receipt +does not authorize a replacement generation. Core uses its current identity-bound +delivery and removes only the unchanged registration after proven group death. +It never rewrites the retained boot identity. + +Execution prints Core's `group_dead` and `record_removed` observations, including +partial results on error. Exit status is successful only when Core returns no +error **and both are true**. Failure can follow partial termination; preserve the +receipt and output and inspect the cause before a new preview or retry. Neither a +nil removal error nor an absent PID alone is reported as complete recovery. +Containers, caches and other registrations are not selected. This explicit +operator workflow is not exposed as an MCP tool. + ### `codefly ps` List signature-matched frontend dev servers (`next dev` / `npm run dev` / `vite`) diff --git a/docs/orchestration.md b/docs/orchestration.md index bf843d9f8..ac30f70ca 100644 --- a/docs/orchestration.md +++ b/docs/orchestration.md @@ -1155,6 +1155,15 @@ and update its agent before restarting. A successful stop must not hide a process it could not authenticate. If registry cleanup fails, stop also skips the signature-based orphan fallback so it cannot bypass that refusal. +An operator can separately select a retained orphan generation using +`codefly recover process-group PGID --preview`, review its Core-produced receipt, +then supply it with `--expect FILE`. This adds no admission to scoped stop or +startup reaping and infers no workspace membership. Core owns legacy boot-format +authentication, unchanged-generation checks, current identity-bound delivery and +conditional removal. The CLI reports completion only when Core returns no error, +the group is dead and the exact record was removed; partial failures remain +errors. See [the command workflow](commands.md#codefly-recover-process-group). + `Stop()` and `Shutdown()` do not fan every manager out at once. Launch order is not completion order, so stopping a database concurrently with the API draining into it can cut off that API's last write. Both walk **reverse topological From 846f8215aca0c00871ad408849b1f0cbde57345b Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 27 Oct 2026 22:24:13 -0400 Subject: [PATCH 08/18] fix(recover): share Core receipt bound and verify encoded round trips --- cmd/recover.go | 7 ++- cmd/recover_receipt_test.go | 93 +++++++++++++++++++++++++++++++++++++ cmd/recover_test.go | 2 +- docs/commands.md | 4 ++ 4 files changed, 101 insertions(+), 5 deletions(-) create mode 100644 cmd/recover_receipt_test.go diff --git a/cmd/recover.go b/cmd/recover.go index ce25d5250..7b9c0fe73 100644 --- a/cmd/recover.go +++ b/cmd/recover.go @@ -73,13 +73,12 @@ An error can report partial termination; inspect the result before any retry.`, defer file.Close() // Bound allocation for operator input; Core enforces its own format // size limit and is the sole decoder of the expectation. - const maxReceiptBytes = 1024 * 1024 - data, err := io.ReadAll(io.LimitReader(file, maxReceiptBytes+1)) + data, err := io.ReadAll(io.LimitReader(file, base.MaxProcessGroupRecoveryExpectationBytes+1)) if err != nil { return fmt.Errorf("read recovery receipt: %w", err) } - if len(data) > maxReceiptBytes { - return errors.New("recovery receipt exceeds 1 MiB") + if len(data) > base.MaxProcessGroupRecoveryExpectationBytes { + return fmt.Errorf("recovery receipt exceeds %d bytes", base.MaxProcessGroupRecoveryExpectationBytes) } expected, err := base.ParseProcessGroupRecoveryExpectation(data) if err != nil { diff --git a/cmd/recover_receipt_test.go b/cmd/recover_receipt_test.go new file mode 100644 index 000000000..83f677139 --- /dev/null +++ b/cmd/recover_receipt_test.go @@ -0,0 +1,93 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/codefly-dev/core/runners/base" +) + +// This exercises receipt transport, not authentication of a large live group. +// Core owns producer admission; the CLI must preserve its accepted byte range. +func TestRecoveryPreviewReceiptByteBoundary(t *testing.T) { + for name, size := range map[string]int{ + "above registry storage limit": 20 * 1024, + "inclusive receipt limit": base.MaxProcessGroupRecoveryExpectationBytes, + "one byte above limit": base.MaxProcessGroupRecoveryExpectationBytes + 1, + } { + t.Run(name, func(t *testing.T) { + expected, raw := recoveryReceipt(t) + member := expected.Leader + member.PID = 100000 + memberJSON, err := json.Marshal(member) + if err != nil { + t.Fatal(err) + } + // Each extra member adds one comma plus its JSON; keep PID widths + // constant, then pad one executable by less than a member's size. + count := (size - len(raw) - 1) / (len(memberJSON) + 1) + for i := 0; i < count; i++ { + member.PID = 100000 + i + expected.Members = append(expected.Members, member) + } + raw, err = json.Marshal(expected) + if err != nil { + t.Fatal(err) + } + expected.Members[len(expected.Members)-1].Executable += strings.Repeat("x", size-len(raw)-1) + var out bytes.Buffer + calls := 0 + command := newRecoverCommand(func(context.Context, int) (*base.ProcessGroupRecoveryExpectation, error) { + return &expected, nil + }, func(_ context.Context, got base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error) { + calls++ + if !reflect.DeepEqual(got, expected) { + t.Fatal("receipt transport changed the owner expectation") + } + return &base.ProcessGroupRecoveryResult{GroupDead: true, RecordRemoved: true}, nil + }) + command.SetOut(&out) + command.SetErr(&bytes.Buffer{}) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetArgs([]string{"process-group", "12345", "--preview"}) + if err := command.Execute(); err != nil { + t.Fatal(err) + } + if out.Len() != size || out.Bytes()[size-1] != '\n' || calls != 0 { + t.Fatalf("preview bytes=%d, want %d including newline; effects=%d", out.Len(), size, calls) + } + oversized := size > base.MaxProcessGroupRecoveryExpectationBytes + parsed, err := base.ParseProcessGroupRecoveryExpectation(out.Bytes()) + if oversized && err == nil { + t.Fatal("Core parser accepted oversized receipt") + } + if !oversized && (err != nil || !reflect.DeepEqual(parsed, expected)) { + t.Fatalf("actual preview output does not round-trip through Core: %v", err) + } + path := filepath.Join(t.TempDir(), "receipt.json") + if err := os.WriteFile(path, out.Bytes(), 0o600); err != nil { + t.Fatal(err) + } + out.Reset() + command.SetArgs([]string{"process-group", "12345", "--preview=false", "--expect", path}) + if oversized { + // Injected oversized preview tests the consumer's bound; Core's + // real producer refuses this expectation before returning it. + if err := command.Execute(); err == nil || calls != 0 { + t.Fatalf("oversized receipt reached effects: %v, calls=%d", err, calls) + } + return + } + if err := command.Execute(); err != nil || calls != 1 { + t.Fatalf("accepted preview cannot be consumed: %v, calls=%d", err, calls) + } + t.Logf("%d-byte command preview, including newline, consumed unchanged", size) + }) + } +} diff --git a/cmd/recover_test.go b/cmd/recover_test.go index 6f7721fcf..a15aca60a 100644 --- a/cmd/recover_test.go +++ b/cmd/recover_test.go @@ -75,7 +75,7 @@ func TestRecoveryRefusesBeforeOwnerEffects(t *testing.T) { {"trailing", nil, string(valid) + " {}"}, {"unknown field", nil, strings.Replace(string(valid), "{", `{"unknown":true,`, 1)}, {"duplicate field", nil, strings.Replace(string(valid), "{", `{"pgid":12345,`, 1)}, - {"oversized", nil, strings.Repeat(" ", 1024*1024+1)}, + {"oversized", nil, strings.Repeat(" ", base.MaxProcessGroupRecoveryExpectationBytes+1)}, } { t.Run(tc.name, func(t *testing.T) { path := filepath.Join(t.TempDir(), "receipt.json") diff --git a/docs/commands.md b/docs/commands.md index 0198e51fa..b431d4905 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4421,6 +4421,10 @@ namespace, registration SHA256, PGID, recorded leader/owner identities and obser member identities. Save it unchanged. The CLI requires one PGID and exactly one mode; execution uses Core's strict receipt parser and rejects a different PGID. There is no name/port selection, `--all`, inferred workspace or automatic retry. +Core bounds the complete receipt to 1 MiB, including the preview's trailing +newline. Preview refuses a generation whose receipt cannot fit; the CLI uses +the same owner-defined bound when reading it. This is independent of the +smaller registry-record storage limit. Core owns format compatibility and revalidates unchanged registration evidence, original-owner absence, supported boot identity and initial member births before From 4513cfe8ee0ed7c9dad1fc6b4b6bd13dd8b31976 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 27 Oct 2026 23:14:30 -0400 Subject: [PATCH 09/18] docs(generate): name the runtime owner artifact renderer --- docs/commands.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/commands.md b/docs/commands.md index cebd5880c..f3761ae09 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -3646,7 +3646,8 @@ policy rows the authority applies, `" "`, are derived from it exactly as from a REST module's contract, so the document describes the routes as the backend serves them: no `basePath` when its paths already carry the prefix the authority's edge forwards them on (`/v1//…`). A solution -built on solution-runtime-go renders it with `solution.InterfaceDocument`. +built on solution-runtime-go renders it with `Server.InterfaceArtifact`, which +also emits the operation markers consumed by `generate runnables`. An exported endpoint that carries no machine-readable contract is skipped with a notice, never a failure: `tcp` and `mcp` endpoints, an `http` endpoint whose From c16938eff31f25f76e35379b1501d5df1063746e Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Tue, 27 Oct 2026 23:27:51 -0400 Subject: [PATCH 10/18] feat(recover): stop an explicit container through Core receipt checks --- cmd/recover_container.go | 101 +++++++++++++++ cmd/recover_container_test.go | 229 ++++++++++++++++++++++++++++++++++ docs/cli-reference.md | 28 +++++ docs/commands.md | 29 +++++ 4 files changed, 387 insertions(+) create mode 100644 cmd/recover_container.go create mode 100644 cmd/recover_container_test.go diff --git a/cmd/recover_container.go b/cmd/recover_container.go new file mode 100644 index 000000000..fd68b60cf --- /dev/null +++ b/cmd/recover_container.go @@ -0,0 +1,101 @@ +package cmd + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + + "github.com/codefly-dev/cli/cmd/common" + "github.com/codefly-dev/core/runners/dockerrun" + "github.com/spf13/cobra" +) + +func init() { + RecoverCmd.AddCommand(newRecoverContainerCommand(dockerrun.PreviewContainerStop, dockerrun.StopContainer)) +} + +// Core owns receipt validation, exact-object comparison, stop and readback. +// This command selects one operator-supplied ID; it infers no service ownership. +func newRecoverContainerCommand( + preview func(context.Context, string) (*dockerrun.ContainerStopExpectation, error), + stopContainer func(context.Context, dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error), +) *cobra.Command { + var previewOnly bool + var expectationPath string + command := &cobra.Command{ + Use: "container ID", + Short: "Preview or stop one explicitly selected container, retaining its data", + Long: `Select one container by its full lowercase 64-hex Docker ID. +Use --preview to inspect it without effects and print Core's JSON receipt. +Review and save the complete receipt, then use --expect FILE with the same ID. +Core compares the identity, creation time, name, labels and mounts before stopping +that ID only. AutoRemove containers refuse. No name lookup, parent-service claim, +label rewrite, container removal, volume removal or restart is performed. +Success requires the same container retained and stopped with unchanged mounts +and labels. This does not prove stored data contents or qualify producer reuse. +An error may follow a stop; inspect the reported result before any retry.`, + Example: ` codefly recover container --preview > container-stop.json + codefly recover container --expect container-stop.json`, + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + id := args[0] + if cmd.Flags().Changed("expect") && expectationPath == "" { + return errors.New("--expect requires a receipt file") + } + if previewOnly == (expectationPath != "") { + return errors.New("select exactly one of --preview or --expect FILE") + } + ctx, done := common.NewContext() + defer done() + ctx, stop := common.SignalContext(ctx) + defer stop() + if previewOnly { + expected, err := preview(ctx, id) + if err != nil { + return err + } + if expected == nil || expected.ID != id { + return errors.New("core returned no matching container stop preview") + } + return json.NewEncoder(cmd.OutOrStdout()).Encode(expected) + } + file, err := os.Open(expectationPath) + if err != nil { + return fmt.Errorf("read container stop receipt: %w", err) + } + defer file.Close() + data, err := io.ReadAll(io.LimitReader(file, dockerrun.MaxContainerStopExpectationBytes+1)) + if err != nil { + return fmt.Errorf("read container stop receipt: %w", err) + } + if len(data) > dockerrun.MaxContainerStopExpectationBytes { + return fmt.Errorf("container stop receipt exceeds %d bytes", dockerrun.MaxContainerStopExpectationBytes) + } + expected, err := dockerrun.ParseContainerStopExpectation(data) + if err != nil { + return fmt.Errorf("invalid container stop receipt: %w", err) + } + if expected.ID != id { + return errors.New("receipt ID does not match the explicitly selected container ID") + } + result, stopErr := stopContainer(ctx, expected) + var reportErr error + if result != nil { + reportErr = json.NewEncoder(cmd.OutOrStdout()).Encode(result) + } + if stopErr != nil { + return errors.Join(stopErr, reportErr) + } + if result == nil || !result.Stopped || !result.Retained || !result.MountsUnchanged { + return errors.Join(errors.New("container stop incomplete: stopped, retained and unchanged mounts were not all confirmed"), reportErr) + } + return reportErr + }, + } + command.Flags().BoolVar(&previewOnly, "preview", false, "Inspect one full container ID and print its receipt without stopping it") + command.Flags().StringVar(&expectationPath, "expect", "", "Stop only the exact container matching this reviewed Core JSON receipt") + return command +} diff --git a/cmd/recover_container_test.go b/cmd/recover_container_test.go new file mode 100644 index 000000000..6bebd6984 --- /dev/null +++ b/cmd/recover_container_test.go @@ -0,0 +1,229 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/codefly-dev/core/runners/dockerrun" + "github.com/docker/docker/api/types/container" + "github.com/stretchr/testify/require" +) + +func containerStopReceipt(t *testing.T) (dockerrun.ContainerStopExpectation, []byte) { + t.Helper() + expected := dockerrun.ContainerStopExpectation{ + ID: strings.Repeat("a", 64), Created: "2026-10-28T00:38:43.921733968Z", Name: "codefly-fixture", + Labels: map[string]string{dockerrun.LabelCodeflyOwner: "true", dockerrun.LabelCodeflyName: "codefly-fixture"}, + Mounts: []container.MountPoint{}, + } + var encoded bytes.Buffer + require.NoError(t, json.NewEncoder(&encoded).Encode(expected)) + return expected, encoded.Bytes() +} + +func TestContainerRecoveryPreviewReceiptRoundTrip(t *testing.T) { + for _, atLimit := range []bool{false, true} { + expected, data := containerStopReceipt(t) + if atLimit { + expected.Labels["padding"] = "" + initial, err := json.Marshal(expected) + require.NoError(t, err) + expected.Labels["padding"] = strings.Repeat("x", dockerrun.MaxContainerStopExpectationBytes-len(initial)-1) + } + var out bytes.Buffer + preview := newRecoverContainerCommand(func(_ context.Context, id string) (*dockerrun.ContainerStopExpectation, error) { + require.Equal(t, expected.ID, id) + return &expected, nil + }, func(context.Context, dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + t.Fatal("preview invoked effects") + return nil, nil + }) + preview.SetOut(&out) + preview.SetArgs([]string{expected.ID, "--preview"}) + require.NoError(t, preview.Execute()) + if atLimit { + require.Equal(t, dockerrun.MaxContainerStopExpectationBytes, out.Len()) + } else { + require.Equal(t, string(data), out.String()) + require.Contains(t, out.String(), `"mounts":[]`) + } + parsed, err := dockerrun.ParseContainerStopExpectation(out.Bytes()) + require.NoError(t, err) + require.Equal(t, expected, parsed) + file := filepath.Join(t.TempDir(), "expected.json") + require.NoError(t, os.WriteFile(file, out.Bytes(), 0o600)) + calls := 0 + command := newRecoverContainerCommand(nil, func(_ context.Context, actual dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + calls++ + require.Equal(t, expected, actual) + return &dockerrun.ContainerStopResult{Stopped: true, Retained: true, MountsUnchanged: true}, nil + }) + command.SetOut(&bytes.Buffer{}) + command.SetArgs([]string{expected.ID, "--expect", file}) + require.NoError(t, command.Execute()) + require.Equal(t, 1, calls) + } +} + +func TestContainerRecoveryRefusesBeforeEffects(t *testing.T) { + expected, valid := containerStopReceipt(t) + for _, tc := range []struct { + name string + args []string + data string + }{ + {"no ID", []string{"--preview"}, ""}, + {"extra ID", []string{expected.ID, expected.ID, "--preview"}, ""}, + {"no mode", []string{expected.ID}, ""}, + {"both modes", []string{expected.ID, "--preview", "--expect", "RECEIPT"}, string(valid)}, + {"empty path", []string{expected.ID, "--preview", "--expect="}, ""}, + {"missing file", []string{expected.ID, "--expect", "MISSING"}, ""}, + {"wrong target", []string{strings.Repeat("b", 64), "--expect", "RECEIPT"}, string(valid)}, + {"short target", []string{expected.ID[:12], "--expect", "RECEIPT"}, string(valid)}, + {"empty", nil, ""}, + {"malformed", nil, "{"}, + {"incomplete", nil, "{}"}, + {"null", nil, "null"}, + {"trailing", nil, string(valid) + " {}"}, + {"unknown", nil, strings.Replace(string(valid), "{", `{"unknown":true,`, 1)}, + {"duplicate", nil, strings.Replace(string(valid), "{", `{"name":"codefly-fixture",`, 1)}, + {"null mounts", nil, strings.Replace(string(valid), `"mounts":[]`, `"mounts":null`, 1)}, + {"oversize", nil, strings.Repeat(" ", dockerrun.MaxContainerStopExpectationBytes+1)}, + } { + t.Run(tc.name, func(t *testing.T) { + file := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(file, []byte(tc.data), 0o600)) + args := tc.args + if args == nil { + args = []string{expected.ID, "--expect", "RECEIPT"} + } + for i := range args { + if args[i] == "RECEIPT" { + args[i] = file + } else if args[i] == "MISSING" { + args[i] = filepath.Join(t.TempDir(), "missing") + } + } + command := newRecoverContainerCommand(func(context.Context, string) (*dockerrun.ContainerStopExpectation, error) { + t.Fatal("invalid CLI input reached preview") + return nil, nil + }, func(context.Context, dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + t.Fatal("invalid input reached stop") + return nil, nil + }) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs(args) + require.Error(t, command.Execute()) + }) + } +} + +func TestContainerRecoveryReportsAllPostconditions(t *testing.T) { + expected, data := containerStopReceipt(t) + file := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(file, data, 0o600)) + ownerErr := errors.New("owner could not verify stop") + for bits := 0; bits < 8; bits++ { + for _, failed := range []bool{false, true} { + result := &dockerrun.ContainerStopResult{Stopped: bits&1 != 0, Retained: bits&2 != 0, MountsUnchanged: bits&4 != 0} + var out bytes.Buffer + command := newRecoverContainerCommand(nil, func(_ context.Context, actual dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + require.Equal(t, expected, actual) + if failed { + return result, ownerErr + } + return result, nil + }) + command.SetOut(&out) + command.SetErr(&bytes.Buffer{}) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetArgs([]string{expected.ID, "--expect", file}) + err := command.Execute() + require.Equal(t, bits == 7 && !failed, err == nil) + if failed { + require.ErrorIs(t, err, ownerErr) + } + var reported dockerrun.ContainerStopResult + require.NoError(t, json.Unmarshal(out.Bytes(), &reported)) + require.Equal(t, *result, reported) + } + } + command := newRecoverContainerCommand(nil, func(context.Context, dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + return nil, nil + }) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs([]string{expected.ID, "--expect", file}) + require.ErrorContains(t, command.Execute(), "incomplete") +} + +func TestContainerRecoveryPreviewRefusalsAndRegistration(t *testing.T) { + expected, _ := containerStopReceipt(t) + for _, result := range []*dockerrun.ContainerStopExpectation{nil, {ID: strings.Repeat("b", 64)}} { + command := newRecoverContainerCommand(func(context.Context, string) (*dockerrun.ContainerStopExpectation, error) { return result, nil }, nil) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs([]string{expected.ID, "--preview"}) + require.ErrorContains(t, command.Execute(), "no matching") + } + for _, badID := range []string{"fixture-name", expected.ID[:12], strings.ToUpper(expected.ID)} { + // The owner validates positional syntax before opening Docker. + command := newRecoverContainerCommand(dockerrun.PreviewContainerStop, nil) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs([]string{badID, "--preview"}) + require.ErrorContains(t, command.Execute(), "64-hex") + } + command, _, err := RootCmd.Find([]string{"recover", "container"}) + require.NoError(t, err) + require.Equal(t, "container", command.Name()) + var help bytes.Buffer + previous := command.OutOrStdout() + command.SetOut(&help) + t.Cleanup(func() { command.SetOut(previous) }) + require.NoError(t, command.Help()) + require.Contains(t, help.String(), "AutoRemove") + require.Contains(t, help.String(), "--expect") +} + +type containerStopReportWriter struct{ err error } + +func (w containerStopReportWriter) Write([]byte) (int, error) { return 0, w.err } + +func TestContainerRecoveryPreservesOwnerAndReportingErrors(t *testing.T) { + expected, data := containerStopReceipt(t) + ownerErr, outputErr := errors.New("inspection refused"), errors.New("output failed") + preview := newRecoverContainerCommand(func(context.Context, string) (*dockerrun.ContainerStopExpectation, error) { + return nil, ownerErr + }, nil) + preview.SetOut(&bytes.Buffer{}) + preview.SetErr(&bytes.Buffer{}) + preview.SetArgs([]string{expected.ID, "--preview"}) + require.ErrorIs(t, preview.Execute(), ownerErr) + file := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(file, data, 0o600)) + for _, failed := range []bool{false, true} { + command := newRecoverContainerCommand(nil, func(context.Context, dockerrun.ContainerStopExpectation) (*dockerrun.ContainerStopResult, error) { + result := &dockerrun.ContainerStopResult{Stopped: true, Retained: true, MountsUnchanged: true} + if failed { + return result, ownerErr + } + return result, nil + }) + command.SetOut(containerStopReportWriter{err: outputErr}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs([]string{expected.ID, "--expect", file}) + err := command.Execute() + require.ErrorIs(t, err, outputErr) + if failed { + require.ErrorIs(t, err, ownerErr) + } + } +} diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 486153166..90be2731b 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -4190,8 +4190,36 @@ codefly recover Subcommands: +- [`codefly recover container`](#codefly-recover-container) - [`codefly recover process-group`](#codefly-recover-process-group) +## `codefly recover container` + +Preview or stop one explicitly selected container, retaining its data + +``` +Select one container by its full lowercase 64-hex Docker ID. +Use --preview to inspect it without effects and print Core's JSON receipt. +Review and save the complete receipt, then use --expect FILE with the same ID. +Core compares the identity, creation time, name, labels and mounts before stopping +that ID only. AutoRemove containers refuse. No name lookup, parent-service claim, +label rewrite, container removal, volume removal or restart is performed. +Success requires the same container retained and stopped with unchanged mounts +and labels. This does not prove stored data contents or qualify producer reuse. +An error may follow a stop; inspect the reported result before any retry. +``` + +``` +codefly recover container ID +``` + +Flags: + +``` + --expect string Stop only the exact container matching this reviewed Core JSON receipt + --preview Inspect one full container ID and print its receipt without stopping it +``` + ## `codefly recover process-group` Preview or recover one authenticated orphan process-group generation diff --git a/docs/commands.md b/docs/commands.md index f3761ae09..954fdcb89 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4380,6 +4380,35 @@ registration and owner-death evidence; a name or environment marker alone never permits signaling. Missing or ambiguous proof leaves the process running and returns an error. Registry failures prevent fallback cleanup from bypassing refusal. +### `codefly recover container` + +Stop one independently selected container by its full lowercase 64-hex Docker ID, +preserving the container, writable layer and mounts. This explicit operator +operation establishes no parent-service ownership and does not relax normal +runtime adoption or workspace-scoped stop. + +```bash +codefly recover container --preview > container-stop.json +# Review the complete Core receipt and exact target before executing it. +codefly recover container --expect container-stop.json +``` + +Preview is read-only. Core records the ID, creation time, exact name, labels and +mounts; execution uses Core's strict parser and compares the receipt against +that same immutable ID before stopping it. The CLI rejects a positional ID that +differs from the receipt. Core owns the 64 KiB inclusive receipt limit, including +the preview's newline; an empty mount list is explicit `[]`. AutoRemove +containers refuse before effects because stopping them could delete the object. + +Success requires no owner error and all three reported postconditions: +`stopped`, `retained`, and `mounts_unchanged`. Core also checks unchanged labels +and identity. An error may follow an actual stop; partial results are printed, +and uncertain or missing readback never counts as success. No name/prefix +selection, ledger claim, relabeling, removal, restart or automatic retry occurs. +This operation verifies retention and mount identity, not stored object contents +or producer migration. Corrected-producer reuse and data checks remain separate. +It is an explicit operator command; no separate MCP effect tool is added. + ### `codefly recover process-group` Recover one explicitly selected orphan generation when ordinary workspace-scoped From a92a10be2b618285b7f33fb842f04a1b48471529 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 00:29:14 -0400 Subject: [PATCH 11/18] build: adopt released Core v0.19.9 recovery APIs --- docs/supported-matrix.md | 7 +++-- go.mod | 28 ++++++++--------- go.sum | 60 ++++++++++++++++++------------------- pkg/conformance/matrix.json | 2 +- 4 files changed, 49 insertions(+), 48 deletions(-) diff --git a/docs/supported-matrix.md b/docs/supported-matrix.md index 93646fbe9..7376428e1 100644 --- a/docs/supported-matrix.md +++ b/docs/supported-matrix.md @@ -27,11 +27,12 @@ When `go.mod` moves to a new core release line, update the `core` field in `go test ./pkg/conformance/...` before the full coverage and race gates. `TestMatrixCoreMatchesGoMod` refuses a stale declared version. -The current pin targets Core's `v0.19.8` release line. The exact source +The current pin targets Core's published `v0.19.9` release line. The exact source revision is declared in `go.mod`; the CLI consumes Core's container parent -attribution and release acquisition APIs. These source gates qualify the CLI +attribution, release acquisition and explicit process-group/container recovery +APIs. These source gates qualify the CLI against that selected revision. They do not qualify separately compiled agents -or imply that the target Core release has been published. Agents creating +or update the Core code embedded in their binaries. Agents creating sidecar containers must adopt the attribution API in their own source and qualify their lifecycle before consumers select the resulting release. diff --git a/go.mod b/go.mod index 7b049174a..dd71d6284 100644 --- a/go.mod +++ b/go.mod @@ -3,7 +3,7 @@ module github.com/codefly-dev/cli go 1.27.3 require ( - connectrpc.com/connect v1.21.0 + connectrpc.com/connect v1.21.1 connectrpc.com/cors v0.1.0 github.com/Masterminds/semver v1.5.0 github.com/Masterminds/semver/v3 v3.5.0 @@ -16,7 +16,7 @@ require ( github.com/codefly-dev/authority/libraries/delivery/go v0.0.0-20261026023425-000d44495d93 github.com/codefly-dev/authority/libraries/fixture-composition/go v0.0.0-20261027171336-5178f01696e3 github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad7224855 - github.com/codefly-dev/core v0.19.8 + github.com/codefly-dev/core v0.19.9 github.com/codefly-dev/golor v0.1.3 github.com/codefly-dev/llm v0.1.7 github.com/codefly-dev/sdk-go/workcontext v0.1.1-0.20261018052022-70dc7a546458 @@ -43,12 +43,12 @@ require ( go.etcd.io/bbolt v1.5.0 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0 go.uber.org/goleak v1.3.0 - golang.org/x/crypto v0.57.0 - golang.org/x/mod v0.41.0 + golang.org/x/crypto v0.58.0 + golang.org/x/mod v0.42.0 golang.org/x/oauth2 v0.36.0 - golang.org/x/sync v0.23.0 + golang.org/x/sync v0.24.0 golang.org/x/sys v0.49.0 - golang.org/x/term v0.46.0 + golang.org/x/term v0.47.0 google.golang.org/genproto/googleapis/api v0.0.0-20261019180522-ce908d29299b // HELD at v1.83.2: v1.84.0 is affected by GO-2026-6443 (server panic via a // missing :authority/Host header) and the advisory's only fix for the 1.84 @@ -78,7 +78,7 @@ require ( github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect github.com/Masterminds/goutils v1.1.1 // indirect github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 // indirect - github.com/ProtonMail/go-crypto v1.4.1 // indirect + github.com/ProtonMail/go-crypto v1.5.2 // indirect github.com/VividCortex/ewma v1.2.0 // indirect github.com/alecthomas/chroma/v2 v2.27.0 // indirect github.com/anthropics/anthropic-sdk-go v1.63.1 // indirect @@ -109,7 +109,7 @@ require ( github.com/containerd/errdefs v1.0.0 // indirect github.com/containerd/errdefs/pkg v0.3.0 // indirect github.com/containerd/typeurl/v2 v2.3.0 // indirect - github.com/creativeprojects/go-selfupdate v1.6.0 // indirect + github.com/creativeprojects/go-selfupdate v1.7.0 // indirect github.com/cyphar/filepath-securejoin v0.7.0 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/davidmz/go-pageant v1.0.2 // indirect @@ -134,7 +134,7 @@ require ( github.com/go-openapi/jsonreference v1.0.2 // indirect github.com/go-openapi/loads v0.25.3 // indirect github.com/go-openapi/spec v1.0.1 // indirect - github.com/go-openapi/strfmt v0.27.2 // indirect + github.com/go-openapi/strfmt v0.27.3 // indirect github.com/go-openapi/swag v0.29.2 // indirect github.com/go-openapi/swag/cmdutils v0.29.2 // indirect github.com/go-openapi/swag/conv v0.29.2 // indirect @@ -153,7 +153,7 @@ require ( github.com/google/go-github/v86 v86.0.0 // indirect github.com/google/go-querystring v1.2.0 // indirect github.com/gorilla/css v1.0.1 // indirect - github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 // indirect + github.com/grpc-ecosystem/grpc-gateway/v2 v2.32.0 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-cleanhttp v0.5.2 // indirect github.com/hashicorp/go-retryablehttp v0.7.8 // indirect @@ -228,7 +228,7 @@ require ( github.com/yusufpapurcu/wmi v1.2.4 // indirect gitlab.com/gitlab-org/api/client-go v1.46.0 // indirect go.opentelemetry.io/auto/sdk v1.2.1 // indirect - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 // indirect + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0 // indirect go.opentelemetry.io/otel v1.47.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 // indirect @@ -241,9 +241,9 @@ require ( go.yaml.in/yaml/v2 v2.4.4 // indirect go.yaml.in/yaml/v3 v3.0.5 // indirect go.yaml.in/yaml/v4 v4.0.0-rc.6 // indirect - golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba // indirect - golang.org/x/net v0.60.0 // indirect - golang.org/x/text v0.42.0 // indirect + golang.org/x/exp v0.0.0-20261014143641-f0251eb4aa3e // indirect + golang.org/x/net v0.61.0 // indirect + golang.org/x/text v0.43.0 // indirect golang.org/x/time v0.16.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20261012220534-4341edf3f941 // indirect gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect diff --git a/go.sum b/go.sum index e7a05990b..628170996 100644 --- a/go.sum +++ b/go.sum @@ -10,8 +10,8 @@ cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdB cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10= code.gitea.io/sdk/gitea v0.25.1 h1:yywxWwoV+SdjHtbC6unBiXojWdZOtoHuGhEazEXeWuE= code.gitea.io/sdk/gitea v0.25.1/go.mod h1:uDFWYBU8dgZsgOHwe6C/6olxvf8FHguNB3wW1i83fgg= -connectrpc.com/connect v1.21.0 h1:LhqSJt7jHf5NJBo9Jq/t/9FjcYAideif0mg+qe2jCUs= -connectrpc.com/connect v1.21.0/go.mod h1:A2ygJrukXwWy32vkCAAHNVguZrqZ+jeZ9rGRnGR4dN4= +connectrpc.com/connect v1.21.1 h1:h2kacdlzXpZU2OQFSv98xkjYXNUSr/ptxmGVbS/+CRM= +connectrpc.com/connect v1.21.1/go.mod h1:fhMcVJ5w6r/yTYhnkEbW6+sIF7OZ5Ay7KYgmfebh2ek= connectrpc.com/cors v0.1.0 h1:f3gTXJyDZPrDIZCQ567jxfD9PAIpopHiRDnJRt3QuOQ= connectrpc.com/cors v0.1.0/go.mod h1:v8SJZCPfHtGH1zsm+Ttajpozd4cYIUryl4dFB6QEpfg= dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8= @@ -31,8 +31,8 @@ github.com/Masterminds/sprig/v3 v3.3.0/go.mod h1:Zy1iXRYNqNLUolqCpL4uhk6SHUMAOSC github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY= github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 h1:0kQAzHq8vLs7Pptv+7TxjdETLf/nIqJpIB4oC6Ba4vY= github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29/go.mod h1:ZWa7ssZJT30CCDGJ7fk/2SBTq9BIQrrVjrcss0UW2s0= -github.com/ProtonMail/go-crypto v1.4.1 h1:9RfcZHqEQUvP8RzecWEUafnZVtEvrBVL9BiF67IQOfM= -github.com/ProtonMail/go-crypto v1.4.1/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo= +github.com/ProtonMail/go-crypto v1.5.2 h1:cucYnvqcY7UOXVD//mSyjeaPY0SSN3v5cDkYPxumINk= +github.com/ProtonMail/go-crypto v1.5.2/go.mod h1:/RaSu30DaKO4RY+XdV/ACcCcZkGr7AhUIduq5sjzzCo= github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow= github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4= github.com/agext/levenshtein v1.2.3 h1:YB2fHEn0UJagG8T1rrWknE3ZQzWM06O8AMAatNn7lmo= @@ -117,8 +117,8 @@ github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad722 github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad7224855/go.mod h1:rfn41uiORQ36RrOeFEjXHPLC9NUi0j14AoYJtcovsn4= github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-000d44495d93 h1:4EygH7/J0E6aM7hUPdbQGfit1rsLUNCtWJ8kMh2b9PE= github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-000d44495d93/go.mod h1:1rmjrVyr0h7zOuwOHS9i+BoeS9dr3XNvtupLZFwY6As= -github.com/codefly-dev/core v0.19.8 h1:7ZFXXo51P4H0r22QUbTPzs1j9d6iyftvoeqPGKDFAg4= -github.com/codefly-dev/core v0.19.8/go.mod h1:NB7x1UZ01rYTfEZEl4vUFD1A+fgOXhsWbgA3M8x4ThA= +github.com/codefly-dev/core v0.19.9 h1:lTDsRJTceSsHLTmVttxFzNM3rEjuKsT8w5NmwozLu3Y= +github.com/codefly-dev/core v0.19.9/go.mod h1:aTJY3Rw0OSVkbWRThgDzSkqieoF9mW4MiPRXM8Vzxww= github.com/codefly-dev/golor v0.1.3 h1:xmo+ceyJFRYZdvpWE2fNd0jeaadp/Ibm1BnganiGKOc= github.com/codefly-dev/golor v0.1.3/go.mod h1:sl/u/K1l7J0Pr3xyVZp8fOJYQItKKst1No9JqgzLLoY= github.com/codefly-dev/gortk v0.2.0 h1:7bOlS5valYz2zil+fZctQNcPCYBcPj86abcw9N8h1hQ= @@ -140,8 +140,8 @@ github.com/containerd/typeurl/v2 v2.3.0/go.mod h1:Qk+PAdUYArVj41TnGi6rJ+48RF0Pkc github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= -github.com/creativeprojects/go-selfupdate v1.6.0 h1:Bu3cIgdyfI1Pg8XsL8nbaT2uMjfZ8HIoxnBmPJbN0sw= -github.com/creativeprojects/go-selfupdate v1.6.0/go.mod h1:Ids8O474XGQG0jZ5vpBIhWffcGYjUP6ccOI0mMcvQbI= +github.com/creativeprojects/go-selfupdate v1.7.0 h1:hFqLlJHZLp2fZm91j+hdjT9EKRjOZ0b5M6bJAgTQwzg= +github.com/creativeprojects/go-selfupdate v1.7.0/go.mod h1:RVszRkQ0kVQB2hbqrsgrEl+sKGem5nR1qs8qdCicy+U= github.com/cyphar/filepath-securejoin v0.7.0 h1:s0Y3ITPy6sQn5xt54DuYvTF8hu134ooYLUb58DX/HjE= github.com/cyphar/filepath-securejoin v0.7.0/go.mod h1:ymLGms/u3BYaviIiuKFnUx8EkQEZeK6cInNoAPJA3o4= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -214,8 +214,8 @@ github.com/go-openapi/loads v0.25.3 h1:V+jKy/thXWdLJUuYC8sZX2dICyAa8M3DokF70jj34 github.com/go-openapi/loads v0.25.3/go.mod h1:LgLyCSOLBL2Qnj0Ps1oo2YH8jZoEKsMeVZCc+ALIxFo= github.com/go-openapi/spec v1.0.1 h1:lj2vdGpNDcVgwRc6qXdw6qt/KQpCtSa9tnUH6vpDPDk= github.com/go-openapi/spec v1.0.1/go.mod h1:M//GWQGtDUAjnP37gE6fInLgaczB+FatoipV3H1fYw8= -github.com/go-openapi/strfmt v0.27.2 h1:SG32SlbwNy92s0KJiVxt2joJeFdqIYHvwrA0OU6HqzQ= -github.com/go-openapi/strfmt v0.27.2/go.mod h1:M4CKsMO0Fb8qR10+1Ra75wCKNNquy+Vj+4LWZrhTo2E= +github.com/go-openapi/strfmt v0.27.3 h1:cyf4J5Wjpd6bFpAs44MF8PsbWF/mWXhn82Ck/hgQZgw= +github.com/go-openapi/strfmt v0.27.3/go.mod h1:Bl+xadPLa3nhBiaCAxNND1vrQfzyEjF0vBrLx96n/4E= github.com/go-openapi/swag v0.29.2 h1:9n8frkcsuQRA0INU31VbxNxt1zBaHxrHYGEf5BUjFi4= github.com/go-openapi/swag v0.29.2/go.mod h1:RkCiX1gCVXWgLOavDXE3ALJrQ/VO2ziiIzOUs3z10xo= github.com/go-openapi/swag/cmdutils v0.29.2 h1:cXEzX/nWCODon251f+1HgL75/R4XXML5IZxgLCWMoro= @@ -271,8 +271,8 @@ github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8= github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0= github.com/graph-gophers/graphql-go v1.9.0 h1:yu0ucKHLc5qGpRwLYKIWtr9bOoxovkWasuBrPQwlHls= github.com/graph-gophers/graphql-go v1.9.0/go.mod h1:23olKZ7duEvHlF/2ELEoSZaY1aNPfShjP782SOoNTyM= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 h1:Bd7KaOxzULLxtZ/K5s1aLbWhR0+5RToO65TXHsf3bqQ= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0/go.mod h1:nN7ts3dFXKtCZWc//yfkpcQNKJABg16/uDVAZpLDalo= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.32.0 h1:QMILzF8BA7xftE4zVORkRrlv08m12St3csK4RsyRgRs= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.32.0/go.mod h1:+ZQ6iUL8+ATyxGDQNuRKpNwJFdGDUi0e7E1QL9MvI7U= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= @@ -510,8 +510,8 @@ go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0 h1:Tq+E65HaNbZTeqcLw6QUhsV+/EIWdkfUH2CQzcrlE5M= go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0/go.mod h1:A+EVhDakAj1waJBPhmTszZaSr08ww8MRRKy1nB1rqck= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 h1:3g7B90UzBltIDKq1/5mrTGxTnOFDV0ICOhLoxiZ8jlg= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0/go.mod h1:Ef8SuTh59BT7+ofpDxN9z+yOlc4t2GjLmKDgYNJL/NU= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0 h1:LxwW/9ctSCv+QkE/cLR7M91ZIkXNMqJtEMi1vCw9U8s= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0/go.mod h1:tOsftB4SslBwwErVEPaenU2RpThXWPIU8DoJHEC4dyw= go.opentelemetry.io/otel v1.47.0 h1:j7ALJ/zgkS7Z6aeJW09p8VC9804bC+PpeTfCD4XPnOM= go.opentelemetry.io/otel v1.47.0/go.mod h1:8wS9O2qfXrYrzp6hIF/HOYJJf/wIhFPhR2xLuP+iXQU= go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 h1:julhjPeUH/q/7hinbSdDdqt5h7Zw9YWmRlWRhI0jd54= @@ -547,21 +547,21 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8= golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= -golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M= -golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA= -golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba h1:Ck8QetSgk912qxWLMCKxd0in+aiyBQyDSMae6e/xmpU= -golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba/go.mod h1:50RgIsmK7OwqzTTeqcSXQW8SswW0o8fRcDxmqGluJ8E= -golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c= -golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o= +golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54= +golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w= +golang.org/x/exp v0.0.0-20261014143641-f0251eb4aa3e h1:HgI6W1OolKK6tcV74is+rl9SMT0njsHlp0TbQiGXI/A= +golang.org/x/exp v0.0.0-20261014143641-f0251eb4aa3e/go.mod h1:DBoxwWhjaW8DJJw/YeM4zpeOKMC1FMXSblsYy+1mFSw= +golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg= +golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU= -golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg= +golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw= +golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI= golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs= golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q= -golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk= -golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= +golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA= +golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -578,18 +578,18 @@ golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA= golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE= -golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc= +golang.org/x/term v0.47.0 h1:cpyOwQkr2Nzxe2p6KnrTG7Ei2wtzjsuSjhpXXepZ/BM= +golang.org/x/term v0.47.0/go.mod h1:efAfAuG6RKABvg1ArGebRkvX6LXul14SM67o2e4mUCY= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI= -golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E= +golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc= +golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s= golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE= golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.51.0 h1:k4Xc/1Om9jwkBJBo4NVLMSARBoWtK10mx+W5BnXCeAI= -golang.org/x/tools v0.51.0/go.mod h1:9eEncMayCV6zRMGhR5eZEC2iBx98qWcF1HZ9Z7wJOoA= +golang.org/x/tools v0.52.0 h1:ryxg4U63oaUYk7uOLTF37n43f4qgtGvqV4Ioit1BaEs= +golang.org/x/tools v0.52.0/go.mod h1:ItcP5UNaVGG+Yc4eIYmaFLNJ4soEopkx5Y2d4lcEurs= gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4= gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E= google.golang.org/genproto/googleapis/api v0.0.0-20261019180522-ce908d29299b h1:nB3spRIaSPtxJpxb0k41yljROvm2Jrwfs2pSNdKS5y8= diff --git a/pkg/conformance/matrix.json b/pkg/conformance/matrix.json index 7ad1a1552..59a57b66a 100644 --- a/pkg/conformance/matrix.json +++ b/pkg/conformance/matrix.json @@ -1,7 +1,7 @@ { "schema_version": 1, "cli": "source", - "core": "v0.19.8", + "core": "v0.19.9", "rows": [ { "id": "linux-amd64-source", From a7ea869679fbbc231adff34f8c9f014a35ddce8e Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 00:52:56 -0400 Subject: [PATCH 12/18] feat(recover): force an explicitly selected process through Core identity checks --- cmd/recover.go | 2 +- cmd/recover_process.go | 219 ++++++++++++++++++ cmd/recover_process_integration_test.go | 158 +++++++++++++ cmd/recover_process_test.go | 284 ++++++++++++++++++++++++ docs/cli-reference.md | 35 ++- docs/commands.md | 53 +++++ pkg/processgroup/reaper_test.go | 16 ++ pkg/processgroup/registered.go | 2 +- 8 files changed, 766 insertions(+), 3 deletions(-) create mode 100644 cmd/recover_process.go create mode 100644 cmd/recover_process_integration_test.go create mode 100644 cmd/recover_process_test.go diff --git a/cmd/recover.go b/cmd/recover.go index 7b9c0fe73..02c18083c 100644 --- a/cmd/recover.go +++ b/cmd/recover.go @@ -23,7 +23,7 @@ func newRecoverCommand( recoverGroup func(context.Context, base.ProcessGroupRecoveryExpectation) (*base.ProcessGroupRecoveryResult, error), ) *cobra.Command { parent := &cobra.Command{ - Use: "recover", Short: "Recover an explicitly selected retained Codefly resource", + Use: "recover", Short: "Recover an explicitly selected resource or process", } var previewOnly bool var expectationPath string diff --git a/cmd/recover_process.go b/cmd/recover_process.go new file mode 100644 index 000000000..0af980a2d --- /dev/null +++ b/cmd/recover_process.go @@ -0,0 +1,219 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "strconv" + "syscall" + "time" + + "github.com/codefly-dev/cli/cmd/common" + "github.com/codefly-dev/cli/pkg/composition" + "github.com/codefly-dev/core/runners/base" + "github.com/spf13/cobra" +) + +const ( + processExitReceiptLimit = 16 << 10 + processObservationUnknown = "unknown" + processObservationAbsent = "absent" +) + +// Version 1 is a fixed local native-identity exception, not a registry format. +// Its exact owner type shape is frozen by TestProcessExitReceiptNativeShape. +type processExitReceipt struct { + SchemaVersion int `json:"schema_version"` + Process base.ProcessIdentity `json:"process"` +} + +type processExitResult struct { + SchemaVersion int `json:"schema_version"` + PID int `json:"pid"` + SignalAttempted bool `json:"signal_attempted"` + SignalAccepted bool `json:"signal_accepted"` + Observation string `json:"observation"` + ExpectedBirthGone bool `json:"expected_birth_gone"` +} + +func init() { + RecoverCmd.AddCommand(newRecoverProcessCommand(base.InspectProcess, base.SignalProcess)) +} + +func newRecoverProcessCommand(inspect func(int) (base.ProcessIdentity, error), signal func(context.Context, base.ProcessIdentity, syscall.Signal) error) *cobra.Command { + var preview, force bool + var path string + command := &cobra.Command{ + Use: "process PID", Short: "Preview or explicitly force one independently selected process to exit", + Long: `Select one process by PID, independently of workspace or group ownership. +Save --preview verbatim, review its full identity, then use --expect FILE --force. +Execution compares the full current identity and requests SIGKILL through Core +for that process birth only. SIGKILL is irreversible. No graceful signal, +escalation, group operation, registry removal or orphan recovery is performed. +Executable equality is checked at admission only: a same-birth exec can still +receive the signal. Children may react to lost connections or parent death; +this command does not prove that their cleanup or stored data remain untouched. +Success requires observed absence without errors; delivery alone is not exit. +Cancellation cannot revoke in-flight delivery. The five-second polling budget +does not interrupt synchronous inspection or output. An error may follow an +effect: inspect the disposition before any retry. Use the same reviewed binary +on the same host/boot; changing implementation requires a fresh preview.`, + Example: " codefly recover process 12345 --preview > process-exit.json\n codefly recover process 12345 --expect process-exit.json --force", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + pid, parseErr := strconv.Atoi(args[0]) + if parseErr != nil || pid <= 1 || pid == os.Getpid() || strconv.Itoa(pid) != args[0] { + return errors.New("PID must be a canonical decimal integer greater than 1, excluding this CLI") + } + if preview == (path != "") || (cmd.Flags().Changed("expect") && path == "") || force != !preview { + return errors.New("select --preview alone, or --expect FILE with --force") + } + ctx, done := common.NewContext() + defer done() + ctx, stop := common.SignalContext(ctx) + defer stop() + unregister := context.AfterFunc(cmd.Context(), stop) + defer unregister() + if err := cmd.Context().Err(); err != nil { + return err + } + if preview { + identity, err := inspect(pid) + if err != nil { + return err + } + if identity.PID != pid { + return errors.New("core returned a different process identity") + } + raw, err := encodeProcessExitReceipt(processExitReceipt{1, identity}) + if err = errors.Join(err, ctx.Err()); err != nil { + return err + } + _, err = cmd.OutOrStdout().Write(raw) + return err + } + expected, err := readProcessExitReceipt(ctx, path) + if err != nil { + return err + } + if expected.Process.PID != pid { + return errors.New("receipt PID does not match the explicitly selected PID") + } + result, effectErr := forceProcessExit(ctx, expected.Process, inspect, signal) + return errors.Join(effectErr, json.NewEncoder(cmd.OutOrStdout()).Encode(result)) + }, + } + command.Flags().BoolVar(&preview, "preview", false, "Print the current process identity without signaling") + command.Flags().StringVar(&path, "expect", "", "Use the complete verbatim version-1 preview receipt") + command.Flags().BoolVar(&force, "force", false, "Explicitly request irreversible SIGKILL for the selected process birth") + return command +} + +func encodeProcessExitReceipt(receipt processExitReceipt) ([]byte, error) { + p := receipt.Process + if receipt.SchemaVersion != 1 || p.PID <= 1 || p.PID == os.Getpid() || p.PGID <= 0 || p.BootID == "" || p.StartID == 0 || p.Executable == "" { + return nil, errors.New("version-1 receipt requires a complete non-self process identity") + } + raw, err := json.Marshal(receipt) + if err != nil { + return nil, err + } + raw = append(raw, '\n') + if len(raw) > processExitReceiptLimit { + return nil, fmt.Errorf("process receipt exceeds %d bytes including newline", processExitReceiptLimit) + } + return raw, nil +} + +func readProcessExitReceipt(ctx context.Context, path string) (processExitReceipt, error) { + var receipt processExitReceipt + file, err := composition.OpenInputFile(ctx, path) + if err != nil { + return receipt, err + } + defer file.Close() + raw, err := io.ReadAll(io.LimitReader(file, processExitReceiptLimit+1)) + if err != nil { + return receipt, err + } + if len(raw) > processExitReceiptLimit { + return receipt, errors.New("process receipt exceeds the 16 KiB limit") + } + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.DisallowUnknownFields() + if err = decoder.Decode(&receipt); err != nil { + return receipt, err + } + canonical, err := encodeProcessExitReceipt(receipt) + if err != nil { + return receipt, err + } + if !bytes.Equal(raw, canonical) { + return receipt, errors.New("process receipt must match the complete canonical preview bytes") + } + return receipt, ctx.Err() +} + +func sameProcessBirth(first, second base.ProcessIdentity) bool { + return first.PID == second.PID && first.BootID == second.BootID && first.StartID == second.StartID +} + +func forceProcessExit(ctx context.Context, expected base.ProcessIdentity, inspect func(int) (base.ProcessIdentity, error), signal func(context.Context, base.ProcessIdentity, syscall.Signal) error) (processExitResult, error) { + result := processExitResult{SchemaVersion: 1, PID: expected.PID, Observation: processObservationUnknown} + observe := func() (base.ProcessIdentity, error) { + result.Observation, result.ExpectedBirthGone = processObservationUnknown, false + if err := ctx.Err(); err != nil { + return base.ProcessIdentity{}, err + } + current, err := inspect(expected.PID) + switch { + case errors.Is(err, base.ErrProcessNotFound): + result.Observation, result.ExpectedBirthGone = processObservationAbsent, true + err = nil + case err != nil: + case current.PID != expected.PID || current.PGID <= 0 || current.BootID == "" || current.StartID == 0 || current.Executable == "": + err = errors.New("core returned an incomplete or mismatched process observation") + case !sameProcessBirth(current, expected): + result.Observation, result.ExpectedBirthGone = "changed_birth", true + err = errors.New("PID now names a different birth; replacement was not selected") + default: + result.Observation = "same_birth" + } + return current, errors.Join(err, ctx.Err()) + } + current, err := observe() + if err != nil || result.Observation == processObservationAbsent { + return result, err + } + if current != expected { + return result, errors.New("process execution state changed since preview") + } + if err := ctx.Err(); err != nil { + return result, err + } + result.SignalAttempted = true + deliveryErr := signal(ctx, expected, syscall.SIGKILL) + result.SignalAccepted = deliveryErr == nil + // This bounds polling, not synchronous owner inspection or in-flight delivery. + deadline := time.NewTimer(5 * time.Second) + defer deadline.Stop() + ticker := time.NewTicker(100 * time.Millisecond) + defer ticker.Stop() + for { + _, err := observe() + if err != nil || result.Observation == processObservationAbsent { + return result, errors.Join(deliveryErr, err) + } + select { + case <-ctx.Done(): + return result, errors.Join(deliveryErr, ctx.Err()) + case <-deadline.C: + return result, errors.Join(deliveryErr, errors.New("process exit unconfirmed within polling budget")) + case <-ticker.C: + } + } +} diff --git a/cmd/recover_process_integration_test.go b/cmd/recover_process_integration_test.go new file mode 100644 index 000000000..c83d5fd72 --- /dev/null +++ b/cmd/recover_process_integration_test.go @@ -0,0 +1,158 @@ +//go:build unix + +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strconv" + "syscall" + "testing" + "time" + + "github.com/codefly-dev/core/runners/base" + "github.com/stretchr/testify/require" +) + +// These fixtures are independently selected test children, not registered +// Codefly resources. All production delivery goes through Core.SignalProcess. +func startProcessExitChild(t *testing.T, command *exec.Cmd) (base.ProcessIdentity, <-chan error) { + t.Helper() + require.NoError(t, command.Start()) + identity, err := base.InspectProcess(command.Process.Pid) + require.NoError(t, err) + done := make(chan error, 1) + go func() { done <- command.Wait() }() + t.Cleanup(func() { + // The fixture's original birth remains the cleanup expectation. + _ = base.SignalProcess(context.Background(), identity, syscall.SIGKILL) + }) + return identity, done +} + +func assertProcessExitRealReadback(t *testing.T, result processExitResult, err error) { + t.Helper() + require.True(t, result.SignalAttempted) + require.True(t, result.SignalAccepted) + if err != nil { + // Linux can observe the exited child before its parent reaps it. Core + // refuses incomplete inspection; the CLI must preserve that uncertainty. + require.Equal(t, "unknown", result.Observation) + require.False(t, result.ExpectedBirthGone) + t.Logf("accepted delivery with uncertain readback (not successful absence): %v", err) + return + } + require.Equal(t, "absent", result.Observation) + require.True(t, result.ExpectedBirthGone) +} + +func TestProcessExitRealTargetAndSameGroupSibling(t *testing.T) { + target := exec.Command("sleep", "60") + target.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + identity, done := startProcessExitChild(t, target) + sibling := exec.Command("sleep", "60") + sibling.SysProcAttr = &syscall.SysProcAttr{Setpgid: true, Pgid: identity.PGID} + siblingIdentity, _ := startProcessExitChild(t, sibling) + require.Equal(t, identity.PGID, siblingIdentity.PGID) + + var preview bytes.Buffer + command := newRecoverProcessCommand(base.InspectProcess, base.SignalProcess) + command.SetOut(&preview) + command.SetArgs([]string{strconv.Itoa(identity.PID), "--preview"}) + require.NoError(t, command.Execute()) + path := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(path, preview.Bytes(), 0o600)) + var output bytes.Buffer + command = newRecoverProcessCommand(base.InspectProcess, base.SignalProcess) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetOut(&output) + command.SetArgs([]string{strconv.Itoa(identity.PID), "--expect", path, "--force"}) + exitErr := command.Execute() + var result processExitResult + require.NoError(t, json.Unmarshal(output.Bytes(), &result)) + assertProcessExitRealReadback(t, result, exitErr) + select { + case err := <-done: + require.Error(t, err, "force-exited fixture should have a signal exit status") + case <-time.After(time.Second): + t.Fatal("fixture parent did not reap selected process") + } + // A separate invocation after the fixture's parent has reaped it must + // confirm absence without another delivery. Production never auto-retries. + output.Reset() + command = newRecoverProcessCommand(base.InspectProcess, base.SignalProcess) + command.SetOut(&output) + command.SetArgs([]string{strconv.Itoa(identity.PID), "--expect", path, "--force"}) + require.NoError(t, command.Execute()) + require.NoError(t, json.Unmarshal(output.Bytes(), &result)) + require.False(t, result.SignalAttempted) + require.Equal(t, "absent", result.Observation) + require.True(t, result.ExpectedBirthGone) + actual, err := base.InspectProcess(siblingIdentity.PID) + require.NoError(t, err) + require.Equal(t, siblingIdentity, actual, "same-group sibling must survive exact-process delivery") + + // A stale birth expectation aimed at a live PID must not select its current + // occupant. Actual OS PID reuse is nondeterministic; owner handles own that + // delivery race, while this tests the CLI admission boundary with a live PID. + stale := siblingIdentity + stale.StartID++ + result, err = forceProcessExit(t.Context(), stale, base.InspectProcess, base.SignalProcess) + require.Error(t, err) + require.False(t, result.SignalAttempted) + require.Equal(t, "changed_birth", result.Observation) + actual, err = base.InspectProcess(siblingIdentity.PID) + require.NoError(t, err) + require.Equal(t, siblingIdentity, actual) +} + +func TestProcessExitParentReactionHelper(t *testing.T) { + switch os.Getenv("CLI934_PROCESS_EXIT_HELPER") { + case "parent": + child := exec.Command(os.Args[0], "-test.run=^TestProcessExitParentReactionHelper$") + child.Env = append(os.Environ(), "CLI934_PROCESS_EXIT_HELPER=watcher") + require.NoError(t, child.Start()) + // The watcher writes readiness after capturing this parent's identity. + for { + time.Sleep(time.Second) + } + case "watcher": + parent := os.Getppid() + require.NoError(t, os.WriteFile(os.Getenv("CLI934_PROCESS_EXIT_READY"), []byte(strconv.Itoa(os.Getpid())), 0o600)) + deadline := time.Now().Add(15 * time.Second) + for time.Now().Before(deadline) { + if os.Getppid() != parent { + require.NoError(t, os.WriteFile(os.Getenv("CLI934_PROCESS_EXIT_REACTION"), []byte("parent loss triggered child behavior\n"), 0o600)) + return + } + time.Sleep(10 * time.Millisecond) + } + t.Fatal("fixture parent never exited") + } +} + +func TestProcessExitCanTriggerIndirectChildReaction(t *testing.T) { + ready, reaction := filepath.Join(t.TempDir(), "ready"), filepath.Join(t.TempDir(), "reaction") + parent := exec.Command(os.Args[0], "-test.run=^TestProcessExitParentReactionHelper$") + parent.Env = append(os.Environ(), "CLI934_PROCESS_EXIT_HELPER=parent", "CLI934_PROCESS_EXIT_READY="+ready, "CLI934_PROCESS_EXIT_REACTION="+reaction) + parent.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + identity, done := startProcessExitChild(t, parent) + require.Eventually(t, func() bool { _, err := os.Stat(ready); return err == nil }, 5*time.Second, 10*time.Millisecond) + result, err := forceProcessExit(t.Context(), identity, base.InspectProcess, base.SignalProcess) + assertProcessExitRealReadback(t, result, err) + select { + case err := <-done: + require.Error(t, err) + case <-time.After(time.Second): + t.Fatal("fixture parent was not reaped") + } + _, err = base.InspectProcess(identity.PID) + require.ErrorIs(t, err, base.ErrProcessNotFound) + require.Eventually(t, func() bool { _, err := os.Stat(reaction); return err == nil }, 5*time.Second, 10*time.Millisecond) + // This is intentional contrary evidence: one-PID delivery does not prove + // zero child effects. It qualifies no historical agent cleanup implementation. +} diff --git a/cmd/recover_process_test.go b/cmd/recover_process_test.go new file mode 100644 index 000000000..c6e31c2c0 --- /dev/null +++ b/cmd/recover_process_test.go @@ -0,0 +1,284 @@ +package cmd + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "reflect" + "strconv" + "strings" + "syscall" + "testing" + "time" + + "github.com/codefly-dev/core/runners/base" + "github.com/stretchr/testify/require" +) + +func processExitTestIdentity() base.ProcessIdentity { + return base.ProcessIdentity{PID: 1234567, PGID: 1234567, BootID: "boot", StartID: 42, Executable: "fixture"} +} + +func processExitTestReceipt(t *testing.T, identity base.ProcessIdentity) (string, []byte) { + t.Helper() + raw, err := encodeProcessExitReceipt(processExitReceipt{1, identity}) + require.NoError(t, err) + path := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(path, raw, 0o600)) + return path, raw +} + +func TestProcessExitReceiptNativeShape(t *testing.T) { + // An upstream owner field addition/type/tag change requires explicit v1 + // contract review, not silent propagation through the embedded native type. + typeOf := reflect.TypeFor[base.ProcessIdentity]() + fields := []struct{ name, kind string }{{"PID", "int"}, {"PGID", "int"}, {"BootID", "string"}, {"StartID", "uint64"}, {"Executable", "string"}} + require.Equal(t, len(fields), typeOf.NumField()) + for i, field := range fields { + require.Equal(t, field.name, typeOf.Field(i).Name) + require.Equal(t, field.kind, typeOf.Field(i).Type.String()) + require.Empty(t, typeOf.Field(i).Tag) + } + _, raw := processExitTestReceipt(t, processExitTestIdentity()) + require.Equal(t, "{\"schema_version\":1,\"process\":{\"PID\":1234567,\"PGID\":1234567,\"BootID\":\"boot\",\"StartID\":42,\"Executable\":\"fixture\"}}\n", string(raw)) + result, err := json.Marshal(processExitResult{SchemaVersion: 1, PID: 1234567, Observation: "unknown"}) + require.NoError(t, err) + require.JSONEq(t, `{"schema_version":1,"pid":1234567,"signal_attempted":false,"signal_accepted":false,"observation":"unknown","expected_birth_gone":false}`, string(result)) +} + +func TestProcessExitReceiptRejectsAlteredEvidence(t *testing.T) { + _, raw := processExitTestReceipt(t, processExitTestIdentity()) + valid := string(raw) + cases := map[string]string{ + "null": "null\n", "missing process": "{\"schema_version\":1}\n", "null process": "{\"schema_version\":1,\"process\":null}\n", + "unknown": strings.Replace(valid, `"process":`, `"extra":1,"process":`, 1), + "duplicate": strings.Replace(valid, `"PID":1234567,`, `"PID":9,"PID":1234567,`, 1), + "alias": strings.Replace(valid, `"PID"`, `"pid"`, 1), + "escaped duplicate": strings.Replace(valid, `"PID":1234567,`, `"\u0050ID":9,"PID":1234567,`, 1), + "null identity field": strings.Replace(valid, `"PGID":1234567`, `"PGID":null`, 1), + "missing identity field": strings.Replace(valid, `"Executable":"fixture"`, `"Executable":""`, 1), + "unknown version": strings.Replace(valid, `"schema_version":1`, `"schema_version":2`, 1), + "trailing": valid + "{}", "whitespace": " " + valid, "missing newline": strings.TrimSuffix(valid, "\n"), + "oversize": strings.Repeat(" ", processExitReceiptLimit+1), + "missing birth": strings.Replace(valid, `"StartID":42`, `"StartID":0`, 1), + } + for name, raw := range cases { + t.Run(name, func(t *testing.T) { + path := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(path, []byte(raw), 0o600)) + _, err := readProcessExitReceipt(t.Context(), path) + require.Error(t, err) + }) + } +} + +func TestProcessExitPreviewReceiptBoundary(t *testing.T) { + identity := processExitTestIdentity() + _, raw := processExitTestReceipt(t, identity) + identity.Executable += strings.Repeat("x", processExitReceiptLimit-len(raw)) + var output bytes.Buffer + command := newRecoverProcessCommand(func(int) (base.ProcessIdentity, error) { return identity, nil }, func(context.Context, base.ProcessIdentity, syscall.Signal) error { + t.Fatal("preview signaled") + return nil + }) + command.SetOut(&output) + command.SetArgs([]string{strconv.Itoa(identity.PID), "--preview"}) + require.NoError(t, command.Execute()) + require.Len(t, output.Bytes(), processExitReceiptLimit) + path := filepath.Join(t.TempDir(), "receipt.json") + require.NoError(t, os.WriteFile(path, output.Bytes(), 0o600)) + receipt, err := readProcessExitReceipt(t.Context(), path) + require.NoError(t, err) + require.Equal(t, identity, receipt.Process) + identity.Executable += "x" + _, err = encodeProcessExitReceipt(processExitReceipt{1, identity}) + require.Error(t, err) +} + +func TestProcessExitCommandRefusesBeforeEffects(t *testing.T) { + identity := processExitTestIdentity() + path, _ := processExitTestReceipt(t, identity) + pid := strconv.Itoa(identity.PID) + cases := [][]string{ + nil, {"0", "--preview"}, {"1", "--preview"}, {"-2", "--preview"}, {"+2", "--preview"}, {"02", "--preview"}, + {"9999999999999999999999999", "--preview"}, {strconv.Itoa(os.Getpid()), "--preview"}, {pid}, + {pid, "--expect", path}, {pid, "--force"}, {pid, "--preview", "--force"}, {pid, "--preview", "--expect", path}, + {pid, "--expect=", "--force"}, {pid, "--expect", path + "-missing", "--force"}, + {"1234568", "--expect", path, "--force"}, {pid, "another", "--preview"}, + } + for _, args := range cases { + t.Run(strings.Join(args, " "), func(t *testing.T) { + command := newRecoverProcessCommand(func(int) (base.ProcessIdentity, error) { + t.Fatal("invalid selection reached inspection") + return identity, nil + }, func(context.Context, base.ProcessIdentity, syscall.Signal) error { + t.Fatal("invalid selection signaled") + return nil + }) + command.SetOut(&bytes.Buffer{}) + command.SetErr(&bytes.Buffer{}) + command.SetArgs(args) + require.Error(t, command.Execute()) + }) + } +} + +func TestProcessExitOwnerOutcomes(t *testing.T) { + expected := processExitTestIdentity() + changedBirth, changedGroup, changedExecutable, changedBoot := expected, expected, expected, expected + changedBirth.StartID++ + changedBoot.BootID = "other-boot" + changedGroup.PGID++ + changedExecutable.Executable = "execed" + inspectionErr, deliveryErr := errors.New("owner inspection incomplete"), errors.New("delivery uncertain") + type observation struct { + identity base.ProcessIdentity + err error + } + for _, tc := range []struct { + name string + reads []observation + delivery error + cancelSend bool + want processExitResult + wantErr bool + }{ + {"absent initially", []observation{{err: base.ErrProcessNotFound}}, nil, false, processExitResult{Observation: "absent", ExpectedBirthGone: true}, false}, + {"ordinary admission error", []observation{{err: inspectionErr}}, nil, false, processExitResult{Observation: "unknown"}, true}, + {"live replacement", []observation{{identity: changedBirth}}, nil, false, processExitResult{Observation: "changed_birth", ExpectedBirthGone: true}, true}, + {"changed boot", []observation{{identity: changedBoot}}, nil, false, processExitResult{Observation: "changed_birth", ExpectedBirthGone: true}, true}, + {"incomplete owner observation", []observation{{}}, nil, false, processExitResult{Observation: "unknown"}, true}, + {"changed PGID admission", []observation{{identity: changedGroup}}, nil, false, processExitResult{Observation: "same_birth"}, true}, + {"changed executable admission", []observation{{identity: changedExecutable}}, nil, false, processExitResult{Observation: "same_birth"}, true}, + {"exit observed", []observation{{identity: expected}, {err: base.ErrProcessNotFound}}, nil, false, processExitResult{SignalAttempted: true, SignalAccepted: true, Observation: "absent", ExpectedBirthGone: true}, false}, + {"ordinary readback error", []observation{{identity: expected}, {err: inspectionErr}}, nil, false, processExitResult{SignalAttempted: true, SignalAccepted: true, Observation: "unknown"}, true}, + {"error despite absence", []observation{{identity: expected}, {err: base.ErrProcessNotFound}}, deliveryErr, false, processExitResult{SignalAttempted: true, Observation: "absent", ExpectedBirthGone: true}, true}, + {"owner notfound is not absence", []observation{{identity: expected}, {identity: changedBirth}}, base.ErrProcessNotFound, false, processExitResult{SignalAttempted: true, Observation: "changed_birth", ExpectedBirthGone: true}, true}, + {"replacement after accepted send", []observation{{identity: expected}, {identity: changedBirth}}, nil, false, processExitResult{SignalAttempted: true, SignalAccepted: true, Observation: "changed_birth", ExpectedBirthGone: true}, true}, + {"cancel during accepted owner call", []observation{{identity: expected}}, nil, true, processExitResult{SignalAttempted: true, SignalAccepted: true, Observation: "unknown"}, true}, + {"cancel during uncertain owner call", []observation{{identity: expected}}, deliveryErr, true, processExitResult{SignalAttempted: true, Observation: "unknown"}, true}, + {"same birth exec is not exit", []observation{{identity: expected}, {identity: changedExecutable}, {identity: changedGroup}, {err: base.ErrProcessNotFound}}, nil, false, processExitResult{SignalAttempted: true, SignalAccepted: true, Observation: "absent", ExpectedBirthGone: true}, false}, + } { + t.Run(tc.name, func(t *testing.T) { + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + reads, calls := 0, 0 + result, err := forceProcessExit(ctx, expected, func(pid int) (base.ProcessIdentity, error) { + require.Equal(t, expected.PID, pid) + require.Less(t, reads, len(tc.reads)) + r := tc.reads[reads] + reads++ + return r.identity, r.err + }, func(_ context.Context, identity base.ProcessIdentity, sig syscall.Signal) error { + calls++ + require.Equal(t, expected, identity) + require.Positive(t, identity.PID) + require.Equal(t, syscall.SIGKILL, sig) + if tc.cancelSend { + cancel() + } + return tc.delivery + }) + require.Equal(t, tc.wantErr, err != nil, "%v", err) + if tc.delivery != nil { + require.ErrorIs(t, err, tc.delivery) + } + tc.want.SchemaVersion, tc.want.PID = 1, expected.PID + require.Equal(t, tc.want, result) + require.Equal(t, len(tc.reads), reads) + if result.SignalAttempted { + require.Equal(t, 1, calls) + } else { + require.Zero(t, calls) + } + }) + } +} + +func TestProcessExitCancellationAndPollingBudget(t *testing.T) { + expected := processExitTestIdentity() + ctx, cancel := context.WithCancel(t.Context()) + cancel() + result, err := forceProcessExit(ctx, expected, func(int) (base.ProcessIdentity, error) { t.Fatal("canceled inspection"); return expected, nil }, func(context.Context, base.ProcessIdentity, syscall.Signal) error { + t.Fatal("canceled signal") + return nil + }) + require.ErrorIs(t, err, context.Canceled) + require.False(t, result.SignalAttempted) + reads := 0 + result, err = forceProcessExit(t.Context(), expected, func(int) (base.ProcessIdentity, error) { reads++; return expected, nil }, func(context.Context, base.ProcessIdentity, syscall.Signal) error { return nil }) + require.ErrorContains(t, err, "polling budget") + require.Greater(t, reads, 1) + require.Equal(t, "same_birth", result.Observation) + require.False(t, result.ExpectedBirthGone) + require.True(t, result.SignalAccepted) +} + +type processExitFailWriter struct{ err error } + +func (writer processExitFailWriter) Write([]byte) (int, error) { return 0, writer.err } + +func TestProcessExitCommandReportingAndWiring(t *testing.T) { + expected := processExitTestIdentity() + path, _ := processExitTestReceipt(t, expected) + for _, failOutput := range []bool{false, true} { + reads := 0 + var output bytes.Buffer + deliveryErr, outputErr := errors.New("delivery uncertainty"), errors.New("output lost") + command := newRecoverProcessCommand(func(int) (base.ProcessIdentity, error) { + reads++ + if reads == 1 { + return expected, nil + } + return base.ProcessIdentity{}, base.ErrProcessNotFound + }, func(context.Context, base.ProcessIdentity, syscall.Signal) error { return deliveryErr }) + command.SetOut(&output) + if failOutput { + command.SetOut(processExitFailWriter{outputErr}) + } + command.SetErr(&bytes.Buffer{}) + command.SilenceErrors, command.SilenceUsage = true, true + command.SetArgs([]string{strconv.Itoa(expected.PID), "--expect", path, "--force"}) + err := command.Execute() + require.ErrorIs(t, err, deliveryErr) + if failOutput { + require.ErrorIs(t, err, outputErr) + } else { + var result processExitResult + require.NoError(t, json.Unmarshal(output.Bytes(), &result)) + require.True(t, result.SignalAttempted) + require.Equal(t, "absent", result.Observation) + } + } + command, _, err := RootCmd.Find([]string{"recover", "process"}) + require.NoError(t, err) + require.Equal(t, "process", command.Name()) + var help bytes.Buffer + command.SetOut(&help) + t.Cleanup(func() { command.SetOut(nil) }) + require.NoError(t, command.Help()) + for _, phrase := range []string{"SIGKILL", "irreversible", "same-birth exec", "Cancellation", "--force"} { + require.Contains(t, help.String(), phrase) + } +} + +func TestProcessExitReceiptNonRegularRefuses(t *testing.T) { + for _, path := range []string{t.TempDir(), os.DevNull} { + _, err := readProcessExitReceipt(t.Context(), path) + require.Error(t, err) + } + // Nonblocking open must not wait for a FIFO writer before type validation. + path := filepath.Join(t.TempDir(), "fifo") + require.NoError(t, syscall.Mkfifo(path, 0o600)) + done := make(chan error, 1) + go func() { _, err := readProcessExitReceipt(context.Background(), path); done <- err }() + select { + case err := <-done: + require.Error(t, err) + case <-time.After(time.Second): + t.Fatalf("receipt open waited for FIFO peer: %s", path) + } +} diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 90be2731b..95ed6fe61 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -4182,7 +4182,7 @@ Flags: ## `codefly recover` -Recover an explicitly selected retained Codefly resource +Recover an explicitly selected resource or process ``` codefly recover @@ -4191,6 +4191,7 @@ codefly recover Subcommands: - [`codefly recover container`](#codefly-recover-container) +- [`codefly recover process`](#codefly-recover-process) - [`codefly recover process-group`](#codefly-recover-process-group) ## `codefly recover container` @@ -4220,6 +4221,38 @@ Flags: --preview Inspect one full container ID and print its receipt without stopping it ``` +## `codefly recover process` + +Preview or explicitly force one independently selected process to exit + +``` +Select one process by PID, independently of workspace or group ownership. +Save --preview verbatim, review its full identity, then use --expect FILE --force. +Execution compares the full current identity and requests SIGKILL through Core +for that process birth only. SIGKILL is irreversible. No graceful signal, +escalation, group operation, registry removal or orphan recovery is performed. +Executable equality is checked at admission only: a same-birth exec can still +receive the signal. Children may react to lost connections or parent death; +this command does not prove that their cleanup or stored data remain untouched. +Success requires observed absence without errors; delivery alone is not exit. +Cancellation cannot revoke in-flight delivery. The five-second polling budget +does not interrupt synchronous inspection or output. An error may follow an +effect: inspect the disposition before any retry. Use the same reviewed binary +on the same host/boot; changing implementation requires a fresh preview. +``` + +``` +codefly recover process PID +``` + +Flags: + +``` + --expect string Use the complete verbatim version-1 preview receipt + --force Explicitly request irreversible SIGKILL for the selected process birth + --preview Print the current process identity without signaling +``` + ## `codefly recover process-group` Preview or recover one authenticated orphan process-group generation diff --git a/docs/commands.md b/docs/commands.md index 954fdcb89..07cdf4a55 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4380,6 +4380,59 @@ registration and owner-death evidence; a name or environment marker alone never permits signaling. Missing or ambiguous proof leaves the process running and returns an error. Registry failures prevent fallback cleanup from bypassing refusal. +### `codefly recover process` + +Preview or force one independently selected process to exit. This operation +establishes no Codefly, service or workspace ownership and does not select a +process group, descendants, containers or registrations. + +```bash +codefly recover process 12345 --preview > process-exit.json +# Review the complete identity and independently establish the target. +codefly recover process 12345 --expect process-exit.json --force +``` + +Execution requires the same PID and full current identity as the preview, +including process group and executable at admission. PID1 and this CLI's PID +refuse. The only signal is explicit, irreversible SIGKILL through Core's +`SignalProcess`: there is no graceful default, escalation or automatic retry. +Core binds delivery to process birth and group, not an immutable executable; +a same-birth exec after admission can still receive the signal. One CLI owner +call may involve Core's internal kernel-token retries. + +The result reports `signal_attempted`, `signal_accepted`, `observation` and +`expected_birth_gone`. Success requires observed absence and no delivery, +inspection, cancellation or output error. Already-absent is successful without +a signal attempt. A replacement is reported as `changed_birth` with a nonzero +exit and is never selected; its presence proves neither PID availability nor +that this command caused the original exit. Same-birth exec/group changes do +not prove exit. Ordinary owner inspection errors remain `unknown`, not absence. +Delivery errors remain errors even if subsequent observation finds absence. +The five-second polling budget does not interrupt synchronous inspection or +output. Cancellation prevents a new owner call once observed; it cannot revoke +in-flight delivery or its indirect effects. A nonzero exit can follow an effect. + +SIGKILL prevents the selected process's own handlers/deferred cleanup. Surviving +agents can still react to lost pipes/RPCs or parent death. This command does not +prove that older agent cleanup, stored data or descendants are untouched; assess +the actual selected binaries and ordering separately before an operation. +Orphan recovery and registry removal remain separate explicit operations. + +**Fixed local receipt exception:** Core's owner explicitly approved this +separate native-identity exception under its architecture contract policy. +Version1 embeds `base.ProcessIdentity` unchanged in a `schema_version`/`process` +envelope and uses a fixed local result shape. Both ends must use the same +reviewed implementation on the same host/boot; any change requires fresh preview. +The command does not detect the binary that produced a receipt. It is untrusted +comparison evidence, not a credential, ownership grant, registry or remote API. +The complete default JSON encoding plus its single newline must be saved +verbatim and fit within16KiB, inclusive. Reformatted, unknown-version, incomplete, +duplicate, alias, null, trailing or noncanonical input refuses, as do non-regular +files. The native field shape is frozen by a maintained test. Future shape or +meaning changes require a separately versioned contract under the protobuf +default unless another explicit exception is reviewed. This exception does not +extend the registry or Docker receipt contracts. There is no MCP effect tool. + ### `codefly recover container` Stop one independently selected container by its full lowercase 64-hex Docker ID, diff --git a/pkg/processgroup/reaper_test.go b/pkg/processgroup/reaper_test.go index 955f7e9ed..6b8271e06 100644 --- a/pkg/processgroup/reaper_test.go +++ b/pkg/processgroup/reaper_test.go @@ -578,6 +578,22 @@ func TestReaperUsesOwnerInspectionForSupportedRecords(t *testing.T) { if err := syscall.Kill(-pid, 0); err != nil { t.Fatalf("empty-record reconciliation signaled process group %d: %v", pid, err) } + + // A refused owner inspection must identify the candidate, even though its + // workspace remains unauthenticated. Alter only this test's own record. + incompatible := strings.ReplaceAll(string(recordData), leader.bootID, "incompatible-test-boot") + if err := os.WriteFile(recordPath, []byte(incompatible), 0600); err != nil { + t.Fatal(err) + } + _, previewErr := PreviewManagedProcessGroups(context.Background(), InWorkspace(t.TempDir())) + if previewErr == nil || !strings.Contains(previewErr.Error(), fmt.Sprintf("inspect registered group %d:", pid)) || !strings.Contains(previewErr.Error(), "incompatible boot identity") { + t.Fatalf("preview lost refused registration identity: %v", previewErr) + } + after, err := os.ReadFile(recordPath) + if err != nil || string(after) != incompatible { + t.Fatalf("refused preview changed record: %v", err) + } + assertPortHeld(t, port) } func TestReaperClearsVerifiedStaleListenerWhenOwnerPIDWasReused(t *testing.T) { diff --git a/pkg/processgroup/registered.go b/pkg/processgroup/registered.go index 714be68ef..12c90958b 100644 --- a/pkg/processgroup/registered.go +++ b/pkg/processgroup/registered.go @@ -22,7 +22,7 @@ func PreviewManagedProcessGroups(ctx context.Context, scope Scope) ([]int, error } ownership, err := group.InspectOwnership(ctx) if err != nil { - return nil, err + return nil, fmt.Errorf("inspect registered group %d: %w", group.PGID(), err) } if !ownership.Authenticated { return nil, fmt.Errorf("group %d ownership is unproven", group.PGID()) From 6a2d69e9e371a8724c5ecfda90b0fb567dcce68a Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 01:38:11 -0400 Subject: [PATCH 13/18] fix(deps): adopt published Core v0.19.10 process inspection --- docs/supported-matrix.md | 4 +++- go.mod | 2 +- go.sum | 4 ++-- pkg/conformance/matrix.json | 2 +- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/supported-matrix.md b/docs/supported-matrix.md index 7376428e1..c22e6b68f 100644 --- a/docs/supported-matrix.md +++ b/docs/supported-matrix.md @@ -27,7 +27,9 @@ When `go.mod` moves to a new core release line, update the `core` field in `go test ./pkg/conformance/...` before the full coverage and race gates. `TestMatrixCoreMatchesGoMod` refuses a stale declared version. -The current pin targets Core's published `v0.19.9` release line. The exact source +The current pin targets Core's published `v0.19.10` release line, including the +Linux process-inspection correction: a missing executable link while process +state remains is an inspection error, not successful proof of absence. The exact source revision is declared in `go.mod`; the CLI consumes Core's container parent attribution, release acquisition and explicit process-group/container recovery APIs. These source gates qualify the CLI diff --git a/go.mod b/go.mod index dd71d6284..62420b369 100644 --- a/go.mod +++ b/go.mod @@ -16,7 +16,7 @@ require ( github.com/codefly-dev/authority/libraries/delivery/go v0.0.0-20261026023425-000d44495d93 github.com/codefly-dev/authority/libraries/fixture-composition/go v0.0.0-20261027171336-5178f01696e3 github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad7224855 - github.com/codefly-dev/core v0.19.9 + github.com/codefly-dev/core v0.19.10 github.com/codefly-dev/golor v0.1.3 github.com/codefly-dev/llm v0.1.7 github.com/codefly-dev/sdk-go/workcontext v0.1.1-0.20261018052022-70dc7a546458 diff --git a/go.sum b/go.sum index 628170996..f118e5da5 100644 --- a/go.sum +++ b/go.sum @@ -117,8 +117,8 @@ github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad722 github.com/codefly-dev/authority/libraries/hop/go v0.0.0-20261026175541-3afad7224855/go.mod h1:rfn41uiORQ36RrOeFEjXHPLC9NUi0j14AoYJtcovsn4= github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-000d44495d93 h1:4EygH7/J0E6aM7hUPdbQGfit1rsLUNCtWJ8kMh2b9PE= github.com/codefly-dev/authority/libraries/resources/go v0.0.0-20261026023425-000d44495d93/go.mod h1:1rmjrVyr0h7zOuwOHS9i+BoeS9dr3XNvtupLZFwY6As= -github.com/codefly-dev/core v0.19.9 h1:lTDsRJTceSsHLTmVttxFzNM3rEjuKsT8w5NmwozLu3Y= -github.com/codefly-dev/core v0.19.9/go.mod h1:aTJY3Rw0OSVkbWRThgDzSkqieoF9mW4MiPRXM8Vzxww= +github.com/codefly-dev/core v0.19.10 h1:rTeh1vph0xGtpiCCt2Dl1AH2Rz9hzIYt1YGcbv6bLYE= +github.com/codefly-dev/core v0.19.10/go.mod h1:aTJY3Rw0OSVkbWRThgDzSkqieoF9mW4MiPRXM8Vzxww= github.com/codefly-dev/golor v0.1.3 h1:xmo+ceyJFRYZdvpWE2fNd0jeaadp/Ibm1BnganiGKOc= github.com/codefly-dev/golor v0.1.3/go.mod h1:sl/u/K1l7J0Pr3xyVZp8fOJYQItKKst1No9JqgzLLoY= github.com/codefly-dev/gortk v0.2.0 h1:7bOlS5valYz2zil+fZctQNcPCYBcPj86abcw9N8h1hQ= diff --git a/pkg/conformance/matrix.json b/pkg/conformance/matrix.json index 59a57b66a..2b3cd329f 100644 --- a/pkg/conformance/matrix.json +++ b/pkg/conformance/matrix.json @@ -1,7 +1,7 @@ { "schema_version": 1, "cli": "source", - "core": "v0.19.9", + "core": "v0.19.10", "rows": [ { "id": "linux-amd64-source", From af90f9d53329ccf638ef6434696f267d0e27ff74 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 09:08:21 -0400 Subject: [PATCH 14/18] fix(ci): isolate disposable recipe image outputs --- cmd/ci/build.go | 3 + cmd/ci/common.go | 6 + cmd/ci/disposable_images.go | 46 ++ cmd/ci/disposable_images_test.go | 44 ++ cmd/ci/run.go | 5 + cmd/ci/scheduler_flags.go | 1 + cmd/ci/test.go | 5 +- cmd/validation/service.go | 3 + docs/agent-ci-port-isolation.md | 26 ++ docs/cli-reference.md | 4 +- docs/commands.md | 14 + pkg/orchestration/build_plan.go | 50 ++- pkg/orchestration/builder.go | 25 +- pkg/orchestration/disposable_images.go | 167 ++++++++ pkg/orchestration/disposable_images_test.go | 450 ++++++++++++++++++++ pkg/orchestration/flow.go | 11 + pkg/orchestration/local_runtime_image.go | 3 + 17 files changed, 847 insertions(+), 16 deletions(-) create mode 100644 cmd/ci/disposable_images.go create mode 100644 cmd/ci/disposable_images_test.go create mode 100644 pkg/orchestration/disposable_images.go create mode 100644 pkg/orchestration/disposable_images_test.go diff --git a/cmd/ci/build.go b/cmd/ci/build.go index a7ca93bf3..d565bee2e 100644 --- a/cmd/ci/build.go +++ b/cmd/ci/build.go @@ -94,6 +94,9 @@ func initBuildService(ctx context.Context, workspace *resources.Workspace, modul flow.WithBuildCache(cache) flow.WithRebuild(buildCacheFlags.Rebuild()) flow.WithImageSBOM(ciImageSBOM) + if disposableRuntime { + flow.WithDisposableImages() + } flow.WithOutputSink(cli.NewOutputSink()) flow.WithLoadOnly(loadOnly) flow.WithInitOnly(initOnly) diff --git a/cmd/ci/common.go b/cmd/ci/common.go index d39d52b77..f6c87407d 100644 --- a/cmd/ci/common.go +++ b/cmd/ci/common.go @@ -179,6 +179,7 @@ type ScheduleOptions struct { Jobs int FailFast bool LockDependencyClosure bool + DisposableImages bool Phase string Suite string RuntimeContext string @@ -470,6 +471,11 @@ func resolveScheduledTasks(ctx context.Context, workspace *resources.Workspace, if err != nil { return nil, fmt.Errorf("load CI scheduler dependency graph: %w", err) } + if options.DisposableImages { + if err = checkDisposablePlan(ctx, workspace, plan); err != nil { + return nil, err + } + } stage := scheduleStage(options) runtimeDependencies := dependencies dependencies, err = dependencies.ForStage(stage) diff --git a/cmd/ci/disposable_images.go b/cmd/ci/disposable_images.go new file mode 100644 index 000000000..851703461 --- /dev/null +++ b/cmd/ci/disposable_images.go @@ -0,0 +1,46 @@ +package ci + +import ( + "context" + + "github.com/codefly-dev/cli/pkg/orchestration" + "github.com/codefly-dev/core/resources" +) + +// Admit the entire selected request before dispatching any phase or job. Use +// normal flow selection for configuration-reference participants and environment +// exclusions; this constructs no agent managers. Each actual flow checks again. +func checkDisposablePlan(ctx context.Context, workspace *resources.Workspace, plan *Plan) error { + cache, err := buildCacheFlags.Policy() + if err != nil { + return err + } + for _, selected := range plan.Services { + ref, err := resources.ParseServiceWithOptionalModule(selected.Service) + if err != nil { + return err + } + module, err := workspace.LoadModuleFromName(ctx, ref.Module) + if err != nil { + return err + } + service, err := module.LoadServiceFromName(ctx, ref.Name) + if err != nil { + return err + } + env, err := orchestration.SelectEnvironment(workspace, orchestration.LocalEnvironmentName) + if err != nil { + return err + } + flow, err := orchestration.NewFlow(ctx, workspace, module, service, env, orchestration.TestMode) + if err != nil { + return err + } + flow.WithDisposableImages() + flow.WithBuildCache(cache) + if err := flow.CheckDisposableImageInputs(ctx); err != nil { + return err + } + } + return nil +} diff --git a/cmd/ci/disposable_images_test.go b/cmd/ci/disposable_images_test.go new file mode 100644 index 000000000..868e70dd3 --- /dev/null +++ b/cmd/ci/disposable_images_test.go @@ -0,0 +1,44 @@ +package ci + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/codefly-dev/core/resources" + "github.com/stretchr/testify/require" +) + +func TestDisposableCIRefusesPrerequisitesBeforeAnyScheduledAction(t *testing.T) { + root, workspace := loadSchedulerFixture(t) + path := filepath.Join(root, "modules/web/services/frontend/service.codefly.yaml") + raw, err := os.ReadFile(path) + require.NoError(t, err) + require.NoError(t, os.WriteFile(path, []byte(strings.Replace(string(raw), "- name: gateway", "- name: organization\n module: management\n kind: build", 1)), 0600)) + plan := &Plan{Services: []PlannedService{{Service: "management/organization"}, {Service: "web/frontend"}}} + for _, phase := range []string{"lint", "compile", "test", "build"} { + t.Run(phase, func(t *testing.T) { + calls := 0 + err := WithPlanOptions(t.Context(), workspace, plan, func(context.Context, *resources.Workspace, *resources.Module, *resources.Service) error { + calls++ + return nil + }, ScheduleOptions{Jobs: 2, Phase: phase, DisposableImages: true}) + require.ErrorContains(t, err, "disposable image input rebinding is unsupported") + require.Zero(t, calls, "even the producer must not run before rejection") + }) + } +} + +func TestDisposableCISchedulesIndependentServiceWithoutUnrelatedRefusal(t *testing.T) { + _, workspace := loadSchedulerFixture(t) + plan := &Plan{Services: []PlannedService{{Service: "management/worker"}}} + calls := 0 + err := WithPlanOptions(t.Context(), workspace, plan, func(context.Context, *resources.Workspace, *resources.Module, *resources.Service) error { + calls++ + return nil + }, ScheduleOptions{Phase: "build", DisposableImages: true}) + require.NoError(t, err) + require.Equal(t, 1, calls) +} diff --git a/cmd/ci/run.go b/cmd/ci/run.go index f23e2bf5b..a7bd5d172 100644 --- a/cmd/ci/run.go +++ b/cmd/ci/run.go @@ -65,6 +65,11 @@ var RunCmd = &cobra.Command{ return err } return runWithCIReport(ctx, workspace, plan, "codefly ci run", func(reporter *Reporter) error { + if disposableRuntime { + if err := checkDisposablePlan(ctx, workspace, plan); err != nil { + return err + } + } if err := validateAgentVersions(ctx, workspace, plan); err != nil { return err } diff --git a/cmd/ci/scheduler_flags.go b/cmd/ci/scheduler_flags.go index 83db627b8..2f1659488 100644 --- a/cmd/ci/scheduler_flags.go +++ b/cmd/ci/scheduler_flags.go @@ -9,6 +9,7 @@ func bindSchedulingFlags(command *cobra.Command) { func commandScheduleOptions(lockDependencyClosure bool, phase, suite string, reporter *Reporter) ScheduleOptions { return ScheduleOptions{ + DisposableImages: disposableRuntime, Jobs: ciJobs, FailFast: ciFailFast, LockDependencyClosure: lockDependencyClosure, diff --git a/cmd/ci/test.go b/cmd/ci/test.go index 37baeadc6..9c29c1e62 100644 --- a/cmd/ci/test.go +++ b/cmd/ci/test.go @@ -23,7 +23,7 @@ var testSelection SelectionFlags // Ordinary CI keeps the normal Stop semantics of stateful service agents. var disposableRuntime bool -const disposableRuntimeUsage = "Give each validation/test flow a fresh resource scope and destroy its owned runtime resources afterward (for disposable fixtures only)" +const disposableRuntimeUsage = "Isolate runtime resources and host recipe outputs; destroy owned runtime resources but retain images (independent single-recipe fixtures only)" type disposableTestFlow struct{ *orchestration.Flow } @@ -122,6 +122,9 @@ func initTestService(ctx context.Context, workspace *resources.Workspace, module if err != nil { return nil, w.Wrap(err) } + if disposableRuntime { + flow.WithDisposableImages() + } flow.WithOutputSink(cli.NewOutputSink()) flow.WithLoadOnly(loadOnly) flow.WithInitOnly(initOnly) diff --git a/cmd/validation/service.go b/cmd/validation/service.go index 0edb4201b..c8a7eea05 100644 --- a/cmd/validation/service.go +++ b/cmd/validation/service.go @@ -126,6 +126,9 @@ func RunServiceWithOptions( if err != nil { return w.Wrap(err) } + if options.Disposable { + flow.WithDisposableImages() + } flow.WithOutputSink(cli.NewOutputSink()) defer func() { if stopErr := flow.Stop(); stopErr != nil { diff --git a/docs/agent-ci-port-isolation.md b/docs/agent-ci-port-isolation.md index 8293a5bac..af125ce65 100644 --- a/docs/agent-ci-port-isolation.md +++ b/docs/agent-ci-port-isolation.md @@ -77,6 +77,32 @@ running development container; disposable tests explicitly authorize Destroy for their own fresh scope. Neither a prior run nor a retained fixture scope is a cleanup target. Both teardown errors remain part of the gate result. +Disposable flows also isolate **host-executed image recipe outputs**. Artifact +builds and runtime initialization during lint, compile and test use a fresh +flow-owned image namespace, including their runtime dependencies. The actual +destination participates in input identity and cache admission. Runtime images +retain platform selection, input-mutation checks and canonical digest handoff; +image SBOM subjects name the actual output and loaded image ID. Built images +are retained: this policy does not authorize image deletion or restoration. + +This mode currently supports independent single-recipe services with pure +runtime dependencies. Selected build-stage prerequisites (including schema and +legacy dependencies), multiple recipes in one plan, unavailable input identity, +push and registry cache exports are **errors**, not successful skips. Admission +checks the selected roots and their runtime closure before scheduling or loading +managers; a runtime dependency's build prerequisite also refuses. Verified plan +counts are checked before prefetch or host builder provisioning and again before +execution. Read-only cache imports remain supported. Ordinary unscoped CI keeps +its existing prerequisite ordering and image names. + +There is no producer-image input rebinding contract: renaming a producer output +must not let a dependent recipe silently consume an old shared image. Do not +relabel dependencies or alter fixtures to evade this restriction. A successful +input fingerprint does not prove independence: undeclared image inputs, mutable +network downloads, pulled bases and legacy agent-owned effects remain outside +this host-output guarantee. This is not engine-wide isolation or general +qualification of every copied conformance fixture. + Audit and SBOM phases also close their own service-scoped builder connection after use, including load failures. A later build or runtime flow must start an agent under its own recovery scope, not reuse a static builder that inherited diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 95ed6fe61..4c0df52ed 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -1028,7 +1028,7 @@ Flags: --cache-scope string Stable workspace and trust-domain cache scope; service, recipe and platform are added automatically --cache-to stringArray Registry cache repository to publish (repeatable; omit for read-only builds) --changed-file strings Changed path supplied by the CI provider (repeatable; bypasses Git discovery) - --disposable Give each validation/test flow a fresh resource scope and destroy its owned runtime resources afterward (for disposable fixtures only) + --disposable Isolate runtime resources and host recipe outputs; destroy owned runtime resources but retain images (independent single-recipe fixtures only) --fail-fast Stop native tests and service scheduling after the first failure (default true) --fail-on-vuln Fail audit on HIGH or CRITICAL vulnerabilities (default true) --format string Final CI report format: text or json (default "text") @@ -1069,7 +1069,7 @@ Flags: --all Select every service explicitly --base string Base Git revision for affected-service discovery --changed-file strings Changed path supplied by the CI provider (repeatable; bypasses Git discovery) - --disposable Give each validation/test flow a fresh resource scope and destroy its owned runtime resources afterward (for disposable fixtures only) + --disposable Isolate runtime resources and host recipe outputs; destroy owned runtime resources but retain images (independent single-recipe fixtures only) --fail-fast Stop native tests and service scheduling after the first failure (default true) --format string Final CI report format: text or json (default "text") --head string Head Git revision (defaults to HEAD when --base is set) diff --git a/docs/commands.md b/docs/commands.md index 07cdf4a55..543eccbc3 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -4085,6 +4085,20 @@ default gate is `verify`, `sync-drift`, `lint`, `compile`, `test`, `audit`, `sbom`, and `build`; reports retain typed integrity, drift, audit, and artifact evidence. +`ci run --disposable` additionally scopes host-built image outputs in artifact +builds and lint/compile/test runtime initialization, including runtime +dependencies. It currently requires independent single-recipe services: +selected build-stage prerequisites (including schema and legacy edges), +multi-recipe plans and unknown image input identity fail explicitly. The +selected runtime closure is checked too, so a build prerequisite of a runtime +dependency cannot bypass admission. This limitation does not change ordinary +unscoped prerequisite ordering described above. `ci test --disposable` uses the +same runtime image policy. Push and registry cache exports (`--cache-to`) are +refused; read-only cache imports are allowed. Scoped images remain retained; +runtime teardown does not grant image-cleanup authority. Hidden/undeclared +image inputs and legacy agent-owned effects are not qualified by this policy. +See [disposable invocation limits](agent-ci-port-isolation.md#disposable-invocations). + `codefly ci build` and the build phase of `codefly ci run` accept `--image-sbom`, which requires a digest-bound CycloneDX inventory for every image the build produces. Each document is written under `sbom/image/` and named diff --git a/pkg/orchestration/build_plan.go b/pkg/orchestration/build_plan.go index 743099929..7b1665d9a 100644 --- a/pkg/orchestration/build_plan.go +++ b/pkg/orchestration/build_plan.go @@ -55,9 +55,15 @@ const ( // builder's host architecture. func (b *Builder) buildFromPlan(ctx context.Context, outputDir string, plan *builderv0.DockerBuildPlan, prefetched map[string]map[string]string, reuse map[string]imageReuse) error { w := wool.Get(ctx).In("Builder.buildFromPlan", wool.ThisField(b.instance)) + if err := b.world.checkDisposableImagePolicy(); err != nil { + return err + } if err := coreservices.VerifyDockerBuildPlan(outputDir, plan); err != nil { return w.Wrapf(err, "cannot verify build recipe for %s", b.instance.Unique()) } + if err := b.checkDisposableImagePlan(plan); err != nil { + return err + } recipes := plan.GetRecipes() if len(recipes) == 0 { return w.NewError("build plan for %s contains no recipes", b.instance.Unique()) @@ -147,6 +153,9 @@ func (b *Builder) decideImageReuse( return imageReuse{}, err } identity, identityErr := imageBuildIdentity(ctx, recipe, plan, scope, prepared.Dockerfile, probeBuilder, identityArgs, private.Proxies, mirrors) + if err = b.checkDisposableImageIdentity(identity, identityErr); err != nil { + return imageReuse{}, err + } if identityErr != nil { // Not a build failure: an input this cannot account for means the image // is rebuilt, which is the behaviour without a cache at all. It is said @@ -156,16 +165,10 @@ func (b *Builder) decideImageReuse( wool.Field("recipe", recipe.GetName()), wool.ErrField(identityErr)) } decision := imageReuse{identity: identity} - // A local runtime image claims particular inputs, so it is named by them: - // an input that cannot be bound leaves it nothing to claim (local_runtime_image.go). - if b.localRuntime { - name, nameErr := localRuntimeImageName(recipe, identity, identityErr) - if nameErr != nil { - return decision, w.Wrap(nameErr) - } - recipe = proto.CloneOf(recipe) - recipe.Image = name - decision.image = name + // Scoped naming precedes identity; ordinary runtime naming derives from it. + recipe, decision.image, err = b.imageForExecution(recipe, identity, identityErr) + if err != nil { + return decision, err } // A build under a workspace agent override is not the build the release // names, so it is pushed under a tag of its own (override_build.go). @@ -226,7 +229,7 @@ func (b *Builder) decideImageReuse( } entry, found := b.imageBuildCache().lookup(identity.Key, b.instance.Unique(), shouldPush) switch { - case found && b.localRuntime && entry.Image != recipe.GetImage(): + case found && b.cacheImageReferenceMismatch(entry, recipe): // The local tag is the claim about the inputs; an entry recorded under // another name cannot stand in for it. w.Debug("the recorded image is not named by these inputs; building", @@ -307,6 +310,25 @@ func placeholderProxies(recipe *builderv0.DockerBuildRecipe) map[string]string { // the build will actually run and probes the registry the image is pushed to; // the manifests keep the name the cell pulls, with the pushed digest. func (b *Builder) recipeAsBuilt(recipe *builderv0.DockerBuildRecipe, shouldPush bool) (*builderv0.DockerBuildRecipe, error) { + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } + if b.world.disposableImageScope != "" { + if shouldPush { + return nil, fmt.Errorf("disposable images do not support push") + } + if built := b.executionRecipes[recipe.GetName()]; built != nil { + return built, nil + } + if b.localRuntime { + var err error + recipe, err = b.localRuntimeRecipe(recipe) + if err != nil { + return nil, err + } + } + return b.scopedRecipe(recipe), nil + } if !shouldPush { if b.localRuntime { return b.localRuntimeRecipe(recipe) @@ -415,6 +437,9 @@ func (b *Builder) buildRecipe( } reuse = &decided } + if err = b.checkDisposableImageDecision(recipe, reuse); err != nil { + return err + } if reuse.image != "" { recipe = proto.CloneOf(recipe) recipe.Image = reuse.image @@ -560,6 +585,9 @@ func (b *Builder) selectImageBuilder(ctx context.Context, multiArch bool, mirror return name, nil } if b.world.BuildCache != nil { + if b.world.disposableImageScope != "" { + return ensureBuildxBuilder(ctx, mirrors) + } return buildxBuilderFor(mirrors), nil } if !multiArch && len(mirrors) == 0 { diff --git a/pkg/orchestration/builder.go b/pkg/orchestration/builder.go index 941aaaf7f..47ed0193e 100644 --- a/pkg/orchestration/builder.go +++ b/pkg/orchestration/builder.go @@ -51,6 +51,8 @@ type Builder struct { // localRuntimeBuilt is the recipe a local runtime build ran, as built: named // by its inputs and narrowed to the engine's platform (recipeAsBuilt). localRuntimeBuilt *builderv0.DockerBuildRecipe + // Detached scoped execution projections; producer plans remain unchanged. + executionRecipes map[string]*builderv0.DockerBuildRecipe imageEvidence []*builderv0.ImageSBOM resolvedImages []coresbom.ResolvedImage @@ -120,6 +122,9 @@ func (b *Builder) ImageEvidence() []*builderv0.ImageSBOM { } func (b *Builder) Load(ctx context.Context) (*OutputProperty, error) { + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } w := wool.Get(ctx).In("service.NewBuilder", wool.ThisField(b.instance)) b.instance.Builder.Workspace = b.world.Workspace @@ -157,6 +162,9 @@ func (b *Builder) Load(ctx context.Context) (*OutputProperty, error) { } func (b *Builder) Init(ctx context.Context) (*OutputProperty, error) { + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } w := wool.Get(ctx).In("Builder", wool.ThisField(b.instance)) w.Debug("Init") @@ -256,6 +264,9 @@ func (b *Builder) SyncSkipped() bool { } func (b *Builder) dockerBuildContext(ctx context.Context) (*builderv0.DockerBuildContext, error) { + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } dockerContext, err := builder.DockerBuildContext(ctx, b.world.Workspace) if err != nil { return nil, err @@ -266,7 +277,7 @@ func (b *Builder) dockerBuildContext(ctx context.Context) (*builderv0.DockerBuil dockerContext.Cache = scopedBuildCache(b.world.BuildCache, b.world.Workspace.Name, b.instance.Unique(), "") dockerContext.BuildxBuilder = b.world.BuildxBuilder mirrors := registryMirrors(b.world.Env) - if dockerContext.BuildxBuilder == "" && (dockerContext.Cache != nil || len(mirrors) > 0) { + if b.world.disposableImageScope == "" && dockerContext.BuildxBuilder == "" && (dockerContext.Cache != nil || len(mirrors) > 0) { if dockerContext.BuildxBuilder, err = ensureBuildxBuilder(ctx, mirrors); err != nil { return nil, fmt.Errorf("cannot provision buildx builder for %s: %w", b.instance.Unique(), err) } @@ -317,6 +328,9 @@ type plannedBuild struct { func (b *Builder) Plan(ctx context.Context) (*OutputProperty, error) { w := wool.Get(ctx).In("Builder", wool.ThisField(b.instance)) w.Debug("Plan") + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } if b.planned != nil { return OnInit(), nil } @@ -324,6 +338,7 @@ func (b *Builder) Plan(ctx context.Context) (*OutputProperty, error) { return nil, w.NewError("cannot build deployable artifact for %s: agent explicitly advertises artifact build as unsupported", b.instance.Unique()) } + b.resetExecutionProjection() dockerContext, err := b.dockerBuildContext(ctx) if err != nil { return nil, w.Wrapf(err, "cannot create build context") @@ -369,6 +384,9 @@ func (b *Builder) Plan(ctx context.Context) (*OutputProperty, error) { if err := coreservices.VerifyDockerBuildPlan(outputDir, plan); err != nil { return nil, w.Wrapf(err, "cannot verify build recipe for %s", b.instance.Unique()) } + if err := b.checkDisposableImagePlan(plan); err != nil { + return nil, err + } contextRoot := recipeContextRoot(b.instance.Service.Dir(), outputDir, plan) for _, recipe := range plan.GetRecipes() { if len(recipe.GetGoModuleDownloads()) == 0 && len(recipe.GetGitSourceDownloads()) == 0 { @@ -443,6 +461,9 @@ func (b *Builder) Plan(ctx context.Context) (*OutputProperty, error) { func (b *Builder) Build(ctx context.Context) (*OutputProperty, error) { w := wool.Get(ctx).In("Builder", wool.ThisField(b.instance)) w.Debug("Build") + if err := b.world.checkDisposableImagePolicy(); err != nil { + return nil, err + } if b.planned == nil { if _, err := b.Plan(ctx); err != nil { return nil, err @@ -521,7 +542,7 @@ func (b *Builder) Build(ctx context.Context) (*OutputProperty, error) { // built, and on a pushed build already published, so failing the evidence // gate must not also leave the recipe that produced it unrecorded. if b.world.CollectImageSBOM { - if err = b.collectImageEvidence(ctx, plan, buildResult); err != nil { + if err = b.collectImageEvidence(ctx, b.imageEvidencePlan(plan), buildResult); err != nil { return nil, err } } diff --git a/pkg/orchestration/disposable_images.go b/pkg/orchestration/disposable_images.go new file mode 100644 index 000000000..ec766d9ee --- /dev/null +++ b/pkg/orchestration/disposable_images.go @@ -0,0 +1,167 @@ +package orchestration + +import ( + "context" + "crypto/sha256" + "fmt" + + "github.com/codefly-dev/core/architecture" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + "github.com/codefly-dev/core/resources" + "google.golang.org/protobuf/proto" +) + +// WithDisposableImages isolates host-executed recipe outputs for this flow. +// It does not authorize image deletion or imply that image inputs are isolated. +func (flow *Flow) WithDisposableImages() { + if flow.world.disposableImageScope == "" { + flow.world.disposableImageScope = NewInvocationID() + } +} + +func (world *World) checkDisposableImagePolicy() error { + if world.disposableImageScope != "" && (world.Push || len(world.BuildCache.GetExports()) > 0) { + return fmt.Errorf("disposable images do not support push or build-cache exports") + } + return nil +} + +// CheckDisposableImageInputs admits the selected graph without loading agents. +// CI uses it before scheduling; InitManagers checks its final selected graph too. +func (flow *Flow) CheckDisposableImageInputs(ctx context.Context) error { + if flow.world.disposableImageScope == "" { + return nil + } + if err := flow.world.checkDisposableImagePolicy(); err != nil { + return err + } + _, options, err := flow.dependencyGraphOptions(ctx) + if err != nil { + return err + } + dependencies := flow.world.Dependencies + if len(options) > 0 { + dependencies, err = architecture.NewServiceDependencies(ctx, flow.dependencyWorkspace(), options...) + if err != nil { + return err + } + } + return checkDisposableImageDependencies(ctx, dependencies, flow.rootUniques()) +} + +func checkDisposableImageDependencies(ctx context.Context, dependencies *architecture.ServiceDependencies, roots []string) error { + run, err := dependencies.ForStage(resources.StageRun) + if err != nil { + return err + } + build, err := dependencies.ForStage(resources.StageBuild) + if err != nil { + return err + } + seen := map[string]bool{} + for _, root := range roots { + participants, err := run.OrderTo(ctx, root) + if err != nil { + return err + } + participants = append(participants, architecture.Service{Unique: root}) + for _, participant := range participants { + if seen[participant.Unique] { + continue + } + seen[participant.Unique] = true + required, err := build.OrderTo(ctx, participant.Unique) + if err != nil { + return err + } + if len(required) > 0 { + return fmt.Errorf("disposable image input rebinding is unsupported: %s has build prerequisite %s", participant.Unique, required[0].Unique) + } + } + } + return nil +} + +func (b *Builder) checkDisposableImagePlan(plan *builderv0.DockerBuildPlan) error { + if err := b.world.checkDisposableImagePolicy(); err != nil { + return err + } + if b.world.disposableImageScope != "" && len(plan.GetRecipes()) > 1 { + return fmt.Errorf("disposable image input rebinding is unsupported: %s has %d recipes", b.instance.Unique(), len(plan.GetRecipes())) + } + return nil +} + +// scopedRecipe retains one execution projection before identity is computed. +// The digest below only makes a name-safe service/recipe discriminator; it is +// not an input identity or an ownership receipt. +func (b *Builder) scopedRecipe(recipe *builderv0.DockerBuildRecipe) *builderv0.DockerBuildRecipe { + if b.executionRecipes == nil { + b.executionRecipes = map[string]*builderv0.DockerBuildRecipe{} + } + if existing := b.executionRecipes[recipe.GetName()]; existing != nil { + return existing + } + execution := proto.CloneOf(recipe) + discriminator := sha256.Sum256([]byte(b.instance.Unique() + "\x00" + recipe.GetName())) + execution.Image = fmt.Sprintf("codefly-local/disposable/%s:%x", b.world.disposableImageScope, discriminator) + b.executionRecipes[recipe.GetName()] = execution + return execution +} + +func (b *Builder) imageEvidencePlan(plan *builderv0.DockerBuildPlan) *builderv0.DockerBuildPlan { + if plan == nil || b.world.disposableImageScope == "" { + return plan + } + execution := proto.CloneOf(plan) + for i, recipe := range execution.Recipes { + if built := b.executionRecipes[recipe.GetName()]; built != nil { + execution.Recipes[i] = proto.CloneOf(built) + } + } + return execution +} + +func (b *Builder) resetExecutionProjection() { + b.executionRecipes = nil + if b.world.disposableImageScope != "" { + b.resolvedImages = nil + } +} + +func (b *Builder) checkDisposableImageIdentity(identity imageIdentity, cause error) error { + if b.world.disposableImageScope != "" && (cause != nil || identity.Key == "") { + return fmt.Errorf("cannot identify disposable image inputs for %s: %v", b.instance.Unique(), cause) + } + return nil +} + +func (b *Builder) imageForExecution(recipe *builderv0.DockerBuildRecipe, identity imageIdentity, identityErr error) (*builderv0.DockerBuildRecipe, string, error) { + if b.world.disposableImageScope != "" { + return recipe, recipe.GetImage(), nil + } + if !b.localRuntime { + return recipe, "", nil + } + name, err := localRuntimeImageName(recipe, identity, identityErr) + if err != nil { + return nil, "", err + } + execution := proto.CloneOf(recipe) + execution.Image = name + return execution, name, nil +} + +func (b *Builder) cacheImageReferenceMismatch(entry *imageCacheEntry, recipe *builderv0.DockerBuildRecipe) bool { + return (b.localRuntime || b.world.disposableImageScope != "") && entry.Image != recipe.GetImage() +} + +func (b *Builder) checkDisposableImageDecision(recipe *builderv0.DockerBuildRecipe, reuse *imageReuse) error { + if b.world.disposableImageScope == "" { + return nil + } + if reuse.identity.Key == "" || reuse.image != recipe.GetImage() || (reuse.entry != nil && reuse.entry.Image != recipe.GetImage()) { + return fmt.Errorf("disposable image execution does not match its admitted identity and reference") + } + return nil +} diff --git a/pkg/orchestration/disposable_images_test.go b/pkg/orchestration/disposable_images_test.go new file mode 100644 index 000000000..bc69a7e84 --- /dev/null +++ b/pkg/orchestration/disposable_images_test.go @@ -0,0 +1,450 @@ +package orchestration + +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + + coreservices "github.com/codefly-dev/core/agents/services" + "github.com/codefly-dev/core/architecture" + agentv0 "github.com/codefly-dev/core/generated/go/codefly/services/agent/v0" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + "github.com/codefly-dev/core/resources" + "github.com/codefly-dev/core/services" + "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" + "google.golang.org/protobuf/proto" +) + +func disposableGraphWorkspace(t *testing.T, edges map[string]string) *resources.Workspace { + t.Helper() + files := map[string]string{"workspace.codefly.yaml": "name: disposable\nlayout: modules\nmodules:\n - name: app\n", "modules/app/module.codefly.yaml": "name: app\nservices:\n - name: origin\n - name: dependency\n - name: producer\n - name: unrelated\n"} + for _, name := range []string{"origin", "dependency", "producer", "unrelated"} { + files["modules/app/services/"+name+"/service.codefly.yaml"] = "name: " + name + "\nversion: 0.0.0\nagent:\n kind: codefly:service\n publisher: codefly.dev\n name: missing-disposable-peer\n version: 0.0.1\n" + edges[name] + } + return writeTempWorkspace(t, files) +} + +func disposableEdge(name, kind string) string { + s := "service-dependencies:\n - name: " + name + "\n" + if kind != "" { + s += " kind: " + kind + "\n" + } + return s +} + +func TestDisposableImagesAdmitSelectedRuntimeClosure(t *testing.T) { + for _, kind := range []string{"build", "schema", "", "runtime", "completion"} { + t.Run("kind="+kind, func(t *testing.T) { + ws := disposableGraphWorkspace(t, map[string]string{"origin": disposableEdge("dependency", "runtime"), "dependency": disposableEdge("producer", kind)}) + deps, err := architecture.NewServiceDependencies(t.Context(), ws) + require.NoError(t, err) + err = checkDisposableImageDependencies(t.Context(), deps, []string{"app/origin"}) + if kind == "runtime" || kind == "completion" { + require.NoError(t, err) + } else { + require.ErrorContains(t, err, "app/dependency has build prerequisite app/producer") + } + }) + } + ws := disposableGraphWorkspace(t, map[string]string{"unrelated": disposableEdge("producer", "build")}) + deps, err := architecture.NewServiceDependencies(t.Context(), ws) + require.NoError(t, err) + require.NoError(t, checkDisposableImageDependencies(t.Context(), deps, []string{"app/origin"})) + require.ErrorContains(t, checkDisposableImageDependencies(t.Context(), deps, []string{"app/origin", "app/unrelated"}), "build prerequisite") + module, err := ws.LoadModuleFromName(t.Context(), "app") + require.NoError(t, err) + service, err := module.LoadServiceFromName(t.Context(), "origin") + require.NoError(t, err) + env, err := SelectEnvironment(ws, LocalEnvironmentName) + require.NoError(t, err) + flow, err := NewFlow(t.Context(), ws, module, service, env, TestMode) + require.NoError(t, err) + flow.WithDisposableImages() + flow.coRoots = []string{"app/unrelated"} + require.ErrorContains(t, flow.InitManagers(t.Context()), "build prerequisite") + require.Nil(t, flow.hub, "refusal precedes even origin manager creation") +} + +// Uses a real loopback recipe peer. No Docker CLI reaches an engine: every +// invocation runs installDockerStub's isolated executable. +type disposableRecipePeer struct { + sbomRequests chan *builderv0.SBOMRequest + imageCachePeer + multiple bool + unknown bool + calls atomic.Int32 +} + +func (p *disposableRecipePeer) Build(ctx context.Context, req *builderv0.BuildRequest) (*builderv0.BuildResponse, error) { + p.calls.Add(1) + response, err := p.imageCachePeer.Build(ctx, req) + if err != nil { + return nil, err + } + plan := response.GetResult().GetDockerBuildPlan() + if p.multiple { + second := proto.CloneOf(plan.Recipes[0]) + second.Name = "second" + second.Image = "example.test/second:v1" + plan.Recipes = append(plan.Recipes, second) + } + if p.unknown { + if err := os.WriteFile(filepath.Join(req.OutputDirectory, "Dockerfile"), []byte("ARG BASE\nFROM ${BASE}\n"), 0600); err != nil { + return nil, err + } + } + plan, err = coreservices.BuildEmittedDockerBuildPlan(req.OutputDirectory, plan.Recipes, []string{"Dockerfile", "dockerignore"}) + if err != nil { + return nil, err + } + response.Result = &builderv0.BuildResult{Kind: &builderv0.BuildResult_DockerBuildPlan{DockerBuildPlan: plan}} + return response, nil +} + +func disposableBuilder(t *testing.T, p *disposableRecipePeer) (*Builder, *imageCacheHarness) { + t.Helper() + h := newImageCacheHarness(t, false, true) + b := localRuntimeBuilder(t, h) + server := grpc.NewServer() + builderv0.RegisterBuilderServer(server, p) + conn, err := grpc.NewClient(serveGRPC(t, server), grpc.WithTransportCredentials(insecure.NewCredentials())) + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + b.instance.Builder = &services.BuilderInstance{Instance: b.instance, Builder: &coreservices.BuilderAgent{BuilderClient: builderv0.NewBuilderClient(conn)}} + (&Flow{world: b.world}).WithDisposableImages() + return b, h +} + +func TestDisposableImagesRejectPublicationBeforeRPCAndRecheck(t *testing.T) { + for _, kind := range []string{"push", "cache export"} { + t.Run(kind, func(t *testing.T) { + p := &disposableRecipePeer{} + b, h := disposableBuilder(t, p) + reject := func() { + if kind == "push" { + b.world.Push = true + } else { + b.world.BuildCache = &builderv0.BuildCacheOptions{Exports: []string{"example.test/cache"}} + } + } + reject() + _, err := b.Plan(t.Context()) + require.ErrorContains(t, err, "push or build-cache exports") + require.Zero(t, p.calls.Load()) + require.Zero(t, h.docker.calls(t, "build")) + b.world.Push = false + b.world.BuildCache = nil + _, err = b.Plan(t.Context()) + require.NoError(t, err) + plan := b.planned + reject() + _, err = b.Build(t.Context()) + require.ErrorContains(t, err, "push or build-cache exports") + err = b.buildFromPlan(t.Context(), plan.outputDir, plan.response.GetResult().GetDockerBuildPlan(), nil, nil) + require.ErrorContains(t, err, "push or build-cache exports") + require.EqualValues(t, 1, p.calls.Load()) + require.Zero(t, h.docker.calls(t, "build")) + }) + } +} + +func TestDisposableImagesRejectMultiRecipeBeforePrefetchAndProvision(t *testing.T) { + p := &disposableRecipePeer{imageCachePeer: imageCachePeer{goModules: true}, multiple: true} + b, h := disposableBuilder(t, p) + b.world.BuildxBuilder = "" + b.world.BuildCache = &builderv0.BuildCacheOptions{Backend: "registry", Scope: "test", Imports: []string{"example.test/cache"}} + prefetch := newGoModulePrefetch() + var calls []goCall + prefetch.run = recordingGo(&calls, nil) + b.world.goModulePrefetch = prefetch + t.Cleanup(func() { require.NoError(t, prefetch.Close()) }) + _, err := b.Plan(t.Context()) + require.ErrorContains(t, err, "has 2 recipes") + require.Empty(t, calls) + require.NoFileExists(t, filepath.Join(h.docker.dir, "calls")) + output := t.TempDir() + response, err := p.Build(t.Context(), &builderv0.BuildRequest{OutputDirectory: output}) + require.NoError(t, err) + err = b.buildFromPlan(t.Context(), output, response.GetResult().GetDockerBuildPlan(), nil, nil) + require.ErrorContains(t, err, "has 2 recipes") + require.NoFileExists(t, filepath.Join(h.docker.dir, "calls")) +} + +func TestDisposableImagesRefuseUnknownIdentityInPlanAndBuild(t *testing.T) { + for _, prefetch := range []bool{false, true} { + t.Run(fmt.Sprint(prefetch), func(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{imageCachePeer: imageCachePeer{goModules: prefetch}, unknown: true}) + b.world.RebuildImages = true + _, err := b.Plan(t.Context()) + if !prefetch { + require.NoError(t, err) + _, err = b.Build(t.Context()) + } + require.ErrorContains(t, err, "cannot identify disposable image inputs") + require.Zero(t, h.docker.calls(t, "build")) + require.Zero(t, h.docker.calls(t, "imagetools")) + }) + } +} + +func executeDisposablePlan(t *testing.T, b *Builder) *builderv0.DockerBuildPlan { + t.Helper() + _, err := b.Plan(t.Context()) + require.NoError(t, err) + planned := b.planned + original := planned.response.GetResult().GetDockerBuildPlan() + before := proto.CloneOf(original) + require.NoError(t, b.buildFromPlan(t.Context(), planned.outputDir, original, planned.prefetched, planned.reuse)) + require.True(t, proto.Equal(before, original), "verified producer plan remains unchanged") + execution := b.imageEvidencePlan(original) + b.planned = nil + return execution +} + +func TestDisposableImagesProjectionCacheAndActualSubjects(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{}) + b.world.CollectImageSBOM = true + first := executeDisposablePlan(t, b) + image := first.Recipes[0].Image + require.Contains(t, image, "codefly-local/disposable/") + require.Equal(t, 1, h.docker.calls(t, "build")) + assertSubject := func(plan *builderv0.DockerBuildPlan) { + subjects, err := producedSubjects(b.instance.Unique(), plan, false, b.resolvedImages) + require.NoError(t, err) + require.Len(t, subjects, 1) + require.Equal(t, image, subjects[0].Reference) + require.Equal(t, first.Recipes[0].Platforms[0], subjects[0].Platform) + require.Equal(t, fmt.Sprintf("sha256:%064d", 7), subjects[0].Digest) + } + assertSubject(first) + key := h.key(t) + entry, found := b.imageBuildCache().lookup(key, b.instance.Unique(), false) + require.True(t, found) + require.Equal(t, image, entry.Image) + b.resolvedImages = nil + again := executeDisposablePlan(t, b) + require.Equal(t, image, again.Recipes[0].Image) + require.Equal(t, 1, h.docker.calls(t, "build")) + assertSubject(again) + // Same key and loaded ID cannot authorize a differently named cache entry. + entry.Image = "example.test/foreign:retained" + require.NoError(t, b.imageBuildCache().store(t.Context(), entry)) + executeDisposablePlan(t, b) + require.Equal(t, 2, h.docker.calls(t, "build")) + for _, args := range h.docker.buildArgs(t) { + require.Contains(t, args, "-t "+image) + require.NotContains(t, args, "-t example.test/") + } + previousScope := b.world.disposableImageScope + (&Flow{world: b.world}).WithDisposableImages() + require.Equal(t, previousScope, b.world.disposableImageScope) + b.world.disposableImageScope = NewInvocationID() + different := executeDisposablePlan(t, b) + require.NotEqual(t, image, different.Recipes[0].Image) + require.Equal(t, 3, h.docker.calls(t, "build")) +} + +func TestDisposableImagesRetainPlanProjectionBeforePrefetchIdentity(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{imageCachePeer: imageCachePeer{goModules: true}}) + prefetch := newGoModulePrefetch() + var calls []goCall + prefetch.run = recordingGo(&calls, nil) + b.world.goModulePrefetch = prefetch + t.Cleanup(func() { require.NoError(t, prefetch.Close()) }) + _, err := b.Plan(t.Context()) + require.NoError(t, err) + planned := b.planned + recipe := planned.response.GetResult().GetDockerBuildPlan().Recipes[0] + projection := b.executionRecipes[recipe.Name] + decision := planned.reuse[recipe.Name] + require.Equal(t, projection.Image, decision.image) + require.NotEmpty(t, decision.identity.Key) + require.NotEmpty(t, calls) + require.NoError(t, b.buildFromPlan(t.Context(), planned.outputDir, planned.response.GetResult().GetDockerBuildPlan(), planned.prefetched, planned.reuse)) + require.Same(t, projection, b.executionRecipes[recipe.Name]) + require.Contains(t, h.docker.buildArgs(t)[0], "-t "+projection.Image) +} + +func TestDisposableRuntimeImagesKeepCanonicalHandoffAndMutationRefusal(t *testing.T) { + runner, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + (&Flow{world: runner.world}).WithDisposableImages() + _, err := runner.Init(t.Context()) + require.NoError(t, err) + ref := exposedValues(t.Context(), t, runner.world)["runtime-image"] + require.Contains(t, ref, "docker.io/codefly-local/disposable/"+runner.world.disposableImageScope+"@sha256:") + _, err = runner.Init(t.Context()) + require.NoError(t, err) + require.Equal(t, 1, stub.calls(t, "build")) + // A second service in the same World cannot choose the origin destination. + b, err := NewBuilder(t.Context(), &services.Instance{Identity: &resources.ServiceIdentity{Module: "other", Name: "dependency"}}, runner.world) + require.NoError(t, err) + projection := b.scopedRecipe(&builderv0.DockerBuildRecipe{Name: "app", Image: "example.test/app:v1"}) + require.NotContains(t, stub.buildArgs(t)[0], "-t "+projection.Image) + h := newImageCacheHarness(t, false, true) + local := localRuntimeBuilder(t, h) + (&Flow{world: local.world}).WithDisposableImages() + h.docker.editDuringBuild(t, filepath.Join(h.serviceDir, "code/main.go"), "changed during build\n") + image, err := local.buildLocalRuntimeImage(t.Context()) + require.ErrorContains(t, err, "inputs changed while building") + require.Empty(t, image) +} + +func TestDisposableImagesProvisionReadOnlyCacheOnlyAfterAdmission(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{}) + b.world.BuildxBuilder = "" + b.world.BuildCache = &builderv0.BuildCacheOptions{Backend: "registry", Scope: "fixture", Imports: []string{"example.test/cache"}} + // Extend the isolated fake only; no real Docker executable is invoked. + path := filepath.Join(h.docker.dir, "docker") + script, err := os.ReadFile(path) + require.NoError(t, err) + script = []byte(strings.Replace(string(script), "case \"$1 $2\" in", `case "$1 $2" in +"buildx inspect") record provision-inspect; exit 1 ;; +"buildx create") record provision-create; exit 0 ;;`, 1)) + require.NoError(t, os.WriteFile(path, script, 0700)) + _, err = b.Plan(t.Context()) + require.NoError(t, err) + require.Zero(t, h.docker.calls(t, "provision-create")) + executeDisposablePlan(t, b) + require.Equal(t, 1, h.docker.calls(t, "provision-create")) + require.Equal(t, 1, h.docker.calls(t, "build")) + executeDisposablePlan(t, b) + require.Equal(t, 1, h.docker.calls(t, "provision-create")) + require.Equal(t, 1, h.docker.calls(t, "build")) +} + +func TestDisposableImagesRejectInvalidRetainedDecision(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{}) + _, err := b.Plan(t.Context()) + require.NoError(t, err) + p := b.planned + plan := p.response.GetResult().GetDockerBuildPlan() + err = b.buildFromPlan(t.Context(), p.outputDir, plan, nil, map[string]imageReuse{plan.Recipes[0].Name: {image: "example.test/shared:v1"}}) + require.ErrorContains(t, err, "does not match") + require.Zero(t, h.docker.calls(t, "build")) +} + +func TestDisposableImagesCheckConfigurationReferenceParticipants(t *testing.T) { + ws := overrideOrderingWorkspace(t) + path := filepath.Join(ws.Dir(), "modules/platform/services/authority/service.codefly.yaml") + file, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0600) + require.NoError(t, err) + _, err = file.WriteString("service-dependencies:\n - name: gateway\n kind: build\n") + require.NoError(t, err) + require.NoError(t, file.Close()) + module, err := ws.LoadModuleFromName(t.Context(), "payments") + require.NoError(t, err) + service, err := module.LoadServiceFromName(t.Context(), "worker") + require.NoError(t, err) + env, err := SelectEnvironment(ws, LocalEnvironmentName) + require.NoError(t, err) + flow, err := NewFlow(t.Context(), ws, module, service, env, TestMode) + require.NoError(t, err) + flow.WithDisposableImages() + require.ErrorContains(t, flow.InitManagers(t.Context()), "platform/authority has build prerequisite platform/gateway") + require.Nil(t, flow.hub) +} + +func TestDisposableImagesRespectSelectedCutoffs(t *testing.T) { + ws := disposableGraphWorkspace(t, map[string]string{"origin": disposableEdge("dependency", "runtime"), "dependency": disposableEdge("producer", "build")}) + deps, err := architecture.NewServiceDependencies(t.Context(), ws, architecture.SkipDependencyFor("app/dependency")) + require.NoError(t, err) + require.NoError(t, checkDisposableImageDependencies(t.Context(), deps, []string{"app/origin"})) + deps, err = architecture.NewServiceDependencies(t.Context(), ws, architecture.ExcludeServices("app/dependency")) + require.NoError(t, err) + require.NoError(t, checkDisposableImageDependencies(t.Context(), deps, []string{"app/origin"})) +} + +func TestDisposableRuntimePolicyReachesOriginAndDependencyInEveryMode(t *testing.T) { + for _, mode := range []Mode{LintMode, CompileMode, TestMode} { + t.Run(string(mode), func(t *testing.T) { + origin, _ := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + origin.world.Mode = mode + (&Flow{world: origin.world}).WithDisposableImages() + dependency, stub := runtimeImageRunner(t, &runtimeImageBuilderPeer{}) + dependency.instance.Identity.Name = "dependency" + dependency.instance.Service.Name = "dependency" + dependency.world = origin.world + // The second isolated Docker stub remains first on PATH; keep both runners + // on that same fake engine, exactly as their shared World does in a flow. + a, err := origin.localRuntimeImage(t.Context()) + require.NoError(t, err) + d, err := dependency.localRuntimeImage(t.Context()) + require.NoError(t, err) + require.Contains(t, a, "/disposable/"+origin.world.disposableImageScope+"@sha256:") + require.Contains(t, d, "/disposable/"+origin.world.disposableImageScope+"@sha256:") + args := stub.buildArgs(t) + require.Len(t, args, 2) + require.NotEqual(t, strings.Split(strings.Split(args[0], " -t ")[1], " ")[0], strings.Split(strings.Split(args[1], " -t ")[1], " ")[0]) + _, err = origin.localRuntimeImage(t.Context()) + require.NoError(t, err) + require.Equal(t, 2, stub.calls(t, "build")) + }) + } +} + +func (p *disposableRecipePeer) SBOM(_ context.Context, request *builderv0.SBOMRequest) (*builderv0.SBOMResponse, error) { + if p.sbomRequests != nil { + p.sbomRequests <- proto.CloneOf(request) + } + response := &builderv0.SBOMResponse{State: &builderv0.SBOMStatus{State: builderv0.SBOMStatus_COMPLETE}, Scope: builderv0.SBOMScope_SBOM_SCOPE_IMAGE} + for _, subject := range request.Subjects { + response.Images = append(response.Images, &builderv0.ImageSBOM{Digest: subject.Digest, Platform: subject.Platform, Subjects: []*builderv0.ImageSubject{proto.CloneOf(subject)}, Bom: &agentv0.Bom{Components: []*agentv0.Component{{Name: "test-only-inventory"}}}, Tool: "test-peer", Sha256: "test-only-checksum"}) + } + return response, nil +} + +func TestDisposableImagesActualSBOMRequestFreshAndReused(t *testing.T) { + peer := &disposableRecipePeer{sbomRequests: make(chan *builderv0.SBOMRequest, 2)} + b, h := disposableBuilder(t, peer) + b.world.CollectImageSBOM = true + var previous *builderv0.ImageSubject + for i := 0; i < 2; i++ { + _, err := b.Plan(t.Context()) + require.NoError(t, err) + original := b.planned.response.GetResult().GetDockerBuildPlan() + before := proto.CloneOf(original) + _, err = b.Build(t.Context()) + require.NoError(t, err) + require.True(t, proto.Equal(before, original)) + request := <-peer.sbomRequests + require.Len(t, request.Subjects, 1) + subject := request.Subjects[0] + require.Contains(t, subject.Reference, "codefly-local/disposable/"+b.world.disposableImageScope+":") + require.NotEqual(t, original.Recipes[0].Image, subject.Reference) + require.Equal(t, original.Recipes[0].Platforms[0], subject.Platform) + require.Equal(t, fmt.Sprintf("sha256:%064d", 7), subject.Digest) + require.Len(t, b.ImageEvidence(), 1) + if previous != nil { + require.True(t, proto.Equal(previous, subject)) + } + previous = subject + } + require.Equal(t, 1, h.docker.calls(t, "build"), "second SBOM covers an actual cache hit") +} + +func TestDisposableImagesRebuildMissingImageWithinScope(t *testing.T) { + b, h := disposableBuilder(t, &disposableRecipePeer{}) + first := executeDisposablePlan(t, b) + path := filepath.Join(h.docker.dir, "docker") + script, err := os.ReadFile(path) + require.NoError(t, err) + // One absent-image read makes the cache miss. Later inspection after the + // fake build succeeds, so the missing image cannot cause an original-tag fallback. + script = []byte(strings.Replace(string(script), `"image inspect")`, `"image inspect") + if [ -f "$STATE/missing" ]; then rm "$STATE/missing"; exit 1; fi`, 1)) + require.NoError(t, os.WriteFile(path, script, 0700)) + require.NoError(t, os.WriteFile(filepath.Join(h.docker.dir, "missing"), nil, 0600)) + again := executeDisposablePlan(t, b) + require.Equal(t, first.Recipes[0].Image, again.Recipes[0].Image) + require.Equal(t, 2, h.docker.calls(t, "build")) + b.world.RebuildImages = true + forced := executeDisposablePlan(t, b) + require.Equal(t, first.Recipes[0].Image, forced.Recipes[0].Image) + require.Equal(t, 3, h.docker.calls(t, "build")) +} diff --git a/pkg/orchestration/flow.go b/pkg/orchestration/flow.go index 1e979c7fa..e11c38a68 100644 --- a/pkg/orchestration/flow.go +++ b/pkg/orchestration/flow.go @@ -241,6 +241,9 @@ type RuntimeStartAttempt struct { } type World struct { + // Explicit disposable output policy, shared by all origin/dependency builders. + disposableImageScope string + // beforeSourceRestart validates the accepted route inventory before stopping a service. beforeSourceRestart func(context.Context) error @@ -1959,6 +1962,9 @@ func (flow *Flow) managerDependencies(ctx context.Context) ([]architecture.Servi func (flow *Flow) InitManagers(ctx context.Context) error { w := wool.Get(ctx).In("flow.InitManagers") + if err := flow.world.checkDisposableImagePolicy(); err != nil { + return err + } // Project ownership before anything is spawned, and hold it for every spawn // below: New() starts the agent process, which inherits the marker at exec // and can create containers from its Init onwards. @@ -1974,6 +1980,11 @@ func (flow *Flow) InitManagers(ctx context.Context) error { if err = flow.rebuildDependencyGraph(ctx, dependencyOptions); err != nil { return w.Wrap(err) } + if flow.world.disposableImageScope != "" { + if err = checkDisposableImageDependencies(ctx, flow.world.Dependencies, flow.rootUniques()); err != nil { + return err + } + } if err = flow.selectDependencyStage(); err != nil { return w.Wrap(err) } diff --git a/pkg/orchestration/local_runtime_image.go b/pkg/orchestration/local_runtime_image.go index 5b846aee8..b3d1d4e93 100644 --- a/pkg/orchestration/local_runtime_image.go +++ b/pkg/orchestration/local_runtime_image.go @@ -20,6 +20,9 @@ import ( // Stock-image agents may have a builder but emit no recipe. They keep their // own image selection; no agent-name or release-version table is involved. func (runner *Runner) localRuntimeImage(ctx context.Context) (string, error) { + if err := runner.world.checkDisposableImagePolicy(); err != nil { + return "", err + } if runner.remoteEnvironment != nil || runner.runtimeContext != resources.RuntimeContextContainer { return "", nil } From 3f9245114ba7f24d8b2200375f8488d8b58c6fce Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 10:26:31 -0400 Subject: [PATCH 15/18] fix(ci): admit replay and scheduled image prerequisites before dispatch --- cmd/ci/common.go | 10 +- cmd/ci/disposable_images.go | 22 +- cmd/ci/disposable_images_test.go | 111 +++++ cmd/ci/run.go | 2 +- docs/agent-ci-port-isolation.md | 6 +- .../protocoltest/testdata/peer/images.go | 73 ++++ .../protocoltest/testdata/peer/main.go | 7 + pkg/orchestration/disposable_commands_test.go | 193 +++++++++ pkg/orchestration/disposable_engine_test.go | 392 ++++++++++++++++++ pkg/orchestration/disposable_images.go | 42 +- 10 files changed, 835 insertions(+), 23 deletions(-) create mode 100644 pkg/internal/protocoltest/testdata/peer/images.go create mode 100644 pkg/orchestration/disposable_commands_test.go create mode 100644 pkg/orchestration/disposable_engine_test.go diff --git a/cmd/ci/common.go b/cmd/ci/common.go index f6c87407d..b98d61b82 100644 --- a/cmd/ci/common.go +++ b/cmd/ci/common.go @@ -471,11 +471,6 @@ func resolveScheduledTasks(ctx context.Context, workspace *resources.Workspace, if err != nil { return nil, fmt.Errorf("load CI scheduler dependency graph: %w", err) } - if options.DisposableImages { - if err = checkDisposablePlan(ctx, workspace, plan); err != nil { - return nil, err - } - } stage := scheduleStage(options) runtimeDependencies := dependencies dependencies, err = dependencies.ForStage(stage) @@ -630,6 +625,11 @@ func scheduleStage(options ScheduleOptions) resources.Stage { } func buildScheduledTasks(ctx context.Context, workspace *resources.Workspace, plan *Plan, options ScheduleOptions) ([]ciScheduledTask, error) { + if options.DisposableImages { + if err := checkDisposablePlan(ctx, workspace, plan, options.Phase); err != nil { + return nil, err + } + } if plan.replay == nil { return resolveScheduledTasks(ctx, workspace, plan, options) } diff --git a/cmd/ci/disposable_images.go b/cmd/ci/disposable_images.go index 851703461..27a7e34a2 100644 --- a/cmd/ci/disposable_images.go +++ b/cmd/ci/disposable_images.go @@ -2,6 +2,9 @@ package ci import ( "context" + "slices" + + "github.com/codefly-dev/core/architecture" "github.com/codefly-dev/cli/pkg/orchestration" "github.com/codefly-dev/core/resources" @@ -10,7 +13,24 @@ import ( // Admit the entire selected request before dispatching any phase or job. Use // normal flow selection for configuration-reference participants and environment // exclusions; this constructs no agent managers. Each actual flow checks again. -func checkDisposablePlan(ctx context.Context, workspace *resources.Workspace, plan *Plan) error { +func checkDisposablePlan(ctx context.Context, workspace *resources.Workspace, plan *Plan, phases ...string) error { + // Only build scheduling expands prerequisites into additional tasks. Its + // StageBuild closure is actual dispatched work, even if a Flow excludes a + // bridge. Do not traverse raw runtime branches that no Flow would start. + if slices.Contains(phases, string(resources.PhaseBuild)) { + dependencies, err := architecture.NewServiceDependencies(ctx, workspace) + if err != nil { + return err + } + roots := make([]string, 0, len(plan.Services)) + for _, selected := range plan.Services { + roots = append(roots, selected.Service) + } + if err := orchestration.CheckDisposableImageBuildPrerequisites(ctx, dependencies, roots); err != nil { + return err + } + } + cache, err := buildCacheFlags.Policy() if err != nil { return err diff --git a/cmd/ci/disposable_images_test.go b/cmd/ci/disposable_images_test.go index 868e70dd3..ddc0b5321 100644 --- a/cmd/ci/disposable_images_test.go +++ b/cmd/ci/disposable_images_test.go @@ -2,6 +2,7 @@ package ci import ( "context" + "encoding/json" "os" "path/filepath" "strings" @@ -42,3 +43,113 @@ func TestDisposableCISchedulesIndependentServiceWithoutUnrelatedRefusal(t *testi require.NoError(t, err) require.Equal(t, 1, calls) } + +// This fixture has no exported endpoints, so the withdrawal is independently +// valid. Its raw scheduler graph and selected Flow graph deliberately differ. +func disposableAdmissionWorkspace(t *testing.T, edge string, excluded bool) (string, *resources.Workspace) { + t.Helper() + root := t.TempDir() + workspace := "name: disposable-admission\nlayout: modules\nmodules:\n - name: app\nenvironments:\n - name: local\n" + if excluded { + workspace += " exclude-services:\n - service: app/intermediate\n withdrawn-from: [app/origin]\n" + } + files := map[string]string{ + "workspace.codefly.yaml": workspace, + "modules/app/module.codefly.yaml": "name: app\nservices:\n - name: independent\n - name: origin\n - name: intermediate\n - name: producer\n", + } + for _, name := range []string{"independent", "origin", "intermediate", "producer"} { + declaration := "name: " + name + "\nversion: 0.0.0\nagent:\n kind: codefly:service\n publisher: example.test\n name: missing-admission-peer\n version: 0.0.1\n" + if name == "origin" { + declaration += "service-dependencies:\n - name: intermediate\n kind: " + edge + "\n" + } + if name == "intermediate" { + declaration += "service-dependencies:\n - name: producer\n kind: build\n" + } + files["modules/app/services/"+name+"/service.codefly.yaml"] = declaration + } + for name, value := range files { + path := filepath.Join(root, name) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0700)) + require.NoError(t, os.WriteFile(path, []byte(value), 0600)) + } + ws, err := resources.LoadWorkspaceFromDir(t.Context(), root) + require.NoError(t, err) + return root, ws +} + +func validDisposableReplay(t *testing.T, root string, ws *resources.Workspace, phase string, changed []string) *Plan { + t.Helper() + runCacheTestGit(t, root, "init") + runCacheTestGit(t, root, "add", ".") + runCacheTestGit(t, root, "-c", "user.name=CI Test", "-c", "user.email=ci@example.com", "commit", "-m", "fixture") + options := PlanOptions{ChangedFiles: changed} + plan, err := BuildPlan(t.Context(), ws, &options) + require.NoError(t, err) + invocation := ReplayInvocation{Phases: []string{phase}, Suites: []string{"unit"}, RuntimeContext: "free"} + saved, err := buildReplayPlan(t.Context(), ws, plan, invocation) + require.NoError(t, err) + data, err := json.Marshal(saved) + require.NoError(t, err) + path := filepath.Join(t.TempDir(), "plan.json") + require.NoError(t, os.WriteFile(path, data, 0600)) + // Use the same selection entrypoint as ci test --plan, not a forged replay. + flags := SelectionFlags{changedFiles: changed, planFile: path} + actual, err := flags.BuildPlan(t.Context(), ws, invocation) + require.NoError(t, err) + require.NotNil(t, actual.replay) + return actual +} + +func TestDisposableReplayAdmissionBeforeAnyAction(t *testing.T) { + root, ws := disposableAdmissionWorkspace(t, "build", false) + plan := validDisposableReplay(t, root, ws, "test", []string{"modules/app/services/independent/code/input", "modules/app/services/origin/code/input"}) + for _, disposable := range []bool{false, true} { + calls := 0 + err := WithPlanOptions(t.Context(), ws, plan, func(context.Context, *resources.Workspace, *resources.Module, *resources.Service) error { + calls++ + return nil + }, ScheduleOptions{Phase: "test", Suite: "unit", Jobs: 1, DisposableImages: disposable}) + if disposable { + require.ErrorContains(t, err, "disposable image input rebinding is unsupported") + require.Zero(t, calls) + } else { + require.NoError(t, err) + require.GreaterOrEqual(t, calls, 2, "ordinary replay remains supported") + } + } +} + +func TestDisposableAdmissionIncludesScheduledExcludedBridge(t *testing.T) { + for _, replay := range []bool{false, true} { + root, ws := disposableAdmissionWorkspace(t, "build", true) + plan := &Plan{Services: []PlannedService{{Service: "app/origin"}}} + if replay { + plan = validDisposableReplay(t, root, ws, "build", []string{"modules/app/services/origin/code/input"}) + } + // The Flow-only check admits this valid withdrawal; the build scheduler + // still dispatches producer/intermediate, so whole-request refusal wins. + require.NoError(t, checkDisposablePlan(t.Context(), ws, plan, "test")) + require.ErrorContains(t, checkDisposablePlan(t.Context(), ws, plan, "lint", "build"), "build prerequisite") + calls := 0 + err := WithPlanOptions(t.Context(), ws, plan, func(context.Context, *resources.Workspace, *resources.Module, *resources.Service) error { + calls++ + return nil + }, ScheduleOptions{Phase: "build", Jobs: 2, DisposableImages: true}) + require.ErrorContains(t, err, "build prerequisite") + require.Zero(t, calls) + } +} + +func TestDisposableAdmissionDoesNotTraverseUndispatchedRuntimeBranch(t *testing.T) { + _, ws := disposableAdmissionWorkspace(t, "runtime", true) + for _, selected := range []string{"app/origin", "app/independent"} { + plan := &Plan{Services: []PlannedService{{Service: selected}}} + calls := 0 + err := WithPlanOptions(t.Context(), ws, plan, func(context.Context, *resources.Workspace, *resources.Module, *resources.Service) error { + calls++ + return nil + }, ScheduleOptions{Phase: "build", DisposableImages: true}) + require.NoError(t, err) + require.Equal(t, 1, calls, "no scheduler or Flow dispatches the withdrawn runtime branch") + } +} diff --git a/cmd/ci/run.go b/cmd/ci/run.go index a7bd5d172..43337690f 100644 --- a/cmd/ci/run.go +++ b/cmd/ci/run.go @@ -66,7 +66,7 @@ var RunCmd = &cobra.Command{ } return runWithCIReport(ctx, workspace, plan, "codefly ci run", func(reporter *Reporter) error { if disposableRuntime { - if err := checkDisposablePlan(ctx, workspace, plan); err != nil { + if err := checkDisposablePlan(ctx, workspace, plan, phases...); err != nil { return err } } diff --git a/docs/agent-ci-port-isolation.md b/docs/agent-ci-port-isolation.md index af125ce65..43873e2ba 100644 --- a/docs/agent-ci-port-isolation.md +++ b/docs/agent-ci-port-isolation.md @@ -89,7 +89,11 @@ This mode currently supports independent single-recipe services with pure runtime dependencies. Selected build-stage prerequisites (including schema and legacy dependencies), multiple recipes in one plan, unavailable input identity, push and registry cache exports are **errors**, not successful skips. Admission -checks the selected roots and their runtime closure before scheduling or loading +checks fresh and validated replay requests before dispatch. Build scheduling +conservatively refuses prerequisites it would dispatch even if a later Flow +withdraws an intermediate service; it does not traverse withdrawn runtime-only +branches that neither scheduler nor Flow dispatches. It also checks the selected +roots and their runtime closure before scheduling or loading managers; a runtime dependency's build prerequisite also refuses. Verified plan counts are checked before prefetch or host builder provisioning and again before execution. Read-only cache imports remain supported. Ordinary unscoped CI keeps diff --git a/pkg/internal/protocoltest/testdata/peer/images.go b/pkg/internal/protocoltest/testdata/peer/images.go new file mode 100644 index 000000000..7b7312797 --- /dev/null +++ b/pkg/internal/protocoltest/testdata/peer/images.go @@ -0,0 +1,73 @@ +package main + +// Optional recipe behavior for CLI-owned image boundary tests. It never calls +// Docker; the CLI remains the builder. Inventory below is synthetic transport +// evidence, not a package scan. +import ( + "context" + "fmt" + "os" + "path/filepath" + + coreservices "github.com/codefly-dev/core/agents/services" + agentv0 "github.com/codefly-dev/core/generated/go/codefly/services/agent/v0" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + runtimev0 "github.com/codefly-dev/core/generated/go/codefly/services/runtime/v0" + "google.golang.org/protobuf/proto" +) + +func (b *builderPeer) Init(_ context.Context, req *builderv0.InitRequest) (*builderv0.InitResponse, error) { + return &builderv0.InitResponse{State: &builderv0.InitStatus{State: builderv0.InitStatus_SUCCESS}}, b.p.record("Builder.Init", req) +} + +func (*builderPeer) BuildCapabilities(context.Context, *builderv0.BuildCapabilitiesRequest) (*builderv0.BuildCapabilitiesResponse, error) { + return &builderv0.BuildCapabilitiesResponse{BuildxSelection: true}, nil +} + +func (b *builderPeer) Build(_ context.Context, req *builderv0.BuildRequest) (*builderv0.BuildResponse, error) { + if err := b.p.record("Builder.Build", req); err != nil { + return nil, err + } + if os.Getenv("CODEFLY_TEST_PEER_IMAGES") != "1" { + return nil, fmt.Errorf("test recipe behavior was not selected") + } + output := req.GetOutputDirectory() + if err := os.MkdirAll(output, 0700); err != nil { + return nil, err + } + for name, contents := range map[string]string{"Dockerfile": "FROM scratch\nCOPY code /app\n", "dockerignore": ".protocol-test\n"} { + if err := os.WriteFile(filepath.Join(output, name), []byte(contents), 0600); err != nil { + return nil, err + } + } + plan, err := coreservices.SingleImageBuildPlan(output, "codefly-dev/app/subject:0.0.0", []string{"linux/arm64", "linux/amd64"}, []string{"Dockerfile", "dockerignore"}) + if err != nil { + return nil, err + } + if err := b.p.record("Builder.EmittedPlan", plan); err != nil { + return nil, err + } + return &builderv0.BuildResponse{State: &builderv0.BuildStatus{State: builderv0.BuildStatus_SUCCESS}, Result: &builderv0.BuildResult{Kind: &builderv0.BuildResult_DockerBuildPlan{DockerBuildPlan: plan}}}, nil +} + +func (b *builderPeer) SBOM(_ context.Context, req *builderv0.SBOMRequest) (*builderv0.SBOMResponse, error) { + if err := b.p.record("Builder.SBOM", req); err != nil { + return nil, err + } + if os.Getenv("CODEFLY_TEST_PEER_SBOM_ERROR") == "1" { + return nil, fmt.Errorf("intentional test SBOM failure after owned build") + } + response := &builderv0.SBOMResponse{State: &builderv0.SBOMStatus{State: builderv0.SBOMStatus_COMPLETE}, Scope: builderv0.SBOMScope_SBOM_SCOPE_IMAGE} + for _, subject := range req.Subjects { + response.Images = append(response.Images, &builderv0.ImageSBOM{Digest: subject.Digest, Platform: subject.Platform, Subjects: []*builderv0.ImageSubject{proto.CloneOf(subject)}, Bom: &agentv0.Bom{Metadata: &agentv0.Metadata{Component: &agentv0.Component{Name: "test-only-subject", Type: agentv0.ComponentType_CONTAINER}}, Components: []*agentv0.Component{{Name: "test-only-inventory"}}}, Tool: "test-peer", Sha256: "test-only-checksum"}) + } + return response, nil +} + +func (r *runtimePeer) Lint(_ context.Context, req *runtimev0.LintRequest) (*runtimev0.LintResponse, error) { + return &runtimev0.LintResponse{Status: &runtimev0.LintStatus{State: runtimev0.LintStatus_SUCCESS}}, r.p.record("Runtime.Lint", req) +} + +func (r *runtimePeer) Build(_ context.Context, req *runtimev0.BuildRequest) (*runtimev0.BuildResponse, error) { + return &runtimev0.BuildResponse{Status: &runtimev0.BuildStatus{State: runtimev0.BuildStatus_SUCCESS}}, r.p.record("Runtime.Build", req) +} diff --git a/pkg/internal/protocoltest/testdata/peer/main.go b/pkg/internal/protocoltest/testdata/peer/main.go index c7d45374a..beeaca310 100644 --- a/pkg/internal/protocoltest/testdata/peer/main.go +++ b/pkg/internal/protocoltest/testdata/peer/main.go @@ -71,6 +71,10 @@ func (d *discovery) GetAgentInformation(ctx context.Context, req *agentv0.AgentI } declaration := contract.Current() declaration.Capabilities = nil + if os.Getenv("CODEFLY_TEST_PEER_IMAGES") == "1" { + declaration = contract.Current() + declaration.Capabilities = append(declaration.Capabilities, contract.RuntimeInitImage) + } switch os.Getenv("CODEFLY_TEST_PEER_CONTRACT") { case "missing": declaration.ProtocolVersion = 0 @@ -95,6 +99,9 @@ func (d *discovery) GetAgentInformation(ctx context.Context, req *agentv0.AgentI Test: &agentv0.TestValidationCapability{Supported: true, Suites: []*agentv0.TestSuiteCapability{{Name: "unit", DefaultSuite: true, DependencyMode: dependencyMode}}}, } } + if os.Getenv("CODEFLY_TEST_PEER_IMAGES") == "1" && info.Validation != nil { + info.Validation.ArtifactBuild = &agentv0.ValidationOperationCapability{Supported: true} + } return info, nil } diff --git a/pkg/orchestration/disposable_commands_test.go b/pkg/orchestration/disposable_commands_test.go new file mode 100644 index 000000000..6e05fc350 --- /dev/null +++ b/pkg/orchestration/disposable_commands_test.go @@ -0,0 +1,193 @@ +package orchestration + +import ( + "context" + "crypto/sha256" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/codefly-dev/cli/pkg/internal/protocoltest" + "github.com/codefly-dev/core/agents/manager" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + runtimev0 "github.com/codefly-dev/core/generated/go/codefly/services/runtime/v0" + "github.com/codefly-dev/core/resources" + "github.com/codefly-dev/core/runners/recoveryscope" + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/encoding/protojson" +) + +func disposableCommandCLI(t *testing.T) string { + t.Helper() + binary := filepath.Join(t.TempDir(), "codefly") + if os.Getenv("CODEFLY_TEST_DISPOSABLE_ENGINE") == "1" { + binary = filepath.Join(os.Getenv("CODEFLY_TEST_DISPOSABLE_EVIDENCE"), "codefly") + } + command := exec.CommandContext(t.Context(), "go", "build", "-o", binary, "../../cmd/codefly") + command.Env = append(os.Environ(), "GOWORK=off") + output, err := command.CombinedOutput() + require.NoError(t, err, "%s", output) + if os.Getenv("CODEFLY_TEST_DISPOSABLE_ENGINE") == "1" { + data, readErr := os.ReadFile(binary) + require.NoError(t, readErr) + metadata, readErr := exec.CommandContext(t.Context(), "go", "version", "-m", binary).CombinedOutput() + require.NoError(t, readErr) + require.NoError(t, os.WriteFile(binary+".provenance.txt", append([]byte(fmt.Sprintf("sha256 %x\n", sha256.Sum256(data))), metadata...), 0600)) + } + return binary +} + +func disposableCommandWorkspace(t *testing.T) string { + t.Helper() + ws := writeTempWorkspace(t, map[string]string{ + "workspace.codefly.yaml": "name: disposable-commands\nlayout: modules\nmodules:\n - name: app\n", + "modules/app/module.codefly.yaml": "name: app\nservices:\n - name: subject\n - name: dependency\n", + "modules/app/services/subject/service.codefly.yaml": disposableCommandDeclaration("subject") + "service-dependencies:\n - name: dependency\n kind: runtime\n", + "modules/app/services/dependency/service.codefly.yaml": disposableCommandDeclaration("dependency"), + "modules/app/services/subject/code/payload": t.TempDir(), + "modules/app/services/dependency/code/payload": t.TempDir(), + }) + return ws.Dir() +} + +func disposableCommandDeclaration(name string) string { + return "name: " + name + "\nversion: 0.0.0\nagent:\n kind: codefly:service\n publisher: example.test\n name: disposable-command\n version: 0.0.1\n" +} + +// Process-local fake engine for commands: reuse the maintained stub, extending +// only architecture and scoped digest responses. It has no real-engine fallback. +func disposableCommandDocker(t *testing.T) *dockerStub { + t.Helper() + stub := installDockerStub(t, true) + path := filepath.Join(stub.dir, "docker") + data, err := os.ReadFile(path) + require.NoError(t, err) + script := strings.Replace(string(data), `case "$1 $2" in`, `case "$1 $2" in +"info --format") printf 'arm64\n'; exit 0 ;; +"buildx inspect") printf 'running\n'; exit 0 ;;`, 1) + script = strings.Replace(script, `printf '["codefly-local/runtime@sha256:%064d"]\n' 7`, `for arg in "$@"; do image="$arg"; done + repo="${image%:*}" + printf '["%s@sha256:%064d"]\n' "$repo" 7`, 1) + require.NoError(t, os.WriteFile(path, []byte(script), 0700)) + return stub +} + +func disposableCommand(t *testing.T, binary, root, phase string, extraEnv ...string) *exec.Cmd { + t.Helper() + ctx, cancel := context.WithTimeout(t.Context(), 90*time.Second) + t.Cleanup(cancel) + args := []string{"ci", "run", "--phase", phase, "--changed-file", "modules/app/services/subject/code/payload", "--disposable", "--runtime-context", "container", "--image-sbom", "--output", filepath.Join(root, ".reports")} + command := exec.CommandContext(ctx, binary, args...) + command.Dir = root + // Copy only the installed test peer into a new home. Controllers, agent + // registries and logs are never shared by concurrent command processes. + agent := &resources.Agent{Kind: resources.ServiceAgent, Publisher: "example.test", Name: "disposable-command", Version: "0.0.1"} + installed, err := agent.Path(t.Context()) + require.NoError(t, err) + relative, err := filepath.Rel(resources.CodeflyHomeDir(), installed) + require.NoError(t, err) + home := t.TempDir() + target := filepath.Join(home, relative) + require.NoError(t, os.MkdirAll(filepath.Dir(target), 0700)) + require.NoError(t, os.Symlink(installed, target)) + command.Env = append(os.Environ(), resources.CodeflyHomeEnv+"="+home) + command.Env = append(command.Env, extraEnv...) + return command +} + +func TestDisposableCommandsUseSelectedManagersAndArtifactBuilder(t *testing.T) { + protocoltest.Install(t, "disposable-command") + t.Setenv(manager.AgentSourceEnv, "local") + t.Setenv(recoveryscope.EnvironmentVariable, "") + t.Setenv("CODEFLY_TEST_PEER_IMAGES", "1") + t.Setenv("CODEFLY_TEST_PEER_VALIDATION", "stack") + binary := disposableCommandCLI(t) + for _, phase := range []string{"lint", "compile", "test", "build", "build-error", "sbom-error"} { + t.Run(phase, func(t *testing.T) { + stub := disposableCommandDocker(t) + root := disposableCommandWorkspace(t) + actualPhase := phase + var env []string + if phase == "sbom-error" { + actualPhase = "build" + env = []string{"CODEFLY_TEST_PEER_SBOM_ERROR=1"} + } + if phase == "build-error" { + actualPhase = "build" + require.NoError(t, os.WriteFile(filepath.Join(stub.dir, "hook"), []byte("echo intentional-test-build-failure >&2; exit 96\n"), 0600)) + } + output, err := disposableCommand(t, binary, root, actualPhase, env...).CombinedOutput() + if strings.HasSuffix(phase, "-error") { + require.Error(t, err) + require.Contains(t, string(output), "intentional") + require.Equal(t, 1, stub.calls(t, "build")) + return + } + require.NoError(t, err, "%s", output) + require.Positive(t, stub.calls(t, "build")) + assertDisposableCommandDelivery(t, root, phase, false) + }) + } +} + +// Inspect actual recorded gRPC deliveries from command-created managers. Real +// engine qualification additionally compares the delivered subjects to Docker. +func assertDisposableCommandDelivery(t *testing.T, root, phase string, real bool) { + t.Helper() + for _, service := range []string{"subject", "dependency"} { + if phase == "build" && service == "dependency" { + continue + } + calls := protocoltest.Calls(t, filepath.Join(root, "modules/app/services", service)) + var images []string + methods := map[string]bool{} + for _, call := range calls { + methods[call.Method] = true + switch call.Method { + case "Builder.EmittedPlan": + plan := &builderv0.DockerBuildPlan{} + require.NoError(t, protojson.Unmarshal(call.Request, plan)) + require.Len(t, plan.Recipes, 1) + require.Equal(t, "codefly-dev/app/subject:0.0.0", plan.Recipes[0].Image) + case "Runtime.Init": + request := &runtimev0.InitRequest{} + require.NoError(t, protojson.Unmarshal(call.Request, request)) + images = append(images, request.RuntimeImage) + case "Builder.SBOM": + request := &builderv0.SBOMRequest{} + require.NoError(t, protojson.Unmarshal(call.Request, request)) + for _, subject := range request.Subjects { + images = append(images, subject.Reference) + if real { + id, err := inspectLocalImageID(t.Context(), subject.Reference) + require.NoError(t, err) + require.Equal(t, id, subject.Digest) + platform, err := exec.CommandContext(t.Context(), "docker", "image", "inspect", "--format", "{{.Os}}/{{.Architecture}}", subject.Reference).Output() + require.NoError(t, err) + require.Equal(t, strings.TrimSpace(string(platform)), subject.Platform) + } + } + } + } + require.True(t, methods["Builder.Build"]) + if service == "subject" { + expected := map[string]string{"lint": "Runtime.Lint", "compile": "Runtime.Build", "test": "Runtime.Test", "build": "Builder.SBOM"}[phase] + require.True(t, methods[expected], "command must execute its operation, not skip") + } + require.NotEmpty(t, images, "%s/%s must reach actual image delivery", phase, service) + for _, image := range images { + require.Contains(t, image, "codefly-local/disposable/") + if real { + _, err := inspectLocalImageID(t.Context(), image) + require.NoError(t, err) + } + } + if real { + require.NoError(t, os.WriteFile(filepath.Join(os.Getenv("CODEFLY_TEST_DISPOSABLE_EVIDENCE"), phase+"-"+service+"-delivered.txt"), []byte(strings.Join(images, "\n")), 0600)) + } + } +} diff --git a/pkg/orchestration/disposable_engine_test.go b/pkg/orchestration/disposable_engine_test.go new file mode 100644 index 000000000..7a721dc03 --- /dev/null +++ b/pkg/orchestration/disposable_engine_test.go @@ -0,0 +1,392 @@ +package orchestration + +// Opt-in host-boundary qualification. No deletion is registered, even on +// failure. The proxy permits only scoped builds and read-only engine queries. +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/codefly-dev/cli/pkg/internal/protocoltest" + "github.com/codefly-dev/core/agents/manager" + builderv0 "github.com/codefly-dev/core/generated/go/codefly/services/builder/v0" + "github.com/codefly-dev/core/runners/recoveryscope" + "github.com/codefly-dev/core/wool" + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/encoding/protojson" + "google.golang.org/protobuf/proto" +) + +var disposableProtectedImages = map[string]string{ + "codefly-dev/app/subject:0.0.0": "sha256:7bef274c8e7c158a93bd5fcc56f39d530f3e56826cfa34151899747d1ffa561c", + "codefly-local/runtime:sha256-5ac8c7280c1eca0c57ccb065cb29216316c46fe5cbd844e6695e4736c10a1c6b": "sha256:714e6933800934c51975fa2aa49206d01f9a43435ab158a0936ac50b0a5a4e90", +} + +type disposableEngineCall struct { + Args []string + Start, End int64 + Error string +} + +// Runs only as the Docker executable of an explicitly opted-in test. Exiting +// prevents the Go test runner from appending PASS to Docker's protocol output. +func TestDisposableDockerProxy(t *testing.T) { + if os.Getenv("CODEFLY_TEST_DOCKER_PROXY_CHILD") != "1" { + return + } + args := os.Args + for len(args) > 0 && args[0] != "--" { + args = args[1:] + } + if len(args) == 0 { + os.Exit(97) + } + args = args[1:] + build := len(args) > 1 && args[0] == "buildx" && args[1] == "build" + if !allowedDisposableEngineCall(args) { + fmt.Fprintln(os.Stderr, "unallocated engine operation", args) + os.Exit(97) + } + if build { + // Every actual build in this selection is fresh. Same-flow reuse must + // issue no build. Existing output is a collision, never setup authority. + ref := disposableBuiltRef(disposableEngineCall{Args: args}) + probe := exec.Command(os.Getenv("CODEFLY_TEST_DISPOSABLE_REAL_DOCKER"), "image", "inspect", "--format", "{{.Id}}", ref) + output, err := probe.CombinedOutput() + if err == nil || !strings.Contains(strings.ToLower(string(output)), "no such image") { + fmt.Fprintln(os.Stderr, "fresh fixture destination is not proven absent", ref) + os.Exit(97) + } + } + call := disposableEngineCall{Args: args, Start: time.Now().UnixNano()} + // Two separate CLI processes meet at their first owned build. The actual + // command intervals below, not the barrier alone, must overlap. + if build && os.Getenv("CODEFLY_TEST_IMAGE_BARRIER") != "" { + dir := os.Getenv("CODEFLY_TEST_IMAGE_BARRIER") + _ = os.WriteFile(filepath.Join(dir, os.Getenv("CODEFLY_TEST_IMAGE_PARTICIPANT")), nil, 0600) + deadline := time.Now().Add(30 * time.Second) + for { + entries, _ := os.ReadDir(dir) + if len(entries) >= 2 { + break + } + if time.Now().After(deadline) { + os.Exit(98) + } + time.Sleep(10 * time.Millisecond) + } + } + call.Start = time.Now().UnixNano() + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + command := exec.CommandContext(ctx, os.Getenv("CODEFLY_TEST_DISPOSABLE_REAL_DOCKER"), args...) + command.Stdout, command.Stderr = os.Stdout, os.Stderr + err := command.Run() + call.End = time.Now().UnixNano() + if err != nil { + call.Error = err.Error() + } + file, openErr := os.OpenFile(os.Getenv("CODEFLY_TEST_IMAGE_COMMANDS"), os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600) + if openErr != nil { + os.Exit(99) + } + _ = json.NewEncoder(file).Encode(call) + _ = file.Close() + if err != nil { + os.Exit(1) + } + // Inject an evidence-side host failure AFTER this invocation's own image + // was loaded; the caller must report failure without rollback/cleanup. + if build && os.Getenv("CODEFLY_TEST_IMAGE_AFTER_BUILD_ERROR") == "1" { + fmt.Fprintln(os.Stderr, "intentional failure after owned build") + os.Exit(96) + } + os.Exit(0) +} + +func allowedDisposableEngineCall(args []string) bool { + if len(args) < 2 { + return false + } + if args[0] == "image" && args[1] == "inspect" { + return true + } + if args[0] == "info" && args[1] == "--format" { + return true + } + if args[0] != "buildx" { + return false + } + if args[1] == "inspect" { + for _, arg := range args[2:] { + if strings.HasPrefix(arg, "--bootstrap") { + return false + } + } + return true + } + if args[1] != "build" { + return false + } + tags := 0 + for i, arg := range args { + if strings.HasPrefix(arg, "--push") || strings.HasPrefix(arg, "--cache-to") || strings.HasPrefix(arg, "--output") || strings.HasPrefix(arg, "--tag") { + return false + } + if arg == "-t" { + if i+1 == len(args) || !strings.HasPrefix(args[i+1], "codefly-local/disposable/") { + return false + } + tags++ + } + } + return tags == 1 +} + +func TestDisposableEngineProxyRefusesForeignEffects(t *testing.T) { + for _, args := range [][]string{ + {"image", "rm", "anything"}, {"buildx", "create"}, + {"buildx", "build", "-t", "codefly-dev/app/subject:0.0.0"}, + {"buildx", "build", "-t", "codefly-local/disposable/new:one", "-t", "foreign:tag"}, + {"buildx", "build", "-t", "codefly-local/disposable/new:one", "--push=true"}, + {"buildx", "build", "-t", "codefly-local/disposable/new:one", "--output=type=registry"}, + } { + require.False(t, allowedDisposableEngineCall(args), "%v", args) + } + require.True(t, allowedDisposableEngineCall([]string{"buildx", "build", "--load", "-t", "codefly-local/disposable/new:one", "."})) +} + +func disposableEngineSetup(t *testing.T) (string, string) { + t.Helper() + if os.Getenv("CODEFLY_TEST_DISPOSABLE_ENGINE") != "1" { + t.Skip("requires explicit root engine allocation: CODEFLY_TEST_DISPOSABLE_ENGINE=1") + } + evidence := os.Getenv("CODEFLY_TEST_DISPOSABLE_EVIDENCE") + require.True(t, filepath.IsAbs(evidence), "require a new absolute evidence directory") + require.NoError(t, os.Mkdir(evidence, 0700), "refuse reusing another run's evidence") + docker, err := exec.LookPath("docker") + require.NoError(t, err) + binary, err := os.Executable() + require.NoError(t, err) + dir := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(dir, "docker"), []byte("#!/bin/sh\nexec \"$CODEFLY_TEST_DOCKER_PROXY\" -test.run=^TestDisposableDockerProxy$ -- \"$@\"\n"), 0700)) + t.Setenv("CODEFLY_TEST_DISPOSABLE_REAL_DOCKER", docker) + t.Setenv("CODEFLY_TEST_DOCKER_PROXY", binary) + t.Setenv("CODEFLY_TEST_DOCKER_PROXY_CHILD", "1") + t.Setenv("CODEFLY_TEST_IMAGE_COMMANDS", filepath.Join(evidence, "engine.jsonl")) + t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH")) + return evidence, dir +} + +func disposableImageObservation(t *testing.T, ref string) []byte { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + out, err := exec.CommandContext(ctx, "docker", "image", "inspect", "--format", `{"Id":{{json .Id}},"RepoTags":{{json .RepoTags}},"RepoDigests":{{json .RepoDigests}}}`, ref).CombinedOutput() + require.NoError(t, err, "inspect %s: %s", ref, out) + var data struct { + ID string `json:"Id"` + RepoTags, RepoDigests []string + } + require.NoError(t, json.Unmarshal(bytes.TrimSpace(out), &data)) + if expected, ok := disposableProtectedImages[ref]; ok { + require.Equal(t, expected, data.ID, "protected baseline changed; never restore") + } + encoded, err := json.Marshal(data) + require.NoError(t, err) + return encoded +} + +func TestDisposableImagesEngineQualification(t *testing.T) { + evidence, proxy := disposableEngineSetup(t) + baseline := map[string][]byte{} + for ref := range disposableProtectedImages { + baseline[ref] = disposableImageObservation(t, ref) + } + check := func(label string) { + for ref, want := range baseline { + require.Equal(t, want, disposableImageObservation(t, ref), label+": protected reference changed") + } + data, err := json.Marshal(baseline) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(evidence, label+"-protected.json"), data, 0600)) + } + check("preflight") + t.Cleanup(func() { check("cleanup-observation") }) + protocoltest.Install(t, "disposable-command") + t.Setenv(manager.AgentSourceEnv, "local") + t.Setenv(recoveryscope.EnvironmentVariable, "") + t.Setenv("CODEFLY_TEST_PEER_IMAGES", "1") + t.Setenv("CODEFLY_TEST_PEER_VALIDATION", "stack") + cli := disposableCommandCLI(t) + seen := map[string]bool{} + for _, phase := range []string{"lint", "compile", "test", "build"} { + before := len(disposableBuildCalls(t, evidence)) + root := disposableCommandWorkspace(t) + output, err := disposableCommand(t, cli, root, phase).CombinedOutput() + require.NoError(t, err, "%s: %s", phase, output) + assertDisposableCommandDelivery(t, root, phase, true) + require.NoError(t, os.WriteFile(filepath.Join(evidence, phase+".log"), output, 0600)) + for _, call := range disposableBuildCalls(t, evidence)[before:] { + ref := disposableBuiltRef(call) + require.False(t, seen[ref], "sequential flows must not reuse destinations") + seen[ref] = true + } + check(phase) + } + // Independent process environments and workspaces; no t.Setenv in workers. + barrier := t.TempDir() + beforeConcurrent := disposableBuildCalls(t, evidence) + var commands []*exec.Cmd + var outputs [2]bytes.Buffer + for i := 0; i < 2; i++ { + root := disposableCommandWorkspace(t) + command := disposableCommand(t, cli, root, "build", "CODEFLY_TEST_IMAGE_BARRIER="+barrier, fmt.Sprintf("CODEFLY_TEST_IMAGE_PARTICIPANT=%d", i)) + command.Stdout, command.Stderr = &outputs[i], &outputs[i] + if err := command.Start(); err != nil { + for _, started := range commands { + _ = started.Wait() + } + require.NoError(t, err) + } + commands = append(commands, command) + } + var waits []error + for _, command := range commands { + waits = append(waits, command.Wait()) + } + for i, err := range waits { + require.NoError(t, err, "%s", outputs[i].Bytes()) + } + afterConcurrent := disposableBuildCalls(t, evidence)[len(beforeConcurrent):] + require.Len(t, afterConcurrent, 2) + require.Less(t, afterConcurrent[0].Start, afterConcurrent[1].End) + require.Less(t, afterConcurrent[1].Start, afterConcurrent[0].End, "actual build command intervals must overlap") + require.NotEqual(t, disposableBuiltRef(afterConcurrent[0]), disposableBuiltRef(afterConcurrent[1])) + check("concurrent") + for _, fault := range []string{"CODEFLY_TEST_IMAGE_AFTER_BUILD_ERROR=1", "CODEFLY_TEST_PEER_SBOM_ERROR=1"} { + root := disposableCommandWorkspace(t) + before := len(disposableBuildCalls(t, evidence)) + output, err := disposableCommand(t, cli, root, "build", fault).CombinedOutput() + require.Error(t, err, "fault must fail: %s", output) + require.Contains(t, string(output), "intentional") + created := disposableBuildCalls(t, evidence)[before:] + require.Len(t, created, 1) + _ = disposableImageObservation(t, disposableBuiltRef(created[0])) // retained after failure + check(strings.Split(fault, "=")[0]) + } + // Same-flow reuse through the maintained Builder + actual loopback SBOM + // peer. Actual engine IDs/platforms are compared independently of replies. + peer := &disposableRecipePeer{sbomRequests: make(chan *builderv0.SBOMRequest, 2)} + b, _ := disposableBuilder(t, peer) + t.Setenv("PATH", proxy+string(os.PathListSeparator)+os.Getenv("PATH")) + b.world.BuildxBuilder = "" + b.world.hostArchitecture = nil + b.world.CollectImageSBOM = true + beforeReuse := len(disposableBuildCalls(t, evidence)) + var ordinaryRef string + var ordinaryBefore []byte + var previous *builderv0.ImageSubject + for i := 0; i < 2; i++ { + _, err := b.Plan(t.Context()) + require.NoError(t, err) + producerPlan := b.planned.response.GetResult().GetDockerBuildPlan() + original := proto.CloneOf(producerPlan) + if i == 0 { + ordinaryRef = disposableOrdinaryRuntimeRef(t, b) + ordinaryBefore = disposableOptionalObservation(t, ordinaryRef) + require.NoError(t, os.WriteFile(filepath.Join(evidence, "fixture-ordinary-reference.txt"), []byte(ordinaryRef+"\n"+string(ordinaryBefore)), 0600)) + } + _, err = b.Build(t.Context()) + require.NoError(t, err) + require.True(t, proto.Equal(original, producerPlan)) + request := <-peer.sbomRequests + require.Len(t, request.Subjects, 1) + subject := request.Subjects[0] + id, err := inspectLocalImageID(t.Context(), subject.Reference) + require.NoError(t, err) + require.Equal(t, id, subject.Digest) + platform, err := exec.CommandContext(t.Context(), "docker", "image", "inspect", "--format", "{{.Os}}/{{.Architecture}}", subject.Reference).Output() + require.NoError(t, err) + require.Equal(t, strings.TrimSpace(string(platform)), subject.Platform) + payload, err := protojson.Marshal(request) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(evidence, fmt.Sprintf("sbom-%d.json", i)), payload, 0600)) + require.Contains(t, subject.Reference, "codefly-local/disposable/") + if previous != nil { + require.True(t, proto.Equal(previous, subject)) + } + previous = subject + } + require.Equal(t, beforeReuse+1, len(disposableBuildCalls(t, evidence)), "same-flow second SBOM must be a cache hit") + require.Equal(t, ordinaryBefore, disposableOptionalObservation(t, ordinaryRef)) + check("final") +} + +func disposableBuildCalls(t *testing.T, evidence string) []disposableEngineCall { + t.Helper() + file, err := os.Open(filepath.Join(evidence, "engine.jsonl")) + require.NoError(t, err) + defer file.Close() + var result []disposableEngineCall + decoder := json.NewDecoder(file) + for { + var call disposableEngineCall + err = decoder.Decode(&call) + if err == io.EOF { + break + } + require.NoError(t, err) + if len(call.Args) > 1 && call.Args[0] == "buildx" && call.Args[1] == "build" { + result = append(result, call) + } + } + return result +} + +func disposableBuiltRef(call disposableEngineCall) string { + for i, arg := range call.Args { + if arg == "-t" && i+1 < len(call.Args) { + return call.Args[i+1] + } + } + return "" +} + +// Derive the ordinary destination through exactly the existing runtime recipe, +// input identity and name selection. This copy never executes Build or writes a +// cache record; the scoped builder remains untouched. +func disposableOrdinaryRuntimeRef(t *testing.T, b *Builder) string { + t.Helper() + ordinary := *b + world := *b.world + world.disposableImageScope = "" + world.RebuildImages = true // identity/name only; do not query a cache for reuse + ordinary.world = &world + ordinary.localRuntime = true + plan := b.planned.response.GetResult().GetDockerBuildPlan() + recipe, err := ordinary.recipeAsBuilt(plan.Recipes[0], false) + require.NoError(t, err) + decision, err := ordinary.decideImageReuse(t.Context(), wool.Get(t.Context()), b.planned.outputDir, recipeContextRoot(b.instance.Service.Dir(), b.planned.outputDir, plan), plan, recipe, false) + require.NoError(t, err) + require.Regexp(t, `^codefly-local/runtime:sha256-[0-9a-f]{64}$`, decision.image) + return decision.image +} + +func disposableOptionalObservation(t *testing.T, ref string) []byte { + t.Helper() + output, err := exec.CommandContext(t.Context(), "docker", "image", "inspect", "--format", `{"Id":{{json .Id}},"RepoTags":{{json .RepoTags}},"RepoDigests":{{json .RepoDigests}}}`, ref).CombinedOutput() + if err != nil { + require.Contains(t, strings.ToLower(string(output)), "no such image", "ordinary error is not absence") + return []byte("absent") + } + return output +} diff --git a/pkg/orchestration/disposable_images.go b/pkg/orchestration/disposable_images.go index ec766d9ee..364af7666 100644 --- a/pkg/orchestration/disposable_images.go +++ b/pkg/orchestration/disposable_images.go @@ -54,29 +54,41 @@ func checkDisposableImageDependencies(ctx context.Context, dependencies *archite if err != nil { return err } + var participants []string + for _, root := range roots { + required, err := run.OrderTo(ctx, root) + if err != nil { + return err + } + participants = append(participants, root) + for _, service := range required { + participants = append(participants, service.Unique) + } + } + return CheckDisposableImageBuildPrerequisites(ctx, dependencies, participants) +} + +// CheckDisposableImageBuildPrerequisites checks only the supplied participants' +// build closure. CI uses it for the roots whose build prerequisites it schedules; +// Flow supplies its selected runtime participants. It does not select runtime +// participants from an unfiltered workspace graph. +func CheckDisposableImageBuildPrerequisites(ctx context.Context, dependencies *architecture.ServiceDependencies, participants []string) error { build, err := dependencies.ForStage(resources.StageBuild) if err != nil { return err } seen := map[string]bool{} - for _, root := range roots { - participants, err := run.OrderTo(ctx, root) + for _, participant := range participants { + if seen[participant] { + continue + } + seen[participant] = true + required, err := build.OrderTo(ctx, participant) if err != nil { return err } - participants = append(participants, architecture.Service{Unique: root}) - for _, participant := range participants { - if seen[participant.Unique] { - continue - } - seen[participant.Unique] = true - required, err := build.OrderTo(ctx, participant.Unique) - if err != nil { - return err - } - if len(required) > 0 { - return fmt.Errorf("disposable image input rebinding is unsupported: %s has build prerequisite %s", participant.Unique, required[0].Unique) - } + if len(required) > 0 { + return fmt.Errorf("disposable image input rebinding is unsupported: %s has build prerequisite %s", participant, required[0].Unique) } } return nil From 8d0ef7831a8848ac403a2d92c4d1bb2f293a3ba6 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 10:43:28 -0400 Subject: [PATCH 16/18] test(ci): retain disposable command evidence before assertions --- pkg/orchestration/disposable_engine_test.go | 138 ++++++++++++++++++-- 1 file changed, 126 insertions(+), 12 deletions(-) diff --git a/pkg/orchestration/disposable_engine_test.go b/pkg/orchestration/disposable_engine_test.go index 7a721dc03..95c667db1 100644 --- a/pkg/orchestration/disposable_engine_test.go +++ b/pkg/orchestration/disposable_engine_test.go @@ -6,6 +6,7 @@ import ( "bytes" "context" "encoding/json" + "errors" "fmt" "io" "os" @@ -231,10 +232,11 @@ func TestDisposableImagesEngineQualification(t *testing.T) { for _, phase := range []string{"lint", "compile", "test", "build"} { before := len(disposableBuildCalls(t, evidence)) root := disposableCommandWorkspace(t) - output, err := disposableCommand(t, cli, root, phase).CombinedOutput() + command := disposableCommand(t, cli, root, phase) + output, err := command.CombinedOutput() + require.NoError(t, saveDisposableCommandEvidence(evidence, phase, command, output, err)) require.NoError(t, err, "%s: %s", phase, output) assertDisposableCommandDelivery(t, root, phase, true) - require.NoError(t, os.WriteFile(filepath.Join(evidence, phase+".log"), output, 0600)) for _, call := range disposableBuildCalls(t, evidence)[before:] { ref := disposableBuiltRef(call) require.False(t, seen[ref], "sequential flows must not reuse destinations") @@ -246,23 +248,29 @@ func TestDisposableImagesEngineQualification(t *testing.T) { barrier := t.TempDir() beforeConcurrent := disposableBuildCalls(t, evidence) var commands []*exec.Cmd + var waits []error var outputs [2]bytes.Buffer for i := 0; i < 2; i++ { root := disposableCommandWorkspace(t) command := disposableCommand(t, cli, root, "build", "CODEFLY_TEST_IMAGE_BARRIER="+barrier, fmt.Sprintf("CODEFLY_TEST_IMAGE_PARTICIPANT=%d", i)) command.Stdout, command.Stderr = &outputs[i], &outputs[i] - if err := command.Start(); err != nil { - for _, started := range commands { - _ = started.Wait() - } - require.NoError(t, err) - } commands = append(commands, command) + startErr := command.Start() + waits = append(waits, startErr) + if startErr != nil { + break + } } - var waits []error - for _, command := range commands { - waits = append(waits, command.Wait()) + for i, command := range commands { + if waits[i] == nil { + waits[i] = command.Wait() + } + } + var captureErrors []error + for i, command := range commands { + captureErrors = append(captureErrors, saveDisposableCommandEvidence(evidence, fmt.Sprintf("concurrent-%d", i), command, outputs[i].Bytes(), waits[i])) } + require.NoError(t, errors.Join(captureErrors...)) for i, err := range waits { require.NoError(t, err, "%s", outputs[i].Bytes()) } @@ -275,7 +283,9 @@ func TestDisposableImagesEngineQualification(t *testing.T) { for _, fault := range []string{"CODEFLY_TEST_IMAGE_AFTER_BUILD_ERROR=1", "CODEFLY_TEST_PEER_SBOM_ERROR=1"} { root := disposableCommandWorkspace(t) before := len(disposableBuildCalls(t, evidence)) - output, err := disposableCommand(t, cli, root, "build", fault).CombinedOutput() + command := disposableCommand(t, cli, root, "build", fault) + output, err := command.CombinedOutput() + require.NoError(t, saveDisposableCommandEvidence(evidence, "fault-"+strings.Split(fault, "=")[0], command, output, err)) require.Error(t, err, "fault must fail: %s", output) require.Contains(t, string(output), "intentional") created := disposableBuildCalls(t, evidence)[before:] @@ -390,3 +400,107 @@ func disposableOptionalObservation(t *testing.T, ref string) []byte { } return output } + +// Save before interpreting the outcome. A failed command may have created an +// owned image, and temporary workspaces are removed even when assertions fail. +func saveDisposableCommandEvidence(evidence, label string, command *exec.Cmd, output []byte, runErr error) error { + destination := filepath.Join(evidence, "commands", label) + if err := os.MkdirAll(destination, 0700); err != nil { + return err + } + failures := []error{os.WriteFile(filepath.Join(destination, "output.log"), output, 0600)} + result := struct { + Args []string `json:"args"` + Directory string `json:"directory"` + ExitCode *int `json:"exit_code"` + Error string `json:"error,omitempty"` + Files map[string]string `json:"files"` + }{Args: command.Args, Directory: command.Dir, Files: map[string]string{}} + if command.ProcessState != nil { + code := command.ProcessState.ExitCode() + result.ExitCode = &code + } + if runErr != nil { + result.Error = runErr.Error() + } + files := []string{"workspace.codefly.yaml", "modules/app/module.codefly.yaml"} + for _, service := range []string{"subject", "dependency"} { + for _, name := range []string{"service.codefly.yaml", "code/payload", ".protocol-test/calls.jsonl"} { + files = append(files, filepath.Join("modules/app/services", service, name)) + } + } + for _, name := range files { + data, err := os.ReadFile(filepath.Join(command.Dir, name)) + if errors.Is(err, os.ErrNotExist) { + result.Files[name] = "absent" + continue + } + if err == nil { + target := filepath.Join(destination, "fixture", name) + err = os.MkdirAll(filepath.Dir(target), 0700) + if err == nil { + err = os.WriteFile(target, data, 0600) + } + } + if err != nil { + failures = append(failures, err) + result.Files[name] = err.Error() + } else { + result.Files[name] = "saved" + } + } + reports := filepath.Join(command.Dir, ".reports") + if _, err := os.Stat(reports); errors.Is(err, os.ErrNotExist) { + result.Files[".reports"] = "absent" + } else if err != nil { + failures = append(failures, err) + result.Files[".reports"] = err.Error() + } else if err := os.CopyFS(filepath.Join(destination, "fixture/.reports"), os.DirFS(reports)); err != nil { + failures = append(failures, err) + result.Files[".reports"] = err.Error() + } else { + result.Files[".reports"] = "saved" + } + data, err := json.MarshalIndent(result, "", " ") + if err == nil { + err = os.WriteFile(filepath.Join(destination, "result.json"), data, 0600) + } + return errors.Join(append(failures, err)...) +} + +func TestDisposableCommandEvidenceRetainsFailedRequests(t *testing.T) { + root, evidence := t.TempDir(), t.TempDir() + protocol := "modules/app/services/subject/.protocol-test/calls.jsonl" + for name, data := range map[string]string{protocol: "{\"method\":\"Builder.SBOM\"}\n", ".reports/report.json": "{\"status\":\"failed\"}"} { + path := filepath.Join(root, name) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0700)) + require.NoError(t, os.WriteFile(path, []byte(data), 0600)) + } + command := exec.CommandContext(t.Context(), "sh", "-c", "printf 'intentional failure output'; exit 7") + command.Dir = root + output, runErr := command.CombinedOutput() + require.Error(t, runErr) + require.NoError(t, saveDisposableCommandEvidence(evidence, "failed", command, output, runErr)) + // Remove only this test's temporary source to prove retained bytes survive. + require.NoError(t, os.RemoveAll(root)) + base := filepath.Join(evidence, "commands/failed") + saved, err := os.ReadFile(filepath.Join(base, "output.log")) + require.NoError(t, err) + require.Equal(t, output, saved) + saved, err = os.ReadFile(filepath.Join(base, "fixture", protocol)) + require.NoError(t, err) + require.Equal(t, "{\"method\":\"Builder.SBOM\"}\n", string(saved)) + require.FileExists(t, filepath.Join(base, "fixture/.reports/report.json")) + saved, err = os.ReadFile(filepath.Join(base, "result.json")) + require.NoError(t, err) + require.Contains(t, string(saved), `"exit_code": 7`) + require.Contains(t, string(saved), `"modules/app/services/dependency/.protocol-test/calls.jsonl": "absent"`) + missing := exec.CommandContext(t.Context(), filepath.Join(t.TempDir(), "missing-command")) + missing.Dir = t.TempDir() + output, runErr = missing.CombinedOutput() + require.Error(t, runErr) + require.NoError(t, saveDisposableCommandEvidence(evidence, "start-failed", missing, output, runErr)) + saved, err = os.ReadFile(filepath.Join(evidence, "commands/start-failed/result.json")) + require.NoError(t, err) + require.Contains(t, string(saved), `"exit_code": null`) +} From 48a517cd3365bdf20e54bb6948f5d5bd59f824cb Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 10:44:29 -0400 Subject: [PATCH 17/18] fix(generate): derive runnable audiences from owner publications --- cmd/generate/contract_surface_test.go | 4 + .../runnable_source_operation_test.go | 3 + cmd/generate/runnables.go | 35 +++++++- cmd/generate/runnables_audience_test.go | 82 +++++++++++++++++++ cmd/generate/runnables_default_test.go | 2 +- cmd/generate/runnables_fixture_test.go | 20 +++++ cmd/generate/runnables_solution_test.go | 3 + docs/runnable.md | 8 ++ 8 files changed, 152 insertions(+), 5 deletions(-) create mode 100644 cmd/generate/runnables_audience_test.go diff --git a/cmd/generate/contract_surface_test.go b/cmd/generate/contract_surface_test.go index fbe95c86e..6793cdc57 100644 --- a/cmd/generate/contract_surface_test.go +++ b/cmd/generate/contract_surface_test.go @@ -11,6 +11,7 @@ import ( "testing" "github.com/codefly-dev/core/composition" + modulecontract "github.com/codefly-dev/core/contracts/module" basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" "github.com/codefly-dev/core/resources" "github.com/stretchr/testify/require" @@ -23,6 +24,9 @@ func TestContractSurfaceExportAndDerivation(t *testing.T) { Endpoints: []*resources.Endpoint{{Name: "invoke", API: "connect"}, {Name: "receipts", API: "grpc"}}, } root, moduleDir := saveFixtureWorkspace(t, ctx, "documents", nil, service) + saveRunnablePublication(t, moduleDir, "documents.ingestion", + modulecontract.Destination{ID: "invoke", Service: service.Name, Endpoint: "invoke", Kind: "module"}, + modulecontract.Destination{ID: "receipts", Service: service.Name, Endpoint: "receipts", Kind: "module"}) service, err := resources.LoadServiceFromDir(ctx, service.Dir()) require.NoError(t, err) writeFixtureFile(t, filepath.Join(service.Dir(), "generated/surfaces.json"), []byte(`{ diff --git a/cmd/generate/runnable_source_operation_test.go b/cmd/generate/runnable_source_operation_test.go index 56543c457..9b55e0d4d 100644 --- a/cmd/generate/runnable_source_operation_test.go +++ b/cmd/generate/runnable_source_operation_test.go @@ -13,6 +13,7 @@ import ( "github.com/codefly-dev/cli/pkg/environments" runnablespkg "github.com/codefly-dev/cli/pkg/runnables" "github.com/codefly-dev/core/composition" + modulecontract "github.com/codefly-dev/core/contracts/module" basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" "github.com/codefly-dev/core/resources" corerunnable "github.com/codefly-dev/core/runnable" @@ -45,6 +46,8 @@ func TestGenerateRealSourceOperations(t *testing.T) { root, moduleDir := saveFixtureWorkspace(t, context.Background(), "saas-starter", &resources.ModuleInterface{Endpoints: []*resources.InterfaceEndpoint{{Service: "accounts", Endpoint: "authority", Visibility: resources.VisibilityInternal}, {Service: "accounts", Endpoint: "connect", Visibility: resources.VisibilityInternal}}}, service) + // The fixture declares the trust boundary carried by its explicit operations. + saveRunnablePublication(t, moduleDir, "saas-datasource", modulecontract.Destination{ID: "connect", Service: "accounts", Endpoint: "connect", Kind: "module"}) writeFixtureFile(t, filepath.Join(service.Dir(), "generated/surfaces.json"), surfaceBytes) surfacePath := fixtureSurfacePath(t, moduleDir, service.Name, "generated/surfaces.json") surfaces, err := loadContractSurfaces(service, []*basev0.Endpoint{{Name: "authority", Api: "grpc"}, {Name: "connect", Api: "connect"}}, surfacePath) diff --git a/cmd/generate/runnables.go b/cmd/generate/runnables.go index 36f9ab141..4ba41af2e 100644 --- a/cmd/generate/runnables.go +++ b/cmd/generate/runnables.go @@ -17,6 +17,7 @@ import ( "github.com/codefly-dev/cli/pkg/cli" runnablespkg "github.com/codefly-dev/cli/pkg/runnables" "github.com/codefly-dev/core/composition" + modulecontract "github.com/codefly-dev/core/contracts/module" basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" runnablev0 "github.com/codefly-dev/core/generated/go/codefly/runnable/v0" "github.com/codefly-dev/core/resources" @@ -154,8 +155,10 @@ func init() { // derivation is everything one walk produced: the files the tree should hold, // the index that names them, and the methods that were refused. type derivation struct { - index *runnablespkg.Index - files map[string][]byte + index *runnablespkg.Index + files map[string][]byte + publication *modulecontract.Contract + publicationErr error // release is the module's own release version, empty for a module that // declares no package manifest. release string @@ -265,6 +268,9 @@ func deriveRunnablesIn(ctx context.Context, workspaceName, workspaceDir string, files: map[string][]byte{}, origins: map[string]derivedOperation{}, } + // Load the selected owner's declaration once. A module with no operations + // still produces an empty index; an operation cannot publish without it. + derived.publication, derived.publicationErr = modulecontract.Load(module.Dir()) var endpoints []composition.APIContractEndpoint if catalog != nil { @@ -352,6 +358,7 @@ func deriveEndpoint(module *resources.Module, endpoint *composition.APIContractE owner := corerunnable.ServiceOwner{ Module: module.Name, + Audience: derived.audience(), Service: endpoint.Service, Endpoint: endpoint.Endpoint, Agent: agent, @@ -403,7 +410,7 @@ func deriveOpenAPIEndpoint(module *resources.Module, endpoint *composition.APICo if err != nil { return fmt.Errorf("cannot read %s: %w", contract, err) } - owner := corerunnable.ServiceOwner{Module: module.Name, Service: endpoint.Service, Endpoint: endpoint.Endpoint, Agent: agent} + owner := corerunnable.ServiceOwner{Module: module.Name, Audience: derived.audience(), Service: endpoint.Service, Endpoint: endpoint.Endpoint, Agent: agent} paths := make([]string, 0, len(parsed.Paths)) for path := range parsed.Paths { paths = append(paths, path) @@ -528,8 +535,28 @@ func (d *derivation) admitted(subject string, err error) bool { return false } -// record writes one derived operation into the tree and the index. +func (d *derivation) audience() string { + if d.publication == nil { + return "" + } + return d.publication.Principal +} + +// record writes one derived operation into the tree and the index. The +// publication principal is independent of the producer checkout's module name; +// an explicitly declared operation must agree, never be silently rewritten. func (d *derivation) record(endpoint *composition.APIContractEndpoint, name string, pkg *basev0.RunnablePackage, spec *corerunnable.OperationSpec, origin derivedOperation) error { + if d.publicationErr != nil { + return fmt.Errorf("runnable %s/%s/%s needs its owner's module contract: %w", endpoint.Service, endpoint.Endpoint, name, d.publicationErr) + } + if d.publication == nil || !slices.ContainsFunc(d.publication.Destinations, func(destination modulecontract.Destination) bool { + return destination.Service == endpoint.Service && destination.Endpoint == endpoint.Endpoint + }) { + return fmt.Errorf("module contract does not publish runnable destination %s/%s", endpoint.Service, endpoint.Endpoint) + } + if spec.Audience != d.publication.Principal { + return fmt.Errorf("runnable %s/%s/%s audience %q differs from module contract principal %q", endpoint.Service, endpoint.Endpoint, name, spec.Audience, d.publication.Principal) + } operation := pkg.GetServiceOperations()[0] packageBytes, err := runnablespkg.CanonicalJSON(pkg) if err != nil { diff --git a/cmd/generate/runnables_audience_test.go b/cmd/generate/runnables_audience_test.go new file mode 100644 index 000000000..7270322a0 --- /dev/null +++ b/cmd/generate/runnables_audience_test.go @@ -0,0 +1,82 @@ +package generate + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + + runnablespkg "github.com/codefly-dev/cli/pkg/runnables" + modulecontract "github.com/codefly-dev/core/contracts/module" + "github.com/stretchr/testify/require" +) + +func TestRunnableAudienceUsesPublishedPrincipalWithoutRenamingModule(t *testing.T) { + ctx := context.Background() + contract := withHopPolicy(t, descriptorSet(t, ingestionFile(conformingOperation(), hopPolicyMethod("WriteText", "documents", "write")))) + root, moduleDir := saveRunnableFixture(t, ctx, contract, "0.1.0") + t.Chdir(root) + resetRunnablesFlags(t) + require.NoError(t, RunnablesCmd.RunE(RunnablesCmd, []string{"documents"})) + require.Equal(t, "documents", readIndex(t, moduleDir).Module) + for _, method := range []string{"ApplyText", "WriteText"} { + data, err := os.ReadFile(filepath.Join(moduleDir, runnablespkg.DerivedDir, "runtime-worker", "grpc", method, runnablespkg.OperationFileName)) + require.NoError(t, err) + var operation map[string]any + require.NoError(t, json.Unmarshal(data, &operation)) + require.Equal(t, "documents.ingestion", operation["audience"]) + } +} + +func TestRunnableAudienceRefusalsLeaveOutputUnchanged(t *testing.T) { + for _, test := range []struct { + name string + edit func(*testing.T, string) + want string + }{ + {"missing", func(t *testing.T, dir string) { + require.NoError(t, os.Remove(filepath.Join(dir, modulecontract.FileName))) + }, "needs its owner's module contract"}, + {"malformed", func(t *testing.T, dir string) { + writeFixtureFile(t, filepath.Join(dir, modulecontract.FileName), []byte("schema: codefly/module-contract/v1\nprincipal: documents\nunknown: true\n")) + }, "needs its owner's module contract"}, + {"uncovered destination", func(t *testing.T, dir string) { + saveRunnablePublication(t, dir, "documents.ingestion", modulecontract.Destination{ID: "other", Service: "runtime-worker", Endpoint: "other", Kind: "module"}) + }, "does not publish runnable destination runtime-worker/grpc"}, + {"explicit conflict", func(t *testing.T, dir string) { + saveRunnablePublication(t, dir, "other", modulecontract.Destination{ID: "grpc", Service: "runtime-worker", Endpoint: "grpc", Kind: "module"}) + }, "differs from module contract principal"}, + {"escaped publication", func(t *testing.T, dir string) { + outside := filepath.Join(t.TempDir(), "contract.yaml") + data, err := os.ReadFile(filepath.Join(dir, modulecontract.FileName)) + require.NoError(t, err) + writeFixtureFile(t, outside, data) + require.NoError(t, os.Remove(filepath.Join(dir, modulecontract.FileName))) + require.NoError(t, os.Symlink(outside, filepath.Join(dir, modulecontract.FileName))) + }, "needs its owner's module contract"}, + } { + t.Run(test.name, func(t *testing.T) { + ctx := context.Background() + root, moduleDir := saveRunnableFixture(t, ctx, descriptorSet(t, ingestionFile(conformingOperation())), "0.1.0") + t.Chdir(root) + resetRunnablesFlags(t) + require.NoError(t, RunnablesCmd.RunE(RunnablesCmd, []string{"documents"})) + before := snapshotTree(t, moduleDir) + test.edit(t, moduleDir) + require.ErrorContains(t, RunnablesCmd.RunE(RunnablesCmd, []string{"documents"}), test.want) + require.Equal(t, before, snapshotTree(t, moduleDir)) + }) + } +} + +func TestRunnableAudienceChecksOpenAPIPublication(t *testing.T) { + root, moduleDir := saveSolutionFixture(t, context.Background(), releasesDocument(t, nil)) + t.Chdir(root) + resetRunnablesFlags(t) + require.NoError(t, RunnablesCmd.RunE(RunnablesCmd, []string{"releases"})) + before := snapshotTree(t, moduleDir) + saveRunnablePublication(t, moduleDir, "other", modulecontract.Destination{ID: "http", Service: "backend", Endpoint: "http", Kind: "module"}) + require.ErrorContains(t, RunnablesCmd.RunE(RunnablesCmd, []string{"releases"}), "differs from module contract principal") + require.Equal(t, before, snapshotTree(t, moduleDir)) +} diff --git a/cmd/generate/runnables_default_test.go b/cmd/generate/runnables_default_test.go index 0fd8ecea4..82017fe9f 100644 --- a/cmd/generate/runnables_default_test.go +++ b/cmd/generate/runnables_default_test.go @@ -104,7 +104,7 @@ func TestGenerateRunnablesDerivesEveryMethodBehindTheHop(t *testing.T) { require.Equal(t, "RECOVERY_RECOMPUTE", count["recovery"]) require.Empty(t, count["lookup_scopes"]) require.Equal(t, "EFFECT_READ_ONLY", count["tool"].(map[string]any)["effect"]) - require.Equal(t, "documents", count["audience"]) + require.Equal(t, "documents.ingestion", count["audience"]) write := read("WriteText") require.Equal(t, "RECOVERY_RECEIPT", write["recovery"]) diff --git a/cmd/generate/runnables_fixture_test.go b/cmd/generate/runnables_fixture_test.go index 68ba726f4..23f820acf 100644 --- a/cmd/generate/runnables_fixture_test.go +++ b/cmd/generate/runnables_fixture_test.go @@ -8,6 +8,7 @@ import ( "time" "github.com/codefly-dev/core/composition" + modulecontract "github.com/codefly-dev/core/contracts/module" basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" runnablev0 "github.com/codefly-dev/core/generated/go/codefly/runnable/v0" "github.com/codefly-dev/core/resources" @@ -16,6 +17,7 @@ import ( "google.golang.org/protobuf/reflect/protoregistry" "google.golang.org/protobuf/types/descriptorpb" "google.golang.org/protobuf/types/known/durationpb" + "gopkg.in/yaml.v3" ) // The derivation reads descriptors, so the fixtures are descriptors. Building @@ -213,6 +215,11 @@ func saveRunnableFixture(t *testing.T, ctx context.Context, contract []byte, man ) publishContract(t, moduleDir, contract) + destinations := []modulecontract.Destination{{ID: "grpc", Service: "runtime-worker", Endpoint: "grpc", Kind: "module"}} + for _, endpoint := range beside { + destinations = append(destinations, modulecontract.Destination{ID: endpoint.Name, Service: "runtime-worker", Endpoint: endpoint.Name, Kind: "module"}) + } + saveRunnablePublication(t, moduleDir, "documents.ingestion", destinations...) if manifestVersion != "" { writeFixtureFile(t, filepath.Join(moduleDir, composition.PackageManifestFileName), []byte( @@ -222,6 +229,19 @@ func saveRunnableFixture(t *testing.T, ctx context.Context, contract []byte, man return root, moduleDir } +func saveRunnablePublication(t *testing.T, moduleDir, principal string, destinations ...modulecontract.Destination) { + t.Helper() + data, err := yaml.Marshal(&modulecontract.Contract{ + Schema: modulecontract.SchemaV1, Principal: principal, + Namespaces: []string{}, Queues: []string{}, ScopeCeilings: []modulecontract.ScopeCeiling{}, + Bindings: []modulecontract.Binding{}, Destinations: destinations, + }) + if err != nil { + t.Fatal(err) + } + writeFixtureFile(t, filepath.Join(moduleDir, modulecontract.FileName), data) +} + // publishContract writes one endpoint's contract and the catalog that names // it, the way `codefly generate contracts` leaves them behind. func publishContract(t *testing.T, moduleDir string, contract []byte) { diff --git a/cmd/generate/runnables_solution_test.go b/cmd/generate/runnables_solution_test.go index a8bc9af53..5fddb9d03 100644 --- a/cmd/generate/runnables_solution_test.go +++ b/cmd/generate/runnables_solution_test.go @@ -9,6 +9,7 @@ import ( "testing" "github.com/codefly-dev/core/composition" + modulecontract "github.com/codefly-dev/core/contracts/module" basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" runnablev0 "github.com/codefly-dev/core/generated/go/codefly/runnable/v0" "github.com/codefly-dev/core/resources" @@ -121,6 +122,7 @@ func saveSolutionFixture(t *testing.T, ctx context.Context, document []byte) (ro }, ) publishSolutionContract(t, moduleDir, document) + saveRunnablePublication(t, moduleDir, "releases", modulecontract.Destination{ID: "http", Service: "backend", Endpoint: "http", Kind: "module"}) return root, moduleDir } @@ -296,6 +298,7 @@ func saveCommittedSolution(t *testing.T, ctx context.Context, artifact []byte) ( }, }, ) + saveRunnablePublication(t, moduleDir, "releases", modulecontract.Destination{ID: "http", Service: "backend", Endpoint: "http", Kind: "module"}) writeFixtureFile(t, filepath.Join(moduleDir, "services", "backend", filepath.FromSlash(standards.OpenAPIPath)), artifact) writeFixtureFile(t, filepath.Join(moduleDir, composition.PackageManifestFileName), []byte(`schema: codefly/module-package/v2 kind: module-package diff --git a/docs/runnable.md b/docs/runnable.md index 2a6d00e70..578168245 100644 --- a/docs/runnable.md +++ b/docs/runnable.md @@ -43,6 +43,14 @@ owns the option, the descriptor-to-bounded-schema projection and published contracts, the layout on disk and the drift gate. Nothing about the contract is authored twice — it is the projection of the method's own messages. +The selected owner's `module.contract.codefly.yaml` supplies the audience through +Core's strict module-contract reader. Its principal is independent of the +producer checkout's module name; generation keeps the package's module identity +unchanged. Every generated destination must be published by that contract, and +an explicit operation audience must match its principal. Missing, unreadable or +contradictory publication refuses generation before output changes. A module +with no operations still writes an empty index. + Derivation is endpoint-specific: `catalog.codefly.json`'s `endpoints[].services[].procedures` declares the served surface; descriptor bytes supply each method's option and schema. Sharing a descriptor does not make every From dbc000aebba76a615f87cc57d46369ae747efb23 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 11:25:08 -0400 Subject: [PATCH 18/18] test(doctor): declare runnable owner in binding fixture --- cmd/doctor_workspace_binding_values_test.go | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/cmd/doctor_workspace_binding_values_test.go b/cmd/doctor_workspace_binding_values_test.go index 5a399d916..035da4c48 100644 --- a/cmd/doctor_workspace_binding_values_test.go +++ b/cmd/doctor_workspace_binding_values_test.go @@ -28,6 +28,18 @@ func TestDoctorUsesEffectiveRunnableValueAndPreservesAuthoredFile(t *testing.T) - service: api endpoint: http visibility: public +`, + "modules/backend/module.contract.codefly.yaml": `schema: codefly/module-contract/v1 +principal: releases +namespaces: [] +queues: [] +scope_ceilings: [] +bindings: [] +destinations: + - id: http + service: api + endpoint: http + kind: module `, "modules/backend/services/api/openapi/api.swagger.json": string(artifact), })