Skip to content

feat(configuration): deliver public module artifacts through ordinary loads - #1213

Merged
antoinetoussaint-byte merged 4 commits into
mainfrom
configuration/module-artifacts-through-ordinary-loads
Oct 28, 2026
Merged

antoinetoussaint-byte merged 4 commits into
mainfrom
configuration/module-artifacts-through-ordinary-loads

Conversation

@antoinetoussaint-byte

Copy link
Copy Markdown
Contributor

Core acquires a bound module artifact per ordinary configuration load (core#865, v0.19.11-0.20261028153419-de19eba80550); this is the consumer half. Classification: feat at the CLI-owned configuration delivery boundary, carrying two fixes that share the same Init/Start seam — one of them a local-startup blocker. Related: #932, supersedes the artifact half of the merged #934 lineage.

Reuses core's ModuleArtifactResolver/WithModuleArtifactResolver, ValidateModuleArtifactDelivery/Reference/Bytes/Value and FileCarrierThreshold, plus the materializer's own provenance predicates (ResolutionModeFor, pinnedManaged, moduleReceiptAnswers) rather than re-deriving them. No new Core API, no resolver framework, no lifecycle subsystem.

Module artifacts through ordinary loads

A workspace configuration value may be bound to a file in a composed module, declared in the selected profile's module-artifacts.codefly.json. Each ordinary load acquires afresh and carries the exact bytes with the identity core computes from them; a standalone check describes its own observation and proves nothing about a later delivery.

  • The resolver is supplied wherever a configuration is loaded: the run/check reader, the doctor's read, the remote manager, generate runnable-bindings, the four gitops readers and environments.ValidateSecretPolicy (which takes it as a parameter, since it sits below pkg/orchestration).
  • Acquisition opens the declared path confined under the selected module with os.Root, non-blocking, regular-file checked before any byte is read, bounded at the inline threshold, with same-open-file observations around the read and the selection re-asked afterwards. A CLI-materialized path needs a receipt answering the request the workspace makes now, and the resolved directory must be exactly that receipt's (filepath.Clean equality — the receipt records the module dir with the subpath already joined, so nothing legitimately nests).
  • ValidateModuleArtifactDelivery runs on the complete unmerged payload at Init, at the Init response, at Start and at the excluded-root export, with the phase override map computed once and reused by the RPC — and on both public typed exports before any filter, merge or conversion, so a direct caller cannot bypass it.
  • Only a successful Init retains a detached baseline. Start refuses without one rather than handing core a nil comparison, which core reads as "no bound destinations" and passes.
  • Init's image work now runs after the delivery check, so a payload the run will refuse leaves no built image behind.
  • The merged fixture composition no longer replaces a group a module artifact is bound to; it refuses with ErrConfigurationConflict.

Whole-record --output-env limit

The raw batch is held to the grammar sdk-go's reader actually parses — CODEFLY__ plus [A-Z0-9_], and no LF/CR/NUL in any value — across every record and before the first write. One serialization is checked and retained through the write, and the shared writer re-checks both supplied and retained records, so a malformed retained line is refused rather than reproduced.

This surfaced a pre-existing defect, which is not the CLI's to fix: resources.ConfigurationDocumentKey is a prefix plus a lowercase hex SHA256, which that grammar refuses — so a run exporting a structured configuration document (platform-core's merged fixture-composition is exactly that shape) has always written a file the reader rejects wholesale. The CLI now refuses before writing instead of reporting success. Fixing it needs an uppercase document carrier in core or a reader in sdk-go that accepts this one.

Two consequences worth naming: --output-env with a --set whose key is not a CODEFLY__ carrier now refuses rather than writing an unreadable file, and such a document cannot be exported this way at all.

Local runtime image handoff (fix)

The agent's advertisement of runtime-init-image/v1 — not the cardinality of its build plan — decides whether the run relies on the image handoff. A database agent emits exactly one recipe for its migration image; reading the runtime role off that sole recipe built the wrong image and then failed the run for a capability the agent never claimed, which is what blocked local startup for a container-only store. Checked before the builder is loaded or planned, so a non-advertising agent's builder is never touched. Only the absence of the optional capability is waived: the gate tests the capability list rather than contract.Check, because Check reports a protocol-version mismatch with the same ErrIncompatible, and the strict guard in buildLocalRuntimeImage still runs the full check once the handoff is selected.

Runnable audiences (fix)

An explicitly annotated audience may name a host surface the module reaches without being installed as one, so it needs no module.contract.codefly.yaml; its scopes and lookup are carried through as core derived them. A default-derived audience still requires the real principal. Where a contract does exist it governs every operation — destination coverage and principal equality both still refuse a real conflict rather than reading it as host permission. A declaration that exists and cannot be read, a dangling in-root symlink included, is never absence: core's loader reports it as fs.ErrNotExist, so a confined Lstat observation separates the two while core keeps the parse. Declared marks the option's presence, not an audience's.

Verification

All GOWORK=off against the released pin:

  • go build ./..., go vet ./... clean. The disposable_engine_test.go copylock is fixed by pointer, not suppressed: the helper overrides and restores two fields on the scoped builder's own *World instead of copying a World carrying a sync.Once.
  • golangci-lint run ./pkg/... ./cmd/... → 0 issues. gofmt clean.
  • pkg/composition ok, pkg/gitops ok (126s), pkg/environments ok, cmd/environment ok, cmd/generate ok, and the pkg/orchestration artifact/output/runtime-image/audience sets ok.
  • Adopting the real core pin also turned four previously-red GOWORK=off subprocess tests green, and moved pkg/conformance/matrix.json's core line to v0.19.11 for TestMatrixCoreMatchesGoMod.

New coverage: exact-byte acquisition; absolute/traversing/symlink-escape/FIFO/directory/missing/empty-path/unknown-module refusals with no content in the diagnostic; oversize vs inclusive threshold; unmaterialized module; receipt/resolution divergence; every-offer-preserved baseline; detached retained delivery with evidence intact; end-to-end bound delivery with its identity; changed source on the next load; literal-vs-binding conflict; missing artifact naming module and path; check-load own observation; additional-offer and override collisions including byte-equal; excluded-root refusal before any write; the LF-smuggling counterexample and the CR/NUL/key-grammar table with absent-stays-absent and existing-bytes-identical; retained malformed records; writer/reader grammar agreement; the document-carrier refusal; non-advertising agent with zero builder traffic; the agent's own Init error propagated; a bootstrap-named recipe; incompatible advertising agent; dangling declaration; annotated-empty audience.

Limits

Not claimed: any live run, deploy, Docker or engine qualification — the whole pkg/orchestration package has not been rerun since the engine slot returned, because that package provisions a buildx builder and builds disposable images; no SDK process byte roundtrip; no runtime installation or deployment. Root sdk-go was deliberately not added as a test dependency, so the reader grammar is transcribed in the test file with an agreement assertion against production rather than an in-process roundtrip. The four pkg/conformance failures seen locally are pre-existing and environment-only: they scan the repo tree and trip over an untracked .lazybox/checkouts/.

Delivered on a new branch because #934 landed by squash, so dbc000ae is not an ancestor of main and pushing here would have rewritten a merged PR's branch history.

🤖 Generated with Claude Code

Antoine Toussaint and others added 4 commits October 28, 2026 12:54
… loads

Core acquires a bound module artifact per ordinary configuration load; this is
the consumer half. Three CLI-owned corrections ride together because they share
the Init/Start delivery seam and one of them is a local-startup blocker.

Module artifacts. A workspace value may be bound to a file in a composed
module, declared in the selected profile. The resolver is supplied wherever a
configuration is loaded, so run delivery, checks, render and the remote manager
acquire alike. The declared path is opened confined under the selected module,
non-blocking, regular-file only, bounded, with same-open-file observations
around the read and the selection re-asked afterwards; a CLI-materialized path
needs a receipt answering the request the workspace makes now, and the resolved
directory must be that receipt's. The delivery check runs on the complete
unmerged payload at Init, the Init response, Start and the excluded-root export,
with the override map computed once and reused by the request, and on both
public typed exports before any filter or conversion. Only a successful Init
retains a detached baseline; Start refuses without one rather than handing Core
a nil comparison. The merged fixture composition no longer replaces a group a
module artifact is bound to.

Output records. The raw --output-env batch is held to the grammar the supported
reader parses, across every record and before the first write, so a value
carrying a line break can no longer smuggle a second record past semantic key
matching. One serialization is checked and retained through the write, and the
shared writer re-checks supplied and retained records. This surfaces that a
structured configuration document was never loadable from such a file: its
carrier is a lowercase hex digest the reader refuses.

Local runtime image. The agent's advertisement of runtime-init-image/v1, not
the cardinality of its build plan, decides whether the run relies on the image
handoff — a database agent's sole recipe is its migration image, and reading
the runtime role off it built the wrong image and then failed the run for a
capability the agent never claimed. Checked before the builder is loaded, and
only the absence of the optional capability is waived.

Runnable audiences. An explicitly annotated audience may name a host surface
the module reaches without being installed as one, so it needs no module
contract; a default-derived audience still requires the real principal, and an
existing contract still governs destinations and principal equality. A
declaration that exists and cannot be read — a dangling in-root symlink
included — is never absence.

Adopts core v0.19.11-0.20261028153419-de19eba80550 and moves the conformance
matrix core line with it. Updates docs/configuration-contract.md,
docs/commands.md and docs/runnable.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…fact read

The post-read recheck compared the resolved directory and discarded the
provenance the selection had already computed, so it could not see a selection
that moved without its directory moving.

ResolveModule answers from the overlay snapshot the workspace was loaded with.
A non-managed selection — an explicit path or a worktree, which carries no
receipt — therefore resolves to the same directory on both asks even after the
overlay on disk has been re-pointed, and a selection can change mode or the
request it answers while keeping its directory. Either way the path matched and
the read was attributed to a selection this invocation no longer makes.

SelectModule now returns a comparable descriptor carrying the existing owner
evidence beside the directory: the disk-read directive, the receipt's request
and what it produced, the derived mode, and the composed reference. The
resolver holds it across the read and compares it whole. No new framework, no
new source of truth, and every refusal is the one that already existed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The retained-delivery guard sat ahead of the output gathering, so a runner with
no consumer module — which must refuse rather than write dependency addresses
whose visibility nothing can check — reported the missing delivery instead of
the missing module, and TestOutputEnvRefusesToWriteWithoutAConsumerModule went
red on CI.

The guard now sits immediately before the validation it protects. The gathering
only reads and converts, writing nothing, so no no-write guarantee moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A directive path is relative to the overlay file that declares it, so a parent
overlay and a nearer workspace overlay saying the same `path: modules/wiki`
select different directories. The descriptor compared the raw path text, which
is identical in both, and the resolved directory, which is also identical
because ResolveModule answers from the snapshot the workspace was loaded with
— so a nearer overlay appearing mid-read moved the selection and compared
equal.

The selection now carries the directory its overlay and resolution record were
read from, which is NearestOverlayDir's existing answer and already computed by
loadMaterializationState. No new framework and no new source of truth.

Closes the last case from the independent selection-provenance probe; its
TestIndependentOverlayOriginChange and TestIndependentSamePathReceiptChange
both pass, and the behaviour has its own regression here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@antoinetoussaint-byte
antoinetoussaint-byte added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 9608688 Oct 28, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant