Skip to content

runnable.md

Latest commit

 

History

History
464 lines (390 loc) · 26 KB

runnable.md

File metadata and controls

464 lines (390 loc) · 26 KB
Raw

Runnables in the CLI

A Runnable is a packaged implementation of a typed finite operation: typed input in, typed output out, declared dependencies and declared effect semantics. It is declared in runnable.codefly.yaml and owned by a module, next to that module's services and jobs.

Core owns the resource, the wire contracts and the package/binding verification — see core's docs/runnable.md for the declaration format, the bounded schema profile and the installation facts. This page covers what the CLI does with them.

Runnables are experimental. This page states what is implemented and what is not; nothing here is a promise about the unimplemented parts.

What the CLI implements today

Command What it does
codefly add runnable <name> --agent=<name>:<version> --handler=<path> Create a declaration and scaffold through the real Builder agent
codefly build runnable <name> [--output=<new-directory>] [--json] Prepare and package through the agent, verify archive bytes, write artifacts/runnable-package.json
codefly generate runnables [module] [--check] Derive a SERVICE-facility package for every gRPC method a module's contracts mark with the operation option
codefly generate runnable-bindings [--env=<env>] [--naming-scope=<scope>] [--call-endpoint=<module>/<service>/<endpoint>] [--check] Resolve the environment's scope selections, owner endpoints and declared runnable-call-endpoint into prepared bindings
codefly list runnables [--module=<m>] [--json] List the workspace's runnables with their immutable identity, pinned agent, execution facilities and timeout
codefly show runnable <name> [--version=<v>] [--json] Show one runnable's contract, execution bounds and dependency resolution
codefly agent install <language>:<version> --kind=runnable Download a released runnable language agent into the local Codefly cache
codefly agent info runnable --agent=<language>:<version> Load that agent over gRPC and print what it reports

The language is the agent's name, never a branch in the CLI: a runnable agent resolves to the runnable-<language> repository and executable, and installs under runnables/, through the same agent-kind registry that resolves service agents.

Derived SERVICE-facility packages

A unary, idempotent method an owner service already publishes becomes a Runnable by derivation, not by authoring. The method carries core's codefly.runnable.v0.operation option; codefly generate runnables reads it off the contract.binpb that generate contracts already published and writes a SERVICE-facility package per marked method under contracts/runnables. Core owns the option, the descriptor-to-bounded-schema projection and runnable.PackageFromMethod; the CLI owns running them over a module's published contracts, the layout on disk and the drift gate. Nothing about the contract is authored twice — it is the projection of the method's own messages.

Where an operation's audience comes from decides what the module must publish.

A method that annotates its audience explicitly (Core sets OperationSpec.Declared from the method option) carries the author's own audience. That may be a host surface the module reaches without being installed as one — a source-operation audience, say — so Core does not require it to equal any module principal, and neither does generation: such an operation needs no module.contract.codefly.yaml at all. Its served target, scopes and lookup are still checked as always, so an unserved method or a missing paired Lookup refuses.

A method that carries no operation option takes Core's default from ServiceOwner.Audience, which generation fills from the selected owner's module.contract.codefly.yaml through Core's strict module-contract reader. There the contract is required: with no publication there is no principal to derive from, and generation refuses rather than emit an empty audience.

The distinction is the option's presence, not an audience's: Declared marks that the method carries the option. Core validates the audience inside it before setting Declared, and generation refuses an annotated method whose audience is empty rather than treating it as either case — it has no principal to check against and nothing of its own to carry.

When that contract does exist, the module's publication applies to every operation it serves, explicit ones included: each generated destination must be published by it, and an audience differing from its principal is a conflict in the module's own declarations rather than host permission. Its principal is independent of the producer checkout's module name; generation keeps the package's module identity unchanged.

An unreadable, malformed or escaped contract is never read as absence — a module that authored one and got it wrong is refused by name. Every refusal happens before output changes. A module with no operations still writes an empty index.

Derivation is endpoint-specific: catalog.codefly.json's endpoints[].services[].procedures declares the served surface; descriptor bytes supply each method's option and schema. Sharing a descriptor does not make every method callable on every listener. Unknown catalog methods are refused, and a marked method's paired Lookup must also be served on that endpoint. Removing a method from an endpoint removes its derived package; --check detects the change. Owners with restricted listeners generate the procedure inventory from their registration/routing tables and configure the module's contracts.codefly.yaml before exporting.

A solution's routes are the same derivation read from its OpenAPI document rather than from descriptors: generate runnables derives every route marked x-codefly-operation with core's runnable.PackageFromOpenAPIOperation, into the same tree and index, and generate runnable-bindings prepares each keyed <SOLUTION>__<OPERATION> with a rest route — the verb and path — at the solution's http endpoint. solution-runtime-go marks every read a backend serves, so a solution's GET reads become read-only Runnables, and so tools, with nothing hand-written (#1184).

This is a separate path from build runnable, not a branch in it. A SERVICE package has no archive and no launch command: the implementation is the method itself, reached on the owner's endpoint, inside the process the owner already operates. pkg/runnable.assemble only ever emits NATIVE artifacts and is untouched.

Core message fields in operation.json use protobuf JSON, including named tool effects. Unknown nested fields are refused so a newer declaration cannot lose authority metadata silently.

The package is the contract; the execution policy and the authority a binding is minted for sit beside it in operation.json, because they are installed with the binding — two installations of one contract may run under different ones, and digesting them in would make those two installations two releases.

An index.json is data on disk, and in a composed workspace it arrives inside a third-party module package, so every path read out of one is resolved inside the module directory through os.Root — a row naming ../../../.ssh/id_rsa, or an ordinary-looking directory that is a symlink out of the tree, is refused rather than followed.

See generate runnables for the output layout, the refusal rules and how the release version is derived. What this does not do is invoke anything: no CLI command invokes a Runnable, derived or authored. Preparing each operation's binding for an environment is generate runnable-bindings.

Required scope slots and tool exposure

generate runnables preserves every field of Core's Operation in operation.json, including required_scope_slots and optional tool exposure. A slot-only operation may declare no fixed scopes. Generation keeps its slots unresolved; it cannot choose resource IDs for the installing composition.

The composition answers each slot in the existing workspace.codefly.yaml environment selected by generate runnable-bindings --env. Under runnable-scope-selections, the key is the output binding key: <MODULE>__<OPERATION>, upper-cased with non-alphanumeric characters replaced by underscores. The operation name comes from contracts/runnables/index.json. Each value is a list of Core ScopeSelection messages (slot, invoke, lookup), using Core's scope fields. For example:

environments:
  - name: staging
    runnable-scope-selections:
      SAAS_STARTER__ACCOUNTS_CONNECT_INVOKE_SOURCE_OPERATION:
        - slot: source
          invoke:
            - resource_kind: datasource.sources
              actions: [invoke, read]
              resource_ids: [source-a]
          lookup:
            - resource_kind: datasource.sources
              actions: [read]
              resource_ids: [source-a]

source-a is an illustrative ID; select the exact resources this installation may use. Quote IDs that look like YAML numbers. The owner of InvokeSourceOperation requires source with actions invoke and read, and lookup: true, so its lookup must cover the same exact IDs with action read. When an operation is actually served on several published endpoints, each index row is a binding and needs its own selection. Saas-starter serves this operation and its Lookup on connect; its authority listener does not serve them even though both endpoints carry the complete descriptor. Receipt pruning is a separate operation and selection key, SAAS_STARTER__ACCOUNTS_CONNECT_PRUNE_SOURCE_OPERATION_RECEIPTS.

codefly generate runnables saas-starter
codefly generate runnable-bindings --env staging
codefly generate runnable-bindings --env staging --check

Preparation calls Core's OperationSpec.ResolveScopeSlots once per operation and delivers its Policy(): fixed scopes stay in place, selected scopes are appended, and no unresolved slot reaches a prepared binding. Missing selections name the operation, slot and runnable-scope-selections key to set. An owner declaring neither fixed scopes nor slots is still refused as declaring no authority. Wildcards (including omitted IDs), collisions with a fixed scope's kind or another slot's kind, duplicate/undeclared slots, unknown fields and selection keys naming no derived operation are refused. A failed preparation writes no binding output and preserves the existing binding; a pinned module it materialized first stays materialized (dependency cache, overlay, receipt and .gitignore entries), as after any command that reads composed modules.

The generated configurations/<profile>/runnable-bindings.env carries Core's codefly.runnable-prepared/v3 bindings: owner coordinates, the resolved JSON call target, bounded contract and digest, and concrete policy. Tool metadata survives preparation; absence remains ErrNotATool, and exposure grants no authority. A gRPC operation is called only on a Connect endpoint whose published served set contains both the operation and its Lookup; no match or multiple matches are refused. A directly published Connect operation uses that endpoint with the same membership checks. Legacy gRPC-only catalogs must publish the called Connect surface before binding. Endpoint addresses still come from the existing network resolver, in the run's naming scope; see naming scope and a declared call endpoint.

An environment that prepares no binding has no file when no module derives an operation. That absence is an answer, not a missing file: a service that declares runnable-bindings receives nothing for the group on a run, a build and a render, and reads every key as not configured, and codefly doctor workspace accepts the group absent or empty exactly when the environment selects no binding. A selected binding the group does not hold fails there, naming codefly generate runnable-bindings --env <env>.

Binding and selection keys contain the derived service/endpoint/method name. Moving a method between endpoints changes its key: update the composition's selections and consumers and regenerate. Old selection keys are refused as unused; they do not silently grant authority to the new identity. Publishing a method on two endpoints gives two identities with separate selections.

Declining a binding in an environment

A composition that cannot name the resources a slot needs answers the key with an explicitly empty list. The typical case is a base composition: the host's InvokeSourceOperation needs exact source IDs, but sources are tenant data created at runtime, so a composition of modules alone holds none. Composed as saas, it writes:

environments:
  - name: local
    runnable-scope-selections:
      SAAS__ACCOUNTS_CONNECT_INVOKE_SOURCE_OPERATION: []

[] means "this environment installs this operation for no resource":

  • that operation gets no prepared binding: its key is absent from runnable-bindings.env, and it receives no authority;
  • every other derived operation still prepares, so one operation the composition cannot answer no longer fails every binding with it;
  • the file records the decline as a comment line naming the key — and no environment, since the file belongs to a configuration profile several environments may share, so two of them that resolve the same addresses and decline the same key write the same bytes and --check agrees for both — generation output names it, and codefly doctor workspace --env <env> reports it as a runnable_binding_declined warning. The doctor judges each decline with the rule generation applies: a decline generation refuses (see below) is a runnable_binding_decline_invalid failure, never a warning.

Only [] declines. An absent key is still refused exactly as before, so nothing is ever skipped by omission; a null value (KEY: or KEY: ~) is refused when the environment is read, because it would look like either. [] on an operation that declares no required scope slot is refused, since there is nothing to decline and it would hide a typo, and [] on a key that names no derived operation is refused as an unknown key.

A module whose service installs prepared bindings asks for each one in its manifest's authority request, by its key, with the binding's audience, kinds and actions and no resource id; the module's installation scope narrows it to the resources an organization consents to (module manifests). A declined operation has no prepared binding, so nothing asks for it.

A service that installs a declined operation is refused at its own point: the runnable-bindings key it names is not configured, and it reports that key. The decline is the environment's own answer, so the same operation can be selected in another environment that does hold resource IDs. To install it, replace [] with selections and regenerate.

This is not a new vocabulary: the value is still a list of Core ScopeSelection messages, and the empty list is the one answer that selects nothing. Core's resolution is unchanged; a declined operation is simply never handed to it.

Changing a selection requires regeneration, and --check detects that change, including a decline added or removed. The configuration profile chooses the output directory; selections belong to the named environment even when environments share a profile. These declarations are installation input and are not sent to agents as runtime configuration. There is no additional selection file, CLI policy vocabulary or CLI scope resolver. Tests verify the prepared binding and the identical policy as a Core resolved-policy receipt; this command emits the binding configuration only.

Naming scope and a declared call endpoint

A local binding is read by a run, so its address is the one that run binds. --naming-scope follows codefly run: omitted, the environment's declared naming-scope applies; passed, it replaces it for this invocation; passed empty, it clears it. A live run of that scope answers with its concrete allocation, and with no run the address is the one a run of the scope will bind (the resolver codefly endpoint uses). Generate with the scope you run with:

codefly generate runnable-bindings --env local --naming-scope demo
codefly run service ... --env local --naming-scope demo

An environment declares the endpoint every call prepared for it is sent to instead of the owner's listener — the composition's hop in front of its modules, for example:

environments:
  - name: local
    runnable-call-endpoint: authority/hop/http

Declared there, a plain codefly generate runnable-bindings and its --check prepare through it, so the committed file depends on the workspace and not on a flag retyped per invocation. --call-endpoint <module>/<service>/<endpoint> overrides the declaration for one invocation (--call-endpoint "" sends each call to its owner), and an environment that declares nothing keeps working with the flag alone; --check names the flag in its regeneration command only when the file was prepared for an endpoint other than the declared one. The declaration is installation input: it is never sent to an agent. The coordinate is parsed as an endpoint reference, must name a declared endpoint of a workspace module, and must be HTTP-based (http, connect, rest or mcp), because a prepared call carries an HTTP base URL; it resolves in the same environment and scope as the owners. Nothing infers such an endpoint: the composition that runs the hop names it.

Only the address moves. Each owner is still loaded and held to its served Connect surface (operation and Lookup) and its published contract before the address is replaced, so a hop never makes an unserved or stale operation look callable. The operation, the Connect procedure or REST verb and path, the contract, its digest and the policy stay the owner's. One generation names at most one call endpoint, and the output file is the profile's: two generations for different scopes into one profile overwrite each other, so parallel runs use separate workspaces or profiles.

Identity and coexistence

A runnable is identified by workspace/module/name @ version. A version is an immutable release, and two versions of one name coexist: both listings and show report the version, and nothing in the CLI resolves a name to a "latest" release.

Because a name can therefore stand for more than one release, show runnable refuses an ambiguous name instead of picking one — it lists every candidate as module/name@version and --version selects one. A name that resolves to a single release needs no flag.

One projection per runnable

codefly list runnables --json, codefly show runnable --json and the MCP list_runnables tool emit the same identity, agent, protocol and execution fields, from one shared projection (pkg/runnables). show adds the contract, entrypoint and dependency report on top; it does not restate the shared fields differently.

Derived operations are listed through that same projection, so an operator sees what a module exposes without reading the generated JSON. They carry facility service and a source of <service>/<endpoint>/<Method>, which is empty for an authored runnable — the distinction a reader needs, since a derived row is regenerated from a contract rather than edited. show renders a derived operation's method, the published messages it reuses, and the policy and authority in its operation.json; it renders no handler and no dependency report, because a derived operation is reached on a service the workspace already runs.

Listing is strict

A runnable whose declaration does not load fails codefly list runnables rather than shortening the list. A caller about to build or install what it finds must not be told a broken runnable is absent.

Dependency resolution is a report, not a gate

codefly show runnable resolves each declared service dependency against what the workspace declares and reports it as resolved or unresolved with the reason. It mirrors core's binding rule for runnables (runnable/package.go): endpoints are matched by NAME, because resources.Runnable.Proto flattens each selector to its endpoint name on the wire; a runtime edge needs at least one endpoint, while a legacy or external edge needs one only when it names endpoints explicitly. A selector that names no endpoint (- api: grpc) is reported as ::grpc and unresolved, since it flattens to an empty name that binds nothing. It deliberately does not fail on an unresolved dependency: the declaration is still valid, and a caller needs to see which of several dependencies is missing. Reachability, credential resolution and platform compatibility are the installer's and launcher's responsibilities, not this command's.

What the CLI does not implement yet

None of the following exists in the CLI. There are no stub commands for them, because a stub would be indistinguishable from a capability:

  • Image builds. The current Runnable build command accepts native artifacts only. It rejects build/schema/completion prerequisites and internal libraries whose preparation is not implemented. (SERVICE-facility packages are implemented — see above — and need no build, because the owner's own builder already built the method.)
  • Register, activate, invoke, inspect. These are Orchestration surfaces. The CLI calls them; it does not host a Task/effect scheduler. The local runner below is the process boundary they dispatch through, not a way to invoke installed work from the command line: no CLI command invokes a Runnable.
  • The Kubernetes path. Orchestration's adapter owns durable invocation-to-Job identity, result handling and attempt policy.

Completing these paths requires coordinated work:

  • The shared contracts (create, build evidence, invocation framing) are tracked in codefly-dev/core#472.
  • The language agents live in runnable-python and runnable-go. Python implements native Builder gRPC and the shared invocation framing; Go currently has typed bindings.
  • The durable surfaces are Orchestration's, coordinated through obin-ai/module-runtime#63.

codefly-dev/cli#638 is the milestone that tracks all of them.

Native authoring and build

codefly add runnable word-count --agent=python:0.0.1 --handler=handler.py --json
# Edit runnables/word-count/runnable.codefly.yaml and its handler.
codefly build runnable word-count --output=/tmp/word-count-build --json

Both agent and handler are explicit; the CLI has no language-specific defaults. A compatible development agent is required until the fleet publishes qualified releases. Creation rolls back a failed agent call's module reference and source. Building preserves a failed explicit --output for inspection; choose a new directory to retry. The default output, .codefly/build/runnables/module/name/version, is a CLI-owned scratch directory and is replaced on every build, so the ordinary edit/rebuild loop never strands itself on its deterministic path. Release immutability lives in the package digest, which core's CompareRelease uses to tell an idempotent re-registration from a conflict.

The CLI sends Load(RunnableLocation), RunnableBuildInputs and Package over gRPC. It constructs the descriptor from the declaration, shared build evidence and PackageArtifact.command, validates it through core, and hashes both the declared build inputs and the actual archive before writing the descriptor. The input hashes are what tie the evidence to the author's source: without them a stale agent snapshot would report a stale handler digest inside a stale archive whose digest matches it, and nothing would notice the current source never shipped. Paths, interpreter names and language build tools come from the agent. No agent implementation is imported.

The local runner

Removed. pkg/runnable.NativeLauncher was the CLI's side of a physical process boundary: it started an installed native binding's command once per invocation, wrote invocation.json, read result.json, supervised the process group, enforced the deadline with SIGTERM then SIGKILL, and classified what it observed. It is gone with the native placement it served, and so is the Runnable native lifecycle workflow that qualified it.

Nothing replaces it here, because nothing needs to. A Runnable someone runs locally is a service they run locally: the archive codefly build runnable produces carries the generated harness, and the harness serves the contract over Connect/HTTP JSON. It is called the way an invocation Job's container is called — same transport, same headers, same Work Context — so the local path and the production path are one path instead of two. The launcher framing (codefly.runnable/v1, three CODEFLY__RUNNABLE_* variables and two documents on disk) existed only to bridge the difference that no longer exists.

The one thing the launcher owned that mattered beyond it was the line between a proven and an unproven effect: a handler that failed, a process that wrote nothing, an invalid document, a crash, a timeout, a cancellation, and the precedence among them. That distinction is what recovery depends on, so its served equivalent — RunnableServedOutcome, runnable.ClassifyServed and runnable.ServedOutcomeIsCertain — was written in core before any of this was deleted, not after.

What the CLI still owes, and does not yet have, is the command that unpacks a generated-service binding's archive and runs its harness. Until that lands, codefly build runnable produces and verifies the archive and nothing in the CLI starts it.

MCP create/build tools are deferred until the installation and execution surface is qualified. The CLI commands are available for unattended authoring now.