Repository navigation
Callable sources: OAuth2 (authorization code + client_credentials) for an API source, and InvokeSourceOperation — a connected API is called through the host, as the person, under receipts and audit #1049
Description
What we are trying to do
A solution connects a third-party HTTP API once — API key, or OAuth2 — and from then on its backend calls that API through the host via the SDK, and a robin assistant calls it as a tool, both as the asking person, both audited, both under receipts, with the credential never leaving this module and the model never choosing a URL. The host is the owner of the call because a Runnable owner must be a composition endpoint under the mesh dial policy and SP-CELL-03 egress; a third-party API can never be the owner.
Design, measured 2026-10-09 with every file:line: design/callable-sources.md (handbook branch rest-datasource, commit 26c2a93). Read it first; this issue is its H1–H9 rows made concrete for this repo. The handbook contract moves in the sibling handbook issue (proposal); do not wait for it to start — implementation is local first — but do not tag a release before the proposal is accepted.
Sibling issues
Filled in by a comment below once every issue exists.
What is true today (saas-starter main f5a104c, 2026-10-08)
module/services/accounts/code/pkg/datasource/apisource/client.go— the "generic HTTP API with a stored credential" connector: oneGET base_url+resource_path, body ≤ 5 MiB (maxResponseBytes), credential kindsbearer|basic|header|query, OAuth2 handled by the business layer as a bearer;guardDialrefuses loopback/private/link-local on the resolved IP;checkRedirectbounds the chain at 3 and refuses cross-host;RefreshOAuth2at:228sendsgrant_type=refresh_tokenonly.pkg/business/datasources.go:1473 runAPISync— decrypts vias.datasourceCipher.DecryptSecret(ctx, DatasourceConnectorSecretPurpose(source.ID), source.CredentialSecretRef), orresolveOAuth2AccessToken(:1532) underLockDatasourceSourceCredentialRefin one org transaction, rotating the envelope;ErrOAuth2ReauthRequiredis terminal and degrades the source.pkg/datasource/connector/CONFORMANCE.md— the API connector is registered non-conformant (recordsinterface;Registry.Registeradmitsfilesonly) and takes no new sources.DatasourceProviderDescriptor.accepts_new_sources(datasource.proto:456) is false for it.pkg/business/datasource_directory.go:152–190—datasourceAccountLinker{AuthorizeURL, ResolveAccount}is a generic seam; the only implementation isgithubAccountLinker, selected at:161only whenconnectorKey == github.ConnectorKey && s.githubAppClientID != "".BeginDatasourceAccountLinkmints one-time state bound to org + person + connector;CompleteDatasourceAccountLinkresolves the account.- Credential custody: Vault Transit envelopes (
cfs1:vault-transit:…, keyapi-keys,pkg/keyservice/vault.go) stored indatasource_sources.credential_secret_ref. Vault KV holds onlysecret/data/jwt-signing-key. Thekeyserviceseam (branchissue-1013-…, not a PR) adds a KMS backend behind the same seam — usedatasourceCipheras it is; do not touchkeyservice. - Module-facing seams a consumer already reaches a source through:
ModuleCapabilitiesService.FetchDatasourceFiles(module_capabilities.proto:1347, authorized by the caller principal's datasource-queue grant and the source row's own org/boundary; rate limit →RESOURCE_EXHAUSTED+ErrorInfo{DATASOURCE_RATE_LIMITED, reset_at}+RetryInfo),MintSourceOperationContext(:1452, re-checks delegation active / source exists / person still owner-or-admin / binding unchanged on every mint),ExchangeDelegatedOperationAudienceover aMODULE_PRINCIPALSoperation_audiencesentry (module/WORK_CONTEXTS.md:99–165: resource-restricted scopes, lookup ⊆ invoke and read-only, revoked on the next call when the binding changes). SourceDelegation(datasource.proto ~947): a person, by connecting a source, delegates it to one installed operation binding of a consuming module;ListSourceDelegations,RevokeSourceDelegation, seven revocation reasons.- The host marks no method with
codefly.runnable.v0.operationand declares norunnable.codefly.yaml(Rip the job runtime out of the host: audit of what is still in the tree (2026-09-18: nothing removed yet) and the removal checklist #835, Prove the host's 31 handbook stories, trace every operation, keep one plan #820 Part 0 open). The reuse reference is document-storemodules/documents/contracts/api/documents/grpc/proto/documents/v1/api.proto:4076 PlanSyncPages—audience,invoke_scopes,lookup_scopes,lookup_method,max_input_bytes,max_output_bytes,completion: COMPLETION_CALL, served on its Connect endpoint with the SDK receipts interceptor (x-codefly-effect-id, replay on reuse). - Budget:
pkg/datasource/connector/budget.go— one credential's quota is shared by every source using it;PriorityInteractiveis served before background. - Audit vocabulary:
pkg/business/audit_registry.go; datasource events aresaas.datasource.*. - REST projection:
module/REST_SURFACE.md— an RPC is REST-enabled only with agoogle.api.httpannotation andEXPOSURE_PUBLIC|AUTHENTICATED; internal RPCs have none. - Tests:
apisource/client_test.go(11 tests,httptest);business/datasources_test.goruns against real Postgres + Vault booted by codefly (!puretag);connector_conformance_record_test.goholdsCONFORMANCE.mdto the registry. - Related open work — build on, never duplicate: Robin parity C: caller-authorized tool discovery, Runnable dispatch and extension adapters #1023 (Robin parity C: caller-authorized discovery and Runnable dispatch; its discovery projection is where one tool per declared operation will be emitted — coordinate on that thread), Datasource and audit cleanup: grants at connect time and in bulk, a private-repo check at sync, clickable audit cards, connector conformance, and a periodic reconcile #966 (datasource cleanup incl. connector conformance), The host binds the cell's durable Vault (kubernetes auth, CA mount, projected token); the module's dev Vault never serves a deployed product #1006 (durable Vault on the cell — prerequisite for staging qualification, not this issue), accounts' keys behind a key service: a cloud KMS backend for hosted cells (no secrets store, no stored credential) and the existing Vault backend for in-cluster deployments #1013 branch (keyservice).
Deliverable: one self-contained PR, two halves
Half A — OAuth2 for an API source (H2, H3, H5)
A1. client_credentials grant. apisource.ClientCredentials(ctx, OAuth2Config, clientID, clientSecret) (*OAuth2Token, error) beside RefreshOAuth2, same guardedClient, same terminal/transient split. New proto: ApiOAuth2Config.grant enum {REFRESH_TOKEN (default, today's behaviour), CLIENT_CREDENTIALS, AUTHORIZATION_CODE}; for CLIENT_CREDENTIALS the sealed credential is the client secret and no refresh token exists. resolveOAuth2AccessToken branches on the grant, keeps the row lock, caches the access token + expiry in the same envelope exactly as the refresh path does.
A2. Generic authorization-code linker with PKCE. oauthAccountLinker{descriptor} implementing datasourceAccountLinker for any connector whose DatasourceProviderDescriptor carries OAuth metadata (new fields: oauth_authorize_url, oauth_token_url, oauth_scopes); provider app registration is a host configuration group datasource-oauth with client_id as a value and client_secret as a secret (Codefly delivers values and secrets, never files — decisions/security-posture.md). ResolveAccount exchanges the code (with code_verifier) at the token URL, seals the refresh token (+ client secret when confidential) into the source's envelope with credential_kind: OAUTH2, grant: AUTHORIZATION_CODE, and sets PersonalOwnerUserID so the source is readable only by that person (connector.Source.PersonalOwnerUserID, ApplySourcePolicy). State stays one-time, bound to org + person + connector, as today. Per-organisation app registration is out of scope — say so in the PR.
A3. Disclosure guard. Every new error, receipt and audit payload goes through named constructors as status_reason does; add TestNoCredentialByteSequenceLeaks: seal a random 48-byte credential, drive every failure path (bad token URL, 401, 429, redirect, timeout, schema refusal), and assert no 16-byte segment of the plaintext, the client secret or the refresh token appears in any returned error string, receipt JSON, audit row or captured log line.
Half B — a connected source is callable (H1, H4, H6, H7, H8)
B1. Split sync from call on the descriptor. accepts_new_sources stays for sync; add accepts_operations (and operations_gap) to DatasourceProviderDescriptor. The API connector is accepts_sync: false (unchanged, honest) and accepts_operations: true. CONFORMANCE.md gets a section stating that the call path is not the envelope's sync interface and is held to its own suite (B6); connector_conformance_record_test.go extended so a provider cannot be registered for operations without a row.
B2. Declarations. New table datasource_source_operations (org-scoped, RLS like datasource_sources): source_id, name (^[a-z][a-z0-9_]{0,63}$), method (GET|POST|PUT|PATCH|DELETE), path template with {param} placeholders, input_schema (JSON Schema, object, additionalProperties:false), output_schema, effect (READ_ONLY|MUTATION), max_output_bytes, digest over the canonical declaration. Proto: SourceOperation message; DeclareSourceOperations(org_id, source_id, operations) (replaces the set atomically; admin-only; audited); ListSourceOperations. Input → route mapping, v1: fields named by a {param} go to the path (percent-encoded), fields listed in a declared query list go to the query, the remainder is the JSON body for bodied methods and is refused for GET|DELETE; headers never. Where the declaration comes from (solution manifest vs connect form) is an open question on the handbook issue — build the RPC; the manifest import is a later consumer of it.
B3. InvokeSourceOperation + LookupInvokeSourceOperation on DatasourceService, EXPOSURE_AUTHENTICATED, TENANT_REQUIREMENT_ORG_MEMBER, resource target the source's boundary node, new permission datasource:invoke (declare it in the permission catalog beside the read grants; the handler gate and the role check are two layers, keep both). Request: org_id, source_id, operation, input (Struct), effect_id; response: output (Struct), receipt {effect_id, committed_at, status, provider_status}. Behaviour: load + validate the declaration digest; validate input against input_schema (InvalidArgument naming the pointer); resolve the credential exactly as runAPISync does (Half A grants included); build the route from declaration + input, never from the caller; take a budget slot under PriorityInteractive; one call through apisource (extend it with Do(ctx, method, url, body) beside Fetch, same guards, body ≤ declared max_output_bytes); 2xx → output validated against output_schema; 429/provider rate limit → RESOURCE_EXHAUSTED + ErrorInfo{DATASOURCE_RATE_LIMITED} + RetryInfo exactly as FetchDatasourceFiles; 4xx → FailedPrecondition with a named constructor reason; 5xx/timeout on a MUTATION → the receipt stays unresolved, never re-dispatched, Lookup answers unknown; on READ_ONLY → Unavailable (retryable). Receipts: an effect_id reused with a different input is refused; the same input replays the committed receipt and makes no provider call.
B4. Runnable marking. Mark InvokeSourceOperation with codefly.runnable.v0.operation exactly in document-store's shape: audience: "saas-datasource" (or the host's existing operation audience — read MODULE_INSTALLATION.md and pick the one the host already issues for module-facing operations; do not invent a second), invoke_scopes: [{resource_kind: "datasource.sources", actions: ["invoke","read"]}] with resource_ids left to the installing composition via a required_scope_slots slot named source (lookup: true), lookup_scopes read-only, lookup_method: "/saas.accounts.v1.DatasourceService/LookupInvokeSourceOperation", max_input_bytes: 65536, max_output_bytes: <declared cap>, completion: COMPLETION_CALL. Do not put tool: on the method unless the core sibling issue answers that it belongs there — the per-declaration tool is the discovery projection's (#1023). Add codefly generate runnables --check to CI as document-store has, commit the generated contracts/runnables/, and serve the receipts interceptor the document store serves on the same Connect endpoint. Never hand-edit a generated file.
B5. Audit. saas.datasource.operation.declared and saas.datasource.operation.invoked in audit_registry.go: actor, source, operation, effect, outcome (committed|refused|unresolved|rate_limited), effect id; no input, no output, no credential.
B6. Operations conformance suite. connectortest.RunOperations(t, fixture): unknown operation refused; input outside schema refused; declared {param} required; GET with body refused; output over cap refused; 429 typed; a mutation with a lost reply stays unresolved; credential never in any error (A3's check reused). Run it for the API connector.
B7. SDK hook. The SDK regen path is saas-sdk-go/README.md ("regenerate from module/services/accounts/proto at the recorded ref"); nothing to build here beyond keeping the proto generatable. The sibling SDK issues consume your PR head's proto to develop and your tag to release.
Tests — what must exist and pass
apisource/client_test.go:ClientCredentialsrequest shape + 401 terminal + token URL SSRF-guarded + cross-host redirect refused;Dofor each method; body cap;429surfaced withRetry-After.business(real Postgres + Vault,!pure): invoke as a boundary reader withdatasource:invoke→ output; without →PermissionDenied; unknown operation →NotFound; schema refusal →InvalidArgumentwith the pointer; sameeffect_idtwice → one provider call (count at thehttptestprovider) and identical receipt; different input under the sameeffect_id→ refused; two concurrent invokes on aclient_credentialssource → one token exchange (the existing row-lock pattern); revoked delegation / removed permission between two invokes → second refused; audit rows exactly one per call, credential-free; a personal (auth-code) source invoked by another member →PermissionDenied.TestNoCredentialByteSequenceLeaks(A3).connectortest.RunOperationson the API connector (B6).codefly generate runnables --checkgreen; arunnabletest-style prepared binding over the method succeeds in Connect JSON mode and refuses a reused effect id (copy module-runtime'spkg/runnable/runnabletestshape into a host test, or add the host method to the runtime's qualification graph owners — say which and why).- Existing gates: naming gate (tree + records; the records half reads issue/PR titles — keep consumer names out), boundaries gate (zero baseline),
go vet ./...in every Go module,-tags puresuites,GOWORK=off go test ./...for the config-group gate (the newdatasource-oauthgroup must ship defaults forlocal— saas ships no local defaults for its declared workspace-config groups — every composing solution must hand-provide legal/identity/etc. to boot #482's rule), frontend typecheck if you touchsaas-ui(you should not need to; the connect form for OAuth is the sibling of Datasource and audit cleanup: grants at connect time and in bulk, a private-repo check at sync, clickable audit cards, connector conformance, and a periodic reconcile #966 and out of scope unless trivial).
Docs — same PR, named in the body
module/MODULE.md (Data sources: callable sources, the two OAuth2 shapes), module/REST_SURFACE.md (new routes and counts — the pinned extension set test must still pass), module/WORK_CONTEXTS.md (the datasource.sources scope and the slot), pkg/datasource/connector/CONFORMANCE.md (B1), module/configurations/README or wherever configuration groups are documented (datasource-oauth), audit_registry docs if a catalogue page exists, docs/ for the receipts interceptor if you add it.
Rules
- Follow this repo's
AGENTS.mdand the handbook's boundaries: no solution or customer name anywhere, including tests and fixtures. - Prove reuse before you build: the PR body lists what was reused (
datasourceCipher,resolveOAuth2AccessToken's lock,guardedClient, the budgetScheduler,FetchDatasourceFiles' rate-limit shape, the delegated-audience exchange, document-store's operation marking) and what was deliberately not built (a second credential store, a tool-owner-style HTTP proxy, a runtime change, a queue). A second mechanism for a job a seam already does is refused at review. - Classify every change in the PR body as a fix at the owner or a hack; no unlabelled workaround. If a gate blocks you, fix the gate's source.
- One self-contained PR against
main; never a stack; no release/tag until the handbook proposal is accepted (ask on this issue when the PR is green). Releases arecodefly publish's — never hand-tag. - Do not open issues anywhere; ask on this issue. Do not edit another agent's checkout; Robin parity C: caller-authorized tool discovery, Runnable dispatch and extension adapters #1023 is live on this repo — coordinate the tool-projection seam on its thread before touching anything under
module_capabilitiesbeyond B4's marking. - Keep
workingandrole:*labels; they are lazybox's claim state. - Say what you did not verify. If you could not boot real Postgres + Vault, say so and do not claim the business suite green.
Activity
antoinetoussaint-byte commentedon Oct 9, 2026
Sibling issues — one per repo, same design note
Sourcerow, saas-starter + robin page edits, three storiesInvokeSourceOperation/Lookup…, declarations, receipts, audit, the Runnable markingbindingInvoke,Lookup,DeclareOperations, typed errorsOrder: handbook proposal and core answer first (both unblock the host proto) → saas-starter PR + tag → both SDKs from the tag → robin → platform-core pins and the end-to-end run (not filed yet; it needs the tags). Related, build on and never duplicate: module-saas-starter#1023 (Robin parity C), #966, #1006, #1013 branch; module-robin#286, #315.
antoinetoussaint-byte commentedon Oct 9, 2026
Core #754 has answered: one prepared binding, per-declaration discovery projection, with
toolabsent on InvokeSourceOperation. Coordination is posted on #1023: #1023 (comment).One wire-contract decision is needed before the invoke proto: Core's current bounded method profile rejects protobuf Struct/Value, maps and bytes. This issue specifies Struct for both input and output, which cannot pass
codefly generate runnables --checkunder that profile. Proposed owner fix: useinput_jsonandoutput_jsonstrings in the method envelope, validate their decoded object against the declaration schemas in the host, and expose objects through the SDK facades. The method's byte limits still bound the complete envelope. May we use that supported representation, or must the Struct contract wait for a Core profile extension?OAuth and declaration implementation continues independently. No release or issue creation is being attempted.
antoinetoussaint-byte commentedon Oct 9, 2026
Owner's answers (2026-10-09), three of the four open questions
client_credentials(organisation-level source, readers = the boundary node). Today's refresh-token paste stays as the third grant.proposals/2026-10-02-declared-source.md). A person connects the source; nobody types routes into a form. The host'sDeclareSourceOperationsRPC is the import target the manifest install step calls; a connect-time form is not built.Record 1–3 in the proposal's "Decisions taken" with this date; implementers build to them.
antoinetoussaint-byte commentedon Oct 9, 2026
Source-only Runnable scope slot is blocked by Core
The public Invoke/Lookup JSON-string proto and receipt handlers are implemented.
toolremains absent.A concrete Core contradiction blocks the requested resource-bound marking (confirmed in v0.14.0 and current main):
runnable.OperationSpec.Validate()callsauthoritySpec.validateScopes()and refuses zero fixedinvoke_scopes/lookup_scopes, even whenrequired_scope_slots: {name: "source", required_actions: ["invoke", "read"], lookup: true}is declared.datasource.sourcesscopes instead does not solve it:ResolveScopeSlotsrefuses selecting that same kind insourcebecause a fixed scope already binds it (scope_slots.go:152).The owner fix is for Core to admit an unresolved operation whose required slots supply all authority, while continuing to refuse unresolved prepared bindings and empty resolved authority. The host can then retain only the
sourceslot and the composition suppliesdatasource.sourcesinvoke/read for exact source IDs, with read-only lookup.Please have the Core worker address this on the existing Core work, or give an owner-approved equivalent that preserves source-only authority. No new issue is needed. I am continuing the host tests/docs/generation and will publish the PR head for SDK regeneration, with this check explicitly reported as blocked if the fix is not available.
antoinetoussaint-byte commentedon Oct 9, 2026
Callable-source proto available for SDK regeneration
One draft PR against main: #1052
Head:
f4ce896d954101b164c3b511cbeea555f17302f7SDK workers on codefly-dev/saas-sdk-go#30 and codefly-dev/saas-sdk-python#22 can regenerate from
module/services/accounts/protoat that head. The owned declarations are undersaas;vendorcontains the Core option import closure. The full Go, contract and TypeScript generation succeeded, and contract drift and proto lint passed.The Invoke request is
org_id,source_id,operation,input_json,effect_id. Invoke and Lookup both returnoutput_jsonplus a receipt containingeffect_id,committed_at(RFC3339Nano string),status,provider_status, andoutput_json. Lookup takes onlyeffect_id; the host derives tenant/source from verified identity and durable attempt metadata. Each JSON string carries exactly one object; unknown outputs are{}. Effect IDs are at most 128 bytes. The full input envelope is 65,536 bytes; each declaration caps provider output at 65,536 bytes, and the full response envelope is 1 MiB. No Struct, Value, map or bytes are reachable from the Runnable-marked Invoke/Lookup graph; declaration schema fields remain Struct.toolis absent, per Core #754.This PR is not green or deployable. Core's source-only required-slot rejection blocks Runnable generation, the prepared Connect JSON replay test, and accounts receipt initialization. The concrete owner fix is described at #1049 (comment); it must reach both the accounts dependency and CLI toolchain before regeneration can produce the Runnable artifacts. Please route that fix to the Core worker while the SDK workers consume this proto head.
Real Postgres + Vault qualification could not boot: Codefly timed out initializing the migration driver at
SELECT CURRENT_SCHEMA(), so zero real business tests ran. The PR body lists the passing pure/conformance/transport, repository, SDK and vet checks, the failing prepared-binding check, and the compiled-only integration tests. It classifies the changes as fixes and names the documentation. No release or tag; the handbook proposal must be accepted, and the release question stays pending until this PR is green.antoinetoussaint-byte commentedon Oct 9, 2026
Host-contract items for PR #1052 from the SDK deep reviews (saas-sdk-go#30 review 5475129391, saas-sdk-python#22 review 5475124355):
operationUnknown()answersFailedPrecondition+ErrorInfo{reason: SOURCE_OPERATION_OUTCOME_UNKNOWN}. Keep the reason as the contract (both SDKs branch on it) and state it in the proto comment onInvokeSourceOperationand in docs: that reason means the effect's outcome is unknown, will not be re-dispatched, and the receipt stays unresolved; a refusal carries another reason.operationRefused()passes nil metadata, but both SDK facades exposeProviderRefused.statusfrommetadata["provider_status"]. Emitprovider_status(the provider's HTTP status as a decimal string) on every refusal that has one, and on the receipt.InvalidArgumentcarriesBadRequest.FieldViolationwith the JSON pointer; the handler does not emit it at f4ce896. Emit it (the issue asked for "InvalidArgument naming the pointer").NotFound "source effect not found"stays the one signal meaning "never attempted; a same-id re-invoke is safe"; document it beside the unknown-outcome reason.SourceOperationinListSourceOperationsmust carrydescriptionanddigestif it does not — robin's tool projection needs both.Also noted by core#755's deep review: codefly-dev/cli has no
required_scope_slotssupport, socodefly generate runnable-bindingswill refuse the slot-only operation even on the fixed core; record in the PR body that the prepared-binding test and the bindings render are blocked on the CLI, not core.antoinetoussaint-byte commentedon Oct 9, 2026
Owner's instruction (2026-10-09): fix all — the five host-contract items in the comment above, and every finding of the deep review of PR #1052 once it posts (in flight). The CLI gap is tracked as codefly-dev/cli#943 with a worker; record in the PR body that the prepared-binding test and the bindings render wait on it. No deferrals; mark #1052 ready when green and post the head SHA here.
antoinetoussaint-byte commentedon Oct 9, 2026
Deep review of PR #1052 head f4ce896: #1052 (review) (lazybox r8, 9 findings). Fix all, no deferrals.
B1 — BLOCKER, tenant isolation.
pkg/adapters/source_operation_invoke.go:82-99registers the auditdeferbeforerequireOrgMember/tenant checks;DurableAuditEmitter.Emitwrites underWithOrgTx(entry.OrgID)andbindRequestScopenever compares the org with the verified identity, so any authenticated principal can commitsaas.datasource.operation.invokedrows with choseneffect_idtext into a foreign org's audit log. Reorder so no audit row is written before tenant membership is verified (the business engine already orders it correctly); add the negative test: foreign-org Work Context → PermissionDenied and zero audit rows.M1. The deployment-wide OAuth client secret/id/token URL are copied into every linked personal source's envelope and used from the copy at refresh; rotating the secret forces every person to relink and spreads one secret across N tenant envelopes. Reference the configuration group at refresh time; never copy it into a source envelope.
M2. No sweeper for receipts/attempts: one credential at the admitted 60 calls/min can write ~10 GB/day of receipts useful only for the 600 s
total_timeout. Add retention keyed tototal_timeoutplus a lookup grace, as a host operation the runtime executes (jobs-leave-the-host), with a test.Minors, all to fix:
Routedrops the base-URL path (probed); a READ_ONLY commit failure is permanently unknown (should be Unavailable/retryable);ListSourceOperationslackspermissions/scopes; the provider map can shadow the GitHub linker and admits one usable provider; 8+ schema compilations per call (compile once per declaration digest and cache); the small sprawl.Build evidence from the reviewer: with
core@8b32d055the non-test tree builds andTestSourceOperationPreparedConnectJSONReplayAndCurrentAuthoritypasses, but two existing adapter tests fail to compile against that core (solutionhost.SchemaPresenceV2,coreworkcontext.Inspect) — reconcile those APIs in this PR; the unblock is not a pin bump alone.Then the five host-contract items above, the Postgres+Vault retry, PR body, ready, head SHA here.
antoinetoussaint-byte commentedon Oct 9, 2026
PR #1052 head
2ff5708f436ce494314f0dab046e5181a0c10bc8now contains the complete generated Runnable output and the qualification fixes. It remains draft.Using Core
8b32d0555897a3d595a85844bf4633c355b1df12through local overrides, and CLI source1289c657f0dbd336b2296e6b0b7f829bc9b8f6b8built with that Core:codefly generate runnables saas-starterand--checkpassed.go.work, so this test used a temporary local go.mod replacement; build metadata verified it and the replacement was removed. On deliberate shutdown, Vault exceeded its 15-second drain budget; Codefly reported exit 1 for cleanup, then scopedcodefly clear --keep-processesremoved the remaining test containers.No unmerged Core pseudo-version is committed. The released v0.17.0 staging pin still reproduces the scope-slot refusal after the override is removed. It must become the Core release carrying #755.
The latest Core #754 correction (comment 6088882705) is recorded: CLI #943 is also required. Current
operation.jsongeneration loses the slot; a clean drift check does not prove CLI delivery. Our passing prepared test uses Core directly from the descriptor, not the lossy CLI document. I will rerun the CLI document/selection/preparation path when that fix is available.A separate export finding for the CLI owner:
cmd/generate/contracts.go:349exports the whole service descriptor to both accounts/authority and accounts/connect;cmd/generate/runnables.go:252derives every marked method on each. Only Connect serves Invoke/Lookup. The generated authority package is retained unedited but is not claimed callable. Please include endpoint-specific contract projection in the tooling handoff; the authority listener must not be widened to disguise the export.The accepted handbook decision is reflected in docs. The posted deep-review findings are now being fixed in this same PR; the tenant-audit ordering, live OAuth registration refresh, base path, read-only retry, schema cache and SDK error-detail fixes are in progress beyond this pushed head. For retention, deleting the last mutation marker would violate the no-redispatch contract; the safe implementation expires large outputs while retaining compact effect tombstones. No release or tag, and no readiness claim while host review work, CLI delivery and released pins remain outstanding.
antoinetoussaint-byte commentedon Oct 9, 2026
Host review fixes and qualification
PR #1052 is updated at
86ee51fd8b4d66aad9da2509483e14d3c7534764and remains draft: #1052The host review and SDK contract fixes are pushed: verified tenant/membership before audit; live OAuth app rotation without copying the deployment secret/config into sources; base-path preservation; retryable read-only commit failure; List permission/scope policy; API-only OAuth registration; bounded compiled-schema cache; matching effect-ID carriers; provider-status metadata, JSON-pointer BadRequest details and documented unknown/NotFound semantics. Descriptions and digests remain in listed declarations.
Receipt retention is a source-scoped host Runnable (
PruneSourceOperationReceiptsplus Lookup) for runtime scheduling. It removes at most 1,000 expired response bodies per call after the declared Invoke timeout plus 24 hours, atomically with audit and its receipt. Compact effect tombstones remain: deleting the last marker would permit an old mutation or changed-input reuse. Shared sources require an administrator; personal sources require their owning member, with current source scope before replay. There is no host timer or queue. Regression tests cover rollback, replay, tenant/source isolation, owner/admin admission and expired mutation non-redispatch.Qualification uses Core
8b32d0555897a3d595a85844bf4633c355b1df12via external local overrides. The final run uses CLI #9471026677fa6f84471b0524e63cfc52e42fac8425dbuilt against that same Core in this task's own temporary archive. No other worker's checkout was edited; no unmerged pseudo-version or local replacement is committed./tmp/1049-retention-owner-business-full.log. This supersedes the earlier migration-init timeout; no business suite is left unexecuted in this retry.generate runnablesand--checkpassed. An isolated composition using this head's contracts passedgenerate runnable-bindingsand--check; Core verified the four emitted prepared bindings, matching resolved-policy receipts and absent tool exposure. This qualifies the CLI document/selection/preparation path separately from the host's descriptor-based Connect tests./tmp/1049-review-accounts-startup.log.Readiness is still blocked on released tools and CI. Core #755 and CLI #947 are still open. The committed released Core v0.17.0 reproduces the source-only scope refusal; its final pin must be the Core release carrying #755, and the CLI pin must be the release carrying #943/#947. Fresh CI has started on the new head; it is not claimed green. Prior run 37990122604 also hit Docker Hub unauthenticated pull limits in authorization, migration, SDK, build and supply-chain jobs. The host event-catalog drift from that run is fixed here.
The endpoint-export finding remains for the tooling owner: the CLI derives the same marked methods under authority and Connect because both catalog entries contain the full descriptor. Only Connect serves Invoke/Lookup and Prune/Lookup. All generated output is retained untouched, but the authority packages are not callable; do not widen the authority listener to hide the export defect. Live JWT/delegation invocation through a CLI-rendered deployment binding and release/staging qualification remain unverified.
SDK workers can regenerate from this head's
module/services/accounts/proto; the JSON-string Invoke/Lookup wire shape is preserved, error-contract comments are updated, and retention methods are added. Coordinator/Core/CLI owners: supply the released pins and route endpoint-specific export, then this PR can run default CI and become ready once it passes. The handbook decision is accepted; no host release is requested while the PR is not green. No issue, tag or release was created.antoinetoussaint-byte commentedon Oct 9, 2026
Receipt recovery survives declaration removal
Pushed PR #1052 head
2a16c688c61d66645e1063c495fa38eaf2fa52de: #1052The SDK-reported gap is fixed at the host owner. Once an attempt exists, Lookup checks the retained actor/source binding and current source read authority, then reads the saved receipt independently of the current declaration. Removal or replacement cannot turn that evidence into NotFound. A committed receipt is returned unchanged; an unresolved or expired receipt remains
unknown. Revoked source access still refuses recovery.Re-invoking that effect while its declaration is removed returns FailedPrecondition with
ErrorInfo{reason: SOURCE_OPERATION_DECLARATION_REMOVED, metadata: {receipt_status: committed|unknown}}. It does not call the provider. The status comes from retained receipt presence, and the SDK read occurs after releasing the authority transaction. A changed declaration still fences Invoke; Lookup remains evidence recovery.The new public Connect JSON regression reproduced the original NotFound before the fix and now passes. It checks the saved mutation receipt after removal/replacement, the removal reason/status, unknown recovery and revocation. The real Postgres/Vault mutation test invokes and records effects, removes declarations, recovers committed/unknown evidence through the authority and SDK receipt-store path used by Lookup, refuses same-ID reinvocation with the typed reason/status, and asserts no additional provider calls. The public handler regression and real persistence test cover both layers.
Validation on this update:
/tmp/1049-declaration-removed-business-full.log.go vet ./...passed.codefly generate runnables saas-starter --checkpassed.Qualification still uses Core
8b32d0555897a3d595a85844bf4633c355b1df12and CLI #9471026677fa6f84471b0524e63cfc52e42fac8425dthrough the existing local development overrides. No dependency pin changed, no unmerged pseudo-version was committed, and no release was made. The PR remains draft under the existing released Core/CLI and endpoint-export blockers; fresh CI is not claimed green.Updated docs are
module/CALLABLE_SOURCES.md,module/WORK_CONTEXTS.mdand the public proto comments beside the error reasons. The behavior, regressions, documentation and complete generated output are fixes at the host owner; no hacks.SDK workers can consume this head. The JSON-string message fields are unchanged; Lookup no longer needs the removed-declaration caveat, and Invoke now documents the typed removal refusal and
receipt_statusmetadata. Recover the original effect with Lookup; never use a fresh ID as recovery.antoinetoussaint-byte commentedon Oct 9, 2026
A lazybox agent is working on this.
codex(Astra · high) started at 2026-10-19 16:25 UTC, last renewed the claim at 2026-10-29 14:25 UTC, and the claim lapses at 2026-10-29 15:25 UTC unless it is renewed again. A claim is an assertion with an expiry, not proof that a process is alive.claim detail
antoinetoussaint-byte commentedon Oct 9, 2026
Pushed 1927792 to PR #1052. The host endpoint-inventory item from CLI #943/#947 is fixed.
cmd/connect-registrationnow emitsgenerated/api-contract-surfaces.jsonusing Core's existing service records. The manifest selects it withspec.api-contract-surfaces. Authority is exactlyModuleAuthorityProcedures(31 procedures); Connect is the full registration catalog (258, including internal methods). Authority's listener and allowlist are unchanged. Complete descriptors are retained, while regenerated catalogs and Runnables now expose only Connect Invoke/Prune, with their paired Lookups. Both invalid authority packages were removed by the generator.Qualification used CLI #947 3a88b6305a919eb842491ffbc313434be05123b8, built against Core 8b32d0555897a3d595a85844bf4633c355b1df12 through external Go workspace overrides. No branch pseudo-version or local override is committed; the final pins must be the releases carrying those fixes.
Passed:
--checkgates. Exactly two Connect packages remain.--checkfor both Connect selections; Core VerifyPrepared, VerifyResolvedPolicy and BindingMatchesResolvedPolicy; exact source selection and ErrNotATool for both.This generation/manifest/docs follow-up does not change runtime behavior. The preceding real Postgres + Vault business run remains 1,033 passed / 0 failed / 0 skipped at
2a16c688; it and native receipt startup were not rerun for this inventory-only change. Live delegated delivery through a deployed composition remains unverified.Every change is a fix at the host owner, no hacks. Docs updated in accounts AGENTS.md, REST_SURFACE.md and CALLABLE_SOURCES.md; CI now checks the owner inventory before contract/Runnable drift. The PR body records the evidence and limits. The declaration-removal recovery fix remains included in this head; SDK workers can consume this head's proto.
Core #755 and CLI #947 remain open, so #1052 remains draft pending their released pins and green CI. Coordinator: adopt those release pins and rerun default CI before readiness; the host inventory and retained-receipt gaps are resolved.
antoinetoussaint-byte commentedon Oct 10, 2026
Pushed b83cd1e to PR #1052. Core v0.18.0 is pinned and qualified in all eight Go modules that require Core, including policy-log and module-authority qualification. The release resolves to
59cd7b06e18d915afb749356245a9172a2ff2df7; no Core pseudo-version or replacement is committed. Host suites ran withGOWORK=off.The old SDK used by auth-gateway, policy-log, store and telemetry referenced the removed Core
resources.VisibilityExternal. Those four pins now use released SDK v0.3.1; the three listener entrypoints useResolveNetworkInstanceinstead of removedWithDefaultNetwork. No caller-token shim or new declaration-source behavior was needed. All changes are fixes at their owners, with docs and complete regenerated output in this PR.Passed:
codefly agent deps --pin v0.18.0standalone build/tidy verification, pure suites and vet in all eight modules; root full suite including configuration defaults./tmp/1049-core018-business-full.log.--checkpass; authority remains 31 procedures, Connect 258, with exactly two Connect Invoke/Prune packages.Generation still uses CLI #947
3a88b6305a919eb842491ffbc313434be05123b8, rebuilt with an external CLI-only workspace override to released Core v0.18.0. The checked-in CLI pin is unchanged pending #947 and its release. PR body and docs now distinguish that remaining blocker from the resolved Core dependency.Native auth-gateway/policy-log/telemetry startup and live delegated invocation through a deployed composition were not exercised; the real business suite did run. No green-CI or ready claim is made. Coordinator: supply the CLI release pin when available, then rerun default CI before readiness. SDK workers can regenerate from this head.
antoinetoussaint-byte commentedon Oct 10, 2026
PR #1052 build follow-up: 0b07def
Pushed 0b07def to #1052. The PR body was updated before the push with the diagnosis, owner fixes, complete generated output and exact qualification results.
Run 38077896784, Codefly build failed before accounts compilation:
go.mod requires go >= 1.27.2 (running go 1.27.0; GOTOOLCHAIN=local). The pinned image isgolang:1.27.0-alpine3.23@sha256:3747dcba41c8b0db3211fda4db61638b980e17ac5bb3c94460a975a9cfe19395. This was not a Docker Hub 429 or caller-token/SBOM API failure. The image-evidence failure followed the failed accounts build and skipped dependents.Fixed accounts’ accidental 1.27.2 minimum to its supported 1.27.0 minimum. Also fixed both authority integration fixture builders to state Core v0.18.0’s required empty subject declarations, and the exact-grants test to require zero control-plane privileges on the four callable-source tables. Production validation and migration 31 grants are unchanged. All changes are fixes at the host owner; no hacks or evidence-gate exceptions.
Verified with
GOWORK=offand actual Go 1.27.0:go mod tidy -diffpassed. Contract/Runnable drift checks, naming, boundaries and whitespace passed.linux/amd64images; the actual effective image-evidence gate passed for all five against the unchanged image inventory. Cache/vault keep the existing vendor-audit coverage. Image-gate unit tests: 13 passed, three existing opt-in skips.The local shared agent cache contained a development build under the release filename, so that first trial was stopped and excluded. Codefly’s existing
--plugin-pathdownloaded the real pinned releases into an isolated cache. Their complete generated recipe/runtime-access output is committed; shared installations and other checkouts were untouched.Fresh CI run 38079651048 is queued; this is not a green-CI claim. CLI #947 is still open. Generation remains on its requested
3a88b6305a919eb842491ffbc313434be05123b8override with released Core v0.18.0; no new CLI pin is committed. The coordinator must supply the release carrying #947, then default CI must pass before #1052 is marked ready. No release or tag was made.antoinetoussaint-byte commentedon Oct 10, 2026
Recipe proposal finding and fix: e2c1b3d
Pushed e2c1b3d to #1052; it remains draft.
The recipe changes were not caused by CLI #947 or host Core v0.18.0. I reran the affected build with released CI CLI v0.1.179 (binary dependency Core v0.8.1) and freshly downloaded released agents go-grpc 0.1.53, nextjs 0.0.163, postgres 0.0.141. They regenerated byte-identical recipes to
0b07def48, leaving its tree clean. The templates are owned by those service agents. The seven service tasks passed; existing images were reused after generation because their input identities matched, so this probe is not a fresh-image-build claim.The prior local build had refreshed stale checked-in snapshots from already-pinned agents. Committing those unrelated refreshes caused the plan refusal. As requested, I removed the complete refreshes for auth-gateway, frontend, telemetry and store, including companion store bootstrap/runtime-access files and telemetry’s ignore file. Their service manifests are unchanged. Accounts’ generated recipe remains because its manifest changed. The restored baseline snapshots are not claimed to be current generator output; effective builds regenerate from the manifest pins. No gate, image inventory, service pin or manifest was altered to admit the diff. This is a scope fix, with no hack introduced.
My previous local validation omitted
CODEFLY_BASE: without it,build-images.mjs checkchecks coverage but skips proposal admission. The exact CI invocation reproduced all four refusals before this fix and now passes:The only remaining recipe diff against that base is accounts. The effective image-evidence check also passes with the same base set, comparing the accounts committed recipe with pinned-agent output and checking the existing completed BuildKit records. Naming, boundaries and whitespace passed.
RELEASE_GATES.mddocuments the missing local-check input. No authored runtime code changed; the earlier full accounts and real Postgres+Vault results were not rerun.The PR body now explicitly records run 38079651048’s expected SDK-boundary failure until CI pins a release carrying #947’s
api-contract-surfacessupport, and that Release gates only aggregates these failures. Generation remains on the requested #9473a88b630override. Fresh CI run 38081217983 is queued; no green-CI claim is made. The coordinator can supply the released CLI pin after #947 lands; #1052 stays draft until that pin and CI are green.antoinetoussaint-byte commentedon Oct 10, 2026
Pushed 35356b5 to draft PR #1052.
The retained
codefly-ci-report-testartifact from run 38081217983 / job 114298452670 gives the verdict: eight services passed, frontend failed. The complete log at 19:58:39 UTC namesfrontend-catalog.test.ts:36: expected 24 permissions, got 26. This PR addsdatasource:readanddatasource:invoke; the next old API-key scope count was also stale (19 versus 21). Vitest finished with 1 failed / 4,051 passed tests. This was a real stale assertion, not a timeout, later-service failure, or the expected-error stderr.Fix at the owner: the authored frontend test now checks the exact callable-source names, scope membership and permission recognition. The Go cataloggen fixture-equality test still checks the complete generated catalog.
module/CALLABLE_SOURCES.mddocuments the checks and corrects its stale endpoint-export paragraph. No runtime or generated files changed; no hack or weakened gate.Verification:
codefly ci run --all --phase testpassed all nine service tasks, zero reused/failed/skipped/cancelled tasks. Used CI's released CLI v0.1.179, pinned released agents in an isolated cache propagated throughCODEFLY_HOME, Go 1.27.2, Core v0.18.0,GOWORK=off, and fixturedev-admin. Local platform is Darwin/arm64. Selection bounds were the original CI base ande2c1b3d4; the working tree contained exactly this test/docs patch..codefly/ci/quality-test-catalog-final/report.json; log:/tmp/1049-quality-test-catalog-final.log.CODEFLY_BASEall pass. The tree is clean.Earlier local full-phase attempts failed
TestPostgresEventTransportConformance/AckNackHeartbeat's immediate redelivery assertion. It passed 20 isolated runs and the final full phase without source changes; its precise cause remains unproven. An intervening retry also hit a stale exited Postgres container from my preceding run; only that container was removed with scoped Codefly cleanup. The PR body retains those failures and their logs instead of claiming every attempt passed.PR body updated before push. #1052 remains draft; fresh GitHub CI is not claimed green. SDK-boundary drift remains expected until a CLI release carrying #947 is pinned. Contract/Runnable generation stays on CLI #947 3a88b630. The next handoff remains the released CLI pin, then a full green CI run before readiness.
antoinetoussaint-byte commentedon Oct 11, 2026
PR #1052 is ready for review at cab3641: #1052
All applicable CI checks are green: https://github.com/codefly-dev/module-saas-starter/actions/runs/38148237643. This includes SDK boundary, contracts/Runnables drift, image build and effective image evidence, quality and release gates. The retained
codefly-ci-report-testreports 9 service tasks passed, 0 reused/failed/skipped/cancelled. Accounts reports 4,868 passing tests with 12 existing skips; auth-gateway 549 with 1 existing skip; frontend 4,114 passing tests. These agent totals include subtests. Release-only jobs were conditionally skipped.CLI v0.1.185 and Core v0.18.0 are pinned; development overrides are retired. The inventory selector now lives in CLI-owned
module/contracts.codefly.yaml, and the obsolete servicespeckey is removed. Complete regeneration retains 31 authority procedures and the full 262-procedure Connect catalog, deriving exactly the Connect Invoke/Prune operations and their paired Lookups. There is notool:and no authority widening. The explicit inventory remains a documented hack/stopgap pending listener-local served-method metadata; the selector migration and dependency/test corrections are fixes.The real Codefly-booted Postgres+Vault business suite passed 1,082 / 0 / 0 (pass/fail/skip). Direct full accounts qualification passed 3,031 tests, zero failures and 11 documented existing skips across 50 packages, including 597 real Postgres infra tests. Pure suites, vet, prepared-binding/policy-receipt checks, generation/drift, naming and boundaries passed. The PR body distinguishes these direct runs from the final hosted results and records local Docker/disk/cache interruptions and remaining live-deployment/staging limitations.
The new-agent test failure was diagnosed from the CI report: old Redis/Postgres agents returned an UNKNOWN run-level outcome. Released owner fixes are consumed through Redis 0.0.97, Postgres 0.0.144 and Vault 0.0.42; the final hosted phase passes all nine services. Store generation is retained beside its changed manifest. Test fixtures now use isolated dependency sessions; no other checkout or shared database was altered.
SDK workers can regenerate from this head’s
module/services/accounts/proto. Retained receipts remain recoverable after declaration removal, and same-ID re-invocation cannot make a second provider call. JSON-string Invoke/Lookup and receipt output remain the public contract.The coordinator can proceed with review and the merge/publication sequence. No merge, tag or publication was performed here.
antoinetoussaint-byte commentedon Oct 11, 2026
PR #1052 is updated and ready for review at 6ff1e1c: #1052
All r24 findings are fixed:
The PR body explicitly names cab3641's move of
app_control_plane,app_billing_worker,app_webhook_workerandapp_job_workerfrom_rwto the dedicated_control_planelogin, including revocation of old memberships. These follow-up changes are fixes; the pre-existing inventory stopgap remains labelled.Every applicable CI check is green, including SDK boundary, contract/Runnable drift, build/effective image evidence and release gates. The retained test report has 9 service tasks passed, 0 reused/failed/skipped/cancelled. Local pure suites and vet passed in all nine Go modules; full accounts passed 3,033/0/11, including real Codefly-booted Postgres+Vault business 1,082/0/0 and Postgres infra 597/0/0. All four Go service images built successfully. The 11 direct-Go skips are the documented existing opt-ins/helper; no test failure was waived.
The finding-by-finding response is posted on the PR. SDK workers can consume this head; wire contracts are unchanged, with the Runnable package digests regenerated for the released agent.
The coordinator owns merge and publication. Neither was performed here.