Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libraries/source-read-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ provider.

## Regenerate

Requires released Codefly CLI **v0.1.219** (the CI pin). From the repository root:
Use the released Codefly CLI selected by the [CI installer](../../scripts/ci/install-codefly.sh). From the repository root:

```sh
codefly generate client --from saas-starter/accounts/authority --name source-read-sdk \
Expand Down
2 changes: 1 addition & 1 deletion libraries/source-read-sdk/library.codefly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ sources:
module: saas-starter
service: accounts
endpoint: authority
contract-digest: sha256:606e52170c2e55688ce163eb1d54c347f0d19ac40915d7fb6252f2034b787c0e
contract-digest: sha256:9aaed4fe30672e6c11675034ae204cec14ecbb299d68a3898b65780a88f80657
services:
- ModuleCapabilitiesService
generated-by:
Expand Down
64 changes: 64 additions & 0 deletions module/CALLABLE_SOURCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,55 @@ binding are recovered from verified identity and durable attempt metadata.

## Receipts and uncertain outcomes

### Conditional receipt verification (local owner candidate)

`VerifySourceOperationReceipt` is a proposed additional read, not an accepted
release contract. It takes the complete original `InvokeSourceOperationRequest`
through Connect/gRPC/gRPC-Web or POST
`/v1/datasource-operation-receipts/verify` with the whole request in the body.
It has no input-bearing GET or query binding. Its request and response are secret.

The caller must preserve exact `input_json` text and supply the original nonempty
`expected_declaration_digest`. The host normalizes the effect header/field exactly
as Invoke does and applies the same 65,536-byte complete normalized protojson
request bound. Matching headers are allowed; conflicting or empty carriers refuse.
Under the verified tenant, the original actor must still hold read authority to
the **stored** source, including current delegation, membership, boundary and
personal-source checks. Supplied source metadata never grants authority.
The method declares failure-only audit for the existing source-scope/delegation
refusal event (`saas.datasource.operation.invoked`); successful receipt reads
emit no new invocation event. This metadata does not turn verification into a
source mutation or add audit records for every validation or lookup failure.

Only after those checks, the host compares the admitted declaration digest and
the complete normalized request with the permanent attempt's SDK request digest.
This reuses `receipts.RequestDigest` and its `codefly.effect-receipt.request.digest/v1`
format; optional-field presence and exact JSON text participate. There is no second
consumer-side fingerprint. A future digest-format change needs an owner migration
that preserves verification of retained attempts; silently rehashing old markers
would not be compatible. No supplied input is newly persisted or logged.

A committed result means that invocation and admitted declaration match the
stored attempt, and contains the retained response under its existing 1 MiB wire
bound. It says nothing about output correctness, currentness or indefinite retention.
Mismatch returns a generic FailedPrecondition with no output; omitted or empty
declaration preconditions are InvalidArgument. An absent attempt is NotFound;
a matching attempt without retained output is `unknown` with `{}` output and
never authorizes redispatch. Removed/replaced declarations do not prevent the read;
removed sources or revoked authority can refuse it. Legacy invocations that never
pinned a declaration remain recoverable only through the original Lookup.

Old servers answer Unimplemented. Consumers must refuse unsupported, missing,
malformed, unknown or failed verification; ordinary Lookup success is not a
substitute. Legacy Lookup's request/response and paired Runnable recovery remain
unchanged. This method neither reads a replacement declaration nor contacts a
provider, and introduces no Runnable, ledger, queue or retry mechanism. Its
actor-bound, bounded retention does not provide permanent cross-reviewer evidence.
Owner acceptance, composition qualification and any durable sharing/retention
contract remain separate prerequisites for adoption.

### Existing receipt lifecycle

The SDK receipts guard serializes `(tenant, effect_id, method)` and verifies the
request digest. Its Postgres reads use the request pool under tenant RLS.
`receipts.Record` writes the full response using the exact org transaction that
Expand Down Expand Up @@ -402,3 +451,18 @@ is not release qualification. The CLI preserves scope
slots in operation documents and resolves composition selections during
preparation, checking that the called endpoint serves both Invoke and Lookup.
A passing generation drift check alone does not qualify the delivery path.

The conditional receipt candidate additionally has an isolated ingress test.
With the documented CLI on `PATH`, run from accounts/code:

```sh
GOWORK=off go test -race -tags integration ./pkg/adapters -run '^TestSourceOperationVerifyPostgresIngress$' -count=1 -timeout=15m
```

It uses the owner's Codefly dependency harness, Postgres/Vault, real signed
sessions, generated Connect client and HTTP POST gateway. It qualifies real
Invoke persistence, caller response loss, binding refusals, legacy Lookup,
current read access, pruning and receipt-pool reopening. Only outbound provider
transport and the retention clock use test seams. It does not qualify a complete
process restart, module-delegation composition, external gateway/proxy deployment,
TypeScript consumer adoption, or production raw body limits.
15 changes: 11 additions & 4 deletions module/REST_SURFACE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ module ceiling. The complete regeneration includes these additions and the calla

## Current surface

The accounts projection contains 175 descriptor routes across 30 services,
The accounts projection contains 176 descriptor routes across 30 services,
counted from `generated/rest-surface.json`:

- 18 public routes and 157 authenticated routes;
- 148 OpenAPI paths and 175 operations;
- 18 public routes and 158 authenticated routes;
- 149 OpenAPI paths and 176 operations;
- zero internal RPCs;
- fifteen explicit non-protobuf extensions loaded by auth-gateway: magic-link
request/verification, the billing and Resend email webhooks, the two GitHub
delivery receivers, the public status surface, the subscriptions stream,
checkout, free-plan, portal, and the four OAuth 2 authorization-server routes
(metadata, authorize validation, authorize grant, token).

The generated runtime therefore authorizes 190 REST routes in total. Descriptor
The generated runtime therefore authorizes 191 REST routes in total. Descriptor
routes and extensions remain separate so an extension can never be mistaken
for a protobuf procedure or inherit policy by path similarity.

Expand Down Expand Up @@ -119,6 +119,13 @@ from 327 to 324.

## Regeneration

The local candidate `VerifySourceOperationReceipt` is POST-only at
`/v1/datasource-operation-receipts/verify`, with the complete original Invoke
request in its body. It has no query binding, classifies both directions as secret,
and reuses current stored-source read authority and bounded receipt recovery.
See [conditional verification](./CALLABLE_SOURCES.md#conditional-receipt-verification-local-owner-candidate)
for strict declaration, committed/unknown/refusal semantics and adoption limits.

Run from `module/services/accounts`, with Docker running and a Codefly CLI at
or above 0.1.160:

Expand Down
Binary file modified module/contracts/api/accounts/authority/contract.binpb
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -1662,6 +1662,42 @@ service DatasourceService {
};
}

// VerifySourceOperationReceipt conditionally reads retained output using the
// exact original Invoke request, including a nonempty declaration precondition.
// Committed means the normalized request and admitted declaration match the
// stored attempt. Unknown remains unresolved; this method never redispatches.
// Current actor, tenant and stored-source read authority apply before comparison.
// Source-scope refusals emit the existing failure audit; successful reads do not
// record another invocation or change the retained receipt.
// This is independent of the paired Runnable Lookup contract above.
rpc VerifySourceOperationReceipt(InvokeSourceOperationRequest) returns (InvokeSourceOperationResponse) {
option (google.api.http) = {
post: "/v1/datasource-operation-receipts/verify"
body: "*"
};
option (saas.policy.v1.method_policy) = {
exposure: EXPOSURE_AUTHENTICATED
tenant: TENANT_REQUIREMENT_ORG_MEMBER
permissions: "datasource:read"
scopes: "datasource:read"
resource_bindings: {
request_field: "effect_id"
target: RESOURCE_TARGET_OWNED_RESOURCE
lookup: RESOURCE_LOOKUP_RESOURCE_TO_ORGANIZATION
}
mfa: MFA_REQUIREMENT_NONE
audit: {
events: "saas.datasource.operation.invoked"
emission: AUDIT_EMISSION_FAILURE
}
idempotency: IDEMPOTENCY_REQUIREMENT_FORBIDDEN
rate_limit: RATE_LIMIT_CLASS_STANDARD_READ
request_sensitivity: SENSITIVITY_SECRET
response_sensitivity: SENSITIVITY_SECRET
platform_role: PLATFORM_ROLE_REQUIREMENT_NONE
};
}

// ListSourceOperations reads the currently admitted declarations under source authority.
rpc ListSourceOperations(ListSourceOperationsRequest) returns (ListSourceOperationsResponse) {
option (google.api.http) = {get: "/v1/organizations/{org_id}/datasources/{source_id}/operations"};
Expand Down
Binary file modified module/contracts/api/accounts/connect/contract.binpb
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -1662,6 +1662,42 @@ service DatasourceService {
};
}

// VerifySourceOperationReceipt conditionally reads retained output using the
// exact original Invoke request, including a nonempty declaration precondition.
// Committed means the normalized request and admitted declaration match the
// stored attempt. Unknown remains unresolved; this method never redispatches.
// Current actor, tenant and stored-source read authority apply before comparison.
// Source-scope refusals emit the existing failure audit; successful reads do not
// record another invocation or change the retained receipt.
// This is independent of the paired Runnable Lookup contract above.
rpc VerifySourceOperationReceipt(InvokeSourceOperationRequest) returns (InvokeSourceOperationResponse) {
option (google.api.http) = {
post: "/v1/datasource-operation-receipts/verify"
body: "*"
};
option (saas.policy.v1.method_policy) = {
exposure: EXPOSURE_AUTHENTICATED
tenant: TENANT_REQUIREMENT_ORG_MEMBER
permissions: "datasource:read"
scopes: "datasource:read"
resource_bindings: {
request_field: "effect_id"
target: RESOURCE_TARGET_OWNED_RESOURCE
lookup: RESOURCE_LOOKUP_RESOURCE_TO_ORGANIZATION
}
mfa: MFA_REQUIREMENT_NONE
audit: {
events: "saas.datasource.operation.invoked"
emission: AUDIT_EMISSION_FAILURE
}
idempotency: IDEMPOTENCY_REQUIREMENT_FORBIDDEN
rate_limit: RATE_LIMIT_CLASS_STANDARD_READ
request_sensitivity: SENSITIVITY_SECRET
response_sensitivity: SENSITIVITY_SECRET
platform_role: PLATFORM_ROLE_REQUIREMENT_NONE
};
}

// ListSourceOperations reads the currently admitted declarations under source authority.
rpc ListSourceOperations(ListSourceOperationsRequest) returns (ListSourceOperationsResponse) {
option (google.api.http) = {get: "/v1/organizations/{org_id}/datasources/{source_id}/operations"};
Expand Down
7 changes: 4 additions & 3 deletions module/contracts/api/catalog.codefly.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"kind": "protobuf",
"package": "saas.accounts.v1",
"path": "contracts/api/accounts/authority/contract.binpb",
"digest": "sha256:606e52170c2e55688ce163eb1d54c347f0d19ac40915d7fb6252f2034b787c0e",
"digest": "sha256:9aaed4fe30672e6c11675034ae204cec14ecbb299d68a3898b65780a88f80657",
"services": [
{
"name": "ModuleCapabilitiesService",
Expand Down Expand Up @@ -66,7 +66,7 @@
"kind": "protobuf",
"package": "",
"path": "contracts/api/accounts/connect/contract.binpb",
"digest": "sha256:606e52170c2e55688ce163eb1d54c347f0d19ac40915d7fb6252f2034b787c0e",
"digest": "sha256:9aaed4fe30672e6c11675034ae204cec14ecbb299d68a3898b65780a88f80657",
"services": [
{
"name": "ArtifactService",
Expand Down Expand Up @@ -259,7 +259,8 @@
"/saas.accounts.v1.DatasourceService/SyncSource",
"/saas.accounts.v1.DatasourceService/UnbindDatasourceGroup",
"/saas.accounts.v1.DatasourceService/UpdateSource",
"/saas.accounts.v1.DatasourceService/VerifyDatasourceDomain"
"/saas.accounts.v1.DatasourceService/VerifyDatasourceDomain",
"/saas.accounts.v1.DatasourceService/VerifySourceOperationReceipt"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions module/module.package.codefly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ services:
kind: protobuf
package: saas.accounts.v1
path: contracts/api/accounts/authority/contract.binpb
digest: sha256:606e52170c2e55688ce163eb1d54c347f0d19ac40915d7fb6252f2034b787c0e
digest: sha256:9aaed4fe30672e6c11675034ae204cec14ecbb299d68a3898b65780a88f80657
- endpoint: connect
kind: protobuf
package: ""
path: contracts/api/accounts/connect/contract.binpb
digest: sha256:606e52170c2e55688ce163eb1d54c347f0d19ac40915d7fb6252f2034b787c0e
digest: sha256:9aaed4fe30672e6c11675034ae204cec14ecbb299d68a3898b65780a88f80657
- name: auth-gateway
endpoints: [grpc, rest]
api-contracts:
Expand Down
5 changes: 3 additions & 2 deletions module/services/accounts/AUTHZ_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> Generated from protobuf service descriptors and `saas.policy.v1.method_policy`; only the prose description is joined from `pkg/business/introspection.go`. Do not edit by hand. Run `go generate ./pkg/business` from `module/services/accounts/code`.

Inventory: **301 RPCs** across **42 services**. Missing, invalid, or unclassified procedures are denied by both Connect and gRPC interceptors.
Inventory: **302 RPCs** across **42 services**. Missing, invalid, or unclassified procedures are denied by both Connect and gRPC interceptors.

The compact tier is retained for compatibility. Guards, resource bindings, audit events, limiter class, and data sensitivity are descriptor-authoritative. Domain handlers may enforce stronger state-dependent rules but may not weaken this floor.

Expand Down Expand Up @@ -106,6 +106,7 @@ The compact tier is retained for compatibility. Guards, resource bindings, audit
| `/saas.accounts.v1.DatasourceService/UnbindDatasourceGroup` | unary | `—` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.datasource.group_unbound | FORBIDDEN / STANDARD_WRITE | INTERNAL → CONFIDENTIAL | Remove a provider group binding. |
| `/saas.accounts.v1.DatasourceService/UpdateSource` | unary | `—` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.datasource.source.updated | FORBIDDEN / STANDARD_WRITE | INTERNAL → CONFIDENTIAL | Change what a connected source fetches — its branch and path and file-type filters — keeping its credential, history and collection. |
| `/saas.accounts.v1.DatasourceService/VerifyDatasourceDomain` | unary | `—` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.datasource.domain_verified | FORBIDDEN / STANDARD_WRITE | INTERNAL → CONFIDENTIAL | Verify a claimed domain by its DNS TXT record. |
| `/saas.accounts.v1.DatasourceService/VerifySourceOperationReceipt` | unary | `POST /v1/datasource-operation-receipts/verify` | `org_member` | exposure=AUTHENTICATED; tenant=ORG_MEMBER | perm=datasource:read; scope=datasource:read | effect_id → OWNED_RESOURCE/RESOURCE_TO_ORGANIZATION | FAILURE: saas.datasource.operation.invoked | FORBIDDEN / STANDARD_READ | SECRET → SECRET | Read a source receipt only when the original invocation and admitted declaration match. |
| `/saas.accounts.v1.DelegationService/DecideDelegation` | unary | `POST /v1/delegations/{id}:decide` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.delegation.approved, saas.delegation.denied | FORBIDDEN / STANDARD_READ | CONFIDENTIAL → CONFIDENTIAL | Approve or deny a delegation request. |
| `/saas.accounts.v1.DelegationService/ListPendingDelegations` | unary | `GET /v1/delegations:pending` | `org_member` | exposure=AUTHENTICATED; tenant=ORG_MEMBER | — | org_id → ORGANIZATION/DIRECT_ID | — | FORBIDDEN / STANDARD_READ | CONFIDENTIAL → CONFIDENTIAL | List pending organization delegations. |
| `/saas.accounts.v1.DelegationService/RequestDelegation` | unary | `POST /v1/delegations` | `org_member` | exposure=AUTHENTICATED; tenant=ORG_MEMBER | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.delegation.requested, saas.delegation.auto_approved | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Request a scoped authority delegation. |
Expand Down Expand Up @@ -316,7 +317,7 @@ The compact tier is retained for compatibility. Guards, resource bindings, audit
- `internal`: 55
- `mfa`: 5
- `org_admin`: 71
- `org_member`: 57
- `org_member`: 58
- `platform_admin`: 29
- `public`: 32

Expand Down
Loading
Loading