Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
413084a
feat(accounts): the object-storage connector diffs the current listin…
antoinetoussaint-byte Oct 21, 2026
7c53397
feat(accounts,saas-ui): a bucket source takes include and exclude key…
antoinetoussaint-byte Oct 21, 2026
4762aa8
refactor(accounts,saas-ui): the bucket key filter is the connector's …
antoinetoussaint-byte Oct 23, 2026
264f6d6
Merge remote-tracking branch 'origin/issue-1211-bucket-key-patterns' …
antoinetoussaint-byte Oct 23, 2026
6178584
feat(accounts): object-storage sources read through the composed obje…
antoinetoussaint-byte Oct 23, 2026
d2fc5fc
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 23, 2026
36b0b36
build(ci): the object-storage service is covered by the Codefly vendo…
antoinetoussaint-byte Oct 23, 2026
f06038c
build(accounts): the object-storage dependency's generated client, fr…
antoinetoussaint-byte Oct 23, 2026
a3c4836
feat(accounts): object storage reads a batch in one call and is regis…
antoinetoussaint-byte Oct 23, 2026
bafae16
test(accounts): a vanished store never closes an object-storage sync,…
antoinetoussaint-byte Oct 23, 2026
cb69977
build(accounts): service-object-storage at eef1863, #111's head; the …
antoinetoussaint-byte Oct 23, 2026
7210303
fix(accounts): an object-storage source lists its literal prefix, adm…
antoinetoussaint-byte Oct 23, 2026
9d49690
style(accounts): the object-storage connector's two unchecked writes,…
antoinetoussaint-byte Oct 23, 2026
7f3602f
build(accounts): service-object-storage at a3157779, #111's fixed com…
antoinetoussaint-byte Oct 23, 2026
419a96d
test(accounts): the pinned object-storage service is held to the two …
antoinetoussaint-byte Oct 23, 2026
a33f4b5
style(accounts): the contract test's two unchecked closes, which gola…
antoinetoussaint-byte Oct 23, 2026
8dc4ebe
test(accounts): a composed object-storage service that cannot serve a…
antoinetoussaint-byte Oct 23, 2026
a138094
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 24, 2026
32eb6c8
build(accounts): object storage runs on service-object-storage v0.0.1…
antoinetoussaint-byte Oct 24, 2026
fd4af14
fix(accounts): a sync never empties an object-storage source that has…
antoinetoussaint-byte Oct 24, 2026
93d143d
fix(accounts): a refused object-storage source parks through the one …
antoinetoussaint-byte Oct 24, 2026
f47ff97
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 25, 2026
eee04f9
chore(frontend): kit 0.13.3 and saas-sdk 0.3.37, bumped from main's n…
antoinetoussaint-byte Oct 25, 2026
c338179
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 25, 2026
cb21332
fix(store): renumber the object-storage migrations above main's front…
antoinetoussaint-byte Oct 25, 2026
bb92cab
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 26, 2026
3921fc8
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 26, 2026
315dd36
fix(store): renumber the object-storage migrations above main's front…
antoinetoussaint-byte Oct 26, 2026
923f333
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 26, 2026
b60b3f1
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 26, 2026
d48ab00
fix(store): renumber the object-storage migrations above main's front…
antoinetoussaint-byte Oct 26, 2026
36a8469
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 27, 2026
c8d7091
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 27, 2026
1bd7f0d
fix(store): renumber the object-storage migrations above main's front…
antoinetoussaint-byte Oct 27, 2026
72b2ea0
Merge branch 'main' into issue-1211-object-storage-through-the-service
antoinetoussaint-byte Oct 27, 2026
b31a708
Merge branch 'main' into issue-1211-object-storage-through-the-service
antoinetoussaint-byte Oct 27, 2026
248691f
Merge branch 'main' into issue-1211-object-storage-through-the-service
antoinetoussaint-byte Oct 27, 2026
f28c961
Merge remote-tracking branch 'origin/main' into issue-1211-object-sto…
antoinetoussaint-byte Oct 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions RELEASE_GATES.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,10 @@ A successful image build and matching materials alone do not establish that a
generated-file change belongs in the PR. Reproduce recipes using the CI-pinned
CLI and the released agents selected by the service manifests.
Every topology service must declare image coverage, even if its recipe is absent
from the checkout. Redis and Vault explicitly use the existing Codefly vendor
image audit; an unknown agent is an error.
from the checkout. Redis, Vault and object storage explicitly use the existing
Codefly vendor image audit — the object-storage agent runs its gateway and MinIO
images, both pinned by the agent, and builds nothing here; an unknown agent is
an error.

Changes to the image contract force the full service graph through CI. Before
each canonical build attempt, CI snapshots Buildx history. After the build,
Expand Down
2 changes: 1 addition & 1 deletion libraries/source-read-sdk/library.codefly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ sources:
module: saas-starter
service: accounts
endpoint: authority
contract-digest: sha256:10a231ea430313364937808160138bca599ac012230ad70b7047eac27764836a
contract-digest: sha256:743b63bad9a546f105e994c754d845865e189465e6c72816e5785efe38ab7831
services:
- ModuleCapabilitiesService
generated-by:
Expand Down
2 changes: 1 addition & 1 deletion module/DATABASE_AUTHORITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ executable inventory and this table in the same change.
| Scope | Relations | Required database boundary |
|---|---|---|
| `global` | `audit_event_types`, `authority_core_delegations`, `authority_core_grant_epochs`, `authority_core_installations`, `authority_core_module_revisions`, `authority_core_modules`, `authority_core_owner_aware`, `authority_core_person_epochs`, `authority_core_replay`, `authority_core_role_grants`, `authority_core_tombstones`, `bootstrap_state`, `data_retention_policies`, `datasource_credential_budgets`, `email_templates`, `feature_flags`, `hop_admission_keys`, `hop_admission_proofs`, `identity_providers`, `module_envelopes`, `plan_entitlements`, `plans`, `platform_admins`, `solution_delivery_documents`, `solution_generation_history`, `solution_host_bindings`, `solution_registrations`, `solution_targets` | No RLS; exact grants |
| `tenant` | `actor_chain_journal`, `actor_chain_revocations`, `api_keys`, `approval_decisions`, `approval_requests`, `audit_event_idempotency`, `audit_events`, `connector_credentials`, `dashboards`, `datasource_account_links`, `datasource_domains`, `datasource_group_bindings`, `codefly_effect_receipts`, `datasource_operation_attempts`, `datasource_oauth_consumed_states`, `datasource_source_operations`, `datasource_sources`, `delegation_grants`, `domain_events`, `entitlement_overrides`, `executable_artifact_approvals`, `github_app_installations`, `github_app_setups`, `installations`, `invitations`, `membership_integrity_findings`, `org_generic_settings`, `org_identity_providers`, `org_key_bindings`, `org_settings`, `organization_activations`, `organization_authorization_revisions`, `organization_members`, `organizations`, `principal_authorization_revisions`, `principals`, `record_shares`, `role_assignments`, `role_permissions`, `roles`, `scope_grants`, `scope_nodes`, `source_delegations`, `source_read_revisions`, `subscriptions`, `team_members`, `team_membership_quarantine`, `teams`, `usage_events`, `usage_totals`, `webhook_deliveries`, `webhook_subscriptions`, `work_context_replay` | Enabled and forced RLS with at least one policy |
| `tenant` | `actor_chain_journal`, `actor_chain_revocations`, `api_keys`, `approval_decisions`, `approval_requests`, `audit_event_idempotency`, `audit_events`, `connector_credentials`, `dashboards`, `datasource_account_links`, `datasource_domains`, `datasource_group_bindings`, `datasource_object_manifests`, `codefly_effect_receipts`, `datasource_operation_attempts`, `datasource_oauth_consumed_states`, `datasource_source_operations`, `datasource_sources`, `delegation_grants`, `domain_events`, `entitlement_overrides`, `executable_artifact_approvals`, `github_app_installations`, `github_app_setups`, `installations`, `invitations`, `membership_integrity_findings`, `org_generic_settings`, `org_identity_providers`, `org_key_bindings`, `org_settings`, `organization_activations`, `organization_authorization_revisions`, `organization_members`, `organizations`, `principal_authorization_revisions`, `principals`, `record_shares`, `role_assignments`, `role_permissions`, `roles`, `scope_grants`, `scope_nodes`, `source_delegations`, `source_read_revisions`, `subscriptions`, `team_members`, `team_membership_quarantine`, `teams`, `usage_events`, `usage_totals`, `webhook_deliveries`, `webhook_subscriptions`, `work_context_replay` | Enabled and forced RLS with at least one policy |
| `user` | `client_authorization_codes`, `gdpr_requests`, `mfa_backup_codes`, `mfa_devices`, `mfa_login_transactions`, `notifications`, `onboarding_progress`, `resource_follows`, `sessions`, `user_consent_events`, `user_consent_preferences`, `user_identities`, `users`, `webauthn_ceremonies`, `webauthn_credentials` | Enabled and forced RLS with at least one policy |
| `pre_auth` | `magic_links`, `waitlist_entries` | Enabled and forced RLS; fail-closed request policy, accessed only by the control-plane role |
| `job` | `job_messages` | Enabled and forced RLS with at least one policy; no request relation grant — function-only scoped enqueue plus exact job-worker grants |
Expand Down
1 change: 1 addition & 0 deletions module/DEPLOYMENT_TOPOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ REST without a corresponding accounts endpoint, generation fails.
| Caller | Target endpoints | Kubernetes ports |
| --- | --- | --- |
| `accounts` | `cache/read`, `cache/write` | TCP 6379 |
| `accounts` | `object-storage/grpc` | Codefly-assigned gRPC port |
| `accounts` | `store/tcp` | TCP 5432 |
| `accounts` | `vault/http` | TCP 8200 |
| `auth-gateway` | `accounts/connect`, `accounts/rest`, `accounts/grpc` | TCP 8080, 9090 |
Expand Down
Binary file modified module/contracts/api/accounts/authority/contract.binpb
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,48 @@ message UploadDatasourceConfig {
// AWS-style access key id. The matching secret access key is the credential.
string access_key_id = 5;
// Upper bound on objects fetched per sync. Zero applies the connector default.
// Only objects that pass the filters below count against it.
uint32 max_objects = 6;
// Which keys under the prefix are ingested. Each pattern is an RE2 regular
// expression matched against the object key with the prefix removed,
// unanchored and case-sensitive (anchor with ^ and $, fold case with (?i)).
// An empty include admits every key; an exclude match always wins. Applied
// when the prefix is listed, before anything is fetched, so an excluded
// object is never read and never counts against max_objects. An invalid
// expression is refused when the source is added or updated, never at sync:
// the call fails INVALID_ARGUMENT with a google.rpc.BadRequest violation, and
// a message that starts with the field's path, e.g.
// "upload.include_patterns[1]: …".
repeated string include_patterns = 7 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
repeated string exclude_patterns = 8 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
// Case-insensitive key suffixes, e.g. [".md", ".pdf"], intersected with the
// patterns. Empty admits every type. GitHub's file_extensions semantics.
repeated string file_extensions = 9 [(buf.validate.field).repeated = {
max_items: 32
items: {
string: {
min_len: 2
max_len: 33
pattern: "^\\.[A-Za-z0-9][A-Za-z0-9._-]*$"
}
}
}];
}

// Datasource is one connected external datasource. It is org-scoped and carries
Expand Down Expand Up @@ -437,6 +478,27 @@ message UpdateSourceRequest {
// sole reason that the source had reconciled since the form was opened. What
// an editor needs guarded is what they were looking at.
SourceFetchScope expected = 6 [(buf.validate.field).required = true];
// An object-storage source's key patterns (see UploadDatasourceConfig),
// replaced whole like every field here. Its file_extensions is field 5; its
// branch and paths must be empty. Refused for a github source.
repeated string include_patterns = 7 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
repeated string exclude_patterns = 8 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
}

// SourceFetchScope is what a source fetches — the three values UpdateSource
Expand Down Expand Up @@ -467,6 +529,17 @@ message SourceFetchScope {
max_len: 33
}
}
}]; repeated string include_patterns = 4 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {max_len: 256}
}
}];
repeated string exclude_patterns = 5 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {max_len: 256}
}
}];
}

Expand Down
Binary file modified module/contracts/api/accounts/connect/contract.binpb
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,48 @@ message UploadDatasourceConfig {
// AWS-style access key id. The matching secret access key is the credential.
string access_key_id = 5;
// Upper bound on objects fetched per sync. Zero applies the connector default.
// Only objects that pass the filters below count against it.
uint32 max_objects = 6;
// Which keys under the prefix are ingested. Each pattern is an RE2 regular
// expression matched against the object key with the prefix removed,
// unanchored and case-sensitive (anchor with ^ and $, fold case with (?i)).
// An empty include admits every key; an exclude match always wins. Applied
// when the prefix is listed, before anything is fetched, so an excluded
// object is never read and never counts against max_objects. An invalid
// expression is refused when the source is added or updated, never at sync:
// the call fails INVALID_ARGUMENT with a google.rpc.BadRequest violation, and
// a message that starts with the field's path, e.g.
// "upload.include_patterns[1]: …".
repeated string include_patterns = 7 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
repeated string exclude_patterns = 8 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
// Case-insensitive key suffixes, e.g. [".md", ".pdf"], intersected with the
// patterns. Empty admits every type. GitHub's file_extensions semantics.
repeated string file_extensions = 9 [(buf.validate.field).repeated = {
max_items: 32
items: {
string: {
min_len: 2
max_len: 33
pattern: "^\\.[A-Za-z0-9][A-Za-z0-9._-]*$"
}
}
}];
}

// Datasource is one connected external datasource. It is org-scoped and carries
Expand Down Expand Up @@ -437,6 +478,27 @@ message UpdateSourceRequest {
// sole reason that the source had reconciled since the form was opened. What
// an editor needs guarded is what they were looking at.
SourceFetchScope expected = 6 [(buf.validate.field).required = true];
// An object-storage source's key patterns (see UploadDatasourceConfig),
// replaced whole like every field here. Its file_extensions is field 5; its
// branch and paths must be empty. Refused for a github source.
repeated string include_patterns = 7 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
repeated string exclude_patterns = 8 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {
min_len: 1
max_len: 256
}
}
}];
}

// SourceFetchScope is what a source fetches — the three values UpdateSource
Expand Down Expand Up @@ -467,6 +529,17 @@ message SourceFetchScope {
max_len: 33
}
}
}]; repeated string include_patterns = 4 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {max_len: 256}
}
}];
repeated string exclude_patterns = 5 [(buf.validate.field).repeated = {
max_items: 16
items: {
string: {max_len: 256}
}
}];
}

Expand Down
4 changes: 2 additions & 2 deletions module/contracts/api/catalog.codefly.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"kind": "protobuf",
"package": "saas.accounts.v1",
"path": "contracts/api/accounts/authority/contract.binpb",
"digest": "sha256:10a231ea430313364937808160138bca599ac012230ad70b7047eac27764836a",
"digest": "sha256:743b63bad9a546f105e994c754d845865e189465e6c72816e5785efe38ab7831",
"services": [
{
"name": "ModuleCapabilitiesService",
Expand Down Expand Up @@ -66,7 +66,7 @@
"kind": "protobuf",
"package": "",
"path": "contracts/api/accounts/connect/contract.binpb",
"digest": "sha256:10a231ea430313364937808160138bca599ac012230ad70b7047eac27764836a",
"digest": "sha256:743b63bad9a546f105e994c754d845865e189465e6c72816e5785efe38ab7831",
"services": [
{
"name": "ArtifactService",
Expand Down
1 change: 1 addition & 0 deletions module/deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,7 @@ services:
- name: cache
- name: frontend
- name: marketing
- name: object-storage
- name: store
- name: telemetry
# no `vault`: KEY_SERVICE_BACKEND=kms with
Expand Down
23 changes: 23 additions & 0 deletions module/deployment/generated/service-topology.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions module/module.codefly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,6 @@ services:
- name: cache
- name: frontend
- name: marketing
- name: object-storage
- name: store
- name: vault
6 changes: 4 additions & 2 deletions module/module.package.codefly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ services:
kind: protobuf
package: saas.accounts.v1
path: contracts/api/accounts/authority/contract.binpb
digest: sha256:10a231ea430313364937808160138bca599ac012230ad70b7047eac27764836a
digest: sha256:743b63bad9a546f105e994c754d845865e189465e6c72816e5785efe38ab7831
- endpoint: connect
kind: protobuf
package: ""
path: contracts/api/accounts/connect/contract.binpb
digest: sha256:10a231ea430313364937808160138bca599ac012230ad70b7047eac27764836a
digest: sha256:743b63bad9a546f105e994c754d845865e189465e6c72816e5785efe38ab7831
- name: auth-gateway
endpoints: [grpc, rest]
api-contracts:
Expand All @@ -38,6 +38,8 @@ services:
endpoints: [http]
- name: marketing
endpoints: [http]
- name: object-storage
endpoints: [grpc]
- name: store
endpoints: [tcp]
- name: vault
Expand Down
6 changes: 6 additions & 0 deletions module/services/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ wrong, and nothing enforces it.
`marketing` and `frontend` are separate Next.js applications: marketing owns
public apex/`www` content, while frontend remains the authenticated product
behind `auth-gateway`.
`object-storage` is the composed object-storage gateway
(codefly-dev/service-object-storage). The host keeps no store client of its
own: accounts reads every object-storage datasource through it, under a
per-call binding of the source's bucket and credential, and the gateway owns
the store protocol and the public-destination guard — see
`accounts/AGENTS.md`.

Run the complete local dependency graph from `module/`:

Expand Down
Loading
Loading