diff --git a/configurations/local/workload-identity.env b/configurations/local/workload-identity.env new file mode 120000 index 000000000..8f6d6273b --- /dev/null +++ b/configurations/local/workload-identity.env @@ -0,0 +1 @@ +../../module/configurations/local/workload-identity.env \ No newline at end of file diff --git a/module/DEPLOYMENT_TOPOLOGY.md b/module/DEPLOYMENT_TOPOLOGY.md index 76a180eba..6fc5c5dc3 100644 --- a/module/DEPLOYMENT_TOPOLOGY.md +++ b/module/DEPLOYMENT_TOPOLOGY.md @@ -48,11 +48,17 @@ depend on the database whose authority it witnesses. | `accounts` | `vault/http` | TCP 8200 | | `auth-gateway` | `accounts/connect`, `accounts/rest`, `accounts/grpc` | TCP 8080, 9090 | | `auth-gateway` | `cache/write` | TCP 6379 | -| `frontend` | `auth-gateway/rest` | TCP 8080 | +| `frontend` | `auth-gateway/rest` | Codefly-assigned HTTP port | The Codefly module interface exposes the public `frontend/http` and `marketing/http` endpoints; the auth-gateway gRPC ext-authz endpoint has module -visibility. Istio routes apex/`www`/docs hosts to `marketing/http` and `app` to +visibility. Its `rest` and `workload` endpoints are also module-visible; +`rest` serves plain HTTP and `workload` serves the same routes over TLS 1.3 with +required workload client certificates. Both ports are allocated by Codefly. +The gateway declares the platform-provided `workload-identity` configuration +group and refuses startup without its projected certificate, key, trust bundle, +caller allowlist and token files. Solution upstreams require mutual TLS and the +SPIFFE identity from applied presence. Istio routes apex/`www`/docs hosts to `marketing/http` and `app` to `frontend/http`. Accounts, frontend, marketing, and telemetry may reach public IP space only over TCP 443. The public rules exclude private, loopback, link-local, metadata, documentation, benchmark, multicast, and other diff --git a/module/INTERNAL_TRANSPORT.md b/module/INTERNAL_TRANSPORT.md index c05fec240..77e05a180 100644 --- a/module/INTERNAL_TRANSPORT.md +++ b/module/INTERNAL_TRANSPORT.md @@ -167,14 +167,17 @@ fail closed rather than fall back to another key when the set expires. ## The endpoint a composed module depends on: `accounts/authority` -The module interface exports four endpoints with `visibility: internal` and +The module interface exports five endpoints with `visibility: internal` and `allow-modules: ["*"]`. This explicitly preserves their reachability by every composed module; it grants neither public ingress nor application authority. -The retired `module` spelling is refused by current Codefly. Three do not +The retired `module` spelling is refused by current Codefly. Four do not carry this tier: `accounts/connect` and `auth-gateway/grpc` are tenant surfaces -whose tiering interceptor refuses internal methods, and `auth-gateway/rest` +whose tiering interceptor refuses internal methods, `auth-gateway/rest` carries the brokered bootstrap routes (registration, Work Context mints) and the -key set. The fourth, `accounts/authority`, is a gRPC endpoint with a listener of +key set, and `auth-gateway/workload` serves the same gateway routes over mutual +TLS to workloads that present a platform-issued SVID listed in its +allowed-callers file, where a module mint must also come from the SPIFFE ID its +delivered presence declares. The fifth, `accounts/authority`, is a gRPC endpoint with a listener of its own, and it is the internal tier **narrowed**: - **Only the module surface is served.** `business.ModuleAuthorityProcedures` diff --git a/module/MODULE_INSTALLATION.md b/module/MODULE_INSTALLATION.md index cf8638db1..11193d617 100644 --- a/module/MODULE_INSTALLATION.md +++ b/module/MODULE_INSTALLATION.md @@ -133,12 +133,18 @@ capability in `X-Codefly-Work-Context` and the same JSON body: Accounts resolves `organizationSlug` through its existing organization store, then compares the persisted ID with both token tenant and delegation. It checks the current owner's administrative eligibility and role permission -contents before mutation. Inspect returns `state: absent` or `state: ready` and -`changed: false`; apply returns `state: ready` and reports whether it created -anything. Verify fails on an absent or incompatible installation. Ready responses +contents before mutation. Inspect returns `state: absent`, `pending`, `failed`, +or `ready` and `changed: false`; apply reports the current state and whether it +created anything. Solution installations are pending until the workload install +step acknowledges. Reapplying observes that run; it never restarts a failed run. +An organization administrator retries failed consent through RetryInstallation. +Verify refuses absent, pending, failed, and incompatible installations. Existing responses include `organizationId`, `principalId`, `installationId`, `scopeNodeId`, and `grantId`. Absent responses contain only `organizationId` alongside state and changed. IDs are always read from the server, including after a lost response. +Only `ready` may carry a verified authority reference. Declaration resolution +uses the same registry binding as InstallSolution; this delegated path grants +only its approved agent ceiling, with empty module-capability consent. HTTP statuses: 400 invalid shape/unknown fields; 401 missing, expired, wrongly signed or wrong-audience capability; 403 missing/expired/revoked delegation or diff --git a/module/SOLUTION_REGISTRATION.md b/module/SOLUTION_REGISTRATION.md index 7696911d9..fa18d66c1 100644 --- a/module/SOLUTION_REGISTRATION.md +++ b/module/SOLUTION_REGISTRATION.md @@ -290,12 +290,32 @@ credential. The split is the point. | Which release runs | the declaration | `release`, recorded as `applied_release` | | Which route alias it answers on | the declaration | `routes[].alias` | | The workload identity to expect | the declaration | `workload` | -| Resolved upstream and frontend | host reconciliation | `backend_upstream`, `frontend_manifest` | +| Resolved upstream and frontend | the running workload, at its mint | `backend_upstream`, `frontend_manifest` | The document deliberately carries no upstream address and no manifest. An address written into a delivery document is a resolution result that was true on one cluster until something moved; a manifest is what the running build actually -published. Both are observations that the host resolves under declared authority. +published. Both are observations that the host records under declared authority. + +**The workload reports them with its mint.** `POST /modules/_work-context` +carries two optional fields beside `prefix`: `upstream`, the https origin the +gateway dials to reach the workload, and `manifest`, the frontend manifest it +serves. Accounts records them (`RecordModuleUpstream`) only after the projected +token and the running build have been bound exactly as for a mint, only onto the +registration the applied presence already declares, and never onto the static +half — the alias, the target and the workload identity stay the delivery's. No +installation is required: a solution nobody has installed must still be reachable +for its install step, which is delivered to the origin recorded here. + +The dynamic half is accepted only on the `workload` endpoint, where the +reporter's identity is on the connection; on the plain endpoint the request is +refused. The origin is bounded twice, at the gateway and again in accounts: `https`, a +host and a port and nothing else, on a composition-local host (loopback or +private address, a bare service name, or a cluster-local name). It cannot +redirect a solution's traffic to another workload: the gateway reaches a +recorded origin only over TLS to the SPIFFE identity the applied presence names. +The request body is bounded at 80 KiB at the gateway. A report that is refused +refuses the mint. ### The registry key is the route alias @@ -395,7 +415,7 @@ Every key is in the `federation` group. | --- | --- | | `SOLUTION_HOST_COORDINATE` | the coordinate this host answers for, exactly as the operator declared it on the environment the renderer read. Never derived here — an invented coordinate matches nothing delivery wrote, so every document would be refused. It is also the one declaration that turns the surface **on**: empty means no reconciler and no delivery endpoint, and no new declared presence can be reconciled. | | `SOLUTION_HOST_OWNERSHIP_DOMAINS` | the ownership domains this host accepts delivery from. Required with the coordinate and refusing to boot when empty, for the same reason the coordinate is: it is the only thing bounding a binding's first generation. | -| *(no setting)* | WHERE the pinned-key verification policy is read from is **not configurable**: it is a constant, `infra.SolutionHostTrustAnchorPath`. There is no environment value, no flag and no overridable default. Workspace environment is delivered by the **composition**, so an overridable path would let a composition point this verifier at a policy it wrote itself, sign its own presence documents, and pass every downstream check — against an anchor it chose. A deployment needing a different path changes the pod spec that mounts it, which the platform renders and the composition cannot supply. That document also carries the signer-to-domain mapping, keyed by `keyId`, so workspace environment cannot widen a key's authority independently of its pin. A signer granted a domain `SOLUTION_HOST_OWNERSHIP_DOMAINS` does not accept is refused **by name**. | +| *(no setting in a pod)* | WHERE the pinned-key verification policy is read from is **not configurable in a deployment**: it is a constant, `infra.SolutionHostTrustAnchorPath`. In a pod there is no environment value, no flag and no overridable default; the one relocation, for a host started as a process, is refused in a pod and described below. Workspace environment is delivered by the **composition**, so an overridable path would let a composition point this verifier at a policy it wrote itself, sign its own presence documents, and pass every downstream check — against an anchor it chose. A deployment needing a different path changes the pod spec that mounts it, which the platform renders and the composition cannot supply. That document also carries the signer-to-domain mapping, keyed by `keyId`, so workspace environment cannot widen a key's authority independently of its pin. A signer granted a domain `SOLUTION_HOST_OWNERSHIP_DOMAINS` does not accept is refused **by name**. | | `SOLUTION_HOST_TRUST_POLICY` | how a delivered carrier's envelope is checked. `keys` is the only value. Required with the coordinate, with **no default** — every default is wrong somewhere. The trust anchor's **absence** is checked first and refuses the boot naming the anchor, so an unmounted anchor is never reported as a missing setting. | | `SOLUTION_HOST_BINDING_INTERVAL` | optional; how often the inbox is re-read. Empty uses 30s. | @@ -407,6 +427,28 @@ TokenReview can be performed — a delivered carrier is authorised by TokenRevie so a host that cannot reach its API server has no input at all and starting would leave it polling an inbox nothing can write to. +**A process on a developer machine may relocate both, and a pod never can.** +`codefly run` starts the host as a process, where the kubelet's paths do not +exist and writing them needs root. Two settings name a local stand-in, and each +is **refused at boot when the process runs in a pod** (`infra.InCluster()`), so +neither is a way for a composition to move a deployed host's trust: + +- **`SOLUTION_HOST_LOCAL_CLUSTER`** — an absolute directory holding `server` + (the https origin of the local stand-in for the API server), `ca.crt`, + `token` and `namespace`. TokenReview and pod reads go there. It does not make + `InCluster()` true: the process is still not in a pod, and everything keyed + on that stays local. +- **`SOLUTION_HOST_LOCAL_TRUST_ANCHOR`** — an absolute directory holding + `verification_policy.json`. A relocated anchor must be a **local** profile — + one bound to a cluster CA by its `local` block — so it can only ever admit a + developer's key for the one local cluster whose CA it names. + +Neither is a workspace setting and neither is in the table above: both are +read from the process environment by the code that would otherwise read the +kubelet's paths, never through a workspace configuration group. + +In a pod both constants stand exactly as described above. + **There is no mount setting.** It was `SOLUTION_HOST_BINDINGS_DIR`, a projected ConfigMap volume, and both it and the directory reader are deleted. A mount is the CURRENT desired set and nothing more, so a document that arrived, was @@ -454,7 +496,27 @@ pass and every activation reads the CURRENT revision and hands Core's `ValidateAgainst` its projection for the document being judged, so containment is Core's exact rule and a narrowing takes effect at the next check with no restart. It is never read out of a delivered document — "an envelope a document -carried would be a document declaring its own ceiling". Absence and damage are +carried would be a document declaring its own ceiling". + +**Only a pure grant skips the policy-log witness.** A revision is a pure grant +when, against the revision in force, every approved build is still approved, +every principal is still there with a section that is the same JSON value, +every other top-level key is the same value, and no installation loses its +organization. What may differ is the revision, approved builds that were +added, and whole principals that were added. Everything else is written under +`WithPolicyLoggedNarrowing`, so a host restored from an older database cannot +silently give it back. + +A principal's section is compared for equality and never for "at least as +much", because in this schema more JSON does not mean more authority: a +resource list that appears restricts an audience that was unrestricted, a +required kind that appears refuses artifacts that passed, a tenant that is set +removes every other organization, and a unit that gains a field is a different +unit. So a changed section needs the witness, whatever changed. The +classification reads installations on the control plane, and an apply that was +classified a pure grant and then finds an installation to revoke stops and is +taken again under the witness. A host with no witness configured therefore +accepts pure grants and refuses everything else. Absence and damage are different answers: no revision written is a complete deployment that grants no module anything, while a revision that cannot be read leaves what the last pass installed serving, because reading a damaged ceiling as "no ceiling" withdraws @@ -556,6 +618,46 @@ storage column holds the envelope; it does not change the carrier's wire shape. ## 7b. The host's own control paths, and which part a consumer may hardcode +The current solution mint is `POST /modules/_work-context` on the gateway's +`workload` endpoint (`api: http`, transported over TLS 1.3). Codefly resolves its +address and port. A solution presents its X.509-SVID and projected ServiceAccount +token as `Authorization: Bearer`; it needs no shared internal token on this +listener. The gateway verifies the client chain and single SPIFFE URI SAN against +`workload-identity/TRUST_BUNDLE_FILE` and `ALLOWED_CALLERS_FILE`. Accounts' existing +TokenReview/BindExecution path requires that URI to equal both the reviewed +namespace/ServiceAccount identity and the SPIFFE ID in applied presence. A +mismatch returns 403, logs the mint refusal, and issues no credential. + +A principal the registry does not hold is refused, with one exception: while +the delivery inbox holds a document the registry was derived without, the +answer is `503` ("not established yet") and a pass is requested. A workload +that starts beside its own delivery asks in exactly that moment. Once the pass +has read the inbox, a principal it still does not hold is refused as before. + +A workload that is admitted and **installed nowhere** is answered `412` +(`FailedPrecondition` from accounts, on the module's own mint only — a +capability's use and a headless operation still hear the refusal), not `403`: nothing is wrong with the +workload, there is simply no installation to seal a credential to. It serves +its install endpoint and asks again. The origin and manifest it reported with +that mint are recorded all the same, because the install step is delivered +there. + +The `workload-identity` group supplies `CERT_FILE`, `KEY_FILE`, +`TRUST_BUNDLE_FILE` and `ALLOWED_CALLERS_FILE` as readable paths; +missing files fail startup with the key named. The SDK certificate reloader +keeps the last good pair during rotation. Trust is reread for each handshake; +caller admission rereads the newline-separated, `#`-commented allowlist for each +handshake and request, so revocation also affects existing connections. + +Solution proxy traffic uses TLS 1.3, presents the gateway SVID, verifies the +server chain, and requires exactly the SPIFFE URI declared by the applied +presence. The registry read joins matching registration and applied-generation +records; a missing, inconsistent or ambiguous workload identity is not proxied. +Dial-time address revalidation still rejects off-mesh and metadata destinations. +The plain `rest` endpoint retains its existing routes, including module-authority +mints with the internal token and projected bearer. Those module mints and module +upstreams retain their existing in-cluster transport contract. + Three repositories independently invented the string `/platform/_credential` from a design note, and nothing failed at build time. This section exists so the next one reads a contract instead of guessing, and so the hardcoding question has @@ -759,6 +861,13 @@ brokering pattern is the same one `GET /solutions/_entitlements` already uses. ## 8. Local runs: one attestation path, not a local mode +Local solution traffic follows the same mutual-TLS contract as deployed traffic: +the platform projects rotating SVIDs, trust and peer files, and the gateway uses +its resolved `workload` endpoint for solution mints. Its solution proxy presents +the gateway SVID and checks applied presence for the destination's SPIFFE ID. +Plain-HTTP solution listeners and mints are not a supported local mode. Module +SDK mints continue to use the plain in-cluster endpoint described in §7b. + Binding a credential to an approved execution means reading what the workload actually runs: `TokenReview` on its projected service-account token, then the pod by the name that review returns, then the application container's `imageID`. That @@ -949,6 +1058,33 @@ So presence v2 **can** describe a local run honestly. What it cannot describe honestly is a run of bare *processes* — which is an argument for running pods locally, not an argument for a second trust model. +### A run of processes: the CLI answers as the cluster + +The argument above ends at "run pods locally". A run of processes is still the +first loop a developer has, so `codefly run` supplies what the host reads from +Kubernetes instead of the host learning a second path. The host's code is the +same in both: TokenReview on the projected token, the pod by the name the review +returns, the build that pod runs, pinned-key verification of every delivered +carrier, the envelope as an independent ceiling. + +What the CLI stands in for, per run: + +- **the API server** — TokenReview and pod reads over TLS under a CA created for + that run, with projected tokens issued for the same audiences. The host + reaches it through `SOLUTION_HOST_LOCAL_CLUSTER` (§7, Configuration). +- **workload identity** — X.509-SVIDs under that same CA, named as a cell names + them from the namespace and service account. +- **the trust anchor** — a local profile whose `local` block pins that run's CA, + read through `SOLUTION_HOST_LOCAL_TRUST_ANCHOR`. + +What is weaker than a cluster, stated rather than implied: the build a pod is +said to run is the SHA-256 of the executable the CLI started, and the CLI is +both the party that says so and the party that signs the presence naming it. No +kubelet is a second witness. That is the authority a developer already has over +their own machine, and it is bounded to it — the profile admits a person's key +only for the one CA it names, and both relocations are refused in a pod, so no +deployed host can be pointed at a stand-in. + ### Two rules that do not bend locally **Absence is never removal, under any policy.** A local run makes this tempting in diff --git a/module/configurations/local/workload-identity.env b/module/configurations/local/workload-identity.env new file mode 100644 index 000000000..a6a0523e2 --- /dev/null +++ b/module/configurations/local/workload-identity.env @@ -0,0 +1,12 @@ +# The render supplies this group, not a configuration file: in a cell it +# projects the gateway's own X.509-SVID, the trust bundle and the callers the +# gateway admits, and a native `codefly run` supplies the same from the run's +# own authority. This file is what a run that projects nothing resolves. +# +# Empty means "nothing is projected": the gateway then serves its plain +# endpoints and opens no workload endpoint, so solutions cannot mint or be +# proxied. Any value requires all four, and each is a path to a readable file. +CERT_FILE= +KEY_FILE= +TRUST_BUNDLE_FILE= +ALLOWED_CALLERS_FILE= diff --git a/module/contracts/accounts/saas.events.codefly.yaml b/module/contracts/accounts/saas.events.codefly.yaml index 13112f15b..86c849e79 100644 --- a/module/contracts/accounts/saas.events.codefly.yaml +++ b/module/contracts/accounts/saas.events.codefly.yaml @@ -415,6 +415,10 @@ publishes: schema: saas/events/v1/events.proto#EventEnvelope visibility: external retention: 30d + - type: saas.installation.retried + schema: saas/events/v1/events.proto#EventEnvelope + visibility: external + retention: 30d - type: saas.installation.revoked schema: saas/events/v1/events.proto#EventEnvelope visibility: external diff --git a/module/contracts/api/accounts/authority/contract.binpb b/module/contracts/api/accounts/authority/contract.binpb index 5e7e7ab90..e0acaf84a 100644 Binary files a/module/contracts/api/accounts/authority/contract.binpb and b/module/contracts/api/accounts/authority/contract.binpb differ diff --git a/module/contracts/api/accounts/authority/proto/saas/accounts/v1/installations.proto b/module/contracts/api/accounts/authority/proto/saas/accounts/v1/installations.proto index 17fe6a96d..7b8440c1b 100644 --- a/module/contracts/api/accounts/authority/proto/saas/accounts/v1/installations.proto +++ b/module/contracts/api/accounts/authority/proto/saas/accounts/v1/installations.proto @@ -29,6 +29,12 @@ enum InstallationStatus { INSTALLATION_STATUS_UNSPECIFIED = 0; INSTALLATION_STATUS_ACTIVE = 1; INSTALLATION_STATUS_REVOKED = 2; + // Installed, and waiting for the solution to acknowledge its standard install + // step. Nothing can use a pending installation. + INSTALLATION_STATUS_PENDING = 3; + // The solution refused its install step, or never answered within the + // attempt budget. Nothing can use it; an administrator may retry it. + INSTALLATION_STATUS_FAILED = 4; } // InstallationHealth is resolved live from current authority facts, never @@ -88,6 +94,9 @@ message Installation { // uninstalling: the installation-identity cutover, or a tenancy narrowing that // withdrew this organisation. Empty on an active row. string revoked_reason = 12 [(buf.validate.field).string.max_len = 512]; + // Why the install step failed: the solution's bounded refusal reason, or a + // fixed reason when it never answered. Empty unless status is FAILED. + string failure_reason = 13 [(buf.validate.field).string.max_len = 512]; } message InstallSolutionRequest { @@ -177,6 +186,11 @@ message NarrowInstallationRequest { string scope = 3 [(buf.validate.field).string = {min_len: 2 max_bytes: 1048576}]; } +message RetryInstallationRequest { + string org_id = 1 [(buf.validate.field).string.uuid = true]; + string installation_id = 2 [(buf.validate.field).string.uuid = true]; +} + message UninstallSolutionRequest { string org_id = 1 [(buf.validate.field).string.uuid = true]; string installation_id = 2 [(buf.validate.field).string.uuid = true]; @@ -333,6 +347,34 @@ service InstallationService { }; } + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + rpc RetryInstallation(RetryInstallationRequest) returns (Installation) { + option (saas.policy.v1.method_policy) = { + exposure: EXPOSURE_AUTHENTICATED + tenant: TENANT_REQUIREMENT_ORG_ADMIN + resource_bindings: { + request_field: "org_id" + target: RESOURCE_TARGET_ORGANIZATION + lookup: RESOURCE_LOOKUP_DIRECT_ID + } + mfa: MFA_REQUIREMENT_NONE + audit: { + events: "saas.installation.retried" + emission: AUDIT_EMISSION_SUCCESS + } + idempotency: IDEMPOTENCY_REQUIREMENT_FORBIDDEN + rate_limit: RATE_LIMIT_CLASS_STANDARD_WRITE + request_sensitivity: SENSITIVITY_CONFIDENTIAL + response_sensitivity: SENSITIVITY_CONFIDENTIAL + platform_role: PLATFORM_ROLE_REQUIREMENT_NONE + }; + option (google.api.http) = { + post: "/v1/installations/{installation_id}:retry" + body: "*" + }; + } + // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, diff --git a/module/contracts/api/accounts/authority/proto/saas/accounts/v1/module_registration.proto b/module/contracts/api/accounts/authority/proto/saas/accounts/v1/module_registration.proto index ceb5164cd..395aac1c3 100644 --- a/module/contracts/api/accounts/authority/proto/saas/accounts/v1/module_registration.proto +++ b/module/contracts/api/accounts/authority/proto/saas/accounts/v1/module_registration.proto @@ -60,6 +60,28 @@ message ModuleMintWorkContextRequest { (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, (buf.validate.field).string.uuid = true ]; + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + string upstream = 5 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string = { + max_len: 512 + uri: true + } + ]; + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + string manifest = 6 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string.max_len = 65536 + ]; } // ModuleMintWorkContextResponse carries the signed capability plus the identity diff --git a/module/contracts/api/accounts/connect/contract.binpb b/module/contracts/api/accounts/connect/contract.binpb index 5e7e7ab90..e0acaf84a 100644 Binary files a/module/contracts/api/accounts/connect/contract.binpb and b/module/contracts/api/accounts/connect/contract.binpb differ diff --git a/module/contracts/api/accounts/connect/proto/saas/accounts/v1/installations.proto b/module/contracts/api/accounts/connect/proto/saas/accounts/v1/installations.proto index 17fe6a96d..7b8440c1b 100644 --- a/module/contracts/api/accounts/connect/proto/saas/accounts/v1/installations.proto +++ b/module/contracts/api/accounts/connect/proto/saas/accounts/v1/installations.proto @@ -29,6 +29,12 @@ enum InstallationStatus { INSTALLATION_STATUS_UNSPECIFIED = 0; INSTALLATION_STATUS_ACTIVE = 1; INSTALLATION_STATUS_REVOKED = 2; + // Installed, and waiting for the solution to acknowledge its standard install + // step. Nothing can use a pending installation. + INSTALLATION_STATUS_PENDING = 3; + // The solution refused its install step, or never answered within the + // attempt budget. Nothing can use it; an administrator may retry it. + INSTALLATION_STATUS_FAILED = 4; } // InstallationHealth is resolved live from current authority facts, never @@ -88,6 +94,9 @@ message Installation { // uninstalling: the installation-identity cutover, or a tenancy narrowing that // withdrew this organisation. Empty on an active row. string revoked_reason = 12 [(buf.validate.field).string.max_len = 512]; + // Why the install step failed: the solution's bounded refusal reason, or a + // fixed reason when it never answered. Empty unless status is FAILED. + string failure_reason = 13 [(buf.validate.field).string.max_len = 512]; } message InstallSolutionRequest { @@ -177,6 +186,11 @@ message NarrowInstallationRequest { string scope = 3 [(buf.validate.field).string = {min_len: 2 max_bytes: 1048576}]; } +message RetryInstallationRequest { + string org_id = 1 [(buf.validate.field).string.uuid = true]; + string installation_id = 2 [(buf.validate.field).string.uuid = true]; +} + message UninstallSolutionRequest { string org_id = 1 [(buf.validate.field).string.uuid = true]; string installation_id = 2 [(buf.validate.field).string.uuid = true]; @@ -333,6 +347,34 @@ service InstallationService { }; } + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + rpc RetryInstallation(RetryInstallationRequest) returns (Installation) { + option (saas.policy.v1.method_policy) = { + exposure: EXPOSURE_AUTHENTICATED + tenant: TENANT_REQUIREMENT_ORG_ADMIN + resource_bindings: { + request_field: "org_id" + target: RESOURCE_TARGET_ORGANIZATION + lookup: RESOURCE_LOOKUP_DIRECT_ID + } + mfa: MFA_REQUIREMENT_NONE + audit: { + events: "saas.installation.retried" + emission: AUDIT_EMISSION_SUCCESS + } + idempotency: IDEMPOTENCY_REQUIREMENT_FORBIDDEN + rate_limit: RATE_LIMIT_CLASS_STANDARD_WRITE + request_sensitivity: SENSITIVITY_CONFIDENTIAL + response_sensitivity: SENSITIVITY_CONFIDENTIAL + platform_role: PLATFORM_ROLE_REQUIREMENT_NONE + }; + option (google.api.http) = { + post: "/v1/installations/{installation_id}:retry" + body: "*" + }; + } + // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, diff --git a/module/contracts/api/accounts/connect/proto/saas/accounts/v1/module_registration.proto b/module/contracts/api/accounts/connect/proto/saas/accounts/v1/module_registration.proto index ceb5164cd..395aac1c3 100644 --- a/module/contracts/api/accounts/connect/proto/saas/accounts/v1/module_registration.proto +++ b/module/contracts/api/accounts/connect/proto/saas/accounts/v1/module_registration.proto @@ -60,6 +60,28 @@ message ModuleMintWorkContextRequest { (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, (buf.validate.field).string.uuid = true ]; + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + string upstream = 5 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string = { + max_len: 512 + uri: true + } + ]; + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + string manifest = 6 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string.max_len = 65536 + ]; } // ModuleMintWorkContextResponse carries the signed capability plus the identity diff --git a/module/contracts/api/auth-gateway/grpc/contract.binpb b/module/contracts/api/auth-gateway/grpc/contract.binpb index a54defcde..ee70a4cfe 100644 Binary files a/module/contracts/api/auth-gateway/grpc/contract.binpb and b/module/contracts/api/auth-gateway/grpc/contract.binpb differ diff --git a/module/contracts/api/auth-gateway/grpc/proto/saas/accounts/v1/module_registration.proto b/module/contracts/api/auth-gateway/grpc/proto/saas/accounts/v1/module_registration.proto index ceb5164cd..395aac1c3 100644 --- a/module/contracts/api/auth-gateway/grpc/proto/saas/accounts/v1/module_registration.proto +++ b/module/contracts/api/auth-gateway/grpc/proto/saas/accounts/v1/module_registration.proto @@ -60,6 +60,28 @@ message ModuleMintWorkContextRequest { (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, (buf.validate.field).string.uuid = true ]; + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + string upstream = 5 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string = { + max_len: 512 + uri: true + } + ]; + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + string manifest = 6 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string.max_len = 65536 + ]; } // ModuleMintWorkContextResponse carries the signed capability plus the identity diff --git a/module/contracts/api/catalog.codefly.json b/module/contracts/api/catalog.codefly.json index c2afebc75..5772bd6e8 100644 --- a/module/contracts/api/catalog.codefly.json +++ b/module/contracts/api/catalog.codefly.json @@ -10,7 +10,7 @@ "kind": "protobuf", "package": "saas.accounts.v1", "path": "contracts/api/accounts/authority/contract.binpb", - "digest": "sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047", + "digest": "sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82", "services": [ { "name": "ModuleCapabilitiesService", @@ -64,7 +64,7 @@ "kind": "protobuf", "package": "saas.accounts.v1", "path": "contracts/api/accounts/connect/contract.binpb", - "digest": "sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047", + "digest": "sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82", "services": [ { "name": "APIKeyService", @@ -242,6 +242,7 @@ "/saas.accounts.v1.InstallationService/ListInstallations", "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", "/saas.accounts.v1.InstallationService/NarrowInstallation", + "/saas.accounts.v1.InstallationService/RetryInstallation", "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", "/saas.accounts.v1.InstallationService/UninstallSolution" @@ -557,7 +558,7 @@ "kind": "protobuf", "package": "saas.gateway.auth.v1", "path": "contracts/api/auth-gateway/grpc/contract.binpb", - "digest": "sha256:0b293e6c50e9d7ef4921b178ab8575002a4b8bf75689322671ac7ee2421f74d7", + "digest": "sha256:35b09d7e8c2024969abc646b8e18844dc9620c5da11a67da7fc1199dfeac2ca9", "services": [ { "name": "AuthGatewayService", diff --git a/module/deployment/generated/asyncapi.json b/module/deployment/generated/asyncapi.json index 4d283e53e..c822ea19e 100644 --- a/module/deployment/generated/asyncapi.json +++ b/module/deployment/generated/asyncapi.json @@ -1131,6 +1131,17 @@ "x-visibility": "external", "x-retention": "30d" }, + "saas.installation.retried": { + "address": "saas.installation.retried", + "title": "saas.installation.retried", + "messages": { + "envelope": { + "$ref": "#/components/messages/saas.installation.retried" + } + }, + "x-visibility": "external", + "x-retention": "30d" + }, "saas.installation.revoked": { "address": "saas.installation.revoked", "title": "saas.installation.revoked", @@ -2814,6 +2825,13 @@ }, "title": "saas publishes saas.installation.ownership_transferred" }, + "send:saas.installation.retried": { + "action": "send", + "channel": { + "$ref": "#/channels/saas.installation.retried" + }, + "title": "saas publishes saas.installation.retried" + }, "send:saas.installation.revoked": { "action": "send", "channel": { @@ -4140,6 +4158,13 @@ "$ref": "#/components/schemas/saas_events_v1_events.proto_EventEnvelope" } }, + "saas.installation.retried": { + "name": "saas.installation.retried", + "title": "saas.installation.retried", + "payload": { + "$ref": "#/components/schemas/saas_events_v1_events.proto_EventEnvelope" + } + }, "saas.installation.revoked": { "name": "saas.installation.revoked", "title": "saas.installation.revoked", diff --git a/module/deployment/generated/communication.md b/module/deployment/generated/communication.md index 465dc99da..5a3010bc8 100644 --- a/module/deployment/generated/communication.md +++ b/module/deployment/generated/communication.md @@ -824,6 +824,14 @@ Every domain event type, who publishes it, and who consumes it. The machine-read - **Retention:** 30d - **Consumers:** _none_ +## saas.installation.retried + +- **Publisher:** saas +- **Visibility:** external +- **Schema:** `saas/events/v1/events.proto#EventEnvelope` (major v1) +- **Retention:** 30d +- **Consumers:** _none_ + ## saas.installation.revoked - **Publisher:** saas diff --git a/module/deployment/generated/event-catalog.json b/module/deployment/generated/event-catalog.json index 7b179359f..5fd24fa97 100644 --- a/module/deployment/generated/event-catalog.json +++ b/module/deployment/generated/event-catalog.json @@ -10204,6 +10204,106 @@ } ] }, + { + "type": "saas.installation.retried", + "namespace": "saas", + "schema": "saas/events/v1/events.proto#EventEnvelope", + "major": 1, + "visibility": "external", + "retention": "30d", + "fields": [ + { + "name": "id", + "number": 1, + "type": "string" + }, + { + "name": "type", + "number": 2, + "type": "string" + }, + { + "name": "source", + "number": 3, + "type": "string" + }, + { + "name": "subject", + "number": 4, + "type": "string" + }, + { + "name": "time", + "number": 5, + "type": "google.protobuf.Timestamp" + }, + { + "name": "specversion", + "number": 6, + "type": "string" + }, + { + "name": "datacontenttype", + "number": 7, + "type": "string" + }, + { + "name": "dataschema", + "number": 8, + "type": "string" + }, + { + "name": "data", + "number": 9, + "type": "bytes" + }, + { + "name": "tenant_id", + "number": 20, + "type": "string" + }, + { + "name": "boundary_id", + "number": 21, + "type": "string" + }, + { + "name": "partition_key", + "number": 22, + "type": "string" + }, + { + "name": "correlation_id", + "number": 23, + "type": "string" + }, + { + "name": "causation_id", + "number": 24, + "type": "string" + }, + { + "name": "actor_principal_id", + "number": 25, + "type": "string" + }, + { + "name": "owner_principal_id", + "number": 26, + "type": "string" + }, + { + "name": "traceparent", + "number": 27, + "type": "string" + }, + { + "name": "schema_version", + "number": 28, + "type": "uint32" + } + ] + }, { "type": "saas.installation.revoked", "namespace": "saas", diff --git a/module/deployment/generated/service-topology.json b/module/deployment/generated/service-topology.json index f0d29f775..7e441d853 100644 --- a/module/deployment/generated/service-topology.json +++ b/module/deployment/generated/service-topology.json @@ -70,8 +70,12 @@ { "name": "rest", "api": "CODEFLY_API_REST", - "visibility": "ENDPOINT_VISIBILITY_MODULE", - "port": 8080 + "visibility": "ENDPOINT_VISIBILITY_MODULE" + }, + { + "name": "workload", + "api": "CODEFLY_API_HTTP", + "visibility": "ENDPOINT_VISIBILITY_MODULE" } ], "dependencies": [ @@ -210,6 +214,11 @@ "endpoint": "rest", "visibility": "ENDPOINT_VISIBILITY_MODULE" }, + { + "service": "auth-gateway", + "endpoint": "workload", + "visibility": "ENDPOINT_VISIBILITY_MODULE" + }, { "service": "frontend", "endpoint": "http", diff --git a/module/module.codefly.yaml b/module/module.codefly.yaml index cf4c64ec5..eb6579923 100644 --- a/module/module.codefly.yaml +++ b/module/module.codefly.yaml @@ -23,6 +23,9 @@ interface: - service: auth-gateway endpoint: rest visibility: internal + - service: auth-gateway + endpoint: workload + visibility: internal - service: frontend endpoint: http visibility: public diff --git a/module/module.package.codefly.yaml b/module/module.package.codefly.yaml index 74dcc0998..32922c568 100644 --- a/module/module.package.codefly.yaml +++ b/module/module.package.codefly.yaml @@ -18,12 +18,12 @@ services: kind: protobuf package: saas.accounts.v1 path: contracts/api/accounts/authority/contract.binpb - digest: sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047 + digest: sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82 - endpoint: connect kind: protobuf package: saas.accounts.v1 path: contracts/api/accounts/connect/contract.binpb - digest: sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047 + digest: sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82 - name: auth-gateway endpoints: [grpc, rest] api-contracts: @@ -31,7 +31,7 @@ services: kind: protobuf package: saas.gateway.auth.v1 path: contracts/api/auth-gateway/grpc/contract.binpb - digest: sha256:0b293e6c50e9d7ef4921b178ab8575002a4b8bf75689322671ac7ee2421f74d7 + digest: sha256:35b09d7e8c2024969abc646b8e18844dc9620c5da11a67da7fc1199dfeac2ca9 - name: cache endpoints: [read, write] - name: frontend diff --git a/module/services/accounts/AGENTS.md b/module/services/accounts/AGENTS.md index 3cc90ba2d..699a5acc1 100644 --- a/module/services/accounts/AGENTS.md +++ b/module/services/accounts/AGENTS.md @@ -572,7 +572,15 @@ could not run is `ErrExecutionUnbound`, not one. API server has just authenticated. A longer one, or one whose lifetime cannot be read, is refused. -**Replay, stated precisely.** A projected ServiceAccount token is a BEARER +**Workload listener binding.** The gateway's TLS listener forwards its verified +SPIFFE peer on the authenticated internal mint RPC. `BindExecution` compares it +to the namespace and ServiceAccount returned by TokenReview and the full SPIFFE +ID retained from applied presence in `ModuleWorkload`. A mismatch follows the +existing refused-identity log path and is returned as 403 on the workload +endpoint, before signing or recording any credential. Plain module SDK mints +continue on their existing endpoint without this additional TLS attestation. + +**Replay on the plain module endpoint, stated precisely.** A projected ServiceAccount token is a BEARER credential. TokenReview proves the cluster issued it to a named pod and that the pod still exists; it does not prove the pod is the party presenting it. So: diff --git a/module/services/accounts/AUTHZ_MATRIX.md b/module/services/accounts/AUTHZ_MATRIX.md index ae2494cdc..b4b61945a 100644 --- a/module/services/accounts/AUTHZ_MATRIX.md +++ b/module/services/accounts/AUTHZ_MATRIX.md @@ -2,7 +2,7 @@ > Generated from protobuf service descriptors and `saas.policy.v1.method_policy`; only the prose description is joined from `pkg/business/introspection.go`. Do not edit by hand. Run `go generate ./pkg/business` from `module/services/accounts/code`. -Inventory: **266 RPCs** across **36 services**. Missing, invalid, or unclassified procedures are denied by both Connect and gRPC interceptors. +Inventory: **267 RPCs** across **36 services**. Missing, invalid, or unclassified procedures are denied by both Connect and gRPC interceptors. The compact tier is retained for compatibility. Guards, resource bindings, audit events, limiter class, and data sensitivity are descriptor-authoritative. Domain handlers may enforce stronger state-dependent rules but may not weaken this floor. @@ -96,6 +96,7 @@ The compact tier is retained for compatibility. Guards, resource bindings, audit | `/saas.accounts.v1.InstallationService/ListInstallations` | unary | `—` | `internal` | exposure=INTERNAL; tenant=NONE | — | — | — | FORBIDDEN / INTERNAL | CONFIDENTIAL → CONFIDENTIAL | List one organization's installations with their live health. | | `/saas.accounts.v1.InstallationService/ListOrganizationInstallations` | unary | `GET /v1/installations` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | — | FORBIDDEN / STANDARD_READ | CONFIDENTIAL → CONFIDENTIAL | List the caller's organization's active installations with health, installed target, authority root and the teams it is exposed to. | | `/saas.accounts.v1.InstallationService/NarrowInstallation` | unary | `POST /v1/installations/{installation_id}:narrow` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.installation.narrowed | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Narrow an installation's scope, bump its revision and cancel the organization's affected work. | +| `/saas.accounts.v1.InstallationService/RetryInstallation` | unary | `POST /v1/installations/{installation_id}:retry` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.installation.retried | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Run a failed installation's install step again; the installation is pending until the solution acknowledges. | | `/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam` | unary | `POST /v1/installations/{installation_id}/teams/{team_id}:revoke` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN; impersonation=FORBIDDEN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.scope.revoked | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Remove a team's grant at an installation's authority root, under the policy log. | | `/saas.accounts.v1.InstallationService/TransferInstallationOwnership` | unary | `POST /v1/installations/{installation_id}:transferOwnership` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.installation.ownership_transferred | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Reassign an installation's accountable owner of record. | | `/saas.accounts.v1.InstallationService/UninstallSolution` | unary | `POST /v1/installations/{installation_id}:uninstall` | `org_admin` | exposure=AUTHENTICATED; tenant=ORG_ADMIN | — | org_id → ORGANIZATION/DIRECT_ID | SUCCESS: saas.installation.revoked | FORBIDDEN / STANDARD_WRITE | CONFIDENTIAL → CONFIDENTIAL | Uninstall a solution and reverse its composition. | @@ -280,7 +281,7 @@ The compact tier is retained for compatibility. Guards, resource bindings, audit - `auth`: 43 - `internal`: 52 - `mfa`: 5 -- `org_admin`: 63 +- `org_admin`: 64 - `org_member`: 56 - `platform_admin`: 29 - `public`: 18 diff --git a/module/services/accounts/code/pkg/adapters/connect_handlers_installations.go b/module/services/accounts/code/pkg/adapters/connect_handlers_installations.go index 829a4e801..3a1fc4637 100644 --- a/module/services/accounts/code/pkg/adapters/connect_handlers_installations.go +++ b/module/services/accounts/code/pkg/adapters/connect_handlers_installations.go @@ -16,6 +16,10 @@ type installationConnectHandler struct { inner *InstallationServer } +func (h *installationConnectHandler) RetryInstallation(ctx context.Context, req *connect.Request[gen.RetryInstallationRequest]) (*connect.Response[gen.Installation], error) { + return unary(ctx, req, h.inner.RetryInstallation) +} + func (h *installationConnectHandler) InstallSolution(ctx context.Context, req *connect.Request[gen.InstallSolutionRequest]) (*connect.Response[gen.Installation], error) { return unary(ctx, req, h.inner.InstallSolution) } diff --git a/module/services/accounts/code/pkg/adapters/current_installation_test.go b/module/services/accounts/code/pkg/adapters/current_installation_test.go index d803585ef..50560e970 100644 --- a/module/services/accounts/code/pkg/adapters/current_installation_test.go +++ b/module/services/accounts/code/pkg/adapters/current_installation_test.go @@ -132,6 +132,9 @@ func TestCurrentInstallationServedModuleAndParent(t *testing.T) { {"wrong tenant", func(i *gen.Installation) { i.OrgId = uuid.NewString() }}, {"wrong installation", func(i *gen.Installation) { i.Id = uuid.NewString() }}, {"revoked", func(i *gen.Installation) { i.RevokedAt = timestamppb.Now() }}, + // HOST-INST-002, HOST-INST-004: unacknowledged consent is absent. + {"pending", func(i *gen.Installation) { i.Status = gen.InstallationStatus_INSTALLATION_STATUS_PENDING }}, + {"failed", func(i *gen.Installation) { i.Status = gen.InstallationStatus_INSTALLATION_STATUS_FAILED }}, {"inactive", func(i *gen.Installation) { i.Status = gen.InstallationStatus_INSTALLATION_STATUS_UNSPECIFIED }}, {"no consented target", func(i *gen.Installation) { i.TargetId = "" }}, } { diff --git a/module/services/accounts/code/pkg/adapters/installation_jobs_http.go b/module/services/accounts/code/pkg/adapters/installation_jobs_http.go new file mode 100644 index 000000000..7be9a70db --- /dev/null +++ b/module/services/accounts/code/pkg/adapters/installation_jobs_http.go @@ -0,0 +1,153 @@ +package adapters + +import ( + "context" + "log" + "net/http" + "slices" + "strings" + + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/jobs" +) + +// This private REST endpoint is intentionally absent from the gateway's public +// route catalog and from the module authority listener. It serves one queue. +const InstallationStepPath = "/internal/installation-steps/" + +type installationStepService interface { + ClaimInstallationStep(context.Context, string) (*jobsv1.JobEnvelope, error) + WithInstallationStep(context.Context, *jobsv1.JobLeaseReference, func(context.Context, *business.InstallationStepAuthority, []*gen.WorkContextScope) error) error + ValidateInstallationStepLease(context.Context, *jobsv1.JobLeaseReference) error + FinishInstallationStep(context.Context, *jobsv1.JobLeaseReference, uint32, int, string) error +} + +type installationWireScope struct { + ResourceKind string `json:"resourceKind"` + Actions []string `json:"actions"` + ResourceIDs []string `json:"resourceIds"` +} + +func NewInstallationStepHandler(svc *business.Service) http.Handler { + return installationStepHandler(svc, func(ctx context.Context, a *business.InstallationStepAuthority, scopes []*gen.WorkContextScope, lease *jobsv1.JobLeaseReference) (string, error) { + s := WorkContextSingleton() + if s == nil || !s.configured() || s.authority == nil { + return "", business.ErrModuleInstallationInactive + } + audience := "solution:" + a.BindingID + // The leased read has already established this exact live binding. + if a.Facts == nil || a.Facts.Actor == nil || (len(a.Facts.Actor.AllowedAudiences) > 0 && !slices.Contains(a.Facts.Actor.AllowedAudiences, audience)) { + return "", business.ErrModuleInstallationInactive + } + _, authority, err := workContextScopes(scopes, noContentReads) + if err != nil { + return "", err + } + // Sealed to this installation and actored by its agent, with the step's + // own scope beside the installation's (signInstallationStep). The session + // is the issuer's fresh one — the lease token is the job's, not a session + // a caller may name (see mintRequest.Session). + issued, err := s.signInstallationStep(ctx, &gen.StartInstallationTaskRequest{ + OrgId: a.OrganizationID, InstallationId: a.InstallationID, TaskId: a.InstallationID, + Audience: audience, TtlSeconds: int32(business.InstallationStepLease.Seconds()), + }, a.Facts, authority) + if err != nil { + return "", err + } + return issued.GetToken(), nil + }) +} + +func installationStepHandler(svc installationStepService, mint func(context.Context, *business.InstallationStepAuthority, []*gen.WorkContextScope, *jobsv1.JobLeaseReference) (string, error)) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + installerError(w, 405, "POST required") + return + } + if values := r.Header.Values("X-Codefly-Internal-Token"); len(values) != 1 || !validInternalToken(values[0]) { + installerError(w, 401, "internal credential required") + return + } + switch strings.TrimPrefix(r.URL.Path, InstallationStepPath) { + case "claim": + var req struct { + Worker string `json:"worker"` + } + if !installerDecode(w, r, &req) { + return + } + if req.Worker == "" || len(req.Worker) > 128 { + installerError(w, 400, "worker required") + return + } + job, err := svc.ClaimInstallationStep(r.Context(), req.Worker) + if err != nil { + // Named in this host's log: the consumer retries and dead-letters + // on a bare 503, and the operator reading the installation + // needs the refusal, never only its count. + log.Printf("installation step: claim refused: %v", err) + installerError(w, 503, "installation queue unavailable") + return + } + if job == nil { + w.WriteHeader(http.StatusNoContent) + return + } + lease := &jobsv1.JobLeaseReference{JobId: job.GetId(), WorkerId: req.Worker, LeaseToken: job.GetLease().GetToken()} + var result map[string]any + err = svc.WithInstallationStep(r.Context(), lease, func(ctx context.Context, a *business.InstallationStepAuthority, scopes []*gen.WorkContextScope) error { + token, err := mint(ctx, a, scopes, lease) + if err != nil { + return err + } + wire := make([]installationWireScope, 0, len(scopes)) + for _, s := range scopes { + wire = append(wire, installationWireScope{s.GetResourceKind(), s.GetActions(), s.GetResourceIds()}) + } + result = map[string]any{"job_id": lease.JobId, "worker": lease.WorkerId, "lease_token": lease.LeaseToken, "attempt": job.GetAttemptCount(), + "route_alias": a.RouteAlias, "target_id": a.TargetID, "binding_id": a.BindingID, "work_context": token, + "installation": map[string]any{"installation_id": a.InstallationID, "organization_id": a.OrganizationID, "revision": a.Revision, "scope": wire}} + return nil + }) + if err != nil { + // An unavailable mint consumes this attempt just like an unavailable + // solution. If finalization itself fails, lease expiry recovers it. + log.Printf("installation step: job %s attempt %d: authority or mint refused: %v", lease.JobId, job.GetAttemptCount(), err) + _ = svc.FinishInstallationStep(r.Context(), lease, job.GetAttemptCount(), 503, "") + installerError(w, 503, "installation authority unavailable") + return + } + installerJSON(w, 200, result) + case "finish": + var req struct { + JobID string `json:"job_id"` + Worker string `json:"worker"` + Token string `json:"lease_token"` + Attempt uint32 `json:"attempt"` + Status int `json:"status"` + Reason string `json:"reason"` + } + if !installerDecode(w, r, &req) { + return + } + lease := &jobsv1.JobLeaseReference{JobId: req.JobID, WorkerId: req.Worker, LeaseToken: req.Token} + if err := jobs.ValidateCommand(&jobsv1.CompleteJobRequest{Lease: lease}); err != nil { + installerError(w, 400, "invalid lease") + return + } + if err := svc.ValidateInstallationStepLease(r.Context(), lease); err != nil { + installerError(w, 409, "installation lease expired") + return + } + if err := svc.FinishInstallationStep(r.Context(), lease, req.Attempt, req.Status, req.Reason); err != nil { + installerError(w, 503, "installation outcome unavailable") + return + } + w.WriteHeader(http.StatusNoContent) + default: + installerError(w, 404, "unknown installation operation") + } + }) +} diff --git a/module/services/accounts/code/pkg/adapters/installation_jobs_http_test.go b/module/services/accounts/code/pkg/adapters/installation_jobs_http_test.go new file mode 100644 index 000000000..54c31be1a --- /dev/null +++ b/module/services/accounts/code/pkg/adapters/installation_jobs_http_test.go @@ -0,0 +1,179 @@ +package adapters + +import ( + "context" + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + + basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" + "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" +) + +// installStepLease is the job lease the step's consumer holds. It is the +// job's, never a session: the capability's session is the issuer's own. +const installStepLease = "019fec91-2000-7000-8000-0000000000a6" + +type stepServiceFake struct { + claimed, finished int + leaseValid bool + mintErr error + status int +} + +func (f *stepServiceFake) ClaimInstallationStep(context.Context, string) (*jobsv1.JobEnvelope, error) { + f.claimed++ + return &jobsv1.JobEnvelope{Id: installTaskID, AttemptCount: 1, Lease: &jobsv1.JobLease{Token: installStepLease}}, nil +} +func (f *stepServiceFake) WithInstallationStep(ctx context.Context, lease *jobsv1.JobLeaseReference, mint func(context.Context, *business.InstallationStepAuthority, []*gen.WorkContextScope) error) error { + if f.mintErr != nil { + return f.mintErr + } + return mint(ctx, &business.InstallationStepAuthority{InstallationStep: business.InstallationStep{InstallationID: installID, OrganizationID: installOrgID, Revision: 7}, TargetID: "example-target", RouteAlias: "example", BindingID: "example-binding"}, nil) +} +func (f *stepServiceFake) ValidateInstallationStepLease(context.Context, *jobsv1.JobLeaseReference) error { + if !f.leaseValid { + return errors.New("wrong queue or expired lease") + } + return nil +} +func (f *stepServiceFake) FinishInstallationStep(_ context.Context, _ *jobsv1.JobLeaseReference, _ uint32, status int, _ string) error { + f.finished++ + f.status = status + return nil +} + +// HOST-INST-003: only the private consumer can claim. Wire input cannot choose +// an installation, scope or audience, and an invalid lease finalizes nothing. +func TestInstallationStepHTTPIsPrivateAndLeaseBound(t *testing.T) { + old := internalToken + SetInternalToken("example-internal") + t.Cleanup(func() { SetInternalToken(old) }) + f := &stepServiceFake{} + h := installationStepHandler(f, func(context.Context, *business.InstallationStepAuthority, []*gen.WorkContextScope, *jobsv1.JobLeaseReference) (string, error) { + return "scoped-token", nil + }) + call := func(path, body, token string) *httptest.ResponseRecorder { + r := httptest.NewRequest(http.MethodPost, InstallationStepPath+path, strings.NewReader(body)) + r.Header.Set("X-Codefly-Internal-Token", token) + w := httptest.NewRecorder() + h.ServeHTTP(w, r) + return w + } + require.Equal(t, 401, call("claim", `{"worker":"example"}`, "forged").Code) + require.Zero(t, f.claimed) + require.Equal(t, 400, call("claim", `{"worker":"example","installation_id":"foreign"}`, "example-internal").Code) + require.Zero(t, f.claimed) + w := call("claim", `{"worker":"example"}`, "example-internal") + require.Equal(t, 200, w.Code) + var out map[string]any + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &out)) + require.Equal(t, "scoped-token", out["work_context"]) + require.Equal(t, "no-store", w.Header().Get("Cache-Control")) + body := `{"job_id":"` + installTaskID + `","worker":"example","lease_token":"` + installStepLease + `","status":204}` + require.Equal(t, 409, call("finish", body, "example-internal").Code) + require.Zero(t, f.finished) + f.leaseValid = true + require.Equal(t, 204, call("finish", body, "example-internal").Code) + require.Equal(t, 204, f.status) + f.mintErr = errors.New("authority unavailable") + require.Equal(t, 503, call("claim", `{"worker":"example"}`, "example-internal").Code) + require.Equal(t, 503, f.status) +} + +// HOST-INST-003: the install step reuses the headless issuer. Its task names +// exactly one installation, its sole actor is that installation's agent, and +// its scope names only its own root and the step itself. No-scope setup is +// valid: it grants the step and nothing else. +func TestInstallationStepSignedAuthority(t *testing.T) { + s := newInstallationMintServer(t, healthyInstallationFacts(), nil) + for _, scopes := range [][]*gen.WorkContextScope{nil, {{ResourceKind: "doc", Actions: []string{"write"}, ResourceIds: []string{"own-root"}}}} { + _, core, err := workContextScopes(scopes, noContentReads) + require.NoError(t, err) + req := installTaskRequest() + req.TaskId = installID + req.TtlSeconds = 60 + issued, err := s.signInstallationStep(context.Background(), req, healthyInstallationFacts(), core) + require.NoError(t, err) + verified, err := s.verifyIssued(context.Background(), issued.GetToken()) + require.NoError(t, err) + claims := verified.Context() + require.Equal(t, installOrgID, claims.GetTenantId()) + require.Equal(t, installID, claims.GetTaskId()) + require.Len(t, claims.GetActorChain(), 1) + require.Equal(t, installAgentID, claims.GetActorChain()[0].GetPrincipalId()) + // Sealed to the one installation the step is for: the carrier the + // gateway sends beside it names this id and revision, and the + // solution's runtime refuses a step whose carrier and seal disagree. + require.Equal(t, req.GetInstallationId(), claims.GetSeal().GetInstallationId()) + // The agent is delegated the installation's scopes and the step's own, + // and nothing else — so a scope-less installation still has a step. + granted := claims.GetActorChain()[0].GetGrantedScopes() + require.Len(t, granted, len(scopes)+1) + var own, step int + for _, scope := range granted { + switch scope.GetResourceKind() { + case InstallationStepResourceKind: + step++ + require.Equal(t, []string{InstallationStepAction}, scope.GetActions()) + require.Equal(t, []string{req.GetInstallationId()}, scope.GetResourceIds()) + default: + own++ + require.Equal(t, []string{"own-root"}, scope.GetResourceIds()) + } + } + require.Equal(t, 1, step) + require.Equal(t, len(scopes), own) + } +} + +// An installation's own scope may not spell the step's kind: the step's scope is +// the host's to add, for this installation only, so a scope arriving with it +// would be a second, caller-shaped claim on the same word. +func TestInstallationStepRefusesAScopeNamingTheStepItself(t *testing.T) { + s := newInstallationMintServer(t, healthyInstallationFacts(), nil) + req := installTaskRequest() + req.TaskId = installID + _, err := s.signInstallationStep(context.Background(), req, healthyInstallationFacts(), []*basev0.WorkScopeV1{{ + ResourceKind: InstallationStepResourceKind, Actions: []string{InstallationStepAction}, ResourceIds: []string{"another-installation"}, + }}) + require.Equal(t, codes.InvalidArgument, status.Code(err)) +} + +// The step's kind is the issuer's to stamp on every path. An ordinary +// installation task that could name it would carry a second spelling of "run +// this installation's install step" — on another installation's id. +func TestNoInstallationMintAcceptsAScopeNamingTheInstallStep(t *testing.T) { + for _, kind := range []string{"installations", "Installations", "INSTALLATIONS"} { + for name, ceiling := range map[string][]string{"a ceiling naming the kind": {"doc", "installations"}, "no ceiling": nil} { + t.Run(kind+"/"+name, func(t *testing.T) { + facts := healthyInstallationFacts() + facts.Actor.AllowedScopes = ceiling + server := newInstallationMintServer(t, facts, nil) + t.Cleanup(func() { SetInternalToken("") }) + req := installTaskRequest() + req.AuthorityScopes = []*gen.WorkContextScope{{ + ResourceKind: kind, Actions: []string{InstallationStepAction}, ResourceIds: []string{"another-installation"}, + }} + _, err := server.StartInstallationTask(installInternalContext(), req) + require.Equal(t, codes.InvalidArgument, status.Code(err), "an ordinary installation task minted the install step's scope: %v", err) + + step := installTaskRequest() + step.TaskId = installID + _, err = server.signInstallationStep(context.Background(), step, facts, []*basev0.WorkScopeV1{{ + ResourceKind: kind, Actions: []string{InstallationStepAction}, ResourceIds: []string{"another-installation"}, + }}) + require.Equal(t, codes.InvalidArgument, status.Code(err)) + }) + } + } +} diff --git a/module/services/accounts/code/pkg/adapters/installation_rpcs.go b/module/services/accounts/code/pkg/adapters/installation_rpcs.go index b0d444a02..2cea090e7 100644 --- a/module/services/accounts/code/pkg/adapters/installation_rpcs.go +++ b/module/services/accounts/code/pkg/adapters/installation_rpcs.go @@ -20,6 +20,24 @@ type InstallationServer struct { gen.UnsafeInstallationServiceServer } +func (s *InstallationServer) RetryInstallation(ctx context.Context, req *gen.RetryInstallationRequest) (*gen.Installation, error) { + if err := Validate(req); err != nil { + return nil, err + } + actorID, err := requireAuth(ctx) + if err != nil { + return nil, err + } + if err := requireOrgAdmin(ctx, actorID, req.GetOrgId()); err != nil { + return nil, err + } + installation, err := service.RetryInstallation(ctx, actorID, req.GetOrgId(), req.GetInstallationId()) + if err != nil { + return nil, mapInstallationError(err) + } + return installation, nil +} + func (s *InstallationServer) InstallSolution(ctx context.Context, req *gen.InstallSolutionRequest) (*gen.Installation, error) { if err := Validate(req); err != nil { return nil, err diff --git a/module/services/accounts/code/pkg/adapters/installation_scope_rpcs_test.go b/module/services/accounts/code/pkg/adapters/installation_scope_rpcs_test.go index 68ecb50c0..50f6546b3 100644 --- a/module/services/accounts/code/pkg/adapters/installation_scope_rpcs_test.go +++ b/module/services/accounts/code/pkg/adapters/installation_scope_rpcs_test.go @@ -37,3 +37,22 @@ func TestInstallationNarrowingErrorMapping(t *testing.T) { require.Equal(t, codes.Unavailable, status.Code(mapInstallationError(business.ErrPolicyLogUnreachable))) require.Equal(t, codes.PermissionDenied, status.Code(mapInstallationError(status.Error(codes.PermissionDenied, "outside ceiling")))) } + +// HOST-INST-005: retry is an administration act, never an unauthenticated +// resubmission or a way to operate on another organization. +func TestRetryInstallationRequiresOrgAdmin(t *testing.T) { + enableEnforcement(t) + const method = "/saas.accounts.v1.InstallationService/RetryInstallation" + s := &InstallationServer{} + req := &gen.RetryInstallationRequest{OrgId: enforceOrgID, InstallationId: installID} + _, err := s.RetryInstallation(context.Background(), req) + require.Equal(t, codes.Unauthenticated, status.Code(err)) + installEnforceService(t, memberStore(gen.OrgRole_ORG_ROLE_MEMBER)) + _, err = s.RetryInstallation(enforceActorCtx(), req) + require.Equal(t, codes.PermissionDenied, status.Code(err)) + installEnforceService(t, memberStore(gen.OrgRole_ORG_ROLE_ADMIN)) + require.NoError(t, enforceCentralPolicy(enforceActorCtx(), method)) + req.OrgId = exposureOtherOrgID + _, err = s.RetryInstallation(enforceActorCtx(), req) + require.Equal(t, codes.PermissionDenied, status.Code(err)) +} diff --git a/module/services/accounts/code/pkg/adapters/module_capabilities_server.go b/module/services/accounts/code/pkg/adapters/module_capabilities_server.go index 4942a258e..7f8e9cbc1 100644 --- a/module/services/accounts/code/pkg/adapters/module_capabilities_server.go +++ b/module/services/accounts/code/pkg/adapters/module_capabilities_server.go @@ -71,6 +71,24 @@ func moduleIdentityError(err error, denied codes.Code, message string) error { return nil } +// moduleMintWorkloadPeer imports the attestation for the shared BindExecution +// check. An empty or ambiguous attestation is present and fails that check; +// absence preserves the plain module endpoint. +func moduleMintWorkloadPeer(ctx context.Context) context.Context { + // This RPC is on the internal tier. The gateway creates this attestation + // from its verified TLS connection, never from an HTTP request header. + if md, ok := metadata.FromIncomingContext(ctx); ok { + if peers := md.Get("x-codefly-workload-spiffe-id"); len(peers) != 0 { + peer := "" + if len(peers) == 1 { + peer = peers[0] + } + ctx = business.WithWorkloadPeer(ctx, peer) + } + } + return ctx +} + // MintModuleWorkContext issues that Work Context. It takes none itself — this // is where a module obtains its identity — so it authenticates with the pod's // projected ServiceAccount token: accounts reviews it, reads the pod it names @@ -87,6 +105,19 @@ func (s *ModuleCapabilitiesServer) MintModuleWorkContext(ctx context.Context, re if err := Validate(req); err != nil { return nil, err } + ctx = moduleMintWorkloadPeer(ctx) + // The dynamic half first, on identity alone: the proven workload reports + // where it serves, installed or not, so the host can reach it with the + // install step that makes the mint below possible. + if err := service.ModuleReportUpstream(ctx, req.GetPrefix(), req.GetServiceAccountToken(), req.GetUpstream(), req.GetManifest()); err != nil { + if errors.Is(err, business.ErrModuleUpstreamRefused) { + return nil, status.Error(codes.InvalidArgument, err.Error()) + } + if mapped := moduleIdentityError(err, codes.PermissionDenied, "module work context denied"); mapped != nil { + return nil, mapped + } + return nil, err + } authority, err := service.ModuleAuthorizeWorkContext(ctx, req.GetPrefix(), req.GetServiceAccountToken(), req.GetOrganizationId()) if err != nil { if mapped := moduleIdentityError(err, codes.PermissionDenied, "module work context denied"); mapped != nil { @@ -155,6 +186,7 @@ func (s *ModuleCapabilitiesServer) MintModuleOperationContext(ctx context.Contex if err := Validate(req); err != nil { return nil, err } + ctx = moduleMintWorkloadPeer(ctx) authority, err := service.ModuleAuthorizeOperationContext(ctx, req.GetPrefix(), req.GetServiceAccountToken(), req.GetBinding(), req.GetOrganizationId()) if err != nil { if mapped := moduleIdentityError(err, codes.Unauthenticated, "module operation context denied"); mapped != nil { diff --git a/module/services/accounts/code/pkg/adapters/module_installations_http.go b/module/services/accounts/code/pkg/adapters/module_installations_http.go index b87d37fb4..1fbc128d7 100644 --- a/module/services/accounts/code/pkg/adapters/module_installations_http.go +++ b/module/services/accounts/code/pkg/adapters/module_installations_http.go @@ -135,6 +135,10 @@ func (h *ModuleInstallationHTTPHandler) ServeHTTP(w http.ResponseWriter, r *http installerError(w, http.StatusForbidden, "installer delegation denied") return } + if status.Code(err) == codes.PermissionDenied || status.Code(err) == codes.FailedPrecondition { + installerError(w, http.StatusForbidden, "installation declaration is not available to this organization") + return + } var se *business.StoreError if errors.As(err, &se) && (se.StoreErrorType == business.ErrTypeConflict || se.StoreErrorType == business.ErrTypeNotFound) { installerError(w, http.StatusConflict, se.Error()) @@ -160,7 +164,13 @@ func (h *ModuleInstallationHTTPHandler) ServeHTTP(w http.ResponseWriter, r *http return } if mode == "verify" && result.State != "ready" { - installerError(w, http.StatusConflict, "installation is absent; apply the approved declaration") + reason := "installation is absent; apply the approved declaration" + if result.State == "pending" { + reason = "installation is pending acknowledgement" + } else if result.State == "failed" { + reason = "installation failed; an organization administrator must retry or uninstall it" + } + installerError(w, http.StatusConflict, reason) return } installerJSON(w, http.StatusOK, result) diff --git a/module/services/accounts/code/pkg/adapters/module_work_context_test.go b/module/services/accounts/code/pkg/adapters/module_work_context_test.go index 164d4f5a8..f73690668 100644 --- a/module/services/accounts/code/pkg/adapters/module_work_context_test.go +++ b/module/services/accounts/code/pkg/adapters/module_work_context_test.go @@ -5,6 +5,7 @@ import ( "crypto/ed25519" "crypto/rand" "errors" + "strings" "testing" "accounts/pkg/business" @@ -424,3 +425,111 @@ func TestModuleCallerReadsTheVerifiedPrincipalAndTenant(t *testing.T) { require.Equal(t, principalID, caller.PrincipalID) require.Equal(t, moduleWorkContextTenant, caller.BoundOrg) } + +// Exercise the real TokenReview/BindExecution path through the mint adapter. +// A refusal happens before signing or recording a credential. +func TestWorkloadPeerMintMatchesReviewedTokenAndDeliveredPresence(t *testing.T) { + const peer = "spiffe://acme.test/ns/acme-prod/sa/module" + declared := strings.Replace(documentsPrincipals, `"workload":{`, `"workload":{"spiffe_id":"`+peer+`",`, 1) + for _, test := range []struct { + name, peer, declaration string + allowed bool + }{ + {"matching", peer, declared, true}, + {"another service account", "spiffe://acme.test/ns/acme-prod/sa/other", declared, false}, + {"another namespace", "spiffe://acme.test/ns/other/sa/module", declared, false}, + {"another trust domain", "spiffe://other.test/ns/acme-prod/sa/module", declared, false}, + {"absent declared identity", peer, documentsPrincipals, false}, + {"empty attestation", "", declared, false}, + // TLS and presence agree, but the reviewed token names another SA. + {"presence differs from reviewed token", peer, strings.Replace(declared, `"service_account":"module"`, `"service_account":"other"`, 1), false}, + } { + t.Run(test.name, func(t *testing.T) { + store := installModuleWorkContextService(t, test.declaration) + installModuleWorkContextAuthority(t) + ctx := metadata.NewIncomingContext(context.Background(), metadata.Pairs("x-codefly-workload-spiffe-id", test.peer)) + response, err := ModuleCapabilitiesSingleton().MintModuleWorkContext(ctx, &gen.ModuleMintWorkContextRequest{Prefix: "documents", ServiceAccountToken: moduleToken(t, "documents")}) + if test.allowed { + require.NoError(t, err) + require.NotEmpty(t, response.GetToken()) + require.Equal(t, 1, store.recordAttempts()) + } else { + require.Equal(t, codes.PermissionDenied, status.Code(err)) + require.Nil(t, response) + require.Zero(t, store.recordAttempts()) + } + }) + } +} + +func TestWorkloadPeerEveryMintBindsReviewedServiceAccount(t *testing.T) { + const peer = "spiffe://acme.test/ns/acme-prod/sa/module" + for _, kind := range []string{"work context", "operation context", "source delegation"} { + t.Run(kind, func(t *testing.T) { + for _, test := range []struct { + name string + peers []string + declaredPeer string + allowed bool + }{ + {"matching", []string{peer}, peer, true}, + {"another service account", []string{"spiffe://acme.test/ns/acme-prod/sa/other"}, peer, false}, + // Presence and TLS agree, but the reviewed token is for module, not other. + {"TLS and presence differ from token", []string{"spiffe://acme.test/ns/acme-prod/sa/other"}, "spiffe://acme.test/ns/acme-prod/sa/other", false}, + {"another trust domain", []string{"spiffe://other.test/ns/acme-prod/sa/module"}, peer, false}, + {"empty attestation", []string{""}, peer, false}, + {"ambiguous attestation", []string{peer, peer}, peer, false}, + {"plain endpoint", nil, peer, true}, + } { + t.Run(test.name, func(t *testing.T) { + prefix := "documents" + var audit *validatingAudit + switch kind { + case "source delegation": + _, audit = installSourceDelegationService(t) + prefix = "docstore" + default: + _, audit = installModuleOperationContextService(t, headlessPrincipals) + installModuleWorkContextAuthority(t) + } + // Keep the reviewed namespace/ServiceAccount independent of the full + // SPIFFE ID delivered in presence, so agreement with both is required. + registry := moduleExecution.registry + grant := registry[business.ModulePrincipalID(prefix)] + grant.Workload.SPIFFEID = test.declaredPeer + registry[business.ModulePrincipalID(prefix)] = grant + service.SetModuleCapabilities(nil, nil, registry) + ctx := metadata.NewIncomingContext(context.Background(), metadata.MD{"x-codefly-workload-spiffe-id": test.peers}) + token := moduleToken(t, prefix) + var issued string + var err error + denied := codes.Unauthenticated + switch kind { + case "work context": + var response *gen.ModuleMintWorkContextResponse + response, err = ModuleCapabilitiesSingleton().MintModuleWorkContext(ctx, &gen.ModuleMintWorkContextRequest{Prefix: prefix, ServiceAccountToken: token}) + issued = response.GetToken() + denied = codes.PermissionDenied + case "operation context": + var response *gen.ModuleMintOperationContextResponse + response, err = ModuleCapabilitiesSingleton().MintModuleOperationContext(ctx, &gen.ModuleMintOperationContextRequest{Prefix: prefix, ServiceAccountToken: token, Binding: "model"}) + issued = response.GetToken() + case "source delegation": + var response *gen.ModuleMintSourceOperationContextResponse + response, err = ModuleCapabilitiesSingleton().MintSourceOperationContext(ctx, &gen.ModuleMintSourceOperationContextRequest{Prefix: prefix, ServiceAccountToken: token, Delegation: &gen.ModuleMintSourceOperationContextRequest_SourceId{SourceId: delegationSource}}) + issued = response.GetToken() + } + if test.allowed { + require.NoError(t, err) + require.NotEmpty(t, issued) + require.Len(t, audit.entries, 1) + } else { + require.Equal(t, denied, status.Code(err), "%v", err) + require.Empty(t, issued) + require.Empty(t, audit.entries, "a refused identity records no issuance") + } + }) + } + }) + } +} diff --git a/module/services/accounts/code/pkg/adapters/rest_registration_catalog_gen.go b/module/services/accounts/code/pkg/adapters/rest_registration_catalog_gen.go index 007aa2af6..de22597cd 100644 --- a/module/services/accounts/code/pkg/adapters/rest_registration_catalog_gen.go +++ b/module/services/accounts/code/pkg/adapters/rest_registration_catalog_gen.go @@ -268,6 +268,7 @@ var catalogRESTTemplateRoutes = []catalogRESTTemplateRoute{ {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+/teams$")}, {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+/teams/[^/]+:revoke$")}, {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+:narrow$")}, + {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+:retry$")}, {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+:transferOwnership$")}, {method: "POST", path: regexp.MustCompile("^/v1/installations/[^/]+:uninstall$")}, {method: "POST", path: regexp.MustCompile("^/v1/invitations/[^/]+:resend$")}, diff --git a/module/services/accounts/code/pkg/adapters/source_delegation.go b/module/services/accounts/code/pkg/adapters/source_delegation.go index ad0fac8dc..5b29e8c46 100644 --- a/module/services/accounts/code/pkg/adapters/source_delegation.go +++ b/module/services/accounts/code/pkg/adapters/source_delegation.go @@ -69,6 +69,7 @@ func (s *ModuleCapabilitiesServer) MintSourceOperationContext(ctx context.Contex if err := Validate(req); err != nil { return nil, err } + ctx = moduleMintWorkloadPeer(ctx) authority, err := service.AuthorizeSourceOperationContext(ctx, req.GetPrefix(), req.GetServiceAccountToken(), business.SourceDelegationRef{ DelegationID: req.GetDelegationId(), SourceID: req.GetSourceId(), diff --git a/module/services/accounts/code/pkg/adapters/work_context_rpcs.go b/module/services/accounts/code/pkg/adapters/work_context_rpcs.go index 20667e839..23c9c20af 100644 --- a/module/services/accounts/code/pkg/adapters/work_context_rpcs.go +++ b/module/services/accounts/code/pkg/adapters/work_context_rpcs.go @@ -6,6 +6,7 @@ import ( "errors" "fmt" "slices" + "sort" "strings" "time" @@ -806,15 +807,85 @@ func (s *WorkContextAuthorityServer) StartInstallationTask( if err != nil { return nil, err } + return s.signInstallationTask(ctx, req, facts, scopes) +} + +// Shared by the ordinary headless mint and the leased install step. Only the +// latter resolves pending consent; neither path changes ordinary use admission. +func (s *WorkContextAuthorityServer) signInstallationTask(ctx context.Context, req *gen.StartInstallationTaskRequest, facts *business.InstallationAuthorityFacts, scopes []*basev0.WorkScopeV1) (*gen.IssuedWorkContext, error) { // An installation context is always agent-actored; a nil actor means the store // broke its contract (a healthy resolve always names the agent). Fail closed // rather than mint an unattributed token. if facts.Actor == nil { return nil, status.Error(codes.Internal, "installation authority resolved without an agent actor") } + // The step's kind is the issuer's to stamp on every path, not only the + // one that stamps it: an ordinary task that could name it would carry a + // second spelling of "run this installation's install step". + if err := refuseInstallationStepKind(scopes); err != nil { + return nil, err + } + if err := enforceActorCeiling(ctx, facts.Actor, req.GetAudience(), scopes); err != nil { + return nil, err + } + return s.mintInstallationTask(ctx, req, facts, scopes) +} + +// InstallationStepResourceKind and InstallationStepAction name the one scope +// every install step is delegated: to run THIS installation's install step. The +// resource id is the installation's own id, so the scope reaches nothing else. +const ( + InstallationStepResourceKind = "installations" + InstallationStepAction = "install" +) + +// refuseInstallationStepKind refuses a caller-supplied scope that names the +// install step's kind, however it is cased or padded. +func refuseInstallationStepKind(scopes []*basev0.WorkScopeV1) error { + for _, scope := range scopes { + if strings.EqualFold(strings.TrimSpace(scope.GetResourceKind()), InstallationStepResourceKind) { + return status.Errorf(codes.InvalidArgument, "an installation's scope may not name %q: the install step's own scope is the host's to add", InstallationStepResourceKind) + } + } + return nil +} + +// signInstallationStep signs the capability an install step is delivered +// under: the installation's own scopes, held to the agent's ceiling exactly as +// signInstallationTask holds them, plus the step's own scope. +// +// The step's scope is what keeps two rules true at once. An installation whose +// role and grant intersect to nothing is valid — the solution then acts only as +// its viewers, and its agent is delegated nothing — and it still has an install +// step to acknowledge. Core refuses a delegation hop that grants no scope, +// rightly: an empty grant is one consumer's "nothing" and another's "no +// restriction". And an installation capability is never minted unattributed. +// So the hop always grants the one thing the step is: this installation's +// install. It is added after the ceiling, which governs the kinds an agent may +// be delegated over a module's resources and has no say over the step itself. +func (s *WorkContextAuthorityServer) signInstallationStep(ctx context.Context, req *gen.StartInstallationTaskRequest, facts *business.InstallationAuthorityFacts, scopes []*basev0.WorkScopeV1) (*gen.IssuedWorkContext, error) { + if facts.Actor == nil { + return nil, status.Error(codes.Internal, "installation authority resolved without an agent actor") + } + // Before the ceiling, so the refusal names what is wrong with the scope + // rather than what the agent may not hold. + if err := refuseInstallationStepKind(scopes); err != nil { + return nil, err + } if err := enforceActorCeiling(ctx, facts.Actor, req.GetAudience(), scopes); err != nil { return nil, err } + step := append(cloneWorkScopes(scopes), &basev0.WorkScopeV1{ + ResourceKind: InstallationStepResourceKind, + Actions: []string{InstallationStepAction}, + ResourceIds: []string{req.GetInstallationId()}, + }) + sort.SliceStable(step, func(i, j int) bool { return step[i].GetResourceKind() < step[j].GetResourceKind() }) + return s.mintInstallationTask(ctx, req, facts, step) +} + +// mintInstallationTask signs resolved, ceiling-checked installation authority. +func (s *WorkContextAuthorityServer) mintInstallationTask(ctx context.Context, req *gen.StartInstallationTaskRequest, facts *business.InstallationAuthorityFacts, scopes []*basev0.WorkScopeV1) (*gen.IssuedWorkContext, error) { // Sealed to the installation the authority is drawn from, owned by its owner // of record and actored by its agent. The session is a fresh one: this // caller holds a service credential, not a session, and a session id a diff --git a/module/services/accounts/code/pkg/business/audit_registry.go b/module/services/accounts/code/pkg/business/audit_registry.go index af9e64d89..a6ac266ea 100644 --- a/module/services/accounts/code/pkg/business/audit_registry.go +++ b/module/services/accounts/code/pkg/business/audit_registry.go @@ -383,6 +383,7 @@ const ( EventInstallationCreated EventType = "saas.installation.created" EventInstallationNarrowed EventType = "saas.installation.narrowed" + EventInstallationRetried EventType = "saas.installation.retried" EventInstallationRevoked EventType = "saas.installation.revoked" EventInstallationOwnershipTransferred EventType = "saas.installation.ownership_transferred" @@ -673,6 +674,7 @@ var auditEventCatalog = []AuditEventDefinition{ mutation(EventInstallationRevoked, CategoryAccess, "A solution was uninstalled, or the presence it was installed from was withdrawn: its agent principal and standing grant were revoked and its scope node soft-deleted.", uid("target_id"), str("binding_id"), str("generation"), str("revoked_reason")), mutation(EventInstallationNarrowed, CategoryAccess, "An installation scope was narrowed and its affected work canceled.", str("scope"), str("revision"), str("binding_id")), + mutation(EventInstallationRetried, CategoryAccess, "A failed installation's install step was requested again.", str("revision")), mutation(EventInstallationOwnershipTransferred, CategoryAccess, "An installation's owner of record was reassigned.", uid("owner_principal_id")), mutation(EventRecordShared, CategoryAccess, "A record was shared with a principal or team.", uid("role_id"), uid("subject_id")), diff --git a/module/services/accounts/code/pkg/business/installation_ack_fixture_test.go b/module/services/accounts/code/pkg/business/installation_ack_fixture_test.go new file mode 100644 index 000000000..af919119b --- /dev/null +++ b/module/services/accounts/code/pkg/business/installation_ack_fixture_test.go @@ -0,0 +1,50 @@ +//go:build !pure + +package business_test + +import ( + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/infra" + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/types/known/durationpb" +) + +// Existing authority fixtures require an acknowledged installation. Drive the +// actual durable queue and outcome projection, as a setup consumer would; no +// fixture writes active directly. These database suites run sequentially. +func acknowledgeInstallation(t *testing.T, i *gen.Installation) { + t.Helper() + if i.Status != gen.InstallationStatus_INSTALLATION_STATUS_PENDING { + return + } + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + for n := 0; n < 1000; n++ { + batch, err := worker.Claim(testCtx, &jobsv1.ClaimJobsRequest{Queue: business.InstallationStepQueue, WorkerId: "fixture-ack", Limit: 1, LeaseDuration: durationpb.New(time.Minute)}) + require.NoError(t, err) + require.Len(t, batch.Jobs, 1, "expected an enqueued installation step") + job := batch.Jobs[0] + require.NoError(t, worker.Complete(testCtx, &jobsv1.CompleteJobRequest{Lease: &jobsv1.JobLeaseReference{JobId: job.Id, WorkerId: job.Lease.Owner, LeaseToken: job.Lease.Token}})) + if job.Attributes["installation_step.id"] == i.Id { + require.NoError(t, testStore.WithOrgTx(testCtx, i.OrgId, func(ctx context.Context) error { + current, _, err := testStore.GetInstallation(ctx, i.OrgId, i.Id) + if err == nil { + i.Status = current.Status + i.FailureReason = current.FailureReason + } + return err + })) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, i.Status) + return + } + } + t.Fatal("installation step not found") +} diff --git a/module/services/accounts/code/pkg/business/installation_batch_lock_integration_test.go b/module/services/accounts/code/pkg/business/installation_batch_lock_integration_test.go index 249b8b029..dffde85b7 100644 --- a/module/services/accounts/code/pkg/business/installation_batch_lock_integration_test.go +++ b/module/services/accounts/code/pkg/business/installation_batch_lock_integration_test.go @@ -76,6 +76,7 @@ func TestDeleteOrganizationSerializesWithAnotherInstallationUninstall(t *testing TargetID: targetID, RootScopeLabel: "Example Solution", RoleID: roleID, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) ids = append(ids, installation.Id) } sort.Strings(ids) @@ -322,6 +323,7 @@ func TestRevokeInstallationsOfTargetLocksTheBatchBeforeUpdating(t *testing.T) { TargetID: targetID, RootScopeLabel: "Example Solution", RoleID: roleID, }) require.NoError(t, err) + acknowledgeInstallation(t, second) ids := []string{first, second.Id} sort.Strings(ids) ctx, cancel := context.WithTimeout(testCtx, 30*time.Second) diff --git a/module/services/accounts/code/pkg/business/installation_exposure.go b/module/services/accounts/code/pkg/business/installation_exposure.go index 44548e99d..09e353219 100644 --- a/module/services/accounts/code/pkg/business/installation_exposure.go +++ b/module/services/accounts/code/pkg/business/installation_exposure.go @@ -32,7 +32,7 @@ type InstallationExposure struct { InheritedTeams []*gen.CollectionReadGrant } -// OrganizationInstallation is one active installation as its organization's +// OrganizationInstallation is one nonrevoked installation as its organization's // administrator manages it. type OrganizationInstallation struct { Summary *gen.InstallationSummary @@ -47,7 +47,7 @@ type OrganizationInstallation struct { // is the one outcome an administrator granting "exposure" must never be handed. var ErrRoleDoesNotExposeSolution = errors.New("role does not permit using a solution") -// ListOrganizationInstallations lists orgID's ACTIVE installations, one page, with +// ListOrganizationInstallations lists orgID's installations, one page, with // live health, exposure and the catalogue row of each installed target. // // The installations and their exposure are read on ONE read-only REPEATABLE @@ -77,7 +77,7 @@ func (s *Service) ListOrganizationInstallations( if err := s.installationStore().WithOrgReadSnapshot(ctx, orgID, func(ctx context.Context) error { var e error summaries, e = s.installationStore().ListInstallations(ctx, orgID, - gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, pageToken, pageSize+1) + gen.InstallationStatus_INSTALLATION_STATUS_UNSPECIFIED, pageToken, pageSize+1) if e != nil { return e } @@ -121,18 +121,24 @@ func (s *Service) ListOrganizationInstallations( out := make([]*OrganizationInstallation, 0, len(summaries)) for _, summary := range summaries { installation := summary.GetInstallation() + if installation.Status == gen.InstallationStatus_INSTALLATION_STATUS_REVOKED { + continue + } row := &OrganizationInstallation{ Summary: summary, Exposure: exposure[installation.GetId()], Target: targets[installation.GetTargetId()], } - if row.Exposure == nil { + if row.Exposure == nil && installation.Status == gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE { // Both reads share one snapshot and an active installation always has // a root node, so a missing entry is a defect rather than a race. It // fails the listing instead of showing an active row with no root. return nil, "", w.Wrapf(fmt.Errorf("installation %s has no authority root in the same snapshot", installation.GetId()), "cannot list organization installations") } + if row.Exposure == nil { + row.Exposure = &InstallationExposure{} + } out = append(out, row) } return out, next, nil diff --git a/module/services/accounts/code/pkg/business/installation_exposure_integration_test.go b/module/services/accounts/code/pkg/business/installation_exposure_integration_test.go index 03b34cdc8..321bbabac 100644 --- a/module/services/accounts/code/pkg/business/installation_exposure_integration_test.go +++ b/module/services/accounts/code/pkg/business/installation_exposure_integration_test.go @@ -54,6 +54,7 @@ func installForExposure(t *testing.T, adminID, orgID string) (installationID, ta RoleID: agentRole, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) return installation.GetId(), targetID, alias } @@ -285,6 +286,7 @@ func TestGrantInstallationToTeamSerializesWithUninstall(t *testing.T) { RootScopeLabel: "Exposed", RoleID: agentRole, }) require.NoError(t, err) + acknowledgeInstallation(t, reinstalled) require.NotEqual(t, w.installation, reinstalled.GetId()) require.Empty(t, entitledInstallations(t, w.org, w.inTeam)) }) diff --git a/module/services/accounts/code/pkg/business/installation_fixture_test.go b/module/services/accounts/code/pkg/business/installation_fixture_test.go index eb2c9159d..5f85939be 100644 --- a/module/services/accounts/code/pkg/business/installation_fixture_test.go +++ b/module/services/accounts/code/pkg/business/installation_fixture_test.go @@ -52,17 +52,20 @@ func installCapabilityFixture(t *testing.T, svc *business.Service, org, owner st } role := business.NewIDString() scope := business.FullInstallationScope(grant) + var installed *gen.Installation require.NoError(t, testStore.WithOrgTx(testCtx, org, func(ctx context.Context) error { if err := testStore.CreateRole(ctx, &gen.Role{Id: role, OrgId: org, Name: "Example role " + role, Permissions: []*gen.Permission{{Resource: "solution", Action: "use"}}}); err != nil { return err } - _, err := testStore.InstallSolution(ctx, &business.InstallSolutionParams{ + var err error + installed, err = testStore.InstallSolution(ctx, &business.InstallSolutionParams{ OrgID: org, OwnerPrincipalID: owner, GrantedBy: owner, RoleID: role, AgentIdentifier: "acme.example/" + alias + ":1.0.0", TargetID: target, Kind: "solution", RegistryBindingID: grant.PresenceBinding, RegistryPrincipalID: principal, Scope: &scope, EnvelopeRevision: envelope, }) return err })) + acknowledgeInstallation(t, installed) live, err = testStore.LiveModuleAuthority(testCtx, principal, org, "") require.NoError(t, err) require.Equal(t, grant.PresenceBinding, live.RegistryBindingID) @@ -71,5 +74,5 @@ func installCapabilityFixture(t *testing.T, svc *business.Service, org, owner st require.Equal(t, envelope, *live.EnvelopeRevision) registry[principal] = grant } - svc.InstallDerivedModulePrincipals(registry, envelope) + svc.InstallDerivedModulePrincipals(registry, envelope, nil) } diff --git a/module/services/accounts/code/pkg/business/installation_jobs.go b/module/services/accounts/code/pkg/business/installation_jobs.go new file mode 100644 index 000000000..130dfa247 --- /dev/null +++ b/module/services/accounts/code/pkg/business/installation_jobs.go @@ -0,0 +1,255 @@ +package business + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "slices" + "sort" + "time" + "unicode/utf8" + + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/jobs" + + "google.golang.org/protobuf/types/known/durationpb" + "google.golang.org/protobuf/types/known/timestamppb" +) + +type InstallationRetryStore interface { + RetryInstallation(context.Context, string, string) (*gen.Installation, uint64, error) +} + +// RetryInstallation preserves the consent revision. State, outbox and audit +// commit together; a repeated request while pending is a conflict. +func (s *Service) RetryInstallation(ctx context.Context, actorID, orgID, id string) (*gen.Installation, error) { + store, ok := s.store.(InstallationRetryStore) + if !ok { + return nil, errors.New("installation retry store unavailable") + } + actorType := s.actorTypeForCreator(ctx, actorID) + var installation *gen.Installation + // The target lock must precede the installation lock, as on withdrawal. + // The control-plane transaction can read the target; every write names orgID. + err := s.store.WithControlPlane(ctx, func(ctx context.Context) error { + var err error + var revision uint64 + installation, revision, err = store.RetryInstallation(ctx, orgID, id) + if err != nil { + return err + } + return s.emitTx(ctx, actorID, actorType, EventInstallationRetried, "installation", id, orgID, map[string]any{"revision": fmt.Sprint(revision)}) + }) + return installation, err +} + +// InstallationStepAuthority is resolved under locks for the current leased run. +// StandingPermissions is the agent's role at its own root, never the owner's RBAC. +type InstallationStepAuthority struct { + InstallationStep + TargetID, RouteAlias, BindingID, PrincipalID, RootID string + Scope InstallationScope + Facts *InstallationAuthorityFacts + StandingPermissions []WorkContextPermission +} + +type InstallationStepStore interface { + WithInstallationStepAuthority(context.Context, *jobsv1.JobLeaseReference, uint64, func(context.Context, *InstallationStepAuthority) error) error + InstallationStepLeaseValid(context.Context, *jobsv1.JobLeaseReference) (bool, error) +} + +// WithInstallationStep checks delivered authority as well as consent. The +// callback signs while the target and installation locks still fence changes. +func (s *Service) WithInstallationStep(ctx context.Context, lease *jobsv1.JobLeaseReference, mint func(context.Context, *InstallationStepAuthority, []*gen.WorkContextScope) error) error { + store, ok := s.store.(InstallationStepStore) + if !ok { + return errors.New("installation step store unavailable") + } + snapshot := s.moduleRegistrySnapshot() + if err := s.fenceModuleRegistrySnapshot(ctx, snapshot); err != nil { + return err + } + return store.WithInstallationStepAuthority(ctx, lease, snapshot.envelopeRevision, func(ctx context.Context, a *InstallationStepAuthority) error { + grant, ok := snapshot.registry[a.PrincipalID] + if !ok || a.BindingID != grant.PresenceBinding || !moduleAvailableToOrganization(grant, a.OrganizationID) { + return ErrModuleInstallationInactive + } + a.Scope = FullInstallationScope(intersectInstallationScope(grant, a.Scope)) + if err := mint(ctx, a, installationStepScopes(a)); err != nil { + return err + } + if s.moduleRegistrySnapshot() != snapshot { + return ErrModuleRegistryStale + } + return nil + }) +} + +// A setup token is always confined to this installation's root. Only concrete +// actions in delivered operation/read bindings, consent, the agent ceiling and +// its standing role survive. Empty authority is valid for a no-op install. +func installationStepScopes(a *InstallationStepAuthority) []*gen.WorkContextScope { + byKind := map[string][]string{} + add := func(scope ModuleOperationScope) { + if len(scope.ResourceIDs) > 0 && !slices.Contains(scope.ResourceIDs, a.RootID) { + return + } + if len(a.Facts.Actor.AllowedScopes) > 0 && !slices.Contains(a.Facts.Actor.AllowedScopes, scope.ResourceKind) { + return + } + for _, action := range scope.Actions { + if action == "*" || scope.ResourceKind == "*" { + continue + } + for _, p := range a.StandingPermissions { + if (p.ResourceKind == "*" || p.ResourceKind == scope.ResourceKind) && (p.Action == "*" || p.Action == action) { + byKind[scope.ResourceKind] = append(byKind[scope.ResourceKind], action) + break + } + } + } + } + for _, binding := range a.Scope.OperationAudiences { + for _, scope := range binding.InvokeScopes { + add(scope) + } + } + for _, binding := range a.Scope.ReadAudiences { + for _, scope := range readOperationScopes(binding.Scopes) { + add(scope) + } + } + var result []*gen.WorkContextScope + for kind, actions := range byKind { + slices.Sort(actions) + result = append(result, &gen.WorkContextScope{ResourceKind: kind, Actions: slices.Compact(actions), ResourceIds: []string{a.RootID}}) + } + sort.Slice(result, func(i, j int) bool { return result[i].ResourceKind < result[j].ResourceKind }) + return result +} + +func (s *Service) ValidateInstallationStepLease(ctx context.Context, lease *jobsv1.JobLeaseReference) error { + store, ok := s.store.(InstallationStepStore) + if !ok { + return errors.New("installation step store unavailable") + } + valid, err := store.InstallationStepLeaseValid(ctx, lease) + if err != nil { + return err + } + if !valid { + return errors.New("installation step lease is no longer current") + } + return nil +} + +const ( + InstallationStepQueue = "installation-step" + InstallationStepMaxAttempts = 5 + InstallationStepLease = time.Minute + InstallationStepTimeout = 20 * time.Second + InstallationFailureLimit = 512 + InstallationStepFailure = "The installation step could not be completed after repeated attempts." +) + +// InstallationStep identifies consent, never a caller-selected upstream. Scope +// is resolved afresh for every attempt. Tokens are never stored in job payloads. +type InstallationStep struct { + InstallationID string `json:"installation_id"` + OrganizationID string `json:"organization_id"` + Revision uint64 `json:"revision"` +} + +// NewInstallationStepJob gives each operator retry its own durable run while +// keeping the solution's idempotency identity (installation, revision) stable. +// Ordering serializes revisions and retries of one installation. +func NewInstallationStepJob(step InstallationStep, run string) (*jobsv1.EnqueueJobRequest, error) { + payload, err := json.Marshal(step) + if err != nil { + return nil, err + } + req := &jobsv1.EnqueueJobRequest{Job: &jobsv1.NewJob{ + Direction: jobsv1.JobDirection_JOB_DIRECTION_OUTBOX, + Scope: &jobsv1.JobScope{Value: &jobsv1.JobScope_OrganizationId{OrganizationId: step.OrganizationID}}, + Queue: InstallationStepQueue, Topic: "installation.step", Source: "saas.installation", + IdempotencyKey: fmt.Sprintf("%s:%d:%s", step.InstallationID, step.Revision, run), + Ordering: &jobsv1.JobOrderingKey{Namespace: "installation", Components: []string{step.InstallationID}}, + SchemaVersion: 1, Payload: payload, ContentType: "application/json", + MaxAttempts: InstallationStepMaxAttempts, + Attributes: map[string]string{ + "installation_step.id": step.InstallationID, + "installation_step.revision": fmt.Sprint(step.Revision), + "codefly.installation_id": step.InstallationID, + "codefly.organization_id": step.OrganizationID, + "codefly.installation_revision": fmt.Sprint(step.Revision), + }, + }} + if err := jobs.ValidateCommand(req); err != nil { + return nil, err + } + return req, nil +} + +func InstallationStepRetryDelay(attempt uint32) time.Duration { + if attempt == 0 { + attempt = 1 + } + if attempt > InstallationStepMaxAttempts { + attempt = InstallationStepMaxAttempts + } + return 5 * time.Second * time.Duration(1<<(attempt-1)) +} + +// BoundedInstallationReason accepts only the solution's explicit refusal field. +// Invalid/empty diagnostics become a fixed message; arbitrary transport errors +// never enter installation records or job history. +func BoundedInstallationReason(reason string) string { + if !utf8.ValidString(reason) || reason == "" { + return "The solution refused the installation step." + } + for len(reason) > InstallationFailureLimit { + _, size := utf8.DecodeLastRuneInString(reason) + reason = reason[:len(reason)-size] + } + return reason +} + +// ClaimInstallationStep uses the existing isolated job store. A gateway replica +// takes one lease at a time and bounds the workload call below that lease. +func (s *Service) ClaimInstallationStep(ctx context.Context, worker string) (*jobsv1.JobEnvelope, error) { + if s.moduleJobStore == nil { + return nil, errors.New("installation job store unavailable") + } + result, err := s.moduleJobStore.Claim(ctx, &jobsv1.ClaimJobsRequest{ + Queue: InstallationStepQueue, WorkerId: worker, Limit: 1, + LeaseDuration: durationpb.New(InstallationStepLease), + }) + if err != nil || len(result.GetJobs()) == 0 { + return nil, err + } + return result.Jobs[0], nil +} + +// FinishInstallationStep delegates transitions, lease fencing and exhaustion to +// the common job system. The migration projects each transition onto the current +// installation atomically with the job attempt, including expired final leases. +func (s *Service) FinishInstallationStep(ctx context.Context, lease *jobsv1.JobLeaseReference, attempt uint32, status int, reason string) error { + if s.moduleJobStore == nil { + return errors.New("installation job store unavailable") + } + switch status { + case 204: + return s.moduleJobStore.Complete(ctx, &jobsv1.CompleteJobRequest{Lease: lease}) + case 409: + return s.moduleJobStore.DeadLetter(ctx, &jobsv1.DeadLetterJobRequest{Lease: lease, + Failure: &jobsv1.JobFailure{Code: "installation.refused", Message: BoundedInstallationReason(reason)}}) + default: + _, err := s.moduleJobStore.Retry(ctx, &jobsv1.RetryJobRequest{Lease: lease, + Failure: &jobsv1.JobFailure{Code: "installation.unavailable", Message: InstallationStepFailure}, + RetryAt: timestamppb.New(time.Now().Add(InstallationStepRetryDelay(attempt))), + }) + return err + } +} diff --git a/module/services/accounts/code/pkg/business/installation_jobs_test.go b/module/services/accounts/code/pkg/business/installation_jobs_test.go new file mode 100644 index 000000000..7a0a40ea0 --- /dev/null +++ b/module/services/accounts/code/pkg/business/installation_jobs_test.go @@ -0,0 +1,72 @@ +package business_test + +import ( + "context" + "encoding/json" + "fmt" + "strings" + "testing" + "time" + "unicode/utf8" + + "accounts/pkg/business" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/jobs" + + "github.com/stretchr/testify/require" +) + +// HOST-INST-002, HOST-INST-005: retry runs are distinct jobs, but the solution +// sees exactly the same installation/revision identity and no stored credential. +func TestInstallationStepJobIdentity(t *testing.T) { + step := business.InstallationStep{InstallationID: moduleUserA, OrganizationID: moduleTenantA, Revision: 7} + initial, err := business.NewInstallationStepJob(step, "initial") + require.NoError(t, err) + retry, err := business.NewInstallationStepJob(step, "retry-one") + require.NoError(t, err) + require.NotEqual(t, initial.Job.IdempotencyKey, retry.Job.IdempotencyKey) + require.Equal(t, initial.Job.Payload, retry.Job.Payload) + require.Equal(t, uint32(business.InstallationStepMaxAttempts), initial.Job.MaxAttempts) + require.NoError(t, jobs.ValidateCommand(initial)) + var decoded map[string]any + require.NoError(t, json.Unmarshal(initial.Job.Payload, &decoded)) + require.Len(t, decoded, 3) + require.Equal(t, step.InstallationID, decoded["installation_id"]) + require.Equal(t, step.OrganizationID, initial.Job.Scope.GetOrganizationId()) + require.Equal(t, step.InstallationID, initial.Job.Attributes["codefly.installation_id"]) + require.Equal(t, step.OrganizationID, initial.Job.Attributes["codefly.organization_id"]) + require.Equal(t, "7", initial.Job.Attributes["codefly.installation_revision"]) +} + +// HOST-INST-004: only an explicit refusal is permanent, and its reason is +// bounded in bytes without corrupting UTF-8. Transport text is never persisted. +func TestInstallationStepOutcomes(t *testing.T) { + for _, status := range []int{204, 409, 503, 0, 200, 302} { + t.Run(fmt.Sprint(status), func(t *testing.T) { + backend := &fakeJobBackend{} + svc := newModuleService(t, backend) + lease := &jobsv1.JobLeaseReference{JobId: moduleUserA, WorkerId: "example-worker", LeaseToken: moduleUserB} + before := time.Now() + require.NoError(t, svc.FinishInstallationStep(context.Background(), lease, 3, status, "Example safe reason")) + switch status { + case 204: + require.Len(t, backend.completed, 1) + require.Empty(t, backend.retried) + require.Empty(t, backend.deadLetter) + case 409: + require.Len(t, backend.deadLetter, 1) + require.Equal(t, "Example safe reason", backend.deadLetter[0].Failure.Message) + default: + require.Len(t, backend.retried, 1) + require.Equal(t, business.InstallationStepFailure, backend.retried[0].Failure.Message) + require.WithinDuration(t, before.Add(20*time.Second), backend.retried[0].RetryAt.AsTime(), time.Second) + } + }) + } + for _, input := range []string{"", "\xff", strings.Repeat("é", 512)} { + reason := business.BoundedInstallationReason(input) + require.NotEmpty(t, reason) + require.True(t, utf8.ValidString(reason)) + require.LessOrEqual(t, len(reason), business.InstallationFailureLimit) + } +} diff --git a/module/services/accounts/code/pkg/business/installation_review_test.go b/module/services/accounts/code/pkg/business/installation_review_test.go index dbe0ae7f3..3c350652c 100644 --- a/module/services/accounts/code/pkg/business/installation_review_test.go +++ b/module/services/accounts/code/pkg/business/installation_review_test.go @@ -32,6 +32,15 @@ func TestInstallationArtifactPolicyAtEveryDecision(t *testing.T) { _, err = s.DecideExecutableArtifact(context.Background(), caller, tenant, uuid.NewString(), "example", action, req) require.NoError(t, err) } + // HOST-INST-002, HOST-INST-004: even retained approvals are unavailable. + for _, state := range []gen.InstallationStatus{gen.InstallationStatus_INSTALLATION_STATUS_PENDING, gen.InstallationStatus_INSTALLATION_STATUS_FAILED} { + store.installation.Status = state + for _, action := range []string{"approve", "authorize"} { + _, err = s.DecideExecutableArtifact(context.Background(), caller, tenant, uuid.NewString(), "example", action, req) + require.Error(t, err) + } + } + store.installation.Status = gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE live.rows[tenant+"/"+caller.PrincipalID].Scope.ArtifactPolicies = nil live.rows[tenant+"/"+caller.PrincipalID].InstallationRevision++ for _, action := range []string{"approve", "authorize"} { diff --git a/module/services/accounts/code/pkg/business/installation_scope_integration_test.go b/module/services/accounts/code/pkg/business/installation_scope_integration_test.go index 28cfdddd7..d7c31f634 100644 --- a/module/services/accounts/code/pkg/business/installation_scope_integration_test.go +++ b/module/services/accounts/code/pkg/business/installation_scope_integration_test.go @@ -29,6 +29,7 @@ func TestInstallationScopeInstallNarrowAuditAndPolicyLog(t *testing.T) { scope.Queues = []string{"example.ingest"} installed, err := testService.InstallSolution(testCtx, owner, params) require.NoError(t, err) + acknowledgeInstallation(t, installed) state := func() *business.InstallationAuthorityState { t.Helper() var out *business.InstallationAuthorityState diff --git a/module/services/accounts/code/pkg/business/installation_step_scope_test.go b/module/services/accounts/code/pkg/business/installation_step_scope_test.go new file mode 100644 index 000000000..bfb8440a5 --- /dev/null +++ b/module/services/accounts/code/pkg/business/installation_step_scope_test.go @@ -0,0 +1,25 @@ +package business + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +// HOST-INST-003: a job cannot borrow another installation's boundary, an +// undelivered action, or authority outside its agent's ceiling/standing role. +func TestInstallationStepScopesAreRootBounded(t *testing.T) { + a := &InstallationStepAuthority{RootID: "own-root", Facts: &InstallationAuthorityFacts{Actor: &Principal{AllowedScopes: []string{"records"}}}, + StandingPermissions: []WorkContextPermission{{ResourceKind: "records", Action: "read"}}, + Scope: InstallationScope{OperationAudiences: map[string]ModuleOperationAudience{ + "example": {InvokeScopes: []ModuleOperationScope{{ResourceKind: "records", Actions: []string{"read", "write"}}, {ResourceKind: "other", Actions: []string{"read"}}}}, + "foreign": {InvokeScopes: []ModuleOperationScope{{ResourceKind: "records", Actions: []string{"delete"}, ResourceIDs: []string{"other-root"}}}}, + }}, + } + scopes := installationStepScopes(a) + require.Len(t, scopes, 1) + require.Equal(t, []string{"read"}, scopes[0].Actions) + require.Equal(t, []string{"own-root"}, scopes[0].ResourceIds) + a.StandingPermissions = nil + require.Empty(t, installationStepScopes(a)) +} diff --git a/module/services/accounts/code/pkg/business/installation_target_identity_integration_test.go b/module/services/accounts/code/pkg/business/installation_target_identity_integration_test.go index da9c0889c..cfde274cd 100644 --- a/module/services/accounts/code/pkg/business/installation_target_identity_integration_test.go +++ b/module/services/accounts/code/pkg/business/installation_target_identity_integration_test.go @@ -52,6 +52,7 @@ func TestInstallationTargetCannotBeRepointed(t *testing.T) { AllowedScopes: []string{"doc"}, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) // Repointing at ANOTHER live target is refused. The other target is live, so // nothing here is explained by the liveness trigger below. diff --git a/module/services/accounts/code/pkg/business/installations.go b/module/services/accounts/code/pkg/business/installations.go index 7f8b944dc..f98489e4e 100644 --- a/module/services/accounts/code/pkg/business/installations.go +++ b/module/services/accounts/code/pkg/business/installations.go @@ -66,7 +66,7 @@ type InstallationStore interface { InstallSolution(ctx context.Context, params *InstallSolutionParams) (*gen.Installation, error) GetInstallation(ctx context.Context, orgID, installationID string) (*gen.Installation, gen.InstallationHealth, error) TransferInstallationOwnership(ctx context.Context, orgID, installationID, newOwnerPrincipalID string, coOwnerPrincipalIDs []string) (*gen.Installation, error) - // The bool reports whether this call actually flipped an active installation + // The bool reports whether this call actually flipped a nonrevoked installation // to revoked, so the caller emits the audit event exactly once. UninstallSolution(ctx context.Context, orgID, installationID string) (*gen.Installation, bool, error) // ListInstallations returns one page of an org's installations with their live @@ -309,7 +309,7 @@ func (s *Service) uninstallSolutionTx(ctx context.Context, actorID, actorType, o if err != nil { return err } - // Only a real active→revoked transition is a fact worth publishing; + // Only a real transition to revoked is a fact worth publishing; // an idempotent re-uninstall emits neither event nor audit. Publish in // the same transaction (outbox) so the revoke and its event are atomic. if !transitioned { diff --git a/module/services/accounts/code/pkg/business/installations_integration_test.go b/module/services/accounts/code/pkg/business/installations_integration_test.go index d8e44ea8c..3c19866f7 100644 --- a/module/services/accounts/code/pkg/business/installations_integration_test.go +++ b/module/services/accounts/code/pkg/business/installations_integration_test.go @@ -8,6 +8,7 @@ import ( "accounts/pkg/business" gen "accounts/pkg/gen/saas/accounts/v1" + "accounts/pkg/infra/storetx" "github.com/stretchr/testify/require" ) @@ -72,6 +73,26 @@ func TestInstallSolutionEmitsAuditWithGrantorAndCeiling(t *testing.T) { require.Equal(t, roleID, entry.Payload["role_id"]) require.Equal(t, targetID, entry.Payload["target_id"], "the audit trail names the immutable target, not a reusable route alias") + + // HOST-INST-002, HOST-INST-005: the business mutation returns pending; + // retrying failed consent records one audit fact and keeps its revision. + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_PENDING, installation.Status) + require.NoError(t, testStore.WithOrgTx(ctx, orgID, func(ctx context.Context) error { + _, err := storetx.Tx(ctx).Exec(ctx, `UPDATE installations SET status='failed',failure_reason='Example refusal' WHERE org_id=$1 AND id=$2`, orgID, installation.Id) + return err + })) + retried, err := testService.RetryInstallation(ctx, adminID, orgID, installation.Id) + require.NoError(t, err) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_PENDING, retried.Status) + require.Empty(t, retried.FailureReason) + _, err = testService.RetryInstallation(ctx, adminID, orgID, installation.Id) + require.Error(t, err) + events, _, _, err = testService.QueryAuditLog(ctx, business.AuditQuery{OrgID: orgID, EventType: string(business.EventInstallationRetried), PageSize: 10}) + require.NoError(t, err) + require.Len(t, events, 1) + require.Equal(t, "1", events[0].Payload["revision"]) + require.Equal(t, adminID, events[0].ActorID) + require.Equal(t, installation.Id, events[0].ResourceID) } // A malformed agent identifier is a boundary input: it must be rejected as a diff --git a/module/services/accounts/code/pkg/business/introspection.go b/module/services/accounts/code/pkg/business/introspection.go index 6f88aecab..240503d70 100644 --- a/module/services/accounts/code/pkg/business/introspection.go +++ b/module/services/accounts/code/pkg/business/introspection.go @@ -151,6 +151,7 @@ var rpcDescriptions = withWorkContextConsumerDescriptions(map[string]string{ "DelegationService/DecideDelegation": "Approve or deny a delegation request.", "InstallationService/InstallSolution": "Install a solution: compose its agent principal, scope node, standing grant, and installation row.", "InstallationService/UninstallSolution": "Uninstall a solution and reverse its composition.", + "InstallationService/RetryInstallation": "Run a failed installation's install step again; the installation is pending until the solution acknowledges.", "InstallationService/NarrowInstallation": "Narrow an installation's scope, bump its revision and cancel the organization's affected work.", "InstallationService/TransferInstallationOwnership": "Reassign an installation's accountable owner of record.", "InstallationService/GetInstallation": "Get one installation and its live health.", diff --git a/module/services/accounts/code/pkg/business/live_module_authority_integration_test.go b/module/services/accounts/code/pkg/business/live_module_authority_integration_test.go index fb6f48381..15e5fa32b 100644 --- a/module/services/accounts/code/pkg/business/live_module_authority_integration_test.go +++ b/module/services/accounts/code/pkg/business/live_module_authority_integration_test.go @@ -41,6 +41,7 @@ func TestLiveModuleAuthorityOverARealInstallation(t *testing.T) { AllowedScopes: []string{"doc"}, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) // A module principal, which is NOT the installation's agent principal. The // regression assumed these were the same value. @@ -95,6 +96,7 @@ func TestLiveModuleAuthorityRefusesARevokedInstallation(t *testing.T) { AllowedScopes: []string{"doc"}, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) modulePrincipal := business.ModulePrincipalID(alias) // Live first, so the refusal below is the uninstall and not the fixture. diff --git a/module/services/accounts/code/pkg/business/module_capabilities.go b/module/services/accounts/code/pkg/business/module_capabilities.go index 211a86739..f7b0e2c25 100644 --- a/module/services/accounts/code/pkg/business/module_capabilities.go +++ b/module/services/accounts/code/pkg/business/module_capabilities.go @@ -80,6 +80,7 @@ import ( // running image comes from the pod's status, and the approval comes from a // signed document. type ModuleWorkload struct { + SPIFFEID string `json:"spiffe_id"` ServiceAccount string `json:"service_account"` Namespace string `json:"namespace"` Container string `json:"container"` diff --git a/module/services/accounts/code/pkg/business/module_capability_enforcement.go b/module/services/accounts/code/pkg/business/module_capability_enforcement.go index 866c61011..5985e9a6d 100644 --- a/module/services/accounts/code/pkg/business/module_capability_enforcement.go +++ b/module/services/accounts/code/pkg/business/module_capability_enforcement.go @@ -66,6 +66,15 @@ var ErrModuleAuthorityStale = errors.New("module capability authority has moved // this organization, including credentials that do not yet carry a seal. var ErrModuleInstallationInactive = errors.New("the installation this capability names is not active in this organization") +// errModuleInstalledNowhere is ErrModuleInstallationInactive for the one case +// a module's own mint answers differently: the principal has no usable +// installation in the organization at all. Every use of a capability hears it +// as the refusal it is. The module's own mint hears "not yet", because a +// solution nobody has installed still has an install step to answer — see +// ModuleAuthorizeWorkContext. An installation that belongs to another binding +// is not this: that one is a refusal on every path. +var errModuleInstalledNowhere = fmt.Errorf("%w", ErrModuleInstallationInactive) + // ErrModuleAuthorityUnreadable reports a malformed caller the live read refuses // before it reaches the database. // @@ -225,7 +234,7 @@ func (s *Service) authorizeModuleCapabilityFromSnapshot(ctx context.Context, sna ctx, caller.PrincipalID, caller.BoundOrg, sealedInstallation) if err != nil { if errors.Is(err, ErrModuleInstallationInactive) { - return nil, nil, wrapStatus(codes.PermissionDenied, ErrModuleInstallationInactive, + return nil, nil, wrapStatus(codes.PermissionDenied, errModuleInstalledNowhere, "principal %s in organization %s", caller.PrincipalID, caller.BoundOrg) } if errors.Is(err, ErrModuleAuthorityUnreadable) { @@ -241,7 +250,7 @@ func (s *Service) authorizeModuleCapabilityFromSnapshot(ctx context.Context, sna } if live == nil || live.InstallationID == "" || !live.Scope.Capabilities { - return nil, nil, wrapStatus(codes.PermissionDenied, ErrModuleInstallationInactive, "principal %s in organization %s", caller.PrincipalID, caller.BoundOrg) + return nil, nil, wrapStatus(codes.PermissionDenied, errModuleInstalledNowhere, "principal %s in organization %s", caller.PrincipalID, caller.BoundOrg) } if live.RegistryBindingID != grant.PresenceBinding { return nil, nil, wrapStatus(codes.PermissionDenied, ErrModuleInstallationInactive, "installation belongs to a different registry binding") diff --git a/module/services/accounts/code/pkg/business/module_envelope.go b/module/services/accounts/code/pkg/business/module_envelope.go index ca77272bc..d018376b7 100644 --- a/module/services/accounts/code/pkg/business/module_envelope.go +++ b/module/services/accounts/code/pkg/business/module_envelope.go @@ -128,7 +128,7 @@ func (w ModuleWorkloadIdentity) workload() ModuleWorkload { if err != nil { return ModuleWorkload{Container: w.Container} } - return ModuleWorkload{ServiceAccount: identity.ServiceAccount, Namespace: identity.Namespace, Container: w.Container} + return ModuleWorkload{SPIFFEID: w.SPIFFEID, ServiceAccount: identity.ServiceAccount, Namespace: identity.Namespace, Container: w.Container} } // ModuleEnvelopeRecord is one stored revision: the document as written, and who @@ -366,6 +366,11 @@ func parseModuleEnvelopePrincipal(prefix string, wire moduleEnvelopePrincipalWir return ModuleEnvelopePrincipal{}, err } grant := wire.Grant.grant(prefix) + // The install-step queue is the host's own: its jobs are other solutions' + // install steps, and a module granted it could claim and complete one. + if slices.Contains(grant.Queues, InstallationStepQueue) { + return ModuleEnvelopePrincipal{}, fmt.Errorf("principal %q is granted the queue %q, which is the host's own and is granted to no module", prefix, InstallationStepQueue) + } tenant, err := canonicalModuleTenant(prefix, grant.Tenant) if err != nil { return ModuleEnvelopePrincipal{}, err @@ -683,7 +688,10 @@ func (s *Service) PutModuleEnvelope( entry := &PolicyLogEntry{OperationID: newPolicyLogOperationID("module-envelope"), Decision: PolicyLogNarrowed, SubjectKind: PolicyLogModuleEnvelope, SubjectID: "module-envelope", Policy: policy, EnvelopeRevision: envelope.Revision, Actor: actorID} - if err := s.WithPolicyLoggedNarrowing(ctx, entry, func(ctx context.Context) error { + // witnessed is set before apply runs under the policy log; apply refuses + // to revoke anything while it is false. + witnessed := false + apply := func(ctx context.Context) error { if err := s.moduleEnvelopes.AppendModuleEnvelopeRecord(ctx, record, expectedRevision); err != nil { return err } @@ -695,6 +703,13 @@ func (s *Service) PutModuleEnvelope( if installationTenancyAllowed(envelope, installation.PrincipalID, installation.OrgID) { continue } + // An installation acknowledged between the classification and + // this transaction is found here and nowhere else. Revoking it is + // a narrowing, so the write that was classified a pure grant + // stops and is taken again under the witness. + if !witnessed { + return errModuleEnvelopeNarrowsAfterAll + } if err := s.uninstallSolutionTx(ctx, actorID, "user", installation.OrgID, installation.ID); err != nil { return err } @@ -705,7 +720,32 @@ func (s *Service) PutModuleEnvelope( "previous_revision": int64(expectedRevision), //nolint:gosec // a revision is a small positive counter "principals": len(envelope.Principals), }) - }); err != nil { + } + // The policy log witnesses NARROWINGS — "a narrowing nothing witnessed is + // one a restore undoes silently". A write that only grants (a first + // envelope, a build or a principal added, nothing an installation loses) + // is not one: a restore that undoes it leaves less authority, which is the + // fail-closed direction, so it is applied and audited without a witness. + // Anything that removes or changes what was granted is witnessed as today. + narrows, err := s.moduleEnvelopeNarrows(ctx, envelope, document) + if err != nil { + return nil, err + } + underWitness := func(ctx context.Context) error { + witnessed = true + return s.WithPolicyLoggedNarrowing(ctx, entry, apply) + } + write := underWitness + if !narrows { + write = func(ctx context.Context) error { + err := s.store.WithControlPlane(ctx, apply) + if errors.Is(err, errModuleEnvelopeNarrowsAfterAll) { + return underWitness(ctx) + } + return err + } + } + if err := write(ctx); err != nil { if errors.Is(err, ErrModuleEnvelopeConflict) { return nil, NewStoreError(err, ErrTypeConflict) } @@ -727,6 +767,11 @@ func (s *Service) PutModuleEnvelope( // has. var ErrModuleRegistryStale = errors.New("the module registry predates the committed module envelope") +// errModuleEnvelopeNarrowsAfterAll stops an envelope write that was classified +// a pure grant and then found something to revoke. It never leaves this file: +// the write is retried under the witness. +var errModuleEnvelopeNarrowsAfterAll = errors.New("the envelope write revokes an installation and was not witnessed") + // moduleRegistrySnapshot is the installed registry and the envelope revision // it was derived from, loaded ONCE. A decision reads its grant from this value // and fences this value — never a second load — so a registry installed between @@ -849,6 +894,57 @@ func (s *Service) currentModuleGrantFromSnapshot( return grant, true, nil } +// deliveredSince reports that the delivery inbox holds a document the registry +// in snapshot was derived without. +// +// It is the delivery half of the envelope fence above, for the one answer that +// fence does not cover. A principal the registry does not hold is "not a +// declared module principal" — a judgement, which ends the workload that asked. +// That is right when nothing declares it and wrong for the moment between a +// document being accepted and the pass it requested finishing: the host has +// been told, and has not yet read what it was told. A workload that starts +// beside its own delivery asks in exactly that moment, and it must hear "not +// established yet", which it retries, rather than a refusal. +// +// Durable and per replica, like the fence: the inbox is in the store and each +// replica derives its own registry, so the comparison is between the snapshot +// this decision reads and the store, never between two in-memory views. +// +// Nothing is granted here and nothing is widened. The only answer it changes +// is which refusal an undeclared principal hears, and only while a delivered +// document is unread. A document that has been read and grants this principal +// nothing leaves the count equal, and the answer is the judgement it always +// was. Delivery is authenticated and signed, so a caller cannot manufacture +// the condition. Where the registry carries no marker — it was stated +// directly, or published degraded — or the inbox cannot be counted, the answer +// is unchanged. +func (s *Service) deliveredSince(ctx context.Context, snapshot *installedModuleRegistry) bool { + if snapshot == nil || snapshot.delivered == nil { + return false + } + counter, counts := s.deliveryStore.(SolutionDeliveryCounter) + if !counts { + return false + } + var delivered, unapplied uint64 + if err := s.store.WithControlPlane(ctx, func(ctx context.Context) error { + var err error + if delivered, err = counter.CountDeliveredDocuments(ctx); err != nil { + return err + } + unapplied, err = counter.CountUnappliedPresence(ctx) + return err + }); err != nil { + return false + } + // A document the pass did not read, or a presence the pass read before the + // binding reconciler applied it: either way the registry is about to + // change. The second is the common one — the two reconcilers run apart, + // and a registry derived between them omits the principal whose presence + // was applied a moment later. + return delivered > *snapshot.delivered || unapplied > 0 +} + // ModuleInstallationConsent identifies the rows a tenancy change may revoke. // Agent-only legacy installs have no module principal and are not inferred here. type ModuleInstallationConsent struct{ ID, OrgID, PrincipalID string } @@ -861,3 +957,161 @@ func installationTenancyAllowed(envelope *ModuleEnvelope, principalID, orgID str } return false } + +// moduleEnvelopeNarrows reports whether writing next would take authority away +// from anyone, and it is the boundary of the one write that skips the +// policy-log witness, so it recognises a pure grant and calls everything else a +// narrowing. +// +// A pure grant is exactly this, against the envelope in force: every approved +// build is still approved, every principal is still there with a section that +// is the same JSON value, every other top-level key is the same JSON value, +// and no installation loses its organization. What may differ is the +// revision, builds that were added, and whole principals that were added. +// +// A principal's section is compared for equality and never for "at least as +// much". The schema does not let more JSON mean more authority: a resource +// list that appears narrows an audience that was unrestricted, a required +// kind that appears refuses artifacts that passed, a tenant that is set +// removes every other organization, and a unit that gains a field is a +// different unit under core's whole-unit containment. A comparison that tried +// to tell those apart would be a second copy of every rule that reads the +// envelope, wrong whenever one of them moved. So a changed section is a +// narrowing, whatever changed. +// +// A host with no envelope has granted nothing, so its first envelope narrows +// nothing an envelope granted; installations are still checked. +func (s *Service) moduleEnvelopeNarrows(ctx context.Context, next *ModuleEnvelope, nextDocument []byte) (bool, error) { + narrows := false + err := s.store.WithControlPlane(ctx, func(ctx context.Context) error { + current, err := s.moduleEnvelopes.LatestModuleEnvelopeRecord(ctx) + if err != nil { + return err + } + if current != nil && !moduleEnvelopeOnlyAdds(current.Document, nextDocument) { + narrows = true + return nil + } + // Inside the control plane, like the apply's own read: outside it a + // row policy hides every installation, and the answer is "none lose + // anything" for the wrong reason. + installations, err := s.moduleEnvelopes.ModuleInstallations(ctx) + if err != nil { + return err + } + for _, installation := range installations { + if !installationTenancyAllowed(next, installation.PrincipalID, installation.OrgID) { + narrows = true + return nil + } + } + return nil + }) + return narrows, err +} + +// moduleEnvelopeOnlyAdds reports that after is before plus approved builds +// and whole principals, and nothing else: see moduleEnvelopeNarrows. Anything +// it cannot read as that — a document that does not decode, a key it does not +// know — is not an addition. +func moduleEnvelopeOnlyAdds(before, after []byte) bool { + was, ok := envelopeTopLevel(before) + if !ok { + return false + } + is, ok := envelopeTopLevel(after) + if !ok { + return false + } + for key := range was { + if _, kept := is[key]; !kept { + return false + } + } + for key, value := range is { + switch key { + case "revision": + case "approved_builds": + if !envelopeBuildsKept(was[key], value) { + return false + } + case "principals": + if !envelopePrincipalsKept(was[key], value) { + return false + } + default: + previous, existed := was[key] + if !existed || !sameJSONValue(previous, value) { + return false + } + } + } + return true +} + +func envelopeTopLevel(document []byte) (map[string]json.RawMessage, bool) { + var top map[string]json.RawMessage + if err := json.Unmarshal(document, &top); err != nil || top == nil { + return nil, false + } + return top, true +} + +func envelopeBuildsKept(before, after json.RawMessage) bool { + var was, is []string + if len(before) > 0 && json.Unmarshal(before, &was) != nil { + return false + } + if len(after) > 0 && json.Unmarshal(after, &is) != nil { + return false + } + for _, build := range was { + if !slices.Contains(is, build) { + return false + } + } + return true +} + +func envelopePrincipalsKept(before, after json.RawMessage) bool { + var was, is map[string]json.RawMessage + if len(before) > 0 && json.Unmarshal(before, &was) != nil { + return false + } + if len(after) > 0 && json.Unmarshal(after, &is) != nil { + return false + } + for prefix, section := range was { + kept, exists := is[prefix] + if !exists || !sameJSONValue(section, kept) { + return false + } + } + return true +} + +// sameJSONValue reports that two encodings are one JSON value: key order and +// whitespace aside, every member, element and number literal equal. Numbers +// are compared as written, so two revisions past 2^53 that a float would fold +// together stay different. +func sameJSONValue(a, b json.RawMessage) bool { + canonical := func(raw json.RawMessage) ([]byte, bool) { + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.UseNumber() + var value any + if err := decoder.Decode(&value); err != nil { + return nil, false + } + if decoder.More() { + return nil, false + } + encoded, err := json.Marshal(value) + return encoded, err == nil + } + left, ok := canonical(a) + if !ok { + return false + } + right, ok := canonical(b) + return ok && bytes.Equal(left, right) +} diff --git a/module/services/accounts/code/pkg/business/module_envelope_test.go b/module/services/accounts/code/pkg/business/module_envelope_test.go index b74d46869..a9f74827b 100644 --- a/module/services/accounts/code/pkg/business/module_envelope_test.go +++ b/module/services/accounts/code/pkg/business/module_envelope_test.go @@ -362,7 +362,7 @@ func TestOnlyAnApprovedWorkloadBindsThePrincipal(t *testing.T) { workload, err := approvedWorkload(entry, []ModuleWorkloadIdentity{{SPIFFEID: testSPIFFEID, Container: testContainer}}, document) require.NoError(t, err) - require.Equal(t, ModuleWorkload{ServiceAccount: "reports", Namespace: "acme-prod", Container: testContainer}, workload) + require.Equal(t, ModuleWorkload{SPIFFEID: testSPIFFEID, ServiceAccount: "reports", Namespace: "acme-prod", Container: testContainer}, workload) for name, retargeted := range map[string]ModuleWorkloadIdentity{ "another service account": {SPIFFEID: "spiffe://acme.test/ns/acme-prod/sa/intruder", Container: testContainer}, @@ -423,7 +423,7 @@ func TestSPIFFEIDsAreParsedStrictly(t *testing.T) { []any{map[string]any{"spiffe_id": "spiffe://ns/ns/acme-prod/sa/reports", "container": testContainer}} envelope, err := ParseModuleEnvelope(encodeEnvelope(t, document)) require.NoError(t, err) - require.Equal(t, ModuleWorkload{ServiceAccount: "reports", Namespace: "acme-prod", Container: testContainer}, + require.Equal(t, ModuleWorkload{SPIFFEID: "spiffe://ns/ns/acme-prod/sa/reports", ServiceAccount: "reports", Namespace: "acme-prod", Container: testContainer}, envelope.Principals[testModulePrefix].WorkloadIdentities[0].workload()) } @@ -486,3 +486,16 @@ func TestSharedEnvelopeTenantIsNotAnOrganizationGrant(t *testing.T) { _, err = canonicalModuleTenant("example", "not-an-organization") require.Error(t, err) } + +// The install-step queue is the host's own: its jobs are other solutions' +// install steps, and a module granted it could claim and complete one. +func TestTheInstallStepQueueIsGrantedToNoModule(t *testing.T) { + document := testEnvelopeDocument() + principal := document["principals"].(map[string]any)[testModulePrefix].(map[string]any) + grant := principal["grant"].(map[string]any) + queues, _ := grant["queues"].([]any) + grant["queues"] = append(queues, InstallationStepQueue) + _, err := ParseModuleEnvelope(encodeEnvelope(t, document)) + require.ErrorIs(t, err, ErrModuleEnvelopeInvalid) + require.ErrorContains(t, err, InstallationStepQueue) +} diff --git a/module/services/accounts/code/pkg/business/module_execution_binding.go b/module/services/accounts/code/pkg/business/module_execution_binding.go index a6b2a84a8..e43361026 100644 --- a/module/services/accounts/code/pkg/business/module_execution_binding.go +++ b/module/services/accounts/code/pkg/business/module_execution_binding.go @@ -310,6 +310,15 @@ var ErrExecutionAuthorityUnreconciled = errors.New( // audience is what makes the token a credential FOR this host. const executionTokenAudience = "accounts" +// WithWorkloadPeer carries the gateway's verified TLS identity through the +// internal mint. Absence is the existing plain module mint; a present but empty +// value is a refused attestation, never permission to omit the binding. +func WithWorkloadPeer(ctx context.Context, id string) context.Context { + return context.WithValue(ctx, workloadPeerContextKey{}, id) +} + +type workloadPeerContextKey struct{} + // BindExecution establishes what a caller is running and holds it against what // the authority document approves. The caller passes its mint snapshot; both // the declared container and the workload-owner lookup use that same registry. @@ -359,6 +368,13 @@ func (s *Service) BindExecution( // and nothing about the caller has been decided. return nil, fmt.Errorf("%w: %w", ErrExecutionUnbound, err) } + if peer, presented := ctx.Value(workloadPeerContextKey{}).(string); presented { + identity, err := ParseKubernetesSPIFFEID(peer) + if err != nil || peer != grant.Workload.SPIFFEID || + identity.Namespace != reviewed.Namespace || identity.ServiceAccount != reviewed.ServiceAccount { + return nil, fmt.Errorf("%w: TLS workload identity differs from reviewed token or delivered presence", ErrExecutionIdentityMismatch) + } + } if reviewed.PodUID == "" || reviewed.PodName == "" { return nil, fmt.Errorf("%w: the token authenticates %s/%s but names no pod", ErrExecutionNotBoundToken, reviewed.Namespace, reviewed.ServiceAccount) @@ -366,8 +382,8 @@ func (s *Service) BindExecution( // THE TOKEN'S LIFETIME IS THE REPLAY WINDOW. A projected token is a bearer // credential: TokenReview proves the cluster issued it to this pod, never - // that the pod is the one presenting it. The host cannot close that here - // (see "Replay" in services/accounts/AGENTS.md); it can bound it, and it + // that the pod is the one presenting it. The workload listener additionally + // binds its TLS peer above. For plain module mints the host // bounds it to the shortest projection Kubernetes issues. if reviewed.IssuedAt.IsZero() || reviewed.ExpiresAt.IsZero() { return nil, fmt.Errorf("%w: the review reported no issue or expiry time", ErrExecutionTokenLifetime) diff --git a/module/services/accounts/code/pkg/business/module_execution_snapshot_test.go b/module/services/accounts/code/pkg/business/module_execution_snapshot_test.go index daf747b5a..89d82e99f 100644 --- a/module/services/accounts/code/pkg/business/module_execution_snapshot_test.go +++ b/module/services/accounts/code/pkg/business/module_execution_snapshot_test.go @@ -98,7 +98,7 @@ func TestModuleMintFencesARevisionCommittedDuringReview(t *testing.T) { registry := service.ModulePrincipals() grant := registry["principal-1"] registry = ModulePrincipalRegistry{ModulePrincipalID("worker"): grant} - service.InstallDerivedModulePrincipals(registry, 3) + service.InstallDerivedModulePrincipals(registry, 3, nil) envelope := &fakeEnvelopeStore{latest: 3} service.SetModuleEnvelopeStore(envelope) reviewer := &replacingExecutionReviewer{fakeExecutionReviewer: fakeExecutionReviewer{reviewed: reviewedPod(), pod: runningPodWith()}, replace: func() { envelope.latest = 4 }} diff --git a/module/services/accounts/code/pkg/business/module_identity.go b/module/services/accounts/code/pkg/business/module_identity.go index 7b7ff7f4a..824d966c4 100644 --- a/module/services/accounts/code/pkg/business/module_identity.go +++ b/module/services/accounts/code/pkg/business/module_identity.go @@ -98,6 +98,14 @@ func (s *Service) ModuleAuthorizeWorkContext( } installed, _, err := s.authorizeModuleCapabilityFromSnapshot(ctx, snapshot, ModuleCaller{PrincipalID: identity.PrincipalID, BoundOrg: identity.Tenant}) if err != nil { + if errors.Is(err, errModuleInstalledNowhere) { + // Proven, admitted, and installed nowhere: a precondition this + // workload waits on, not a judgement of it. Only here — a + // capability's use and a headless operation have no install step + // to wait for, and hear the refusal. + return ModuleWorkContextAuthority{}, wrapStatus(codes.FailedPrecondition, errModuleInstalledNowhere, + "principal %s in organization %s", identity.PrincipalID, identity.Tenant) + } return ModuleWorkContextAuthority{}, err } identity.InstallationID, identity.InstallationRevision, identity.ProducerEpoch = installed.InstallationID, installed.InstallationRevision, installed.ProducerEpoch @@ -119,6 +127,15 @@ func (s *Service) authorizeModuleIdentity( return ModuleWorkContextAuthority{}, ModulePrincipalGrant{}, err } if !registered { + // A workload that starts beside its own delivery asks before the pass + // that reads it has run. Only here — the mint that names a prefix — is + // that "not established yet": every other caller of the registry, + // a person's revision check among them, hears what it always did. + if s.deliveredSince(ctx, snapshot) { + s.requestRegistryPass() + return ModuleWorkContextAuthority{}, ModulePrincipalGrant{}, wrapStatus(codes.Unavailable, ErrModuleRegistryStale, + "principal %s: a document was delivered after the registry was derived", principalID) + } return ModuleWorkContextAuthority{}, ModulePrincipalGrant{}, fmt.Errorf("%w: %s is not a declared module principal", ErrModuleIdentityDenied, prefix) } diff --git a/module/services/accounts/code/pkg/business/module_installation_authority_test.go b/module/services/accounts/code/pkg/business/module_installation_authority_test.go index 57af65dc3..7486c508b 100644 --- a/module/services/accounts/code/pkg/business/module_installation_authority_test.go +++ b/module/services/accounts/code/pkg/business/module_installation_authority_test.go @@ -42,6 +42,9 @@ func (s *authorityInstallationStore) GetOrganizationBySlug(context.Context, stri func (s *authorityInstallationStore) LiveDeclaredSolutionBindingIDs(context.Context) ([]string, error) { return nil, nil } +func (s *authorityInstallationStore) ListAvailableSolutionTargets(_ context.Context, query business.AvailableSolutionQuery) ([]*business.AvailableSolutionTarget, error) { + return []*business.AvailableSolutionTarget{{TargetID: query.TargetID, BindingID: "acme.test.installation", RouteAlias: "example", Kind: "solution"}}, nil +} func (s *authorityInstallationStore) ReconcileModuleInstallation(context.Context, *business.InstallSolutionParams, []string, bool) (*business.ModuleInstallationResult, error) { s.calls++ result := s.result @@ -76,6 +79,7 @@ func authorityInstallationFixture(t *testing.T, module string) (*business.Servic // installer's path validates it like every other write path. These two are // declared module prefixes, declared the way a composition declares them. service.SetModulePrincipals(business.ModulePrincipalRegistry{ + business.ModulePrincipalID("example"): {Prefix: "example", PresenceBinding: "acme.test.installation"}, business.ModulePrincipalID("example.api"): {Prefix: "example.api"}, business.ModulePrincipalID("example.worker"): {Prefix: "example.worker"}, }) @@ -257,6 +261,14 @@ func TestInstallationAuthorityIsOptInAndAbsentHasNoReference(t *testing.T) { result, err = svc.ReconcileModuleInstallation(context.Background(), caller, policy, req, false) require.NoError(t, err) require.Nil(t, result.AuthorityReference) + // HOST-INST-002, HOST-INST-004: an existing unacknowledged installation + // cannot produce a verified authority reference through the installer. + for _, state := range []string{"pending", "failed"} { + store.result.State = state + result, err = svc.ReconcileModuleInstallation(context.Background(), caller, policy, req, false) + require.NoError(t, err) + require.Nil(t, result.AuthorityReference) + } } func TestInstallationAuthorityCanonicalGolden(t *testing.T) { svc, _, caller, policy, req := authorityInstallationFixture(t, "example-one") diff --git a/module/services/accounts/code/pkg/business/module_installations.go b/module/services/accounts/code/pkg/business/module_installations.go index 9f1cccebf..fdcac41f0 100644 --- a/module/services/accounts/code/pkg/business/module_installations.go +++ b/module/services/accounts/code/pkg/business/module_installations.go @@ -208,16 +208,24 @@ func (s *Service) ReconcileModuleInstallation(ctx context.Context, caller Module if err := s.RequireHostAudiences(ctx, req.AllowedAudiences); err != nil { return nil, err } + params := &InstallSolutionParams{ + OrgID: org.Id, AgentIdentifier: req.AgentIdentifier, TargetID: req.TargetID, + RoleID: req.RoleID, AllowedAudiences: req.AllowedAudiences, AllowedScopes: req.AllowedScopes, + DisplayName: req.DisplayName, RootScopeLabel: req.RootScopeLabel, + OwnerPrincipalID: delegation.OwnerPrincipalID, GrantedBy: delegation.OwnerPrincipalID, + InstallerPrincipalID: caller.PrincipalID, + // This delegation authorizes the agent ceiling, not module capabilities. + Scope: &InstallationScope{}, + } + // All creation paths resolve the same declared binding before opening the + // tenant transaction. A pending job needs that identity to mint its context. + if _, err := s.prepareInstallation(ctx, params); err != nil { + return nil, err + } var result *ModuleInstallationResult err = s.store.WithOrgTx(ctx, org.Id, func(ctx context.Context) error { var err error - result, err = store.ReconcileModuleInstallation(ctx, &InstallSolutionParams{ - OrgID: org.Id, AgentIdentifier: req.AgentIdentifier, TargetID: req.TargetID, - RoleID: req.RoleID, AllowedAudiences: req.AllowedAudiences, AllowedScopes: req.AllowedScopes, - DisplayName: req.DisplayName, RootScopeLabel: req.RootScopeLabel, - OwnerPrincipalID: delegation.OwnerPrincipalID, GrantedBy: delegation.OwnerPrincipalID, - InstallerPrincipalID: caller.PrincipalID, - }, delegation.RolePermissions, apply) + result, err = store.ReconcileModuleInstallation(ctx, params, delegation.RolePermissions, apply) if err != nil { return err } diff --git a/module/services/accounts/code/pkg/business/module_principal_reconcile.go b/module/services/accounts/code/pkg/business/module_principal_reconcile.go index eb523872c..320c72e0d 100644 --- a/module/services/accounts/code/pkg/business/module_principal_reconcile.go +++ b/module/services/accounts/code/pkg/business/module_principal_reconcile.go @@ -101,6 +101,10 @@ type publishedModuleRegistry struct { registry ModulePrincipalRegistry revision uint64 sources map[string]modulePrincipalSource + // delivered is the inbox's size in the snapshot this registry was derived + // from. Nil on every degraded publish: a registry that was not derived + // from a full read of the inbox says nothing about what the inbox holds. + delivered *uint64 } // modulePrincipalSource is what an entry was derived from: enough to re-judge @@ -125,7 +129,7 @@ func NewModulePrincipalReconciler(activation *SolutionAuthorityActivation) (*Mod func (r *ModulePrincipalReconciler) publish(published *publishedModuleRegistry) { r.last = published - r.activation.reconciler.service.InstallDerivedModulePrincipals(published.registry, published.revision) + r.activation.reconciler.service.InstallDerivedModulePrincipals(published.registry, published.revision, published.delivered) } // degraded is the last published registry kept only where it is still @@ -239,7 +243,11 @@ func (r *ModulePrincipalReconciler) RunOnce(ctx context.Context) error { if envelope == nil { // No ceiling has ever been written: nothing is allowed, so nothing is // granted. Not an unreadable input — the empty registry is the answer. - r.publish(&publishedModuleRegistry{registry: ModulePrincipalRegistry{}}) + empty := &publishedModuleRegistry{registry: ModulePrincipalRegistry{}} + if inputsErr == nil { + empty.delivered = inputs.delivered + } + r.publish(empty) return nil } if inputsErr != nil { @@ -276,7 +284,11 @@ func (r *ModulePrincipalReconciler) RunOnce(ctx context.Context) error { } } - next := &publishedModuleRegistry{registry: ModulePrincipalRegistry{}, revision: envelope.Revision, sources: map[string]modulePrincipalSource{}} + next := &publishedModuleRegistry{ + registry: ModulePrincipalRegistry{}, revision: envelope.Revision, sources: map[string]modulePrincipalSource{}, + // Derived from the whole inbox as this snapshot held it. + delivered: inputs.delivered, + } for _, one := range inside { document := one.document candidates, workloadErr := r.workloadsFor(document, applied[document.PresenceBinding]) diff --git a/module/services/accounts/code/pkg/business/module_registry_delivery_fence_test.go b/module/services/accounts/code/pkg/business/module_registry_delivery_fence_test.go new file mode 100644 index 000000000..95124a59d --- /dev/null +++ b/module/services/accounts/code/pkg/business/module_registry_delivery_fence_test.go @@ -0,0 +1,105 @@ +package business + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/require" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" +) + +// countedInbox is a delivery inbox that can say how many documents it holds. +type countedInbox struct { + SolutionDeliveryStore + count uint64 + unapplied uint64 + err error + reads int +} + +func (c *countedInbox) CountDeliveredDocuments(context.Context) (uint64, error) { + c.reads++ + return c.count, c.err +} + +func (c *countedInbox) CountUnappliedPresence(context.Context) (uint64, error) { + return c.unapplied, c.err +} + +// A prefix the registry does not hold is refused at its mint — unless the +// inbox holds a document the registry was derived without. A workload that +// starts beside its own delivery asks in exactly that moment, and a refusal +// there ends it for a state the host is one pass away from leaving. +func TestAnUndeclaredModuleIsNotRefusedWhileADeliveryIsUnread(t *testing.T) { + const prefix = "nobody-declared" + marker := func(n uint64) *uint64 { return &n } + setup := func(delivered *uint64, inbox *countedInbox) (*Service, *int) { + s := reviewService() + s.deliveryStore = inbox + passes := 0 + s.SetModuleRegistryPassTrigger(func() { passes++ }) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1"}}, 0, delivered) + return s, &passes + } + mint := func(s *Service) error { + _, _, err := s.authorizeModuleIdentity(context.Background(), s.moduleRegistrySnapshot(), prefix, "projected-token") + return err + } + + t.Run("nothing delivered since: the refusal stands", func(t *testing.T) { + s, passes := setup(marker(2), &countedInbox{count: 2}) + require.ErrorIs(t, mint(s), ErrModuleIdentityDenied) + require.Zero(t, *passes) + }) + t.Run("a document delivered since: not established yet, and a pass is asked for", func(t *testing.T) { + s, passes := setup(marker(2), &countedInbox{count: 3}) + err := mint(s) + require.ErrorIs(t, err, ErrModuleRegistryStale) + require.Equal(t, codes.Unavailable, status.Code(err)) + require.Equal(t, 1, *passes) + }) + t.Run("the pass read everything but a presence is not applied yet: not established yet", func(t *testing.T) { + s, passes := setup(marker(3), &countedInbox{count: 3, unapplied: 1}) + err := mint(s) + require.ErrorIs(t, err, ErrModuleRegistryStale) + require.Equal(t, codes.Unavailable, status.Code(err)) + require.Equal(t, 1, *passes) + }) + t.Run("the pass read it and still declares nothing: the refusal again", func(t *testing.T) { + s, _ := setup(marker(2), &countedInbox{count: 3}) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{}, 0, marker(3)) + require.ErrorIs(t, mint(s), ErrModuleIdentityDenied) + }) + t.Run("a registry with no marker answers as it always did", func(t *testing.T) { + inbox := &countedInbox{count: 99} + s, _ := setup(nil, inbox) + require.ErrorIs(t, mint(s), ErrModuleIdentityDenied) + require.Zero(t, inbox.reads) + }) + t.Run("an inbox that cannot be counted changes nothing", func(t *testing.T) { + s, passes := setup(marker(2), &countedInbox{err: errors.New("unreadable")}) + require.ErrorIs(t, mint(s), ErrModuleIdentityDenied) + require.Zero(t, *passes) + }) +} + +// Every other reader of the registry — a person's revision check asks it "is +// this owner a module?" — is untouched by an unread delivery: it neither waits +// nor pays for a count. +func TestAnUnreadDeliveryChangesNoOtherRegistryAnswer(t *testing.T) { + inbox := &countedInbox{count: 3} + s := reviewService() + s.deliveryStore = inbox + n := uint64(2) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1"}}, 0, &n) + _, found, err := s.currentModuleGrantFromSnapshot(context.Background(), s.moduleRegistrySnapshot(), "a-person") + require.NoError(t, err) + require.False(t, found) + grant, found, err := s.currentModuleGrantFromSnapshot(context.Background(), s.moduleRegistrySnapshot(), "principal-1") + require.NoError(t, err) + require.True(t, found) + require.Equal(t, "crm", grant.Prefix) + require.Zero(t, inbox.reads) +} diff --git a/module/services/accounts/code/pkg/business/module_registry_review_test.go b/module/services/accounts/code/pkg/business/module_registry_review_test.go index 55c022803..657c08e71 100644 --- a/module/services/accounts/code/pkg/business/module_registry_review_test.go +++ b/module/services/accounts/code/pkg/business/module_registry_review_test.go @@ -80,10 +80,10 @@ func (*fakeEnvelopeStore) AppendModuleEnvelopeRecord(context.Context, *ModuleEnv // load would see revision 4, pass, and hand out revision 3's queue. func TestTheFenceChecksTheSnapshotTheGrantCameFrom(t *testing.T) { s := reviewService() - s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1", Queues: []string{"a"}}}, 3) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1", Queues: []string{"a"}}}, 3, nil) store := &fakeEnvelopeStore{latest: 4} s.SetModuleEnvelopeStore(store) - store.duringRead = func() { s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{}, 4) } + store.duringRead = func() { s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{}, 4, nil) } authority, err := s.AuthorizeModuleCapability(context.Background(), reviewCaller()) require.ErrorIs(t, err, ErrModuleRegistryStale, "the grant's own registry predates the committed revision") @@ -102,7 +102,7 @@ func TestTheFenceRefusesWhenTheCommittedRevisionCannotBeRead(t *testing.T) { // pays one revision read however many decisions it makes. func TestOneRequestReadsTheCommittedRevisionOnce(t *testing.T) { s := reviewService() - s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1", Queues: []string{"a"}}}, 3) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{"principal-1": {Prefix: "crm", Tenant: "org-1", Queues: []string{"a"}}}, 3, nil) store := &fakeEnvelopeStore{latest: 3} s.SetModuleEnvelopeStore(store) ctx := WithModuleEnvelopeRevisionMemo(context.Background()) @@ -123,7 +123,7 @@ func TestMintsAndRevisionConfirmationReadTheCommittedRevision(t *testing.T) { s := &Service{store: reviewControlPlaneStore{}} grant := parseTestEnvelope(t).Principals[testModulePrefix].Grant grant.Workload = ModuleWorkload{Namespace: "acme-prod", ServiceAccount: "reports", Container: "app"} - s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{ModulePrincipalID(testModulePrefix): grant}, 3) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{ModulePrincipalID(testModulePrefix): grant}, 3, nil) s.SetModuleEnvelopeStore(store) return s, bindTestModule(s, testModulePrefix) } @@ -649,7 +649,7 @@ func referenceExchange(t *testing.T) (*Service, *referenceExchangeStore, *fakeEn s.SetModuleAuthorityReads(reviewLiveAuthority{service: s}, nil) envelopes := &fakeEnvelopeStore{latest: 3} s.SetModuleEnvelopeStore(envelopes) - s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{ModulePrincipalID("worker"): worker}, 3) + s.InstallDerivedModulePrincipals(ModulePrincipalRegistry{ModulePrincipalID("worker"): worker}, 3, nil) return s, store, envelopes, ModuleCaller{PrincipalID: ModulePrincipalID("worker"), BoundOrg: testEnvelopeTenant}, ModulePrincipalRegistry{ModulePrincipalID("worker"): narrowed, ModulePrincipalID("source"): source} } @@ -666,7 +666,7 @@ func TestAReferenceExchangeReadsBothPrincipalsFromOneSnapshot(t *testing.T) { store.duringGet = func() { envelopes.latest = 4 - s.InstallDerivedModulePrincipals(newer, 4) + s.InstallDerivedModulePrincipals(newer, 4, nil) } _, err = s.AuthorizeDelegationReferenceExchange(WithModuleEnvelopeRevisionMemo(context.Background()), caller, store.delegation.ID, "run", false) require.ErrorIs(t, err, ErrSourceDelegationInvalid, "the delegator is read from the caller's snapshot, never a later install") diff --git a/module/services/accounts/code/pkg/business/module_upstream.go b/module/services/accounts/code/pkg/business/module_upstream.go new file mode 100644 index 000000000..9ace721e4 --- /dev/null +++ b/module/services/accounts/code/pkg/business/module_upstream.go @@ -0,0 +1,152 @@ +package business + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net" + "net/url" + "strings" + "time" +) + +// ErrModuleUpstreamRefused names an upstream a minting module reported that the +// host will not route to. +var ErrModuleUpstreamRefused = errors.New("module upstream refused") + +// RecordModuleUpstream writes the dynamic half of a declared module's +// registration: the origin its runtime reports at mint, once BindExecution has +// proven the caller is the workload the declared presence names (#952: "the +// lease, upstream and manifest on solution_registrations, reported by the +// runtime"). The static half — the declaration, its alias and target — is the +// delivery's and is never touched here; a module whose declaration is not +// applied and live has no registration to complete. +// +// The gateway reaches the recorded origin only over TLS to the SVID the applied +// presence names, so this origin cannot redirect the module's traffic to a +// workload that is not the module. It is still bounded like every registered +// upstream was: https, host and port only, and a composition-local host. +func (s *Service) RecordModuleUpstream(ctx context.Context, authority ModuleWorkContextAuthority, upstream, manifest string) error { + upstream, manifest = strings.TrimSpace(upstream), strings.TrimSpace(manifest) + if upstream == "" && manifest == "" { + return nil + } + origin := "" + if upstream != "" { + var err error + if origin, err = moduleUpstreamOrigin(upstream); err != nil { + return err + } + } + if manifest != "" && !json.Valid([]byte(manifest)) { + return fmt.Errorf("%w: the reported manifest is not JSON", ErrModuleUpstreamRefused) + } + grant, registered, err := s.currentModuleGrantFromSnapshot(ctx, authority.snapshot, authority.PrincipalID) + if err != nil { + return err + } + if !registered || grant.PresenceBinding == "" { + return fmt.Errorf("%w: %s has no delivered presence to complete", ErrModuleUpstreamRefused, authority.PrincipalID) + } + now := time.Now().UTC() + return s.store.WithControlPlane(ctx, func(ctx context.Context) error { + binding, err := s.store.GetSolutionHostBindingForUpdate(ctx, grant.PresenceBinding) + if err != nil { + return err + } + if binding == nil || binding.Applied == nil || binding.Applied.Removed || binding.Applied.SolutionID == "" { + return fmt.Errorf("%w: presence %q is not applied", ErrModuleUpstreamRefused, grant.PresenceBinding) + } + solutionID := binding.Applied.SolutionID + current, err := s.store.GetSolutionRegistrationForUpdate(ctx, solutionID) + if err != nil { + return err + } + if current == nil || current.TombstonedAt != nil || current.Declared == nil || current.Declared.BindingID != grant.PresenceBinding { + return fmt.Errorf("%w: %q is not declared by presence %q", ErrModuleUpstreamRefused, solutionID, grant.PresenceBinding) + } + backendSame := origin == "" || (current.Backend != nil && current.Backend.Upstream == origin && current.Backend.ServiceAlias == solutionID) + frontendSame := manifest == "" || (current.Frontend != nil && current.Frontend.Manifest == manifest) + if backendSame && frontendSame { + return nil + } + revision, err := s.store.NextSolutionRegistryRevision(ctx) + if err != nil { + return err + } + next := *current + next.Revision = revision + next.UpdatedAt = now + if !backendSame { + half := SolutionBackendHalf{Revision: revision, Upstream: origin, ServiceAlias: solutionID} + if current.Backend != nil { + half.ContractVersion = current.Backend.ContractVersion + } + next.Backend = &half + } + if !frontendSame { + half := SolutionFrontendHalf{Revision: revision, Manifest: manifest} + if current.Frontend != nil { + half.ContractVersion = current.Frontend.ContractVersion + } + next.Frontend = &half + } + return s.store.SaveSolutionRegistration(ctx, &next) + }) +} + +// ModuleReportUpstream records the origin a workload reports with nothing but +// its proven identity: the projected token and the build it runs are bound +// exactly as for a mint, and no installation is required — a solution nobody +// has installed yet must still be reachable for its install step, which is +// delivered to the origin recorded here. +func (s *Service) ModuleReportUpstream(ctx context.Context, prefix, serviceAccountToken, upstream, manifest string) error { + if strings.TrimSpace(upstream) == "" && strings.TrimSpace(manifest) == "" { + return nil + } + ctx, releaseAuthority, fenceErr := s.BeginModuleAuthorityUse(ctx) + if fenceErr != nil { + return fenceErr + } + defer releaseAuthority() + snapshot := s.moduleRegistrySnapshot() + identity, _, err := s.authorizeModuleIdentity(ctx, snapshot, prefix, serviceAccountToken) + if err != nil { + return err + } + return s.RecordModuleUpstream(ctx, identity, upstream, manifest) +} + +// moduleUpstreamOrigin admits an https origin with nothing but a host and a +// port, on a host that is composition-local: loopback or private, a bare +// service name, or a cluster-local name. Anything else is refused by name. +func moduleUpstreamOrigin(raw string) (string, error) { + parsed, err := url.Parse(raw) + if err != nil || !parsed.IsAbs() || parsed.Opaque != "" { + return "", fmt.Errorf("%w: %q is not an absolute URL", ErrModuleUpstreamRefused, raw) + } + if parsed.Scheme != "https" { + return "", fmt.Errorf("%w: %q is not https", ErrModuleUpstreamRefused, raw) + } + if parsed.User != nil || (parsed.Path != "" && parsed.Path != "/") || parsed.RawQuery != "" || parsed.ForceQuery || parsed.Fragment != "" { + return "", fmt.Errorf("%w: %q carries more than an origin", ErrModuleUpstreamRefused, raw) + } + host := strings.ToLower(parsed.Hostname()) + if host == "" || parsed.Port() == "" { + return "", fmt.Errorf("%w: %q names no host and port", ErrModuleUpstreamRefused, raw) + } + if ip := net.ParseIP(host); ip != nil { + if !ip.IsLoopback() && !ip.IsPrivate() { + return "", fmt.Errorf("%w: %q is not composition-local", ErrModuleUpstreamRefused, raw) + } + } else if strings.Contains(host, ".") { + labels := strings.Split(strings.TrimSuffix(host, "."), ".") + clusterLocal := (len(labels) == 3 && labels[2] == "svc") || + (len(labels) == 5 && labels[2] == "svc" && labels[3] == "cluster" && labels[4] == "local") + if !clusterLocal { + return "", fmt.Errorf("%w: %q is not composition-local", ErrModuleUpstreamRefused, raw) + } + } + return "https://" + net.JoinHostPort(host, parsed.Port()), nil +} diff --git a/module/services/accounts/code/pkg/business/module_upstream_test.go b/module/services/accounts/code/pkg/business/module_upstream_test.go new file mode 100644 index 000000000..f423170fd --- /dev/null +++ b/module/services/accounts/code/pkg/business/module_upstream_test.go @@ -0,0 +1,88 @@ +package business + +import ( + "errors" + "testing" + + "github.com/stretchr/testify/require" +) + +// The origin a minting module reports is where the gateway will dial it, so it +// is admitted only as exactly an https host and port on a composition-local +// host — and normalised, so the registry never holds two spellings of one. +func TestAReportedOriginIsAnHTTPSCompositionLocalHostAndPort(t *testing.T) { + for raw, want := range map[string]string{ + "https://backend.cell-documents.svc.cluster.local:8080": "https://backend.cell-documents.svc.cluster.local:8080", + "https://backend.cell-documents.svc:8080": "https://backend.cell-documents.svc:8080", + "https://BACKEND.cell-documents.svc.cluster.local:8080/": "https://backend.cell-documents.svc.cluster.local:8080", + "https://backend:8080": "https://backend:8080", + "https://localhost:22631": "https://localhost:22631", + "https://127.0.0.1:22631": "https://127.0.0.1:22631", + "https://10.64.3.7:8080": "https://10.64.3.7:8080", + "https://[fd00::12]:8080": "https://[fd00::12]:8080", + } { + t.Run(raw, func(t *testing.T) { + got, err := moduleUpstreamOrigin(raw) + require.NoError(t, err) + require.Equal(t, want, got) + }) + } +} + +func TestAReportedOriginThatIsNotOneIsRefused(t *testing.T) { + for name, raw := range map[string]string{ + "plaintext": "http://backend.cell-documents.svc.cluster.local:8080", + "no port": "https://backend.cell-documents.svc.cluster.local", + "a path": "https://backend.cell-documents.svc.cluster.local:8080/api", + "a query": "https://backend.cell-documents.svc.cluster.local:8080?x=1", + "a bare query mark": "https://backend.cell-documents.svc.cluster.local:8080?", + "a fragment": "https://backend.cell-documents.svc.cluster.local:8080#x", + "userinfo": "https://ops@backend.cell-documents.svc.cluster.local:8080", + "a public name": "https://example.com:443", + "a public address": "https://8.8.8.8:443", + "a name that only ends svc": "https://a.b.c.svc:8080", + "another cluster's suffix": "https://backend.cell-documents.svc.cluster.example:8080", + "opaque": "https:backend:8080", + "not a URL": "backend:8080", + "relative": "//backend:8080", + } { + t.Run(name, func(t *testing.T) { + got, err := moduleUpstreamOrigin(raw) + require.Empty(t, got) + require.True(t, errors.Is(err, ErrModuleUpstreamRefused), "%q: %v", raw, err) + }) + } +} + +// Only a pure grant skips the policy-log witness, so this comparison is the +// security boundary of an envelope write: it must recognise "the same, plus +// builds and whole principals" and nothing else. +func TestAnEnvelopeWriteIsAPureGrantOnlyWhenNothingGrantedChanges(t *testing.T) { + const before = `{"revision":3,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}}}}` + for _, tc := range []struct { + name string + after string + adds bool + }{ + {"the same document at the next revision", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}}}}`, true}, + {"key order and whitespace differ", `{"principals":{"crm":{"grant":{"queues":["q1"],"tenant":""},"bindings":[{"revision":9007199254740992,"id":"u1"}]}}, "approved_builds":["sha256:a"], "revision":4}`, true}, + {"a build added", `{"revision":4,"approved_builds":["sha256:a","sha256:b"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}}}}`, true}, + {"a whole principal added", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}},"docs":{"grant":{"tenant":"t1"}}}}`, true}, + + {"a build removed", `{"revision":4,"approved_builds":[],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}}}}`, false}, + {"a principal removed", `{"revision":4,"approved_builds":["sha256:a"],"principals":{}}`, false}, + {"the principals key removed", `{"revision":4,"approved_builds":["sha256:a"]}`, false}, + {"a queue added inside a granted principal", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1","q2"]}}}}`, false}, + {"the shared tenancy pinned to one organization", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"t1","queues":["q1"]}}}}`, false}, + {"a restricting key that was absent appears", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992,"queue":"q1"}],"grant":{"tenant":"","queues":["q1"]}}}}`, false}, + {"a unit revision moves past what a float can tell apart", `{"revision":4,"approved_builds":["sha256:a"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740993}],"grant":{"tenant":"","queues":["q1"]}}}}`, false}, + {"an unknown top-level key appears", `{"revision":4,"approved_builds":["sha256:a"],"deny":["crm"],"principals":{"crm":{"bindings":[{"id":"u1","revision":9007199254740992}],"grant":{"tenant":"","queues":["q1"]}}}}`, false}, + {"not a document", `[]`, false}, + {"not JSON", `{"revision":4`, false}, + } { + t.Run(tc.name, func(t *testing.T) { + require.Equal(t, tc.adds, moduleEnvelopeOnlyAdds([]byte(before), []byte(tc.after))) + }) + } + require.False(t, moduleEnvelopeOnlyAdds([]byte(`not json`), []byte(before)), "an envelope in force that cannot be read is never added to") +} diff --git a/module/services/accounts/code/pkg/business/policy_log_production_path_test.go b/module/services/accounts/code/pkg/business/policy_log_production_path_test.go index 5b1688ee0..b4bfff0ff 100644 --- a/module/services/accounts/code/pkg/business/policy_log_production_path_test.go +++ b/module/services/accounts/code/pkg/business/policy_log_production_path_test.go @@ -210,6 +210,7 @@ func TestARealUninstallAppendsAndThenCommitsItsReceipt(t *testing.T) { AllowedScopes: []string{"doc"}, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) // An install is not a narrowing, so it must not have been offered to the // log. A protocol that appended on every write would make "was authority @@ -346,6 +347,7 @@ func TestANarrowingTheLogRefusesChangesNothing(t *testing.T) { AllowedScopes: []string{"doc"}, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) testPolicyLog.mu.Lock() testPolicyLog.failWith = errPolicyLogDown diff --git a/module/services/accounts/code/pkg/business/service.go b/module/services/accounts/code/pkg/business/service.go index 097002c38..c5ac4a1c5 100644 --- a/module/services/accounts/code/pkg/business/service.go +++ b/module/services/accounts/code/pkg/business/service.go @@ -304,6 +304,10 @@ func (s *Service) moduleRegistryFenced() bool { type installedModuleRegistry struct { registry ModulePrincipalRegistry envelopeRevision uint64 + // delivered is the size of the delivery inbox this registry was derived + // from, nil when it was not derived from a counted read of it. See + // deliveredSince. + delivered *uint64 } // SetModuleCapabilities wires the module-facing capability surface (issue #463): @@ -352,9 +356,13 @@ func (s *Service) SetModulePrincipals(registry ModulePrincipalRegistry) { // InstallDerivedModulePrincipals publishes a registry derived from one envelope // revision. Callers must derive the entries from that revision before publishing; // this operation does not infer or advance a revision from the current database. -func (s *Service) InstallDerivedModulePrincipals(registry ModulePrincipalRegistry, envelopeRevision uint64) { +// +// delivered is the size of the delivery inbox the registry was derived from, +// read in the same snapshot as its inputs, or nil when it was not derived from +// a counted read of the inbox. See deliveredSince. +func (s *Service) InstallDerivedModulePrincipals(registry ModulePrincipalRegistry, envelopeRevision uint64, delivered *uint64) { s.observeModuleEnvelopeRevision(envelopeRevision) - s.modulePrincipals.Store(&installedModuleRegistry{registry: maps.Clone(registry), envelopeRevision: envelopeRevision}) + s.modulePrincipals.Store(&installedModuleRegistry{registry: maps.Clone(registry), envelopeRevision: envelopeRevision, delivered: delivered}) } // observeModuleEnvelopeRevision records that a revision is committed. It only diff --git a/module/services/accounts/code/pkg/business/solution_delivery.go b/module/services/accounts/code/pkg/business/solution_delivery.go index be390b0c4..4ca97719a 100644 --- a/module/services/accounts/code/pkg/business/solution_delivery.go +++ b/module/services/accounts/code/pkg/business/solution_delivery.go @@ -146,6 +146,20 @@ type SolutionDeliveryStore interface { ListDeliveredDocuments(ctx context.Context, kind SolutionDeliveryKind) ([]*SolutionDeliveryRecord, error) } +// SolutionDeliveryCounter is the inbox's size, where the store can answer it. +// +// Optional, and asked for by assertion: a store that cannot count leaves the +// registry with no marker, and a mint then answers exactly as it did before +// the marker existed. +type SolutionDeliveryCounter interface { + // CountDeliveredDocuments is the number of delivered documents of every + // kind. The inbox is append-only, so it never decreases. + CountDeliveredDocuments(ctx context.Context) (uint64, error) + // CountUnappliedPresence is the number of delivered presence documents + // newer than what their binding has applied and not refused. + CountUnappliedPresence(ctx context.Context) (uint64, error) +} + // SolutionDeliveryCarrierAuthorizer decides whether the CALLER may deliver this // kind of document. // diff --git a/module/services/accounts/code/pkg/business/solution_delivery_activation.go b/module/services/accounts/code/pkg/business/solution_delivery_activation.go index bdfdbccbf..f0b0d5c48 100644 --- a/module/services/accounts/code/pkg/business/solution_delivery_activation.go +++ b/module/services/accounts/code/pkg/business/solution_delivery_activation.go @@ -381,6 +381,9 @@ type registryInputs struct { authorityHistory []*SolutionDeliveryRecord newestAuthority []*SolutionDeliveryRecord newestPresence []*SolutionDeliveryRecord + // delivered is the inbox's size in this same snapshot, nil where the store + // cannot count: the marker a registry derived from these inputs carries. + delivered *uint64 } func (a *SolutionAuthorityActivation) readRegistryInputs(ctx context.Context) (*registryInputs, error) { @@ -403,6 +406,13 @@ func (a *SolutionAuthorityActivation) readRegistryInputs(ctx context.Context) (* if inputs.newestPresence, err = service.deliveryStore.ListNewestDeliveredDocuments(ctx, SolutionDeliveryPresence); err != nil { return fmt.Errorf("the newest presence: %w", err) } + if counter, counts := service.deliveryStore.(SolutionDeliveryCounter); counts { + delivered, err := counter.CountDeliveredDocuments(ctx) + if err != nil { + return fmt.Errorf("the inbox's size: %w", err) + } + inputs.delivered = &delivered + } return nil }); err != nil { return nil, fmt.Errorf("read the delivered authority's inputs in one snapshot: %w", err) diff --git a/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go b/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go index 89d026716..03c14124e 100644 --- a/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go +++ b/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go @@ -72,19 +72,17 @@ func TestDeploymentTopologyIsDeterministicAndCurrent(t *testing.T) { require.Equal(t, string(readFixture(t, "testdata/mesh-policy.golden.yaml")), string(first.MeshPolicy), "run: go generate ./pkg/cataloggen") require.Len(t, first.Catalog.GetServices(), 9) - require.Len(t, first.Catalog.GetInterfaceEndpoints(), 6) + require.Len(t, first.Catalog.GetInterfaceEndpoints(), 7) require.Len(t, first.Catalog.GetPublicEgress(), 5) endpointCount, dependencyCount := 0, 0 for _, service := range first.Catalog.GetServices() { endpointCount += len(service.GetEndpoints()) dependencyCount += len(service.GetDependencies()) } - require.Equal(t, 14, endpointCount) + require.Equal(t, 15, endpointCount) require.Equal(t, 8, dependencyCount) - // The accounts REST surface is reachable only through the gateway; the - // gateway's REST surface is module-visible because composed modules and - // solutions use it (Work Context minting, solution registration) and the - // interface exports it to them. + // Accounts REST stays private. Gateway REST is exported for module mints; + // solutions resolve its separately exported workload listener. restVisibility := map[string]catalogv1.EndpointVisibility{ "accounts": catalogv1.EndpointVisibility_ENDPOINT_VISIBILITY_PRIVATE, "auth-gateway": catalogv1.EndpointVisibility_ENDPOINT_VISIBILITY_MODULE, @@ -739,8 +737,9 @@ func TestDeploymentTopologyAuthorityPortIsAllocatedAtRender(t *testing.T) { require.NoError(t, err) for _, service := range artifacts.Catalog.GetServices() { for _, endpoint := range service.GetEndpoints() { - if service.GetName() == "accounts" && endpoint.GetName() == "authority" { - require.Zero(t, endpoint.GetPort(), "authority's pod port is allocated at render") + if (service.GetName() == "accounts" && endpoint.GetName() == "authority") || + (service.GetName() == "auth-gateway" && (endpoint.GetName() == "rest" || endpoint.GetName() == "workload")) { + require.Zero(t, endpoint.GetPort(), "%s/%s is allocated at render", service.GetName(), endpoint.GetName()) } else { require.NotZero(t, endpoint.GetPort(), "%s/%s declares its pod port", service.GetName(), endpoint.GetName()) } @@ -773,7 +772,7 @@ func TestDeploymentTopologyPodPortIsTheAllocationUnlessDeclared(t *testing.T) { } for endpoint, want := range map[string]uint32{ "accounts/connect": 8081, "accounts/grpc": 9090, "accounts/rest": 8080, "accounts/authority": 0, - "auth-gateway/grpc": 9090, "auth-gateway/rest": 8080, "telemetry/grpc": 9090, + "auth-gateway/grpc": 9090, "auth-gateway/rest": 0, "auth-gateway/workload": 0, "telemetry/grpc": 9090, "cache/read": 6379, "cache/write": 6379, "store/tcp": 5432, "vault/http": 8200, "frontend/http": 3000, "marketing/http": 3000, } { diff --git a/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml b/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml index 3acdbfc7b..14cdf6b35 100644 --- a/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml +++ b/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml @@ -144,8 +144,6 @@ spec: matchLabels: app: frontend ports: - - protocol: TCP - port: 8080 --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy @@ -337,8 +335,6 @@ spec: matchLabels: app: auth-gateway ports: - - protocol: TCP - port: 8080 --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy diff --git a/module/services/accounts/code/pkg/eventcatalog/catalog_gen.go b/module/services/accounts/code/pkg/eventcatalog/catalog_gen.go index 68c78da8f..5676a1ab1 100644 --- a/module/services/accounts/code/pkg/eventcatalog/catalog_gen.go +++ b/module/services/accounts/code/pkg/eventcatalog/catalog_gen.go @@ -127,6 +127,7 @@ var published = [...]PublishedEvent{ {Type: "saas.installation.created", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, {Type: "saas.installation.narrowed", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, {Type: "saas.installation.ownership_transferred", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, + {Type: "saas.installation.retried", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, {Type: "saas.installation.revoked", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, {Type: "saas.invitation.accepted", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, {Type: "saas.invitation.created", Namespace: "saas", Schema: "saas/events/v1/events.proto#EventEnvelope", Major: 1, Visibility: "external", Partition: "", Retention: "30d"}, diff --git a/module/services/accounts/code/pkg/gen/saas/accounts/v1/accountsv1connect/installations.connect.go b/module/services/accounts/code/pkg/gen/saas/accounts/v1/accountsv1connect/installations.connect.go index 5d5423cb1..7e6285a5f 100644 --- a/module/services/accounts/code/pkg/gen/saas/accounts/v1/accountsv1connect/installations.connect.go +++ b/module/services/accounts/code/pkg/gen/saas/accounts/v1/accountsv1connect/installations.connect.go @@ -44,6 +44,9 @@ const ( // InstallationServiceNarrowInstallationProcedure is the fully-qualified name of the // InstallationService's NarrowInstallation RPC. InstallationServiceNarrowInstallationProcedure = "/saas.accounts.v1.InstallationService/NarrowInstallation" + // InstallationServiceRetryInstallationProcedure is the fully-qualified name of the + // InstallationService's RetryInstallation RPC. + InstallationServiceRetryInstallationProcedure = "/saas.accounts.v1.InstallationService/RetryInstallation" // InstallationServiceTransferInstallationOwnershipProcedure is the fully-qualified name of the // InstallationService's TransferInstallationOwnership RPC. InstallationServiceTransferInstallationOwnershipProcedure = "/saas.accounts.v1.InstallationService/TransferInstallationOwnership" @@ -82,6 +85,9 @@ type InstallationServiceClient interface { // Narrowing is witnessed in the policy log before commit, then atomically // bumps the installation revision, cancels its org's work and records audit. NarrowInstallation(context.Context, *connect.Request[v1.NarrowInstallationRequest]) (*connect.Response[emptypb.Empty], error) + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + RetryInstallation(context.Context, *connect.Request[v1.RetryInstallationRequest]) (*connect.Response[v1.Installation], error) // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, @@ -182,6 +188,12 @@ func NewInstallationServiceClient(httpClient connect.HTTPClient, baseURL string, connect.WithSchema(installationServiceMethods.ByName("NarrowInstallation")), connect.WithClientOptions(opts...), ), + retryInstallation: connect.NewClient[v1.RetryInstallationRequest, v1.Installation]( + httpClient, + baseURL+InstallationServiceRetryInstallationProcedure, + connect.WithSchema(installationServiceMethods.ByName("RetryInstallation")), + connect.WithClientOptions(opts...), + ), transferInstallationOwnership: connect.NewClient[v1.TransferInstallationOwnershipRequest, v1.Installation]( httpClient, baseURL+InstallationServiceTransferInstallationOwnershipProcedure, @@ -232,6 +244,7 @@ type installationServiceClient struct { installSolution *connect.Client[v1.InstallSolutionRequest, v1.Installation] uninstallSolution *connect.Client[v1.UninstallSolutionRequest, emptypb.Empty] narrowInstallation *connect.Client[v1.NarrowInstallationRequest, emptypb.Empty] + retryInstallation *connect.Client[v1.RetryInstallationRequest, v1.Installation] transferInstallationOwnership *connect.Client[v1.TransferInstallationOwnershipRequest, v1.Installation] getInstallation *connect.Client[v1.GetInstallationRequest, v1.GetInstallationResponse] listInstallations *connect.Client[v1.ListInstallationsRequest, v1.ListInstallationsResponse] @@ -256,6 +269,11 @@ func (c *installationServiceClient) NarrowInstallation(ctx context.Context, req return c.narrowInstallation.CallUnary(ctx, req) } +// RetryInstallation calls saas.accounts.v1.InstallationService.RetryInstallation. +func (c *installationServiceClient) RetryInstallation(ctx context.Context, req *connect.Request[v1.RetryInstallationRequest]) (*connect.Response[v1.Installation], error) { + return c.retryInstallation.CallUnary(ctx, req) +} + // TransferInstallationOwnership calls // saas.accounts.v1.InstallationService.TransferInstallationOwnership. func (c *installationServiceClient) TransferInstallationOwnership(ctx context.Context, req *connect.Request[v1.TransferInstallationOwnershipRequest]) (*connect.Response[v1.Installation], error) { @@ -309,6 +327,9 @@ type InstallationServiceHandler interface { // Narrowing is witnessed in the policy log before commit, then atomically // bumps the installation revision, cancels its org's work and records audit. NarrowInstallation(context.Context, *connect.Request[v1.NarrowInstallationRequest]) (*connect.Response[emptypb.Empty], error) + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + RetryInstallation(context.Context, *connect.Request[v1.RetryInstallationRequest]) (*connect.Response[v1.Installation], error) // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, @@ -405,6 +426,12 @@ func NewInstallationServiceHandler(svc InstallationServiceHandler, opts ...conne connect.WithSchema(installationServiceMethods.ByName("NarrowInstallation")), connect.WithHandlerOptions(opts...), ) + installationServiceRetryInstallationHandler := connect.NewUnaryHandler( + InstallationServiceRetryInstallationProcedure, + svc.RetryInstallation, + connect.WithSchema(installationServiceMethods.ByName("RetryInstallation")), + connect.WithHandlerOptions(opts...), + ) installationServiceTransferInstallationOwnershipHandler := connect.NewUnaryHandler( InstallationServiceTransferInstallationOwnershipProcedure, svc.TransferInstallationOwnership, @@ -455,6 +482,8 @@ func NewInstallationServiceHandler(svc InstallationServiceHandler, opts ...conne installationServiceUninstallSolutionHandler.ServeHTTP(w, r) case InstallationServiceNarrowInstallationProcedure: installationServiceNarrowInstallationHandler.ServeHTTP(w, r) + case InstallationServiceRetryInstallationProcedure: + installationServiceRetryInstallationHandler.ServeHTTP(w, r) case InstallationServiceTransferInstallationOwnershipProcedure: installationServiceTransferInstallationOwnershipHandler.ServeHTTP(w, r) case InstallationServiceGetInstallationProcedure: @@ -490,6 +519,10 @@ func (UnimplementedInstallationServiceHandler) NarrowInstallation(context.Contex return nil, connect.NewError(connect.CodeUnimplemented, errors.New("saas.accounts.v1.InstallationService.NarrowInstallation is not implemented")) } +func (UnimplementedInstallationServiceHandler) RetryInstallation(context.Context, *connect.Request[v1.RetryInstallationRequest]) (*connect.Response[v1.Installation], error) { + return nil, connect.NewError(connect.CodeUnimplemented, errors.New("saas.accounts.v1.InstallationService.RetryInstallation is not implemented")) +} + func (UnimplementedInstallationServiceHandler) TransferInstallationOwnership(context.Context, *connect.Request[v1.TransferInstallationOwnershipRequest]) (*connect.Response[v1.Installation], error) { return nil, connect.NewError(connect.CodeUnimplemented, errors.New("saas.accounts.v1.InstallationService.TransferInstallationOwnership is not implemented")) } diff --git a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.go b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.go index 21d178a5c..9069c690d 100644 --- a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.go +++ b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.go @@ -36,6 +36,12 @@ const ( InstallationStatus_INSTALLATION_STATUS_UNSPECIFIED InstallationStatus = 0 InstallationStatus_INSTALLATION_STATUS_ACTIVE InstallationStatus = 1 InstallationStatus_INSTALLATION_STATUS_REVOKED InstallationStatus = 2 + // Installed, and waiting for the solution to acknowledge its standard install + // step. Nothing can use a pending installation. + InstallationStatus_INSTALLATION_STATUS_PENDING InstallationStatus = 3 + // The solution refused its install step, or never answered within the + // attempt budget. Nothing can use it; an administrator may retry it. + InstallationStatus_INSTALLATION_STATUS_FAILED InstallationStatus = 4 ) // Enum value maps for InstallationStatus. @@ -44,11 +50,15 @@ var ( 0: "INSTALLATION_STATUS_UNSPECIFIED", 1: "INSTALLATION_STATUS_ACTIVE", 2: "INSTALLATION_STATUS_REVOKED", + 3: "INSTALLATION_STATUS_PENDING", + 4: "INSTALLATION_STATUS_FAILED", } InstallationStatus_value = map[string]int32{ "INSTALLATION_STATUS_UNSPECIFIED": 0, "INSTALLATION_STATUS_ACTIVE": 1, "INSTALLATION_STATUS_REVOKED": 2, + "INSTALLATION_STATUS_PENDING": 3, + "INSTALLATION_STATUS_FAILED": 4, } ) @@ -176,6 +186,9 @@ type Installation struct { // uninstalling: the installation-identity cutover, or a tenancy narrowing that // withdrew this organisation. Empty on an active row. RevokedReason string `protobuf:"bytes,12,opt,name=revoked_reason,json=revokedReason,proto3" json:"revoked_reason,omitempty"` + // Why the install step failed: the solution's bounded refusal reason, or a + // fixed reason when it never answered. Empty unless status is FAILED. + FailureReason string `protobuf:"bytes,13,opt,name=failure_reason,json=failureReason,proto3" json:"failure_reason,omitempty"` unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } @@ -287,6 +300,13 @@ func (x *Installation) GetRevokedReason() string { return "" } +func (x *Installation) GetFailureReason() string { + if x != nil { + return x.FailureReason + } + return "" +} + type InstallSolutionRequest struct { state protoimpl.MessageState `protogen:"open.v1"` OrgId string `protobuf:"bytes,1,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"` @@ -512,6 +532,58 @@ func (x *NarrowInstallationRequest) GetScope() string { return "" } +type RetryInstallationRequest struct { + state protoimpl.MessageState `protogen:"open.v1"` + OrgId string `protobuf:"bytes,1,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"` + InstallationId string `protobuf:"bytes,2,opt,name=installation_id,json=installationId,proto3" json:"installation_id,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache +} + +func (x *RetryInstallationRequest) Reset() { + *x = RetryInstallationRequest{} + mi := &file_saas_accounts_v1_installations_proto_msgTypes[3] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) +} + +func (x *RetryInstallationRequest) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*RetryInstallationRequest) ProtoMessage() {} + +func (x *RetryInstallationRequest) ProtoReflect() protoreflect.Message { + mi := &file_saas_accounts_v1_installations_proto_msgTypes[3] + if x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use RetryInstallationRequest.ProtoReflect.Descriptor instead. +func (*RetryInstallationRequest) Descriptor() ([]byte, []int) { + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{3} +} + +func (x *RetryInstallationRequest) GetOrgId() string { + if x != nil { + return x.OrgId + } + return "" +} + +func (x *RetryInstallationRequest) GetInstallationId() string { + if x != nil { + return x.InstallationId + } + return "" +} + type UninstallSolutionRequest struct { state protoimpl.MessageState `protogen:"open.v1"` OrgId string `protobuf:"bytes,1,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"` @@ -522,7 +594,7 @@ type UninstallSolutionRequest struct { func (x *UninstallSolutionRequest) Reset() { *x = UninstallSolutionRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[3] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[4] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -534,7 +606,7 @@ func (x *UninstallSolutionRequest) String() string { func (*UninstallSolutionRequest) ProtoMessage() {} func (x *UninstallSolutionRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[3] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[4] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -547,7 +619,7 @@ func (x *UninstallSolutionRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use UninstallSolutionRequest.ProtoReflect.Descriptor instead. func (*UninstallSolutionRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{3} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{4} } func (x *UninstallSolutionRequest) GetOrgId() string { @@ -578,7 +650,7 @@ type TransferInstallationOwnershipRequest struct { func (x *TransferInstallationOwnershipRequest) Reset() { *x = TransferInstallationOwnershipRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[4] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[5] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -590,7 +662,7 @@ func (x *TransferInstallationOwnershipRequest) String() string { func (*TransferInstallationOwnershipRequest) ProtoMessage() {} func (x *TransferInstallationOwnershipRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[4] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[5] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -603,7 +675,7 @@ func (x *TransferInstallationOwnershipRequest) ProtoReflect() protoreflect.Messa // Deprecated: Use TransferInstallationOwnershipRequest.ProtoReflect.Descriptor instead. func (*TransferInstallationOwnershipRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{4} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{5} } func (x *TransferInstallationOwnershipRequest) GetOrgId() string { @@ -644,7 +716,7 @@ type GetInstallationRequest struct { func (x *GetInstallationRequest) Reset() { *x = GetInstallationRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[5] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[6] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -656,7 +728,7 @@ func (x *GetInstallationRequest) String() string { func (*GetInstallationRequest) ProtoMessage() {} func (x *GetInstallationRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[5] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[6] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -669,7 +741,7 @@ func (x *GetInstallationRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use GetInstallationRequest.ProtoReflect.Descriptor instead. func (*GetInstallationRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{5} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{6} } func (x *GetInstallationRequest) GetOrgId() string { @@ -697,7 +769,7 @@ type GetInstallationResponse struct { func (x *GetInstallationResponse) Reset() { *x = GetInstallationResponse{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[6] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[7] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -709,7 +781,7 @@ func (x *GetInstallationResponse) String() string { func (*GetInstallationResponse) ProtoMessage() {} func (x *GetInstallationResponse) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[6] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[7] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -722,7 +794,7 @@ func (x *GetInstallationResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use GetInstallationResponse.ProtoReflect.Descriptor instead. func (*GetInstallationResponse) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{6} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{7} } func (x *GetInstallationResponse) GetInstallation() *Installation { @@ -761,7 +833,7 @@ type ListInstallationsRequest struct { func (x *ListInstallationsRequest) Reset() { *x = ListInstallationsRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[7] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[8] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -773,7 +845,7 @@ func (x *ListInstallationsRequest) String() string { func (*ListInstallationsRequest) ProtoMessage() {} func (x *ListInstallationsRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[7] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[8] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -786,7 +858,7 @@ func (x *ListInstallationsRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use ListInstallationsRequest.ProtoReflect.Descriptor instead. func (*ListInstallationsRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{7} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{8} } func (x *ListInstallationsRequest) GetOrgId() string { @@ -831,7 +903,7 @@ type InstallationSummary struct { func (x *InstallationSummary) Reset() { *x = InstallationSummary{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[8] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[9] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -843,7 +915,7 @@ func (x *InstallationSummary) String() string { func (*InstallationSummary) ProtoMessage() {} func (x *InstallationSummary) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[8] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[9] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -856,7 +928,7 @@ func (x *InstallationSummary) ProtoReflect() protoreflect.Message { // Deprecated: Use InstallationSummary.ProtoReflect.Descriptor instead. func (*InstallationSummary) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{8} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{9} } func (x *InstallationSummary) GetInstallation() *Installation { @@ -885,7 +957,7 @@ type ListInstallationsResponse struct { func (x *ListInstallationsResponse) Reset() { *x = ListInstallationsResponse{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[9] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[10] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -897,7 +969,7 @@ func (x *ListInstallationsResponse) String() string { func (*ListInstallationsResponse) ProtoMessage() {} func (x *ListInstallationsResponse) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[9] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[10] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -910,7 +982,7 @@ func (x *ListInstallationsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ListInstallationsResponse.ProtoReflect.Descriptor instead. func (*ListInstallationsResponse) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{9} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{10} } func (x *ListInstallationsResponse) GetInstallations() []*InstallationSummary { @@ -940,7 +1012,7 @@ type ListOrganizationInstallationsRequest struct { func (x *ListOrganizationInstallationsRequest) Reset() { *x = ListOrganizationInstallationsRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[10] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[11] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -952,7 +1024,7 @@ func (x *ListOrganizationInstallationsRequest) String() string { func (*ListOrganizationInstallationsRequest) ProtoMessage() {} func (x *ListOrganizationInstallationsRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[10] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[11] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -965,7 +1037,7 @@ func (x *ListOrganizationInstallationsRequest) ProtoReflect() protoreflect.Messa // Deprecated: Use ListOrganizationInstallationsRequest.ProtoReflect.Descriptor instead. func (*ListOrganizationInstallationsRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{10} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{11} } func (x *ListOrganizationInstallationsRequest) GetOrgId() string { @@ -1017,7 +1089,7 @@ type OrganizationInstallation struct { func (x *OrganizationInstallation) Reset() { *x = OrganizationInstallation{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[11] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[12] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1029,7 +1101,7 @@ func (x *OrganizationInstallation) String() string { func (*OrganizationInstallation) ProtoMessage() {} func (x *OrganizationInstallation) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[11] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[12] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1042,7 +1114,7 @@ func (x *OrganizationInstallation) ProtoReflect() protoreflect.Message { // Deprecated: Use OrganizationInstallation.ProtoReflect.Descriptor instead. func (*OrganizationInstallation) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{11} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{12} } func (x *OrganizationInstallation) GetInstallation() *Installation { @@ -1098,7 +1170,7 @@ type ListOrganizationInstallationsResponse struct { func (x *ListOrganizationInstallationsResponse) Reset() { *x = ListOrganizationInstallationsResponse{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[12] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[13] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1110,7 +1182,7 @@ func (x *ListOrganizationInstallationsResponse) String() string { func (*ListOrganizationInstallationsResponse) ProtoMessage() {} func (x *ListOrganizationInstallationsResponse) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[12] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[13] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1123,7 +1195,7 @@ func (x *ListOrganizationInstallationsResponse) ProtoReflect() protoreflect.Mess // Deprecated: Use ListOrganizationInstallationsResponse.ProtoReflect.Descriptor instead. func (*ListOrganizationInstallationsResponse) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{12} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{13} } func (x *ListOrganizationInstallationsResponse) GetInstallations() []*OrganizationInstallation { @@ -1156,7 +1228,7 @@ type GrantInstallationToTeamRequest struct { func (x *GrantInstallationToTeamRequest) Reset() { *x = GrantInstallationToTeamRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[13] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[14] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1168,7 +1240,7 @@ func (x *GrantInstallationToTeamRequest) String() string { func (*GrantInstallationToTeamRequest) ProtoMessage() {} func (x *GrantInstallationToTeamRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[13] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[14] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1181,7 +1253,7 @@ func (x *GrantInstallationToTeamRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use GrantInstallationToTeamRequest.ProtoReflect.Descriptor instead. func (*GrantInstallationToTeamRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{13} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{14} } func (x *GrantInstallationToTeamRequest) GetOrgId() string { @@ -1221,7 +1293,7 @@ type GrantInstallationToTeamResponse struct { func (x *GrantInstallationToTeamResponse) Reset() { *x = GrantInstallationToTeamResponse{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[14] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[15] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1233,7 +1305,7 @@ func (x *GrantInstallationToTeamResponse) String() string { func (*GrantInstallationToTeamResponse) ProtoMessage() {} func (x *GrantInstallationToTeamResponse) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[14] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[15] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1246,7 +1318,7 @@ func (x *GrantInstallationToTeamResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use GrantInstallationToTeamResponse.ProtoReflect.Descriptor instead. func (*GrantInstallationToTeamResponse) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{14} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{15} } func (x *GrantInstallationToTeamResponse) GetGrant() *ScopeGrant { @@ -1270,7 +1342,7 @@ type RevokeInstallationFromTeamRequest struct { func (x *RevokeInstallationFromTeamRequest) Reset() { *x = RevokeInstallationFromTeamRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[15] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1282,7 +1354,7 @@ func (x *RevokeInstallationFromTeamRequest) String() string { func (*RevokeInstallationFromTeamRequest) ProtoMessage() {} func (x *RevokeInstallationFromTeamRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[15] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[16] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1295,7 +1367,7 @@ func (x *RevokeInstallationFromTeamRequest) ProtoReflect() protoreflect.Message // Deprecated: Use RevokeInstallationFromTeamRequest.ProtoReflect.Descriptor instead. func (*RevokeInstallationFromTeamRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{15} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{16} } func (x *RevokeInstallationFromTeamRequest) GetOrgId() string { @@ -1358,7 +1430,7 @@ type AvailableSolution struct { func (x *AvailableSolution) Reset() { *x = AvailableSolution{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[16] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[17] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1370,7 +1442,7 @@ func (x *AvailableSolution) String() string { func (*AvailableSolution) ProtoMessage() {} func (x *AvailableSolution) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[16] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[17] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1383,7 +1455,7 @@ func (x *AvailableSolution) ProtoReflect() protoreflect.Message { // Deprecated: Use AvailableSolution.ProtoReflect.Descriptor instead. func (*AvailableSolution) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{16} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{17} } func (x *AvailableSolution) GetTargetId() string { @@ -1462,7 +1534,7 @@ type ListAvailableSolutionsRequest struct { func (x *ListAvailableSolutionsRequest) Reset() { *x = ListAvailableSolutionsRequest{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[17] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[18] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1474,7 +1546,7 @@ func (x *ListAvailableSolutionsRequest) String() string { func (*ListAvailableSolutionsRequest) ProtoMessage() {} func (x *ListAvailableSolutionsRequest) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[17] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[18] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1487,7 +1559,7 @@ func (x *ListAvailableSolutionsRequest) ProtoReflect() protoreflect.Message { // Deprecated: Use ListAvailableSolutionsRequest.ProtoReflect.Descriptor instead. func (*ListAvailableSolutionsRequest) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{17} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{18} } func (x *ListAvailableSolutionsRequest) GetOrgId() string { @@ -1521,7 +1593,7 @@ type ListAvailableSolutionsResponse struct { func (x *ListAvailableSolutionsResponse) Reset() { *x = ListAvailableSolutionsResponse{} - mi := &file_saas_accounts_v1_installations_proto_msgTypes[18] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[19] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1533,7 +1605,7 @@ func (x *ListAvailableSolutionsResponse) String() string { func (*ListAvailableSolutionsResponse) ProtoMessage() {} func (x *ListAvailableSolutionsResponse) ProtoReflect() protoreflect.Message { - mi := &file_saas_accounts_v1_installations_proto_msgTypes[18] + mi := &file_saas_accounts_v1_installations_proto_msgTypes[19] if x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1546,7 +1618,7 @@ func (x *ListAvailableSolutionsResponse) ProtoReflect() protoreflect.Message { // Deprecated: Use ListAvailableSolutionsResponse.ProtoReflect.Descriptor instead. func (*ListAvailableSolutionsResponse) Descriptor() ([]byte, []int) { - return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{18} + return file_saas_accounts_v1_installations_proto_rawDescGZIP(), []int{19} } func (x *ListAvailableSolutionsResponse) GetSolutions() []*AvailableSolution { @@ -1567,7 +1639,7 @@ var File_saas_accounts_v1_installations_proto protoreflect.FileDescriptor const file_saas_accounts_v1_installations_proto_rawDesc = "" + "\n" + - "$saas/accounts/v1/installations.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1cgoogle/api/annotations.proto\x1a\x1bgoogle/protobuf/empty.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a$saas/accounts/v1/authorization.proto\x1a\x1csaas/policy/v1/options.proto\"\xe3\x04\n" + + "$saas/accounts/v1/installations.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1cgoogle/api/annotations.proto\x1a\x1bgoogle/protobuf/empty.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a$saas/accounts/v1/authorization.proto\x1a\x1csaas/policy/v1/options.proto\"\x94\x05\n" + "\fInstallation\x12\x18\n" + "\x02id\x18\x01 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x02id\x12\x1f\n" + "\x06org_id\x18\x02 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x05orgId\x126\n" + @@ -1582,7 +1654,8 @@ const file_saas_accounts_v1_installations_proto_rawDesc = "" + "revoked_at\x18\n" + " \x01(\v2\x1a.google.protobuf.TimestampH\x00R\trevokedAt\x88\x01\x01\x12(\n" + "\ttarget_id\x18\v \x01(\tB\v\xbaH\b\xd8\x01\x01r\x03\xb0\x01\x01R\btargetId\x12/\n" + - "\x0erevoked_reason\x18\f \x01(\tB\b\xbaH\x05r\x03\x18\x80\x04R\rrevokedReasonB\r\n" + + "\x0erevoked_reason\x18\f \x01(\tB\b\xbaH\x05r\x03\x18\x80\x04R\rrevokedReason\x12/\n" + + "\x0efailure_reason\x18\r \x01(\tB\b\xbaH\x05r\x03\x18\x80\x04R\rfailureReasonB\r\n" + "\v_revoked_atJ\x04\b\x04\x10\x05R\x13solution_identifier\"\xf6\x04\n" + "\x16InstallSolutionRequest\x12\x1f\n" + "\x06org_id\x18\x01 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x05orgId\x125\n" + @@ -1603,6 +1676,9 @@ const file_saas_accounts_v1_installations_proto_rawDesc = "" + "\x06org_id\x18\x01 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x05orgId\x121\n" + "\x0finstallation_id\x18\x02 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x0einstallationId\x12!\n" + "\x05scope\x18\x03 \x01(\tB\v\xbaH\br\x06\x10\x02(\x80\x80@R\x05scope\"n\n" + + "\x18RetryInstallationRequest\x12\x1f\n" + + "\x06org_id\x18\x01 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x05orgId\x121\n" + + "\x0finstallation_id\x18\x02 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x0einstallationId\"n\n" + "\x18UninstallSolutionRequest\x12\x1f\n" + "\x06org_id\x18\x01 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x05orgId\x121\n" + "\x0finstallation_id\x18\x02 \x01(\tB\b\xbaH\x05r\x03\xb0\x01\x01R\x0einstallationId\"\xff\x01\n" + @@ -1678,18 +1754,20 @@ const file_saas_accounts_v1_installations_proto_rawDesc = "" + "page_token\x18\x03 \x01(\tR\tpageToken\"\x8b\x01\n" + "\x1eListAvailableSolutionsResponse\x12A\n" + "\tsolutions\x18\x01 \x03(\v2#.saas.accounts.v1.AvailableSolutionR\tsolutions\x12&\n" + - "\x0fnext_page_token\x18\x02 \x01(\tR\rnextPageToken*z\n" + + "\x0fnext_page_token\x18\x02 \x01(\tR\rnextPageToken*\xbb\x01\n" + "\x12InstallationStatus\x12#\n" + "\x1fINSTALLATION_STATUS_UNSPECIFIED\x10\x00\x12\x1e\n" + "\x1aINSTALLATION_STATUS_ACTIVE\x10\x01\x12\x1f\n" + - "\x1bINSTALLATION_STATUS_REVOKED\x10\x02*\x84\x02\n" + + "\x1bINSTALLATION_STATUS_REVOKED\x10\x02\x12\x1f\n" + + "\x1bINSTALLATION_STATUS_PENDING\x10\x03\x12\x1e\n" + + "\x1aINSTALLATION_STATUS_FAILED\x10\x04*\x84\x02\n" + "\x12InstallationHealth\x12#\n" + "\x1fINSTALLATION_HEALTH_UNSPECIFIED\x10\x00\x12\x1f\n" + "\x1bINSTALLATION_HEALTH_HEALTHY\x10\x01\x12)\n" + "%INSTALLATION_HEALTH_NO_ELIGIBLE_OWNER\x10\x02\x12%\n" + "!INSTALLATION_HEALTH_AGENT_REVOKED\x10\x03\x12&\n" + "\"INSTALLATION_HEALTH_AGENT_DISABLED\x10\x04\x12.\n" + - "*INSTALLATION_HEALTH_STANDING_GRANT_MISSING\x10\x052\xd1\x10\n" + + "*INSTALLATION_HEALTH_STANDING_GRANT_MISSING\x10\x052\xaa\x12\n" + "\x13InstallationService\x12\xba\x01\n" + "\x0fInstallSolution\x12(.saas.accounts.v1.InstallSolutionRequest\x1a\x1e.saas.accounts.v1.Installation\"]\xc2\xf3\x18=\b\x02\x10\x04*\f\n" + "\x06org_id\x10\x02\x18\x010\x01:\x1d\n" + @@ -1699,7 +1777,10 @@ const file_saas_accounts_v1_installations_proto_rawDesc = "" + "\x19saas.installation.revoked\x10\x02@\x01H\x04P\x03X\x03`\x01\x82\xd3\xe4\x93\x022:\x01*\"-/v1/installations/{installation_id}:uninstall\x12\xd2\x01\n" + "\x12NarrowInstallation\x12+.saas.accounts.v1.NarrowInstallationRequest\x1a\x16.google.protobuf.Empty\"w\xc2\xf3\x18>\b\x02\x10\x04*\f\n" + "\x06org_id\x10\x02\x18\x010\x01:\x1e\n" + - "\x1asaas.installation.narrowed\x10\x02@\x01H\x04P\x03X\x03`\x01\x82\xd3\xe4\x93\x02/:\x01*\"*/v1/installations/{installation_id}:narrow\x12\x89\x02\n" + + "\x1asaas.installation.narrowed\x10\x02@\x01H\x04P\x03X\x03`\x01\x82\xd3\xe4\x93\x02/:\x01*\"*/v1/installations/{installation_id}:narrow\x12\xd6\x01\n" + + "\x11RetryInstallation\x12*.saas.accounts.v1.RetryInstallationRequest\x1a\x1e.saas.accounts.v1.Installation\"u\xc2\xf3\x18=\b\x02\x10\x04*\f\n" + + "\x06org_id\x10\x02\x18\x010\x01:\x1d\n" + + "\x19saas.installation.retried\x10\x02@\x01H\x04P\x03X\x03`\x01\x82\xd3\xe4\x93\x02.:\x01*\")/v1/installations/{installation_id}:retry\x12\x89\x02\n" + "\x1dTransferInstallationOwnership\x126.saas.accounts.v1.TransferInstallationOwnershipRequest\x1a\x1e.saas.accounts.v1.Installation\"\x8f\x01\xc2\xf3\x18K\b\x02\x10\x04*\f\n" + "\x06org_id\x10\x02\x18\x010\x01:+\n" + "'saas.installation.ownership_transferred\x10\x02@\x01H\x04P\x03X\x03`\x01\x82\xd3\xe4\x93\x02::\x01*\"5/v1/installations/{installation_id}:transferOwnership\x12\xb9\x01\n" + @@ -1731,74 +1812,77 @@ func file_saas_accounts_v1_installations_proto_rawDescGZIP() []byte { } var file_saas_accounts_v1_installations_proto_enumTypes = make([]protoimpl.EnumInfo, 2) -var file_saas_accounts_v1_installations_proto_msgTypes = make([]protoimpl.MessageInfo, 19) +var file_saas_accounts_v1_installations_proto_msgTypes = make([]protoimpl.MessageInfo, 20) var file_saas_accounts_v1_installations_proto_goTypes = []any{ (InstallationStatus)(0), // 0: saas.accounts.v1.InstallationStatus (InstallationHealth)(0), // 1: saas.accounts.v1.InstallationHealth (*Installation)(nil), // 2: saas.accounts.v1.Installation (*InstallSolutionRequest)(nil), // 3: saas.accounts.v1.InstallSolutionRequest (*NarrowInstallationRequest)(nil), // 4: saas.accounts.v1.NarrowInstallationRequest - (*UninstallSolutionRequest)(nil), // 5: saas.accounts.v1.UninstallSolutionRequest - (*TransferInstallationOwnershipRequest)(nil), // 6: saas.accounts.v1.TransferInstallationOwnershipRequest - (*GetInstallationRequest)(nil), // 7: saas.accounts.v1.GetInstallationRequest - (*GetInstallationResponse)(nil), // 8: saas.accounts.v1.GetInstallationResponse - (*ListInstallationsRequest)(nil), // 9: saas.accounts.v1.ListInstallationsRequest - (*InstallationSummary)(nil), // 10: saas.accounts.v1.InstallationSummary - (*ListInstallationsResponse)(nil), // 11: saas.accounts.v1.ListInstallationsResponse - (*ListOrganizationInstallationsRequest)(nil), // 12: saas.accounts.v1.ListOrganizationInstallationsRequest - (*OrganizationInstallation)(nil), // 13: saas.accounts.v1.OrganizationInstallation - (*ListOrganizationInstallationsResponse)(nil), // 14: saas.accounts.v1.ListOrganizationInstallationsResponse - (*GrantInstallationToTeamRequest)(nil), // 15: saas.accounts.v1.GrantInstallationToTeamRequest - (*GrantInstallationToTeamResponse)(nil), // 16: saas.accounts.v1.GrantInstallationToTeamResponse - (*RevokeInstallationFromTeamRequest)(nil), // 17: saas.accounts.v1.RevokeInstallationFromTeamRequest - (*AvailableSolution)(nil), // 18: saas.accounts.v1.AvailableSolution - (*ListAvailableSolutionsRequest)(nil), // 19: saas.accounts.v1.ListAvailableSolutionsRequest - (*ListAvailableSolutionsResponse)(nil), // 20: saas.accounts.v1.ListAvailableSolutionsResponse - (*timestamppb.Timestamp)(nil), // 21: google.protobuf.Timestamp - (*CollectionReadGrant)(nil), // 22: saas.accounts.v1.CollectionReadGrant - (*ScopeGrant)(nil), // 23: saas.accounts.v1.ScopeGrant - (*emptypb.Empty)(nil), // 24: google.protobuf.Empty + (*RetryInstallationRequest)(nil), // 5: saas.accounts.v1.RetryInstallationRequest + (*UninstallSolutionRequest)(nil), // 6: saas.accounts.v1.UninstallSolutionRequest + (*TransferInstallationOwnershipRequest)(nil), // 7: saas.accounts.v1.TransferInstallationOwnershipRequest + (*GetInstallationRequest)(nil), // 8: saas.accounts.v1.GetInstallationRequest + (*GetInstallationResponse)(nil), // 9: saas.accounts.v1.GetInstallationResponse + (*ListInstallationsRequest)(nil), // 10: saas.accounts.v1.ListInstallationsRequest + (*InstallationSummary)(nil), // 11: saas.accounts.v1.InstallationSummary + (*ListInstallationsResponse)(nil), // 12: saas.accounts.v1.ListInstallationsResponse + (*ListOrganizationInstallationsRequest)(nil), // 13: saas.accounts.v1.ListOrganizationInstallationsRequest + (*OrganizationInstallation)(nil), // 14: saas.accounts.v1.OrganizationInstallation + (*ListOrganizationInstallationsResponse)(nil), // 15: saas.accounts.v1.ListOrganizationInstallationsResponse + (*GrantInstallationToTeamRequest)(nil), // 16: saas.accounts.v1.GrantInstallationToTeamRequest + (*GrantInstallationToTeamResponse)(nil), // 17: saas.accounts.v1.GrantInstallationToTeamResponse + (*RevokeInstallationFromTeamRequest)(nil), // 18: saas.accounts.v1.RevokeInstallationFromTeamRequest + (*AvailableSolution)(nil), // 19: saas.accounts.v1.AvailableSolution + (*ListAvailableSolutionsRequest)(nil), // 20: saas.accounts.v1.ListAvailableSolutionsRequest + (*ListAvailableSolutionsResponse)(nil), // 21: saas.accounts.v1.ListAvailableSolutionsResponse + (*timestamppb.Timestamp)(nil), // 22: google.protobuf.Timestamp + (*CollectionReadGrant)(nil), // 23: saas.accounts.v1.CollectionReadGrant + (*ScopeGrant)(nil), // 24: saas.accounts.v1.ScopeGrant + (*emptypb.Empty)(nil), // 25: google.protobuf.Empty } var file_saas_accounts_v1_installations_proto_depIdxs = []int32{ 0, // 0: saas.accounts.v1.Installation.status:type_name -> saas.accounts.v1.InstallationStatus - 21, // 1: saas.accounts.v1.Installation.created_at:type_name -> google.protobuf.Timestamp - 21, // 2: saas.accounts.v1.Installation.revoked_at:type_name -> google.protobuf.Timestamp + 22, // 1: saas.accounts.v1.Installation.created_at:type_name -> google.protobuf.Timestamp + 22, // 2: saas.accounts.v1.Installation.revoked_at:type_name -> google.protobuf.Timestamp 2, // 3: saas.accounts.v1.GetInstallationResponse.installation:type_name -> saas.accounts.v1.Installation 1, // 4: saas.accounts.v1.GetInstallationResponse.health:type_name -> saas.accounts.v1.InstallationHealth 0, // 5: saas.accounts.v1.ListInstallationsRequest.status:type_name -> saas.accounts.v1.InstallationStatus 2, // 6: saas.accounts.v1.InstallationSummary.installation:type_name -> saas.accounts.v1.Installation 1, // 7: saas.accounts.v1.InstallationSummary.health:type_name -> saas.accounts.v1.InstallationHealth - 10, // 8: saas.accounts.v1.ListInstallationsResponse.installations:type_name -> saas.accounts.v1.InstallationSummary + 11, // 8: saas.accounts.v1.ListInstallationsResponse.installations:type_name -> saas.accounts.v1.InstallationSummary 2, // 9: saas.accounts.v1.OrganizationInstallation.installation:type_name -> saas.accounts.v1.Installation 1, // 10: saas.accounts.v1.OrganizationInstallation.health:type_name -> saas.accounts.v1.InstallationHealth - 18, // 11: saas.accounts.v1.OrganizationInstallation.target:type_name -> saas.accounts.v1.AvailableSolution - 22, // 12: saas.accounts.v1.OrganizationInstallation.granted_teams:type_name -> saas.accounts.v1.CollectionReadGrant - 22, // 13: saas.accounts.v1.OrganizationInstallation.inherited_teams:type_name -> saas.accounts.v1.CollectionReadGrant - 13, // 14: saas.accounts.v1.ListOrganizationInstallationsResponse.installations:type_name -> saas.accounts.v1.OrganizationInstallation - 23, // 15: saas.accounts.v1.GrantInstallationToTeamResponse.grant:type_name -> saas.accounts.v1.ScopeGrant - 18, // 16: saas.accounts.v1.ListAvailableSolutionsResponse.solutions:type_name -> saas.accounts.v1.AvailableSolution + 19, // 11: saas.accounts.v1.OrganizationInstallation.target:type_name -> saas.accounts.v1.AvailableSolution + 23, // 12: saas.accounts.v1.OrganizationInstallation.granted_teams:type_name -> saas.accounts.v1.CollectionReadGrant + 23, // 13: saas.accounts.v1.OrganizationInstallation.inherited_teams:type_name -> saas.accounts.v1.CollectionReadGrant + 14, // 14: saas.accounts.v1.ListOrganizationInstallationsResponse.installations:type_name -> saas.accounts.v1.OrganizationInstallation + 24, // 15: saas.accounts.v1.GrantInstallationToTeamResponse.grant:type_name -> saas.accounts.v1.ScopeGrant + 19, // 16: saas.accounts.v1.ListAvailableSolutionsResponse.solutions:type_name -> saas.accounts.v1.AvailableSolution 3, // 17: saas.accounts.v1.InstallationService.InstallSolution:input_type -> saas.accounts.v1.InstallSolutionRequest - 5, // 18: saas.accounts.v1.InstallationService.UninstallSolution:input_type -> saas.accounts.v1.UninstallSolutionRequest + 6, // 18: saas.accounts.v1.InstallationService.UninstallSolution:input_type -> saas.accounts.v1.UninstallSolutionRequest 4, // 19: saas.accounts.v1.InstallationService.NarrowInstallation:input_type -> saas.accounts.v1.NarrowInstallationRequest - 6, // 20: saas.accounts.v1.InstallationService.TransferInstallationOwnership:input_type -> saas.accounts.v1.TransferInstallationOwnershipRequest - 7, // 21: saas.accounts.v1.InstallationService.GetInstallation:input_type -> saas.accounts.v1.GetInstallationRequest - 9, // 22: saas.accounts.v1.InstallationService.ListInstallations:input_type -> saas.accounts.v1.ListInstallationsRequest - 19, // 23: saas.accounts.v1.InstallationService.ListAvailableSolutions:input_type -> saas.accounts.v1.ListAvailableSolutionsRequest - 12, // 24: saas.accounts.v1.InstallationService.ListOrganizationInstallations:input_type -> saas.accounts.v1.ListOrganizationInstallationsRequest - 15, // 25: saas.accounts.v1.InstallationService.GrantInstallationToTeam:input_type -> saas.accounts.v1.GrantInstallationToTeamRequest - 17, // 26: saas.accounts.v1.InstallationService.RevokeInstallationFromTeam:input_type -> saas.accounts.v1.RevokeInstallationFromTeamRequest - 2, // 27: saas.accounts.v1.InstallationService.InstallSolution:output_type -> saas.accounts.v1.Installation - 24, // 28: saas.accounts.v1.InstallationService.UninstallSolution:output_type -> google.protobuf.Empty - 24, // 29: saas.accounts.v1.InstallationService.NarrowInstallation:output_type -> google.protobuf.Empty - 2, // 30: saas.accounts.v1.InstallationService.TransferInstallationOwnership:output_type -> saas.accounts.v1.Installation - 8, // 31: saas.accounts.v1.InstallationService.GetInstallation:output_type -> saas.accounts.v1.GetInstallationResponse - 11, // 32: saas.accounts.v1.InstallationService.ListInstallations:output_type -> saas.accounts.v1.ListInstallationsResponse - 20, // 33: saas.accounts.v1.InstallationService.ListAvailableSolutions:output_type -> saas.accounts.v1.ListAvailableSolutionsResponse - 14, // 34: saas.accounts.v1.InstallationService.ListOrganizationInstallations:output_type -> saas.accounts.v1.ListOrganizationInstallationsResponse - 16, // 35: saas.accounts.v1.InstallationService.GrantInstallationToTeam:output_type -> saas.accounts.v1.GrantInstallationToTeamResponse - 24, // 36: saas.accounts.v1.InstallationService.RevokeInstallationFromTeam:output_type -> google.protobuf.Empty - 27, // [27:37] is the sub-list for method output_type - 17, // [17:27] is the sub-list for method input_type + 5, // 20: saas.accounts.v1.InstallationService.RetryInstallation:input_type -> saas.accounts.v1.RetryInstallationRequest + 7, // 21: saas.accounts.v1.InstallationService.TransferInstallationOwnership:input_type -> saas.accounts.v1.TransferInstallationOwnershipRequest + 8, // 22: saas.accounts.v1.InstallationService.GetInstallation:input_type -> saas.accounts.v1.GetInstallationRequest + 10, // 23: saas.accounts.v1.InstallationService.ListInstallations:input_type -> saas.accounts.v1.ListInstallationsRequest + 20, // 24: saas.accounts.v1.InstallationService.ListAvailableSolutions:input_type -> saas.accounts.v1.ListAvailableSolutionsRequest + 13, // 25: saas.accounts.v1.InstallationService.ListOrganizationInstallations:input_type -> saas.accounts.v1.ListOrganizationInstallationsRequest + 16, // 26: saas.accounts.v1.InstallationService.GrantInstallationToTeam:input_type -> saas.accounts.v1.GrantInstallationToTeamRequest + 18, // 27: saas.accounts.v1.InstallationService.RevokeInstallationFromTeam:input_type -> saas.accounts.v1.RevokeInstallationFromTeamRequest + 2, // 28: saas.accounts.v1.InstallationService.InstallSolution:output_type -> saas.accounts.v1.Installation + 25, // 29: saas.accounts.v1.InstallationService.UninstallSolution:output_type -> google.protobuf.Empty + 25, // 30: saas.accounts.v1.InstallationService.NarrowInstallation:output_type -> google.protobuf.Empty + 2, // 31: saas.accounts.v1.InstallationService.RetryInstallation:output_type -> saas.accounts.v1.Installation + 2, // 32: saas.accounts.v1.InstallationService.TransferInstallationOwnership:output_type -> saas.accounts.v1.Installation + 9, // 33: saas.accounts.v1.InstallationService.GetInstallation:output_type -> saas.accounts.v1.GetInstallationResponse + 12, // 34: saas.accounts.v1.InstallationService.ListInstallations:output_type -> saas.accounts.v1.ListInstallationsResponse + 21, // 35: saas.accounts.v1.InstallationService.ListAvailableSolutions:output_type -> saas.accounts.v1.ListAvailableSolutionsResponse + 15, // 36: saas.accounts.v1.InstallationService.ListOrganizationInstallations:output_type -> saas.accounts.v1.ListOrganizationInstallationsResponse + 17, // 37: saas.accounts.v1.InstallationService.GrantInstallationToTeam:output_type -> saas.accounts.v1.GrantInstallationToTeamResponse + 25, // 38: saas.accounts.v1.InstallationService.RevokeInstallationFromTeam:output_type -> google.protobuf.Empty + 28, // [28:39] is the sub-list for method output_type + 17, // [17:28] is the sub-list for method input_type 17, // [17:17] is the sub-list for extension type_name 17, // [17:17] is the sub-list for extension extendee 0, // [0:17] is the sub-list for field type_name @@ -1818,7 +1902,7 @@ func file_saas_accounts_v1_installations_proto_init() { GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: unsafe.Slice(unsafe.StringData(file_saas_accounts_v1_installations_proto_rawDesc), len(file_saas_accounts_v1_installations_proto_rawDesc)), NumEnums: 2, - NumMessages: 19, + NumMessages: 20, NumExtensions: 0, NumServices: 1, }, diff --git a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.gw.go b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.gw.go index 753f03c4c..253f569f0 100644 --- a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.gw.go +++ b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations.pb.gw.go @@ -152,6 +152,51 @@ func local_request_InstallationService_NarrowInstallation_0(ctx context.Context, return msg, metadata, err } +func request_InstallationService_RetryInstallation_0(ctx context.Context, marshaler runtime.Marshaler, client InstallationServiceClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) { + var ( + protoReq RetryInstallationRequest + metadata runtime.ServerMetadata + err error + ) + if err := marshaler.NewDecoder(req.Body).Decode(&protoReq); err != nil && !errors.Is(err, io.EOF) { + return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err) + } + if req.Body != nil { + _, _ = io.Copy(io.Discard, req.Body) + } + val, ok := pathParams["installation_id"] + if !ok { + return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "installation_id") + } + protoReq.InstallationId, err = runtime.String(val) + if err != nil { + return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "installation_id", err) + } + msg, err := client.RetryInstallation(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD)) + return msg, metadata, err +} + +func local_request_InstallationService_RetryInstallation_0(ctx context.Context, marshaler runtime.Marshaler, server InstallationServiceServer, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) { + var ( + protoReq RetryInstallationRequest + metadata runtime.ServerMetadata + err error + ) + if err := marshaler.NewDecoder(req.Body).Decode(&protoReq); err != nil && !errors.Is(err, io.EOF) { + return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err) + } + val, ok := pathParams["installation_id"] + if !ok { + return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "installation_id") + } + protoReq.InstallationId, err = runtime.String(val) + if err != nil { + return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "installation_id", err) + } + msg, err := server.RetryInstallation(ctx, &protoReq) + return msg, metadata, err +} + func request_InstallationService_TransferInstallationOwnership_0(ctx context.Context, marshaler runtime.Marshaler, client InstallationServiceClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) { var ( protoReq TransferInstallationOwnershipRequest @@ -492,6 +537,26 @@ func RegisterInstallationServiceHandlerServer(ctx context.Context, mux *runtime. } forward_InstallationService_NarrowInstallation_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...) }) + mux.Handle(http.MethodPost, pattern_InstallationService_RetryInstallation_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) { + ctx, cancel := context.WithCancel(req.Context()) + defer cancel() + var stream runtime.ServerTransportStream + ctx = grpc.NewContextWithServerTransportStream(ctx, &stream) + inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req) + annotatedContext, err := runtime.AnnotateIncomingContext(ctx, mux, req, "/saas.accounts.v1.InstallationService/RetryInstallation", runtime.WithHTTPPathPattern("/v1/installations/{installation_id}:retry")) + if err != nil { + runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err) + return + } + resp, md, err := local_request_InstallationService_RetryInstallation_0(annotatedContext, inboundMarshaler, server, req, pathParams) + md.HeaderMD, md.TrailerMD = metadata.Join(md.HeaderMD, stream.Header()), metadata.Join(md.TrailerMD, stream.Trailer()) + annotatedContext = runtime.NewServerMetadataContext(annotatedContext, md) + if err != nil { + runtime.HTTPError(annotatedContext, mux, outboundMarshaler, w, req, err) + return + } + forward_InstallationService_RetryInstallation_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...) + }) mux.Handle(http.MethodPost, pattern_InstallationService_TransferInstallationOwnership_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) { ctx, cancel := context.WithCancel(req.Context()) defer cancel() @@ -703,6 +768,23 @@ func RegisterInstallationServiceHandlerClient(ctx context.Context, mux *runtime. } forward_InstallationService_NarrowInstallation_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...) }) + mux.Handle(http.MethodPost, pattern_InstallationService_RetryInstallation_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) { + ctx, cancel := context.WithCancel(req.Context()) + defer cancel() + inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req) + annotatedContext, err := runtime.AnnotateContext(ctx, mux, req, "/saas.accounts.v1.InstallationService/RetryInstallation", runtime.WithHTTPPathPattern("/v1/installations/{installation_id}:retry")) + if err != nil { + runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err) + return + } + resp, md, err := request_InstallationService_RetryInstallation_0(annotatedContext, inboundMarshaler, client, req, pathParams) + annotatedContext = runtime.NewServerMetadataContext(annotatedContext, md) + if err != nil { + runtime.HTTPError(annotatedContext, mux, outboundMarshaler, w, req, err) + return + } + forward_InstallationService_RetryInstallation_0(annotatedContext, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...) + }) mux.Handle(http.MethodPost, pattern_InstallationService_TransferInstallationOwnership_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) { ctx, cancel := context.WithCancel(req.Context()) defer cancel() @@ -812,6 +894,7 @@ var ( pattern_InstallationService_InstallSolution_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1}, []string{"v1", "installations"}, "")) pattern_InstallationService_UninstallSolution_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 1, 0, 4, 1, 5, 2}, []string{"v1", "installations", "installation_id"}, "uninstall")) pattern_InstallationService_NarrowInstallation_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 1, 0, 4, 1, 5, 2}, []string{"v1", "installations", "installation_id"}, "narrow")) + pattern_InstallationService_RetryInstallation_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 1, 0, 4, 1, 5, 2}, []string{"v1", "installations", "installation_id"}, "retry")) pattern_InstallationService_TransferInstallationOwnership_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 1, 0, 4, 1, 5, 2}, []string{"v1", "installations", "installation_id"}, "transferOwnership")) pattern_InstallationService_GetInstallation_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1, 1, 0, 4, 1, 5, 2}, []string{"v1", "installations", "installation_id"}, "")) pattern_InstallationService_ListAvailableSolutions_0 = runtime.MustPattern(runtime.NewPattern(1, []int{2, 0, 2, 1}, []string{"v1", "installations"}, "listAvailable")) @@ -824,6 +907,7 @@ var ( forward_InstallationService_InstallSolution_0 = runtime.ForwardResponseMessage forward_InstallationService_UninstallSolution_0 = runtime.ForwardResponseMessage forward_InstallationService_NarrowInstallation_0 = runtime.ForwardResponseMessage + forward_InstallationService_RetryInstallation_0 = runtime.ForwardResponseMessage forward_InstallationService_TransferInstallationOwnership_0 = runtime.ForwardResponseMessage forward_InstallationService_GetInstallation_0 = runtime.ForwardResponseMessage forward_InstallationService_ListAvailableSolutions_0 = runtime.ForwardResponseMessage diff --git a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations_grpc.pb.go b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations_grpc.pb.go index 5a8131a70..cee20fabe 100644 --- a/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations_grpc.pb.go +++ b/module/services/accounts/code/pkg/gen/saas/accounts/v1/installations_grpc.pb.go @@ -24,6 +24,7 @@ const ( InstallationService_InstallSolution_FullMethodName = "/saas.accounts.v1.InstallationService/InstallSolution" InstallationService_UninstallSolution_FullMethodName = "/saas.accounts.v1.InstallationService/UninstallSolution" InstallationService_NarrowInstallation_FullMethodName = "/saas.accounts.v1.InstallationService/NarrowInstallation" + InstallationService_RetryInstallation_FullMethodName = "/saas.accounts.v1.InstallationService/RetryInstallation" InstallationService_TransferInstallationOwnership_FullMethodName = "/saas.accounts.v1.InstallationService/TransferInstallationOwnership" InstallationService_GetInstallation_FullMethodName = "/saas.accounts.v1.InstallationService/GetInstallation" InstallationService_ListInstallations_FullMethodName = "/saas.accounts.v1.InstallationService/ListInstallations" @@ -54,6 +55,9 @@ type InstallationServiceClient interface { // Narrowing is witnessed in the policy log before commit, then atomically // bumps the installation revision, cancels its org's work and records audit. NarrowInstallation(ctx context.Context, in *NarrowInstallationRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + RetryInstallation(ctx context.Context, in *RetryInstallationRequest, opts ...grpc.CallOption) (*Installation, error) // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, @@ -163,6 +167,16 @@ func (c *installationServiceClient) NarrowInstallation(ctx context.Context, in * return out, nil } +func (c *installationServiceClient) RetryInstallation(ctx context.Context, in *RetryInstallationRequest, opts ...grpc.CallOption) (*Installation, error) { + cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) + out := new(Installation) + err := c.cc.Invoke(ctx, InstallationService_RetryInstallation_FullMethodName, in, out, cOpts...) + if err != nil { + return nil, err + } + return out, nil +} + func (c *installationServiceClient) TransferInstallationOwnership(ctx context.Context, in *TransferInstallationOwnershipRequest, opts ...grpc.CallOption) (*Installation, error) { cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) out := new(Installation) @@ -254,6 +268,9 @@ type InstallationServiceServer interface { // Narrowing is witnessed in the policy log before commit, then atomically // bumps the installation revision, cancels its org's work and records audit. NarrowInstallation(context.Context, *NarrowInstallationRequest) (*emptypb.Empty, error) + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + RetryInstallation(context.Context, *RetryInstallationRequest) (*Installation, error) // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, @@ -342,6 +359,9 @@ func (UnimplementedInstallationServiceServer) UninstallSolution(context.Context, func (UnimplementedInstallationServiceServer) NarrowInstallation(context.Context, *NarrowInstallationRequest) (*emptypb.Empty, error) { return nil, status.Error(codes.Unimplemented, "method NarrowInstallation not implemented") } +func (UnimplementedInstallationServiceServer) RetryInstallation(context.Context, *RetryInstallationRequest) (*Installation, error) { + return nil, status.Error(codes.Unimplemented, "method RetryInstallation not implemented") +} func (UnimplementedInstallationServiceServer) TransferInstallationOwnership(context.Context, *TransferInstallationOwnershipRequest) (*Installation, error) { return nil, status.Error(codes.Unimplemented, "method TransferInstallationOwnership not implemented") } @@ -438,6 +458,24 @@ func _InstallationService_NarrowInstallation_Handler(srv interface{}, ctx contex return interceptor(ctx, in, info, handler) } +func _InstallationService_RetryInstallation_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(RetryInstallationRequest) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(InstallationServiceServer).RetryInstallation(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: InstallationService_RetryInstallation_FullMethodName, + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(InstallationServiceServer).RetryInstallation(ctx, req.(*RetryInstallationRequest)) + } + return interceptor(ctx, in, info, handler) +} + func _InstallationService_TransferInstallationOwnership_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(TransferInstallationOwnershipRequest) if err := dec(in); err != nil { @@ -583,6 +621,10 @@ var InstallationService_ServiceDesc = grpc.ServiceDesc{ MethodName: "NarrowInstallation", Handler: _InstallationService_NarrowInstallation_Handler, }, + { + MethodName: "RetryInstallation", + Handler: _InstallationService_RetryInstallation_Handler, + }, { MethodName: "TransferInstallationOwnership", Handler: _InstallationService_TransferInstallationOwnership_Handler, diff --git a/module/services/accounts/code/pkg/gen/saas/accounts/v1/module_registration.pb.go b/module/services/accounts/code/pkg/gen/saas/accounts/v1/module_registration.pb.go index 616117d73..e3c76e897 100644 --- a/module/services/accounts/code/pkg/gen/saas/accounts/v1/module_registration.pb.go +++ b/module/services/accounts/code/pkg/gen/saas/accounts/v1/module_registration.pb.go @@ -44,8 +44,21 @@ type ModuleMintWorkContextRequest struct { // tenant-pinned principal, omit or match its tenant. Accounts validates the // requested organization's live installation after authenticating execution. OrganizationId string `protobuf:"bytes,4,opt,name=organization_id,json=organizationId,proto3" json:"organization_id,omitempty"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + Upstream string `protobuf:"bytes,5,opt,name=upstream,proto3" json:"upstream,omitempty"` + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + Manifest string `protobuf:"bytes,6,opt,name=manifest,proto3" json:"manifest,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *ModuleMintWorkContextRequest) Reset() { @@ -99,6 +112,20 @@ func (x *ModuleMintWorkContextRequest) GetOrganizationId() string { return "" } +func (x *ModuleMintWorkContextRequest) GetUpstream() string { + if x != nil { + return x.Upstream + } + return "" +} + +func (x *ModuleMintWorkContextRequest) GetManifest() string { + if x != nil { + return x.Manifest + } + return "" +} + // ModuleMintWorkContextResponse carries the signed capability plus the identity // it asserts, so a module can attribute its own work without parsing the token. type ModuleMintWorkContextResponse struct { @@ -575,11 +602,13 @@ var File_saas_accounts_v1_module_registration_proto protoreflect.FileDescriptor const file_saas_accounts_v1_module_registration_proto_rawDesc = "" + "\n" + - "*saas/accounts/v1/module_registration.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1fgoogle/protobuf/timestamp.proto\"\xe9\x01\n" + + "*saas/accounts/v1/module_registration.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1fgoogle/protobuf/timestamp.proto\"\xbf\x02\n" + "\x1cModuleMintWorkContextRequest\x12D\n" + "\x06prefix\x18\x01 \x01(\tB,\xbaH)r'\x10\x01\x18?2!^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$R\x06prefix\x12?\n" + "\x15service_account_token\x18\x03 \x01(\tB\v\xbaH\br\x06\x10\x01\x18\x80\x80\x04R\x13serviceAccountToken\x124\n" + - "\x0forganization_id\x18\x04 \x01(\tB\v\xbaH\b\xd8\x01\x01r\x03\xb0\x01\x01R\x0eorganizationIdJ\x04\b\x02\x10\x03R\x06secret\"\xab\x01\n" + + "\x0forganization_id\x18\x04 \x01(\tB\v\xbaH\b\xd8\x01\x01r\x03\xb0\x01\x01R\x0eorganizationId\x12*\n" + + "\bupstream\x18\x05 \x01(\tB\x0e\xbaH\v\xd8\x01\x01r\x06\x18\x80\x04\x88\x01\x01R\bupstream\x12(\n" + + "\bmanifest\x18\x06 \x01(\tB\f\xbaH\t\xd8\x01\x01r\x04\x18\x80\x80\x04R\bmanifestJ\x04\b\x02\x10\x03R\x06secret\"\xab\x01\n" + "\x1dModuleMintWorkContextResponse\x12\x14\n" + "\x05token\x18\x01 \x01(\tR\x05token\x129\n" + "\n" + diff --git a/module/services/accounts/code/pkg/infra/consumer_invoke_ceiling_db_test.go b/module/services/accounts/code/pkg/infra/consumer_invoke_ceiling_db_test.go index a23f6a08c..fbade1ff4 100644 --- a/module/services/accounts/code/pkg/infra/consumer_invoke_ceiling_db_test.go +++ b/module/services/accounts/code/pkg/infra/consumer_invoke_ceiling_db_test.go @@ -52,6 +52,9 @@ func consumerCeilingInstall(t *testing.T, installed business.ModulePrincipalGran OwnerPrincipalID: owner, GrantedBy: owner, RoleID: role, Kind: "solution", RegistryBindingID: binding, RegistryPrincipalID: principal, Scope: &scope, EnvelopeRevision: revision}) return err })) + // An installation is pending until its install step is acknowledged, and a + // ceiling is read for an installation in use. + acknowledgeInstallation(t, installation) return dynamicInstall{org, owner, target, binding, principal, installation, installed, revision} } @@ -60,7 +63,7 @@ func consumerCeilingService(t *testing.T, a dynamicInstall, current business.Mod current.PresenceBinding, current.Tenant, current.CrossTenant = a.binding, a.org, true svc, err := business.NewService(testStore) require.NoError(t, err) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: current}, a.envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: current}, a.envelope, nil) svc.SetModuleAuthorityReads(testStore, nil) return svc } @@ -119,6 +122,9 @@ func TestConsumerInvokeCeilingDBRowWithoutDeliveredAuthorityIsRefusedByName(t *t OwnerPrincipalID: owner, GrantedBy: owner, RoleID: role, Kind: "solution", RegistryBindingID: binding}) return err })) + // In use, like every installation whose ceiling is read: the refusal under + // test is the missing delivered authority, not a step still pending. + acknowledgeInstallation(t, installation) a := dynamicInstall{org: org, binding: binding, principal: business.ModulePrincipalID(alias), installation: installation} svc := consumerCeilingService(t, a, grant) _, err := svc.ConsumerInvokeCeiling(testCtx, a.org, a.installation.Id) diff --git a/module/services/accounts/code/pkg/infra/installation_ack_fixture_test.go b/module/services/accounts/code/pkg/infra/installation_ack_fixture_test.go new file mode 100644 index 000000000..f9a660ccc --- /dev/null +++ b/module/services/accounts/code/pkg/infra/installation_ack_fixture_test.go @@ -0,0 +1,50 @@ +//go:build !pure + +package infra_test + +import ( + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/infra" + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/types/known/durationpb" +) + +// Existing authority fixtures require an acknowledged installation. Drive the +// actual durable queue and outcome projection, as a setup consumer would; no +// fixture writes active directly. These database suites run sequentially. +func acknowledgeInstallation(t *testing.T, i *gen.Installation) { + t.Helper() + if i.Status != gen.InstallationStatus_INSTALLATION_STATUS_PENDING { + return + } + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + for n := 0; n < 1000; n++ { + batch, err := worker.Claim(testCtx, &jobsv1.ClaimJobsRequest{Queue: business.InstallationStepQueue, WorkerId: "fixture-ack", Limit: 1, LeaseDuration: durationpb.New(time.Minute)}) + require.NoError(t, err) + require.Len(t, batch.Jobs, 1, "expected an enqueued installation step") + job := batch.Jobs[0] + require.NoError(t, worker.Complete(testCtx, &jobsv1.CompleteJobRequest{Lease: &jobsv1.JobLeaseReference{JobId: job.Id, WorkerId: job.Lease.Owner, LeaseToken: job.Lease.Token}})) + if job.Attributes["installation_step.id"] == i.Id { + require.NoError(t, testStore.WithOrgTx(testCtx, i.OrgId, func(ctx context.Context) error { + current, _, err := testStore.GetInstallation(ctx, i.OrgId, i.Id) + if err == nil { + i.Status = current.Status + i.FailureReason = current.FailureReason + } + return err + })) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, i.Status) + return + } + } + t.Fatal("installation step not found") +} diff --git a/module/services/accounts/code/pkg/infra/installation_credential_test.go b/module/services/accounts/code/pkg/infra/installation_credential_test.go index e706eb792..73a993cc0 100644 --- a/module/services/accounts/code/pkg/infra/installation_credential_test.go +++ b/module/services/accounts/code/pkg/infra/installation_credential_test.go @@ -106,7 +106,7 @@ func TestSealedModuleCredentialDBLiveInstallationAtEveryEntryPoint(t *testing.T) svc, err := business.NewService(testStore) require.NoError(t, err) svc.SetModuleCapabilities(testStore, worker, nil) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, installParams.EnvelopeRevision) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, installParams.EnvelopeRevision, nil) svc.SetDatasourceConnector(installationCipher{}, testStore, "") svc.SetDatasourceGitHubClientFactory(func(string) business.GitHubContentClient { return installationContent{} }) source := seedDatasourceSource(t, org) @@ -239,6 +239,7 @@ func TestSealedModuleCredentialDBLiveInstallationAtEveryEntryPoint(t *testing.T) replacement, err = testStore.InstallSolution(ctx, &installParams) return err })) + acknowledgeInstallation(t, replacement) require.NotEqual(t, installation.Id, replacement.Id) _, err = issuer.VerifyModuleWorkContext(testCtx, credential) require.Error(t, err, "a reinstall does not revive a credential sealed to the old installation") diff --git a/module/services/accounts/code/pkg/infra/installation_dynamic_test.go b/module/services/accounts/code/pkg/infra/installation_dynamic_test.go index aaf97c7fb..2bda796ea 100644 --- a/module/services/accounts/code/pkg/infra/installation_dynamic_test.go +++ b/module/services/accounts/code/pkg/infra/installation_dynamic_test.go @@ -22,7 +22,7 @@ type dynamicInstall struct { envelope uint64 } -func newDynamicInstall(t *testing.T, target, binding, principal string) dynamicInstall { +func newDynamicInstall(t *testing.T, target, binding, principal string, pending ...bool) dynamicInstall { t.Helper() owner := seedUser(t) seedHumanPrincipal(t, owner, "Jane Doe") @@ -53,6 +53,9 @@ func newDynamicInstall(t *testing.T, target, binding, principal string) dynamicI OwnerPrincipalID: owner, GrantedBy: owner, RoleID: role, Kind: "solution", RegistryBindingID: binding, RegistryPrincipalID: principal, Scope: &scope, EnvelopeRevision: revision}) return err })) + if len(pending) == 0 || !pending[0] { + acknowledgeInstallation(t, installation) + } return dynamicInstall{org, owner, target, binding, principal, installation, grant, revision} } @@ -167,7 +170,7 @@ func TestDynamicInstallationDBCancelsOnlyItsOrgAndFencesLeases(t *testing.T) { foreignPendingID := enqueue(b) svc, err := business.NewService(testStore) require.NoError(t, err) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: a.grant}, a.envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: a.grant}, a.envelope, nil) emitter, err := business.NewDurableAuditEmitter(testStore, testStore) require.NoError(t, err) svc.SetAuditEmitter(emitter) diff --git a/module/services/accounts/code/pkg/infra/installation_fixture_test.go b/module/services/accounts/code/pkg/infra/installation_fixture_test.go index 6935452f9..c8928e69a 100644 --- a/module/services/accounts/code/pkg/infra/installation_fixture_test.go +++ b/module/services/accounts/code/pkg/infra/installation_fixture_test.go @@ -58,17 +58,20 @@ func installCapabilityFixture(t *testing.T, svc *business.Service, org, owner st } role := business.NewIDString() scope := business.FullInstallationScope(grant) + var installed *gen.Installation require.NoError(t, testStore.WithOrgTx(testCtx, org, func(ctx context.Context) error { if err := testStore.CreateRole(ctx, &gen.Role{Id: role, OrgId: org, Name: "Example role " + role, Permissions: []*gen.Permission{{Resource: "solution", Action: "use"}}}); err != nil { return err } - _, err := testStore.InstallSolution(ctx, &business.InstallSolutionParams{ + var err error + installed, err = testStore.InstallSolution(ctx, &business.InstallSolutionParams{ OrgID: org, OwnerPrincipalID: owner, GrantedBy: owner, RoleID: role, AgentIdentifier: "acme.example/" + alias + ":1.0.0", TargetID: target, Kind: "solution", RegistryBindingID: grant.PresenceBinding, RegistryPrincipalID: principal, Scope: &scope, EnvelopeRevision: envelope, }) return err })) + acknowledgeInstallation(t, installed) live, err = testStore.LiveModuleAuthority(testCtx, principal, org, "") require.NoError(t, err) require.Equal(t, grant.PresenceBinding, live.RegistryBindingID) @@ -77,5 +80,5 @@ func installCapabilityFixture(t *testing.T, svc *business.Service, org, owner st require.Equal(t, envelope, *live.EnvelopeRevision) registry[principal] = grant } - svc.InstallDerivedModulePrincipals(registry, envelope) + svc.InstallDerivedModulePrincipals(registry, envelope, nil) } diff --git a/module/services/accounts/code/pkg/infra/installation_lifecycle_test.go b/module/services/accounts/code/pkg/infra/installation_lifecycle_test.go new file mode 100644 index 000000000..a3cdcdf1a --- /dev/null +++ b/module/services/accounts/code/pkg/infra/installation_lifecycle_test.go @@ -0,0 +1,284 @@ +//go:build !pure + +package infra_test + +import ( + "context" + "errors" + "testing" + "time" + + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/infra" + "accounts/pkg/infra/storetx" + + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/types/known/durationpb" +) + +func claimInstallationFor(t *testing.T, worker *infra.PostgresJobStore, id string) *jobsv1.JobEnvelope { + t.Helper() + for n := 0; n < 1000; n++ { + out, err := worker.Claim(testCtx, &jobsv1.ClaimJobsRequest{Queue: business.InstallationStepQueue, WorkerId: "example-install-consumer", Limit: 1, LeaseDuration: durationpb.New(time.Minute)}) + require.NoError(t, err) + require.Len(t, out.Jobs, 1) + job := out.Jobs[0] + if job.Attributes["installation_step.id"] == id { + return job + } + require.NoError(t, worker.Complete(testCtx, &jobsv1.CompleteJobRequest{Lease: installationLease(job)})) + } + t.Fatal("installation job not found") + return nil +} +func installationLease(job *jobsv1.JobEnvelope) *jobsv1.JobLeaseReference { + return &jobsv1.JobLeaseReference{JobId: job.Id, WorkerId: job.Lease.Owner, LeaseToken: job.Lease.Token} +} +func readLifecycle(t *testing.T, i dynamicInstall) (*gen.Installation, uint64, int) { + t.Helper() + var current *gen.Installation + var revision uint64 + var attempts int + require.NoError(t, testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + var err error + current, _, err = testStore.GetInstallation(ctx, i.org, i.installation.Id) + if err != nil { + return err + } + summaries, err := testStore.ListInstallations(ctx, i.org, current.Status, "", 100) + if err != nil { + return err + } + require.Len(t, summaries, 1) + require.Equal(t, current.Status, summaries[0].Installation.Status) + require.Equal(t, current.FailureReason, summaries[0].Installation.FailureReason) + return storetx.Tx(ctx).QueryRow(ctx, `SELECT revision,install_attempts FROM installations WHERE id=$1 AND org_id=$2`, i.installation.Id, i.org).Scan(&revision, &attempts) + })) + return current, revision, attempts +} + +// HOST-INST-002..005: actual queue leases, pending-only mint admission, atomic +// refusal/ack projection and a manual retry at the same consent revision. +func TestInstallationLifecycleRefusalRetryAndAcknowledgement(t *testing.T) { + i := newDynamicInstall(t, "", "", "", true) + current, revision, attempts := readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_PENDING, current.Status) + require.Equal(t, uint64(1), revision) + require.Zero(t, attempts) + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + svc, err := business.NewService(testStore) + require.NoError(t, err) + svc.SetModuleCapabilities(testStore, worker, nil) + rows, _, err := svc.ListOrganizationInstallations(testCtx, i.org, "", 100) + require.NoError(t, err) + require.Len(t, rows, 1) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_PENDING, rows[0].Summary.Installation.Status) + require.Empty(t, rows[0].Exposure.GrantedTeams) + job := claimInstallationFor(t, worker, i.installation.Id) + lease := installationLease(job) + _, err = testStore.ResolveInstallationAuthority(testCtx, i.org, i.installation.Id, nil) + require.Error(t, err) + require.NoError(t, testStore.WithInstallationStepAuthority(testCtx, lease, i.envelope, func(_ context.Context, a *business.InstallationStepAuthority) error { + require.Equal(t, i.installation.Id, a.InstallationID) + require.Equal(t, i.org, a.OrganizationID) + require.Equal(t, i.installation.AgentPrincipalId, a.Facts.Actor.ID) + require.Equal(t, i.installation.RootScopeNodeId, a.RootID) + return nil + })) + wrong := &jobsv1.JobLeaseReference{JobId: lease.JobId, WorkerId: lease.WorkerId, LeaseToken: business.NewIDString()} + require.Error(t, testStore.WithInstallationStepAuthority(testCtx, wrong, i.envelope, func(context.Context, *business.InstallationStepAuthority) error { + t.Fatal("wrong lease minted") + return nil + })) + require.NoError(t, svc.FinishInstallationStep(testCtx, lease, job.AttemptCount, 409, "Example refusal")) + current, revision, attempts = readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_FAILED, current.Status) + require.Equal(t, "Example refusal", current.FailureReason) + require.Equal(t, 1, attempts) + err = testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + _, err := testStore.InstallSolution(ctx, &business.InstallSolutionParams{OrgID: i.org, TargetID: i.target, OwnerPrincipalID: i.owner}) + return err + }) + require.ErrorContains(t, err, "already exists") + require.NoError(t, testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + retried, rev, err := testStore.RetryInstallation(ctx, i.org, i.installation.Id) + if err == nil { + require.Equal(t, revision, rev) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_PENDING, retried.Status) + require.Empty(t, retried.FailureReason) + var role, org string + err = storetx.Tx(ctx).QueryRow(ctx, `SELECT current_user, coalesce(current_setting('app.current_org_id',true),'')`).Scan(&role, &org) + require.Equal(t, "app_control_plane", role, "enqueue restores the enclosing transaction's role") + require.Empty(t, org, "enqueue does not leave tenant state on the control plane") + } + return err + })) + require.Error(t, svc.FinishInstallationStep(testCtx, lease, job.AttemptCount, 204, ""), "old lease cannot acknowledge a retry") + job = claimInstallationFor(t, worker, i.installation.Id) + require.NotEqual(t, lease.JobId, job.Id) + require.NoError(t, svc.FinishInstallationStep(testCtx, installationLease(job), job.AttemptCount, 204, "")) + current, revision, attempts = readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, current.Status) + require.Equal(t, uint64(1), revision) + // An active narrowing gets a fresh revision/run without losing availability. + scope := business.FullInstallationScope(i.grant) + scope.Queues = nil + require.NoError(t, testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + return testStore.NarrowInstallationScope(ctx, i.org, i.installation.Id, revision, scope, business.ModulePrincipalRegistry{i.principal: i.grant}, i.envelope) + })) + current, revision, _ = readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, current.Status) + require.Equal(t, uint64(2), revision) + job = claimInstallationFor(t, worker, i.installation.Id) + require.Equal(t, "2", job.Attributes["installation_step.revision"]) + require.NoError(t, svc.FinishInstallationStep(testCtx, installationLease(job), job.AttemptCount, 409, "Replay refused")) + current, _, _ = readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, current.Status) + require.Empty(t, current.FailureReason) +} + +func TestInstallationStepCancellationAndAtomicEnqueue(t *testing.T) { + for _, state := range []string{"pending", "processing", "retrying"} { + t.Run(state, func(t *testing.T) { + i := newDynamicInstall(t, "", "", "", true) + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + svc, err := business.NewService(testStore) + require.NoError(t, err) + svc.SetModuleCapabilities(testStore, worker, nil) + var job *jobsv1.JobEnvelope + if state != "pending" { + job = claimInstallationFor(t, worker, i.installation.Id) + if state == "retrying" { + require.NoError(t, svc.FinishInstallationStep(testCtx, installationLease(job), job.AttemptCount, 503, "")) + } + } + other := newDynamicInstall(t, i.target, i.binding, i.principal, true) + var jobID string + require.NoError(t, pool.QueryRow(testCtx, `SELECT id::text FROM job_messages + WHERE attributes->>'codefly.installation_id'=$1 AND attributes->>'codefly.organization_id'=$2`, i.installation.Id, i.org).Scan(&jobID)) + rollback := errors.New("roll back the lifecycle transaction") + err = testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + if err := testStore.EnqueueInstallationStep(ctx, i.org, i.installation.Id, "rolled-back"); err != nil { + return err + } + return rollback + }) + require.ErrorIs(t, err, rollback) + var count int + require.NoError(t, pool.QueryRow(testCtx, `SELECT count(*) FROM job_messages + WHERE attributes->>'codefly.installation_id'=$1`, i.installation.Id).Scan(&count)) + require.Equal(t, 1, count, "rolled-back enqueue leaves no orphan job") + var current, foreign bool + require.NoError(t, pool.QueryRow(testCtx, `SELECT public.installation_step_job_current(id,attributes), + public.installation_step_job_current(id,attributes || jsonb_build_object('codefly.organization_id',$2::text)) + FROM job_messages WHERE id=$1`, jobID, other.org).Scan(¤t, &foreign)) + require.True(t, current, "rollback preserves the exact current run") + require.False(t, foreign, "setup authority is bound to the installation's organization") + require.NoError(t, testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + _, _, err := testStore.UninstallSolution(ctx, i.org, i.installation.Id) + return err + })) + var got string + require.NoError(t, pool.QueryRow(testCtx, `SELECT state FROM job_messages WHERE id=$1`, jobID).Scan(&got)) + require.Equal(t, "canceled", got) + require.NoError(t, pool.QueryRow(testCtx, `SELECT state FROM job_messages + WHERE attributes->>'codefly.installation_id'=$1`, other.installation.Id).Scan(&got)) + require.Equal(t, "pending", got, "revocation preserves another organization's setup") + if job != nil { + require.Error(t, svc.FinishInstallationStep(testCtx, installationLease(job), job.AttemptCount, 204, "")) + } + }) + } +} + +func TestInstallationStepNarrowingFencesActiveReplay(t *testing.T) { + i := newDynamicInstall(t, "", "", "") + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + narrow := func(scope business.InstallationScope, revision uint64) { + t.Helper() + require.NoError(t, testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + return testStore.NarrowInstallationScope(ctx, i.org, i.installation.Id, revision, scope, + business.ModulePrincipalRegistry{i.principal: i.grant}, i.envelope) + })) + } + scope := business.FullInstallationScope(i.grant) + scope.Queues = nil + narrow(scope, 1) + old := claimInstallationFor(t, worker, i.installation.Id) + scope.Resources = nil + narrow(scope, 2) + var state string + require.NoError(t, pool.QueryRow(testCtx, `SELECT state FROM job_messages WHERE id=$1`, old.Id).Scan(&state)) + require.Equal(t, "canceled", state) + require.Error(t, worker.Complete(testCtx, &jobsv1.CompleteJobRequest{Lease: installationLease(old)})) + job := claimInstallationFor(t, worker, i.installation.Id) + require.NotEqual(t, old.Id, job.Id) + require.Equal(t, "3", job.Attributes["codefly.installation_revision"]) + require.NoError(t, worker.Complete(testCtx, &jobsv1.CompleteJobRequest{Lease: installationLease(job)})) + current, revision, _ := readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, current.Status) + require.Equal(t, uint64(3), revision) +} + +// HOST-INST-004: retryable outages exhaust the declared budget, with a fixed +// public reason. The test advances availability, never sleeps through backoff. +func TestInstallationLifecycleExhaustion(t *testing.T) { + i := newDynamicInstall(t, "", "", "", true) + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + svc, err := business.NewService(testStore) + require.NoError(t, err) + svc.SetModuleCapabilities(testStore, worker, nil) + for attempt := 1; attempt <= business.InstallationStepMaxAttempts; attempt++ { + job := claimInstallationFor(t, worker, i.installation.Id) + require.Equal(t, uint32(attempt), job.AttemptCount) + require.NoError(t, svc.FinishInstallationStep(testCtx, installationLease(job), job.AttemptCount, 503, "private diagnostic")) + if attempt < business.InstallationStepMaxAttempts { + _, err := pool.Exec(testCtx, `UPDATE job_messages SET available_at=clock_timestamp() WHERE id=$1 AND state='retrying'`, job.Id) + require.NoError(t, err) + } + } + current, revision, attempts := readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_FAILED, current.Status) + require.Equal(t, uint64(1), revision) + require.Equal(t, business.InstallationStepMaxAttempts, attempts) + require.Equal(t, business.InstallationStepFailure, current.FailureReason) +} + +// HOST-INST-004: losing the final worker is an exhausted attempt too. The +// common queue's recovery must project failure without a finish callback. +func TestInstallationLifecycleFinalLeaseExpiry(t *testing.T) { + i := newDynamicInstall(t, "", "", "", true) + pool, err := infra.NewJobWorkerPool(testCtx) + require.NoError(t, err) + defer pool.Close() + worker := infra.NewPostgresJobStore(pool) + for attempt := 1; attempt <= business.InstallationStepMaxAttempts; attempt++ { + job := claimInstallationFor(t, worker, i.installation.Id) + require.Equal(t, uint32(attempt), job.AttemptCount) + _, err := pool.Exec(testCtx, `UPDATE job_messages SET lease_expires_at=now()-interval '1 second' WHERE id=$1`, job.Id) + require.NoError(t, err) + } + // A claim pass recovers the exhausted lease even though it cannot claim it. + _, err = worker.Claim(testCtx, &jobsv1.ClaimJobsRequest{Queue: business.InstallationStepQueue, + WorkerId: "example-recovery", Limit: 1, LeaseDuration: durationpb.New(time.Minute)}) + require.NoError(t, err) + current, revision, attempts := readLifecycle(t, i) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_FAILED, current.Status) + require.Equal(t, uint64(1), revision) + require.Equal(t, business.InstallationStepMaxAttempts, attempts) + require.Equal(t, business.InstallationStepFailure, current.FailureReason) +} diff --git a/module/services/accounts/code/pkg/infra/installation_pending_test.go b/module/services/accounts/code/pkg/infra/installation_pending_test.go new file mode 100644 index 000000000..ca047b6b5 --- /dev/null +++ b/module/services/accounts/code/pkg/infra/installation_pending_test.go @@ -0,0 +1,110 @@ +//go:build !pure + +package infra_test + +import ( + "context" + "encoding/json" + "testing" + + "accounts/pkg/business" + "accounts/pkg/infra/storetx" + "accounts/pkg/jobs" + + "github.com/stretchr/testify/require" +) + +// HOST-INST-002, HOST-INST-004, HOST-INST-006: every enforcement-at-use +// installation reader withholds pending/failed consent, including the worker's +// SQL predicate and lock, the gateway projection, grants, and ordinary mints. +func TestPendingInstallationEnforcementReaders(t *testing.T) { + for _, state := range []string{"pending", "failed"} { + t.Run(state, func(t *testing.T) { + i := newDynamicInstall(t, "", "", "") + team := seedTeamWith(t, i.org, i.owner) + grantAtNode(t, i.org, team, solutionRole(t, i.org), i.installation) + require.Equal(t, []string{i.target}, entitledIdentifiers(t, i.org, i.owner)) + _, err := testStore.ResolveInstallationAuthority(testCtx, i.org, i.installation.Id, nil) + require.NoError(t, err, "positive control: ordinary mint resolves active consent") + require.NoError(t, testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + _, err := storetx.Tx(ctx).Exec(ctx, `UPDATE installations SET status=$2, + failure_reason=CASE WHEN $2='failed' THEN 'Example refusal' ELSE NULL END WHERE id=$1`, i.installation.Id, state) + return err + })) + _, err = testStore.LiveModuleAuthority(testCtx, i.principal, i.org, i.installation.Id) + require.ErrorIs(t, err, business.ErrModuleInstallationInactive) + _, err = testStore.ResolveInstallationAuthority(testCtx, i.org, i.installation.Id, nil) + require.Error(t, err, "ordinary StartInstallationTask must refuse") + require.NoError(t, testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + _, err := testStore.InstallationAuthorityState(ctx, i.org, i.installation.Id) + require.ErrorContains(t, err, business.ErrModuleInstallationInactive.Error()) + _, err = testStore.InstallationRootScopePath(ctx, i.org, i.installation.Id) + require.Error(t, err, "team grants require an active root") + exposure, err := testStore.ListInstallationExposure(ctx, i.org, []string{i.installation.Id}) + require.NoError(t, err) + require.Empty(t, exposure) + entitlements, err := testStore.ListSolutionEntitlements(ctx, i.org, i.owner, "", 100) + require.NoError(t, err) + require.Empty(t, entitlements, "gateway projection must omit unacknowledged consent") + stamp, err := json.Marshal(jobs.ModuleAuthority{PrincipalID: i.principal, OrganizationID: i.org, + InstallationID: i.installation.Id, InstallationRevision: 1, BindingID: i.binding}.Attributes()) + if err != nil { + return err + } + var allowed bool + err = storetx.Tx(ctx).QueryRow(ctx, `SELECT public.module_job_authority_current($1::jsonb)`, stamp).Scan(&allowed) + require.NoError(t, err) + require.False(t, allowed) + return nil + })) + err = testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + stamp, err := json.Marshal(jobs.ModuleAuthority{PrincipalID: i.principal, OrganizationID: i.org, + InstallationID: i.installation.Id, InstallationRevision: 1, BindingID: i.binding}.Attributes()) + if err != nil { + return err + } + _, err = storetx.Tx(ctx).Exec(ctx, `SELECT public.lock_module_job_authority($1::jsonb)`, stamp) + return err + }) + require.ErrorContains(t, err, "revoked_authority") + }) + } +} + +// HOST-INST-002: the install step is the one capability minted to a pending +// installation, so a verifier holds a member of the organization to a pending +// installation exactly as it does to an active one — otherwise the step that +// would acknowledge it can never be delivered. Nothing else is: a failed or +// revoked installation is held by nobody, and no state is held for someone +// outside the organization. +func TestTheSealHoldsAPendingInstallationAndNoEndedOne(t *testing.T) { + const stranger = "00000000-0000-4000-8000-0000000000aa" + for _, tc := range []struct { + state string + held bool + }{ + {"active", true}, + {"pending", true}, + {"failed", false}, + {"revoked", false}, + } { + t.Run(tc.state, func(t *testing.T) { + i := newDynamicInstall(t, "", "", "") + require.NoError(t, testStore.WithOrgTx(testCtx, i.org, func(ctx context.Context) error { + _, err := storetx.Tx(ctx).Exec(ctx, `UPDATE installations SET status=$2, + failure_reason=CASE WHEN $2='failed' THEN 'Example refusal' ELSE NULL END, + revoked_at=CASE WHEN $2='revoked' THEN CURRENT_TIMESTAMP ELSE NULL END WHERE id=$1`, i.installation.Id, tc.state) + return err + })) + revision, err := testStore.WorkContextHeldInstallation(testCtx, i.org, i.owner, i.installation.Id) + if tc.held { + require.NoError(t, err) + require.NotZero(t, revision) + } else { + require.ErrorIs(t, err, business.ErrWorkContextSealUnheld) + } + _, err = testStore.WorkContextHeldInstallation(testCtx, i.org, stranger, i.installation.Id) + require.ErrorIs(t, err, business.ErrWorkContextSealUnheld, "membership is required in every state") + }) + } +} diff --git a/module/services/accounts/code/pkg/infra/installation_review_test.go b/module/services/accounts/code/pkg/infra/installation_review_test.go index 948fccc13..9f93e154f 100644 --- a/module/services/accounts/code/pkg/infra/installation_review_test.go +++ b/module/services/accounts/code/pkg/infra/installation_review_test.go @@ -47,7 +47,7 @@ func TestInstallationDBFenceSerializesPlacementAndNarrowing(t *testing.T) { }} svc, err := business.NewService(store) require.NoError(t, err) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: a.grant}, a.envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{a.principal: a.grant}, a.envelope, nil) svc.SetModuleAuthorityReads(testStore, nil) caller := business.ModuleCaller{PrincipalID: a.principal, BoundOrg: a.org} placed := make(chan error, 1) diff --git a/module/services/accounts/code/pkg/infra/kubernetes_token_review.go b/module/services/accounts/code/pkg/infra/kubernetes_token_review.go index 247aaba0a..119440c5d 100644 --- a/module/services/accounts/code/pkg/infra/kubernetes_token_review.go +++ b/module/services/accounts/code/pkg/infra/kubernetes_token_review.go @@ -11,7 +11,9 @@ import ( "fmt" "io/fs" "net/http" + "net/url" "os" + "path/filepath" "strings" "time" @@ -150,6 +152,9 @@ func newKubernetesHTTPClient(roots *x509.CertPool) *http.Client { // pinned-key trust policy refuses at boot: "I cannot perform this check" and "this // caller failed the check" must not be the same observable. func NewKubernetesClient() (*KubernetesClient, error) { + if dir := strings.TrimSpace(os.Getenv(LocalClusterEnvironment)); dir != "" { + return localClusterClient(dir, InCluster()) + } host := strings.TrimSpace(os.Getenv("KUBERNETES_SERVICE_HOST")) port := strings.TrimSpace(os.Getenv("KUBERNETES_SERVICE_PORT")) if host == "" || port == "" { @@ -163,6 +168,57 @@ func NewKubernetesClient() (*KubernetesClient, error) { return kubernetesClientFrom("https://"+host+":"+port, authority, inClusterTokenPath, inClusterNamespacePath) } +// LocalClusterEnvironment names a directory that stands in for the kubelet's +// projection when this host is NOT a pod: a developer's run of the composition +// on a laptop, where the run itself is the cluster — it serves the TokenReview +// and pod reads, issues each workload its projected token, and writes into this +// directory what a kubelet would project into the host's pod: +// +// server the API server's https origin +// ca.crt the one certificate authority that API server serves under +// token this service's own token +// namespace the namespace this service runs in +// +// The in-cluster paths stay fixed and unconfigurable, for the reason given +// above them, and this does not change that: IN A POD THIS VARIABLE IS REFUSED, +// not ignored. A pod's identity is the kubelet's to project, so a pod that +// names a stand-in is a pod whose identity somebody is trying to point +// elsewhere, and the host says so at boot rather than serving. Outside a pod +// there is no kubelet and nothing to redirect: the stand-in is the only cluster +// there is, and the host's local trust profile is still proven against it +// (BindToAuthority) before a single delivery is admitted. +const LocalClusterEnvironment = "SOLUTION_HOST_LOCAL_CLUSTER" + +// ErrLocalClusterInAPod refuses a stand-in named inside a real cluster. +var ErrLocalClusterInAPod = errors.New("a pod may not name a local stand-in for its cluster") + +// localClusterClient builds the client from a stand-in directory. inPod is +// InCluster's answer, passed in so a test drives the refusal through this code. +func localClusterClient(dir string, inPod bool) (*KubernetesClient, error) { + if inPod { + return nil, fmt.Errorf("%w: %s is set and this process carries what a kubelet puts in a pod; "+ + "a pod reads its identity from the kubelet's projection and from nowhere else", + ErrLocalClusterInAPod, LocalClusterEnvironment) + } + if !filepath.IsAbs(dir) { + return nil, fmt.Errorf("%w: %s must be an absolute directory, got %q", ErrKubernetesUnavailable, LocalClusterEnvironment, dir) + } + raw, err := os.ReadFile(filepath.Join(dir, "server")) + if err != nil { + return nil, fmt.Errorf("%w: cannot read the local cluster's API server address: %w", ErrKubernetesUnavailable, err) + } + server, err := url.Parse(strings.TrimSpace(string(raw))) + if err != nil || server.Scheme != "https" || server.Host == "" || server.Port() == "" || server.User != nil || + (server.Path != "" && server.Path != "/") || server.RawQuery != "" || server.Fragment != "" { + return nil, fmt.Errorf("%w: the local cluster's API server address must be an https origin and nothing more", ErrKubernetesUnavailable) + } + authority, err := os.ReadFile(filepath.Join(dir, "ca.crt")) + if err != nil { + return nil, fmt.Errorf("%w: cannot read the local cluster's CA: %w", ErrKubernetesUnavailable, err) + } + return kubernetesClientFrom("https://"+server.Host, authority, filepath.Join(dir, "token"), filepath.Join(dir, "namespace")) +} + // kubernetesClientFrom is the construction NewKubernetesClient performs, over // pieces it has already located — separated so a test builds its client through // the same code (the same roots, the same redirect refusal) rather than a diff --git a/module/services/accounts/code/pkg/infra/local_cluster_test.go b/module/services/accounts/code/pkg/infra/local_cluster_test.go new file mode 100644 index 000000000..b940f75fc --- /dev/null +++ b/module/services/accounts/code/pkg/infra/local_cluster_test.go @@ -0,0 +1,115 @@ +package infra + +import ( + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/x509" + "crypto/x509/pkix" + "encoding/pem" + "errors" + "math/big" + "os" + "path/filepath" + "testing" + "time" +) + +func testCertificateAuthorityPEM(t *testing.T) string { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + t.Fatal(err) + } + template := &x509.Certificate{ + SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "local stand-in"}, + NotBefore: time.Now().Add(-time.Minute), NotAfter: time.Now().Add(time.Hour), + IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign, + } + der, err := x509.CreateCertificate(rand.Reader, template, template, key.Public(), key) + if err != nil { + t.Fatal(err) + } + return string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})) +} + +// A stand-in for the kubelet's projection is a developer's run, and only that: +// the same variable inside a pod is a refusal at boot, never an alternative +// identity. These drive the refusal and the acceptance through the production +// constructors. + +func writeLocalCluster(t *testing.T, server string) string { + t.Helper() + dir := t.TempDir() + for name, content := range map[string]string{ + "server": server, "ca.crt": testCertificateAuthorityPEM(t), "token": "token", "namespace": "host\n", + } { + if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil { + t.Fatal(err) + } + } + return dir +} + +func TestAPodMayNotNameALocalStandInForItsCluster(t *testing.T) { + dir := writeLocalCluster(t, "https://127.0.0.1:6443") + if _, err := localClusterClient(dir, true); !errors.Is(err, ErrLocalClusterInAPod) { + t.Fatalf("a pod naming a stand-in was not refused: %v", err) + } + client, err := localClusterClient(dir, false) + if err != nil { + t.Fatalf("a process outside a pod was refused its stand-in: %v", err) + } + if client.endpoint != "https://127.0.0.1:6443" || client.tokenPath != filepath.Join(dir, "token") || client.namespacePath != filepath.Join(dir, "namespace") { + t.Fatalf("the client reads %q, %q, %q", client.endpoint, client.tokenPath, client.namespacePath) + } +} + +func TestALocalStandInIsAnHTTPSOriginInAnAbsoluteDirectory(t *testing.T) { + for name, server := range map[string]string{ + "plaintext": "http://127.0.0.1:6443", + "a path": "https://127.0.0.1:6443/api", + "credentials": "https://user@127.0.0.1:6443", + "no port": "https://127.0.0.1", + "a query": "https://127.0.0.1:6443?x=1", + "empty": "", + } { + t.Run(name, func(t *testing.T) { + if _, err := localClusterClient(writeLocalCluster(t, server), false); !errors.Is(err, ErrKubernetesUnavailable) { + t.Fatalf("server %q was accepted: %v", server, err) + } + }) + } + if _, err := localClusterClient("relative/dir", false); !errors.Is(err, ErrKubernetesUnavailable) { + t.Fatalf("a relative directory was accepted: %v", err) + } +} + +func TestAPodMayNotRelocateItsTrustAnchor(t *testing.T) { + if dir, err := trustAnchorDir("", true); err != nil || dir != SolutionHostTrustAnchorPath { + t.Fatalf("a pod naming nothing reads %q, %v", dir, err) + } + if _, err := trustAnchorDir("/tmp/anchor", true); !errors.Is(err, ErrLocalTrustAnchorInAPod) { + t.Fatalf("a pod relocating its anchor was not refused: %v", err) + } + if dir, err := trustAnchorDir("/tmp/anchor", false); err != nil || dir != "/tmp/anchor" { + t.Fatalf("outside a pod the named anchor reads %q, %v", dir, err) + } + if _, err := trustAnchorDir("anchor", false); !errors.Is(err, ErrSolutionHostTrustRootUnavailable) { + t.Fatalf("a relative anchor was accepted: %v", err) + } +} + +func TestARelocatedAnchorMustBeALocalProfile(t *testing.T) { + release := &keyBundleVerifier{} + if _, err := requireLocalProfileOffTheAnchor("/tmp/anchor", release); !errors.Is(err, ErrSolutionHostTrustRootUnavailable) { + t.Fatalf("a release policy read from a relocated anchor was accepted: %v", err) + } + if _, err := requireLocalProfileOffTheAnchor(SolutionHostTrustAnchorPath, release); err != nil { + t.Fatalf("the constant anchor's release policy was refused: %v", err) + } + local := &keyBundleVerifier{local: &SolutionHostLocalProfile{}} + if _, err := requireLocalProfileOffTheAnchor("/tmp/anchor", local); err != nil { + t.Fatalf("a local profile on a relocated anchor was refused: %v", err) + } +} diff --git a/module/services/accounts/code/pkg/infra/module_envelope_witness_test.go b/module/services/accounts/code/pkg/infra/module_envelope_witness_test.go new file mode 100644 index 000000000..d3a0a238c --- /dev/null +++ b/module/services/accounts/code/pkg/infra/module_envelope_witness_test.go @@ -0,0 +1,208 @@ +//go:build !pure + +package infra_test + +import ( + "bytes" + "context" + "encoding/json" + "strings" + "testing" + + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + + "github.com/stretchr/testify/require" +) + +func envelopeJSON(t *testing.T, raw []byte) map[string]any { + t.Helper() + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.UseNumber() + var out map[string]any + require.NoError(t, decoder.Decode(&out)) + return out +} + +func envelopeJSONClone(t *testing.T, doc map[string]any) map[string]any { + t.Helper() + raw, err := json.Marshal(doc) + require.NoError(t, err) + return envelopeJSON(t, raw) +} + +func envelopeJSONAt(t *testing.T, doc map[string]any, revision uint64) []byte { + t.Helper() + doc["revision"] = json.Number(jsonNumber(revision)) + raw, err := json.Marshal(doc) + require.NoError(t, err) + return raw +} + +// Only a pure grant skips the policy-log witness: approved builds added, +// whole principals added, everything else the same value. With no policy log +// wired, a write that needs the witness is refused and writes nothing, so "it +// landed" is the proof a change was applied unwitnessed. +// +// Every shape below is a real narrowing that a comparison by "more JSON means +// more authority" let through: a resource list that appears restricts an +// audience that was unrestricted, a required kind that appears refuses +// artifacts that passed, a unit that gains a field is a different unit, a +// tenant that is set removes every other organization, and two revisions past +// 2^53 are one float. +func TestAnEnvelopeChangeToAGrantedPrincipalNeedsTheWitness(t *testing.T) { + s := newRegistryScenario(t, "rvf") + digest := "sha256:" + strings.Repeat("ab", 32) + base := envelopeJSON(t, s.envelope(t, 1, envelopeOptions{resources: true})) + principal := func(doc map[string]any) map[string]any { + return doc["principals"].(map[string]any)[s.prefix].(map[string]any) + } + grant := func(doc map[string]any) map[string]any { return principal(doc)["grant"].(map[string]any) } + grant(base)["tenant"] = "" + grant(base)["read_audiences"] = map[string]any{ + "docs": map[string]any{"audience": "example-reader", "scopes": []any{map[string]any{"resource_kind": "documents"}}}, + } + grant(base)["artifact_policies"] = map[string]any{ + "p1": map[string]any{ + "schema": "example/v1", + "sources": map[string]any{"src": s.primary.binding}, + "activate": map[string]any{"resource": "artifact", "action": "activate"}, + "run": map[string]any{"resource": "artifact", "action": "run"}, + "contracts": []any{ + map[string]any{"kind": "a", "name": "one", "digest": digest}, + map[string]any{"kind": "b", "name": "two", "digest": digest}, + }, + "required_kinds": []any{"a"}, + }, + } + // The lookup unit sits at revision 2^53 so the float64 comparison can be probed. + principal(base)["bindings"].([]any)[1].(map[string]any)["revision"] = json.Number("9007199254740992") + + current := s.currentRevision(t) + _, err := s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, envelopeJSONAt(t, base, current+1)) + require.NoError(t, err, "the base envelope (witnessed) must be written") + state := base + + // From here the host has no witness at all. + s.delivery.service.SetPolicyLog(nil, nil) + + opAudience := func(doc map[string]any) map[string]any { + return grant(doc)["operation_audiences"].(map[string]any)["run"].(map[string]any) + } + cases := []struct { + name string + mutate func(doc map[string]any) + }{ + {"a content resource removed", func(doc map[string]any) { grant(doc)["resources"] = []any{} }}, + // Even an addition inside a section: the section is compared for + // equality, never for "at least as much". + {"a queue added to a granted principal", func(doc map[string]any) { grant(doc)["queues"] = []any{"reports.render", "reports.extra"} }}, + {"lookup scope restricted from every resource to one id (resource_ids added)", func(doc map[string]any) { + opAudience(doc)["lookup_scopes"].([]any)[0].(map[string]any)["resource_ids"] = []any{"p1"} + }}, + {"lookup and invoke scopes restricted from every resource to one id (resource_ids added)", func(doc map[string]any) { + opAudience(doc)["lookup_scopes"].([]any)[0].(map[string]any)["resource_ids"] = []any{"p1"} + opAudience(doc)["invoke_scopes"].([]any)[0].(map[string]any)["resource_ids"] = []any{"p1"} + }}, + {"read audience restricted from every resource to one id (resource_ids added)", func(doc map[string]any) { + grant(doc)["read_audiences"].(map[string]any)["docs"].(map[string]any)["scopes"].([]any)[0].(map[string]any)["resource_ids"] = []any{"d1"} + }}, + {"artifact policy gains a REQUIRED contract kind", func(doc map[string]any) { + grant(doc)["artifact_policies"].(map[string]any)["p1"].(map[string]any)["required_kinds"] = []any{"a", "b"} + }}, + {"an admitted unit is replaced by a different unit under the same id (queue set)", func(doc map[string]any) { + principal(doc)["bindings"].([]any)[0].(map[string]any)["queue"] = "reports.render" + }}, + {"an admitted unit's revision moves 2^53 -> 2^53+1", func(doc map[string]any) { + principal(doc)["bindings"].([]any)[1].(map[string]any)["revision"] = json.Number("9007199254740993") + }}, + {"tenancy pinned: shared (every organization) -> one organization", func(doc map[string]any) { grant(doc)["tenant"] = s.tenant }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + next := envelopeJSONClone(t, state) + tc.mutate(next) + current := s.currentRevision(t) + raw := envelopeJSONAt(t, next, current+1) + _, parseErr := business.ParseModuleEnvelope(raw) + require.NoError(t, parseErr, "the document must be a valid envelope") + _, err := s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, raw) + require.ErrorIs(t, err, business.ErrPolicyLogUnreachable, "the change must need the witness") + require.Equal(t, current, s.currentRevision(t), "a refused write must write nothing") + }) + } + + // And the pure grant: a build approved beside the ones already approved. + t.Run("an approved build added lands with no witness", func(t *testing.T) { + next := envelopeJSONClone(t, state) + builds, _ := next["approved_builds"].([]any) + next["approved_builds"] = append(builds, "sha256:"+strings.Repeat("cd", 32)) + current := s.currentRevision(t) + _, err := s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, envelopeJSONAt(t, next, current+1)) + require.NoError(t, err) + require.Equal(t, current+1, s.currentRevision(t)) + state = next + }) + t.Run("an approved build removed needs the witness", func(t *testing.T) { + next := envelopeJSONClone(t, state) + builds := next["approved_builds"].([]any) + next["approved_builds"] = builds[:len(builds)-1] + current := s.currentRevision(t) + _, err := s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, envelopeJSONAt(t, next, current+1)) + require.ErrorIs(t, err, business.ErrPolicyLogUnreachable) + }) + s.delivery.service.SetPolicyLog(&productionPathPolicyLog{}, testStore) +} + +// The installation half: an envelope that would take an organization's +// installation away is a narrowing, and with no witness it is refused and the +// installation stays as it was. The classification reads installations on the +// control plane; read anywhere else a row policy hides them all, and the write +// revoked an active installation unwitnessed. +func TestAnEnvelopeThatWouldRevokeAnInstallationNeedsTheWitness(t *testing.T) { + s := newRegistryScenario(t, "rvi") + orgB := seedOrg(t, s.admin) + require.NoError(t, testStore.As(business.Identity{OrgID: orgB}).AddOrgMember(testCtx, s.admin, "admin")) + shared := envelopeJSON(t, s.envelope(t, 1, envelopeOptions{resources: true})) + grant := func(doc map[string]any) map[string]any { + return doc["principals"].(map[string]any)[s.prefix].(map[string]any)["grant"].(map[string]any) + } + grant(shared)["tenant"] = "" + current := s.currentRevision(t) + _, err := s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, envelopeJSONAt(t, shared, current+1)) + require.NoError(t, err) + revision := current + 1 + s.deliverPresence(t, s.primary, 1, revision) + s.pass(t) + s.deliverAuthority(t, s.authority(t, s.primary, 1, revision, false, s.grantsTo(s.primary))) + s.pass(t) + _, present := s.entry() + require.True(t, present) + + status := func(org, id string) gen.InstallationStatus { + var out gen.InstallationStatus + require.NoError(t, testStore.WithOrgTx(testCtx, org, func(ctx context.Context) error { + row, _, err := testStore.GetInstallation(ctx, org, id) + if err == nil { + out = row.Status + } + return err + })) + return out + } + + active := s.install(t, orgB) // acknowledged: ACTIVE + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, status(orgB, active.Id)) + _, err = s.delivery.service.AuthorizeModuleCapability(testCtx, s.caller(orgB)) + require.NoError(t, err, "org B's installation is usable under the shared tenancy") + + s.delivery.service.SetPolicyLog(nil, nil) + pinned := envelopeJSONClone(t, shared) + grant(pinned)["tenant"] = s.tenant + current = s.currentRevision(t) + _, err = s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, envelopeJSONAt(t, pinned, current+1)) + require.ErrorIs(t, err, business.ErrPolicyLogUnreachable) + require.Equal(t, gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE, status(orgB, active.Id), + "an unwitnessed envelope write must not revoke an installation") + s.delivery.service.SetPolicyLog(&productionPathPolicyLog{}, testStore) +} diff --git a/module/services/accounts/code/pkg/infra/module_principal_registry_test.go b/module/services/accounts/code/pkg/infra/module_principal_registry_test.go index 635a6613b..dd32182f2 100644 --- a/module/services/accounts/code/pkg/infra/module_principal_registry_test.go +++ b/module/services/accounts/code/pkg/infra/module_principal_registry_test.go @@ -378,7 +378,7 @@ func TestDeliveringAuthorityChangesTheRegistryWithNoRestart(t *testing.T) { grant, _ := s.entry() require.Equal(t, s.prefix, grant.Prefix, "the principal is the module's own name") require.Equal(t, s.tenant, grant.Tenant) - require.Equal(t, business.ModuleWorkload{ServiceAccount: "api", Namespace: s.primary.namespace, Container: "api"}, grant.Workload, + require.Equal(t, business.ModuleWorkload{SPIFFEID: s.primary.spiffeID(), ServiceAccount: "api", Namespace: s.primary.namespace, Container: "api"}, grant.Workload, "the workload is the applied presence's, and one the envelope allows") require.Equal(t, []string{"reports.render"}, grant.Queues) require.Equal(t, []string{s.resource}, grant.Resources) @@ -783,5 +783,6 @@ func (s *registryScenario) install(t *testing.T, org string) *gen.Installation { OrgID: org, TargetID: target, AgentIdentifier: "acme.example/registry:1.0.0", RoleID: role, }) require.NoError(t, err) + acknowledgeInstallation(t, installation) return installation } diff --git a/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal.go b/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal.go index 645b978ad..b99990ff1 100644 --- a/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal.go +++ b/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal.go @@ -7,6 +7,7 @@ import ( "fmt" "accounts/pkg/business" + "accounts/pkg/infra/storetx" "github.com/google/uuid" "github.com/jackc/pgx/v5" @@ -34,7 +35,7 @@ func (s *PostgresStore) AppendActorChainHop( return nil, errors.New("actor chain hop requires id") } var stored *business.ActorChainHop - err := s.WithOrgTx(ctx, hop.OrgID, func(ctx context.Context) error { + appendHop := func(ctx context.Context) error { q := s.getQueryExecutor(ctx) prevHash := "" @@ -88,7 +89,16 @@ func (s *PostgresStore) AppendActorChainHop( } stored = row return nil - }) + } + var err error + if storetx.Tx(ctx) != nil { + // The leased installation mint keeps its consent locks through signing + // and this append. Reuse that transaction instead of borrowing a pool + // connection while holding its locks. SQL remains explicitly org-bound. + err = appendHop(ctx) + } else { + err = s.WithOrgTx(ctx, hop.OrgID, appendHop) + } if err != nil { return nil, err } diff --git a/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal_test.go b/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal_test.go index 128f7948f..806688afd 100644 --- a/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_actor_chain_journal_test.go @@ -4,6 +4,7 @@ package infra_test import ( "context" + "errors" "testing" "github.com/stretchr/testify/require" @@ -56,6 +57,29 @@ func TestActorChainJournal_AppendContentAddressesAndIsLive(t *testing.T) { require.False(t, revoked) } +func TestActorChainJournal_AppendRollsBackWithAuthorityTransaction(t *testing.T) { + owner := seedUser(t) + orgID := seedOrg(t, owner) + agent := seedAgentPrincipal(t, orgID, "acme.test/agent:0.1.0") + in := newRootHopInput(orgID, owner, agent.ID) + rejected := errors.New("authority changed before commit") + err := testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + hop, err := testStore.AppendActorChainHop(ctx, in) + require.NoError(t, err) + require.Equal(t, in.ID, hop.ID) + return rejected + }) + require.ErrorIs(t, err, rejected) + require.NoError(t, testStore.WithOrgTx(testCtx, orgID, func(ctx context.Context) error { + var exists bool + err := storetx.Tx(ctx).QueryRow(ctx, + `SELECT EXISTS (SELECT 1 FROM actor_chain_journal WHERE org_id=$1 AND id=$2)`, orgID, in.ID).Scan(&exists) + require.NoError(t, err) + require.False(t, exists, "a rejected mint must not leave a journal hop") + return nil + })) +} + // A retried append of the SAME hop id is absorbed (crash-safety). This is // store-level id idempotency, not issuance-level dedup — production always mints // a fresh hop id per issuance, so this path only guards a retry of one append. diff --git a/module/services/accounts/code/pkg/infra/postgres_executable_artifacts_test.go b/module/services/accounts/code/pkg/infra/postgres_executable_artifacts_test.go index 07f04cdd7..1ad01ea12 100644 --- a/module/services/accounts/code/pkg/infra/postgres_executable_artifacts_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_executable_artifacts_test.go @@ -48,7 +48,7 @@ func TestExecutableArtifactPostgresConsentRevocationAndTenantFloor(t *testing.T) envelope, err = testStore.LatestModuleEnvelopeRevision(ctx) return err })) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, envelope, nil) request := business.ExecutableArtifactRequest{Installation: installation.Id, Policy: "content", Identity: business.ExecutableArtifactIdentity{Schema: policy.Schema, Source: "acme.example.solution", Subject: []byte(`{"version":9007199254740993,"dependencies":[]}`), Contracts: policy.Contracts, ExpectedRevision: 9007199254740993}} decide := func(action string) (*business.ExecutableArtifactApproval, error) { return svc.DecideExecutableArtifact(auth.WithVerifiedDatabaseIdentity(testCtx, owner, org), caller, org, owner, "example", action, request) @@ -119,9 +119,9 @@ func TestExecutableArtifactPostgresConsentRevocationAndTenantFloor(t *testing.T) // A removed policy and unhealthy installation must not trap old consent. withoutPolicy := grant withoutPolicy.ArtifactPolicies = nil - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: withoutPolicy}, envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: withoutPolicy}, envelope, nil) _, _, err = svc.RevokeApprovedExecutableArtifact(auth.WithVerifiedDatabaseIdentity(testCtx, owner, org), caller, org, owner, "example", installation.Id, first.ID, func(business.ArtifactPermission) error { return nil }) - svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, envelope) + svc.InstallDerivedModulePrincipals(business.ModulePrincipalRegistry{principal: grant}, envelope, nil) require.NoError(t, err) _, err = decide("authorize") require.ErrorContains(t, err, "revoked") diff --git a/module/services/accounts/code/pkg/infra/postgres_installation_jobs.go b/module/services/accounts/code/pkg/infra/postgres_installation_jobs.go new file mode 100644 index 000000000..a45a3b4f9 --- /dev/null +++ b/module/services/accounts/code/pkg/infra/postgres_installation_jobs.go @@ -0,0 +1,168 @@ +package infra + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + "accounts/pkg/business" + gen "accounts/pkg/gen/saas/accounts/v1" + jobsv1 "accounts/pkg/gen/saas/jobs/v1" + "accounts/pkg/infra/internal/txbind" + "accounts/pkg/jobs" + + "github.com/jackc/pgx/v5" +) + +func (s *PostgresStore) InstallationStepLeaseValid(ctx context.Context, lease *jobsv1.JobLeaseReference) (bool, error) { + var valid bool + // The gateway's headless consumer has no tenant context; this fixed-queue + // definer checks only its exact lease and exposes no job-table read access. + err := s.WithControlPlane(ctx, func(ctx context.Context) error { + return s.getQueryExecutor(ctx).QueryRow(ctx, `SELECT public.installation_step_lease($1,$2,$3)`, lease.GetJobId(), lease.GetWorkerId(), lease.GetLeaseToken()).Scan(&valid) + }) + return valid, err +} + +func (s *PostgresStore) WithInstallationStepAuthority(ctx context.Context, lease *jobsv1.JobLeaseReference, envelope uint64, use func(context.Context, *business.InstallationStepAuthority) error) error { + // Setup resolves the target and registry across tenants from a verified + // lease, then holds that installation's authority through signing. + return s.WithControlPlane(ctx, func(ctx context.Context) error { + executor := s.getQueryExecutor(ctx) + // The queue-specific lease check never grants table access to this pool. + if err := checkInstallationEnvelope(ctx, executor, envelope); err != nil { + return err + } + var valid bool + err := executor.QueryRow(ctx, `SELECT public.installation_step_lease($1,$2,$3)`, lease.GetJobId(), lease.GetWorkerId(), lease.GetLeaseToken()).Scan(&valid) + if err != nil { + return err + } + if !valid { + return errors.New("installation step lease expired") + } + if _, err := executor.Exec(ctx, `SELECT 1 FROM solution_targets WHERE id IN + (SELECT target_id FROM installations WHERE install_job_id=$1) ORDER BY id FOR SHARE`, lease.GetJobId()); err != nil { + return err + } + a := &business.InstallationStepAuthority{} + var raw []byte + err = executor.QueryRow(ctx, `SELECT i.id::text,i.org_id::text,i.revision,i.target_id::text, + t.solution_id,i.registry_binding_id,i.registry_principal_id::text,i.root_scope_node_id::text,i.scope + FROM installations i JOIN solution_targets t ON t.id=i.target_id + WHERE i.install_job_id=$1 AND i.status IN ('pending','active') AND t.closed_generation IS NULL + FOR SHARE OF i`, lease.GetJobId()).Scan(&a.InstallationID, &a.OrganizationID, &a.Revision, &a.TargetID, &a.RouteAlias, &a.BindingID, &a.PrincipalID, &a.RootID, &raw) + if err != nil { + return err + } + if err := json.Unmarshal(raw, &a.Scope); err != nil { + return err + } + a.Facts, err = resolveInstallationAuthority(ctx, executor, a.OrganizationID, a.InstallationID, nil, true) + if err != nil { + return err + } + rows, err := executor.Query(ctx, `SELECT DISTINCT rp.resource,rp.action FROM scope_grants g + JOIN role_permissions rp ON rp.role_id=g.role_id JOIN scope_nodes n ON n.id=$3 + WHERE g.org_id=$1 AND g.subject_kind='principal' AND g.subject_id=$2 + AND g.scope_path @> n.scope_path AND (g.expires_at IS NULL OR g.expires_at>now()) + ORDER BY rp.resource,rp.action`, a.OrganizationID, a.Facts.Actor.ID, a.RootID) + if err != nil { + return err + } + permissions, err := pgx.CollectRows(rows, func(row pgx.CollectableRow) (business.WorkContextPermission, error) { + var p business.WorkContextPermission + err := row.Scan(&p.ResourceKind, &p.Action) + return p, err + }) + if err != nil { + return err + } + if len(permissions) == 0 { + return errors.New("installation standing grant is absent") + } + a.StandingPermissions = permissions + return use(ctx, a) + }) +} + +// EnqueueInstallationStep attaches the outbox job in the lifecycle transaction. +// The row lock fences retries and revision changes; no credential is persisted. +func (s *PostgresStore) EnqueueInstallationStep(ctx context.Context, orgID, id, run string) error { + bound, ok := txbind.Lookup(ctx) + if !ok { + return jobs.ErrTransactionRequired + } + if bound.Pool != txbind.ControlPlane && (bound.Pool != txbind.Request || bound.OrgID != orgID) { + return errors.New("installation enqueue requires its organization's transaction or the control plane") + } + executor := s.getQueryExecutor(ctx) + var revision uint64 + if err := executor.QueryRow(ctx, `SELECT revision FROM installations + WHERE org_id=$1 AND id=$2 AND status IN ('pending','active') FOR UPDATE`, orgID, id).Scan(&revision); err != nil { + return err + } + req, err := business.NewInstallationStepJob(business.InstallationStep{InstallationID: id, OrganizationID: orgID, Revision: revision}, run) + if err != nil { + return err + } + prepared, err := prepareJobEnqueue(req) + if err != nil { + return err + } + // Setup is host work for pending consent, not an active module's enqueue. + // The installation lock above is its authority and survives through commit. + // Like the webhook relay, use the dedicated enqueue_job_message path with + // tenant scope on the existing transaction. The control-plane login cannot + // assume app_tenant; its declared job-worker role can enqueue tenant work. + // No callback or product-table access runs while that role is selected. + if bound.Pool == txbind.ControlPlane { + if _, err := executor.Exec(ctx, "SET LOCAL ROLE app_job_worker"); err != nil { + return err + } + if err := bindRequestScope(ctx, bound.Tx, orgID, ""); err != nil { + return err + } + } + job, err := insertPreparedJob(ctx, executor, prepared) + if err != nil { + return err + } + if bound.Pool == txbind.ControlPlane { + if err := bindRequestScope(ctx, bound.Tx, "", ""); err != nil { + return err + } + if _, err := executor.Exec(ctx, "SET LOCAL ROLE "+controlPlaneDatabaseRole); err != nil { + return err + } + } + _, err = executor.Exec(ctx, `UPDATE installations SET install_job_id=$3, install_attempts=0 WHERE org_id=$1 AND id=$2`, orgID, id, job.GetJobId()) + return err +} + +func (s *PostgresStore) RetryInstallation(ctx context.Context, orgID, id string) (*gen.Installation, uint64, error) { + // Match target withdrawal's target -> installation lock order. + executor := s.getQueryExecutor(ctx) + if _, err := executor.Exec(ctx, `SELECT 1 FROM solution_targets WHERE id IN + (SELECT target_id FROM installations WHERE org_id=$1 AND id=$2) FOR SHARE`, orgID, id); err != nil { + return nil, 0, err + } + i, err := scanInstallation(executor.QueryRow(ctx, `UPDATE installations + SET status='pending', failure_reason=NULL, install_attempts=0 + WHERE org_id=$1 AND id=$2 AND status='failed' RETURNING `+installationColumns, orgID, id)) + if errors.Is(err, pgx.ErrNoRows) { + return nil, 0, business.NewStoreError(fmt.Errorf("only a failed installation may be retried"), business.ErrTypeConflict) + } + if err != nil { + return nil, 0, err + } + if err := s.EnqueueInstallationStep(ctx, orgID, id, business.NewIDString()); err != nil { + return nil, 0, err + } + var revision uint64 + if err := executor.QueryRow(ctx, `SELECT revision FROM installations WHERE org_id=$1 AND id=$2`, orgID, id).Scan(&revision); err != nil { + return nil, 0, err + } + return i, revision, nil +} diff --git a/module/services/accounts/code/pkg/infra/postgres_installation_scope.go b/module/services/accounts/code/pkg/infra/postgres_installation_scope.go index 03a30b205..62f3f2320 100644 --- a/module/services/accounts/code/pkg/infra/postgres_installation_scope.go +++ b/module/services/accounts/code/pkg/infra/postgres_installation_scope.go @@ -65,6 +65,16 @@ func (s *PostgresStore) NarrowInstallationScope(ctx context.Context, orgID, id s } // The trigger owns revision increments and cancellation for ALL writers, // including target withdrawal and organization deletion. - _, err = executor.Exec(ctx, `UPDATE public.installations SET scope=$3::jsonb WHERE id=$1::uuid AND org_id=$2::uuid AND scope IS DISTINCT FROM $3::jsonb`, id, orgID, encoded) - return err + tag, err := executor.Exec(ctx, `UPDATE public.installations SET scope=$3::jsonb WHERE id=$1::uuid AND org_id=$2::uuid AND scope IS DISTINCT FROM $3::jsonb`, id, orgID, encoded) + if err != nil || tag.RowsAffected() == 0 { + return err + } + var kind string + if err := executor.QueryRow(ctx, `SELECT kind FROM installations WHERE id=$1 AND org_id=$2`, id, orgID).Scan(&kind); err != nil { + return err + } + if kind == "module" { + return nil + } + return s.EnqueueInstallationStep(ctx, orgID, id, "narrow") } diff --git a/module/services/accounts/code/pkg/infra/postgres_installations.go b/module/services/accounts/code/pkg/infra/postgres_installations.go index 0ff550dd3..d7b904c74 100644 --- a/module/services/accounts/code/pkg/infra/postgres_installations.go +++ b/module/services/accounts/code/pkg/infra/postgres_installations.go @@ -29,18 +29,18 @@ import ( const installationColumns = `id::text, org_id::text, agent_principal_id::text, owner_principal_id::text, co_owner_principal_ids::text[], root_scope_node_id::text, status, created_at, revoked_at, - COALESCE(target_id::text, ''), COALESCE(revoked_reason, '')` + COALESCE(target_id::text, ''), COALESCE(revoked_reason, ''), COALESCE(failure_reason, '')` func scanInstallation(row rowScanner) (*gen.Installation, error) { var ( id, orgID, agentID, ownerID, rootNodeID, status string - targetID, revokedReason string + targetID, revokedReason, failureReason string coOwners []string createdAt time.Time revokedAt *time.Time ) if err := row.Scan(&id, &orgID, &agentID, &ownerID, &coOwners, - &rootNodeID, &status, &createdAt, &revokedAt, &targetID, &revokedReason); err != nil { + &rootNodeID, &status, &createdAt, &revokedAt, &targetID, &revokedReason, &failureReason); err != nil { return nil, err } out := &gen.Installation{ @@ -54,6 +54,7 @@ func scanInstallation(row rowScanner) (*gen.Installation, error) { CreatedAt: timestamppb.New(createdAt), TargetId: targetID, RevokedReason: revokedReason, + FailureReason: failureReason, } if revokedAt != nil { out.RevokedAt = timestamppb.New(*revokedAt) @@ -63,6 +64,10 @@ func scanInstallation(row rowScanner) (*gen.Installation, error) { func installationStatusFromColumn(status string) gen.InstallationStatus { switch status { + case "pending": + return gen.InstallationStatus_INSTALLATION_STATUS_PENDING + case "failed": + return gen.InstallationStatus_INSTALLATION_STATUS_FAILED case "active": return gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE case "revoked": @@ -102,8 +107,10 @@ func firstEligibleOwner(ctx context.Context, executor QueryExecutor, orgID strin } // InstallSolution composes the agent principal, solution scope node, standing -// grant, and installation row inside the caller's WithOrgTx. Idempotent per -// (org, solution): a re-install of an active solution returns the existing row. +// grant, pending installation and durable step inside the caller's WithOrgTx. +// Only acknowledged consent is usable. An active identical install is idempotent; +// pending/failed consent requires its existing lifecycle, never a second install. +// Composed modules retain their installation path; the callback is solution-owned. func (s *PostgresStore) InstallSolution(ctx context.Context, params *business.InstallSolutionParams) (*gen.Installation, error) { w := wool.Get(ctx).In("InstallSolution", wool.Field("org_id", params.OrgID), @@ -138,9 +145,12 @@ func (s *PostgresStore) InstallSolution(ctx context.Context, params *business.In // than silently ignored — a changed ceiling is applied by uninstall + reinstall. existing, err := scanInstallation(executor.QueryRow(ctx, `SELECT `+installationColumns+` FROM installations - WHERE org_id = $1 AND target_id = $2::uuid AND status = 'active'`, + WHERE org_id = $1 AND target_id = $2::uuid AND status <> 'revoked'`, params.OrgID, params.TargetID)) if err == nil { + if existing.Status != gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE { + return nil, installerConflict("installation already exists; retry a failed installation or uninstall it") + } if e := reconcileExistingInstallation(ctx, executor, existing, params); e != nil { return nil, e } @@ -192,7 +202,7 @@ func (s *PostgresStore) InstallSolution(ctx context.Context, params *business.In INSERT INTO installations (id, org_id, agent_principal_id, target_id, owner_principal_id, co_owner_principal_ids, root_scope_node_id, status, installer_principal_id, kind, registry_binding_id, registry_principal_id, scope) - VALUES (gen_random_uuid(), $1, $2, $3::uuid, $4, $5, $6, 'active', NULLIF($7, '')::uuid, COALESCE(NULLIF($8,''),'solution'), NULLIF($9,''), NULLIF($10, '')::uuid, $11::jsonb) + VALUES (gen_random_uuid(), $1, $2, $3::uuid, $4, $5, $6, CASE WHEN $8='module' THEN 'active' ELSE 'pending' END, NULLIF($7, '')::uuid, COALESCE(NULLIF($8,''),'solution'), NULLIF($9,''), NULLIF($10, '')::uuid, $11::jsonb) RETURNING `+installationColumns, params.OrgID, agentID, params.TargetID, params.OwnerPrincipalID, stringArray(params.CoOwnerPrincipalIDs), nodeID, params.InstallerPrincipalID, @@ -211,6 +221,11 @@ func (s *PostgresStore) InstallSolution(ctx context.Context, params *business.In } return nil, w.Wrapf(err, "failed to insert installation") } + if installation.Status == gen.InstallationStatus_INSTALLATION_STATUS_PENDING { + if err := s.EnqueueInstallationStep(ctx, params.OrgID, installation.Id, "initial"); err != nil { + return nil, err + } + } return installation, nil } @@ -369,6 +384,10 @@ func sameStringSet(a, b []string) bool { // column value an enum selects, or "" for UNSPECIFIED, which selects every row. func installationStatusColumn(status gen.InstallationStatus) string { switch status { + case gen.InstallationStatus_INSTALLATION_STATUS_PENDING: + return "pending" + case gen.InstallationStatus_INSTALLATION_STATUS_FAILED: + return "failed" case gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE: return "active" case gen.InstallationStatus_INSTALLATION_STATUS_REVOKED: @@ -474,7 +493,7 @@ func installationHealthFromLabel(label string) gen.InstallationHealth { const installationListingColumns = `i.id::text, i.org_id::text, i.agent_principal_id::text, i.owner_principal_id::text, i.co_owner_principal_ids::text[], i.root_scope_node_id::text, i.status, i.created_at, i.revoked_at, - COALESCE(i.target_id::text, ''), COALESCE(i.revoked_reason, ''), ` + installationHealthLabel + COALESCE(i.target_id::text, ''), COALESCE(i.revoked_reason, ''), COALESCE(i.failure_reason, ''), ` + installationHealthLabel // withHealthLabel scans one listing row: the installation's own columns through // scanInstallation, so the wire mapping has one definition, then the label. @@ -691,8 +710,8 @@ func (s *PostgresStore) UninstallSolution(ctx context.Context, orgID, installati } revoked, err := scanInstallation(executor.QueryRow(ctx, ` - UPDATE installations SET status = 'revoked', revoked_at = CURRENT_TIMESTAMP - WHERE id = $1 AND org_id = $2 AND status = 'active' + UPDATE installations SET status = 'revoked', revoked_at = CURRENT_TIMESTAMP, failure_reason = NULL + WHERE id = $1 AND org_id = $2 AND status <> 'revoked' RETURNING `+installationColumns, installationID, orgID, )) @@ -710,7 +729,7 @@ func (s *PostgresStore) ResolveInstallationAuthority(ctx context.Context, orgID, var facts *business.InstallationAuthorityFacts err := s.WithControlPlane(ctx, func(ctx context.Context) error { var resolveErr error - facts, resolveErr = resolveInstallationAuthority(ctx, s.getQueryExecutor(ctx), orgID, installationID, permissions) + facts, resolveErr = resolveInstallationAuthority(ctx, s.getQueryExecutor(ctx), orgID, installationID, permissions, false) return resolveErr }) if err != nil { @@ -719,7 +738,7 @@ func (s *PostgresStore) ResolveInstallationAuthority(ctx context.Context, orgID, return facts, nil } -func resolveInstallationAuthority(ctx context.Context, executor QueryExecutor, orgID, installationID string, permissions []business.WorkContextPermission) (*business.InstallationAuthorityFacts, error) { +func resolveInstallationAuthority(ctx context.Context, executor QueryExecutor, orgID, installationID string, permissions []business.WorkContextPermission, installationStep bool) (*business.InstallationAuthorityFacts, error) { var ( agentID, ownerID, rootNodeID, rootScopePath string coOwners []string @@ -729,8 +748,8 @@ func resolveInstallationAuthority(ctx context.Context, executor QueryExecutor, o i.co_owner_principal_ids::text[], i.root_scope_node_id::text, n.scope_path::text FROM installations i JOIN scope_nodes n ON n.id = i.root_scope_node_id - WHERE i.id = $1 AND i.org_id = $2 AND i.status = 'active'`, - installationID, orgID, + WHERE i.id = $1 AND i.org_id = $2 AND (i.status = 'active' OR ($3 AND i.status = 'pending'))`, + installationID, orgID, installationStep, ).Scan(&agentID, &ownerID, &coOwners, &rootNodeID, &rootScopePath) if errors.Is(err, pgx.ErrNoRows) { return nil, business.NewStoreError( diff --git a/module/services/accounts/code/pkg/infra/postgres_installations_test.go b/module/services/accounts/code/pkg/infra/postgres_installations_test.go index bcf61a9d8..d19cb04de 100644 --- a/module/services/accounts/code/pkg/infra/postgres_installations_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_installations_test.go @@ -73,6 +73,7 @@ func installFixture(t *testing.T, resourceKind, action string, configure ...func installation, err = testStore.InstallSolution(ctx, params) return err })) + acknowledgeInstallation(t, installation) // The root path is derived server-side from the created node; read it back so // descendant boundaries can be registered beneath the real authority root. rootPath = scopeNodePath(t, orgID, installation.RootScopeNodeId) diff --git a/module/services/accounts/code/pkg/infra/postgres_job_producer.go b/module/services/accounts/code/pkg/infra/postgres_job_producer.go index e0395dbf5..e9b816192 100644 --- a/module/services/accounts/code/pkg/infra/postgres_job_producer.go +++ b/module/services/accounts/code/pkg/infra/postgres_job_producer.go @@ -184,6 +184,12 @@ func enqueuePreparedJob( return nil, err } } + return insertPreparedJob(ctx, executor, job) +} + +// insertPreparedJob shares the database enqueue and idempotency protocol. Its +// caller must already hold the applicable producer authority lock. +func insertPreparedJob(ctx context.Context, executor QueryExecutor, job preparedJobEnqueue) (*jobsv1.EnqueueJobResponse, error) { var ( jobID string storedFingerprint []byte diff --git a/module/services/accounts/code/pkg/infra/postgres_jobs.go b/module/services/accounts/code/pkg/infra/postgres_jobs.go index 39aaa4618..1a0151893 100644 --- a/module/services/accounts/code/pkg/infra/postgres_jobs.go +++ b/module/services/accounts/code/pkg/infra/postgres_jobs.go @@ -417,7 +417,11 @@ const claimJobsSQL = ` FROM job_messages message WHERE message.queue = $1 AND message.state IN ('pending', 'retrying') - AND (NOT (message.attributes ? 'codefly.installation_id') OR public.module_job_authority_current(message.attributes)) + AND (CASE WHEN message.queue='installation-step' AND message.source='saas.installation' + AND message.topic='installation.step' AND message.scope_kind='tenant' + AND message.direction='outbox' + THEN public.installation_step_job_current(message.id, message.attributes) + ELSE NOT (message.attributes ? 'codefly.installation_id') OR public.module_job_authority_current(message.attributes) END) AND (message.consumer_authority IS NULL OR public.module_job_authority_current(message.consumer_authority)) AND ($5::jsonb = 'null'::jsonb OR ( public.module_job_authority_current($5::jsonb) @@ -612,7 +616,10 @@ func cancelRevokedModuleJobs(ctx context.Context, tx pgx.Tx, queue string) error lease_owner=NULL, lease_token=NULL, lease_expires_at=NULL, heartbeat_at=NULL, last_error_code='revoked_authority', last_error_message='installation authority changed' WHERE queue=$1 AND state IN ('pending','processing','retrying') - AND ((attributes ? 'codefly.installation_id' AND NOT public.module_job_authority_current(attributes)) + AND ((CASE WHEN queue='installation-step' AND source='saas.installation' + AND topic='installation.step' AND scope_kind='tenant' AND direction='outbox' + THEN NOT public.installation_step_job_current(id, attributes) + ELSE attributes ? 'codefly.installation_id' AND NOT public.module_job_authority_current(attributes) END) OR (consumer_authority IS NOT NULL AND NOT public.module_job_authority_current(consumer_authority))) RETURNING id) UPDATE public.job_attempts a SET finished_at=clock_timestamp(), outcome='canceled', diff --git a/module/services/accounts/code/pkg/infra/postgres_module_installations.go b/module/services/accounts/code/pkg/infra/postgres_module_installations.go index 735f5a307..f87c8dc68 100644 --- a/module/services/accounts/code/pkg/infra/postgres_module_installations.go +++ b/module/services/accounts/code/pkg/infra/postgres_module_installations.go @@ -55,7 +55,7 @@ func (s *PostgresStore) ReconcileModuleInstallation(ctx context.Context, p *busi if owner != p.InstallerPrincipalID || existing.OwnerPrincipalId != p.OwnerPrincipalID { return nil, installerConflict("installation belongs to another owner; automatic adoption is forbidden") } - if existing.Status != gen.InstallationStatus_INSTALLATION_STATUS_ACTIVE { + if existing.Status == gen.InstallationStatus_INSTALLATION_STATUS_REVOKED { return nil, installerConflict("installation was revoked; explicit lifecycle action is required") } if err = reconcileExistingInstallation(ctx, executor, existing, p); err != nil { @@ -98,7 +98,14 @@ func installerResult(ctx context.Context, executor QueryExecutor, i *gen.Install if len(ids) != 1 || ids[0] == "" { return nil, installerConflict("installation must have exactly one nonexpiring standing grant at its own root; unrelated grants require review") } - return &business.ModuleInstallationResult{State: "ready", Changed: changed, OrganizationID: i.OrgId, PrincipalID: i.AgentPrincipalId, InstallationID: i.Id, ScopeNodeID: i.RootScopeNodeId, GrantID: ids[0]}, nil + state := "ready" + switch i.Status { + case gen.InstallationStatus_INSTALLATION_STATUS_PENDING: + state = "pending" + case gen.InstallationStatus_INSTALLATION_STATUS_FAILED: + state = "failed" + } + return &business.ModuleInstallationResult{State: state, Changed: changed, OrganizationID: i.OrgId, PrincipalID: i.AgentPrincipalId, InstallationID: i.Id, ScopeNodeID: i.RootScopeNodeId, GrantID: ids[0]}, nil } func lockSolutionInstallation(ctx context.Context, executor QueryExecutor, orgID, solution string) error { diff --git a/module/services/accounts/code/pkg/infra/postgres_module_installations_test.go b/module/services/accounts/code/pkg/infra/postgres_module_installations_test.go index 95c58d9b1..be0a852eb 100644 --- a/module/services/accounts/code/pkg/infra/postgres_module_installations_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_module_installations_test.go @@ -68,6 +68,7 @@ func TestModuleInstallationPostgresRepeatConcurrentAndLostResponse(t *testing.T) firstID := "" changes := 0 for out := range results { + require.Equal(t, "pending", out.State, "HOST-INST-002: a queued install is not ready") if firstID == "" { firstID = out.InstallationID } @@ -164,6 +165,23 @@ func TestModuleInstallationPostgresHTTPAuthenticationAndAudit(t *testing.T) { business.ModulePrincipalID("example.api"): {Prefix: "example.api"}, }) installCapabilityFixture(t, svc, p.OrgID, p.OwnerPrincipalID) + // The target being installed has its own declared identity; it has no + // installation yet and must not inherit the installer's capability fixture. + registry := svc.ModulePrincipals() + var envelope uint64 + require.NoError(t, testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + targets, err := testStore.ListAvailableSolutionTargets(ctx, business.AvailableSolutionQuery{TargetID: p.TargetID, Limit: 1}) + if err != nil { + return err + } + require.Len(t, targets, 1) + registry[business.ModulePrincipalID("example-installed")] = business.ModulePrincipalGrant{ + Prefix: "example-installed", Tenant: p.OrgID, PresenceBinding: targets[0].BindingID, + } + envelope, err = testStore.LatestModuleEnvelopeRevision(ctx) + return err + })) + svc.InstallDerivedModulePrincipals(registry, envelope, nil) // The installer proves which module it is with its pod's projected token, // bound to the build its authority approves (#1056). cluster := executiontest.New() @@ -321,6 +339,10 @@ func TestModuleInstallationPostgresHTTPAuthenticationAndAudit(t *testing.T) { require.NoError(t, json.Unmarshal(raw, &repeat)) require.False(t, repeat.Changed) require.Equal(t, result.InstallationID, repeat.InstallationID) + post("verify", request, issued.Token, 409) + installed, _, err := svc.GetInstallation(testCtx, p.OrgID, result.InstallationID) + require.NoError(t, err) + acknowledgeInstallation(t, installed) post("verify", request, issued.Token, 200) require.Nil(t, result.AuthorityReference) require.Nil(t, repeat.AuthorityReference) diff --git a/module/services/accounts/code/pkg/infra/postgres_org_lifecycle.go b/module/services/accounts/code/pkg/infra/postgres_org_lifecycle.go index 3c39f6ed0..df4e6fb1b 100644 --- a/module/services/accounts/code/pkg/infra/postgres_org_lifecycle.go +++ b/module/services/accounts/code/pkg/infra/postgres_org_lifecycle.go @@ -166,7 +166,7 @@ func (s *PostgresStore) RevokeOrgPendingInvitations(ctx context.Context, orgID s // grant deletion, whose trigger locks the organization's source-read revision. func (s *PostgresStore) ActiveInstallationIDs(ctx context.Context, orgID string) ([]string, error) { rows, err := s.getQueryExecutor(ctx).Query(ctx, - `SELECT id FROM installations WHERE org_id = $1 AND status = 'active' ORDER BY id FOR UPDATE`, orgID) + `SELECT id FROM installations WHERE org_id = $1 AND status <> 'revoked' ORDER BY id FOR UPDATE`, orgID) if err != nil { return nil, wool.Get(ctx).In("ActiveInstallationIDs").Wrapf(err, "failed to list installations") } diff --git a/module/services/accounts/code/pkg/infra/postgres_solution_delivery.go b/module/services/accounts/code/pkg/infra/postgres_solution_delivery.go index 600a0072e..543f3baa4 100644 --- a/module/services/accounts/code/pkg/infra/postgres_solution_delivery.go +++ b/module/services/accounts/code/pkg/infra/postgres_solution_delivery.go @@ -162,3 +162,39 @@ func (s *PostgresStore) ListDeliveredDocuments( } var _ business.SolutionDeliveryStore = (*PostgresStore)(nil) + +// CountDeliveredDocuments is how many documents the inbox holds, of every kind. +// +// The inbox is append-only — a generation is inserted once and never rewritten +// or removed — so the count only grows, and it grows exactly when a document a +// registry pass has not read yet arrives. That makes it the marker a registry +// snapshot is compared with: see business.Service.deliveredSince. +func (s *PostgresStore) CountDeliveredDocuments(ctx context.Context) (uint64, error) { + var count int64 + if err := s.getQueryExecutor(ctx).QueryRow(ctx, + `SELECT count(*) FROM public.solution_delivery_documents`).Scan(&count); err != nil { + return 0, err + } + return uint64(count), nil +} + +// CountUnappliedPresence is how many presence documents the inbox holds at a +// generation newer than the one their binding has applied, and that the host +// has not refused. Each is a declaration the binding reconciler has yet to +// apply: until it has, the registry cannot hold the principal it declares. +func (s *PostgresStore) CountUnappliedPresence(ctx context.Context) (uint64, error) { + var count int64 + if err := s.getQueryExecutor(ctx).QueryRow(ctx, ` + SELECT count(*) + FROM (SELECT document_id, max(generation) AS generation + FROM public.solution_delivery_documents + WHERE kind = $1 + GROUP BY document_id) newest + LEFT JOIN public.solution_host_bindings binding ON binding.binding_id = newest.document_id + WHERE COALESCE(binding.applied_generation, 0) < newest.generation + AND binding.pending_reason IS NULL`, + string(business.SolutionDeliveryPresence)).Scan(&count); err != nil { + return 0, err + } + return uint64(count), nil +} diff --git a/module/services/accounts/code/pkg/infra/postgres_solution_entitlements_test.go b/module/services/accounts/code/pkg/infra/postgres_solution_entitlements_test.go index 5d7f79766..f82fa68ae 100644 --- a/module/services/accounts/code/pkg/infra/postgres_solution_entitlements_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_solution_entitlements_test.go @@ -73,6 +73,7 @@ func installSolutionTarget(t *testing.T, orgID, ownerID, roleID, label, targetID }) return err })) + acknowledgeInstallation(t, installation) return installation } diff --git a/module/services/accounts/code/pkg/infra/postgres_solution_targets.go b/module/services/accounts/code/pkg/infra/postgres_solution_targets.go index 838ec9234..8a1aea2b3 100644 --- a/module/services/accounts/code/pkg/infra/postgres_solution_targets.go +++ b/module/services/accounts/code/pkg/infra/postgres_solution_targets.go @@ -172,7 +172,7 @@ func (s *PostgresStore) RevokeInstallationsOfTarget( // Consume the whole result first, then update only the locked batch. locked, err := executor.Query(ctx, ` SELECT id FROM public.installations - WHERE target_id = $1::uuid AND status = 'active' + WHERE target_id = $1::uuid AND status <> 'revoked' ORDER BY id FOR UPDATE`, targetID) if err != nil { return nil, err @@ -183,8 +183,8 @@ func (s *PostgresStore) RevokeInstallationsOfTarget( } rows, err := executor.Query(ctx, ` UPDATE public.installations - SET status = 'revoked', revoked_at = $3, revoked_reason = $2 - WHERE target_id = $1::uuid AND status = 'active' AND id = ANY($4::uuid[]) + SET status = 'revoked', revoked_at = $3, revoked_reason = $2, failure_reason = NULL + WHERE target_id = $1::uuid AND status <> 'revoked' AND id = ANY($4::uuid[]) RETURNING id::text, org_id::text`, targetID, reason, now, ids) if err != nil { return nil, err @@ -236,7 +236,7 @@ func (s *PostgresStore) ListAvailableSolutionTargets( b.applied_release, t.opened_generation, b.applied_generation, ($1 <> '' AND EXISTS ( SELECT 1 FROM public.installations i - WHERE i.target_id = t.id AND i.org_id = $1::uuid AND i.status = 'active' + WHERE i.target_id = t.id AND i.org_id = $1::uuid AND i.status <> 'revoked' )) FROM public.solution_targets t JOIN public.solution_host_bindings b ON b.binding_id = t.binding_id diff --git a/module/services/accounts/code/pkg/infra/postgres_work_context_seal.go b/module/services/accounts/code/pkg/infra/postgres_work_context_seal.go index e9325d0d2..6440f218a 100644 --- a/module/services/accounts/code/pkg/infra/postgres_work_context_seal.go +++ b/module/services/accounts/code/pkg/infra/postgres_work_context_seal.go @@ -34,7 +34,13 @@ func (s *PostgresStore) WorkContextHeldInstallation( FROM public.installations i WHERE i.org_id = $1::uuid AND i.id::text = $3 - AND i.status = 'active' + -- A member of the organization holds its installations that are + -- active AND the ones still pending: an installation is pending + -- until its install step completes, and that step is the one + -- capability minted to a pending installation — sealing it is + -- what lets the step be delivered at all. A failed or + -- revoked installation is held by nobody. + AND i.status IN ('active', 'pending') AND EXISTS (SELECT 1 FROM public.organization_members m WHERE m.org_id = i.org_id AND m.user_id::text = $2)`, orgID, principalID, installationID, diff --git a/module/services/accounts/code/pkg/infra/solution_host_bundle_verifier.go b/module/services/accounts/code/pkg/infra/solution_host_bundle_verifier.go index 15c4ce354..b9cf753bf 100644 --- a/module/services/accounts/code/pkg/infra/solution_host_bundle_verifier.go +++ b/module/services/accounts/code/pkg/infra/solution_host_bundle_verifier.go @@ -6,6 +6,7 @@ import ( "fmt" "os" "path/filepath" + "strings" "github.com/codefly-dev/core/solutionhost" ) @@ -75,6 +76,42 @@ type solutionHostBundleVerifierWithPolicy interface { // That is the property, and the constant is what makes it one. const SolutionHostTrustAnchorPath = "/etc/obin/delivery-trust" +// LocalTrustAnchorEnvironment names the trust anchor's directory when this host +// is NOT a pod — the same developer run LocalClusterEnvironment describes, which +// writes the local profile there. The constant above stays the only place a pod +// reads its anchor from, for the reason given above it: IN A POD THIS VARIABLE +// IS REFUSED, so nothing a composition can set moves a cell's anchor. Outside a +// pod, nobody mounts anything; the anchor it names is a local profile, which +// the host accepts only after proving it runs in the cluster that profile is +// bound to. +const LocalTrustAnchorEnvironment = "SOLUTION_HOST_LOCAL_TRUST_ANCHOR" + +// ErrLocalTrustAnchorInAPod refuses a relocated anchor named inside a cluster. +var ErrLocalTrustAnchorInAPod = errors.New("a pod may not relocate its delivery trust anchor") + +// solutionHostTrustAnchorDir is where this process reads its anchor from. +func solutionHostTrustAnchorDir() (string, error) { + return trustAnchorDir(os.Getenv(LocalTrustAnchorEnvironment), InCluster()) +} + +// trustAnchorDir is solutionHostTrustAnchorDir over facts already read, so a +// test drives the refusal through this code. +func trustAnchorDir(local string, inPod bool) (string, error) { + local = strings.TrimSpace(local) + if local == "" { + return SolutionHostTrustAnchorPath, nil + } + if inPod { + return "", fmt.Errorf("%w: %s is set and this process carries what a kubelet puts in a pod; "+ + "a pod reads its anchor from %s and from nowhere else", + ErrLocalTrustAnchorInAPod, LocalTrustAnchorEnvironment, SolutionHostTrustAnchorPath) + } + if !filepath.IsAbs(local) { + return "", fmt.Errorf("%w: %s must be an absolute directory, got %q", ErrSolutionHostTrustRootUnavailable, LocalTrustAnchorEnvironment, local) + } + return local, nil +} + // RequireSolutionHostTrustAnchor refuses when the trust anchor is not mounted. // // It exists so the ABSENCE OF THE ANCHOR IS A BOOT FAILURE THAT NAMES THE @@ -90,7 +127,11 @@ const SolutionHostTrustAnchorPath = "/etc/obin/delivery-trust" // anyone's choice. A deployment that needs a different path changes the pod spec // that mounts it. func RequireSolutionHostTrustAnchor() error { - path := filepath.Join(SolutionHostTrustAnchorPath, verificationPolicyFileName) + dir, err := solutionHostTrustAnchorDir() + if err != nil { + return err + } + path := filepath.Join(dir, verificationPolicyFileName) info, err := os.Stat(path) if err != nil { return fmt.Errorf("%w: the delivery trust anchor %s is not mounted or readable: %w; "+ diff --git a/module/services/accounts/code/pkg/infra/solution_host_key_verifier.go b/module/services/accounts/code/pkg/infra/solution_host_key_verifier.go index 9f9600e65..64efeff57 100644 --- a/module/services/accounts/code/pkg/infra/solution_host_key_verifier.go +++ b/module/services/accounts/code/pkg/infra/solution_host_key_verifier.go @@ -89,7 +89,33 @@ type keyBundleVerifier struct { // NewSolutionHostKeyVerifier reads the one policy document at the constant // anchor. A path parameter here would let composition nominate its own trust. func NewSolutionHostKeyVerifier() (solutionHostBundleVerifierWithPolicy, error) { - return readSolutionHostKeyVerifier(SolutionHostTrustAnchorPath) + directory, err := solutionHostTrustAnchorDir() + if err != nil { + return nil, err + } + verifier, err := readSolutionHostKeyVerifier(directory) + if err != nil { + return nil, err + } + return requireLocalProfileOffTheAnchor(directory, verifier) +} + +// requireLocalProfileOffTheAnchor holds a relocated anchor to being a local +// profile. The constant anchor may carry a release policy, because only the +// platform can mount there. A directory a developer's run names may carry only +// a policy bound to one cluster's certificate authority, which the host must +// then prove it runs in — so a relocated anchor can never be a set of keys +// trusted on nobody's say-so. +func requireLocalProfileOffTheAnchor(directory string, verifier solutionHostBundleVerifierWithPolicy) (solutionHostBundleVerifierWithPolicy, error) { + if directory == SolutionHostTrustAnchorPath { + return verifier, nil + } + if keyed, ok := verifier.(*keyBundleVerifier); !ok || keyed.local == nil { + return nil, fmt.Errorf("%w: the anchor at %q was named by %s and is not a local profile; "+ + "only a policy bound to a cluster's certificate authority may be read from outside %s", + ErrSolutionHostTrustRootUnavailable, directory, LocalTrustAnchorEnvironment, SolutionHostTrustAnchorPath) + } + return verifier, nil } // The file reader is private so only tests can substitute a temporary mount. diff --git a/module/services/accounts/code/pkg/infra/source_delegation_test.go b/module/services/accounts/code/pkg/infra/source_delegation_test.go index f9e5f4e23..e71bc38ae 100644 --- a/module/services/accounts/code/pkg/infra/source_delegation_test.go +++ b/module/services/accounts/code/pkg/infra/source_delegation_test.go @@ -662,12 +662,12 @@ func TestSourceDelegation_DedicatedTenantCannotBorrowInstallation(t *testing.T) revision, err = testStore.LatestModuleEnvelopeRevision(ctx) return err })) - w.svc.InstallDerivedModulePrincipals(registry, revision) + w.svc.InstallDerivedModulePrincipals(registry, revision, nil) _, err := w.mint(business.SourceDelegationRef{SourceID: source}) require.ErrorIs(t, err, business.ErrModuleTenantNotCurrent) grant.Tenant = "" registry[principal] = grant - w.svc.InstallDerivedModulePrincipals(registry, revision) + w.svc.InstallDerivedModulePrincipals(registry, revision, nil) authority, err := w.mint(business.SourceDelegationRef{SourceID: source}) require.NoError(t, err) require.NotEmpty(t, authority.InstallationID) diff --git a/module/services/accounts/code/pkg/infra/testdata/control_plane_call_sites.txt b/module/services/accounts/code/pkg/infra/testdata/control_plane_call_sites.txt index 43230e94e..87a5c4edf 100644 --- a/module/services/accounts/code/pkg/infra/testdata/control_plane_call_sites.txt +++ b/module/services/accounts/code/pkg/infra/testdata/control_plane_call_sites.txt @@ -54,6 +54,7 @@ pkg/business/follow_fanout.go: (*Service).revokeUndeclaredFollowSubscriptions: W pkg/business/follow_fanout.go: (*Service).undeliveredFollowers: WithControlPlane pkg/business/gdpr.go: (*Service).PurgeExpiredPrivacyArtifacts: WithControlPlane pkg/business/installation_exposure.go: (*Service).ListOrganizationInstallations: WithControlPlane +pkg/business/installation_jobs.go: (*Service).RetryInstallation: WithControlPlane pkg/business/installation_registry.go: (*Service).prepareInstallation: WithControlPlane pkg/business/installations.go: (*Service).ListAvailableSolutions: WithControlPlane pkg/business/installations.go: (*Service).resolveInstallableTarget: WithControlPlane @@ -77,11 +78,15 @@ pkg/business/module_capabilities.go: (*Service).moduleAuditSubjectIsMemberOf: Wi pkg/business/module_capabilities.go: (*Service).requireTenantMember: WithControlPlane pkg/business/module_envelope.go: (*Service).CurrentModuleEnvelope: WithControlPlane pkg/business/module_envelope.go: (*Service).GetModuleEnvelope: WithControlPlane +pkg/business/module_envelope.go: (*Service).PutModuleEnvelope: WithControlPlane pkg/business/module_envelope.go: (*Service).committedModuleEnvelopeRevision: WithControlPlane +pkg/business/module_envelope.go: (*Service).deliveredSince: WithControlPlane +pkg/business/module_envelope.go: (*Service).moduleEnvelopeNarrows: WithControlPlane pkg/business/module_identity.go: (*Service).RecordModuleWorkContextMint: WithControlPlane pkg/business/module_identity.go: (*Service).VerifyModuleTenant: WithControlPlane pkg/business/module_installations.go: (*Service).ReconcileModuleInstallation: WithControlPlane pkg/business/module_operation_context.go: (*Service).RecordModuleOperationContextMint: WithControlPlane +pkg/business/module_upstream.go: (*Service).RecordModuleUpstream: WithControlPlane pkg/business/notifications.go: (*Service).resolveNotificationUser: WithControlPlane pkg/business/onboarding.go: (*Service).organizationConfigured: WithControlPlane pkg/business/organization_lifecycle.go: (*Service).GetOrganizationRoster: WithControlPlane @@ -171,6 +176,8 @@ pkg/infra/postgres_auth_bootstrap.go: (*PostgresStore).WithAuthBootstrapTx: with pkg/infra/postgres_auth_bootstrap.go: (*PostgresStore).WithAuthLookupTx: withControlPlaneTx pkg/infra/postgres_client_authorization.go: (*PostgresStore).ConsumeClientAuthorizationCode: WithControlPlane pkg/infra/postgres_datasource_sources.go: (*PostgresStore).GetDatasourceSourceByID: WithControlPlane +pkg/infra/postgres_installation_jobs.go: (*PostgresStore).InstallationStepLeaseValid: WithControlPlane +pkg/infra/postgres_installation_jobs.go: (*PostgresStore).WithInstallationStepAuthority: WithControlPlane pkg/infra/postgres_installations.go: (*PostgresStore).ResolveInstallationAuthority: WithControlPlane pkg/infra/postgres_keyservice_reseal.go: (*PostgresStore).organizationsWithForeignKeys: WithControlPlane pkg/infra/postgres_keyservice_reseal.go: (*PostgresStore).resealColumn: WithControlPlane diff --git a/module/services/accounts/code/pkg/infra/unapplied_presence_test.go b/module/services/accounts/code/pkg/infra/unapplied_presence_test.go new file mode 100644 index 000000000..76d32e71d --- /dev/null +++ b/module/services/accounts/code/pkg/infra/unapplied_presence_test.go @@ -0,0 +1,42 @@ +//go:build !pure + +package infra_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" +) + +// A presence the inbox holds at a generation its binding has not applied is +// counted until the binding reconciler applies it, and not after. +func TestUnappliedPresenceIsCountedUntilItsBindingApplies(t *testing.T) { + s := newRegistryScenario(t, "unap") + // On the control plane, as its one production caller reads it: the + // request role sees neither table. + unapplied := func() (uint64, error) { + var count uint64 + err := testStore.WithControlPlane(testCtx, func(ctx context.Context) error { + var err error + count, err = testStore.CountUnappliedPresence(ctx) + return err + }) + return count, err + } + before, err := unapplied() + require.NoError(t, err) + + current := s.currentRevision(t) + _, err = s.delivery.service.PutModuleEnvelope(testCtx, s.admin, current, s.envelope(t, current+1, envelopeOptions{resources: true})) + require.NoError(t, err) + s.deliverPresence(t, s.primary, 1, current+1) + pending, err := unapplied() + require.NoError(t, err) + require.Equal(t, before+1, pending, "a delivered presence nothing has applied is pending") + + s.pass(t) + after, err := unapplied() + require.NoError(t, err) + require.Equal(t, before, after, "an applied presence is no longer pending") +} diff --git a/module/services/accounts/code/work.go b/module/services/accounts/code/work.go index f3505efd4..ee05270ad 100644 --- a/module/services/accounts/code/work.go +++ b/module/services/accounts/code/work.go @@ -210,6 +210,8 @@ func doWork(ctx context.Context) (Clean, error) { // first pass installs what delivery and the envelope agree on. Nothing in // this process's environment can name a module principal. service.SetModuleCapabilities(store, jobStore, nil) + // codefly:gateway-route-exempt Internal install consumer protocol; only the gateway calls accounts directly with the internal credential. + adapters.RegisterHTTPRoute(adapters.InstallationStepPath, adapters.NewInstallationStepHandler(service)) // The envelope's persistence, wired whether or not this host receives // deliveries: a platform super_admin can write the ceiling before the first // document arrives. diff --git a/module/services/accounts/generated/api-contract-surfaces.json b/module/services/accounts/generated/api-contract-surfaces.json index 4660b35a1..749e0093d 100644 --- a/module/services/accounts/generated/api-contract-surfaces.json +++ b/module/services/accounts/generated/api-contract-surfaces.json @@ -221,6 +221,7 @@ "/saas.accounts.v1.InstallationService/ListInstallations", "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", "/saas.accounts.v1.InstallationService/NarrowInstallation", + "/saas.accounts.v1.InstallationService/RetryInstallation", "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", "/saas.accounts.v1.InstallationService/UninstallSolution" diff --git a/module/services/accounts/generated/authz-methods.json b/module/services/accounts/generated/authz-methods.json index 846c0ff2a..47b4db1b7 100644 --- a/module/services/accounts/generated/authz-methods.json +++ b/module/services/accounts/generated/authz-methods.json @@ -3805,6 +3805,51 @@ "policy_sha256": "e259b72e4c614317294f349748195b97379b956f162fdb7a4e2d6f603db27478", "rate_limit_backend_failure_mode": "RATE_LIMIT_BACKEND_FAILURE_MODE_FAIL_OPEN" }, + { + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "full_service": "saas.accounts.v1.InstallationService", + "method": "RetryInstallation", + "input_type": "saas.accounts.v1.RetryInstallationRequest", + "output_type": "saas.accounts.v1.Installation", + "client_streaming": false, + "server_streaming": false, + "policy": { + "exposure": "EXPOSURE_AUTHENTICATED", + "tenant": "TENANT_REQUIREMENT_ORG_ADMIN", + "permissions": [], + "scopes": [], + "resource_bindings": [ + { + "request_field": "org_id", + "target": "RESOURCE_TARGET_ORGANIZATION", + "lookup": "RESOURCE_LOOKUP_DIRECT_ID" + } + ], + "mfa": "MFA_REQUIREMENT_NONE", + "audit": { + "events": [ + "saas.installation.retried" + ], + "emission": "AUDIT_EMISSION_SUCCESS" + }, + "idempotency": "IDEMPOTENCY_REQUIREMENT_FORBIDDEN", + "rate_limit": "RATE_LIMIT_CLASS_STANDARD_WRITE", + "request_sensitivity": "SENSITIVITY_CONFIDENTIAL", + "response_sensitivity": "SENSITIVITY_CONFIDENTIAL", + "platform_role": "PLATFORM_ROLE_REQUIREMENT_NONE", + "authentication_factor_attempt": false, + "conditions": [], + "impersonation": "IMPERSONATION_REQUIREMENT_UNSPECIFIED" + }, + "policy_tier": "org_admin", + "owner": { + "module": "saas-starter", + "service": "accounts" + }, + "source_proto": "saas/accounts/v1/installations.proto", + "policy_sha256": "9d2f469757039b1c657d8f26aa3882f120633ec7426eabc127bb7cbc7d4126fb", + "rate_limit_backend_failure_mode": "RATE_LIMIT_BACKEND_FAILURE_MODE_FAIL_OPEN" + }, { "procedure": "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "full_service": "saas.accounts.v1.InstallationService", diff --git a/module/services/accounts/generated/gateway-routes.json b/module/services/accounts/generated/gateway-routes.json index 6cd75acbf..70a35eb51 100644 --- a/module/services/accounts/generated/gateway-routes.json +++ b/module/services/accounts/generated/gateway-routes.json @@ -1345,6 +1345,22 @@ "rewrite_path": "/saas.accounts.v1.InstallationService/NarrowInstallation", "remove_after": "2026-10-11" }, + { + "protocol": "GATEWAY_PROTOCOL_CONNECT", + "method": "POST", + "path": "/customers.InstallationService/RetryInstallation", + "match": "GATEWAY_MATCH_EXACT", + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "owner": { + "module": "saas-starter", + "service": "accounts" + }, + "upstream_endpoint": "connect", + "exposure": "EXPOSURE_AUTHENTICATED", + "source": "GATEWAY_ROUTE_SOURCE_COMPATIBILITY_ALIAS", + "rewrite_path": "/saas.accounts.v1.InstallationService/RetryInstallation", + "remove_after": "2026-10-11" + }, { "protocol": "GATEWAY_PROTOCOL_CONNECT", "method": "POST", @@ -4769,6 +4785,22 @@ "rewrite_path": "", "remove_after": "" }, + { + "protocol": "GATEWAY_PROTOCOL_CONNECT", + "method": "POST", + "path": "/saas.accounts.v1.InstallationService/RetryInstallation", + "match": "GATEWAY_MATCH_EXACT", + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "owner": { + "module": "saas-starter", + "service": "accounts" + }, + "upstream_endpoint": "connect", + "exposure": "EXPOSURE_AUTHENTICATED", + "source": "GATEWAY_ROUTE_SOURCE_DESCRIPTOR", + "rewrite_path": "", + "remove_after": "" + }, { "protocol": "GATEWAY_PROTOCOL_CONNECT", "method": "POST", @@ -8625,6 +8657,22 @@ "rewrite_path": "", "remove_after": "" }, + { + "protocol": "GATEWAY_PROTOCOL_REST", + "method": "POST", + "path": "/v1/installations/{installation_id}:retry", + "match": "GATEWAY_MATCH_PATH_TEMPLATE", + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "owner": { + "module": "saas-starter", + "service": "accounts" + }, + "upstream_endpoint": "rest", + "exposure": "EXPOSURE_AUTHENTICATED", + "source": "GATEWAY_ROUTE_SOURCE_DESCRIPTOR", + "rewrite_path": "", + "remove_after": "" + }, { "protocol": "GATEWAY_PROTOCOL_REST", "method": "POST", diff --git a/module/services/accounts/generated/openapi-raw/api.swagger.json b/module/services/accounts/generated/openapi-raw/api.swagger.json index bc318dc5b..72aaa430c 100644 --- a/module/services/accounts/generated/openapi-raw/api.swagger.json +++ b/module/services/accounts/generated/openapi-raw/api.swagger.json @@ -1970,6 +1970,45 @@ ] } }, + "/v1/installations/{installationId}:retry": { + "post": { + "summary": "RetryInstallation runs a failed installation's install step again, at the\nsame revision. The installation is pending until the solution acknowledges.", + "operationId": "InstallationService_RetryInstallation", + "responses": { + "200": { + "description": "A successful response.", + "schema": { + "$ref": "#/definitions/v1Installation" + } + }, + "default": { + "description": "An unexpected error response.", + "schema": { + "$ref": "#/definitions/rpcStatus" + } + } + }, + "parameters": [ + { + "name": "installationId", + "in": "path", + "required": true, + "type": "string" + }, + { + "name": "body", + "in": "body", + "required": true, + "schema": { + "$ref": "#/definitions/InstallationServiceRetryInstallationBody" + } + } + ], + "tags": [ + "InstallationService" + ] + } + }, "/v1/installations/{installationId}:transferOwnership": { "post": { "summary": "TransferInstallationOwnership reassigns the owner of record and replaces the\nco-owner succession set. This is how a \"connection owner left the org\"\nfailure is repaired: an admin transfers the installation to a current admin,\nrestoring its health.", @@ -6492,6 +6531,14 @@ }, "description": "An org administrator may only remove installed authority. The host reads\nthe current revision and serializes the narrowing with other lifecycle acts." }, + "InstallationServiceRetryInstallationBody": { + "type": "object", + "properties": { + "orgId": { + "type": "string" + } + } + }, "InstallationServiceRevokeInstallationFromTeamBody": { "type": "object", "properties": { @@ -8889,6 +8936,10 @@ "revokedReason": { "type": "string", "description": "Why a revoked installation was revoked, when it was not an administrator\nuninstalling: the installation-identity cutover, or a tenancy narrowing that\nwithdrew this organisation. Empty on an active row." + }, + "failureReason": { + "type": "string", + "description": "Why the install step failed: the solution's bounded refusal reason, or a\nfixed reason when it never answered. Empty unless status is FAILED." } }, "description": "Installation is one solution's bound identity in an org. Server-assigned ids;\nthe browser cannot attest to tenancy, the agent principal, or the scope node." @@ -8911,10 +8962,12 @@ "enum": [ "INSTALLATION_STATUS_UNSPECIFIED", "INSTALLATION_STATUS_ACTIVE", - "INSTALLATION_STATUS_REVOKED" + "INSTALLATION_STATUS_REVOKED", + "INSTALLATION_STATUS_PENDING", + "INSTALLATION_STATUS_FAILED" ], "default": "INSTALLATION_STATUS_UNSPECIFIED", - "description": "InstallationStatus is the lifecycle of an installation row. An uninstall is a\nsoft delete (status=revoked, revoked_at set) so the agent, grants, and audit\ntrail remain resolvable." + "description": "InstallationStatus is the lifecycle of an installation row. An uninstall is a\nsoft delete (status=revoked, revoked_at set) so the agent, grants, and audit\ntrail remain resolvable.\n\n - INSTALLATION_STATUS_PENDING: Installed, and waiting for the solution to acknowledge its standard install\nstep. Nothing can use a pending installation.\n - INSTALLATION_STATUS_FAILED: The solution refused its install step, or never answered within the\nattempt budget. Nothing can use it; an administrator may retry it." }, "v1Invitation": { "type": "object", diff --git a/module/services/accounts/generated/rest-surface.json b/module/services/accounts/generated/rest-surface.json index c26e7dd2d..54a7998bc 100644 --- a/module/services/accounts/generated/rest-surface.json +++ b/module/services/accounts/generated/rest-surface.json @@ -1781,6 +1781,22 @@ "rewrite_path": "", "remove_after": "" }, + { + "protocol": "GATEWAY_PROTOCOL_REST", + "method": "POST", + "path": "/v1/installations/{installation_id}:retry", + "match": "GATEWAY_MATCH_PATH_TEMPLATE", + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "owner": { + "module": "saas-starter", + "service": "accounts" + }, + "upstream_endpoint": "rest", + "exposure": "EXPOSURE_AUTHENTICATED", + "source": "GATEWAY_ROUTE_SOURCE_DESCRIPTOR", + "rewrite_path": "", + "remove_after": "" + }, { "protocol": "GATEWAY_PROTOCOL_REST", "method": "POST", diff --git a/module/services/accounts/generated/service-catalog.json b/module/services/accounts/generated/service-catalog.json index de5ce9c37..2108d178e 100644 --- a/module/services/accounts/generated/service-catalog.json +++ b/module/services/accounts/generated/service-catalog.json @@ -183,6 +183,7 @@ "/saas.accounts.v1.InstallationService/ListInstallations", "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", "/saas.accounts.v1.InstallationService/NarrowInstallation", + "/saas.accounts.v1.InstallationService/RetryInstallation", "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", "/saas.accounts.v1.InstallationService/UninstallSolution" @@ -5211,6 +5212,65 @@ "service": "accounts" } }, + { + "procedure": "/saas.accounts.v1.InstallationService/RetryInstallation", + "service": "InstallationService", + "full_service": "saas.accounts.v1.InstallationService", + "method": "RetryInstallation", + "input_type": "saas.accounts.v1.RetryInstallationRequest", + "output_type": "saas.accounts.v1.Installation", + "client_streaming": false, + "server_streaming": false, + "deprecated": false, + "source_proto": "saas/accounts/v1/installations.proto", + "protocols": [ + "PROTOCOL_GRPC", + "PROTOCOL_CONNECT", + "PROTOCOL_REST" + ], + "http_bindings": [ + { + "method": "POST", + "path": "/v1/installations/{installation_id}:retry", + "body": "*", + "response_body": "" + } + ], + "policy": { + "exposure": "EXPOSURE_AUTHENTICATED", + "tenant": "TENANT_REQUIREMENT_ORG_ADMIN", + "permissions": [], + "scopes": [], + "resource_bindings": [ + { + "request_field": "org_id", + "target": "RESOURCE_TARGET_ORGANIZATION", + "lookup": "RESOURCE_LOOKUP_DIRECT_ID" + } + ], + "mfa": "MFA_REQUIREMENT_NONE", + "audit": { + "events": [ + "saas.installation.retried" + ], + "emission": "AUDIT_EMISSION_SUCCESS" + }, + "idempotency": "IDEMPOTENCY_REQUIREMENT_FORBIDDEN", + "rate_limit": "RATE_LIMIT_CLASS_STANDARD_WRITE", + "request_sensitivity": "SENSITIVITY_CONFIDENTIAL", + "response_sensitivity": "SENSITIVITY_CONFIDENTIAL", + "platform_role": "PLATFORM_ROLE_REQUIREMENT_NONE", + "authentication_factor_attempt": false, + "conditions": [], + "impersonation": "IMPERSONATION_REQUIREMENT_UNSPECIFIED" + }, + "policy_tier": "org_admin", + "description": "Run a failed installation's install step again; the installation is pending until the solution acknowledges.", + "owner": { + "module": "saas-starter", + "service": "accounts" + } + }, { "procedure": "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "service": "InstallationService", diff --git a/module/services/accounts/openapi/api.swagger.json b/module/services/accounts/openapi/api.swagger.json index bcc61322d..684ac6c2c 100644 --- a/module/services/accounts/openapi/api.swagger.json +++ b/module/services/accounts/openapi/api.swagger.json @@ -90,6 +90,14 @@ }, "type": "object" }, + "InstallationServiceRetryInstallationBody": { + "properties": { + "orgId": { + "type": "string" + } + }, + "type": "object" + }, "InstallationServiceRevokeInstallationFromTeamBody": { "properties": { "orgId": { @@ -2415,6 +2423,10 @@ "format": "date-time", "type": "string" }, + "failureReason": { + "description": "Why the install step failed: the solution's bounded refusal reason, or a\nfixed reason when it never answered. Empty unless status is FAILED.", + "type": "string" + }, "id": { "type": "string" }, @@ -2460,11 +2472,13 @@ }, "v1InstallationStatus": { "default": "INSTALLATION_STATUS_UNSPECIFIED", - "description": "InstallationStatus is the lifecycle of an installation row. An uninstall is a\nsoft delete (status=revoked, revoked_at set) so the agent, grants, and audit\ntrail remain resolvable.", + "description": "InstallationStatus is the lifecycle of an installation row. An uninstall is a\nsoft delete (status=revoked, revoked_at set) so the agent, grants, and audit\ntrail remain resolvable.\n\n - INSTALLATION_STATUS_PENDING: Installed, and waiting for the solution to acknowledge its standard install\nstep. Nothing can use a pending installation.\n - INSTALLATION_STATUS_FAILED: The solution refused its install step, or never answered within the\nattempt budget. Nothing can use it; an administrator may retry it.", "enum": [ "INSTALLATION_STATUS_UNSPECIFIED", "INSTALLATION_STATUS_ACTIVE", - "INSTALLATION_STATUS_REVOKED" + "INSTALLATION_STATUS_REVOKED", + "INSTALLATION_STATUS_PENDING", + "INSTALLATION_STATUS_FAILED" ], "type": "string" }, @@ -7774,6 +7788,45 @@ ] } }, + "/v1/installations/{installationId}:retry": { + "post": { + "operationId": "InstallationService_RetryInstallation", + "parameters": [ + { + "in": "path", + "name": "installationId", + "required": true, + "type": "string" + }, + { + "in": "body", + "name": "body", + "required": true, + "schema": { + "$ref": "#/definitions/InstallationServiceRetryInstallationBody" + } + } + ], + "responses": { + "200": { + "description": "A successful response.", + "schema": { + "$ref": "#/definitions/v1Installation" + } + }, + "default": { + "description": "An unexpected error response.", + "schema": { + "$ref": "#/definitions/rpcStatus" + } + } + }, + "summary": "RetryInstallation runs a failed installation's install step again, at the\nsame revision. The installation is pending until the solution acknowledges.", + "tags": [ + "InstallationService" + ] + } + }, "/v1/installations/{installationId}:transferOwnership": { "post": { "operationId": "InstallationService_TransferInstallationOwnership", diff --git a/module/services/accounts/proto/saas/accounts/v1/installations.proto b/module/services/accounts/proto/saas/accounts/v1/installations.proto index 17fe6a96d..7b8440c1b 100644 --- a/module/services/accounts/proto/saas/accounts/v1/installations.proto +++ b/module/services/accounts/proto/saas/accounts/v1/installations.proto @@ -29,6 +29,12 @@ enum InstallationStatus { INSTALLATION_STATUS_UNSPECIFIED = 0; INSTALLATION_STATUS_ACTIVE = 1; INSTALLATION_STATUS_REVOKED = 2; + // Installed, and waiting for the solution to acknowledge its standard install + // step. Nothing can use a pending installation. + INSTALLATION_STATUS_PENDING = 3; + // The solution refused its install step, or never answered within the + // attempt budget. Nothing can use it; an administrator may retry it. + INSTALLATION_STATUS_FAILED = 4; } // InstallationHealth is resolved live from current authority facts, never @@ -88,6 +94,9 @@ message Installation { // uninstalling: the installation-identity cutover, or a tenancy narrowing that // withdrew this organisation. Empty on an active row. string revoked_reason = 12 [(buf.validate.field).string.max_len = 512]; + // Why the install step failed: the solution's bounded refusal reason, or a + // fixed reason when it never answered. Empty unless status is FAILED. + string failure_reason = 13 [(buf.validate.field).string.max_len = 512]; } message InstallSolutionRequest { @@ -177,6 +186,11 @@ message NarrowInstallationRequest { string scope = 3 [(buf.validate.field).string = {min_len: 2 max_bytes: 1048576}]; } +message RetryInstallationRequest { + string org_id = 1 [(buf.validate.field).string.uuid = true]; + string installation_id = 2 [(buf.validate.field).string.uuid = true]; +} + message UninstallSolutionRequest { string org_id = 1 [(buf.validate.field).string.uuid = true]; string installation_id = 2 [(buf.validate.field).string.uuid = true]; @@ -333,6 +347,34 @@ service InstallationService { }; } + // RetryInstallation runs a failed installation's install step again, at the + // same revision. The installation is pending until the solution acknowledges. + rpc RetryInstallation(RetryInstallationRequest) returns (Installation) { + option (saas.policy.v1.method_policy) = { + exposure: EXPOSURE_AUTHENTICATED + tenant: TENANT_REQUIREMENT_ORG_ADMIN + resource_bindings: { + request_field: "org_id" + target: RESOURCE_TARGET_ORGANIZATION + lookup: RESOURCE_LOOKUP_DIRECT_ID + } + mfa: MFA_REQUIREMENT_NONE + audit: { + events: "saas.installation.retried" + emission: AUDIT_EMISSION_SUCCESS + } + idempotency: IDEMPOTENCY_REQUIREMENT_FORBIDDEN + rate_limit: RATE_LIMIT_CLASS_STANDARD_WRITE + request_sensitivity: SENSITIVITY_CONFIDENTIAL + response_sensitivity: SENSITIVITY_CONFIDENTIAL + platform_role: PLATFORM_ROLE_REQUIREMENT_NONE + }; + option (google.api.http) = { + post: "/v1/installations/{installation_id}:retry" + body: "*" + }; + } + // TransferInstallationOwnership reassigns the owner of record and replaces the // co-owner succession set. This is how a "connection owner left the org" // failure is repaired: an admin transfers the installation to a current admin, diff --git a/module/services/accounts/proto/saas/accounts/v1/module_registration.proto b/module/services/accounts/proto/saas/accounts/v1/module_registration.proto index ceb5164cd..395aac1c3 100644 --- a/module/services/accounts/proto/saas/accounts/v1/module_registration.proto +++ b/module/services/accounts/proto/saas/accounts/v1/module_registration.proto @@ -60,6 +60,28 @@ message ModuleMintWorkContextRequest { (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, (buf.validate.field).string.uuid = true ]; + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + string upstream = 5 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string = { + max_len: 512 + uri: true + } + ]; + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + string manifest = 6 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string.max_len = 65536 + ]; } // ModuleMintWorkContextResponse carries the signed capability plus the identity diff --git a/module/services/auth-gateway/AGENTS.md b/module/services/auth-gateway/AGENTS.md index 1498870a4..6d15dc6da 100644 --- a/module/services/auth-gateway/AGENTS.md +++ b/module/services/auth-gateway/AGENTS.md @@ -330,12 +330,56 @@ bearer: identity is stripped there and no ext_authz check runs, so the gateway has nothing to bind, and both halves say no rather than one promising what the other drops. +## Workload TLS endpoint and solution upstreams + +`workload` (`api: http`) serves the same mux over TLS 1.3 with required, verified +client certificates when the `workload-identity` workspace group supplies +`CERT_FILE`, `KEY_FILE`, `TRUST_BUNDLE_FILE`, and `ALLOWED_CALLERS_FILE`. If all +four values are absent, the gateway boots with only its existing plain endpoints, +does not resolve or open the workload endpoint, and logs once at info that no +workload identity is projected. Solution mints remain unreachable over mTLS and +solution proxying stays refused without a workload identity. Any partial set of +values or missing/unreadable supplied file fails boot naming its key. Keep the +`workload-identity` workspace dependency declared: renders use it to select +consumers of the projection. +The pinned SDK's `CertificateReloader` supplies both certificate callbacks. +Every handshake checks a single SPIFFE URI against the live caller file; every +request checks it again so keep-alive connections cannot outlive revocation. +The caller file must be published atomically, as the kubelet publishes a +ConfigMap volume (written whole, swapped behind one symlink); a mid-write mix +of two lists is indistinguishable from a complete list, so an in-place rewrite +is outside the contract. A truncated write is refused whole. + +On this endpoint every module mint (`/modules/_work-context`, +`/modules/_operation-context`, `/modules/_source-operation-context`) conveys +the verified SVID peer to accounts on the authenticated internal RPC. Accounts +binds it to TokenReview and the applied presence before signing. Caller-supplied +identity headers never attest a peer. Plain `rest` module mints still require +the internal token and keep their existing behavior. + +SPIFFE comparisons use the certificate's raw URI SAN, without URL normalization. +The live caller file must end in a newline; an incomplete or invalid snapshot +admits nobody, including on an existing connection. Entries trim only ASCII +space, tab and CR, with `#` introducing a comment. + +Solutions are proxied only over TLS 1.3 with the gateway SVID. The server's chain +and single SPIFFE URI must match the applied presence read alongside its registry +record. Missing or ambiguous identities fail closed. The existing DNS/address +guard remains the only dial path. Module upstream transport remains unchanged. + ## Brokering a module's Work Context `POST /modules/_work-context` takes the cluster-internal token plus the module pod's **projected ServiceAccount token** as `Authorization: Bearer ` (exactly one header, that scheme), body -`{prefix, organization_id?}`. A shared module selects an organization; accounts +`{prefix, organization_id?, upstream?, manifest?}`, bounded at 80 KiB. +`upstream` and `manifest` are the dynamic half of the workload's registration +([SOLUTION_REGISTRATION.md](../../SOLUTION_REGISTRATION.md#the-document-and-who-owns-which-field)): +they are accepted only on the `workload` endpoint and refused `400` on the +plain one; the gateway checks the origin's shape — `https`, a host and a port, a host a +registered upstream may name — refuses `400` before asking accounts otherwise, +and forwards both unread beyond that. Accounts answers `FailedPrecondition` +for a workload that is admitted and installed nowhere, relayed as `412`. A shared module selects an organization; accounts requires its live installation and the envelope's `cross_tenant` grant. A tenant-pinned module may omit the selector or name only its pinned tenant. No secret header is read: `X-Codefly-Module-Secret` is deleted with diff --git a/module/services/auth-gateway/code/authz_catalog_gen.go b/module/services/auth-gateway/code/authz_catalog_gen.go index 4df60066a..45f198f92 100644 --- a/module/services/auth-gateway/code/authz_catalog_gen.go +++ b/module/services/auth-gateway/code/authz_catalog_gen.go @@ -99,6 +99,7 @@ var generatedAuthorizationByProcedure = map[string]generatedAuthorizationMetadat "/saas.accounts.v1.InstallationService/ListInstallations": {exposure: edgeExposureInternal, rateLimitClass: edgeRateLimitClassInternal, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "45df71bfa7f313b565b0e8c314557847bab913990d43ba59b97d755412b4a4fd"}, "/saas.accounts.v1.InstallationService/ListOrganizationInstallations": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardRead, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "ef6a5e6e68c7115ca8783422cdc54febdd089c9b2660ac3774c7a4a4847e7270"}, "/saas.accounts.v1.InstallationService/NarrowInstallation": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardWrite, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "e259b72e4c614317294f349748195b97379b956f162fdb7a4e2d6f603db27478"}, + "/saas.accounts.v1.InstallationService/RetryInstallation": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardWrite, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "9d2f469757039b1c657d8f26aa3882f120633ec7426eabc127bb7cbc7d4126fb"}, "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardWrite, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "27a0a7b814576c5b823848915610b58388ab31947e230e3c4c630ec25ffdffbe"}, "/saas.accounts.v1.InstallationService/TransferInstallationOwnership": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardWrite, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "8c63edb387aca3cf7afbd955dcbb58520aab0845ab47f031b512a3b784b037de"}, "/saas.accounts.v1.InstallationService/UninstallSolution": {exposure: edgeExposureAuthenticated, rateLimitClass: edgeRateLimitClassStandardWrite, rateLimitBackendFailClosed: false, authenticationFactorAttempt: false, policySHA256: "6ba619197f162c34b6166baecc383ebad32232ed83d3f19b2d89f3e4bd2f046f"}, @@ -329,6 +330,7 @@ var generatedRESTProcedureByRoute = map[string]string{ "GET /v1/installations:listAvailable": "/saas.accounts.v1.InstallationService/ListAvailableSolutions", "GET /v1/installations": "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", "POST /v1/installations/{installation_id}:narrow": "/saas.accounts.v1.InstallationService/NarrowInstallation", + "POST /v1/installations/{installation_id}:retry": "/saas.accounts.v1.InstallationService/RetryInstallation", "POST /v1/installations/{installation_id}/teams/{team_id}:revoke": "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "POST /v1/installations/{installation_id}:transferOwnership": "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", "POST /v1/installations/{installation_id}:uninstall": "/saas.accounts.v1.InstallationService/UninstallSolution", diff --git a/module/services/auth-gateway/code/gateway.go b/module/services/auth-gateway/code/gateway.go index d263ccb50..1f0fdf1db 100644 --- a/module/services/auth-gateway/code/gateway.go +++ b/module/services/auth-gateway/code/gateway.go @@ -51,13 +51,11 @@ type Gateway struct { // clients is this replica's view of the registered-client registry: which // browser origins each first-party client speaks from (track 0016). clients *clientRegistryCache - // registeredTransport re-validates a runtime-registered upstream's resolved - // address at dial time (SSRF / DNS-rebinding defense). Both federated module - // and solution routes use it: a solution upstream is durable now (#534) but - // still names a host its registrant chose, and it receives forwarded user - // bearers. Catalog upstreams are static trusted config and keep the default - // transport. + // registeredTransport guards module upstream DNS/address resolution. + // Solutions use the same guarded dialer with per-destination workload TLS; + // catalog upstreams keep the default transport. registeredTransport http.RoundTripper + workload *workloadIdentity // solutionEntitlements answers what one viewer may use (#949). Set after // construction, like workContext: a nil client fails the entitlement surface // closed rather than answering an empty projection, which would retract every @@ -473,6 +471,7 @@ func (g *Gateway) proxyTo(w http.ResponseWriter, r *http.Request, upstream *url. // token. Never expose these capabilities to the frontend upstream. r.Header.Del("X-Codefly-Gateway-Token") r.Header.Del("X-Codefly-Internal-Token") + r.Header.Del(workloadSPIFFEHeader) r.Header.Del("X-Codefly-Public-Origin") if isAccountsRoute(entry) && g.authz != nil && g.authz.gatewayToken != "" { r.Header.Set("X-Codefly-Gateway-Token", g.authz.gatewayToken) @@ -513,6 +512,15 @@ func (g *Gateway) proxyTo(w http.ResponseWriter, r *http.Request, upstream *url. // transport. proxy.Transport = g.registeredTransport } + if entry != nil && strings.HasPrefix(entry.Service, solutionServicePrefix) { + transport, err := g.workload.upstreamTransport(entry.WorkloadSPIFFEID) + if err != nil || upstream.Scheme != "https" { + httpError(w, http.StatusBadGateway, "solution workload identity unavailable") + return + } + defer transport.CloseIdleConnections() + proxy.Transport = transport + } proxy.ErrorHandler = func(w http.ResponseWriter, _ *http.Request, err error) { httpError(w, http.StatusBadGateway, "upstream error: "+err.Error()) } diff --git a/module/services/auth-gateway/code/gateway_cors_test.go b/module/services/auth-gateway/code/gateway_cors_test.go index a24efd24c..70f1cb04c 100644 --- a/module/services/auth-gateway/code/gateway_cors_test.go +++ b/module/services/auth-gateway/code/gateway_cors_test.go @@ -218,7 +218,7 @@ func TestCORS_UpstreamGrantIsReplacedNotDuplicated(t *testing.T) { twoRegisteredClients(t, gw) permissive := &fakeUpstream{body: "solution-response"} - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + srv := newSolutionTestServer(t, gw, "example", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Access-Control-Allow-Origin", "*") w.Header().Set("Access-Control-Allow-Credentials", "true") permissive.ServeHTTP(w, r) @@ -405,7 +405,7 @@ func TestCORS_StreamedResponseCarriesTheGrantWithItsFirstChunk(t *testing.T) { twoRegisteredClients(t, gw) release := make(chan struct{}) - solution := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + solution := newSolutionTestServer(t, gw, "example", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.Header().Set("Content-Type", "text/event-stream") _, _ = w.Write([]byte("data: one\n\n")) http.NewResponseController(w).Flush() @@ -618,7 +618,7 @@ func TestCORS_VaryIsNotDuplicatedAgainstAnUpstreamThatSetsIt(t *testing.T) { gw, _, _, priv := newGatewayHarness(t) twoRegisteredClients(t, gw) - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + srv := newSolutionTestServer(t, gw, "example", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.Header().Add("Vary", "Origin") w.Header().Add("Vary", "Accept-Encoding") w.WriteHeader(http.StatusOK) diff --git a/module/services/auth-gateway/code/gateway_installation_jobs.go b/module/services/auth-gateway/code/gateway_installation_jobs.go new file mode 100644 index 000000000..41880ae9e --- /dev/null +++ b/module/services/auth-gateway/code/gateway_installation_jobs.go @@ -0,0 +1,174 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "log" + "net/http" + "strconv" + "time" + + "github.com/codefly-dev/sdk-go/workcontext" + "github.com/google/uuid" +) + +const installationCallTimeout = 20 * time.Second + +type installationStepClaim struct { + JobID string `json:"job_id"` + Worker string `json:"worker"` + LeaseToken string `json:"lease_token"` + Attempt uint32 `json:"attempt"` + RouteAlias string `json:"route_alias"` + TargetID string `json:"target_id"` + BindingID string `json:"binding_id"` + WorkContext string `json:"work_context"` + Installation json.RawMessage `json:"installation"` +} + +// Accounts retains the queue and signer. This consumer is here because the +// gateway already owns the verified registry resolution and workload transport. +// No user route can submit an upstream, credential, installation or job to it. +type installationJobConsumer struct { + accountsURL, internalToken, worker string + client *http.Client + deliver func(context.Context, *installationStepClaim) (int, string) +} + +func (g *Gateway) runInstallationJobs(ctx context.Context, accountsURL string) { + c := &installationJobConsumer{accountsURL: accountsURL, internalToken: g.authz.internalToken, + worker: "gateway-install-" + uuid.NewString(), client: &http.Client{Timeout: 10 * time.Second, + CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}, deliver: g.deliverInstallationStep} + ticker := time.NewTicker(time.Second) + defer ticker.Stop() + for { + if err := c.runOnce(ctx); err != nil && ctx.Err() == nil { + log.Printf("installation consumer: %v", err) + } + select { + case <-ctx.Done(): + return + case <-ticker.C: + } + } +} + +func (c *installationJobConsumer) post(ctx context.Context, operation string, body any) (*http.Response, error) { + raw, err := json.Marshal(body) + if err != nil { + return nil, err + } + req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.accountsURL+"/internal/installation-steps/"+operation, bytes.NewReader(raw)) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-Codefly-Internal-Token", c.internalToken) + return c.client.Do(req) +} + +func (c *installationJobConsumer) runOnce(ctx context.Context) error { + resp, err := c.post(ctx, "claim", map[string]string{"worker": c.worker}) + if err != nil { + return errors.New("accounts claim unavailable") + } + defer resp.Body.Close() + if resp.StatusCode == http.StatusNoContent { + return nil + } + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("accounts claim returned %d", resp.StatusCode) + } + var claim installationStepClaim + if err := json.NewDecoder(io.LimitReader(resp.Body, 128<<10)).Decode(&claim); err != nil { + return errors.New("invalid accounts claim") + } + if claim.Worker != c.worker || claim.JobID == "" || claim.LeaseToken == "" || claim.WorkContext == "" || claim.TargetID == "" || claim.BindingID == "" { + return errors.New("incomplete accounts claim") + } + callCtx, cancel := context.WithTimeout(ctx, installationCallTimeout) + status, reason := c.deliver(callCtx, &claim) + cancel() + outcome, err := c.post(ctx, "finish", map[string]any{"job_id": claim.JobID, "worker": claim.Worker, "lease_token": claim.LeaseToken, "attempt": claim.Attempt, "status": status, "reason": reason}) + if err != nil { + return errors.New("accounts outcome unavailable; lease expiry will recover") + } + defer outcome.Body.Close() + if outcome.StatusCode != http.StatusNoContent { + return fmt.Errorf("accounts outcome returned %d", outcome.StatusCode) + } + return nil +} + +func (g *Gateway) deliverInstallationStep(ctx context.Context, claim *installationStepClaim) (int, string) { + entry, resolution := g.solutions.resolveRoutingOfKind(ctx, claim.RouteAlias, solutionSurface) + if resolution != solutionRoutable || entry.TargetID != claim.TargetID || entry.BindingID != claim.BindingID { + // Named, because a step that cannot be delivered is retried and then + // dead-lettered with a generic reason, and the operator reading the + // installation needs to know which of these it was. + log.Printf("installation step: route %q is not deliverable: resolution=%v target=%q/%q binding=%q/%q", + claim.RouteAlias, resolution, entry.TargetID, claim.TargetID, entry.BindingID, claim.BindingID) + return http.StatusServiceUnavailable, "" + } + transport, err := g.workload.upstreamTransport(entry.WorkloadSPIFFEID) + if err != nil { + log.Printf("installation step: route %q: no transport to workload %q: %v", claim.RouteAlias, entry.WorkloadSPIFFEID, err) + return http.StatusServiceUnavailable, "" + } + defer transport.CloseIdleConnections() + return sendInstallationStep(ctx, &http.Client{Transport: transport, Timeout: installationCallTimeout, + CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}, entry.Upstream.String()+"/_codefly/installations", claim) +} + +func sendInstallationStep(ctx context.Context, client *http.Client, destination string, claim *installationStepClaim) (int, string) { + req, err := http.NewRequestWithContext(ctx, http.MethodPost, destination, bytes.NewReader(claim.Installation)) + if err != nil { + return http.StatusServiceUnavailable, "" + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set(workcontext.HeaderName, claim.WorkContext) + // The capability is sealed to one installation and revision; the carrier + // names them beside it, and the runtime refuses a step whose carrier and + // seal disagree (sdk-go/workcontext checkCarriedInstallation). They are + // read off the step itself, which accounts wrote from the same record. + var sealed struct { + InstallationID string `json:"installation_id"` + Revision uint64 `json:"revision"` + } + if err := json.Unmarshal(claim.Installation, &sealed); err != nil || sealed.InstallationID == "" { + log.Printf("installation step: %s: the step names no installation: %v", destination, err) + return http.StatusServiceUnavailable, "" + } + req.Header.Set(workcontext.InstallationIDHeaderName, sealed.InstallationID) + req.Header.Set(workcontext.InstallationRevisionHeaderName, strconv.FormatUint(sealed.Revision, 10)) + resp, err := client.Do(req) + if err != nil { + log.Printf("installation step: %s: %v", destination, err) + return http.StatusServiceUnavailable, "" + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusNoContent && resp.StatusCode != http.StatusConflict { + log.Printf("installation step: %s answered %d", destination, resp.StatusCode) + } + switch resp.StatusCode { + case http.StatusNoContent: + return http.StatusNoContent, "" + case http.StatusConflict: + // Only the explicit public field crosses into job history. Accounts + // bounds it again in UTF-8 bytes before persisting the refusal. + var refusal struct { + Reason string `json:"reason"` + } + decoder := json.NewDecoder(io.LimitReader(resp.Body, 4097)) + if err := decoder.Decode(&refusal); err != nil || decoder.Decode(new(any)) != io.EOF { + return http.StatusConflict, "" + } + return http.StatusConflict, refusal.Reason + default: + return http.StatusServiceUnavailable, "" + } +} diff --git a/module/services/auth-gateway/code/gateway_installation_jobs_test.go b/module/services/auth-gateway/code/gateway_installation_jobs_test.go new file mode 100644 index 000000000..a8fa16f31 --- /dev/null +++ b/module/services/auth-gateway/code/gateway_installation_jobs_test.go @@ -0,0 +1,111 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "io" + "net/http" + "strings" + "testing" + + "github.com/codefly-dev/sdk-go/workcontext" + "github.com/stretchr/testify/require" +) + +type installationRoundTrip func(*http.Request) (*http.Response, error) + +func (f installationRoundTrip) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) } + +// HOST-INST-003..005: the worker carries the installation credential and the +// exact revision/scope, acknowledges only 204, and retries every transport fault. +func TestInstallationDeliveryOutcomes(t *testing.T) { + for _, tc := range []struct { + name string + status int + body string + err error + want int + reason string + }{ + {"acknowledged", 204, "", nil, 204, ""}, + {"refused", 409, `{"reason":"Example refusal"}`, nil, 409, "Example refusal"}, + {"malformed refusal", 409, `private diagnostic`, nil, 409, ""}, + {"unavailable", 503, `private diagnostic`, nil, 503, ""}, + {"unexpected success", 200, "", nil, 503, ""}, + {"redirect", 302, "", nil, 503, ""}, + {"transport", 0, "", errors.New("private transport diagnostic"), 503, ""}, + } { + t.Run(tc.name, func(t *testing.T) { + claim := &installationStepClaim{WorkContext: "installation-credential", Installation: json.RawMessage(`{"installation_id":"example-id","organization_id":"example-org","revision":7,"scope":[]}`)} + client := &http.Client{Transport: installationRoundTrip(func(r *http.Request) (*http.Response, error) { + require.Equal(t, "POST", r.Method) + require.Equal(t, "/_codefly/installations", r.URL.Path) + require.Equal(t, claim.WorkContext, r.Header.Get(workcontext.HeaderName)) + // The carrier names the installation the capability is sealed to, + // read off the step accounts wrote: the runtime refuses a step + // whose carrier and seal disagree. + require.Equal(t, "example-id", r.Header.Get(workcontext.InstallationIDHeaderName)) + require.Equal(t, "7", r.Header.Get(workcontext.InstallationRevisionHeaderName)) + require.Empty(t, r.Header.Get("Authorization")) + require.Empty(t, r.Header.Get("X-Codefly-Internal-Token")) + raw, err := io.ReadAll(r.Body) + require.NoError(t, err) + require.JSONEq(t, string(claim.Installation), string(raw)) + if tc.err != nil { + return nil, tc.err + } + return &http.Response{StatusCode: tc.status, Header: http.Header{}, Body: io.NopCloser(strings.NewReader(tc.body))}, nil + })} + status, reason := sendInstallationStep(context.Background(), client, "https://solution.example/_codefly/installations", claim) + require.Equal(t, tc.want, status) + require.Equal(t, tc.reason, reason) + }) + } +} + +func TestInstallationConsumerReportsLeaseAndOutcome(t *testing.T) { + var operations []string + c := &installationJobConsumer{accountsURL: "http://accounts.example", internalToken: "private-token", worker: "example-worker"} + c.client = &http.Client{Transport: installationRoundTrip(func(r *http.Request) (*http.Response, error) { + operations = append(operations, r.URL.Path) + require.Equal(t, "private-token", r.Header.Get("X-Codefly-Internal-Token")) + if strings.HasSuffix(r.URL.Path, "/claim") { + return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader(`{"job_id":"job","worker":"example-worker","lease_token":"fence","attempt":3,"target_id":"target","binding_id":"binding","work_context":"credential","installation":{"revision":7}}`))}, nil + } + var result map[string]any + require.NoError(t, json.NewDecoder(r.Body).Decode(&result)) + require.Equal(t, map[string]any{"job_id": "job", "worker": "example-worker", "lease_token": "fence", "attempt": float64(3), "status": float64(409), "reason": "Example refusal"}, result) + return &http.Response{StatusCode: 204, Body: io.NopCloser(strings.NewReader(""))}, nil + })} + c.deliver = func(ctx context.Context, claim *installationStepClaim) (int, string) { + _, ok := ctx.Deadline() + require.True(t, ok) + require.Equal(t, "credential", claim.WorkContext) + return 409, "Example refusal" + } + require.NoError(t, c.runOnce(context.Background())) + require.Equal(t, []string{"/internal/installation-steps/claim", "/internal/installation-steps/finish"}, operations) +} + +// A step that names no installation has nothing to seal the carrier to, so it +// is never sent: the solution would refuse it, and the attempt is spent the way +// an unavailable solution spends one. +func TestAnInstallationStepNamingNoInstallationIsNotSent(t *testing.T) { + for name, installation := range map[string]string{ + "no id": `{"organization_id":"example-org","revision":7,"scope":[]}`, + "not json": `not json`, + "empty id": `{"installation_id":"","revision":7}`, + } { + t.Run(name, func(t *testing.T) { + claim := &installationStepClaim{WorkContext: "installation-credential", Installation: json.RawMessage(installation)} + client := &http.Client{Transport: installationRoundTrip(func(*http.Request) (*http.Response, error) { + t.Fatal("a step naming no installation reached the solution") + return nil, nil + })} + status, reason := sendInstallationStep(context.Background(), client, "https://solution.example/_codefly/installations", claim) + require.Equal(t, http.StatusServiceUnavailable, status) + require.Empty(t, reason) + }) + } +} diff --git a/module/services/auth-gateway/code/gateway_modules.go b/module/services/auth-gateway/code/gateway_modules.go index ac65babbd..c85897d24 100644 --- a/module/services/auth-gateway/code/gateway_modules.go +++ b/module/services/auth-gateway/code/gateway_modules.go @@ -9,6 +9,7 @@ import ( "log" "net" "net/http" + "net/url" "strings" "time" @@ -178,7 +179,8 @@ func (g *Gateway) handleModuleWorkContext(w http.ResponseWriter, r *http.Request return true } - if g.authz == nil || !g.authz.acceptsInternalToken(r.Header.Get("X-Codefly-Internal-Token")) { + _, workloadRequest := workloadPeer(r.Context()) + if g.authz == nil || (!workloadRequest && !g.authz.acceptsInternalToken(r.Header.Get("X-Codefly-Internal-Token"))) { httpError(w, http.StatusUnauthorized, "unauthorized") return true } @@ -191,8 +193,16 @@ func (g *Gateway) handleModuleWorkContext(w http.ResponseWriter, r *http.Request var payload struct { Prefix string `json:"prefix"` OrganizationID string `json:"organization_id,omitempty"` - } - if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&payload); err != nil { + // Upstream is the origin the workload serves its backend at: the + // dynamic half of its registration, carried to accounts with the + // token that proves who reports it. Bounded here as every registered + // upstream was, before accounts bounds it again. + Upstream string `json:"upstream,omitempty"` + // Manifest is the solution's frontend manifest, the other dynamic + // half, bounded by accounts at 64 KiB. + Manifest string `json:"manifest,omitempty"` + } + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 80<<10)).Decode(&payload); err != nil { httpError(w, http.StatusBadRequest, "invalid json") return true } @@ -200,17 +210,42 @@ func (g *Gateway) handleModuleWorkContext(w http.ResponseWriter, r *http.Request httpError(w, http.StatusBadRequest, "invalid prefix") return true } + // The dynamic half is reported only where the reporter's identity is on + // the connection. On the plain endpoint the internal token is all that + // separates a caller from a workload's projected token, and a holder of + // both could move that workload's recorded origin and manifest. + if !workloadRequest && (payload.Upstream != "" || payload.Manifest != "") { + httpError(w, http.StatusBadRequest, "upstream and manifest are reported on the workload endpoint") + return true + } + if payload.Upstream != "" { + parsed, err := url.Parse(payload.Upstream) + if err != nil || parsed.Scheme != "https" || parsed.Host == "" || parsed.Port() == "" || parsed.User != nil || + (parsed.Path != "" && parsed.Path != "/") || parsed.RawQuery != "" || parsed.Fragment != "" || + isDisallowedRegisteredUpstreamHost(parsed.Hostname()) { + httpError(w, http.StatusBadRequest, "invalid upstream") + return true + } + } ctx, cancel := context.WithTimeout(r.Context(), moduleExchangeTimeout) defer cancel() - issued, err := g.authz.mintModuleWorkContext(ctx, payload.Prefix, serviceAccountToken, payload.OrganizationID) + issued, err := g.authz.mintModuleWorkContext(ctx, payload.Prefix, serviceAccountToken, payload.OrganizationID, payload.Upstream, payload.Manifest) if err != nil { logModuleExchangeRefusal("module-work-context", payload.Prefix, err) switch status.Code(err) { case codes.PermissionDenied: - httpError(w, http.StatusUnauthorized, "unauthorized") + if workloadRequest { + httpError(w, http.StatusForbidden, "forbidden") + } else { + httpError(w, http.StatusUnauthorized, "unauthorized") + } case codes.InvalidArgument: httpError(w, http.StatusBadRequest, "invalid request") + case codes.FailedPrecondition: + // accounts admits the workload but no organization has installed it + // yet: the module serves its install endpoint and waits. + httpError(w, http.StatusPreconditionFailed, "installation pending") case codes.Unavailable: // accounts could not establish what the caller is running: nothing // was decided, so the module retries rather than stops. @@ -336,15 +371,29 @@ func (g *Gateway) handleModuleOperationContext(w http.ResponseWriter, r *http.Re return true } +// moduleMintContext carries the listener attestation on every module mint. +func (s *ExtAuthz) moduleMintContext(ctx context.Context) context.Context { + ctx = metadata.AppendToOutgoingContext(ctx, "x-codefly-internal-token", s.internalToken) + // Replace, never append to, caller metadata. Only the verified listener + // context may attest a workload to accounts' internal mint. + md, _ := metadata.FromOutgoingContext(ctx) + md = md.Copy() + md.Delete(workloadSPIFFEHeader) + if id, ok := workloadPeer(ctx); ok { + md.Set(workloadSPIFFEHeader, id) + } + return metadata.NewOutgoingContext(ctx, md) +} + func (s *ExtAuthz) mintModuleWorkContext( - ctx context.Context, prefix, serviceAccountToken, organizationID string, + ctx context.Context, prefix, serviceAccountToken, organizationID, upstream, manifest string, ) (*accountsv1.ModuleMintWorkContextResponse, error) { if s.backendConn == nil { return nil, fmt.Errorf("accounts connection not configured") } - ctx = metadata.AppendToOutgoingContext(ctx, "x-codefly-internal-token", s.internalToken) + ctx = s.moduleMintContext(ctx) response := &accountsv1.ModuleMintWorkContextResponse{} - request := &accountsv1.ModuleMintWorkContextRequest{Prefix: prefix, ServiceAccountToken: serviceAccountToken, OrganizationId: organizationID} + request := &accountsv1.ModuleMintWorkContextRequest{Prefix: prefix, ServiceAccountToken: serviceAccountToken, OrganizationId: organizationID, Upstream: upstream, Manifest: manifest} if err := s.backendConn.Invoke(ctx, mintModuleWorkContextMethod, request, response); err != nil { return nil, err } @@ -357,7 +406,7 @@ func (s *ExtAuthz) mintModuleOperationContext( if s.backendConn == nil { return nil, fmt.Errorf("accounts connection not configured") } - ctx = metadata.AppendToOutgoingContext(ctx, "x-codefly-internal-token", s.internalToken) + ctx = s.moduleMintContext(ctx) response := &accountsv1.ModuleMintOperationContextResponse{} request := &accountsv1.ModuleMintOperationContextRequest{Prefix: prefix, ServiceAccountToken: serviceAccountToken, Binding: binding, OrganizationId: organizationID} if err := s.backendConn.Invoke(ctx, mintModuleOperationContextMethod, request, response); err != nil { @@ -529,7 +578,7 @@ func (s *ExtAuthz) mintSourceOperationContext( if s.backendConn == nil { return nil, fmt.Errorf("accounts connection not configured") } - ctx = metadata.AppendToOutgoingContext(ctx, "x-codefly-internal-token", s.internalToken) + ctx = s.moduleMintContext(ctx) response := &accountsv1.ModuleMintSourceOperationContextResponse{} if err := s.backendConn.Invoke(ctx, mintSourceOperationContextMethod, request, response); err != nil { return nil, err diff --git a/module/services/auth-gateway/code/gateway_modules_test.go b/module/services/auth-gateway/code/gateway_modules_test.go index e6c7d7aaa..106a02287 100644 --- a/module/services/auth-gateway/code/gateway_modules_test.go +++ b/module/services/auth-gateway/code/gateway_modules_test.go @@ -1,6 +1,7 @@ package main import ( + "bytes" "context" "encoding/json" "fmt" @@ -188,7 +189,10 @@ func TestGateway_ModuleWorkContext_TakesOnlyTheProjectedTokenBearer(t *testing.T type fakeAccountsWorkContextMint struct { code codes.Code lastInternal string + lastPeer string lastPrefix string + lastUpstream string + lastManifest string lastOrganization string lastToken string requestCount int @@ -201,9 +205,11 @@ func (f *fakeAccountsWorkContextMint) handle(ctx context.Context, dec func(any) } f.requestCount++ f.lastPrefix = req.GetPrefix() + f.lastUpstream, f.lastManifest = req.GetUpstream(), req.GetManifest() f.lastOrganization = req.GetOrganizationId() f.lastToken = req.GetServiceAccountToken() if md, ok := metadata.FromIncomingContext(ctx); ok { + f.lastPeer = strings.Join(md.Get(workloadSPIFFEHeader), ",") if values := md.Get("x-codefly-internal-token"); len(values) > 0 { f.lastInternal = values[0] } @@ -325,6 +331,9 @@ func TestGateway_ModuleWorkContext_RelaysAccountsOutcome(t *testing.T) { "rejected request": {codes.InvalidArgument, http.StatusBadRequest}, // accounts could not establish what the pod runs: retryable, not a verdict. "execution unbound": {codes.Unavailable, http.StatusServiceUnavailable}, + // Admitted and installed nowhere: its own answer, so the module serves + // its install endpoint and waits instead of stopping on a refusal. + "installed nowhere": {codes.FailedPrecondition, http.StatusPreconditionFailed}, "outage": {codes.Internal, http.StatusBadGateway}, } { t.Run(name, func(t *testing.T) { @@ -340,6 +349,107 @@ func TestGateway_ModuleWorkContext_RelaysAccountsOutcome(t *testing.T) { } } +// asWorkload marks a request as arrived on the workload endpoint from peer, as +// the mutual-TLS listener's middleware does once it has verified the peer. +func asWorkload(req *http.Request, peer string) *http.Request { + return req.WithContext(context.WithValue(req.Context(), workloadPeerKey{}, peer)) +} + +const testWorkloadPeer = "spiffe://cluster.local/ns/cell-documents/sa/backend" + +// The dynamic half of a registration travels with the mint, to accounts, under +// the projected token that proves who reports it. The gateway forwards it and +// decides nothing about it beyond its shape. +func TestGateway_ModuleWorkContext_ForwardsTheReportedOriginAndManifest(t *testing.T) { + gw, mint := newWorkContextExchangeHarness(t) + + req := workContextRequest("documents", "documents-projected-token", "test-internal-token") + req.Body = io.NopCloser(strings.NewReader(`{"prefix":"documents","upstream":"https://backend.cell-documents.svc.cluster.local:8080","manifest":"{\"id\":\"documents\"}"}`)) + w := httptest.NewRecorder() + gw.ServeHTTP(w, asWorkload(req, testWorkloadPeer)) + + require.Equal(t, http.StatusOK, w.Code) + require.Equal(t, "https://backend.cell-documents.svc.cluster.local:8080", mint.lastUpstream) + require.Equal(t, `{"id":"documents"}`, mint.lastManifest) +} + +// An origin that is not exactly an https host and port, or names a host no +// registered upstream may name, is refused at the edge: accounts is never asked +// to record it, and no credential is minted beside it. +func TestGateway_ModuleWorkContext_RefusesAnOriginThatIsNotOne(t *testing.T) { + for name, upstream := range map[string]string{ + "plaintext": "http://backend.cell-documents.svc.cluster.local:8080", + "no port": "https://backend.cell-documents.svc.cluster.local", + "a path": "https://backend.cell-documents.svc.cluster.local:8080/api", + "a query": "https://backend.cell-documents.svc.cluster.local:8080?x=1", + "userinfo": "https://ops@backend.cell-documents.svc.cluster.local:8080", + "cloud metadata": "https://169.254.169.254:443", + "a public host": "https://example.com:443", + "not a URL": "backend:8080", + "another scheme": "ftp://backend.cell-documents.svc.cluster.local:8080", + "fragment": "https://backend.cell-documents.svc.cluster.local:8080#x", + "only whitespace": " ", + } { + t.Run(name, func(t *testing.T) { + gw, mint := newWorkContextExchangeHarness(t) + body, err := json.Marshal(map[string]string{"prefix": "documents", "upstream": upstream}) + require.NoError(t, err) + req := workContextRequest("documents", "documents-projected-token", "test-internal-token") + req.Body = io.NopCloser(bytes.NewReader(body)) + w := httptest.NewRecorder() + gw.ServeHTTP(w, asWorkload(req, testWorkloadPeer)) + + require.Equal(t, http.StatusBadRequest, w.Code) + require.Zero(t, mint.requestCount) + }) + } +} + +// On the plain endpoint the internal token is all that separates a caller from +// a workload's projected token, so the dynamic half is not accepted there: a +// holder of both could move that workload's recorded origin and manifest. +func TestGateway_ModuleWorkContext_TheOriginIsReportedOnlyOnTheWorkloadEndpoint(t *testing.T) { + for name, body := range map[string]string{ + "an origin": `{"prefix":"documents","upstream":"https://backend.cell-documents.svc.cluster.local:8080"}`, + "a manifest": `{"prefix":"documents","manifest":"{}"}`, + } { + t.Run(name, func(t *testing.T) { + gw, mint := newWorkContextExchangeHarness(t) + req := workContextRequest("documents", "documents-projected-token", "test-internal-token") + req.Body = io.NopCloser(strings.NewReader(body)) + w := httptest.NewRecorder() + gw.ServeHTTP(w, req) + + require.Equal(t, http.StatusBadRequest, w.Code) + require.Zero(t, mint.requestCount) + }) + } +} + +// The body is bounded: the manifest is the only large field, and a request +// past the bound is refused before accounts is asked anything. +func TestGateway_ModuleWorkContext_BoundsTheRequest(t *testing.T) { + gw, mint := newWorkContextExchangeHarness(t) + body, err := json.Marshal(map[string]string{"prefix": "documents", "manifest": strings.Repeat("a", 81<<10)}) + require.NoError(t, err) + req := workContextRequest("documents", "documents-projected-token", "test-internal-token") + req.Body = io.NopCloser(bytes.NewReader(body)) + w := httptest.NewRecorder() + gw.ServeHTTP(w, asWorkload(req, testWorkloadPeer)) + + require.Equal(t, http.StatusBadRequest, w.Code) + require.Zero(t, mint.requestCount) + + body, err = json.Marshal(map[string]string{"prefix": "documents", "manifest": strings.Repeat("a", 60<<10)}) + require.NoError(t, err) + req = workContextRequest("documents", "documents-projected-token", "test-internal-token") + req.Body = io.NopCloser(bytes.NewReader(body)) + w = httptest.NewRecorder() + gw.ServeHTTP(w, asWorkload(req, testWorkloadPeer)) + require.Equal(t, http.StatusOK, w.Code) + require.Equal(t, 1, mint.requestCount) +} + func TestGateway_ModuleWorkContext_MethodNotAllowed(t *testing.T) { gw, _ := newWorkContextExchangeHarness(t) diff --git a/module/services/auth-gateway/code/gateway_solution_admission_test.go b/module/services/auth-gateway/code/gateway_solution_admission_test.go index 86b67c026..f42638183 100644 --- a/module/services/auth-gateway/code/gateway_solution_admission_test.go +++ b/module/services/auth-gateway/code/gateway_solution_admission_test.go @@ -527,7 +527,7 @@ func replacementArrivesMidRequest( t.Helper() predecessor = registerSolutionUpstream(t, gw, alias) replacement = &fakeUpstream{body: "replacement-response"} - replacementSrv := httptest.NewServer(replacement) + replacementSrv := newSolutionTestServer(t, gw, alias, replacement) t.Cleanup(replacementSrv.Close) registry := solutionRegistryFake(t, gw) diff --git a/module/services/auth-gateway/code/gateway_solution_registry.go b/module/services/auth-gateway/code/gateway_solution_registry.go index 81c7f8e03..3258d3de9 100644 --- a/module/services/auth-gateway/code/gateway_solution_registry.go +++ b/module/services/auth-gateway/code/gateway_solution_registry.go @@ -17,6 +17,7 @@ import ( "google.golang.org/grpc" "google.golang.org/grpc/metadata" + "google.golang.org/protobuf/proto" ) const ( @@ -38,11 +39,17 @@ const ( solutionRefreshFloor = 250 * time.Millisecond ) -// solutionRegistryClient is the registry surface the gateway consumes. The -// interface exists so the cache and the HTTP handlers can be exercised without -// an accounts process. +// solutionRegistrySnapshot joins only matching applied generations. The compact +// registration has no workload field; the host-binding read carries Core's +// complete presence document. No new wire schema duplicates that contract. +type solutionRegistrySnapshot struct { + *accountsv1.ListSolutionRegistrationsResponse + workloadIDs map[string]string +} + +// solutionRegistryClient is the registry surface used by the cache and handlers. type solutionRegistryClient interface { - List(ctx context.Context, req *accountsv1.ListSolutionRegistrationsRequest) (*accountsv1.ListSolutionRegistrationsResponse, error) + List(ctx context.Context, req *accountsv1.ListSolutionRegistrationsRequest) (*solutionRegistrySnapshot, error) } // accountsSolutionRegistry invokes the registry by method name over the @@ -62,9 +69,29 @@ func (c *accountsSolutionRegistry) invoke(ctx context.Context, method string, re return c.conn.Invoke(ctx, method, req, resp) } -func (c *accountsSolutionRegistry) List(ctx context.Context, req *accountsv1.ListSolutionRegistrationsRequest) (*accountsv1.ListSolutionRegistrationsResponse, error) { +func (c *accountsSolutionRegistry) List(ctx context.Context, req *accountsv1.ListSolutionRegistrationsRequest) (*solutionRegistrySnapshot, error) { resp := &accountsv1.ListSolutionRegistrationsResponse{} - return resp, c.invoke(ctx, listSolutionRegistrationsMethod, req, resp) + if err := c.invoke(ctx, listSolutionRegistrationsMethod, req, resp); err != nil { + return nil, err + } + bindings := &accountsv1.ListSolutionHostBindingsResponse{} + if err := c.invoke(ctx, "/saas.accounts.v1.SolutionRegistryService/ListSolutionHostBindings", &accountsv1.ListSolutionHostBindingsRequest{}, bindings); err != nil { + return nil, err + } + out := &solutionRegistrySnapshot{ListSolutionRegistrationsResponse: resp, workloadIDs: make(map[string]string)} + for _, record := range resp.GetRegistrations() { + for _, binding := range bindings.GetBindings() { + // These reads may straddle reconciliation. Never attach a successor's + // identity to its predecessor's address, even within one generation. + if !proto.Equal(record, binding.GetRegistration()) { + continue + } + if id := appliedWorkloadSPIFFEID(binding); id != "" { + out.workloadIDs[record.GetSolutionId()] = id + } + } + } + return out, nil } // solutionResolution is why a solution is or is not routable right now. The @@ -99,10 +126,11 @@ type solutionRegistryCache struct { // it at time.Now. now func() time.Time - mu sync.RWMutex - records map[string]*accountsv1.SolutionRegistration - revision int64 - loaded bool + mu sync.RWMutex + records map[string]*accountsv1.SolutionRegistration + workloadIDs map[string]string + revision int64 + loaded bool // attemptedAt is when a read was last ATTEMPTED, successfully or not. The // refresh floor is measured from it rather than from the last SUCCESS: a // failed read leaves the snapshot's age where it was, so measuring from that @@ -182,6 +210,7 @@ func (c *solutionRegistryCache) load(ctx context.Context) error { records[record.GetSolutionId()] = record } c.records = records + c.workloadIDs = resp.workloadIDs c.revision = resp.GetRegistryRevision() c.loaded = true c.lastErr = nil @@ -210,11 +239,11 @@ func (c *solutionRegistryCache) reconcile(ctx context.Context) { } // lookup reads the cached record without touching the network. -func (c *solutionRegistryCache) lookup(id string) (*accountsv1.SolutionRegistration, bool, bool) { +func (c *solutionRegistryCache) lookup(id string) (*accountsv1.SolutionRegistration, string, bool, bool) { c.mu.RLock() defer c.mu.RUnlock() record, ok := c.records[id] - return record, ok, c.loaded + return record, c.workloadIDs[id], ok, c.loaded } func (c *solutionRegistryCache) staleEnoughToRefresh() bool { @@ -227,8 +256,9 @@ func (c *solutionRegistryCache) staleEnoughToRefresh() bool { // a verdict must never be derived without: whether the registry could actually // be consulted about it. type solutionLookup struct { - record *accountsv1.SolutionRegistration - found bool + record *accountsv1.SolutionRegistration + workloadID string + found bool // loaded is false until a snapshot has ever loaded on this replica. loaded bool // undecidable is set when this alias had no conclusive record in the @@ -273,17 +303,18 @@ func (c *solutionRegistryCache) conclusive( func (c *solutionRegistryCache) lookupFresh( ctx context.Context, id string, surface routingSurface, ) solutionLookup { - record, found, loaded := c.lookup(id) + record, workloadID, found, loaded := c.lookup(id) if c.conclusive(record, found, surface) { - return solutionLookup{record: record, found: found, loaded: loaded} + return solutionLookup{record: record, workloadID: workloadID, found: found, loaded: loaded} } refreshErr := c.refreshIfStale(ctx) if refreshErr != nil { log.Printf("solution registry: on-demand refresh for %q failed: %v", id, refreshErr) } - record, found, loaded = c.lookup(id) + record, workloadID, found, loaded = c.lookup(id) return solutionLookup{ record: record, + workloadID: workloadID, found: found, loaded: loaded, undecidable: refreshErr != nil && !c.conclusive(record, found, surface), @@ -310,6 +341,7 @@ func (c *solutionRegistryCache) lookupFresh( // that must refuse it is the one holding the viewer, and it refuses it as a // verdict rather than as a missing route. type solutionRouting struct { + WorkloadSPIFFEID string // TargetID is the immutable solution target this alias resolves to, empty // for a registration no declaration opened a target for. TargetID string @@ -409,10 +441,14 @@ func (c *solutionRegistryCache) resolveRoutingOfKind( log.Printf("solution registry: record %q has an unusable upstream", id) return nil, solutionNotActive } + if surface == solutionSurface { + upstream.Scheme = "https" + } return &solutionRouting{ - TargetID: record.GetDeclared().GetTargetId(), - BindingID: record.GetDeclared().GetBindingId(), - Upstream: &url.URL{Scheme: upstream.Scheme, Host: upstream.Host}, + WorkloadSPIFFEID: read.workloadID, + TargetID: record.GetDeclared().GetTargetId(), + BindingID: record.GetDeclared().GetBindingId(), + Upstream: &url.URL{Scheme: upstream.Scheme, Host: upstream.Host}, }, solutionRoutable } diff --git a/module/services/auth-gateway/code/gateway_solution_registry_fake_test.go b/module/services/auth-gateway/code/gateway_solution_registry_fake_test.go index 28529428e..63e4791b9 100644 --- a/module/services/auth-gateway/code/gateway_solution_registry_fake_test.go +++ b/module/services/auth-gateway/code/gateway_solution_registry_fake_test.go @@ -141,7 +141,7 @@ func (f *fakeSolutionRegistry) undeclare(alias string) { func (f *fakeSolutionRegistry) List( _ context.Context, req *accountsv1.ListSolutionRegistrationsRequest, -) (*accountsv1.ListSolutionRegistrationsResponse, error) { +) (*solutionRegistrySnapshot, error) { f.mu.Lock() defer f.mu.Unlock() f.listCalls++ @@ -157,7 +157,11 @@ func (f *fakeSolutionRegistry) List( listed.Status = fakeSolutionStatus(listed, time.Now()) out.Registrations = append(out.Registrations, listed) } - return out, nil + ids := make(map[string]string) + for _, record := range out.Registrations { + ids[record.GetSolutionId()] = "spiffe://acme.test/ns/acme-prod/sa/" + record.GetSolutionId() + } + return &solutionRegistrySnapshot{ListSolutionRegistrationsResponse: out, workloadIDs: ids}, nil } // fakeSolutionStatus mirrors business.(*SolutionRegistration).Status, ordering diff --git a/module/services/auth-gateway/code/gateway_solution_workload.go b/module/services/auth-gateway/code/gateway_solution_workload.go new file mode 100644 index 000000000..546ced037 --- /dev/null +++ b/module/services/auth-gateway/code/gateway_solution_workload.go @@ -0,0 +1,36 @@ +package main + +import ( + accountsv1 "auth-gateway/pkg/gen/saas/accounts/v1" + + "github.com/codefly-dev/core/solutionhost" +) + +// Read APPLIED presence only. Desired documents do not authorize traffic. The +// registration, applied generation and parsed document must name the same +// binding and generation. Presence can declare several workloads; until it +// identifies which serves an endpoint, only one distinct identity is routable. +func appliedWorkloadSPIFFEID(binding *accountsv1.SolutionHostBindingState) string { + applied := binding.GetApplied() + record := binding.GetRegistration() + declared := record.GetDeclared() + if applied == nil || applied.GetRemoved() || declared == nil || + declared.GetBindingId() != binding.GetBindingId() || + declared.GetGeneration() != applied.GetGeneration().GetGeneration() || + applied.GetSolutionId() != record.GetSolutionId() || record.GetTombstonedAt() != nil { + return "" + } + document, err := solutionhost.Parse([]byte(applied.GetGeneration().GetDocument())) + if err != nil || document.Binding != declared.GetBindingId() || document.Generation != declared.GetGeneration() { + return "" + } + id := "" + for _, workload := range document.Workloads { + candidate := workload.Identity.SPIFFEID + if !workloadSPIFFEPattern.MatchString(candidate) || (id != "" && id != candidate) { + return "" + } + id = candidate + } + return id +} diff --git a/module/services/auth-gateway/code/gateway_solution_workload_test.go b/module/services/auth-gateway/code/gateway_solution_workload_test.go new file mode 100644 index 000000000..106e338dc --- /dev/null +++ b/module/services/auth-gateway/code/gateway_solution_workload_test.go @@ -0,0 +1,89 @@ +package main + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + accountsv1 "auth-gateway/pkg/gen/saas/accounts/v1" + + "github.com/codefly-dev/core/solutionhost" + "github.com/stretchr/testify/require" + "google.golang.org/protobuf/proto" +) + +func TestAppliedSolutionWorkloadIdentity(t *testing.T) { + var document *solutionhost.SolutionHostBinding + for _, fixture := range solutionhost.FixturesOf(solutionhost.DocumentTypePresence) { + parsed, err := solutionhost.Parse(fixture.Document) + if err == nil && len(parsed.Workloads) > 0 { + document = parsed + break + } + } + require.NotNil(t, document, "Core must ship a readable workload presence fixture") + body, err := json.Marshal(document) + require.NoError(t, err) + binding := &accountsv1.SolutionHostBindingState{ + BindingId: document.Binding, + Applied: &accountsv1.SolutionHostBindingAppliedGeneration{SolutionId: "example", Generation: &accountsv1.SolutionHostBindingGeneration{Generation: document.Generation, Document: string(body)}}, + Registration: &accountsv1.SolutionRegistration{SolutionId: "example", Declared: &accountsv1.SolutionDeclaredBinding{BindingId: document.Binding, Generation: document.Generation}}, + } + require.Equal(t, document.Workloads[0].Identity.SPIFFEID, appliedWorkloadSPIFFEID(binding)) + for _, test := range []struct { + name string + mutate func(*accountsv1.SolutionHostBindingState) + }{ + {"another applied generation", func(b *accountsv1.SolutionHostBindingState) { b.Registration.Declared.Generation++ }}, + {"another binding", func(b *accountsv1.SolutionHostBindingState) { b.BindingId += "-other" }}, + {"another registry alias", func(b *accountsv1.SolutionHostBindingState) { b.Applied.SolutionId = "other" }}, + {"tombstone", func(b *accountsv1.SolutionHostBindingState) { b.Applied.Removed = true }}, + {"desired only", func(b *accountsv1.SolutionHostBindingState) { b.Desired = b.Applied.Generation; b.Applied = nil }}, + {"malformed document", func(b *accountsv1.SolutionHostBindingState) { b.Applied.Generation.Document = "{}" }}, + {"no workload", func(b *accountsv1.SolutionHostBindingState) { + copy := *document + copy.Workloads = nil + body, err := json.Marshal(copy) + require.NoError(t, err) + b.Applied.Generation.Document = string(body) + }}, + {"ambiguous workloads", func(b *accountsv1.SolutionHostBindingState) { + copy := *document + second := copy.Workloads[0] + second.Name += "-other" + second.Identity.SPIFFEID = "spiffe://acme.test/ns/acme-prod/sa/other" + copy.Workloads = append(append([]solutionhost.Workload{}, copy.Workloads...), second) + body, err := json.Marshal(copy) + require.NoError(t, err) + b.Applied.Generation.Document = string(body) + }}, + } { + t.Run(test.name, func(t *testing.T) { + b := proto.Clone(binding).(*accountsv1.SolutionHostBindingState) + test.mutate(b) + require.Empty(t, appliedWorkloadSPIFFEID(b)) + }) + } +} + +// Exercises the actual reverse-proxy path and guarded network dial. Unlike the +// in-memory TLS tests this test requires permission to bind loopback. +func TestSolutionProxyWorkloadIdentity(t *testing.T) { + gateway := &Gateway{} + reached := 0 + server := newSolutionTestServer(t, gateway, "example", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { reached++; _, _ = w.Write([]byte("solution")) })) + for _, test := range []struct { + id string + status int + }{ + {"spiffe://acme.test/ns/acme-prod/sa/example", http.StatusOK}, + {"spiffe://acme.test/ns/acme-prod/sa/other", http.StatusBadGateway}, + {"", http.StatusBadGateway}, + } { + recorder := httptest.NewRecorder() + gateway.proxyTo(recorder, httptest.NewRequest(http.MethodGet, "/assets/entry.js", nil), MustURL(server.URL), &RouteEntry{Service: "solution:example", WorkloadSPIFFEID: test.id}) + require.Equal(t, test.status, recorder.Code) + } + require.Equal(t, 1, reached) +} diff --git a/module/services/auth-gateway/code/gateway_solutions.go b/module/services/auth-gateway/code/gateway_solutions.go index 45a4681a4..07d16c581 100644 --- a/module/services/auth-gateway/code/gateway_solutions.go +++ b/module/services/auth-gateway/code/gateway_solutions.go @@ -190,9 +190,10 @@ func (g *Gateway) handleSolutionRequest(w http.ResponseWriter, r *http.Request) // strictly worse exposure than the authenticated path's. removePersonsSessionCredential(r.Header) entry := &RouteEntry{ - Service: "solution:" + id, - UpstreamPath: publicPath, - RateLimitClass: edgeRateLimitClassPublic, + Service: "solution:" + id, + WorkloadSPIFFEID: routing.WorkloadSPIFFEID, + UpstreamPath: publicPath, + RateLimitClass: edgeRateLimitClassPublic, } g.rateLimitThenProxy(w, r, routing.Upstream, entry) return true @@ -302,10 +303,11 @@ func (g *Gateway) handleSolutionRequest(w http.ResponseWriter, r *http.Request) class = edgeRateLimitClassStandardWrite } entry := &RouteEntry{ - Service: "solution:" + id, - UpstreamPath: "/" + path, - Protected: true, - RateLimitClass: class, + Service: "solution:" + id, + WorkloadSPIFFEID: routing.WorkloadSPIFFEID, + UpstreamPath: "/" + path, + Protected: true, + RateLimitClass: class, } // Metered FIRST, and the admission check runs inside the budget. // diff --git a/module/services/auth-gateway/code/gateway_solutions_test.go b/module/services/auth-gateway/code/gateway_solutions_test.go index 37de3fa9b..4bad955d2 100644 --- a/module/services/auth-gateway/code/gateway_solutions_test.go +++ b/module/services/auth-gateway/code/gateway_solutions_test.go @@ -22,7 +22,7 @@ import ( func registerSolutionUpstream(t *testing.T, gw *Gateway, id string) *fakeUpstream { t.Helper() fake := &fakeUpstream{body: "solution-response"} - srv := httptest.NewServer(fake) + srv := newSolutionTestServer(t, gw, id, fake) t.Cleanup(srv.Close) registerSolutionHalves(t, gw, id, srv.URL) return fake diff --git a/module/services/auth-gateway/code/main.go b/module/services/auth-gateway/code/main.go index 70d004e50..afd280f08 100644 --- a/module/services/auth-gateway/code/main.go +++ b/module/services/auth-gateway/code/main.go @@ -4,6 +4,7 @@ import ( "context" "fmt" "log" + "log/slog" "net" "net/http" "net/url" @@ -51,6 +52,31 @@ func main() { panic(fmt.Sprintf("Codefly REST endpoint is unavailable: %v", httpNetErr)) } httpPort = httpNet.Port + workload, err := loadWorkloadIdentity(func(key string) (string, error) { + return codefly.For(ctx).WorkspaceValue("workload-identity", key) + }) + if err != nil { + panic(err) + } + workloadListener, workloadTLS, err := workload.listen(func() (net.Listener, error) { + workloadNet, err := codefly.For(ctx).Endpoint("workload").API("http").ResolveNetworkInstance() + if err != nil || workloadNet == nil || workloadNet.Port == 0 { + return nil, fmt.Errorf("Codefly workload endpoint is unavailable: %v", err) + } + listener, err := net.Listen("tcp", fmt.Sprintf(":%d", workloadNet.Port)) + if err != nil { + return nil, fmt.Errorf("workload listener: %w", err) + } + return listener, nil + }) + if err != nil { + panic(err) + } + if workloadListener != nil { + defer workloadListener.Close() + } else { + slog.Info("workload endpoint is off because no workload identity is projected") + } var otelProvider *wooltel.Provider var otelMetricProvider *otelMetrics if observabilityEnabled() { @@ -185,7 +211,7 @@ func main() { routeEntries := append(restEntries, connectEntries...) // Private HTTP API gateway listener behind the frontend product entry. - var httpServer *http.Server + var httpServer, workloadServer *http.Server var rateLimiter *RateLimiter if httpPort > 0 { // Build upstream map from codefly network mappings. @@ -250,6 +276,8 @@ func main() { // reach this replica without restarting it. clientRegistry := &accountsClientRegistry{conn: internalAPIConn, internalToken: authz.internalToken} gateway := NewGateway(authz, matcher, upstreams, rateLimiter, solutionRegistry, clientRegistry) + gateway.workload = workload + go gateway.runInstallationJobs(ctx, apiHTTPURL) // The per-viewer solution entitlement read (#949), over the same internal // connection. It is not a cache: the answer is per viewer and must not // outlive a revoke, so every request asks. @@ -278,6 +306,19 @@ func main() { IdleTimeout: 120 * time.Second, MaxHeaderBytes: 1 << 20, // 1 MiB } + if workloadListener != nil { + workloadServer = &http.Server{ + Handler: workload.handler(httpServer.Handler), TLSConfig: workloadTLS, + ReadHeaderTimeout: httpServer.ReadHeaderTimeout, ReadTimeout: httpServer.ReadTimeout, + IdleTimeout: httpServer.IdleTimeout, MaxHeaderBytes: httpServer.MaxHeaderBytes, + } + go func() { + if err := workloadServer.ServeTLS(workloadListener, "", ""); err != nil && err != http.ErrServerClosed { + log.Printf("workload server: %v", err) + stop() + } + }() + } go func() { fmt.Printf("auth-gateway API gateway (HTTP) listening on :%d (api: %s, routes: %d)\n", httpPort, apiHTTPURL, len(routeEntries)) @@ -303,6 +344,14 @@ func main() { drainCancel() } + if workloadServer != nil { + drainCtx, drainCancel := context.WithTimeout(context.Background(), 30*time.Second) + if err := workloadServer.Shutdown(drainCtx); err != nil { + log.Printf("shutdown: workload drain: %v", err) + } + drainCancel() + } + // 2. Gracefully stop gRPC server (finishes in-flight RPCs). log.Println("shutdown: stopping gRPC server...") grpcServer.GracefulStop() diff --git a/module/services/auth-gateway/code/perimeter_workos_free_test.go b/module/services/auth-gateway/code/perimeter_workos_free_test.go index fffe2a43b..b4f62b01b 100644 --- a/module/services/auth-gateway/code/perimeter_workos_free_test.go +++ b/module/services/auth-gateway/code/perimeter_workos_free_test.go @@ -53,12 +53,16 @@ var allowedPerimeterConfigDeps = map[string]bool{ // to require a token's resource audience to name it, so the origin a client // is sent to and the origin a token is accepted at are one operator-pinned // value rather than two headers that can disagree. - "application": true, - "gateway": true, - "gateway-trust": true, - "internal-auth": true, - "observability": true, - "security": true, + "application": true, + // workload-identity carries only file paths to this workload's own SVID, + // trust bundle and admitted-caller list, all projected by the platform. + // Reviewed for #1074: no external-IdP material and no provider endpoint. + "workload-identity": true, + "gateway": true, + "gateway-trust": true, + "internal-auth": true, + "observability": true, + "security": true, } // TestPerimeter_WorkspaceConfigExcludesIdentity asserts the ext_authz check's @@ -148,6 +152,12 @@ func fileImports(t *testing.T, path string) []string { var httpAllowedPerimeterFiles = map[string]bool{ "gateway.go": true, "gateway_solutions.go": true, + // Calls the installed solution's install endpoint over the in-mesh workload-identity transport; no external IdP. + "gateway_installation_jobs.go": true, + // workload_identity.go builds the gateway's mutual-TLS listener and the + // in-mesh transport it proxies to solutions with, holding each peer to a + // SPIFFE ID from delivered presence. It reaches no external IdP. + "workload_identity.go": true, // gateway_solution_credential.go serves the solution registration-token // exchange and verifies the resulting credential with local Ed25519 crypto // against this cluster's own published key set. Its only network call is the diff --git a/module/services/auth-gateway/code/pkg/gen/saas/accounts/v1/module_registration.pb.go b/module/services/auth-gateway/code/pkg/gen/saas/accounts/v1/module_registration.pb.go index a19441f6c..1a10713a5 100644 --- a/module/services/auth-gateway/code/pkg/gen/saas/accounts/v1/module_registration.pb.go +++ b/module/services/auth-gateway/code/pkg/gen/saas/accounts/v1/module_registration.pb.go @@ -44,8 +44,21 @@ type ModuleMintWorkContextRequest struct { // tenant-pinned principal, omit or match its tenant. Accounts validates the // requested organization's live installation after authenticating execution. OrganizationId string `protobuf:"bytes,4,opt,name=organization_id,json=organizationId,proto3" json:"organization_id,omitempty"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + Upstream string `protobuf:"bytes,5,opt,name=upstream,proto3" json:"upstream,omitempty"` + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + Manifest string `protobuf:"bytes,6,opt,name=manifest,proto3" json:"manifest,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *ModuleMintWorkContextRequest) Reset() { @@ -99,6 +112,20 @@ func (x *ModuleMintWorkContextRequest) GetOrganizationId() string { return "" } +func (x *ModuleMintWorkContextRequest) GetUpstream() string { + if x != nil { + return x.Upstream + } + return "" +} + +func (x *ModuleMintWorkContextRequest) GetManifest() string { + if x != nil { + return x.Manifest + } + return "" +} + // ModuleMintWorkContextResponse carries the signed capability plus the identity // it asserts, so a module can attribute its own work without parsing the token. type ModuleMintWorkContextResponse struct { @@ -575,11 +602,13 @@ var File_saas_accounts_v1_module_registration_proto protoreflect.FileDescriptor const file_saas_accounts_v1_module_registration_proto_rawDesc = "" + "\n" + - "*saas/accounts/v1/module_registration.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1fgoogle/protobuf/timestamp.proto\"\xe9\x01\n" + + "*saas/accounts/v1/module_registration.proto\x12\x10saas.accounts.v1\x1a\x1bbuf/validate/validate.proto\x1a\x1fgoogle/protobuf/timestamp.proto\"\xbf\x02\n" + "\x1cModuleMintWorkContextRequest\x12D\n" + "\x06prefix\x18\x01 \x01(\tB,\xbaH)r'\x10\x01\x18?2!^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$R\x06prefix\x12?\n" + "\x15service_account_token\x18\x03 \x01(\tB\v\xbaH\br\x06\x10\x01\x18\x80\x80\x04R\x13serviceAccountToken\x124\n" + - "\x0forganization_id\x18\x04 \x01(\tB\v\xbaH\b\xd8\x01\x01r\x03\xb0\x01\x01R\x0eorganizationIdJ\x04\b\x02\x10\x03R\x06secret\"\xab\x01\n" + + "\x0forganization_id\x18\x04 \x01(\tB\v\xbaH\b\xd8\x01\x01r\x03\xb0\x01\x01R\x0eorganizationId\x12*\n" + + "\bupstream\x18\x05 \x01(\tB\x0e\xbaH\v\xd8\x01\x01r\x06\x18\x80\x04\x88\x01\x01R\bupstream\x12(\n" + + "\bmanifest\x18\x06 \x01(\tB\f\xbaH\t\xd8\x01\x01r\x04\x18\x80\x80\x04R\bmanifestJ\x04\b\x02\x10\x03R\x06secret\"\xab\x01\n" + "\x1dModuleMintWorkContextResponse\x12\x14\n" + "\x05token\x18\x01 \x01(\tR\x05token\x129\n" + "\n" + diff --git a/module/services/auth-gateway/code/routing.go b/module/services/auth-gateway/code/routing.go index 55744dbea..58878b5f4 100644 --- a/module/services/auth-gateway/code/routing.go +++ b/module/services/auth-gateway/code/routing.go @@ -39,6 +39,7 @@ type RouteExtension struct { // RouteEntry is the internal representation used by the gateway. type RouteEntry struct { + WorkloadSPIFFEID string // declared TLS peer for a solution upstream Service string // upstream service name (from directory path) Method string // HTTP method Path string // public HTTP path (may contain {param} segments) diff --git a/module/services/auth-gateway/code/routing_catalog_gen.go b/module/services/auth-gateway/code/routing_catalog_gen.go index 537bc0de6..dae8fd049 100644 --- a/module/services/auth-gateway/code/routing_catalog_gen.go +++ b/module/services/auth-gateway/code/routing_catalog_gen.go @@ -90,6 +90,7 @@ func generatedCatalogConnectRoutes() []*RouteEntry { {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/ListAvailableSolutions", UpstreamPath: "/saas.accounts.v1.InstallationService/ListAvailableSolutions", Procedure: "/saas.accounts.v1.InstallationService/ListAvailableSolutions"}, {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/ListOrganizationInstallations", UpstreamPath: "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", Procedure: "/saas.accounts.v1.InstallationService/ListOrganizationInstallations"}, {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/NarrowInstallation", UpstreamPath: "/saas.accounts.v1.InstallationService/NarrowInstallation", Procedure: "/saas.accounts.v1.InstallationService/NarrowInstallation"}, + {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/RetryInstallation", UpstreamPath: "/saas.accounts.v1.InstallationService/RetryInstallation", Procedure: "/saas.accounts.v1.InstallationService/RetryInstallation"}, {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/RevokeInstallationFromTeam", UpstreamPath: "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", Procedure: "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam"}, {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/TransferInstallationOwnership", UpstreamPath: "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", Procedure: "/saas.accounts.v1.InstallationService/TransferInstallationOwnership"}, {Service: "accounts_connect", Method: "POST", Path: "/customers.InstallationService/UninstallSolution", UpstreamPath: "/saas.accounts.v1.InstallationService/UninstallSolution", Procedure: "/saas.accounts.v1.InstallationService/UninstallSolution"}, @@ -304,6 +305,7 @@ func generatedCatalogConnectRoutes() []*RouteEntry { {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/ListAvailableSolutions", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/ListAvailableSolutions"}, {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/ListOrganizationInstallations"}, {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/NarrowInstallation", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/NarrowInstallation"}, + {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/RetryInstallation", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/RetryInstallation"}, {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam"}, {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/TransferInstallationOwnership"}, {Service: "accounts_connect", Method: "POST", Path: "/saas.accounts.v1.InstallationService/UninstallSolution", UpstreamPath: "", Procedure: "/saas.accounts.v1.InstallationService/UninstallSolution"}, diff --git a/module/services/auth-gateway/code/routing_rest_catalog_gen.go b/module/services/auth-gateway/code/routing_rest_catalog_gen.go index 72d4fa20c..734a3759c 100644 --- a/module/services/auth-gateway/code/routing_rest_catalog_gen.go +++ b/module/services/auth-gateway/code/routing_rest_catalog_gen.go @@ -115,6 +115,7 @@ func generatedCatalogRESTRoutes() []*RouteEntry { {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}/teams", Procedure: "/saas.accounts.v1.InstallationService/GrantInstallationToTeam"}, {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}/teams/{team_id}:revoke", Procedure: "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam"}, {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}:narrow", Procedure: "/saas.accounts.v1.InstallationService/NarrowInstallation"}, + {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}:retry", Procedure: "/saas.accounts.v1.InstallationService/RetryInstallation"}, {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}:transferOwnership", Procedure: "/saas.accounts.v1.InstallationService/TransferInstallationOwnership"}, {Service: "accounts", Method: "POST", Path: "/v1/installations/{installation_id}:uninstall", Procedure: "/saas.accounts.v1.InstallationService/UninstallSolution"}, {Service: "accounts", Method: "POST", Path: "/v1/invitations", Procedure: "/saas.accounts.v1.InvitationService/CreateInvitation"}, diff --git a/module/services/auth-gateway/code/workload_identity.go b/module/services/auth-gateway/code/workload_identity.go new file mode 100644 index 000000000..fea1de5d0 --- /dev/null +++ b/module/services/auth-gateway/code/workload_identity.go @@ -0,0 +1,289 @@ +package main + +import ( + "context" + "crypto/tls" + "crypto/x509" + "encoding/asn1" + "fmt" + "net" + "net/http" + "os" + "regexp" + "strings" + + codefly "github.com/codefly-dev/sdk-go" +) + +const workloadSPIFFEHeader = "x-codefly-workload-spiffe-id" + +type workloadPeerKey struct{} + +// Only the workload listener supplies this context value. HTTP headers never +// attest a peer, including on the plain listener. +func workloadPeer(ctx context.Context) (string, bool) { + id, ok := ctx.Value(workloadPeerKey{}).(string) + return id, ok +} + +type workloadIdentity struct { + certificate *codefly.CertificateReloader + trustFile string + callersFile string +} + +// Resolve only the declared group: bare environment variables are not a second +// source of workload credentials. An entirely absent projection disables the +// workload endpoint; any supplied value requires the complete TLS identity. +func loadWorkloadIdentity(value func(string) (string, error)) (*workloadIdentity, error) { + paths := make(map[string]string) + keys := []string{"CERT_FILE", "KEY_FILE", "TRUST_BUNDLE_FILE", "ALLOWED_CALLERS_FILE"} + var missing error + for _, key := range keys { + path, err := value(key) + if err != nil || strings.TrimSpace(path) == "" { + if missing == nil { + missing = fmt.Errorf("workload-identity %s: missing path (%v)", key, err) + } + } + if strings.TrimSpace(path) != "" { + paths[key] = path + } + } + if len(paths) == 0 { + return nil, nil + } + if missing != nil { + return nil, missing + } + for _, key := range keys { + if _, err := os.ReadFile(paths[key]); err != nil { + return nil, fmt.Errorf("workload-identity %s: %w", key, err) + } + } + reloader, err := codefly.NewCertificateReloader(paths["CERT_FILE"], paths["KEY_FILE"]) + if err != nil { + return nil, fmt.Errorf("workload-identity CERT_FILE/KEY_FILE: %w", err) + } + w := &workloadIdentity{certificate: reloader, trustFile: paths["TRUST_BUNDLE_FILE"], callersFile: paths["ALLOWED_CALLERS_FILE"]} + if _, err := w.roots(); err != nil { + return nil, err + } + if _, err := w.allowedCallers(); err != nil { + return nil, err + } + if _, err := certificateSPIFFEID(reloader.Certificate().Leaf); err != nil { + return nil, fmt.Errorf("workload-identity CERT_FILE: %w", err) + } + return w, nil +} + +// Resolve and open the endpoint only when an identity is projected. Plain-only +// startup must not depend on workload endpoint discovery or bind its socket. +func (w *workloadIdentity) listen(listen func() (net.Listener, error)) (net.Listener, *tls.Config, error) { + if w == nil { + return nil, nil, nil + } + config, err := w.serverTLSConfig() + if err != nil { + return nil, nil, err + } + listener, err := listen() + if err != nil { + return nil, nil, err + } + return listener, config, nil +} + +var workloadSPIFFEPattern = regexp.MustCompile(`^spiffe://[a-z0-9._-]{1,255}(?:/[A-Za-z0-9._-]+)+$`) + +var subjectAltNameOID = asn1.ObjectIdentifier{2, 5, 29, 17} + +func certificateSPIFFEID(cert *x509.Certificate) (string, error) { + if cert == nil { + return "", fmt.Errorf("certificate required") + } + // cert.URIs has already passed through net/url, which lowercases schemes + // and drops empty fragments. Admission compares the original SAN bytes. + var id string + foundSAN := false + for _, extension := range cert.Extensions { + if !extension.Id.Equal(subjectAltNameOID) { + continue + } + if foundSAN { + return "", fmt.Errorf("duplicate subjectAltName extension") + } + foundSAN = true + var names []asn1.RawValue + rest, err := asn1.Unmarshal(extension.Value, &names) + if err != nil || len(rest) != 0 { + return "", fmt.Errorf("invalid subjectAltName extension") + } + for _, name := range names { + if name.Class != asn1.ClassContextSpecific { + return "", fmt.Errorf("invalid subjectAltName GeneralName") + } + if name.Tag != 6 { + continue + } + if id != "" || name.IsCompound || !workloadSPIFFEPattern.Match(name.Bytes) { + return "", fmt.Errorf("expected exactly one canonical SPIFFE URI SAN") + } + id = string(name.Bytes) + } + } + if id == "" { + return "", fmt.Errorf("expected exactly one canonical SPIFFE URI SAN") + } + return id, nil +} + +func (w *workloadIdentity) roots() (*x509.CertPool, error) { + pem, err := os.ReadFile(w.trustFile) + if err != nil { + return nil, fmt.Errorf("workload-identity TRUST_BUNDLE_FILE: %w", err) + } + roots := x509.NewCertPool() + if !roots.AppendCertsFromPEM(pem) { + return nil, fmt.Errorf("workload-identity TRUST_BUNDLE_FILE: no certificates") + } + return roots, nil +} + +// Read on every admission, including requests on an existing connection. A +// removed identity, missing file or malformed replacement revokes immediately. +// +// The file must be published atomically — written whole and swapped in, as the +// kubelet publishes a ConfigMap volume (a new ..data directory behind one +// symlink rename). No reader can tell a mid-write mix of an old and a new list +// from a complete one when both end in a newline, so an in-place rewrite is +// outside the contract, not a case this reader repairs. Only a regular file +// (reached directly or through that symlink) is read. +func (w *workloadIdentity) allowedCallers() (map[string]bool, error) { + info, err := os.Stat(w.callersFile) + if err != nil { + return nil, fmt.Errorf("workload-identity ALLOWED_CALLERS_FILE: %w", err) + } + if !info.Mode().IsRegular() { + return nil, fmt.Errorf("workload-identity ALLOWED_CALLERS_FILE: not a regular file") + } + body, err := os.ReadFile(w.callersFile) + if err != nil { + return nil, fmt.Errorf("workload-identity ALLOWED_CALLERS_FILE: %w", err) + } + // A truncated write that has not reached its final newline is refused whole, + // including any complete entries before it. + if len(body) > 0 && body[len(body)-1] != '\n' { + return nil, fmt.Errorf("workload-identity ALLOWED_CALLERS_FILE: missing final newline") + } + allowed := make(map[string]bool) + for _, line := range strings.Split(string(body), "\n") { + line, _, _ = strings.Cut(line, "#") + id := strings.Trim(line, " \t\r") + if id == "" { + continue + } + if !workloadSPIFFEPattern.MatchString(id) { + return nil, fmt.Errorf("workload-identity ALLOWED_CALLERS_FILE: invalid SPIFFE ID") + } + allowed[id] = true + } + return allowed, nil +} + +func (w *workloadIdentity) admit(state tls.ConnectionState) (string, error) { + if len(state.VerifiedChains) == 0 || len(state.PeerCertificates) == 0 { + return "", fmt.Errorf("verified client certificate required") + } + id, err := certificateSPIFFEID(state.PeerCertificates[0]) + if err != nil { + return "", err + } + allowed, err := w.allowedCallers() + if err != nil { + return "", err + } + if !allowed[id] { + return "", fmt.Errorf("workload caller is not allowed") + } + return id, nil +} + +func (w *workloadIdentity) serverTLSConfig() (*tls.Config, error) { + roots, err := w.roots() + if err != nil { + return nil, err + } + config := &tls.Config{ + MinVersion: tls.VersionTLS13, ClientAuth: tls.RequireAndVerifyClientCert, + ClientCAs: roots, GetCertificate: w.certificate.GetCertificate, + // Revalidate the chain on every handshake, including after revocation. + SessionTicketsDisabled: true, + VerifyConnection: func(state tls.ConnectionState) error { _, err := w.admit(state); return err }, + } + config.GetConfigForClient = func(*tls.ClientHelloInfo) (*tls.Config, error) { + current := config.Clone() + current.GetConfigForClient = nil + roots, err := w.roots() + current.ClientCAs = roots + return current, err + } + return config, nil +} + +func (w *workloadIdentity) handler(next http.Handler) http.Handler { + return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) { + if r.TLS == nil { + httpError(rw, http.StatusForbidden, "workload identity required") + return + } + id, err := w.admit(*r.TLS) + if err != nil { + httpError(rw, http.StatusForbidden, "workload identity refused") + return + } + next.ServeHTTP(rw, r.WithContext(context.WithValue(r.Context(), workloadPeerKey{}, id))) + }) +} + +func (w *workloadIdentity) upstreamTransport(expected string) (*http.Transport, error) { + if w == nil || !workloadSPIFFEPattern.MatchString(expected) { + return nil, fmt.Errorf("solution workload identity unavailable") + } + transport := newModuleUpstreamTransport(net.DefaultResolver) + // Each proxy exchange authenticates the current peer and projected leaf. + transport.DisableKeepAlives = true + transport.TLSClientConfig = &tls.Config{ + MinVersion: tls.VersionTLS13, GetClientCertificate: w.certificate.GetClientCertificate, + // SPIFFE authenticates a URI, not a DNS name. VerifyConnection performs + // the full chain/EKU verification below before comparing that URI. + InsecureSkipVerify: true, + VerifyConnection: func(state tls.ConnectionState) error { + if len(state.PeerCertificates) == 0 { + return fmt.Errorf("solution certificate required") + } + roots, err := w.roots() + if err != nil { + return err + } + intermediates := x509.NewCertPool() + for _, cert := range state.PeerCertificates[1:] { + intermediates.AddCert(cert) + } + leaf := state.PeerCertificates[0] + if _, err := leaf.Verify(x509.VerifyOptions{Roots: roots, Intermediates: intermediates, KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}); err != nil { + return err + } + id, err := certificateSPIFFEID(leaf) + if err != nil { + return err + } + if id != expected { + return fmt.Errorf("solution SPIFFE ID mismatch") + } + return nil + }, + } + return transport, nil +} diff --git a/module/services/auth-gateway/code/workload_identity_test.go b/module/services/auth-gateway/code/workload_identity_test.go new file mode 100644 index 000000000..e1fc3646f --- /dev/null +++ b/module/services/auth-gateway/code/workload_identity_test.go @@ -0,0 +1,467 @@ +package main + +import ( + "bufio" + "context" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/tls" + "crypto/x509" + "crypto/x509/pkix" + "encoding/asn1" + "encoding/pem" + "fmt" + "io" + "math/big" + "net" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/require" +) + +const gatewayTestSPIFFE = "spiffe://acme.test/ns/acme-prod/sa/gateway" +const callerTestSPIFFE = "spiffe://acme.test/ns/acme-prod/sa/worker" + +type workloadTestCA struct { + cert *x509.Certificate + key *ecdsa.PrivateKey + pem []byte +} + +func newWorkloadTestCA(t *testing.T) *workloadTestCA { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + require.NoError(t, err) + serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128)) + require.NoError(t, err) + tmpl := &x509.Certificate{SerialNumber: serial, Subject: pkix.Name{CommonName: "Example CA"}, NotBefore: time.Now().Add(-time.Minute), NotAfter: time.Now().Add(time.Hour), IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign} + der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key) + require.NoError(t, err) + cert, err := x509.ParseCertificate(der) + require.NoError(t, err) + return &workloadTestCA{cert: cert, key: key, pem: pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})} +} + +func (ca *workloadTestCA) leaf(t *testing.T, ids ...string) (tls.Certificate, []byte, []byte) { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + require.NoError(t, err) + serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128)) + require.NoError(t, err) + tmpl := &x509.Certificate{SerialNumber: serial, NotBefore: time.Now().Add(-time.Minute), NotAfter: time.Now().Add(time.Hour), KeyUsage: x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth}} + // Encode the actual SAN bytes: url.Parse would normalize adversarial IDs + // before the certificate under test was even issued. + if len(ids) != 0 { + var names []asn1.RawValue + for _, id := range ids { + names = append(names, asn1.RawValue{Class: asn1.ClassContextSpecific, Tag: 6, Bytes: []byte(id)}) + } + san, err := asn1.Marshal(names) + require.NoError(t, err) + tmpl.ExtraExtensions = []pkix.Extension{{Id: asn1.ObjectIdentifier{2, 5, 29, 17}, Value: san}} + } + der, err := x509.CreateCertificate(rand.Reader, tmpl, ca.cert, &key.PublicKey, ca.key) + require.NoError(t, err) + keyDER, err := x509.MarshalPKCS8PrivateKey(key) + require.NoError(t, err) + certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}) + keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyDER}) + cert, err := tls.X509KeyPair(certPEM, keyPEM) + require.NoError(t, err) + return cert, certPEM, keyPEM +} + +func workloadTestIdentity(t *testing.T, ca *workloadTestCA, id, allowed string) (*workloadIdentity, map[string]string) { + t.Helper() + _, cert, key := ca.leaf(t, id) + paths := map[string]string{} + dir := t.TempDir() + for name, body := range map[string][]byte{"CERT_FILE": cert, "KEY_FILE": key, "TRUST_BUNDLE_FILE": ca.pem, "ALLOWED_CALLERS_FILE": []byte(allowed)} { + paths[name] = filepath.Join(dir, name) + require.NoError(t, os.WriteFile(paths[name], body, 0600)) + } + identity, err := loadWorkloadIdentity(func(key string) (string, error) { return paths[key], nil }) + require.NoError(t, err) + require.NotNil(t, identity) + return identity, paths +} + +func TestWorkloadIdentityAbsentServesOnlyPlainEndpoints(t *testing.T) { + for _, test := range []struct { + name string + value func(string) (string, error) + }{ + {"empty values", func(string) (string, error) { return "", nil }}, + {"blank values", func(string) (string, error) { return " \t\n", nil }}, + {"unprojected group", func(key string) (string, error) { + return "", fmt.Errorf("no workspace configuration value for workload-identity/%s", key) + }}, + } { + t.Run(test.name, func(t *testing.T) { + identity, err := loadWorkloadIdentity(test.value) + require.NoError(t, err) + require.Nil(t, identity) + listener, config, err := identity.listen(func() (net.Listener, error) { + t.Fatal("absent identity must not resolve or open the workload endpoint") + return nil, nil + }) + require.NoError(t, err) + require.Nil(t, listener) + require.Nil(t, config) + + gateway := NewGateway(&ExtAuthz{}, NewRouteMatcher([]*RouteEntry{ + {Service: "self", Method: http.MethodGet, Path: "/health"}, + }, nil), nil, nil, newFakeSolutionRegistry(), nil) + gateway.workload = identity + recorder := httptest.NewRecorder() + gateway.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/health", nil)) + require.Equal(t, http.StatusOK, recorder.Code) + + // Even a declared solution with a valid SPIFFE identity cannot be + // reached when the gateway has no identity of its own. + registerSolutionHalves(t, gateway, "example", "https://solution") + recorder = httptest.NewRecorder() + gateway.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/solutions/example/assets/entry.js", nil)) + require.Equal(t, http.StatusBadGateway, recorder.Code) + require.Contains(t, recorder.Body.String(), "solution workload identity unavailable") + }) + } +} + +// net.Pipe preserves the temporary deadline errors TLS requires when HTTP +// cancels a background read between keep-alive requests. bufconn's deadline +// error is permanent, causing TLS to close after the first request. +type workloadTestListener struct { + connections chan net.Conn + done chan struct{} + once sync.Once +} + +func (l *workloadTestListener) Accept() (net.Conn, error) { + select { + case conn := <-l.connections: + return conn, nil + case <-l.done: + return nil, net.ErrClosed + } +} +func (l *workloadTestListener) Close() error { l.once.Do(func() { close(l.done) }); return nil } +func (l *workloadTestListener) Addr() net.Addr { return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1)} } +func (l *workloadTestListener) DialContext(ctx context.Context) (net.Conn, error) { + server, client := net.Pipe() + select { + case l.connections <- server: + return client, nil + case <-ctx.Done(): + _ = server.Close() + _ = client.Close() + return nil, ctx.Err() + case <-l.done: + _ = server.Close() + _ = client.Close() + return nil, net.ErrClosed + } +} + +// Real TLS and HTTP over an in-memory listener: these security checks also run +// where the sandbox forbids binding a loopback socket. +func workloadMemoryServer(t *testing.T, config *tls.Config, handler http.Handler) *workloadTestListener { + t.Helper() + listener := &workloadTestListener{connections: make(chan net.Conn), done: make(chan struct{})} + server := &http.Server{Handler: handler, TLSConfig: config} + go func() { _ = server.ServeTLS(listener, "", "") }() + t.Cleanup(func() { _ = server.Close(); _ = listener.Close() }) + return listener +} + +func workloadMemoryClient(t *testing.T, identity *workloadIdentity, listener *workloadTestListener, expected string) *http.Client { + t.Helper() + transport, err := identity.upstreamTransport(expected) + require.NoError(t, err) + transport.DialContext = func(ctx context.Context, _, _ string) (net.Conn, error) { return listener.DialContext(ctx) } + t.Cleanup(transport.CloseIdleConnections) + return &http.Client{Transport: transport, Timeout: 3 * time.Second} +} + +func TestWorkloadListenerAdmissionAndRevocation(t *testing.T) { + ca := newWorkloadTestCA(t) + gateway, paths := workloadTestIdentity(t, ca, gatewayTestSPIFFE, "# callers\n"+callerTestSPIFFE+"\n") + listener := &workloadTestListener{connections: make(chan net.Conn), done: make(chan struct{})} + opened, config, err := gateway.listen(func() (net.Listener, error) { return listener, nil }) + require.NoError(t, err) + require.Same(t, listener, opened) + server := &http.Server{TLSConfig: config, Handler: gateway.handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + id, ok := workloadPeer(r.Context()) + require.True(t, ok) + require.Equal(t, callerTestSPIFFE, id) + w.WriteHeader(http.StatusNoContent) + }))} + go func() { _ = server.ServeTLS(opened, "", "") }() + t.Cleanup(func() { _ = server.Close(); _ = listener.Close() }) + for _, test := range []struct { + name string + ca *workloadTestCA + ids []string + noCert bool + allowed bool + }{ + {name: "no certificate", noCert: true}, + {name: "untrusted chain", ca: newWorkloadTestCA(t), ids: []string{callerTestSPIFFE}}, + {name: "unlisted identity", ca: ca, ids: []string{"spiffe://acme.test/ns/acme-prod/sa/intruder"}}, + {name: "two URI SANs", ca: ca, ids: []string{callerTestSPIFFE, callerTestSPIFFE}}, + {name: "uppercase raw scheme", ca: ca, ids: []string{strings.Replace(callerTestSPIFFE, "spiffe://", "SPIFFE://", 1)}}, + {name: "empty raw fragment", ca: ca, ids: []string{callerTestSPIFFE + "#"}}, + {name: "non SPIFFE URI", ca: ca, ids: []string{"https://example.com/worker"}}, + {name: "listed identity", ca: ca, ids: []string{callerTestSPIFFE}, allowed: true}, + } { + t.Run(test.name, func(t *testing.T) { + client := workloadMemoryClient(t, gateway, listener, gatewayTestSPIFFE) + cfg := client.Transport.(*http.Transport).TLSClientConfig + cfg.GetClientCertificate = nil + if !test.noCert { + leaf, _, _ := test.ca.leaf(t, test.ids...) + cfg.Certificates = []tls.Certificate{leaf} + } + response, err := client.Get("https://workload/health") + if !test.allowed { + require.Error(t, err) + return + } + require.NoError(t, err) + _ = response.Body.Close() + require.Equal(t, http.StatusNoContent, response.StatusCode) + }) + } + // Keep this connection alive across the revocation; the middleware must + // re-read the allowlist even when there is no new TLS handshake. + caller, _ := workloadTestIdentity(t, ca, callerTestSPIFFE, gatewayTestSPIFFE+"\n") + client := workloadMemoryClient(t, caller, listener, gatewayTestSPIFFE) + wire, err := listener.DialContext(context.Background()) + require.NoError(t, err) + connection := tls.Client(wire, client.Transport.(*http.Transport).TLSClientConfig) + t.Cleanup(func() { _ = connection.Close() }) + require.NoError(t, connection.SetDeadline(time.Now().Add(3*time.Second))) + reader := bufio.NewReader(connection) + request, err := http.NewRequest(http.MethodGet, "https://workload/health", nil) + require.NoError(t, err) + require.NoError(t, request.Write(connection)) + response, err := http.ReadResponse(reader, request) + require.NoError(t, err) + _ = response.Body.Close() + require.NoError(t, os.WriteFile(paths["ALLOWED_CALLERS_FILE"], []byte("# revoked\n"), 0600)) + require.NoError(t, request.Write(connection)) + response, err = http.ReadResponse(reader, request) + require.NoError(t, err) + _ = response.Body.Close() + require.Equal(t, http.StatusForbidden, response.StatusCode) + _, err = client.Get("https://workload/health") + require.Error(t, err) +} + +func TestWorkloadSolutionTLSIdentity(t *testing.T) { + ca := newWorkloadTestCA(t) + gateway, _ := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n") + solution, _ := workloadTestIdentity(t, ca, callerTestSPIFFE, gatewayTestSPIFFE+"\n") + config, err := solution.serverTLSConfig() + require.NoError(t, err) + listener := workloadMemoryServer(t, config, solution.handler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + id, ok := workloadPeer(r.Context()) + require.True(t, ok) + require.Equal(t, gatewayTestSPIFFE, id) + _, _ = io.WriteString(w, "solution") + }))) + for _, expected := range []string{callerTestSPIFFE, "spiffe://acme.test/ns/acme-prod/sa/different"} { + client := workloadMemoryClient(t, gateway, listener, expected) + response, err := client.Get("https://solution/assets/entry.js") + if expected != callerTestSPIFFE { + require.ErrorContains(t, err, "SPIFFE ID mismatch") + continue + } + require.NoError(t, err) + body, err := io.ReadAll(response.Body) + require.NoError(t, err) + _ = response.Body.Close() + require.Equal(t, "solution", string(body)) + } +} + +func TestWorkloadSolutionRejectsNoncanonicalRawSAN(t *testing.T) { + ca := newWorkloadTestCA(t) + gateway, _ := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n") + for _, id := range []string{ + strings.Replace(callerTestSPIFFE, "spiffe://", "SPIFFE://", 1), + callerTestSPIFFE + "#", + } { + t.Run(id, func(t *testing.T) { + leaf, _, _ := ca.leaf(t, id) + listener := workloadMemoryServer(t, &tls.Config{MinVersion: tls.VersionTLS13, Certificates: []tls.Certificate{leaf}}, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusNoContent) + })) + client := workloadMemoryClient(t, gateway, listener, callerTestSPIFFE) + response, err := client.Get("https://solution/assets/entry.js") + if response != nil { + _ = response.Body.Close() + } + require.Error(t, err) + }) + } +} + +func TestWorkloadCertificateSPIFFEIDUsesCanonicalRawSAN(t *testing.T) { + ca := newWorkloadTestCA(t) + for _, test := range []struct { + name string + ids []string + allowed bool + }{ + {"Kubernetes identity", []string{callerTestSPIFFE}, true}, + {"generic path", []string{"spiffe://acme.test/Example_1/worker.v2-test"}, true}, + {"no URI", nil, false}, + {"duplicate URI", []string{callerTestSPIFFE, callerTestSPIFFE}, false}, + {"additional other URI", []string{callerTestSPIFFE, "https://example.com"}, false}, + {"uppercase scheme", []string{"SPIFFE://acme.test/ns/acme-prod/sa/worker"}, false}, + {"uppercase trust domain", []string{"spiffe://Acme.test/ns/acme-prod/sa/worker"}, false}, + {"empty path", []string{"spiffe://acme.test"}, false}, + {"empty path segment", []string{"spiffe://acme.test/ns//worker"}, false}, + {"trailing slash", []string{callerTestSPIFFE + "/"}, false}, + {"query", []string{callerTestSPIFFE + "?key=value"}, false}, + {"empty query", []string{callerTestSPIFFE + "?"}, false}, + {"fragment", []string{callerTestSPIFFE + "#fragment"}, false}, + {"empty fragment", []string{callerTestSPIFFE + "#"}, false}, + {"percent encoding", []string{"spiffe://acme.test/ns/acme-prod/sa/%77orker"}, false}, + } { + t.Run(test.name, func(t *testing.T) { + leaf, _, _ := ca.leaf(t, test.ids...) + id, err := certificateSPIFFEID(leaf.Leaf) + if test.allowed { + require.NoError(t, err) + require.Equal(t, test.ids[0], id) + } else { + require.Error(t, err) + require.Empty(t, id) + } + }) + } +} + +func TestWorkloadAllowedCallersRequiresCompleteSnapshot(t *testing.T) { + ca := newWorkloadTestCA(t) + gateway, paths := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n") + leaf, _, _ := ca.leaf(t, callerTestSPIFFE) + state := tls.ConnectionState{PeerCertificates: []*x509.Certificate{leaf.Leaf}, VerifiedChains: [][]*x509.Certificate{{leaf.Leaf, ca.cert}}} + for _, test := range []struct { + name, body string + allowed bool + }{ + {"complete", callerTestSPIFFE + "\n", true}, + {"partial identity", callerTestSPIFFE, false}, + {"complete entry then partial identity", callerTestSPIFFE + "\nspiffe://acme.test/ns/acme-prod/sa/oth", false}, + {"complete replacement", callerTestSPIFFE + "-admin\n", false}, + {"empty", "", false}, + {"malformed", callerTestSPIFFE + "\ninvalid\n", false}, + {"ASCII whitespace and comments", " # callers\n\t " + callerTestSPIFFE + " \t\r# worker\r\n", true}, + {"Unicode whitespace", "\u00a0" + callerTestSPIFFE + "\n", false}, + {"restored", callerTestSPIFFE + "\n", true}, + } { + t.Run(test.name, func(t *testing.T) { + require.NoError(t, os.WriteFile(paths["ALLOWED_CALLERS_FILE"], []byte(test.body), 0600)) + id, err := gateway.admit(state) + if test.allowed { + require.NoError(t, err) + require.Equal(t, callerTestSPIFFE, id) + } else { + require.Error(t, err) + require.Empty(t, id) + } + }) + } +} + +func TestWorkloadFilesRequiredAndRotation(t *testing.T) { + ca := newWorkloadTestCA(t) + identity, paths := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n") + for key := range paths { + t.Run(key, func(t *testing.T) { + _, err := loadWorkloadIdentity(func(k string) (string, error) { + if k == key { + return "", nil + } + return paths[k], nil + }) + require.ErrorContains(t, err, key) + _, err = loadWorkloadIdentity(func(k string) (string, error) { + if k == key { + return "", fmt.Errorf("no workspace configuration value") + } + return paths[k], nil + }) + require.ErrorContains(t, err, key) + _, err = loadWorkloadIdentity(func(k string) (string, error) { + if k == key { + return filepath.Join(t.TempDir(), "absent"), nil + } + return paths[k], nil + }) + require.ErrorContains(t, err, key) + _, err = loadWorkloadIdentity(func(k string) (string, error) { + if k == key { + return t.TempDir(), nil // A directory cannot be read as a file. + } + return paths[k], nil + }) + require.ErrorContains(t, err, key) + }) + } + first := identity.certificate.Certificate() + replacement, cert, key := ca.leaf(t, gatewayTestSPIFFE) + require.NoError(t, os.WriteFile(paths["CERT_FILE"], cert, 0600)) + future := time.Now().Add(time.Minute) + require.NoError(t, os.Chtimes(paths["CERT_FILE"], future, future)) + require.Equal(t, first, identity.certificate.Certificate(), "half-written rotation keeps the last good pair") + require.NoError(t, os.WriteFile(paths["KEY_FILE"], key, 0600)) + require.NoError(t, os.Chtimes(paths["KEY_FILE"], future, future)) + served, err := identity.certificate.GetCertificate(nil) + require.NoError(t, err) + presented, err := identity.certificate.GetClientCertificate(nil) + require.NoError(t, err) + require.Equal(t, replacement.Certificate, served.Certificate) + require.Equal(t, served, presented) +} + +// Existing solution routing tests use the same mTLS contract as production. +func newSolutionTestServer(t *testing.T, gateway *Gateway, id string, handler http.Handler) *httptest.Server { + t.Helper() + ca := newWorkloadTestCA(t) + if gateway.workload == nil { + gateway.workload, _ = workloadTestIdentity(t, ca, gatewayTestSPIFFE, "") + } + bundle, err := os.ReadFile(gateway.workload.trustFile) + require.NoError(t, err) + require.NoError(t, os.WriteFile(gateway.workload.trustFile, append(bundle, ca.pem...), 0600)) + solution, _ := workloadTestIdentity(t, ca, "spiffe://acme.test/ns/acme-prod/sa/"+id, gatewayTestSPIFFE+"\n") + solution.trustFile = gateway.workload.trustFile + config, err := solution.serverTLSConfig() + require.NoError(t, err) + server := httptest.NewUnstartedServer(solution.handler(handler)) + server.TLS = config + server.TLS.Certificates = []tls.Certificate{*solution.certificate.Certificate()} + server.StartTLS() + t.Cleanup(server.Close) + return server +} + +func TestWorkloadProxyRequiresDeclaredIdentity(t *testing.T) { + gateway := &Gateway{} + recorder := httptest.NewRecorder() + gateway.proxyTo(recorder, httptest.NewRequest(http.MethodGet, "/assets/entry.js", nil), MustURL("https://solution"), &RouteEntry{Service: "solution:example"}) + require.Equal(t, http.StatusBadGateway, recorder.Code) + require.NotContains(t, strings.ToLower(recorder.Body.String()), "credential") +} diff --git a/module/services/auth-gateway/code/workload_mint_test.go b/module/services/auth-gateway/code/workload_mint_test.go new file mode 100644 index 000000000..b60e419e2 --- /dev/null +++ b/module/services/auth-gateway/code/workload_mint_test.go @@ -0,0 +1,181 @@ +package main + +import ( + "context" + "net" + "net/http" + "net/http/httptest" + "sync" + "time" + + accountsv1 "auth-gateway/pkg/gen/saas/accounts/v1" + "strings" + "testing" + + "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/credentials/insecure" + "google.golang.org/grpc/metadata" + "google.golang.org/grpc/status" + "google.golang.org/grpc/test/bufconn" + "google.golang.org/protobuf/proto" + "google.golang.org/protobuf/types/known/timestamppb" +) + +func TestWorkloadMintForwardsOnlyVerifiedPeerAndMapsRefusal(t *testing.T) { + mint := &fakeAccountsWorkContextMint{} + backend := grpc.NewServer() + backend.RegisterService(&grpc.ServiceDesc{ServiceName: "saas.accounts.v1.ModuleCapabilitiesService", HandlerType: (*any)(nil), Methods: []grpc.MethodDesc{{MethodName: "MintModuleWorkContext", Handler: func(_ any, ctx context.Context, dec func(any) error, _ grpc.UnaryServerInterceptor) (any, error) { + return mint.handle(ctx, dec) + }}}}, mint) + listener := bufconn.Listen(1 << 20) + go func() { _ = backend.Serve(listener) }() + t.Cleanup(backend.Stop) + conn, err := grpc.NewClient("passthrough:///accounts", grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) { return listener.DialContext(ctx) })) + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + gateway := NewGateway(&ExtAuthz{internalToken: "internal-perimeter", backendConn: conn}, NewRouteMatcher(nil, nil), nil, nil, newFakeSolutionRegistry(), nil) + ca := newWorkloadTestCA(t) + identity, _ := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n") + config, err := identity.serverTLSConfig() + require.NoError(t, err) + httpListener := workloadMemoryServer(t, config, identity.handler(gateway)) + caller, _ := workloadTestIdentity(t, ca, callerTestSPIFFE, gatewayTestSPIFFE+"\n") + client := workloadMemoryClient(t, caller, httpListener, gatewayTestSPIFFE) + for _, code := range []codes.Code{codes.OK, codes.PermissionDenied} { + mint.code = code + request, err := http.NewRequest(http.MethodPost, "https://workload/modules/_work-context", strings.NewReader(`{"prefix":"example"}`)) + require.NoError(t, err) + request.Header.Set("Authorization", "Bearer projected-token") + request.Header.Set(workloadSPIFFEHeader, "spiffe://acme.test/ns/acme-prod/sa/spoofed") + // No shared internal credential: this listener authenticated the caller. + response, err := client.Do(request) + require.NoError(t, err) + _ = response.Body.Close() + if code == codes.OK { + require.Equal(t, http.StatusOK, response.StatusCode) + } else { + require.Equal(t, http.StatusForbidden, response.StatusCode) + } + require.Equal(t, callerTestSPIFFE, mint.lastPeer) + require.Equal(t, "projected-token", mint.lastToken) + require.Equal(t, "internal-perimeter", mint.lastInternal) + } +} + +// Exercise each real HTTP route over TLS and the internal gRPC transport. The +// backend here checks the attestation; the accounts adapter tests exercise the +// real TokenReview binding and signer separately. +func TestWorkloadEveryMintForwardsAttestation(t *testing.T) { + for _, mint := range []struct { + path, method, body string + request proto.Message + response proto.Message + }{ + { + path: moduleWorkContextPath, method: "MintModuleWorkContext", body: `{"prefix":"example"}`, + request: &accountsv1.ModuleMintWorkContextRequest{}, + response: &accountsv1.ModuleMintWorkContextResponse{Token: "signed", ExpiresAt: timestamppb.New(time.Now().Add(time.Minute))}, + }, + { + path: moduleOperationContextPath, method: "MintModuleOperationContext", body: `{"prefix":"example","binding":"sync"}`, + request: &accountsv1.ModuleMintOperationContextRequest{}, + response: &accountsv1.ModuleMintOperationContextResponse{Token: "signed", ExpiresAt: timestamppb.New(time.Now().Add(time.Minute))}, + }, + { + path: moduleSourceOperationContextPath, method: "MintSourceOperationContext", body: `{"prefix":"example","source_id":"` + testSourceID + `"}`, + request: &accountsv1.ModuleMintSourceOperationContextRequest{}, + response: &accountsv1.ModuleMintSourceOperationContextResponse{Token: "signed", ExpiresAt: timestamppb.New(time.Now().Add(time.Minute))}, + }, + } { + t.Run(mint.method, func(t *testing.T) { + var mu sync.Mutex + var peers []string + calls := 0 + backend := grpc.NewServer() + backend.RegisterService(&grpc.ServiceDesc{ServiceName: "saas.accounts.v1.ModuleCapabilitiesService", HandlerType: (*any)(nil), Methods: []grpc.MethodDesc{{MethodName: mint.method, Handler: func(_ any, ctx context.Context, dec func(any) error, _ grpc.UnaryServerInterceptor) (any, error) { + req := proto.Clone(mint.request) + if err := dec(req); err != nil { + return nil, err + } + md, _ := metadata.FromIncomingContext(ctx) + if token := req.(interface{ GetServiceAccountToken() string }).GetServiceAccountToken(); token != "projected-token" { + return nil, status.Error(codes.Unauthenticated, "projected token missing") + } + if tokens := md.Get("x-codefly-internal-token"); len(tokens) != 1 || tokens[0] != "internal-perimeter" { + return nil, status.Error(codes.Unauthenticated, "internal credential missing") + } + mu.Lock() + peers = append([]string(nil), md.Get(workloadSPIFFEHeader)...) + calls++ + mu.Unlock() + if len(md.Get(workloadSPIFFEHeader)) != 0 && md.Get(workloadSPIFFEHeader)[0] != callerTestSPIFFE { + return nil, status.Error(codes.PermissionDenied, "workload mismatch") + } + return proto.Clone(mint.response), nil + }}}}, nil) + listener := bufconn.Listen(1 << 20) + go func() { _ = backend.Serve(listener) }() + t.Cleanup(backend.Stop) + conn, err := grpc.NewClient("passthrough:///accounts", grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) { return listener.DialContext(ctx) })) + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + gateway := NewGateway(&ExtAuthz{internalToken: "internal-perimeter", backendConn: conn}, NewRouteMatcher(nil, nil), nil, nil, newFakeSolutionRegistry(), nil) + ca := newWorkloadTestCA(t) + const otherPeer = "spiffe://acme.test/ns/acme-prod/sa/other" + identity, _ := workloadTestIdentity(t, ca, gatewayTestSPIFFE, callerTestSPIFFE+"\n"+otherPeer+"\n") + config, err := identity.serverTLSConfig() + require.NoError(t, err) + handler := identity.handler(gateway) + httpListener := workloadMemoryServer(t, config, handler) + request := func() *http.Request { + req := httptest.NewRequest(http.MethodPost, "https://workload"+mint.path, strings.NewReader(mint.body)) + req.RequestURI = "" + req.Header.Set("Authorization", "Bearer projected-token") + req.Header.Set("X-Codefly-Internal-Token", "internal-perimeter") + req.Header.Set(workloadSPIFFEHeader, callerTestSPIFFE) + req.Header.Set("Grpc-Metadata-"+workloadSPIFFEHeader, callerTestSPIFFE) + return req + } + for _, peer := range []string{callerTestSPIFFE, otherPeer} { + caller, _ := workloadTestIdentity(t, ca, peer, gatewayTestSPIFFE+"\n") + client := workloadMemoryClient(t, caller, httpListener, gatewayTestSPIFFE) + response, err := client.Do(request()) + require.NoError(t, err) + _ = response.Body.Close() + want := http.StatusOK + if peer != callerTestSPIFFE { + want = http.StatusForbidden + } + require.Equal(t, want, response.StatusCode) + mu.Lock() + got := append([]string(nil), peers...) + mu.Unlock() + require.Equal(t, []string{peer}, got) + } + // Neither direct nor grpc-gateway-style HTTP headers attest a plain call. + // Even outgoing metadata already on the context must be replaced. + plain := request() + plain.TLS = nil + plain = plain.WithContext(metadata.NewOutgoingContext(plain.Context(), metadata.Pairs(workloadSPIFFEHeader, callerTestSPIFFE))) + rec := httptest.NewRecorder() + gateway.ServeHTTP(rec, plain) + require.Equal(t, http.StatusOK, rec.Code) + mu.Lock() + got := append([]string(nil), peers...) + before := calls + mu.Unlock() + require.Empty(t, got) + // The workload handler cannot fall back to the plain mint when TLS peer + // verification is absent, even with both bearer and internal credentials. + rec = httptest.NewRecorder() + handler.ServeHTTP(rec, plain) + require.Equal(t, http.StatusForbidden, rec.Code) + mu.Lock() + after := calls + mu.Unlock() + require.Equal(t, before, after) + }) + } +} diff --git a/module/services/auth-gateway/proto/saas/accounts/v1/module_registration.proto b/module/services/auth-gateway/proto/saas/accounts/v1/module_registration.proto index ceb5164cd..395aac1c3 100644 --- a/module/services/auth-gateway/proto/saas/accounts/v1/module_registration.proto +++ b/module/services/auth-gateway/proto/saas/accounts/v1/module_registration.proto @@ -60,6 +60,28 @@ message ModuleMintWorkContextRequest { (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, (buf.validate.field).string.uuid = true ]; + // upstream is where the minting workload serves its backend, as an absolute + // https origin with no path: the dynamic half of its registration, reported + // by the runtime that proved its identity with the token above (#952). The + // gateway reaches it only under the SVID the applied presence names for the + // workload, so a claimed origin that is not that workload serves nothing. + // Empty leaves the registration's backend half as it is. + string upstream = 5 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string = { + max_len: 512 + uri: true + } + ]; + // manifest is the solution's frontend manifest as its runtime serves it: the + // other dynamic half of a solution's registration (a module has none), the + // document the host's frontend renders the solution's page from. Reported + // with the same proven identity as upstream. Empty leaves the frontend half + // as it is. + string manifest = 6 [ + (buf.validate.field).ignore = IGNORE_IF_ZERO_VALUE, + (buf.validate.field).string.max_len = 65536 + ]; } // ModuleMintWorkContextResponse carries the signed capability plus the identity diff --git a/module/services/auth-gateway/service.codefly.yaml b/module/services/auth-gateway/service.codefly.yaml index 2e1a5246f..406625239 100644 --- a/module/services/auth-gateway/service.codefly.yaml +++ b/module/services/auth-gateway/service.codefly.yaml @@ -1,11 +1,12 @@ name: auth-gateway version: 0.0.0 description: | - Single ingress + auth brain. Hosts two listeners: + Single ingress + auth brain. Hosts three listeners: - gRPC ext_authz Check. Its answer carries identity but never the gateway credential, so an Envoy in front of accounts must stamp that credential itself. - private HTTP API gateway behind the frontend entry + - workload API gateway with TLS 1.3 and verified workload identities agent: kind: codefly:service name: go-grpc @@ -35,11 +36,15 @@ workspace-configuration-dependencies: - internal-auth - observability - security + - workload-identity endpoints: - name: grpc visibility: internal - name: rest visibility: internal + - name: workload + api: http + visibility: internal spec: hot-reload: true protocol-output-dirs: diff --git a/module/services/frontend/code/package-lock.json b/module/services/frontend/code/package-lock.json index d000dd953..981d650f2 100644 --- a/module/services/frontend/code/package-lock.json +++ b/module/services/frontend/code/package-lock.json @@ -15,7 +15,7 @@ "@bufbuild/protobuf": "^2.11.0", "@codefly-dev/saas-plugin-contract": "2.4.3", "@codefly-dev/saas-plugin-react": "0.4.4", - "@codefly-dev/saas-sdk": "0.3.28", + "@codefly-dev/saas-sdk": "0.3.29", "@codefly-dev/saas-ui": "0.12.8", "@codefly-dev/ui": "0.12.8", "@codefly/saas-plugin-manifest": "0.1.2", @@ -15474,7 +15474,7 @@ }, "packages/saas-sdk": { "name": "@codefly-dev/saas-sdk", - "version": "0.3.28", + "version": "0.3.29", "license": "Apache-2.0", "dependencies": { "@bufbuild/protobuf": "^2.11.0", diff --git a/module/services/frontend/code/package.json b/module/services/frontend/code/package.json index d3367b17b..0653b26f4 100644 --- a/module/services/frontend/code/package.json +++ b/module/services/frontend/code/package.json @@ -38,7 +38,7 @@ "@bufbuild/protobuf": "^2.11.0", "@codefly-dev/saas-plugin-contract": "2.4.3", "@codefly-dev/saas-plugin-react": "0.4.4", - "@codefly-dev/saas-sdk": "0.3.28", + "@codefly-dev/saas-sdk": "0.3.29", "@codefly-dev/saas-ui": "0.12.8", "@codefly-dev/ui": "0.12.8", "@codefly/saas-plugin-manifest": "0.1.2", diff --git a/module/services/frontend/code/packages/saas-sdk/generated/contract/catalog.codefly.json b/module/services/frontend/code/packages/saas-sdk/generated/contract/catalog.codefly.json index 303256d83..ffa26be3f 100644 --- a/module/services/frontend/code/packages/saas-sdk/generated/contract/catalog.codefly.json +++ b/module/services/frontend/code/packages/saas-sdk/generated/contract/catalog.codefly.json @@ -10,7 +10,7 @@ "kind": "protobuf", "package": "saas.accounts.v1", "path": "contract.binpb", - "digest": "sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047", + "digest": "sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82", "services": [ { "name": "APIKeyService", @@ -188,6 +188,7 @@ "/saas.accounts.v1.InstallationService/ListInstallations", "/saas.accounts.v1.InstallationService/ListOrganizationInstallations", "/saas.accounts.v1.InstallationService/NarrowInstallation", + "/saas.accounts.v1.InstallationService/RetryInstallation", "/saas.accounts.v1.InstallationService/RevokeInstallationFromTeam", "/saas.accounts.v1.InstallationService/TransferInstallationOwnership", "/saas.accounts.v1.InstallationService/UninstallSolution" diff --git a/module/services/frontend/code/packages/saas-sdk/generated/contract/contract.binpb b/module/services/frontend/code/packages/saas-sdk/generated/contract/contract.binpb index 5e7e7ab90..e0acaf84a 100644 Binary files a/module/services/frontend/code/packages/saas-sdk/generated/contract/contract.binpb and b/module/services/frontend/code/packages/saas-sdk/generated/contract/contract.binpb differ diff --git a/module/services/frontend/code/packages/saas-sdk/generated/library.codefly.yaml b/module/services/frontend/code/packages/saas-sdk/generated/library.codefly.yaml index 279079015..f4c5e0e66 100644 --- a/module/services/frontend/code/packages/saas-sdk/generated/library.codefly.yaml +++ b/module/services/frontend/code/packages/saas-sdk/generated/library.codefly.yaml @@ -12,7 +12,7 @@ sources: module: saas-starter service: accounts endpoint: connect - contract-digest: sha256:3ba50fe62c505c2f6a2a16b8604f0abf151982dc5bb4d46bda5fa9fa72878047 + contract-digest: sha256:98b3d725895d0ddfcdcf6fba1a079b11a50969c78ac9e159871e2f1e35474b82 services: - APIKeyService - AccessibleScopeService diff --git a/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/installations_pb.ts b/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/installations_pb.ts index 340af05d7..a0f38962d 100644 --- a/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/installations_pb.ts +++ b/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/installations_pb.ts @@ -17,7 +17,7 @@ import type { Message } from "@bufbuild/protobuf"; * Describes the file saas/accounts/v1/installations.proto. */ export const file_saas_accounts_v1_installations: GenFile = /*@__PURE__*/ - fileDesc("CiRzYWFzL2FjY291bnRzL3YxL2luc3RhbGxhdGlvbnMucHJvdG8SEHNhYXMuYWNjb3VudHMudjEi1wMKDEluc3RhbGxhdGlvbhIUCgJpZBgBIAEoCUIIukgFcgOwAQESGAoGb3JnX2lkGAIgASgJQgi6SAVyA7ABARIkChJhZ2VudF9wcmluY2lwYWxfaWQYAyABKAlCCLpIBXIDsAEBEiQKEm93bmVyX3ByaW5jaXBhbF9pZBgFIAEoCUIIukgFcgOwAQESHgoWY29fb3duZXJfcHJpbmNpcGFsX2lkcxgGIAMoCRIkChJyb290X3Njb3BlX25vZGVfaWQYByABKAlCCLpIBXIDsAEBEjQKBnN0YXR1cxgIIAEoDjIkLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uU3RhdHVzEi4KCmNyZWF0ZWRfYXQYCSABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wEjMKCnJldm9rZWRfYXQYCiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wSACIAQESHgoJdGFyZ2V0X2lkGAsgASgJQgu6SAjYAQFyA7ABARIgCg5yZXZva2VkX3JlYXNvbhgMIAEoCUIIukgFcgMYgARCDQoLX3Jldm9rZWRfYXRKBAgEEAVSE3NvbHV0aW9uX2lkZW50aWZpZXIi4AMKFkluc3RhbGxTb2x1dGlvblJlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIkChBhZ2VudF9pZGVudGlmaWVyGAIgASgJQgq6SAdyBRABGIAEEh4KDGRpc3BsYXlfbmFtZRgEIAEoCUIIukgFcgMYgAQSIgoQcm9vdF9zY29wZV9sYWJlbBgGIAEoCUIIukgFcgMYgAQSGQoHcm9sZV9pZBgHIAEoCUIIukgFcgOwAQESJwoSb3duZXJfcHJpbmNpcGFsX2lkGAggASgJQgu6SAjYAQFyA7ABARIvChZjb19vd25lcl9wcmluY2lwYWxfaWRzGAkgAygJQg+6SAySAQkQICIFcgOwAQESKgoRYWxsb3dlZF9hdWRpZW5jZXMYCiADKAlCD7pIDJIBCRBAIgVyAxiABBInCg5hbGxvd2VkX3Njb3BlcxgLIAMoCUIPukgMkgEJEEAiBXIDGIABEhsKCXRhcmdldF9pZBgMIAEoCUIIukgFcgOwAQESHwoFc2NvcGUYDSABKAlCC7pICHIGEAIogIBASACIAQFCCAoGX3Njb3BlSgQIAxAESgQIBRAGUhNzb2x1dGlvbl9pZGVudGlmaWVyUg9yb290X3Njb3BlX3BhdGgidAoZTmFycm93SW5zdGFsbGF0aW9uUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQESGgoFc2NvcGUYAyABKAlCC7pICHIGEAIogIBAIlcKGFVuaW5zdGFsbFNvbHV0aW9uUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQEivgEKJFRyYW5zZmVySW5zdGFsbGF0aW9uT3duZXJzaGlwUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQESKAoWbmV3X293bmVyX3ByaW5jaXBhbF9pZBgDIAEoCUIIukgFcgOwAQESLwoWY29fb3duZXJfcHJpbmNpcGFsX2lkcxgEIAMoCUIPukgMkgEJECAiBXIDsAEBIlUKFkdldEluc3RhbGxhdGlvblJlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIhCg9pbnN0YWxsYXRpb25faWQYAiABKAlCCLpIBXIDsAEBIoUBChdHZXRJbnN0YWxsYXRpb25SZXNwb25zZRI0CgxpbnN0YWxsYXRpb24YASABKAsyHi5zYWFzLmFjY291bnRzLnYxLkluc3RhbGxhdGlvbhI0CgZoZWFsdGgYAiABKA4yJC5zYWFzLmFjY291bnRzLnYxLkluc3RhbGxhdGlvbkhlYWx0aCKdAQoYTGlzdEluc3RhbGxhdGlvbnNSZXF1ZXN0EhgKBm9yZ19pZBgBIAEoCUIIukgFcgOwAQESNAoGc3RhdHVzGAIgASgOMiQuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25TdGF0dXMSHQoJcGFnZV9zaXplGAMgASgFQgq6SAcaBRj0AygAEhIKCnBhZ2VfdG9rZW4YBCABKAkigQEKE0luc3RhbGxhdGlvblN1bW1hcnkSNAoMaW5zdGFsbGF0aW9uGAEgASgLMh4uc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb24SNAoGaGVhbHRoGAIgASgOMiQuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25IZWFsdGgicgoZTGlzdEluc3RhbGxhdGlvbnNSZXNwb25zZRI8Cg1pbnN0YWxsYXRpb25zGAEgAygLMiUuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25TdW1tYXJ5EhcKD25leHRfcGFnZV90b2tlbhgCIAEoCSJzCiRMaXN0T3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uc1JlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIdCglwYWdlX3NpemUYAiABKAVCCrpIBxoFGPQDKAASEgoKcGFnZV90b2tlbhgDIAEoCSLSAgoYT3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uEjQKDGluc3RhbGxhdGlvbhgBIAEoCzIeLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uEjQKBmhlYWx0aBgCIAEoDjIkLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uSGVhbHRoEjMKBnRhcmdldBgDIAEoCzIjLnNhYXMuYWNjb3VudHMudjEuQXZhaWxhYmxlU29sdXRpb24SFwoPcm9vdF9zY29wZV9wYXRoGAQgASgJEjwKDWdyYW50ZWRfdGVhbXMYBSADKAsyJS5zYWFzLmFjY291bnRzLnYxLkNvbGxlY3Rpb25SZWFkR3JhbnQSPgoPaW5oZXJpdGVkX3RlYW1zGAYgAygLMiUuc2Fhcy5hY2NvdW50cy52MS5Db2xsZWN0aW9uUmVhZEdyYW50IoMBCiVMaXN0T3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uc1Jlc3BvbnNlEkEKDWluc3RhbGxhdGlvbnMYASADKAsyKi5zYWFzLmFjY291bnRzLnYxLk9yZ2FuaXphdGlvbkluc3RhbGxhdGlvbhIXCg9uZXh0X3BhZ2VfdG9rZW4YAiABKAkikwEKHkdyYW50SW5zdGFsbGF0aW9uVG9UZWFtUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQESGQoHdGVhbV9pZBgDIAEoCUIIukgFcgOwAQESGQoHcm9sZV9pZBgEIAEoCUIIukgFcgOwAQEiTgofR3JhbnRJbnN0YWxsYXRpb25Ub1RlYW1SZXNwb25zZRIrCgVncmFudBgBIAEoCzIcLnNhYXMuYWNjb3VudHMudjEuU2NvcGVHcmFudCKWAQohUmV2b2tlSW5zdGFsbGF0aW9uRnJvbVRlYW1SZXF1ZXN0EhgKBm9yZ19pZBgBIAEoCUIIukgFcgOwAQESIQoPaW5zdGFsbGF0aW9uX2lkGAIgASgJQgi6SAVyA7ABARIZCgd0ZWFtX2lkGAMgASgJQgi6SAVyA7ABARIZCgdyb2xlX2lkGAQgASgJQgi6SAVyA7ABASLtAQoRQXZhaWxhYmxlU29sdXRpb24SGwoJdGFyZ2V0X2lkGAEgASgJQgi6SAVyA7ABARISCgpiaW5kaW5nX2lkGAIgASgJEhMKC3JvdXRlX2FsaWFzGAMgASgJEhkKEXJlbGVhc2VfcHVibGlzaGVyGAQgASgJEhQKDHJlbGVhc2VfbmFtZRgFIAEoCRIXCg9yZWxlYXNlX3ZlcnNpb24YBiABKAkSGQoRb3BlbmVkX2dlbmVyYXRpb24YByABKAQSGgoSYXBwbGllZF9nZW5lcmF0aW9uGAggASgEEhEKCWluc3RhbGxlZBgJIAEoCCJsCh1MaXN0QXZhaWxhYmxlU29sdXRpb25zUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEh0KCXBhZ2Vfc2l6ZRgCIAEoBUIKukgHGgUY9AMoABISCgpwYWdlX3Rva2VuGAMgASgJInEKHkxpc3RBdmFpbGFibGVTb2x1dGlvbnNSZXNwb25zZRI2Cglzb2x1dGlvbnMYASADKAsyIy5zYWFzLmFjY291bnRzLnYxLkF2YWlsYWJsZVNvbHV0aW9uEhcKD25leHRfcGFnZV90b2tlbhgCIAEoCSp6ChJJbnN0YWxsYXRpb25TdGF0dXMSIwofSU5TVEFMTEFUSU9OX1NUQVRVU19VTlNQRUNJRklFRBAAEh4KGklOU1RBTExBVElPTl9TVEFUVVNfQUNUSVZFEAESHwobSU5TVEFMTEFUSU9OX1NUQVRVU19SRVZPS0VEEAIqhAIKEkluc3RhbGxhdGlvbkhlYWx0aBIjCh9JTlNUQUxMQVRJT05fSEVBTFRIX1VOU1BFQ0lGSUVEEAASHwobSU5TVEFMTEFUSU9OX0hFQUxUSF9IRUFMVEhZEAESKQolSU5TVEFMTEFUSU9OX0hFQUxUSF9OT19FTElHSUJMRV9PV05FUhACEiUKIUlOU1RBTExBVElPTl9IRUFMVEhfQUdFTlRfUkVWT0tFRBADEiYKIklOU1RBTExBVElPTl9IRUFMVEhfQUdFTlRfRElTQUJMRUQQBBIuCipJTlNUQUxMQVRJT05fSEVBTFRIX1NUQU5ESU5HX0dSQU5UX01JU1NJTkcQBTLREAoTSW5zdGFsbGF0aW9uU2VydmljZRK6AQoPSW5zdGFsbFNvbHV0aW9uEiguc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsU29sdXRpb25SZXF1ZXN0Gh4uc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb24iXcLzGD0IAhAEKgwKBm9yZ19pZBACGAEwATodChlzYWFzLmluc3RhbGxhdGlvbi5jcmVhdGVkEAJAAUgEUANYA2ABgtPkkwIWOgEqIhEvdjEvaW5zdGFsbGF0aW9ucxLSAQoRVW5pbnN0YWxsU29sdXRpb24SKi5zYWFzLmFjY291bnRzLnYxLlVuaW5zdGFsbFNvbHV0aW9uUmVxdWVzdBoWLmdvb2dsZS5wcm90b2J1Zi5FbXB0eSJ5wvMYPQgCEAQqDAoGb3JnX2lkEAIYATABOh0KGXNhYXMuaW5zdGFsbGF0aW9uLnJldm9rZWQQAkABSARQA1gDYAGC0+STAjI6ASoiLS92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9OnVuaW5zdGFsbBLSAQoSTmFycm93SW5zdGFsbGF0aW9uEisuc2Fhcy5hY2NvdW50cy52MS5OYXJyb3dJbnN0YWxsYXRpb25SZXF1ZXN0GhYuZ29vZ2xlLnByb3RvYnVmLkVtcHR5InfC8xg+CAIQBCoMCgZvcmdfaWQQAhgBMAE6Hgoac2Fhcy5pbnN0YWxsYXRpb24ubmFycm93ZWQQAkABSARQA1gDYAGC0+STAi86ASoiKi92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9Om5hcnJvdxKJAgodVHJhbnNmZXJJbnN0YWxsYXRpb25Pd25lcnNoaXASNi5zYWFzLmFjY291bnRzLnYxLlRyYW5zZmVySW5zdGFsbGF0aW9uT3duZXJzaGlwUmVxdWVzdBoeLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uIo8BwvMYSwgCEAQqDAoGb3JnX2lkEAIYATABOisKJ3NhYXMuaW5zdGFsbGF0aW9uLm93bmVyc2hpcF90cmFuc2ZlcnJlZBACQAFIBFADWANgAYLT5JMCOjoBKiI1L3YxL2luc3RhbGxhdGlvbnMve2luc3RhbGxhdGlvbl9pZH06dHJhbnNmZXJPd25lcnNoaXASuQEKD0dldEluc3RhbGxhdGlvbhIoLnNhYXMuYWNjb3VudHMudjEuR2V0SW5zdGFsbGF0aW9uUmVxdWVzdBopLnNhYXMuYWNjb3VudHMudjEuR2V0SW5zdGFsbGF0aW9uUmVzcG9uc2UiUcLzGCIIAhADKgwKBm9yZ19pZBACGAEwAToCEAFAAUgDUANYA2ABgtPkkwIlEiMvdjEvaW5zdGFsbGF0aW9ucy97aW5zdGFsbGF0aW9uX2lkfRKGAQoRTGlzdEluc3RhbGxhdGlvbnMSKi5zYWFzLmFjY291bnRzLnYxLkxpc3RJbnN0YWxsYXRpb25zUmVxdWVzdBorLnNhYXMuYWNjb3VudHMudjEuTGlzdEluc3RhbGxhdGlvbnNSZXNwb25zZSIYwvMYFAgDEAEwAToCEAFAAUgHUANYA2ABEsoBChZMaXN0QXZhaWxhYmxlU29sdXRpb25zEi8uc2Fhcy5hY2NvdW50cy52MS5MaXN0QXZhaWxhYmxlU29sdXRpb25zUmVxdWVzdBowLnNhYXMuYWNjb3VudHMudjEuTGlzdEF2YWlsYWJsZVNvbHV0aW9uc1Jlc3BvbnNlIk3C8xgiCAIQBCoMCgZvcmdfaWQQAhgBMAE6AhABQAFIA1ADWANgAYLT5JMCIRIfL3YxL2luc3RhbGxhdGlvbnM6bGlzdEF2YWlsYWJsZRLRAQodTGlzdE9yZ2FuaXphdGlvbkluc3RhbGxhdGlvbnMSNi5zYWFzLmFjY291bnRzLnYxLkxpc3RPcmdhbml6YXRpb25JbnN0YWxsYXRpb25zUmVxdWVzdBo3LnNhYXMuYWNjb3VudHMudjEuTGlzdE9yZ2FuaXphdGlvbkluc3RhbGxhdGlvbnNSZXNwb25zZSI/wvMYIggCEAQqDAoGb3JnX2lkEAIYATABOgIQAUABSANQA1gDYAGC0+STAhMSES92MS9pbnN0YWxsYXRpb25zEvABChdHcmFudEluc3RhbGxhdGlvblRvVGVhbRIwLnNhYXMuYWNjb3VudHMudjEuR3JhbnRJbnN0YWxsYXRpb25Ub1RlYW1SZXF1ZXN0GjEuc2Fhcy5hY2NvdW50cy52MS5HcmFudEluc3RhbGxhdGlvblRvVGVhbVJlc3BvbnNlInDC8xg4CAIQBCoMCgZvcmdfaWQQAhgBMAE6FgoSc2Fhcy5zY29wZS5ncmFudGVkEAJAAUgEUANYA2ABeAKC0+STAi46ASoiKS92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9L3RlYW1zEu0BChpSZXZva2VJbnN0YWxsYXRpb25Gcm9tVGVhbRIzLnNhYXMuYWNjb3VudHMudjEuUmV2b2tlSW5zdGFsbGF0aW9uRnJvbVRlYW1SZXF1ZXN0GhYuZ29vZ2xlLnByb3RvYnVmLkVtcHR5IoEBwvMYOAgCEAQqDAoGb3JnX2lkEAIYATABOhYKEnNhYXMuc2NvcGUucmV2b2tlZBACQAFIBFADWANgAXgCgtPkkwI/OgEqIjovdjEvaW5zdGFsbGF0aW9ucy97aW5zdGFsbGF0aW9uX2lkfS90ZWFtcy97dGVhbV9pZH06cmV2b2tlYgZwcm90bzM", [file_buf_validate_validate, file_google_api_annotations, file_google_protobuf_empty, file_google_protobuf_timestamp, file_saas_accounts_v1_authorization, file_saas_policy_v1_options]); + fileDesc("CiRzYWFzL2FjY291bnRzL3YxL2luc3RhbGxhdGlvbnMucHJvdG8SEHNhYXMuYWNjb3VudHMudjEi+QMKDEluc3RhbGxhdGlvbhIUCgJpZBgBIAEoCUIIukgFcgOwAQESGAoGb3JnX2lkGAIgASgJQgi6SAVyA7ABARIkChJhZ2VudF9wcmluY2lwYWxfaWQYAyABKAlCCLpIBXIDsAEBEiQKEm93bmVyX3ByaW5jaXBhbF9pZBgFIAEoCUIIukgFcgOwAQESHgoWY29fb3duZXJfcHJpbmNpcGFsX2lkcxgGIAMoCRIkChJyb290X3Njb3BlX25vZGVfaWQYByABKAlCCLpIBXIDsAEBEjQKBnN0YXR1cxgIIAEoDjIkLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uU3RhdHVzEi4KCmNyZWF0ZWRfYXQYCSABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wEjMKCnJldm9rZWRfYXQYCiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wSACIAQESHgoJdGFyZ2V0X2lkGAsgASgJQgu6SAjYAQFyA7ABARIgCg5yZXZva2VkX3JlYXNvbhgMIAEoCUIIukgFcgMYgAQSIAoOZmFpbHVyZV9yZWFzb24YDSABKAlCCLpIBXIDGIAEQg0KC19yZXZva2VkX2F0SgQIBBAFUhNzb2x1dGlvbl9pZGVudGlmaWVyIuADChZJbnN0YWxsU29sdXRpb25SZXF1ZXN0EhgKBm9yZ19pZBgBIAEoCUIIukgFcgOwAQESJAoQYWdlbnRfaWRlbnRpZmllchgCIAEoCUIKukgHcgUQARiABBIeCgxkaXNwbGF5X25hbWUYBCABKAlCCLpIBXIDGIAEEiIKEHJvb3Rfc2NvcGVfbGFiZWwYBiABKAlCCLpIBXIDGIAEEhkKB3JvbGVfaWQYByABKAlCCLpIBXIDsAEBEicKEm93bmVyX3ByaW5jaXBhbF9pZBgIIAEoCUILukgI2AEBcgOwAQESLwoWY29fb3duZXJfcHJpbmNpcGFsX2lkcxgJIAMoCUIPukgMkgEJECAiBXIDsAEBEioKEWFsbG93ZWRfYXVkaWVuY2VzGAogAygJQg+6SAySAQkQQCIFcgMYgAQSJwoOYWxsb3dlZF9zY29wZXMYCyADKAlCD7pIDJIBCRBAIgVyAxiAARIbCgl0YXJnZXRfaWQYDCABKAlCCLpIBXIDsAEBEh8KBXNjb3BlGA0gASgJQgu6SAhyBhACKICAQEgAiAEBQggKBl9zY29wZUoECAMQBEoECAUQBlITc29sdXRpb25faWRlbnRpZmllclIPcm9vdF9zY29wZV9wYXRoInQKGU5hcnJvd0luc3RhbGxhdGlvblJlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIhCg9pbnN0YWxsYXRpb25faWQYAiABKAlCCLpIBXIDsAEBEhoKBXNjb3BlGAMgASgJQgu6SAhyBhACKICAQCJXChhSZXRyeUluc3RhbGxhdGlvblJlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIhCg9pbnN0YWxsYXRpb25faWQYAiABKAlCCLpIBXIDsAEBIlcKGFVuaW5zdGFsbFNvbHV0aW9uUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQEivgEKJFRyYW5zZmVySW5zdGFsbGF0aW9uT3duZXJzaGlwUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQESKAoWbmV3X293bmVyX3ByaW5jaXBhbF9pZBgDIAEoCUIIukgFcgOwAQESLwoWY29fb3duZXJfcHJpbmNpcGFsX2lkcxgEIAMoCUIPukgMkgEJECAiBXIDsAEBIlUKFkdldEluc3RhbGxhdGlvblJlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIhCg9pbnN0YWxsYXRpb25faWQYAiABKAlCCLpIBXIDsAEBIoUBChdHZXRJbnN0YWxsYXRpb25SZXNwb25zZRI0CgxpbnN0YWxsYXRpb24YASABKAsyHi5zYWFzLmFjY291bnRzLnYxLkluc3RhbGxhdGlvbhI0CgZoZWFsdGgYAiABKA4yJC5zYWFzLmFjY291bnRzLnYxLkluc3RhbGxhdGlvbkhlYWx0aCKdAQoYTGlzdEluc3RhbGxhdGlvbnNSZXF1ZXN0EhgKBm9yZ19pZBgBIAEoCUIIukgFcgOwAQESNAoGc3RhdHVzGAIgASgOMiQuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25TdGF0dXMSHQoJcGFnZV9zaXplGAMgASgFQgq6SAcaBRj0AygAEhIKCnBhZ2VfdG9rZW4YBCABKAkigQEKE0luc3RhbGxhdGlvblN1bW1hcnkSNAoMaW5zdGFsbGF0aW9uGAEgASgLMh4uc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb24SNAoGaGVhbHRoGAIgASgOMiQuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25IZWFsdGgicgoZTGlzdEluc3RhbGxhdGlvbnNSZXNwb25zZRI8Cg1pbnN0YWxsYXRpb25zGAEgAygLMiUuc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb25TdW1tYXJ5EhcKD25leHRfcGFnZV90b2tlbhgCIAEoCSJzCiRMaXN0T3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uc1JlcXVlc3QSGAoGb3JnX2lkGAEgASgJQgi6SAVyA7ABARIdCglwYWdlX3NpemUYAiABKAVCCrpIBxoFGPQDKAASEgoKcGFnZV90b2tlbhgDIAEoCSLSAgoYT3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uEjQKDGluc3RhbGxhdGlvbhgBIAEoCzIeLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uEjQKBmhlYWx0aBgCIAEoDjIkLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uSGVhbHRoEjMKBnRhcmdldBgDIAEoCzIjLnNhYXMuYWNjb3VudHMudjEuQXZhaWxhYmxlU29sdXRpb24SFwoPcm9vdF9zY29wZV9wYXRoGAQgASgJEjwKDWdyYW50ZWRfdGVhbXMYBSADKAsyJS5zYWFzLmFjY291bnRzLnYxLkNvbGxlY3Rpb25SZWFkR3JhbnQSPgoPaW5oZXJpdGVkX3RlYW1zGAYgAygLMiUuc2Fhcy5hY2NvdW50cy52MS5Db2xsZWN0aW9uUmVhZEdyYW50IoMBCiVMaXN0T3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uc1Jlc3BvbnNlEkEKDWluc3RhbGxhdGlvbnMYASADKAsyKi5zYWFzLmFjY291bnRzLnYxLk9yZ2FuaXphdGlvbkluc3RhbGxhdGlvbhIXCg9uZXh0X3BhZ2VfdG9rZW4YAiABKAkikwEKHkdyYW50SW5zdGFsbGF0aW9uVG9UZWFtUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEiEKD2luc3RhbGxhdGlvbl9pZBgCIAEoCUIIukgFcgOwAQESGQoHdGVhbV9pZBgDIAEoCUIIukgFcgOwAQESGQoHcm9sZV9pZBgEIAEoCUIIukgFcgOwAQEiTgofR3JhbnRJbnN0YWxsYXRpb25Ub1RlYW1SZXNwb25zZRIrCgVncmFudBgBIAEoCzIcLnNhYXMuYWNjb3VudHMudjEuU2NvcGVHcmFudCKWAQohUmV2b2tlSW5zdGFsbGF0aW9uRnJvbVRlYW1SZXF1ZXN0EhgKBm9yZ19pZBgBIAEoCUIIukgFcgOwAQESIQoPaW5zdGFsbGF0aW9uX2lkGAIgASgJQgi6SAVyA7ABARIZCgd0ZWFtX2lkGAMgASgJQgi6SAVyA7ABARIZCgdyb2xlX2lkGAQgASgJQgi6SAVyA7ABASLtAQoRQXZhaWxhYmxlU29sdXRpb24SGwoJdGFyZ2V0X2lkGAEgASgJQgi6SAVyA7ABARISCgpiaW5kaW5nX2lkGAIgASgJEhMKC3JvdXRlX2FsaWFzGAMgASgJEhkKEXJlbGVhc2VfcHVibGlzaGVyGAQgASgJEhQKDHJlbGVhc2VfbmFtZRgFIAEoCRIXCg9yZWxlYXNlX3ZlcnNpb24YBiABKAkSGQoRb3BlbmVkX2dlbmVyYXRpb24YByABKAQSGgoSYXBwbGllZF9nZW5lcmF0aW9uGAggASgEEhEKCWluc3RhbGxlZBgJIAEoCCJsCh1MaXN0QXZhaWxhYmxlU29sdXRpb25zUmVxdWVzdBIYCgZvcmdfaWQYASABKAlCCLpIBXIDsAEBEh0KCXBhZ2Vfc2l6ZRgCIAEoBUIKukgHGgUY9AMoABISCgpwYWdlX3Rva2VuGAMgASgJInEKHkxpc3RBdmFpbGFibGVTb2x1dGlvbnNSZXNwb25zZRI2Cglzb2x1dGlvbnMYASADKAsyIy5zYWFzLmFjY291bnRzLnYxLkF2YWlsYWJsZVNvbHV0aW9uEhcKD25leHRfcGFnZV90b2tlbhgCIAEoCSq7AQoSSW5zdGFsbGF0aW9uU3RhdHVzEiMKH0lOU1RBTExBVElPTl9TVEFUVVNfVU5TUEVDSUZJRUQQABIeChpJTlNUQUxMQVRJT05fU1RBVFVTX0FDVElWRRABEh8KG0lOU1RBTExBVElPTl9TVEFUVVNfUkVWT0tFRBACEh8KG0lOU1RBTExBVElPTl9TVEFUVVNfUEVORElORxADEh4KGklOU1RBTExBVElPTl9TVEFUVVNfRkFJTEVEEAQqhAIKEkluc3RhbGxhdGlvbkhlYWx0aBIjCh9JTlNUQUxMQVRJT05fSEVBTFRIX1VOU1BFQ0lGSUVEEAASHwobSU5TVEFMTEFUSU9OX0hFQUxUSF9IRUFMVEhZEAESKQolSU5TVEFMTEFUSU9OX0hFQUxUSF9OT19FTElHSUJMRV9PV05FUhACEiUKIUlOU1RBTExBVElPTl9IRUFMVEhfQUdFTlRfUkVWT0tFRBADEiYKIklOU1RBTExBVElPTl9IRUFMVEhfQUdFTlRfRElTQUJMRUQQBBIuCipJTlNUQUxMQVRJT05fSEVBTFRIX1NUQU5ESU5HX0dSQU5UX01JU1NJTkcQBTKqEgoTSW5zdGFsbGF0aW9uU2VydmljZRK6AQoPSW5zdGFsbFNvbHV0aW9uEiguc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsU29sdXRpb25SZXF1ZXN0Gh4uc2Fhcy5hY2NvdW50cy52MS5JbnN0YWxsYXRpb24iXcLzGD0IAhAEKgwKBm9yZ19pZBACGAEwATodChlzYWFzLmluc3RhbGxhdGlvbi5jcmVhdGVkEAJAAUgEUANYA2ABgtPkkwIWOgEqIhEvdjEvaW5zdGFsbGF0aW9ucxLSAQoRVW5pbnN0YWxsU29sdXRpb24SKi5zYWFzLmFjY291bnRzLnYxLlVuaW5zdGFsbFNvbHV0aW9uUmVxdWVzdBoWLmdvb2dsZS5wcm90b2J1Zi5FbXB0eSJ5wvMYPQgCEAQqDAoGb3JnX2lkEAIYATABOh0KGXNhYXMuaW5zdGFsbGF0aW9uLnJldm9rZWQQAkABSARQA1gDYAGC0+STAjI6ASoiLS92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9OnVuaW5zdGFsbBLSAQoSTmFycm93SW5zdGFsbGF0aW9uEisuc2Fhcy5hY2NvdW50cy52MS5OYXJyb3dJbnN0YWxsYXRpb25SZXF1ZXN0GhYuZ29vZ2xlLnByb3RvYnVmLkVtcHR5InfC8xg+CAIQBCoMCgZvcmdfaWQQAhgBMAE6Hgoac2Fhcy5pbnN0YWxsYXRpb24ubmFycm93ZWQQAkABSARQA1gDYAGC0+STAi86ASoiKi92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9Om5hcnJvdxLWAQoRUmV0cnlJbnN0YWxsYXRpb24SKi5zYWFzLmFjY291bnRzLnYxLlJldHJ5SW5zdGFsbGF0aW9uUmVxdWVzdBoeLnNhYXMuYWNjb3VudHMudjEuSW5zdGFsbGF0aW9uInXC8xg9CAIQBCoMCgZvcmdfaWQQAhgBMAE6HQoZc2Fhcy5pbnN0YWxsYXRpb24ucmV0cmllZBACQAFIBFADWANgAYLT5JMCLjoBKiIpL3YxL2luc3RhbGxhdGlvbnMve2luc3RhbGxhdGlvbl9pZH06cmV0cnkSiQIKHVRyYW5zZmVySW5zdGFsbGF0aW9uT3duZXJzaGlwEjYuc2Fhcy5hY2NvdW50cy52MS5UcmFuc2Zlckluc3RhbGxhdGlvbk93bmVyc2hpcFJlcXVlc3QaHi5zYWFzLmFjY291bnRzLnYxLkluc3RhbGxhdGlvbiKPAcLzGEsIAhAEKgwKBm9yZ19pZBACGAEwATorCidzYWFzLmluc3RhbGxhdGlvbi5vd25lcnNoaXBfdHJhbnNmZXJyZWQQAkABSARQA1gDYAGC0+STAjo6ASoiNS92MS9pbnN0YWxsYXRpb25zL3tpbnN0YWxsYXRpb25faWR9OnRyYW5zZmVyT3duZXJzaGlwErkBCg9HZXRJbnN0YWxsYXRpb24SKC5zYWFzLmFjY291bnRzLnYxLkdldEluc3RhbGxhdGlvblJlcXVlc3QaKS5zYWFzLmFjY291bnRzLnYxLkdldEluc3RhbGxhdGlvblJlc3BvbnNlIlHC8xgiCAIQAyoMCgZvcmdfaWQQAhgBMAE6AhABQAFIA1ADWANgAYLT5JMCJRIjL3YxL2luc3RhbGxhdGlvbnMve2luc3RhbGxhdGlvbl9pZH0ShgEKEUxpc3RJbnN0YWxsYXRpb25zEiouc2Fhcy5hY2NvdW50cy52MS5MaXN0SW5zdGFsbGF0aW9uc1JlcXVlc3QaKy5zYWFzLmFjY291bnRzLnYxLkxpc3RJbnN0YWxsYXRpb25zUmVzcG9uc2UiGMLzGBQIAxABMAE6AhABQAFIB1ADWANgARLKAQoWTGlzdEF2YWlsYWJsZVNvbHV0aW9ucxIvLnNhYXMuYWNjb3VudHMudjEuTGlzdEF2YWlsYWJsZVNvbHV0aW9uc1JlcXVlc3QaMC5zYWFzLmFjY291bnRzLnYxLkxpc3RBdmFpbGFibGVTb2x1dGlvbnNSZXNwb25zZSJNwvMYIggCEAQqDAoGb3JnX2lkEAIYATABOgIQAUABSANQA1gDYAGC0+STAiESHy92MS9pbnN0YWxsYXRpb25zOmxpc3RBdmFpbGFibGUS0QEKHUxpc3RPcmdhbml6YXRpb25JbnN0YWxsYXRpb25zEjYuc2Fhcy5hY2NvdW50cy52MS5MaXN0T3JnYW5pemF0aW9uSW5zdGFsbGF0aW9uc1JlcXVlc3QaNy5zYWFzLmFjY291bnRzLnYxLkxpc3RPcmdhbml6YXRpb25JbnN0YWxsYXRpb25zUmVzcG9uc2UiP8LzGCIIAhAEKgwKBm9yZ19pZBACGAEwAToCEAFAAUgDUANYA2ABgtPkkwITEhEvdjEvaW5zdGFsbGF0aW9ucxLwAQoXR3JhbnRJbnN0YWxsYXRpb25Ub1RlYW0SMC5zYWFzLmFjY291bnRzLnYxLkdyYW50SW5zdGFsbGF0aW9uVG9UZWFtUmVxdWVzdBoxLnNhYXMuYWNjb3VudHMudjEuR3JhbnRJbnN0YWxsYXRpb25Ub1RlYW1SZXNwb25zZSJwwvMYOAgCEAQqDAoGb3JnX2lkEAIYATABOhYKEnNhYXMuc2NvcGUuZ3JhbnRlZBACQAFIBFADWANgAXgCgtPkkwIuOgEqIikvdjEvaW5zdGFsbGF0aW9ucy97aW5zdGFsbGF0aW9uX2lkfS90ZWFtcxLtAQoaUmV2b2tlSW5zdGFsbGF0aW9uRnJvbVRlYW0SMy5zYWFzLmFjY291bnRzLnYxLlJldm9rZUluc3RhbGxhdGlvbkZyb21UZWFtUmVxdWVzdBoWLmdvb2dsZS5wcm90b2J1Zi5FbXB0eSKBAcLzGDgIAhAEKgwKBm9yZ19pZBACGAEwAToWChJzYWFzLnNjb3BlLnJldm9rZWQQAkABSARQA1gDYAF4AoLT5JMCPzoBKiI6L3YxL2luc3RhbGxhdGlvbnMve2luc3RhbGxhdGlvbl9pZH0vdGVhbXMve3RlYW1faWR9OnJldm9rZWIGcHJvdG8z", [file_buf_validate_validate, file_google_api_annotations, file_google_protobuf_empty, file_google_protobuf_timestamp, file_saas_accounts_v1_authorization, file_saas_policy_v1_options]); /** * Installation is one solution's bound identity in an org. Server-assigned ids; @@ -100,6 +100,14 @@ export type Installation = Message<"saas.accounts.v1.Installation"> & { * @generated from field: string revoked_reason = 12; */ revokedReason: string; + + /** + * Why the install step failed: the solution's bounded refusal reason, or a + * fixed reason when it never answered. Empty unless status is FAILED. + * + * @generated from field: string failure_reason = 13; + */ + failureReason: string; }; /** @@ -255,6 +263,28 @@ export type NarrowInstallationRequest = Message<"saas.accounts.v1.NarrowInstalla export const NarrowInstallationRequestSchema: GenMessage = /*@__PURE__*/ messageDesc(file_saas_accounts_v1_installations, 2); +/** + * @generated from message saas.accounts.v1.RetryInstallationRequest + */ +export type RetryInstallationRequest = Message<"saas.accounts.v1.RetryInstallationRequest"> & { + /** + * @generated from field: string org_id = 1; + */ + orgId: string; + + /** + * @generated from field: string installation_id = 2; + */ + installationId: string; +}; + +/** + * Describes the message saas.accounts.v1.RetryInstallationRequest. + * Use `create(RetryInstallationRequestSchema)` to create a new message. + */ +export const RetryInstallationRequestSchema: GenMessage = /*@__PURE__*/ + messageDesc(file_saas_accounts_v1_installations, 3); + /** * @generated from message saas.accounts.v1.UninstallSolutionRequest */ @@ -275,7 +305,7 @@ export type UninstallSolutionRequest = Message<"saas.accounts.v1.UninstallSoluti * Use `create(UninstallSolutionRequestSchema)` to create a new message. */ export const UninstallSolutionRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 3); + messageDesc(file_saas_accounts_v1_installations, 4); /** * @generated from message saas.accounts.v1.TransferInstallationOwnershipRequest @@ -311,7 +341,7 @@ export type TransferInstallationOwnershipRequest = Message<"saas.accounts.v1.Tra * Use `create(TransferInstallationOwnershipRequestSchema)` to create a new message. */ export const TransferInstallationOwnershipRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 4); + messageDesc(file_saas_accounts_v1_installations, 5); /** * @generated from message saas.accounts.v1.GetInstallationRequest @@ -333,7 +363,7 @@ export type GetInstallationRequest = Message<"saas.accounts.v1.GetInstallationRe * Use `create(GetInstallationRequestSchema)` to create a new message. */ export const GetInstallationRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 5); + messageDesc(file_saas_accounts_v1_installations, 6); /** * @generated from message saas.accounts.v1.GetInstallationResponse @@ -357,7 +387,7 @@ export type GetInstallationResponse = Message<"saas.accounts.v1.GetInstallationR * Use `create(GetInstallationResponseSchema)` to create a new message. */ export const GetInstallationResponseSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 6); + messageDesc(file_saas_accounts_v1_installations, 7); /** * ListInstallationsRequest asks what one organization has installed. Every other @@ -403,7 +433,7 @@ export type ListInstallationsRequest = Message<"saas.accounts.v1.ListInstallatio * Use `create(ListInstallationsRequestSchema)` to create a new message. */ export const ListInstallationsRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 7); + messageDesc(file_saas_accounts_v1_installations, 8); /** * InstallationSummary is one row of the listing: the installation, plus the health @@ -430,7 +460,7 @@ export type InstallationSummary = Message<"saas.accounts.v1.InstallationSummary" * Use `create(InstallationSummarySchema)` to create a new message. */ export const InstallationSummarySchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 8); + messageDesc(file_saas_accounts_v1_installations, 9); /** * @generated from message saas.accounts.v1.ListInstallationsResponse @@ -455,7 +485,7 @@ export type ListInstallationsResponse = Message<"saas.accounts.v1.ListInstallati * Use `create(ListInstallationsResponseSchema)` to create a new message. */ export const ListInstallationsResponseSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 9); + messageDesc(file_saas_accounts_v1_installations, 10); /** * @generated from message saas.accounts.v1.ListOrganizationInstallationsRequest @@ -486,7 +516,7 @@ export type ListOrganizationInstallationsRequest = Message<"saas.accounts.v1.Lis * Use `create(ListOrganizationInstallationsRequestSchema)` to create a new message. */ export const ListOrganizationInstallationsRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 10); + messageDesc(file_saas_accounts_v1_installations, 11); /** * OrganizationInstallation is one active installation as its organisation's @@ -547,7 +577,7 @@ export type OrganizationInstallation = Message<"saas.accounts.v1.OrganizationIns * Use `create(OrganizationInstallationSchema)` to create a new message. */ export const OrganizationInstallationSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 11); + messageDesc(file_saas_accounts_v1_installations, 12); /** * @generated from message saas.accounts.v1.ListOrganizationInstallationsResponse @@ -571,7 +601,7 @@ export type ListOrganizationInstallationsResponse = Message<"saas.accounts.v1.Li * Use `create(ListOrganizationInstallationsResponseSchema)` to create a new message. */ export const ListOrganizationInstallationsResponseSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 12); + messageDesc(file_saas_accounts_v1_installations, 13); /** * @generated from message saas.accounts.v1.GrantInstallationToTeamRequest @@ -608,7 +638,7 @@ export type GrantInstallationToTeamRequest = Message<"saas.accounts.v1.GrantInst * Use `create(GrantInstallationToTeamRequestSchema)` to create a new message. */ export const GrantInstallationToTeamRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 13); + messageDesc(file_saas_accounts_v1_installations, 14); /** * @generated from message saas.accounts.v1.GrantInstallationToTeamResponse @@ -625,7 +655,7 @@ export type GrantInstallationToTeamResponse = Message<"saas.accounts.v1.GrantIns * Use `create(GrantInstallationToTeamResponseSchema)` to create a new message. */ export const GrantInstallationToTeamResponseSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 14); + messageDesc(file_saas_accounts_v1_installations, 15); /** * @generated from message saas.accounts.v1.RevokeInstallationFromTeamRequest @@ -660,7 +690,7 @@ export type RevokeInstallationFromTeamRequest = Message<"saas.accounts.v1.Revoke * Use `create(RevokeInstallationFromTeamRequestSchema)` to create a new message. */ export const RevokeInstallationFromTeamRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 15); + messageDesc(file_saas_accounts_v1_installations, 16); /** * AvailableSolution is one installable solution target: its immutable identity, @@ -739,7 +769,7 @@ export type AvailableSolution = Message<"saas.accounts.v1.AvailableSolution"> & * Use `create(AvailableSolutionSchema)` to create a new message. */ export const AvailableSolutionSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 16); + messageDesc(file_saas_accounts_v1_installations, 17); /** * @generated from message saas.accounts.v1.ListAvailableSolutionsRequest @@ -769,7 +799,7 @@ export type ListAvailableSolutionsRequest = Message<"saas.accounts.v1.ListAvaila * Use `create(ListAvailableSolutionsRequestSchema)` to create a new message. */ export const ListAvailableSolutionsRequestSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 17); + messageDesc(file_saas_accounts_v1_installations, 18); /** * @generated from message saas.accounts.v1.ListAvailableSolutionsResponse @@ -791,7 +821,7 @@ export type ListAvailableSolutionsResponse = Message<"saas.accounts.v1.ListAvail * Use `create(ListAvailableSolutionsResponseSchema)` to create a new message. */ export const ListAvailableSolutionsResponseSchema: GenMessage = /*@__PURE__*/ - messageDesc(file_saas_accounts_v1_installations, 18); + messageDesc(file_saas_accounts_v1_installations, 19); /** * InstallationStatus is the lifecycle of an installation row. An uninstall is a @@ -815,6 +845,22 @@ export enum InstallationStatus { * @generated from enum value: INSTALLATION_STATUS_REVOKED = 2; */ REVOKED = 2, + + /** + * Installed, and waiting for the solution to acknowledge its standard install + * step. Nothing can use a pending installation. + * + * @generated from enum value: INSTALLATION_STATUS_PENDING = 3; + */ + PENDING = 3, + + /** + * The solution refused its install step, or never answered within the + * attempt budget. Nothing can use it; an administrator may retry it. + * + * @generated from enum value: INSTALLATION_STATUS_FAILED = 4; + */ + FAILED = 4, } /** @@ -916,6 +962,17 @@ export const InstallationService: GenService<{ input: typeof NarrowInstallationRequestSchema; output: typeof EmptySchema; }, + /** + * RetryInstallation runs a failed installation's install step again, at the + * same revision. The installation is pending until the solution acknowledges. + * + * @generated from rpc saas.accounts.v1.InstallationService.RetryInstallation + */ + retryInstallation: { + methodKind: "unary"; + input: typeof RetryInstallationRequestSchema; + output: typeof InstallationSchema; + }, /** * TransferInstallationOwnership reassigns the owner of record and replaces the * co-owner succession set. This is how a "connection owner left the org" diff --git a/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/module_registration_pb.ts b/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/module_registration_pb.ts index 90288e0f0..1060fbc08 100644 --- a/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/module_registration_pb.ts +++ b/module/services/frontend/code/packages/saas-sdk/generated/typescript/src/gen/saas/accounts/v1/module_registration_pb.ts @@ -13,7 +13,7 @@ import type { Message } from "@bufbuild/protobuf"; * Describes the file saas/accounts/v1/module_registration.proto. */ export const file_saas_accounts_v1_module_registration: GenFile = /*@__PURE__*/ - fileDesc("CipzYWFzL2FjY291bnRzL3YxL21vZHVsZV9yZWdpc3RyYXRpb24ucHJvdG8SEHNhYXMuYWNjb3VudHMudjEivAEKHE1vZHVsZU1pbnRXb3JrQ29udGV4dFJlcXVlc3QSPAoGcHJlZml4GAEgASgJQiy6SClyJxABGD8yIV5bYS16MC05XSg/OlthLXowLTktXSpbYS16MC05XSk/JBIqChVzZXJ2aWNlX2FjY291bnRfdG9rZW4YAyABKAlCC7pICHIGEAEYgIAEEiQKD29yZ2FuaXphdGlvbl9pZBgEIAEoCUILukgI2AEBcgOwAQFKBAgCEANSBnNlY3JldCKEAQodTW9kdWxlTWludFdvcmtDb250ZXh0UmVzcG9uc2USDQoFdG9rZW4YASABKAkSLgoKZXhwaXJlc19hdBgCIAEoCzIaLmdvb2dsZS5wcm90b2J1Zi5UaW1lc3RhbXASFAoMcHJpbmNpcGFsX2lkGAMgASgJEg4KBnRlbmFudBgEIAEoCSLeAQohTW9kdWxlTWludE9wZXJhdGlvbkNvbnRleHRSZXF1ZXN0EjwKBnByZWZpeBgBIAEoCUIsukgpcicQARg/MiFeW2EtejAtOV0oPzpbYS16MC05LV0qW2EtejAtOV0pPyQSGwoHYmluZGluZxgDIAEoCUIKukgHcgUQARiAARIqChVzZXJ2aWNlX2FjY291bnRfdG9rZW4YBCABKAlCC7pICHIGEAEYgIAEEiQKD29yZ2FuaXphdGlvbl9pZBgFIAEoCUILukgI2AEBcgOwAQFKBAgCEANSBnNlY3JldCKsAQoiTW9kdWxlTWludE9wZXJhdGlvbkNvbnRleHRSZXNwb25zZRINCgV0b2tlbhgBIAEoCRIuCgpleHBpcmVzX2F0GAIgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIUCgxwcmluY2lwYWxfaWQYAyABKAkSDgoGdGVuYW50GAQgASgJEhAKCGF1ZGllbmNlGAUgASgJEg8KB2JpbmRpbmcYBiABKAki+AEKJ01vZHVsZU1pbnRTb3VyY2VPcGVyYXRpb25Db250ZXh0UmVxdWVzdBI8CgZwcmVmaXgYASABKAlCLLpIKXInEAEYPzIhXlthLXowLTldKD86W2EtejAtOS1dKlthLXowLTldKT8kEioKFXNlcnZpY2VfYWNjb3VudF90b2tlbhgFIAEoCUILukgIcgYQARiAgAQSIQoNZGVsZWdhdGlvbl9pZBgDIAEoCUIIukgFcgOwAQFIABIdCglzb3VyY2VfaWQYBCABKAlCCLpIBXIDsAEBSABCEwoKZGVsZWdhdGlvbhIFukgCCAFKBAgCEANSBnNlY3JldCL4AQooTW9kdWxlTWludFNvdXJjZU9wZXJhdGlvbkNvbnRleHRSZXNwb25zZRINCgV0b2tlbhgBIAEoCRIuCgpleHBpcmVzX2F0GAIgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIUCgxwcmluY2lwYWxfaWQYAyABKAkSDgoGdGVuYW50GAQgASgJEhAKCGF1ZGllbmNlGAUgASgJEg8KB2JpbmRpbmcYBiABKAkSFQoNZGVsZWdhdGlvbl9pZBgHIAEoCRIRCglzb3VyY2VfaWQYCCABKAkSGgoSb3duZXJfcHJpbmNpcGFsX2lkGAkgASgJYgZwcm90bzM", [file_buf_validate_validate, file_google_protobuf_timestamp]); + fileDesc("CipzYWFzL2FjY291bnRzL3YxL21vZHVsZV9yZWdpc3RyYXRpb24ucHJvdG8SEHNhYXMuYWNjb3VudHMudjEi/gEKHE1vZHVsZU1pbnRXb3JrQ29udGV4dFJlcXVlc3QSPAoGcHJlZml4GAEgASgJQiy6SClyJxABGD8yIV5bYS16MC05XSg/OlthLXowLTktXSpbYS16MC05XSk/JBIqChVzZXJ2aWNlX2FjY291bnRfdG9rZW4YAyABKAlCC7pICHIGEAEYgIAEEiQKD29yZ2FuaXphdGlvbl9pZBgEIAEoCUILukgI2AEBcgOwAQESIAoIdXBzdHJlYW0YBSABKAlCDrpIC9gBAXIGGIAEiAEBEh4KCG1hbmlmZXN0GAYgASgJQgy6SAnYAQFyBBiAgARKBAgCEANSBnNlY3JldCKEAQodTW9kdWxlTWludFdvcmtDb250ZXh0UmVzcG9uc2USDQoFdG9rZW4YASABKAkSLgoKZXhwaXJlc19hdBgCIAEoCzIaLmdvb2dsZS5wcm90b2J1Zi5UaW1lc3RhbXASFAoMcHJpbmNpcGFsX2lkGAMgASgJEg4KBnRlbmFudBgEIAEoCSLeAQohTW9kdWxlTWludE9wZXJhdGlvbkNvbnRleHRSZXF1ZXN0EjwKBnByZWZpeBgBIAEoCUIsukgpcicQARg/MiFeW2EtejAtOV0oPzpbYS16MC05LV0qW2EtejAtOV0pPyQSGwoHYmluZGluZxgDIAEoCUIKukgHcgUQARiAARIqChVzZXJ2aWNlX2FjY291bnRfdG9rZW4YBCABKAlCC7pICHIGEAEYgIAEEiQKD29yZ2FuaXphdGlvbl9pZBgFIAEoCUILukgI2AEBcgOwAQFKBAgCEANSBnNlY3JldCKsAQoiTW9kdWxlTWludE9wZXJhdGlvbkNvbnRleHRSZXNwb25zZRINCgV0b2tlbhgBIAEoCRIuCgpleHBpcmVzX2F0GAIgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIUCgxwcmluY2lwYWxfaWQYAyABKAkSDgoGdGVuYW50GAQgASgJEhAKCGF1ZGllbmNlGAUgASgJEg8KB2JpbmRpbmcYBiABKAki+AEKJ01vZHVsZU1pbnRTb3VyY2VPcGVyYXRpb25Db250ZXh0UmVxdWVzdBI8CgZwcmVmaXgYASABKAlCLLpIKXInEAEYPzIhXlthLXowLTldKD86W2EtejAtOS1dKlthLXowLTldKT8kEioKFXNlcnZpY2VfYWNjb3VudF90b2tlbhgFIAEoCUILukgIcgYQARiAgAQSIQoNZGVsZWdhdGlvbl9pZBgDIAEoCUIIukgFcgOwAQFIABIdCglzb3VyY2VfaWQYBCABKAlCCLpIBXIDsAEBSABCEwoKZGVsZWdhdGlvbhIFukgCCAFKBAgCEANSBnNlY3JldCL4AQooTW9kdWxlTWludFNvdXJjZU9wZXJhdGlvbkNvbnRleHRSZXNwb25zZRINCgV0b2tlbhgBIAEoCRIuCgpleHBpcmVzX2F0GAIgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIUCgxwcmluY2lwYWxfaWQYAyABKAkSDgoGdGVuYW50GAQgASgJEhAKCGF1ZGllbmNlGAUgASgJEg8KB2JpbmRpbmcYBiABKAkSFQoNZGVsZWdhdGlvbl9pZBgHIAEoCRIRCglzb3VyY2VfaWQYCCABKAkSGgoSb3duZXJfcHJpbmNpcGFsX2lkGAkgASgJYgZwcm90bzM", [file_buf_validate_validate, file_google_protobuf_timestamp]); /** * ModuleMintWorkContextRequest asks for the Work Context a composed module @@ -51,6 +51,29 @@ export type ModuleMintWorkContextRequest = Message<"saas.accounts.v1.ModuleMintW * @generated from field: string organization_id = 4; */ organizationId: string; + + /** + * upstream is where the minting workload serves its backend, as an absolute + * https origin with no path: the dynamic half of its registration, reported + * by the runtime that proved its identity with the token above (#952). The + * gateway reaches it only under the SVID the applied presence names for the + * workload, so a claimed origin that is not that workload serves nothing. + * Empty leaves the registration's backend half as it is. + * + * @generated from field: string upstream = 5; + */ + upstream: string; + + /** + * manifest is the solution's frontend manifest as its runtime serves it: the + * other dynamic half of a solution's registration (a module has none), the + * document the host's frontend renders the solution's page from. Reported + * with the same proven identity as upstream. Empty leaves the frontend half + * as it is. + * + * @generated from field: string manifest = 6; + */ + manifest: string; }; /** diff --git a/module/services/frontend/code/packages/saas-sdk/package.json b/module/services/frontend/code/packages/saas-sdk/package.json index 7e0814f14..8357b1dcb 100644 --- a/module/services/frontend/code/packages/saas-sdk/package.json +++ b/module/services/frontend/code/packages/saas-sdk/package.json @@ -1,6 +1,6 @@ { "name": "@codefly-dev/saas-sdk", - "version": "0.3.28", + "version": "0.3.29", "description": "Typed Connect client, gateway-bound facades, and audit data-graph tooling for the saas-starter public API.", "repository": { "type": "git", diff --git a/module/services/frontend/code/src/features/solution-installations/model/installations.ts b/module/services/frontend/code/src/features/solution-installations/model/installations.ts index dc41e2139..094befca1 100644 --- a/module/services/frontend/code/src/features/solution-installations/model/installations.ts +++ b/module/services/frontend/code/src/features/solution-installations/model/installations.ts @@ -6,6 +6,7 @@ import type { Role } from "@/gen/saas/accounts/v1/common_pb"; import { type AvailableSolution, InstallationHealth, + InstallationStatus, type OrganizationInstallation, } from "@/gen/saas/accounts/v1/installations_pb"; import type { Badge } from "@/shared/ui"; @@ -75,6 +76,25 @@ export function availableRows(solutions: AvailableSolution[]): AvailableRow[] { })); } +export function installationStatusView(status: InstallationStatus): FactView { + switch (status) { + case InstallationStatus.PENDING: + return { kind: "value", text: "Pending", tone: "warning" }; + case InstallationStatus.FAILED: + return { kind: "value", text: "Failed", tone: "danger" }; + case InstallationStatus.ACTIVE: + return { kind: "value", text: "Active", tone: "success" }; + case InstallationStatus.REVOKED: + return { kind: "value", text: "Revoked", tone: "danger" }; + default: + return { + kind: "gap", + label: "Not reported", + detail: "The host did not report this installation's status.", + }; + } +} + export function healthView(health: InstallationHealth): FactView { switch (health) { case InstallationHealth.HEALTHY: @@ -125,6 +145,9 @@ export interface InstalledRow { solution: FactView; revision: FactView; health: FactView; + status: InstallationStatus; + lifecycle: FactView; + failureReason: string; installedAt: string | undefined; /** Granted at this installation's authority root: revocable here. */ grantedTeams: TeamGrantRow[]; @@ -183,6 +206,9 @@ export function installedRows( } : NOT_OFFERED, health: healthView(health), + status: installation.status, + lifecycle: installationStatusView(installation.status), + failureReason: installation.failureReason, installedAt: installation.createdAt ? timestampDate(installation.createdAt).toISOString() : undefined, diff --git a/module/services/frontend/code/src/features/solution-installations/service/mutations.ts b/module/services/frontend/code/src/features/solution-installations/service/mutations.ts index d8ef27bb6..1982aa6fb 100644 --- a/module/services/frontend/code/src/features/solution-installations/service/mutations.ts +++ b/module/services/frontend/code/src/features/solution-installations/service/mutations.ts @@ -51,6 +51,16 @@ export function useUninstallSolution(orgId: string) { }); } +export function useRetryInstallation(orgId: string) { + const service = useInstallationService(); + const queryClient = useQueryClient(); + return useMutation({ + mutationFn: (installationId: string) => + service.retryInstallation({ orgId, installationId }), + onSettled: () => invalidate(queryClient, orgId), + }); +} + export interface TeamExposure { installationId: string; teamId: string; diff --git a/module/services/frontend/code/src/features/solution-installations/service/queries.ts b/module/services/frontend/code/src/features/solution-installations/service/queries.ts index 79318be19..60accf833 100644 --- a/module/services/frontend/code/src/features/solution-installations/service/queries.ts +++ b/module/services/frontend/code/src/features/solution-installations/service/queries.ts @@ -1,7 +1,8 @@ import { useQuery } from "@tanstack/react-query"; -import type { - AvailableSolution, - OrganizationInstallation, +import { + type AvailableSolution, + InstallationStatus, + type OrganizationInstallation, } from "@/gen/saas/accounts/v1/installations_pb"; import { useInstallationService } from "@/lib/hooks/use-api-client"; @@ -42,7 +43,7 @@ export function useAvailableSolutions(orgId: string) { } /** - * This organization's active installations, from ListOrganizationInstallations: + * This organization's current installations, from ListOrganizationInstallations: * each with its live health, the catalogue row of the target it installed, and * the teams it is exposed to. The organization is bound to the caller's own * administration by the host, so this reads nothing but its own. @@ -52,6 +53,12 @@ export function useInstallations(orgId: string) { return useQuery({ queryKey: solutionInstallationKeys.installed(orgId), enabled: !!orgId, + refetchInterval: (query) => + query.state.data?.some( + (row) => row.installation?.status === InstallationStatus.PENDING, + ) + ? 2000 + : false, queryFn: async () => { const installations: OrganizationInstallation[] = []; let pageToken = ""; diff --git a/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.container.test.tsx b/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.container.test.tsx index a7c5ad047..85874de61 100644 --- a/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.container.test.tsx +++ b/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.container.test.tsx @@ -8,11 +8,11 @@ import { within, } from "@testing-library/react"; import { HttpResponse, http, type JsonBodyType } from "msw"; -import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { toast } from "sonner"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import Page from "@/app/admin/solutions/page"; -import type { OrgRole, PlatformRole } from "@/lib/auth-session"; import { FRONTEND_NAVIGATION } from "@/gen/saas/frontend/v1/plugin_catalog"; +import type { OrgRole, PlatformRole } from "@/lib/auth-session"; import { canPresent } from "@/lib/plugins/presentation"; import { renderInApp, rpc } from "@/test/container"; import { server } from "@/test/setup"; @@ -408,7 +408,7 @@ describe("SolutionInstallationsPage admin container", () => { screen.getByRole("table", { name: "Installed solutions" }), ).getByText("ledger"), ).toBeTruthy(); - expect(toast.success).toHaveBeenCalledWith("ledger installed"); + expect(toast.success).toHaveBeenCalledWith("ledger installation pending"); }); it("does not install without a role for the solution's agent", async () => { @@ -1455,3 +1455,57 @@ describe("accessible validation and host text", () => { expect(toast.success).not.toHaveBeenCalled(); }); }); + +// HOST-INST-002, HOST-INST-004, HOST-INST-005: no grants while unacknowledged; +// a refusal remains visible and Retry reuses the same installation identity. +it("shows pending and failed installations and retries the failed step", async () => { + let retried = false; + const failed = structuredClone(installations); + failed.installations[0].installation.status = "INSTALLATION_STATUS_FAILED"; + Object.assign(failed.installations[0].installation, { + failureReason: "Example setup refused", + }); + failed.installations[0].grantedTeams = []; + failed.installations[0].inheritedTeams = []; + server.use( + http.post(rpc("InstallationService", "ListAvailableSolutions"), () => + HttpResponse.json(available), + ), + http.post( + rpc("InstallationService", "ListOrganizationInstallations"), + () => { + if (retried) { + const pending = structuredClone(failed); + pending.installations[0].installation.status = + "INSTALLATION_STATUS_PENDING"; + Object.assign(pending.installations[0].installation, { + failureReason: "", + }); + return HttpResponse.json(pending); + } + return HttpResponse.json(failed); + }, + ), + http.post( + rpc("InstallationService", "RetryInstallation"), + async ({ request }) => { + await assertBody(request, { + orgId: "org-1", + installationId: EXPENSES_INSTALLATION, + }); + retried = true; + return HttpResponse.json({ + ...failed.installations[0].installation, + status: "INSTALLATION_STATUS_PENDING", + failureReason: "", + }); + }, + ), + ); + renderInApp(); + expect(await screen.findByText("Example setup refused")).toBeTruthy(); + expect(screen.queryByRole("button", { name: /grant.*team/i })).toBeNull(); + fireEvent.click(screen.getByRole("button", { name: "Retry expenses" })); + expect(await screen.findByText("Pending")).toBeTruthy(); + expect(screen.queryByRole("button", { name: "Retry expenses" })).toBeNull(); +}); diff --git a/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.tsx b/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.tsx index 435803564..6cc244756 100644 --- a/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.tsx +++ b/module/services/frontend/code/src/features/solution-installations/ui/solution-installations-page.tsx @@ -5,7 +5,10 @@ import { useState } from "react"; import { toast } from "sonner"; import { EmptyState } from "@/components/empty-state"; import { OrgSelector } from "@/components/org-selector"; -import type { AvailableSolution } from "@/gen/saas/accounts/v1/installations_pb"; +import { + type AvailableSolution, + InstallationStatus, +} from "@/gen/saas/accounts/v1/installations_pb"; import { useAuth } from "@/lib/auth"; import { mayKeepRetainedRows, @@ -51,6 +54,7 @@ import { import { useGrantInstallationToTeam, useInstallSolution, + useRetryInstallation, useRevokeInstallationFromTeam, useUninstallSolution, } from "../service/mutations"; @@ -172,7 +176,9 @@ function SolutionInstallationsForOrganization({ orgId }: { orgId: string }) { { targetId: installing.targetId, values }, { onSuccess: () => { - toast.success(`${installing.routeAlias} installed`); + toast.success( + `${installing.routeAlias} installation pending`, + ); setInstalling(null); }, onError: (error) => @@ -293,6 +299,7 @@ function InstalledSection({ query: ReturnType; }) { const uninstall = useUninstallSolution(orgId); + const retry = useRetryInstallation(orgId); const revoke = useRevokeInstallationFromTeam(orgId); const grant = useGrantInstallationToTeam(orgId); const [confirming, setConfirming] = useState(null); @@ -310,6 +317,18 @@ function InstalledSection({ const name = (row: InstalledRow) => row.solution.kind === "value" ? row.solution.text : "this installation"; + const runRetry = (row: InstalledRow) => { + setRefusal(null); + retry.mutate(row.installationId, { + onSuccess: () => toast.success(`${name(row)} installation pending`), + onError: (error) => + setRefusal({ + title: `The host refused to retry ${name(row)}`, + reason: hostRefusal(error), + }), + }); + }; + const runUninstall = (row: InstalledRow) => { setRefusal(null); uninstall.mutate(row.installationId, { @@ -389,6 +408,7 @@ function InstalledSection({ Solution Revision + Status Health Teams Installed @@ -405,25 +425,56 @@ function InstalledSection({ - + + {row.failureReason && ( +

+ {row.failureReason} +

+ )} + + + {row.status === InstallationStatus.ACTIVE ? ( + + ) : ( + + Unavailable + + )} - { - grant.reset(); - setGranting(row); - }} - onRevoke={(team) => runRevoke(row, team)} - /> + {row.status === InstallationStatus.ACTIVE ? ( + { + grant.reset(); + setGranting(row); + }} + onRevoke={(team) => runRevoke(row, team)} + /> + ) : ( + + Available after activation + + )} {formatDate(row.installedAt)} + {row.status === InstallationStatus.FAILED && ( + + )}