diff --git a/.claude/skills/run-the-starter-locally/SKILL.md b/.claude/skills/run-the-starter-locally/SKILL.md index c0618d62a..add8f24a5 100644 --- a/.claude/skills/run-the-starter-locally/SKILL.md +++ b/.claude/skills/run-the-starter-locally/SKILL.md @@ -5,8 +5,8 @@ description: Boot this module's whole service graph on a developer machine — w # Running the starter locally -The graph is vault → store → cache → telemetry → accounts → auth-gateway → -frontend, plus marketing. One command boots all of it with a fake-auth fixture: +The graph is vault → store → cache → accounts → auth-gateway → frontend, plus +marketing. One command boots all of it with a fake-auth fixture: ```bash codefly run service --fixture dev-admin @@ -23,6 +23,14 @@ codefly run service --fixture dev-admin - No TTY (CI, pipes, MCP) auto-enables `--headless`. - Docker must be running. `codefly doctor` checks prerequisites; `codefly clear` reaps stray processes and containers between runs. +- The module ships no collector. Standard OpenTelemetry process variables + select export: `OTEL_EXPORTER_OTLP_ENDPOINT` names the collector, with + `OTEL_SERVICE_NAME` and `OTEL_RESOURCE_ATTRIBUTES` supplying identity. + CLIs implementing [CLI #1097](https://github.com/codefly-dev/cli/pull/1097) + deliver these for every service through an environment's optional `telemetry:` + block, pointing at the HTTPS operations intake. + Without an endpoint, SDK Init enables no tracing, including for `local`. + No module configuration group is required. For a real external identity provider and the production-grade provider stack, use the `local-dogfood` environment and its configuration groups: diff --git a/.gitattributes b/.gitattributes index 2a5579c7c..522c2b612 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,14 @@ # protoc-gen-es emits one separator line after the final declaration. # Keep normal whitespace checks everywhere else. module/services/frontend/code/src/gen/** whitespace=-blank-at-eof + +# Generated artifacts changed by the telemetry service removal. +# Their owning generators remain responsible for the committed outputs. +module/deployment/generated/** linguist-generated=true +module/contracts/runnables/** linguist-generated=true +module/services/telemetry/code/pkg/gen/** linguist-generated=true + +# Build recipes emitted by the pinned Go service agent. +module/services/accounts/builder/** linguist-generated=true +module/services/auth-gateway/builder/** linguist-generated=true +module/services/policy-log/builder/** linguist-generated=true diff --git a/.gitignore b/.gitignore index ab0e56701..3391b2285 100644 --- a/.gitignore +++ b/.gitignore @@ -46,7 +46,6 @@ module/services/*/nix/ /configurations/local-dogfood/error-tracking.env /configurations/local-dogfood/error-tracking.secret.env /configurations/local-dogfood/observability.env -/configurations/local-dogfood/observability.secret.env /configurations/local-dogfood/abuse-protection.env /configurations/local-dogfood/abuse-protection.secret.env /configurations/local-dogfood/legal.env diff --git a/AGENTS.md b/AGENTS.md index 70dc5cf6f..8aa54ade4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,7 +11,7 @@ in `.claude/skills/`. See [Where the depth lives](#where-the-depth-lives). `saas-starter` is a Codefly **module**: a multi-tenant SaaS backend (three-layer authorization, Postgres RLS, RBAC, impersonation, audit), an authenticated Next.js product, a separately deployable marketing site, and the -supporting cache/store/vault/telemetry services. It is published as an +supporting cache/store/vault services. It is published as an immutable module package that downstream workspaces **compose** (never fork). - Architecture, service graph, and capability ownership: [MODULE.md](./MODULE.md) @@ -173,7 +173,7 @@ Depth sits with the code that owns it: | File | Covers | | --- | --- | -| [module/AGENTS.md](./module/AGENTS.md) | the shipped tree: generated vs authored files, the six Go modules and how to test each, configuration groups | +| [module/AGENTS.md](./module/AGENTS.md) | the shipped tree: generated vs authored files, the Go modules and how to test each, configuration groups | | [module/deployment/AGENTS.md](./module/deployment/AGENTS.md) | the manifests as the model, `spec.deployment`, and moving an agent version | | [module/services/auth-gateway/AGENTS.md](./module/services/auth-gateway/AGENTS.md) | upstream registration, the composed-module REST prefix, the credentials both take | | [module/services/accounts/AGENTS.md](./module/services/accounts/AGENTS.md) | the durable registry record, module principals, minting a module Work Context, mesh reachability | @@ -200,7 +200,7 @@ Skills in `.claude/skills/`, loaded when the task calls for them: such a list with a count: nothing enforces one, so two changes that each add a step and each bump the same number merge cleanly into a total that is silently wrong. -- Go suites live in six independent Go modules with no `go.work` +- Go suites live in independent Go modules with no `go.work` ([module/AGENTS.md](./module/AGENTS.md#go-suites)). - Vulnerability policy: the complete audit runs non-blocking so vendor-image findings stay in the evidence report, while a separate fail-closed step enforces diff --git a/CLAIM_INVENTORY.md b/CLAIM_INVENTORY.md index 25090924f..a82e67e12 100644 --- a/CLAIM_INVENTORY.md +++ b/CLAIM_INVENTORY.md @@ -77,7 +77,7 @@ must not be inferred from a row here. See | The documented repository-specific gate list is the enforced one | RELEASE_GATES.md § Repository-specific gates; AGENTS.md | `release-gates.test.mjs` compares the table and both prose counts against `REQUIRED_GATES` | implemented | #541 | | The complete audit runs `--fail-on-vuln=false`; a separate step enforces first-party findings fail-closed | RELEASE_GATES.md § Vulnerability policy | `.github/workflows/ci.yml` `codefly-supply-chain` | configured | Codefly + this repo | | No artifact-writing job runs unless every mandatory gate actually succeeded | RELEASE_GATES.md § Publication gating | `scripts/ci/release-gates.mjs check` + `decide`, `release-gates.test.mjs` | implemented (#535) | this repo | -| Root `go test ./...` does not cover nested service modules | RELEASE_GATES.md, AGENTS.md | six independent `go.mod` files; there is no `go.work` | implemented (documented) | #541 | +| Root `go test ./...` does not cover nested service modules | RELEASE_GATES.md, AGENTS.md | independent `go.mod` files (the root, `module/tools`, one per Go service); there is no `go.work` | implemented (documented) | #541 | | `main` here is the only default branch in the organization merging through a queue, and `secure-saas-platform` the only other requiring any check | RELEASE_GATES.md § The fleet beyond this repository | `scripts/ci/release-gates.mjs fleet` — operator-run: reading branch configuration needs `Administration: read`, which no workflow token can hold, so no gate can carry this | reviewed narrative | #617 | ## Public registry contract diff --git a/DOGFOODING.md b/DOGFOODING.md index 5869f606d..a7b4cf83e 100644 --- a/DOGFOODING.md +++ b/DOGFOODING.md @@ -24,7 +24,7 @@ cd ~/Development/deus/codefly/module-saas-starter codefly run service --fixture dev-admin ``` -Expected graph: `vault + store + cache + telemetry → accounts → +Expected graph: `vault + store + cache → accounts → auth-gateway → frontend` (plus marketing). Independent services may start concurrently. If any service hangs at "waiting for ready", check that step's `--debug` output. @@ -214,12 +214,16 @@ Admin Portal integration; `IDENTITY_CLIENT_ID` and Configure the remaining adapters as Codefly configuration groups, then start the same Codefly-managed graph. For each of `email` (Resend), -`product-analytics` (PostHog), `error-tracking` (Sentry), `observability` and +`product-analytics` (PostHog), `error-tracking` (Sentry) and `abuse-protection` (Turnstile), copy `configurations/local-dogfood/.env.example` and `.secret.env.example` to their Git-ignored names and fill them in ([LOCAL_DOGFOODING.md](./LOCAL_DOGFOODING.md#configure-providers-through-codefly)). -Keep `observability.env.example` as it is for local (`debug`) telemetry. +Operational telemetry has no configuration group. Use the environment's optional +`telemetry:` block with the HTTPS operations intake. +[CLI #1097](https://github.com/codefly-dev/cli/pull/1097) delivers the standard +endpoint, service identity and resource variables to every service. +With no endpoint, no traces or metrics are exported, including native local runs. Turnstile's published test keys drive the three abuse-protection cases: | Case | `NEXT_PUBLIC_TURNSTILE_SITE_KEY` | `TURNSTILE_SECRET_KEY` | @@ -239,7 +243,7 @@ codefly run service --env local-dogfood - [ ] Grant analytics consent, navigate through onboarding, and verify bounded browser plus durable backend events in PostHog. - [ ] Withdraw analytics consent or log out. Browser capture stops immediately and identity resets. - [ ] Trigger controlled browser and backend errors. Sentry correlates release and environment without creating performance transactions. -- [ ] With the `debug` exporter, exercise login/onboarding and observe trace/metric/log summaries from the in-graph telemetry service. +- [ ] With `OTEL_EXPORTER_OTLP_ENDPOINT` unset, exercise login/onboarding and observe stdout logs. With a reachable collector configured, confirm trace and metric delivery. An invalid exporter setting must be logged without preventing the service from starting. - [ ] Switch Turnstile to the **fail** keys. Registration and waitlist submission fail without database writes. - [ ] Switch Turnstile to the **replay** keys. The first deterministic verification follows Cloudflare's fixture behavior; replay rejection leaves state unchanged. diff --git a/LOCAL_DOGFOODING.md b/LOCAL_DOGFOODING.md index d2c758ccb..4dbec8d8b 100644 --- a/LOCAL_DOGFOODING.md +++ b/LOCAL_DOGFOODING.md @@ -11,7 +11,7 @@ Two Codefly inputs are intentionally independent: | Input | Authority | Purpose | |---|---|---| -| `--env local-dogfood` | `configurations/local-dogfood/*` | Select independently configured identity, billing, email, analytics, error, telemetry, and abuse adapters | +| `--env local-dogfood` | `configurations/local-dogfood/*` | Select independently configured identity, billing, email, analytics, error, and abuse adapters | | optional `--fixture ` | `module/fixtures/.yaml` | Seed a useful starting product state | A fixture never changes the identity provider. The ordinary `local` @@ -33,12 +33,11 @@ production/background fallback; a local port never belongs in product config. ## Configure providers through Codefly WorkOS is the identity adapter. The production-grade dogfood stack also -includes Stripe, Resend, PostHog, Sentry, the in-graph OpenTelemetry gateway, -and Cloudflare Turnstile. Each one is a Codefly configuration group on this +includes Stripe, Resend, PostHog, Sentry, and Cloudflare Turnstile. Each one is a Codefly configuration group on this profile: copy the committed `configurations/local-dogfood/.env.example` (and `.secret.env.example`, where there is one) to the Git-ignored `.env` / `.secret.env`, and fill in the values. Every group is -optional except `observability` and `email` — see the notes below. Stripe setup belongs to +optional except `email` — see the notes below. Stripe setup belongs to the [`codefly-dev/provider-stripe`](https://github.com/codefly-dev/provider-stripe) plugin, which validates the account, observes the webhook and projects the `billing` group. @@ -50,19 +49,17 @@ plugin, which validates the account, observes the webhook and projects the | `email` | Resend | | `product-analytics` | PostHog | | `error-tracking` | Sentry | -| `observability` | the in-graph OpenTelemetry gateway | | `abuse-protection` | Cloudflare Turnstile | Validate the result with `codefly doctor workspace --env local-dogfood`. -`observability` is the one group this profile cannot skip. The `telemetry` -service declares it as a workspace-configuration-dependency, and on -`local-dogfood` the real `observability.env` / `observability.secret.env` are -Git-ignored, so there is no committed default to fall back on. Codefly skips a -declared group whose files are absent rather than failing, and the collector -refuses to start without an explicit `OBSERVABILITY_EXPORTER` — it will not -silently downgrade itself to `debug` and drop every span. Copy -`observability.env.example` as it is to keep telemetry local. +Operational telemetry uses standard process variables rather than a configuration +group. With no `OTEL_EXPORTER_OTLP_ENDPOINT`, the SDK exports no traces in any +environment, and the services build no metrics exporter. To export, use the +environment's optional `telemetry:` block with the HTTPS operations intake. +[CLI #1097](https://github.com/codefly-dev/cli/pull/1097) delivers standard OTEL +variables to every service. SDK exporter failures are +logged without stopping the service; logs always go to stdout. `email` is required because this profile is not the `local` environment, and accounts refuses to start there without an explicit `EMAIL_PROVIDER`: `resend` diff --git a/MODULE.md b/MODULE.md index ead05a026..456abb2b1 100644 --- a/MODULE.md +++ b/MODULE.md @@ -100,7 +100,7 @@ capability; it does not claim that every planned provider is released yet. | Product analytics | PostHog | The `product-analytics@1` browser/server APIs may capture registered events and perform consent-gated identify, alias, organization grouping, and privacy suppression. An event may carry a variant already evaluated by Unleash for experiment analysis. | Flag definition or evaluation, exception capture, and traces, metrics, or logs | | Session replay | PostHog | A browser-only recorder may start after separate replay consent and redaction policy resolve to allow. It has no server SDK or implicit analytics-consent fallback. | Flags, errors, and APM signals | | Error tracking | Sentry | Browser/server exception events, release/environment tags, source-map upload, and the error-issue workflow. Provisioning credentials remain build/provider-only. | Performance transactions, profiles, replay, logs, metrics, and feature flags | -| APM traces, metrics, and logs | SigNoz | Applications use OpenTelemetry SDKs and standard OTLP configuration through the in-graph collector. SigNoz receives those signals as the OTLP backend. | Product analytics, replay, feature flags, and the Sentry error-issue contract | +| APM traces, metrics, and logs | SigNoz | Applications use OpenTelemetry SDKs and export traces and metrics over OTLP/gRPC to the HTTPS operations intake, whose address the environment's `telemetry:` declaration delivers as the standard `OTEL_EXPORTER_OTLP_ENDPOINT` variable (or delivers nothing, and then nothing is exported); logs go to stdout. SigNoz receives those signals as the OTLP backend. | Product analytics, replay, feature flags, and the Sentry error-issue contract | Provider manifests are allowlists. `provider-posthog` may project only `product-analytics@1`; its browser initialization must disable flag remote @@ -111,8 +111,9 @@ Sentry trace sampling at zero and installs no Sentry tracing integration. `provider-signoz`, if API qualification succeeds, may manage exactly owned dashboards and alerts but projects no application runtime configuration. In particular, dashboard provisioning is not an application telemetry contract: -application OTLP endpoints remain instrumentation configuration resolved -through the telemetry service. +the application OTLP endpoint remains instrumentation configuration, delivered +as the standard `OTEL_EXPORTER_OTLP_ENDPOINT` variable through the +environment's optional `telemetry:` declaration. The starter keeps the capability configurations independent. Selecting PostHog never selects flags, errors, or observability; selecting Sentry never @@ -687,7 +688,7 @@ names the source repository and a package version rather than a path: modules: - name: saas-starter source: codefly-dev/module-saas-starter - version: "0.1.0" + version: "0.2.0" ``` Resolution fails closed, so the workspace must also say who it trusts to have diff --git a/README.md b/README.md index 6bb76e61f..df0633c3e 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ not this list, is the enumeration. Beyond the API, the module ships the authenticated product frontend (whose routes and navigation modules contribute to), a separately deployable public -marketing site, and the cache, store, vault, telemetry, and gateway services +marketing site, and the cache, store, vault, and gateway services the above runs on. ## What it does not own diff --git a/RELEASE_GATES.md b/RELEASE_GATES.md index 77740124f..0ce3e1f79 100644 --- a/RELEASE_GATES.md +++ b/RELEASE_GATES.md @@ -193,7 +193,7 @@ Agent-generated `module/services/*/builder/Dockerfile` recipes must **not** be configured in Dependabot: agents replace them before building. The coverage gate rejects those entries. `scripts/ci/build-images.json` records the expected build images and the owning agent source for Go, -Next.js and the Postgres migration builder (including telemetry's generated Go +Next.js and the Postgres migration builder (including the Go services' generated recipe, which is not checked in). The weekly `build-images.yml` monitor compares registry digests for the effective @@ -448,7 +448,7 @@ reverse, fails the `release-contract` job. | `scoped-pools` | the authority of the pools the store's logins reach: the scoped reader, the scoped writer and the request pool, judged against a native PostgreSQL the suite provisions and widens one grant at a time | `python3 scripts/qualify-scoped-pools.py --postgres-bin "$(pg_config --bindir)"`, which runs `go test -race ./qualification/scopedpools` and fails unless every declared test in that package ran and passed | Two of those `go test ./...` invocations are worth reading carefully. This -repository holds **six independent Go modules** — root, `module/tools`, and one +repository holds **independent Go modules** — root, `module/tools`, and one per Go service — with no `go.work`, so `go test ./...` covers only the module it is run in. The `sdk-boundary` invocation runs in the root module (the module agent, the host, the generated reference composition) and the `module-package` diff --git a/configurations/local-dogfood/observability.env.example b/configurations/local-dogfood/observability.env.example deleted file mode 100644 index 2ad133ad0..000000000 --- a/configurations/local-dogfood/observability.env.example +++ /dev/null @@ -1,6 +0,0 @@ -# Copy to observability.env and fill it in. SigNoz is the -# designated OTLP backend; this remains application instrumentation config. -# "debug" keeps telemetry local; "otlphttp" forwards through the collector. -OBSERVABILITY_EXPORTER=debug -OTEL_EXPORTER_OTLP_ENDPOINT= -OTEL_EXPORTER_OTLP_HEADERS= diff --git a/configurations/local-dogfood/observability.secret.env.example b/configurations/local-dogfood/observability.secret.env.example deleted file mode 100644 index 87476f787..000000000 --- a/configurations/local-dogfood/observability.secret.env.example +++ /dev/null @@ -1,3 +0,0 @@ -# Copy to observability.secret.env. This file is ignored by Git. -# Example: Authorization=Basic BASE64_INSTANCE_ID_API_TOKEN -OTEL_EXPORTER_OTLP_HEADERS= diff --git a/configurations/local/observability.env b/configurations/local/observability.env deleted file mode 120000 index 717462ff9..000000000 --- a/configurations/local/observability.env +++ /dev/null @@ -1 +0,0 @@ -../../module/configurations/local/observability.env \ No newline at end of file diff --git a/gitops.go b/gitops.go index f1ae8ccd9..86f568148 100644 --- a/gitops.go +++ b/gitops.go @@ -263,10 +263,9 @@ type topologyModule struct { } type topologyInterface struct { - Service string `yaml:"service"` - Endpoint string `yaml:"endpoint"` - Visibility string `yaml:"visibility"` - AllowModules []string `yaml:"allow-modules,omitempty"` + Service string `yaml:"service"` + Endpoint string `yaml:"endpoint"` + Visibility string `yaml:"visibility"` } type topologyService struct { @@ -2474,7 +2473,6 @@ func bootstrapJobIngressPolicy(namespace string, labels map[string]string, servi "ingress": []any{map[string]any{ "from": []any{map[string]any{"podSelector": map[string]any{"matchLabels": map[string]string{ "codefly.dev/bootstrap-service": service, - "job-name": service, }}}}, "ports": networkPorts(ports), }}, @@ -2490,7 +2488,6 @@ func bootstrapJobEgressPolicy(namespace string, labels map[string]string, servic Spec: map[string]any{ "podSelector": map[string]any{"matchLabels": map[string]string{ "codefly.dev/bootstrap-service": service, - "job-name": service, }}, "policyTypes": []string{"Egress"}, "egress": []any{map[string]any{ diff --git a/gitops_test.go b/gitops_test.go index b6e380362..94c6571b1 100644 --- a/gitops_test.go +++ b/gitops_test.go @@ -2365,9 +2365,6 @@ func serviceManifestFixture(name, endpoint, api, visibility string, port int, to } if visibility != "private" { manifest.WriteString(" visibility: " + visibility + "\n") - if visibility == "internal" { - manifest.WriteString(" allow-modules: [\"*\"]\n") - } } manifest.WriteString("spec:\n") manifest.WriteString(spec) diff --git a/module/AGENTS.md b/module/AGENTS.md index 8280463cb..4a6932097 100644 --- a/module/AGENTS.md +++ b/module/AGENTS.md @@ -60,11 +60,13 @@ adding a REST-enabled RPC means editing it. Do not go looking for a generator. ## Go suites Every Go service here carries **its own `go.mod`** — `accounts`, `auth-gateway`, -`store`, `telemetry` — and `tools` is another, with **no `go.work`** tying them -together. So `go test ./...` only ever covers the module you run it in, and a -service's suite is outside whatever you run from the tree root. The canonical -repository adds its own root module (the module agent, the host, and the generated -reference composition), for six in total; a consumer's root module is its own. +`store` and `policy-log` — and `tools` is another, with **no `go.work`** tying them together. So +`go test ./...` only ever covers the module you run it in, and a service's suite +is outside whatever you run from the tree root. The canonical repository adds its +own root module (the module agent, the host, and the generated reference +composition); a consumer's root module is its own. +`libraries/source-read-sdk/go` and `qualification/module-authority` are two more +modules, outside this tree, that CI exercises in steps of their own. To exercise a service, run its suite from its own directory — its DB-backed suites need Codefly and Docker — or let `codefly ci run` do it. @@ -83,8 +85,8 @@ which is only for older CLIs and reuses fixed local database directories across otherwise independent runs and migration revisions. Core is pinned per Go module. Accounts and auth-gateway currently select -candidate presence/delivery revisions for the lifecycle PR; the gateway also -selects the SDK candidate providing `ResolveDeliveredNetworkInstance`. +candidate presence/delivery revisions for the lifecycle PR. Released SDK +v0.4.1 includes the lifecycle APIs and owns tracing startup and shutdown. Their `go.mod` files are the exact source pins. The other services and tools retain their own released pins. These are development candidates, not release qualification: never ship a host release on an untagged pseudo-version. @@ -103,15 +105,14 @@ Postgres 0.0.144 and Vault 0.0.42, which include their owners' outcome fixes. An empty legacy response is not evidence that tests passed; do not bypass that check or disable the service's test phase to accept it. -Lifecycle CI selects the exact CLI source candidate in -`scripts/ci/install-codefly.sh` for every planner/executor job. It recognises -the CLI-owned `artifact-cache` group without an authored default. This uses -the canonical versioned Go install entrypoint; it does not require a CLI -release for development qualification. The installer refuses release tags -while the source candidate is selected. Restore a qualified released pin -before publishing the host. +CI selects released CLI v0.1.197 in `scripts/ci/install-codefly.sh` for every +planner/executor job. This release includes the merged lifecycle/artifact-cache +implementation and environment-wide standard OTEL delivery from the optional +`telemetry:` declaration. The historical source candidate remains available only +for explicit development qualification; release tags refuse source candidates. -Outside the gateway's delivery candidate, services use released SDK v0.4.0. +Accounts and auth-gateway use released SDK v0.4.1; other services retain their +own released pins. Their listeners resolve allocated endpoints through `ResolveNetworkInstance`, which selects the delivered or local endpoint from the runtime environment; the removed `WithDefaultNetwork` switch diff --git a/module/DEPLOYMENT_TOPOLOGY.md b/module/DEPLOYMENT_TOPOLOGY.md index 3ec5dab26..5e700a935 100644 --- a/module/DEPLOYMENT_TOPOLOGY.md +++ b/module/DEPLOYMENT_TOPOLOGY.md @@ -44,7 +44,6 @@ depend on the database whose authority it witnesses. | --- | --- | --- | | `accounts` | `cache/read`, `cache/write` | TCP 6379 | | `accounts` | `store/tcp` | TCP 5432 | -| `accounts` | `telemetry/grpc` | Codefly-assigned OTLP gRPC port | | `accounts` | `vault/http` | TCP 8200 | | `auth-gateway` | `accounts/connect`, `accounts/rest`, `accounts/grpc` | TCP 8080, 9090 | | `auth-gateway` | `cache/write` | TCP 6379 | @@ -59,26 +58,50 @@ own that the cell's mesh admits from composed workloads alone (an delivers). Both ports are allocated by Codefly. Solution upstreams are reached over plain HTTP at the address the Codefly SDK resolves for the endpoint their applied presence declares; no workload serves or presents an SVID. Istio routes apex/`www`/docs hosts to `marketing/http` and `app` to -`frontend/http`. Accounts, frontend, marketing, and telemetry may reach -public IP space only over TCP 443. The public rules exclude private, loopback, +`frontend/http`. Accounts, frontend, and marketing may reach +public IP space only over TCP 443; `auth-gateway` has no public egress. The public rules exclude private, loopback, link-local, metadata, documentation, benchmark, multicast, and other special-purpose IPv4/IPv6 ranges. Temporal's gRPC frontend and HTTP UI remain module-visible without a public ingress route. ## Network-policy model -The topology-policy golden contains 25 `NetworkPolicy` resources: +Authored endpoints and module-interface exports declare `visibility: internal` +without `allow-modules`. Core derives the modules that can reach an internal +endpoint from the consumers' declared runtime dependencies. A target never +lists its consumers. Interface visibility defaults to internal; a service +endpoint still defaults to private. The catalog retains its `MODULE` enum for +internal reach, while network and mesh policies admit only declared dependency +edges. Public frontend and marketing endpoints explicitly declare +`exposure: public`, preserving their existing public addresses. + +Promotion requires a released CLI with Core's consumer-derived allow-list +support. The module agent, +accounts, auth-gateway, store migrator, and module tools require stable Core 0.14 +or later to load these declarations and derive consumer access. The runtime SDK +also loads the module when resolving local endpoints or secrets. Core 0.13 refuses +internal interface exports without an authored list. The CI pin at CLI 0.1.171 +still loads the module with its older Core 0.7, but does not derive cross-module +access: an internal endpoint with no authored list denies other modules. It is +not a compatible promotion driver for this declaration model. + +The topology-policy golden contains 19 `NetworkPolicy` resources: - one namespace-wide ingress/egress default deny; - DNS and Istio control-plane egress for all injected workloads; - Istio ingress only to the public frontend and marketing HTTP ports; - target ingress and caller egress policies for every declared dependency; -- HTTPS public egress only for accounts, frontend, marketing, and telemetry. +- HTTPS public egress only for accounts, frontend, and marketing. There is no `allow-intra-namespace` rule. Adding a service dependency or port requires changing the topology binding and reviewing both generated directions of the edge. +Store bootstrap ingress and egress select the agent's stable +`codefly.dev/bootstrap-service: store` pod label. Postgres names the immutable +migration Job with a digest of its rendered pod template, so its Kubernetes +`job-name` changes with the image or inputs and cannot be a fixed selector. + Pod selectors use the `app: ` labels emitted by the pinned Codefly agents. Services whose agent uses a different Kubernetes identity declare its Service name and app label in the topology. Endpoint ports are explicit @@ -88,10 +111,20 @@ topology binding in the same change; pinned deployment schema/render validation is tracked by `P1-CI-004`. The generated Codefly dependency declarations contain exact endpoint -references. Accounts resolves `telemetry/grpc` through the SDK and passes that -exact Codefly-owned address to both tracing and metrics; no product -configuration owns a local collector port. The generated NetworkPolicy remains -the hard endpoint/port enforcement boundary. +references. The module owns no collector: accounts and auth-gateway export traces +and metrics to the HTTPS operations intake. The environment's optional +`telemetry:` declaration lets the CLI deliver the standard OTEL variables to +every service. Without an endpoint the services export nothing. + +The generated NetworkPolicy remains the endpoint/port enforcement boundary for +services the module declares. Auth-gateway has no direct public TCP 443 egress. +The selected ambient waypoint route uses the existing +[`allow-ambient-hbone-transport` policy](../gitops.go#L2255), which admits TCP +15008 transport for every pod without a destination restriction. The route does +not require a new module collector-egress field or a broad public-egress rule. +Live CNI handling of HBONE, waypoint routing, DNS, TLS and actual telemetry +export remain unqualified; successful new-environment export is a rollout +prerequisite. The AWS overlay replaces stateful services with managed dependencies. A pod-selector rule cannot authorize an RDS, ElastiCache, external Vault, or S3 @@ -301,7 +334,7 @@ exports, and missing descriptor-required accounts protocols. Parity tests build every artifact twice, compare all checked-in outputs, parse the generated files through Codefly's resource model, and strictly inspect all -25 NetworkPolicy golden documents. After the module generator creates the +19 NetworkPolicy golden documents. After the module generator creates the consumer-owned GitOps tree, render an environment with: ```sh diff --git a/module/FEATURES.md b/module/FEATURES.md index f14e5cac2..ad5fb83bd 100644 --- a/module/FEATURES.md +++ b/module/FEATURES.md @@ -49,8 +49,8 @@ the auth-gateway in front so this code path isn't reached. | Test infra | Playwright e2e against the real stack via `withDependencies` | Everything is orchestrated by Codefly: `codefly run service --fixture -dev-admin` resolves the module's service graph and brings up all eight -services—Postgres + Vault + Redis + object storage + telemetry + Accounts + +dev-admin` resolves the module's service graph and brings up its +services—Postgres + Vault + Redis + object storage + Accounts + the private auth gateway + frontend—with seed data in one command. Marketing remains a separate runtime and may be started, deployed, rolled back, or disabled without changing the authenticated product. @@ -558,8 +558,8 @@ extraction contracts. |----------------|--------|-----------------------------------------------------------------| | Structured logs| ✅ | `wool` everywhere; user/org/action context auto-attached | | Audit trail | ✅ | Separate from app logs; queryable | -| Metrics | ✅ | Job-worker OTel instruments, durable queue projections, and an in-graph OTLP gateway | -| Tracing | ✅ | Accounts resolves the Codefly collector endpoint and exports OTLP for the designated SigNoz backend | +| Metrics | ✅ | Job-worker OTel instruments, durable queue projections, and Go runtime and request metrics pushed over OTLP to the HTTPS operations intake | +| Tracing | ✅ | Accounts and auth-gateway export OTLP to the HTTPS operations intake, whose address the platform delivers as `OTEL_EXPORTER_OTLP_ENDPOINT`, for the designated SigNoz backend | | Error tracking | ✅ | Explicit fail-closed Sentry mode for server/browser errors; trace sampling is fixed at zero | | Dashboards | 🟡 | Versioned provider-neutral business dashboard pack; [SigNoz provisioning remains unsupported pending a pinned service qualification](SIGNOZ_PROVISIONING.md) | @@ -756,20 +756,38 @@ Environment variables consumed by the api: | `POSTHOG_API_HOST` | Separate PostHog management/deletion origin | | `ERROR_TRACKING_MODE` | Explicit `disabled` or `sentry`; rejects partial config | | `SENTRY_DSN` | Server Sentry DSN, required in Sentry mode | -| `OBSERVABILITY_EXPORTER` | Required in-graph collector output: `debug` or `otlphttp`; the collector refuses to start when it is unset | -| `OTEL_EXPORTER_OTLP_ENDPOINT` | External OTLP/HTTP destination used only by the collector | -| `OTEL_EXPORTER_OTLP_HEADERS` | Secret external collector headers | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | The OTLP/gRPC target in the process environment: the operations intake's HTTPS OTLP/gRPC address, delivered from the environment's optional `telemetry:` declaration to every service. Unset means nothing is exported over OTLP; `https://` uses TLS, while `http://` is available for local plaintext receivers | | `ABUSE_PROTECTION_MODE` | Explicit `disabled` or `turnstile` | | `TURNSTILE_SECRET_KEY` | Server-only Siteverify credential | | `TURNSTILE_ALLOWED_HOSTNAMES` | Exact accepted Turnstile response hostnames | | `CODEFLY__FIXTURE` | Loads fixture YAML (e.g. `dev-admin`); FE login picker too | -When `OTEL_EXPORTER_OTLP_ENDPOINT` is configured, the accounts service and the -auth-gateway export unsampled request and Go runtime metrics through the in-graph -OpenTelemetry collector. The auth-gateway covers both its HTTP gateway and gRPC ext_authz -authorization service. Prometheus can alternatively scrape `/metrics` on each -service's private REST endpoint; neither route is a module or public interface -endpoint. +The SDK's `Init` enables tracing when `OTEL_EXPORTER_OTLP_ENDPOINT` is +present, and `ShutdownTelemetry` flushes traces on service shutdown. Services +with no endpoint export nothing, including native `local` runs. Invalid exporter +configuration is logged without stopping the service. Accounts and auth-gateway +also install an OTLP metrics provider for unsampled request and Go runtime +metrics; no scrape endpoint is served. + +Delivery uses OpenTelemetry process variables, not a module configuration group. +The environment's optional `telemetry:` declaration is the platform delivery +contract. [CLI #1097](https://github.com/codefly-dev/cli/pull/1097) supplies the +standard endpoint, service identity and resource variables to every service; +services that do not instrument telemetry can ignore them. The endpoint names +the HTTPS operations intake, rather than a collector deployed by this module. +`OTEL_SERVICE_NAME` selects trace identity, falling back to the SDK's +`module/service` identity. +Metrics also consume `OTEL_RESOURCE_ATTRIBUTES`. Core currently forwards only +`service.name` and `library.name` to traces, so arbitrary resource attributes +are not yet attached to trace exports. + +| Build or reuse | Verified implementation | Decision | +| --- | --- | --- | +| Environment delivery (user choice, 2026-10-21) | Merged [CLI #1097](https://github.com/codefly-dev/cli/pull/1097) environment telemetry injection; operations intake owned by infrastructure | Reuse every-service standard OTEL delivery; remove the overlapping scoped in-cell carrier. Released CLI v0.1.197 includes both this delivery and the squash-merged lifecycle implementation. | +| Tracing lifecycle and identity | Released SDK v0.4.1 [Init / ShutdownTelemetry](https://github.com/codefly-dev/sdk-go/blob/v0.4.1/codefly.go#L66), using Core wool/otel | Reuse SDK lifecycle; remove per-service tracing bootstrap and its duplicate tests. | +| Endpoint declaration validation | Core [Endpoint](https://github.com/codefly-dev/core/blob/v0.18.0/resources/endpoint.go#L134) / [InterfaceEndpoint](https://github.com/codefly-dev/core/blob/v0.18.0/resources/module.go#L69) YAML decoders | Reuse the Core decoders before projecting deployment fields. | +| Metrics export and runtime instruments | OpenTelemetry OTLP metrics exporter and runtime instrumentation v0.71.0; Core/SDK provide no metrics bootstrap | Keep service wiring with the standard exporter. The services remain separate Go modules; no additional shared framework is introduced. | +| GC activity instruments | Runtime v0.71.0 default instruments omit cycle count and cumulative GC pause CPU time; deprecated pause instruments measure wall time instead | Retain existing `go.gc.cycle.count` and `go.gc.pause.cpu_time` callbacks and their export assertions. | Frontend browser configuration. The server reads each value from its Codefly group per request (`identity`, `legal`, `product-features`, `product-analytics`, `error-tracking`, diff --git a/module/PRODUCT_ANALYTICS.md b/module/PRODUCT_ANALYTICS.md index 886315b33..9c5a00f72 100644 --- a/module/PRODUCT_ANALYTICS.md +++ b/module/PRODUCT_ANALYTICS.md @@ -200,9 +200,10 @@ active organization. ## Operational metrics, alerts, and recovery SigNoz is the single backend for application traces, metrics, and logs. The -application surface remains standard OpenTelemetry/OTLP through the in-graph -collector; an optional SigNoz provider may provision dashboards and alerts but -does not project an application SDK, endpoint, or credential. +application surface remains standard OpenTelemetry/OTLP to the cell's collector, +whose address the platform delivers as `OTEL_EXPORTER_OTLP_ENDPOINT`; an +optional SigNoz provider may provision dashboards and alerts but does not project +an application SDK, endpoint, or credential. Job workers emit OpenTelemetry `saas.jobs.polls`, `claimed`, `active`, `completed`, and `duration` instruments. Labels are limited to queue plus diff --git a/module/SIGNOZ_PROVISIONING.md b/module/SIGNOZ_PROVISIONING.md index f1bdb6f5f..ad6c91bca 100644 --- a/module/SIGNOZ_PROVISIONING.md +++ b/module/SIGNOZ_PROVISIONING.md @@ -11,11 +11,12 @@ deployment against which to prove the lifecycle. Creating a provider before that boundary exists would turn an untested upstream API into a claimed Codefly contract. -This decision does not affect application telemetry. Applications continue to -send OTLP to the in-graph collector endpoint resolved through Codefly; they do -not read or hard-code that address. `OTEL_EXPORTER_OTLP_ENDPOINT` remains only -the collector's external OTLP/HTTP destination. Dashboard provisioning is not -an application runtime capability. +This decision does not affect application telemetry. Applications export +OTLP/gRPC to the cell's collector, at the address the platform delivers as +`OTEL_EXPORTER_OTLP_ENDPOINT` (or nothing, when the cell has no collector for them, +and then they export nothing); they do not hard-code it, and the module ships no +collector of its own. Dashboard provisioning is not an application runtime +capability. ## Qualification result diff --git a/module/configurations/local/observability.env b/module/configurations/local/observability.env deleted file mode 100644 index de82d1d00..000000000 --- a/module/configurations/local/observability.env +++ /dev/null @@ -1,3 +0,0 @@ -OBSERVABILITY_EXPORTER=debug -OTEL_EXPORTER_OTLP_ENDPOINT= -OTEL_EXPORTER_OTLP_HEADERS= diff --git a/module/contracts/api/catalog.codefly.json b/module/contracts/api/catalog.codefly.json index 5a4135c62..905df972e 100644 --- a/module/contracts/api/catalog.codefly.json +++ b/module/contracts/api/catalog.codefly.json @@ -1,7 +1,7 @@ { "schema": "codefly/module-api-contracts/v1", "package": "codefly/saas-starter", - "version": "0.1.0", + "version": "0.2.0", "endpoints": [ { "service": "accounts", diff --git a/module/contracts/reference/module.codefly.yaml b/module/contracts/reference/module.codefly.yaml index 3cb683573..c5bb83ce7 100644 --- a/module/contracts/reference/module.codefly.yaml +++ b/module/contracts/reference/module.codefly.yaml @@ -2,7 +2,7 @@ kind: composed-module name: reference-product base: id: codefly/saas-starter - version: ">=0.1.0 <0.2.0" + version: ">=0.2.0 <0.3.0" contributions: frontend: - path: frontend diff --git a/module/contracts/runnables/accounts/connect/InvokeSourceOperation/runnable-package.json b/module/contracts/runnables/accounts/connect/InvokeSourceOperation/runnable-package.json index 6d965672e..1cd13f306 100644 --- a/module/contracts/runnables/accounts/connect/InvokeSourceOperation/runnable-package.json +++ b/module/contracts/runnables/accounts/connect/InvokeSourceOperation/runnable-package.json @@ -1 +1 @@ -{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"operation","type":"STRING"},{"name":"input_json","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"output_json","type":"STRING"},{"fields":[{"name":"effect_id","type":"STRING"},{"name":"committed_at","type":"STRING"},{"name":"status","type":"STRING"},{"name":"provider_status","type":"INTEGER"},{"name":"output_json","type":"STRING"}],"name":"receipt","optional":true,"type":"OBJECT"}]},"protocol":"codefly.runnable.service/v1"},"digest":"989d265873ecdcf658dc37c29fe5a308c51a6626f11a5849e041fbcec2e7bede","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"65536","max_output_bytes":"1048576","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-invoke-source-operation","version":"0.1.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.InvokeSourceOperationRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/InvokeSourceOperation","output_message":"saas.accounts.v1.InvokeSourceOperationResponse"}]} +{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"operation","type":"STRING"},{"name":"input_json","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"output_json","type":"STRING"},{"fields":[{"name":"effect_id","type":"STRING"},{"name":"committed_at","type":"STRING"},{"name":"status","type":"STRING"},{"name":"provider_status","type":"INTEGER"},{"name":"output_json","type":"STRING"}],"name":"receipt","optional":true,"type":"OBJECT"}]},"protocol":"codefly.runnable.service/v1"},"digest":"2b15d5c58084f594b5dfddab8afda44b9030e7a32b5cdbce6d9e8c8be621e883","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"65536","max_output_bytes":"1048576","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-invoke-source-operation","version":"0.2.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.InvokeSourceOperationRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/InvokeSourceOperation","output_message":"saas.accounts.v1.InvokeSourceOperationResponse"}]} diff --git a/module/contracts/runnables/accounts/connect/PruneSourceOperationReceipts/runnable-package.json b/module/contracts/runnables/accounts/connect/PruneSourceOperationReceipts/runnable-package.json index c3a7b9a0d..6793aa6fa 100644 --- a/module/contracts/runnables/accounts/connect/PruneSourceOperationReceipts/runnable-package.json +++ b/module/contracts/runnables/accounts/connect/PruneSourceOperationReceipts/runnable-package.json @@ -1 +1 @@ -{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"effect_id","type":"STRING"},{"name":"receipts","type":"INTEGER"},{"name":"status","type":"STRING"}]},"protocol":"codefly.runnable.service/v1"},"digest":"7d729bfe247aa63a86815162a10ae06ef15d18be4b826939b22187816987cdda","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"4096","max_output_bytes":"4096","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-prune-source-operation-receipts","version":"0.1.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.PruneSourceOperationReceiptsRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/PruneSourceOperationReceipts","output_message":"saas.accounts.v1.PruneSourceOperationReceiptsResponse"}]} +{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"effect_id","type":"STRING"},{"name":"receipts","type":"INTEGER"},{"name":"status","type":"STRING"}]},"protocol":"codefly.runnable.service/v1"},"digest":"a843f385794a3bc4fa1d864b221e1bec5482c45180d6b644514d4ccac0cd9851","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"4096","max_output_bytes":"4096","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-prune-source-operation-receipts","version":"0.2.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.PruneSourceOperationReceiptsRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/PruneSourceOperationReceipts","output_message":"saas.accounts.v1.PruneSourceOperationReceiptsResponse"}]} diff --git a/module/contracts/runnables/accounts/connect/ReconcileSource/runnable-package.json b/module/contracts/runnables/accounts/connect/ReconcileSource/runnable-package.json index e792da93c..f357d98f9 100644 --- a/module/contracts/runnables/accounts/connect/ReconcileSource/runnable-package.json +++ b/module/contracts/runnables/accounts/connect/ReconcileSource/runnable-package.json @@ -1 +1 @@ -{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"effect_id","type":"STRING"},{"name":"status","type":"STRING"},{"name":"job_id","type":"STRING"}]},"protocol":"codefly.runnable.service/v1"},"digest":"ea914c8ee5dfc307d64f492010002873d9df50ed766e21f1f8256a969468681b","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"4096","max_output_bytes":"4096","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-reconcile-source","version":"0.1.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.ReconcileSourceRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/ReconcileSource","output_message":"saas.accounts.v1.ReconcileSourceResponse"}]} +{"agent":{"kind":"SERVICE","name":"go-grpc","publisher":"codefly.dev","version":"0.1.58"},"contract":{"input":{"fields":[{"name":"org_id","type":"STRING"},{"name":"source_id","type":"STRING"},{"name":"effect_id","type":"STRING"}]},"output":{"fields":[{"name":"effect_id","type":"STRING"},{"name":"status","type":"STRING"},{"name":"job_id","type":"STRING"}]},"protocol":"codefly.runnable.service/v1"},"digest":"5782459eaa3ad22b429857eea3b0d50805eb73a2cd8a7c364a6c77d82751f13e","execution":{"cancellation":"CANCELLATION_NONE","completion":"COMPLETION_CALL","facilities":[{"kind":"SERVICE"}],"max_input_bytes":"4096","max_output_bytes":"4096","recovery":"RECOVERY_RECEIPT","timeout":"600s"},"identity":{"module":"saas-starter","name":"accounts-connect-reconcile-source","version":"0.2.0","workspace":"saas-starter-dev"},"schema":"codefly.runnable-package/v1","service_operations":[{"adaptation":"ADAPTATION_BOUNDED_JSON_V1","endpoint":"connect","input_message":"saas.accounts.v1.ReconcileSourceRequest","module":"saas-starter","name":"accounts","operation":"/saas.accounts.v1.DatasourceService/ReconcileSource","output_message":"saas.accounts.v1.ReconcileSourceResponse"}]} diff --git a/module/contracts/runnables/index.json b/module/contracts/runnables/index.json index 93e486df7..96b147583 100644 --- a/module/contracts/runnables/index.json +++ b/module/contracts/runnables/index.json @@ -5,8 +5,8 @@ "operations": [ { "name": "accounts-connect-invoke-source-operation", - "version": "0.1.0", - "digest": "989d265873ecdcf658dc37c29fe5a308c51a6626f11a5849e041fbcec2e7bede", + "version": "0.2.0", + "digest": "2b15d5c58084f594b5dfddab8afda44b9030e7a32b5cdbce6d9e8c8be621e883", "service": "accounts", "endpoint": "connect", "method": "/saas.accounts.v1.DatasourceService/InvokeSourceOperation", @@ -16,8 +16,8 @@ }, { "name": "accounts-connect-prune-source-operation-receipts", - "version": "0.1.0", - "digest": "7d729bfe247aa63a86815162a10ae06ef15d18be4b826939b22187816987cdda", + "version": "0.2.0", + "digest": "a843f385794a3bc4fa1d864b221e1bec5482c45180d6b644514d4ccac0cd9851", "service": "accounts", "endpoint": "connect", "method": "/saas.accounts.v1.DatasourceService/PruneSourceOperationReceipts", @@ -27,8 +27,8 @@ }, { "name": "accounts-connect-reconcile-source", - "version": "0.1.0", - "digest": "ea914c8ee5dfc307d64f492010002873d9df50ed766e21f1f8256a969468681b", + "version": "0.2.0", + "digest": "5782459eaa3ad22b429857eea3b0d50805eb73a2cd8a7c364a6c77d82751f13e", "service": "accounts", "endpoint": "connect", "method": "/saas.accounts.v1.DatasourceService/ReconcileSource", diff --git a/module/deployment/generated/service-topology.json b/module/deployment/generated/service-topology.json index 7e441d853..895e1cb00 100644 --- a/module/deployment/generated/service-topology.json +++ b/module/deployment/generated/service-topology.json @@ -44,12 +44,6 @@ "tcp" ] }, - { - "service": "telemetry", - "endpoints": [ - "grpc" - ] - }, { "service": "vault", "endpoints": [ @@ -92,12 +86,6 @@ "endpoints": [ "write" ] - }, - { - "service": "telemetry", - "endpoints": [ - "grpc" - ] } ] }, @@ -170,24 +158,13 @@ } ] }, - { - "name": "telemetry", - "endpoints": [ - { - "name": "grpc", - "api": "CODEFLY_API_GRPC", - "visibility": "ENDPOINT_VISIBILITY_PRIVATE", - "port": 9090 - } - ] - }, { "name": "vault", "endpoints": [ { "name": "http", "api": "CODEFLY_API_HTTP", - "visibility": "ENDPOINT_VISIBILITY_MODULE", + "visibility": "ENDPOINT_VISIBILITY_PRIVATE", "port": 8200 } ] @@ -254,12 +231,6 @@ "ports": [ 443 ] - }, - { - "service": "telemetry", - "ports": [ - 443 - ] } ] } diff --git a/module/docs/composition-v2-migration.md b/module/docs/composition-v2-migration.md index 9a376f607..e61b14c03 100644 --- a/module/docs/composition-v2-migration.md +++ b/module/docs/composition-v2-migration.md @@ -11,7 +11,7 @@ contract. The copied tree is removed only after the v1 and v2 projections produc the same service catalog, endpoint graph, frontend inventory, permissions, settings, fixtures, and runtime smoke results. -The `0.1.x` package line supports migration from Starter `0.0.32` and later. Earlier +The `0.1.x` and `0.2.x` package lines support migration from Starter `0.0.32` and later. Earlier copies must first update to a release the package line supports. The v2 descriptor points directly at contribution sources; Starter does not @@ -23,7 +23,7 @@ kind: composed-module name: my-product base: id: codefly/saas-starter - version: ">=0.1.0 <0.2.0" + version: ">=0.2.0 <0.3.0" contributions: frontend: - {path: frontend, export: productPlugin} diff --git a/module/module.codefly.yaml b/module/module.codefly.yaml index eb6579923..77349e096 100644 --- a/module/module.codefly.yaml +++ b/module/module.codefly.yaml @@ -40,5 +40,4 @@ services: - name: marketing - name: policy-log - name: store - - name: telemetry - name: vault diff --git a/module/module.package.codefly.yaml b/module/module.package.codefly.yaml index 34ea2ee34..5401faf8b 100644 --- a/module/module.package.codefly.yaml +++ b/module/module.package.codefly.yaml @@ -1,7 +1,7 @@ schema: codefly/module-package/v2 kind: module-package id: codefly/saas-starter -version: 0.1.0 +version: 0.2.0 minimum-codefly-version: ">=0.3.32 <1.0.0" artifact-roots: - contracts @@ -42,8 +42,6 @@ services: endpoints: [grpc] - name: store endpoints: [tcp] - - name: telemetry - endpoints: [grpc] - name: vault endpoints: [http] fixtures: @@ -102,6 +100,7 @@ migrations: breaking-changes: - Consumers resolve the immutable package into a disposable projection instead of editing Starter-owned files. - "Audit event types are namespaced under `saas.*` (`auth.login` -> `saas.auth.login`). The wire value of `audit_events.event_type`, the `X-Webhook-Event` header, and webhook routing keys all change; migration 116 rewrites stored history and existing subscriptions in place, so a live subscriber keeps firing, but a consumer that re-creates a subscription from a hardcoded pre-namespace name, or emits one through `ModuleEmitAuditEvent`, is now rejected. No alias layer." + - "The telemetry service and its dependency contract are removed. SDK Init exports traces when OTEL_EXPORTER_OTLP_ENDPOINT is delivered; no endpoint means no tracing, including native local runs. Exporter initialization failures are logged without stopping the service. Accounts and auth-gateway push metrics to the same endpoint and no longer serve /metrics. Telemetry uses standard process variables, not the removed observability configuration group. Deliver standard OTEL variables to every service through the environment telemetry declaration using a compatible CLI. Existing consumers of the telemetry service must migrate to the HTTPS operations intake." reserved-namespaces: - "@codefly/saas-" - "@codefly-dev/saas-" diff --git a/module/services/accounts/TESTING.md b/module/services/accounts/TESTING.md index 3456af2ab..bd83c7f5a 100644 --- a/module/services/accounts/TESTING.md +++ b/module/services/accounts/TESTING.md @@ -23,8 +23,8 @@ The default `go test ./...` and `codefly ci run` retain all existing default tes The pure command is an additional fast check, not a replacement release gate. Database targets also retain their package's pure tests. They serialize setup, execution and teardown through the existing package lock. Business tests use -Vault for hashing and encrypted credentials. None of them uses telemetry, and -only the Redis target starts the `cache` service: it proves the org-membership +Vault for hashing and encrypted credentials. Only the Redis target starts the +`cache` service: it proves the org-membership cache stack across two instances and the Redis-held revocation, nonce and rate-limit state against the real server. Dependency exclusions prevent every other service from building or starting. diff --git a/module/services/accounts/code/go.mod b/module/services/accounts/code/go.mod index eb56fd445..a4ddd44b0 100644 --- a/module/services/accounts/code/go.mod +++ b/module/services/accounts/code/go.mod @@ -20,7 +20,6 @@ require ( github.com/google/uuid v1.6.0 github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 github.com/jackc/pgx/v5 v5.10.0 - github.com/prometheus/client_golang v1.24.1 github.com/redis/go-redis/v9 v9.22.0 github.com/rs/cors v1.11.1 github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 @@ -29,7 +28,6 @@ require ( go.opentelemetry.io/contrib/instrumentation/runtime v0.71.0 go.opentelemetry.io/otel v1.47.0 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.46.0 - go.opentelemetry.io/otel/exporters/prometheus v0.68.0 go.opentelemetry.io/otel/metric v1.47.0 go.opentelemetry.io/otel/sdk v1.47.0 go.opentelemetry.io/otel/sdk/metric v1.47.0 @@ -56,7 +54,6 @@ require ( github.com/Masterminds/semver/v3 v3.5.0 // indirect github.com/VividCortex/ewma v1.2.0 // indirect github.com/antlr4-go/antlr/v4 v4.13.1 // indirect - github.com/beorn7/perks v1.0.1 // indirect github.com/blang/semver v3.5.1+incompatible // indirect github.com/cenkalti/backoff/v5 v5.0.3 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect @@ -99,7 +96,6 @@ require ( github.com/mattn/go-isatty v0.0.24 // indirect github.com/mattn/go-runewidth v0.0.28 // indirect github.com/mitchellh/mapstructure v1.5.0 // indirect - github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/oklog/ulid/v2 v2.1.2 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect @@ -107,10 +103,6 @@ require ( github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect github.com/pkg/errors v0.9.1 // indirect github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6 // indirect - github.com/prometheus/client_model v0.6.3 // indirect - github.com/prometheus/common v0.71.0 // indirect - github.com/prometheus/otlptranslator v1.0.0 // indirect - github.com/prometheus/procfs v0.22.0 // indirect github.com/shirou/gopsutil/v3 v3.24.5 // indirect github.com/shoenig/go-m1cpu v0.2.2 // indirect github.com/tinylib/msgp v1.6.4 // indirect diff --git a/module/services/accounts/code/go.sum b/module/services/accounts/code/go.sum index 00576cbaf..506f712bd 100644 --- a/module/services/accounts/code/go.sum +++ b/module/services/accounts/code/go.sum @@ -36,8 +36,6 @@ github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1o github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4= github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= -github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= -github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ= github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= @@ -217,8 +215,6 @@ github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRg github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M= github.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRtuthU= github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k= -github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk= -github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE= github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= @@ -243,8 +239,6 @@ github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 h1:n6/2gBQ3RWajuToeY6ZtZTIKv2v7ThUy5KKusIT0yc0= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/oklog/ulid/v2 v2.1.2 h1:IEclFb9JNvzYA6MW2SCxbLzcHTVsfqm3PrqGQJH5zec= github.com/oklog/ulid/v2 v2.1.2/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= @@ -263,16 +257,6 @@ github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6 h1:jL3a8soXdzuTCcRnKhOmtcsVOObdDTFf4O2B403HPRU= github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= -github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU= -github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE= -github.com/prometheus/client_model v0.6.3 h1:O0jaTVAYNxTHYInEPFJt5I3+sN8zqBtVMPTB1qyxiEo= -github.com/prometheus/client_model v0.6.3/go.mod h1:gpN5P9S7Rr6Yr92PiQ+Ixvhf6JZEkF1dnxsYL2aPBEM= -github.com/prometheus/common v0.71.0 h1:9KDAKb7Mj3HEVKyFCK6Dc/HIwlBzZIN2l7/lrHl3KK8= -github.com/prometheus/common v0.71.0/go.mod h1:CLJ5H8TEsGX8bl31BdMkfhIZ+QmZ9tBPPotUxUbfcmk= -github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEoIwkU+A6qos= -github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM= -github.com/prometheus/procfs v0.22.0 h1:6q9+/JL9IKAPbCmBrv9n5O5Ty3NKnciV5X7YGw0oics= -github.com/prometheus/procfs v0.22.0/go.mod h1:CvmFr/GVhIjIvWJZW3tgkODBQMRIf0EyWMQLHCHab58= github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0= github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4= github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8= @@ -334,8 +318,6 @@ go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 h1:julhjPeUH/q/7hinbSd go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0/go.mod h1:Ao2mz688LH/tFf0yMAenidq6k2YNSx6SIY2q6jDACck= go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 h1:UFpxOpYPmMNUtOWhdb+nC1WELIgVf8z9higURrbzYU4= go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0/go.mod h1:YBGjxe3lt0jtXoEXxGrGhv/jM7oUBUDVK7zhAvNyEjc= -go.opentelemetry.io/otel/exporters/prometheus v0.68.0 h1:QOf2IftqQwITVRJpnn0M7M9ZCbgWfxz4P7i9C9yc2N4= -go.opentelemetry.io/otel/exporters/prometheus v0.68.0/go.mod h1:bgSvqu2TWGXiz7yr5UTMfObH8oqxJWHTnubQ3ef9BO4= go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0 h1:N3YQCxjxQ/bMjyc3heladfRm9t9RTksGQH8z4w6yU/0= go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0/go.mod h1:Mp8HOFqcaUyypCuGv9IhDdTHnJ56lSudSHMd+pVSCEA= go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lqCcI= diff --git a/module/services/accounts/code/pkg/auth/pg/session_store_test.go b/module/services/accounts/code/pkg/auth/pg/session_store_test.go index 4a61966e3..ba1d8e044 100644 --- a/module/services/accounts/code/pkg/auth/pg/session_store_test.go +++ b/module/services/accounts/code/pkg/auth/pg/session_store_test.go @@ -63,7 +63,7 @@ func runSessionStoreTests(m *testing.M) int { setupDone := testdb.MeasureSetup("auth-db", []string{"store"}, 120*time.Second) deps, err := sdk.WithDependencies(ctx, sdk.WithDebug(), - sdk.WithExcludedDependencies("cache", "vault", "telemetry"), + sdk.WithExcludedDependencies("cache", "vault"), sdk.WithNamingScope("pgauth-test"), sdk.WithTimeout(120*time.Second), sdk.WithSilence("store"), diff --git a/module/services/accounts/code/pkg/billing/pg/store_test.go b/module/services/accounts/code/pkg/billing/pg/store_test.go index 1b0912df6..d672d63a8 100644 --- a/module/services/accounts/code/pkg/billing/pg/store_test.go +++ b/module/services/accounts/code/pkg/billing/pg/store_test.go @@ -44,7 +44,7 @@ func runBillingStoreTests(m *testing.M) int { setupDone := testdb.MeasureSetup("billing-db", []string{"store"}, 120*time.Second) deps, err := sdk.WithDependencies(ctx, sdk.WithDebug(), - sdk.WithExcludedDependencies("cache", "vault", "telemetry"), + sdk.WithExcludedDependencies("cache", "vault"), sdk.WithNamingScope("pgbilling-test"), sdk.WithTimeout(120*time.Second), sdk.WithSilence("store"), diff --git a/module/services/accounts/code/pkg/business/service_test.go b/module/services/accounts/code/pkg/business/service_test.go index ee0baf90b..e5aef92e2 100644 --- a/module/services/accounts/code/pkg/business/service_test.go +++ b/module/services/accounts/code/pkg/business/service_test.go @@ -109,7 +109,7 @@ func runBusinessTests(m *testing.M) int { setupDone := testdb.MeasureSetup("business-db", []string{"store", "vault"}, 10*time.Minute) deps, err := sdk.WithDependencies(ctx, sdk.WithDebug(), - sdk.WithExcludedDependencies("cache", "telemetry"), + sdk.WithExcludedDependencies("cache"), // Keep this package-owned integration stack distinct from the outer // service runtime and the other package TestMain stacks. The SDK gives // each short-lived flow temporary host ports; the scope also isolates diff --git a/module/services/accounts/code/pkg/cataloggen/deployment_model.go b/module/services/accounts/code/pkg/cataloggen/deployment_model.go index 7aeebf63f..23853385d 100644 --- a/module/services/accounts/code/pkg/cataloggen/deployment_model.go +++ b/module/services/accounts/code/pkg/cataloggen/deployment_model.go @@ -9,6 +9,7 @@ import ( "path/filepath" "sort" + "github.com/codefly-dev/core/resources" "github.com/codefly-dev/core/standards" "gopkg.in/yaml.v3" ) @@ -145,11 +146,11 @@ func assembleDeploymentBindings(documents DeploymentDocuments) (deploymentBindin } for i := range bindings.Interface { exported := &bindings.Interface[i] - visibility, err := manifestCatalogVisibility(exported.Visibility, "", exported.AllowModules) + visibility, err := manifestCatalogVisibility(exported.Visibility, "", true) if err != nil { return deploymentBindings{}, fmt.Errorf("interface %s/%s: %w", exported.Service, exported.Endpoint, err) } - exported.Visibility, exported.AllowModules = visibility, nil + exported.Visibility = visibility } declared := make(map[string]bool, len(module.Services)) for _, reference := range module.Services { @@ -225,7 +226,7 @@ func assembleServiceBinding(moduleName, name string, document []byte) (deploymen if api == "" { api = endpoint.Name } - visibility, err := manifestCatalogVisibility(endpoint.Visibility, endpoint.Location, endpoint.AllowModules) + visibility, err := manifestCatalogVisibility(endpoint.Visibility, endpoint.Location, false) if err != nil { return deploymentServiceBinding{}, fmt.Errorf("service %q endpoint %q: %w", name, endpoint.Name, err) } @@ -270,45 +271,26 @@ func assembleServiceBinding(moduleName, name string, document []byte) (deploymen return service, nil } -// manifestCatalogVisibility projects Codefly's independent visibility/location -// axes into this catalog's categories. MODULE represents every module, so -// narrower allow-lists refuse rather than silently widening the generated policy. -// -// `internal` WITH NO allow-list is the v0.14.0 spelling of what `internal` plus -// `allow-modules: ["*"]` said, and maps to the same category. That is not a -// loosening: core v0.14.0 refuses the key outright — "an allow-list is derived -// from the consumers' declared service dependencies, never written by the module -// it would grant" — so the wildcard was a module answering a question that was -// never its to answer, and deleting it removes a redundant spelling rather than a -// constraint. Which modules actually reach the endpoint is decided by their own -// declared dependencies either way, which is exactly what MODULE means here. -// -// The wildcard is still ACCEPTED, because this generator walks every composed -// module's manifests and the fleet has not finished moving; and a narrower -// allow-list is still REFUSED, because it would need an allow-list-aware catalog -// and core will not let one be authored anyway. So the only two reachable inputs -// both mean "every module that declares the dependency", and nothing silently -// widens. -func manifestCatalogVisibility(visibility, location string, allowed []string) (string, error) { +// manifestCatalogVisibility projects the authored reach into the catalog's legacy MODULE +// category. It does not grant callers: only declared dependency edges render +// network access, and the composing Core derives cross-module reach from asks. +func manifestCatalogVisibility(visibility, location string, exported bool) (string, error) { if visibility == "" { visibility = "private" + if exported { + visibility = "internal" + } } if location != "" { - if location != "external" || visibility != "private" || len(allowed) != 0 { + if location != "external" || visibility != "private" { return "", fmt.Errorf("unsupported external endpoint policy") } return "external", nil } switch visibility { case "private", "public": - if len(allowed) != 0 { - return "", fmt.Errorf("allow-modules requires internal visibility") - } return visibility, nil case "internal": - if len(allowed) != 0 && (len(allowed) != 1 || allowed[0] != "*") { - return "", fmt.Errorf("deployment catalog accepts internal visibility with no allow-modules, or the legacy [*]; a narrower export needs an allow-list-aware catalog") - } return "module", nil default: return "", fmt.Errorf("unsupported authored endpoint visibility %q", visibility) @@ -368,3 +350,12 @@ func standardAllocation(name, api string, endpoints []manifestEndpoint) (uint32, } return port, true } + +func (endpoint *deploymentInterfaceBinding) UnmarshalYAML(node *yaml.Node) error { + var declaration resources.InterfaceEndpoint + if err := node.Decode(&declaration); err != nil { + return err + } + endpoint.Service, endpoint.Endpoint, endpoint.Visibility = declaration.Service, declaration.Endpoint, declaration.Visibility + return nil +} diff --git a/module/services/accounts/code/pkg/cataloggen/deployment_topology.go b/module/services/accounts/code/pkg/cataloggen/deployment_topology.go index 12fc8c552..83585aae4 100644 --- a/module/services/accounts/code/pkg/cataloggen/deployment_topology.go +++ b/module/services/accounts/code/pkg/cataloggen/deployment_topology.go @@ -8,6 +8,7 @@ import ( "sort" "strings" + "github.com/codefly-dev/core/resources" "google.golang.org/protobuf/encoding/protojson" "gopkg.in/yaml.v3" @@ -69,10 +70,9 @@ type deploymentModuleBinding struct { } type deploymentInterfaceBinding struct { - Service string `yaml:"service"` - Endpoint string `yaml:"endpoint"` - Visibility string `yaml:"visibility"` - AllowModules []string `yaml:"allow-modules,omitempty"` + Service string `yaml:"service"` + Endpoint string `yaml:"endpoint"` + Visibility string `yaml:"visibility"` } type deploymentServiceBinding struct { @@ -732,13 +732,7 @@ type manifestEndpointReference struct { Name string `yaml:"name"` } -type manifestEndpoint struct { - Name string `yaml:"name"` - Visibility string `yaml:"visibility,omitempty"` - API string `yaml:"api,omitempty"` - AllowModules []string `yaml:"allow-modules,omitempty"` - Location string `yaml:"location,omitempty"` -} +type manifestEndpoint = resources.Endpoint func marshalGeneratedYAML(source string, value any) ([]byte, error) { document, err := yaml.Marshal(value) @@ -1350,9 +1344,8 @@ spec: - podSelector: matchLabels: codefly.dev/bootstrap-service: %s - job-name: %s ports: -`, service, namespace, serviceApp, service, service) +`, service, namespace, serviceApp, service) writeNetworkPorts(source, ports, 8) fmt.Fprintf(source, `--- apiVersion: networking.k8s.io/v1 @@ -1364,7 +1357,6 @@ spec: podSelector: matchLabels: codefly.dev/bootstrap-service: %s - job-name: %s policyTypes: - Egress egress: @@ -1373,7 +1365,7 @@ spec: matchLabels: app: %s ports: -`, service, service, namespace, service, service, serviceApp) +`, service, service, namespace, service, serviceApp) writeNetworkPorts(source, ports, 8) } diff --git a/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go b/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go index 03c14124e..68daf09fc 100644 --- a/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go +++ b/module/services/accounts/code/pkg/cataloggen/deployment_topology_test.go @@ -71,16 +71,16 @@ func TestDeploymentTopologyIsDeterministicAndCurrent(t *testing.T) { require.Equal(t, string(readFixture(t, "testdata/network-policy.golden.yaml")), string(first.NetworkPolicy), "run: go generate ./pkg/cataloggen") require.Equal(t, string(readFixture(t, "testdata/mesh-policy.golden.yaml")), string(first.MeshPolicy), "run: go generate ./pkg/cataloggen") - require.Len(t, first.Catalog.GetServices(), 9) + require.Len(t, first.Catalog.GetServices(), 8) require.Len(t, first.Catalog.GetInterfaceEndpoints(), 7) - require.Len(t, first.Catalog.GetPublicEgress(), 5) + require.Len(t, first.Catalog.GetPublicEgress(), 4) endpointCount, dependencyCount := 0, 0 for _, service := range first.Catalog.GetServices() { endpointCount += len(service.GetEndpoints()) dependencyCount += len(service.GetDependencies()) } - require.Equal(t, 15, endpointCount) - require.Equal(t, 8, dependencyCount) + require.Equal(t, 14, endpointCount) + require.Equal(t, 6, dependencyCount) // Accounts REST stays private. Gateway REST is exported for module mints; // solutions resolve its separately exported workload listener. restVisibility := map[string]catalogv1.EndpointVisibility{ @@ -88,7 +88,6 @@ func TestDeploymentTopologyIsDeterministicAndCurrent(t *testing.T) { "auth-gateway": catalogv1.EndpointVisibility_ENDPOINT_VISIBILITY_MODULE, } privateREST := map[string]bool{"accounts": false, "auth-gateway": false} - authGatewayTelemetry := false for _, service := range first.Catalog.GetServices() { if _, ok := privateREST[service.GetName()]; !ok { continue @@ -99,17 +98,23 @@ func TestDeploymentTopologyIsDeterministicAndCurrent(t *testing.T) { privateREST[service.GetName()] = true } } - if service.GetName() == "auth-gateway" { - for _, dependency := range service.GetDependencies() { - if dependency.GetService() == "telemetry" { - require.Equal(t, []string{"grpc"}, dependency.GetEndpoints()) - authGatewayTelemetry = true - } - } - } } require.Equal(t, map[string]bool{"accounts": true, "auth-gateway": true}, privateREST) - require.True(t, authGatewayTelemetry) + // The module owns no collector: telemetry is exported to the cell's, which is + // reached in-mesh, so no service has a telemetry dependency and the gateway, + // whose job is to be the perimeter, has no public egress to reach a backend. + for _, service := range first.Catalog.GetServices() { + require.NotEqual(t, "telemetry", service.GetName()) + for _, dependency := range service.GetDependencies() { + require.NotEqual(t, "telemetry", dependency.GetService(), "%s depends on a telemetry service", service.GetName()) + } + } + egress := map[string]bool{} + for _, grant := range first.Catalog.GetPublicEgress() { + egress[grant.GetService()] = true + } + require.Equal(t, map[string]bool{"accounts": true, "frontend": true, "marketing": true, "policy-log": true}, egress) + require.False(t, egress["auth-gateway"], "auth-gateway must have no public egress") accountsConnectExposed, accountsAuthorityExposed, gatewayRESTExposed := false, false, false for _, endpoint := range first.Catalog.GetInterfaceEndpoints() { if endpoint.GetService() == "accounts" && endpoint.GetEndpoint() == "connect" { @@ -129,36 +134,27 @@ func TestDeploymentTopologyIsDeterministicAndCurrent(t *testing.T) { require.True(t, accountsConnectExposed) require.True(t, accountsAuthorityExposed) require.True(t, gatewayRESTExposed) - require.Equal(t, 21, strings.Count(string(first.NetworkPolicy), "\nkind: NetworkPolicy\n")) + require.Equal(t, 19, strings.Count(string(first.NetworkPolicy), "\nkind: NetworkPolicy\n")) require.NotContains(t, string(first.NetworkPolicy), "allow-intra-namespace") for _, name := range []string{ "allow-accounts-from-dependents", "allow-auth-gateway-from-dependents", "allow-auth-gateway-to-dependencies", "allow-frontend-to-dependencies", "allow-store-from-bootstrap", "allow-store-bootstrap-to-store", - "allow-frontend-public-egress", "allow-marketing-public-egress", "allow-telemetry-from-dependents", - "allow-telemetry-public-egress", "allow-istio-ingress-to-marketing", "allow-istio-ingress-to-frontend", + "allow-frontend-public-egress", "allow-marketing-public-egress", + "allow-istio-ingress-to-marketing", "allow-istio-ingress-to-frontend", } { require.Contains(t, string(first.NetworkPolicy), "name: "+name) } + require.NotContains(t, string(first.NetworkPolicy), "telemetry") + require.NotContains(t, string(first.NetworkPolicy), "name: allow-auth-gateway-public-egress") require.NotContains(t, string(first.NetworkPolicy), "name: allow-istio-ingress-to-auth-gateway") require.Equal(t, 2, strings.Count(string(first.NetworkPolicy), "codefly.dev/bootstrap-service: store")) + require.NotContains(t, string(first.NetworkPolicy), "job-name:", "bootstrap Job names contain a content digest; policies select the stable service label") require.Contains(t, string(first.NetworkPolicy), "192.175.48.0/24") require.Contains(t, string(first.NetworkPolicy), "64:ff9b::/96") } -// MODULE is the catalog category for every composed module, not a license to -// erase a narrower authored allow-list. Legacy Codefly spellings also refuse. -// -// `visibility: internal` WITH NO allow-list has moved OUT of the refused set and -// is now what the manifests author (issue #952). It is the v0.14.0 spelling of -// what `internal` plus `allow-modules: ["*"]` said — core refuses the key on an -// export by name, because an allow-list is derived from the consumers' declared -// dependencies and never written by the module it would grant — so the two mean -// the same thing and map to the same category. -// -// The narrower cases stay refused, which is the half that matters: `[example]` -// names SOME modules, and this catalog has no way to express that, so admitting -// it would silently widen the generated policy to every module. The legacy -// `visibility: module` stays refused too. +// Legacy reach spellings and authored consumer lists refuse. MODULE remains +// a catalog category; declared dependencies determine which callers reach it. func TestDeploymentTopologyRefusesUnrepresentableEndpointPolicies(t *testing.T) { catalog := readFixture(t, "../../../generated/service-catalog.json") documents := readDeploymentDocuments(t) @@ -183,13 +179,13 @@ func TestDeploymentTopologyRefusesUnrepresentableEndpointPolicies(t *testing.T) // quietly accepting everything: an unchanged tree must still produce artifacts. _, err := cataloggen.BuildDeploymentArtifacts(catalog, documents) require.NoError(t, err, "the authored `internal` with no allow-list must build") - // The legacy wildcard also still builds, because this generator walks every - // composed module's manifests and the fleet has not finished moving. + // A legacy wildcard is also an authored consumer grant and must refuse, + // matching the Core declaration boundary used to load these manifests. legacy := withService(t, documents, "accounts", " - name: connect\n visibility: internal\n", " - name: connect\n visibility: internal\n allow-modules: [\"*\"]\n") _, err = cataloggen.BuildDeploymentArtifacts(catalog, legacy) - require.NoError(t, err, "the legacy wildcard must still build while the fleet moves") + require.ErrorIs(t, err, resources.ErrInvalidEndpointDeclaration, "legacy wildcard grants must refuse") } // The manifests are the model: a deployment fact lives in the service manifest @@ -210,11 +206,11 @@ func TestDeploymentSpecIsStrictAndComplete(t *testing.T) { require.ErrorContains(t, err, `names unknown endpoint "ghost"`) _, err = cataloggen.BuildDeploymentArtifacts(serviceCatalog, - withService(t, documents, "telemetry", " deployment:\n", " deployment-typo:\n")) + withService(t, documents, "vault", " deployment:\n", " deployment-typo:\n")) require.ErrorContains(t, err, "has no spec.deployment block") _, err = cataloggen.BuildDeploymentArtifacts(serviceCatalog, - withServiceDocument(documents, "ghost", documents.Services["telemetry"])) + withServiceDocument(documents, "ghost", documents.Services["vault"])) require.ErrorContains(t, err, "not declared by module.codefly.yaml") _, err = cataloggen.BuildDeploymentArtifacts(serviceCatalog, @@ -229,7 +225,7 @@ func TestGeneratedCodeflyAndNetworkManifestsParseStrictly(t *testing.T) { require.NoError(t, loadedModule.ValidateInterface(ctx)) loadedServices, err := loadedModule.LoadServices(ctx) require.NoError(t, err) - require.Len(t, loadedServices, 9) + require.Len(t, loadedServices, 8) moduleDocument := readFixture(t, "../../../../../module.codefly.yaml") var moduleEntry struct { @@ -241,7 +237,7 @@ func TestGeneratedCodeflyAndNetworkManifestsParseStrictly(t *testing.T) { require.NoError(t, yaml.Unmarshal(moduleDocument, &module)) _, err = module.Proto(ctx) require.NoError(t, err) - require.Len(t, module.ServiceReferences, 9) + require.Len(t, module.ServiceReferences, 8) for _, reference := range module.ServiceReferences { document := readFixture(t, filepath.Join("../../../../../services", reference.Name, "service.codefly.yaml")) @@ -283,14 +279,14 @@ func TestGeneratedCodeflyAndNetworkManifestsParseStrictly(t *testing.T) { require.False(t, names[document.Metadata.Name], "duplicate NetworkPolicy %s", document.Metadata.Name) names[document.Metadata.Name] = true } - require.Len(t, names, 21) + require.Len(t, names, 19) require.True(t, names["allow-istio-ingress-to-marketing"]) require.True(t, names["allow-istio-ingress-to-frontend"]) require.False(t, names["allow-istio-ingress-to-auth-gateway"]) require.True(t, names["allow-store-from-bootstrap"]) require.True(t, names["allow-store-bootstrap-to-store"]) - require.True(t, names["allow-telemetry-from-dependents"]) - require.True(t, names["allow-telemetry-public-egress"]) + require.False(t, names["allow-telemetry-from-dependents"]) + require.False(t, names["allow-telemetry-public-egress"]) } func TestGeneratedMeshPolicyGatesInternalAuthorityByCallerIdentity(t *testing.T) { @@ -772,7 +768,7 @@ func TestDeploymentTopologyPodPortIsTheAllocationUnlessDeclared(t *testing.T) { } for endpoint, want := range map[string]uint32{ "accounts/connect": 8081, "accounts/grpc": 9090, "accounts/rest": 8080, "accounts/authority": 0, - "auth-gateway/grpc": 9090, "auth-gateway/rest": 0, "auth-gateway/workload": 0, "telemetry/grpc": 9090, + "auth-gateway/grpc": 9090, "auth-gateway/rest": 0, "auth-gateway/workload": 0, "cache/read": 6379, "cache/write": 6379, "store/tcp": 5432, "vault/http": 8200, "frontend/http": 3000, "marketing/http": 3000, } { diff --git a/module/services/accounts/code/pkg/cataloggen/deployment_visibility_pin_test.go b/module/services/accounts/code/pkg/cataloggen/deployment_visibility_pin_test.go index e214bac20..1b154690b 100644 --- a/module/services/accounts/code/pkg/cataloggen/deployment_visibility_pin_test.go +++ b/module/services/accounts/code/pkg/cataloggen/deployment_visibility_pin_test.go @@ -3,7 +3,8 @@ package cataloggen import "testing" // The OTHER HALF OF A PIN. The module agent at the repository root holds the same -// projection as endpointVisibility (topology_manifests.go), in a separate Go module +// projection as this package's manifestCatalogVisibility (deployment_model.go), +// in topology_manifests.go and a separate Go module // so neither side can import the other, and topology_visibility_test.go carries the // matching table. // @@ -16,36 +17,25 @@ import "testing" func TestManifestCatalogVisibilityMatchesTheRootProjection(t *testing.T) { for _, tc := range []struct { visibility, location string - allowed []string + exported bool want string }{ - // `internal` with NO allow-list is what the manifests author at core - // v0.14.0, and it maps to the same category the legacy wildcard did. - {"internal", "", nil, "module"}, - {"internal", "", []string{}, "module"}, - {"internal", "", []string{"*"}, "module"}, - {"private", "external", nil, "external"}, - {"public", "", nil, "public"}, - {"", "", nil, "private"}, - // A NARROWER list still refuses: this catalog cannot express "some - // modules", so admitting one would silently widen the generated policy to - // every module. - {"internal", "", []string{"example"}, ""}, - {"internal", "", []string{"*", "example"}, ""}, - {"public", "", []string{"example"}, ""}, - {"module", "", nil, ""}, - {"external", "", nil, ""}, - {"private", "elsewhere", nil, ""}, - {"internal", "external", []string{"example"}, ""}, + {"internal", "", false, "module"}, + {"private", "external", false, "external"}, + {"public", "", false, "public"}, + {"", "", false, "private"}, + {"internal", "", true, "module"}, + {"module", "", false, ""}, + {"external", "", false, ""}, + {"private", "elsewhere", false, ""}, + {"internal", "external", false, ""}, } { - got, err := manifestCatalogVisibility(tc.visibility, tc.location, tc.allowed) + got, err := manifestCatalogVisibility(tc.visibility, tc.location, tc.exported) if tc.want == "" { if err == nil { t.Errorf("accepted unsupported policy %#v as %q", tc, got) } - continue - } - if err != nil || got != tc.want { + } else if err != nil || got != tc.want { t.Errorf("policy %#v: got %q, %v", tc, got, err) } } diff --git a/module/services/accounts/code/pkg/cataloggen/endpoint_declarations_test.go b/module/services/accounts/code/pkg/cataloggen/endpoint_declarations_test.go new file mode 100644 index 000000000..62d923939 --- /dev/null +++ b/module/services/accounts/code/pkg/cataloggen/endpoint_declarations_test.go @@ -0,0 +1,43 @@ +package cataloggen_test + +import ( + "testing" + + "accounts/pkg/cataloggen" + + "github.com/codefly-dev/core/resources" + "github.com/stretchr/testify/require" +) + +func TestDeploymentTopologyAcceptsConsumerDerivedInternalReach(t *testing.T) { + documents := readDeploymentDocuments(t) + artifacts, err := cataloggen.BuildDeploymentArtifacts(readFixture(t, "../../../generated/service-catalog.json"), documents) + require.NoError(t, err) + // The reach category does not grant callers. The policies are still the + // exact dependency edges, and internal RPCs still require caller identity. + require.Equal(t, string(readFixture(t, "testdata/network-policy.golden.yaml")), string(artifacts.NetworkPolicy)) + require.Equal(t, string(readFixture(t, "testdata/mesh-policy.golden.yaml")), string(artifacts.MeshPolicy)) + // Core defaults an interface export to internal, while a service endpoint + // still defaults to private. Omitting the interface reach keeps its policy. + defaultExport := withModule(t, documents, " visibility: internal\n", "") + defaults, err := cataloggen.BuildDeploymentArtifacts(readFixture(t, "../../../generated/service-catalog.json"), defaultExport) + require.NoError(t, err) + require.Equal(t, string(artifacts.CatalogJSON), string(defaults.CatalogJSON)) +} + +func TestDeploymentTopologyRefusesAuthoredConsumersByKeyPresence(t *testing.T) { + documents := readDeploymentDocuments(t) + catalog := readFixture(t, "../../../generated/service-catalog.json") + for _, key := range []string{"allow-modules", "allow_modules", "allowModules", "Allow Modules"} { + for _, value := range []string{"[\"*\"]", "[example]", "[]", "null"} { + t.Run(key+"/"+value, func(t *testing.T) { + _, err := cataloggen.BuildDeploymentArtifacts(catalog, withModule(t, documents, + " visibility: internal\n", " visibility: internal\n "+key+": "+value+"\n")) + require.ErrorIs(t, err, resources.ErrInvalidEndpointDeclaration) + _, err = cataloggen.BuildDeploymentArtifacts(catalog, withService(t, documents, "accounts", + " visibility: internal\n", " visibility: internal\n "+key+": "+value+"\n")) + require.ErrorIs(t, err, resources.ErrInvalidEndpointDeclaration) + }) + } + } +} diff --git a/module/services/accounts/code/pkg/cataloggen/test_targets_test.go b/module/services/accounts/code/pkg/cataloggen/test_targets_test.go index 93bcbfef3..0340ff899 100644 --- a/module/services/accounts/code/pkg/cataloggen/test_targets_test.go +++ b/module/services/accounts/code/pkg/cataloggen/test_targets_test.go @@ -110,7 +110,6 @@ func TestAccountsTargetMetadataRetainsConservativeInputs(t *testing.T) { require.Contains(t, target.Command, "-tags=pure") } else { require.Contains(t, target.RuntimeServices, "store") - require.NotContains(t, target.RuntimeServices, "telemetry") if startsCache[target.Suite] { require.Contains(t, target.RuntimeServices, "cache") } else { diff --git a/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml b/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml index 14cdf6b35..af981918c 100644 --- a/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml +++ b/module/services/accounts/code/pkg/cataloggen/testdata/network-policy.golden.yaml @@ -194,33 +194,6 @@ spec: --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy -metadata: - name: allow-telemetry-from-dependents - namespace: saas-starter -spec: - podSelector: - matchLabels: - app: telemetry - policyTypes: - - Ingress - ingress: - - from: - - podSelector: - matchLabels: - app: accounts - ports: - - protocol: TCP - port: 9090 - - from: - - podSelector: - matchLabels: - app: auth-gateway - ports: - - protocol: TCP - port: 9090 ---- -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy metadata: name: allow-vault-from-dependents namespace: saas-starter @@ -265,13 +238,6 @@ spec: ports: - protocol: TCP port: 5432 - - to: - - podSelector: - matchLabels: - app: telemetry - ports: - - protocol: TCP - port: 9090 - to: - podSelector: matchLabels: @@ -310,13 +276,6 @@ spec: ports: - protocol: TCP port: 6379 - - to: - - podSelector: - matchLabels: - app: telemetry - ports: - - protocol: TCP - port: 9090 --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy @@ -352,7 +311,6 @@ spec: - podSelector: matchLabels: codefly.dev/bootstrap-service: store - job-name: store ports: - protocol: TCP port: 5432 @@ -366,7 +324,6 @@ spec: podSelector: matchLabels: codefly.dev/bootstrap-service: store - job-name: store policyTypes: - Egress egress: @@ -585,55 +542,3 @@ spec: ports: - protocol: TCP port: 443 ---- -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: allow-telemetry-public-egress - namespace: saas-starter -spec: - podSelector: - matchLabels: - app: telemetry - policyTypes: - - Egress - egress: - - to: - - ipBlock: - cidr: 0.0.0.0/0 - except: - - 0.0.0.0/8 - - 10.0.0.0/8 - - 100.64.0.0/10 - - 127.0.0.0/8 - - 169.254.0.0/16 - - 172.16.0.0/12 - - 192.0.0.0/24 - - 192.0.2.0/24 - - 192.31.196.0/24 - - 192.52.193.0/24 - - 192.88.99.0/24 - - 192.168.0.0/16 - - 192.175.48.0/24 - - 198.18.0.0/15 - - 198.51.100.0/24 - - 203.0.113.0/24 - - 224.0.0.0/4 - - 240.0.0.0/4 - - ipBlock: - cidr: ::/0 - except: - - ::/128 - - ::1/128 - - 64:ff9b::/96 - - 64:ff9b:1::/48 - - 100::/64 - - 2001::/23 - - 2001:db8::/32 - - 2002::/16 - - fc00::/7 - - fe80::/10 - - ff00::/8 - ports: - - protocol: TCP - port: 443 diff --git a/module/services/accounts/code/pkg/infra/postgres_webhooks_test.go b/module/services/accounts/code/pkg/infra/postgres_webhooks_test.go index c9a1f2ce2..c3568d915 100644 --- a/module/services/accounts/code/pkg/infra/postgres_webhooks_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_webhooks_test.go @@ -102,7 +102,7 @@ func runPostgresInfraTests(m *testing.M) int { setupDone := testdb.MeasureSetup("infra-db", []string{"store"}, 90*time.Second) deps, err := sdk.WithDependencies(ctx, sdk.WithDebug(), - sdk.WithExcludedDependencies("cache", "vault", "telemetry"), + sdk.WithExcludedDependencies("cache", "vault"), sdk.WithNamingScope("test-infra"), sdk.WithTimeout(90*time.Second), sdk.WithSilence("store"), diff --git a/module/services/accounts/code/pkg/redisintegration/main_test.go b/module/services/accounts/code/pkg/redisintegration/main_test.go index a14f72f41..10bbfa6d3 100644 --- a/module/services/accounts/code/pkg/redisintegration/main_test.go +++ b/module/services/accounts/code/pkg/redisintegration/main_test.go @@ -46,7 +46,7 @@ func runRedisIntegrationTests(m *testing.M) int { setupDone := testdb.MeasureSetup("redis-db", []string{"cache", "store"}, 120*time.Second) deps, err := sdk.WithDependencies(ctx, sdk.WithDebug(), - sdk.WithExcludedDependencies("vault", "telemetry"), + sdk.WithExcludedDependencies("vault"), sdk.WithNamingScope("redis-integration"), sdk.WithTimeout(120*time.Second), sdk.WithSilence("store", "cache"), diff --git a/module/services/accounts/code/retention_telemetry_test.go b/module/services/accounts/code/retention_telemetry_test.go index 39232c176..899062f61 100644 --- a/module/services/accounts/code/retention_telemetry_test.go +++ b/module/services/accounts/code/retention_telemetry_test.go @@ -3,15 +3,11 @@ package main import ( "context" "errors" - "net/http" - "net/http/httptest" "os" "strings" "testing" "time" - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" "github.com/stretchr/testify/require" sdkmetric "go.opentelemetry.io/otel/sdk/metric" "go.opentelemetry.io/otel/sdk/metric/metricdata" @@ -121,37 +117,44 @@ func TestAScheduledRetentionRunReportsToTheGauge(t *testing.T) { require.Equal(t, map[string]int64{"audit_events provision": 0}, retentionFailing(t, reader)) } -// The alert pack's retention alert reads the gauge under the name the /metrics -// scrape serves it by, which is also the series the pack's promtool tests feed -// the alert: a failed run's first scrape carries a 1, with no earlier sample. -func TestTheRetentionAlertReadsTheGaugeAsItIsScraped(t *testing.T) { - registry := prometheus.NewRegistry() - exporter, err := newScrapeExporter(registry) +// A failed retention run pushes its gauge to the collector. The alert pack and +// its promtool fixtures use the Prometheus translation of that instrument name. +func TestTheRetentionAlertReadsTheGaugeExportedOverOTLP(t *testing.T) { + endpoint, capture := startMetricCapture(t) + t.Setenv(otlpEndpointVariable, "http://"+endpoint) + metricsProvider, err := enableOTELMetrics(t.Context()) require.NoError(t, err) - provider := sdkmetric.NewMeterProvider(sdkmetric.WithReader(exporter)) - t.Cleanup(func() { _ = provider.Shutdown(context.Background()) }) - telemetry, err := newRetentionTelemetry(provider.Meter("github.com/codefly-dev/module-saas-starter/accounts")) + t.Cleanup(func() { _ = metricsProvider.Shutdown(context.Background()) }) + telemetry, err := newRetentionTelemetry(metricsProvider.provider.Meter("github.com/codefly-dev/module-saas-starter/accounts")) require.NoError(t, err) telemetry.report(context.Background(), &business.RetentionError{Failures: []business.RetentionFailure{ {Resource: "audit_events", Step: business.RetentionStepRemove, Err: errors.New("lock not available")}, }}) - recorder := httptest.NewRecorder() - promhttp.HandlerFor(registry, promhttp.HandlerOpts{}).ServeHTTP(recorder, - httptest.NewRequest(http.MethodGet, "http://accounts.internal/metrics", nil)) - require.Equal(t, http.StatusOK, recorder.Code) - scraped := strings.ReplaceAll(retentionFailingMetric, ".", "_") - body := recorder.Body.String() - require.Contains(t, body, "# TYPE "+scraped+" gauge\n") - var sample string - for _, line := range strings.Split(body, "\n") { - if strings.HasPrefix(line, scraped+"{") { - sample = line + require.NoError(t, metricsProvider.provider.ForceFlush(t.Context())) + request := receiveMetrics(t, capture.requests) + found := false + for _, resource := range request.ResourceMetrics { + for _, scope := range resource.ScopeMetrics { + for _, instrument := range scope.Metrics { + if instrument.Name != retentionFailingMetric { + continue + } + for _, point := range instrument.GetGauge().DataPoints { + labels := map[string]string{} + for _, attribute := range point.Attributes { + labels[attribute.Key] = attribute.Value.GetStringValue() + } + if labels["resource"] == "audit_events" && labels["step"] == "remove" { + require.EqualValues(t, 1, point.GetAsInt()) + found = true + } + } + } } } - require.Contains(t, sample, `resource="audit_events"`) - require.Contains(t, sample, `step="remove"`) - require.True(t, strings.HasSuffix(sample, "} 1"), "the failed part reads 1: %s", sample) + require.True(t, found, "the failed retention part is exported as a gauge") + scraped := strings.ReplaceAll(retentionFailingMetric, ".", "_") pack, err := metrics.DefaultSLOPack() require.NoError(t, err) diff --git a/module/services/accounts/code/telemetry.go b/module/services/accounts/code/telemetry.go new file mode 100644 index 000000000..74ba1ed51 --- /dev/null +++ b/module/services/accounts/code/telemetry.go @@ -0,0 +1,87 @@ +package main + +import ( + "context" + "errors" + "net/url" + "os" + "strings" + + "github.com/codefly-dev/core/wool" + codefly "github.com/codefly-dev/sdk-go" + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/sdk/resource" +) + +// otlpEndpointVariable is the whole telemetry contract: OpenTelemetry's own name +// for the collector's address. The platform delivers it when the cell has a +// collector this workload may reach, and delivers nothing when it does not. +// +// Deliver it and the process exports traces and metrics over OTLP/gRPC; deliver +// none and it exports nothing over OTLP. Nothing here reads a state, a reason or +// a protocol: core's wool/otel reads this variable itself, tells a URL from a +// host:port, takes the transport from the scheme, validates the URL and refuses a +// portless one, and the metrics exporter reads it from the same environment. +const otlpEndpointVariable = "OTEL_EXPORTER_OTLP_ENDPOINT" + +// otlpEndpointConfigured reports whether the process was given a collector. A +// whitespace-only value is none: the OpenTelemetry metrics exporter trims what it +// reads and treats an empty result as unset, and an exporter with nothing +// configured dials its own default, localhost:4317. +func otlpEndpointConfigured() bool { + return strings.TrimSpace(os.Getenv(otlpEndpointVariable)) != "" +} + +// requireMetricsEndpoint refuses what the metrics exporter cannot read. It reads +// the variable only as an http:// or https:// URL, and on anything else keeps its +// default and dials localhost:4317 without saying so, so a value of another shape +// is rejected here instead of creating a metrics exporter at a default address. +func requireMetricsEndpoint() error { + if !otlpEndpointConfigured() { + return errors.New("no " + otlpEndpointVariable + " to export metrics to") + } + parsed, err := url.Parse(strings.TrimSpace(os.Getenv(otlpEndpointVariable))) + if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") { + return errors.New(otlpEndpointVariable + " must be an http:// or https:// URL for metrics to be exported; the exporter ignores any other form and would fall back to its own default address") + } + return nil +} + +// telemetryResource builds the metrics resource. service.name is +// whatever the environment says — OTEL_SERVICE_NAME, or service.name in +// OTEL_RESOURCE_ATTRIBUTES — and otherwise the service identity the Codefly SDK +// holds. No name is typed here: a literal would override the environment, so the +// platform's identity could never reach the backend. +func telemetryResource(ctx context.Context, identity string) (*resource.Resource, error) { + options := []resource.Option{resource.WithService()} + if identity != "" { + options = append(options, resource.WithAttributes(attribute.String("service.name", identity))) + } + // Last, so what the environment declares wins over the identity above. + options = append(options, resource.WithFromEnv()) + return resource.New(ctx, options...) +} + +// currentTelemetryResource is telemetryResource for this process. +func currentTelemetryResource(ctx context.Context) (*resource.Resource, error) { + return telemetryResource(ctx, codeflyServiceName(codefly.Context())) +} + +// codeflyServiceName is the service identity the Codefly SDK holds: the +// `module/service` identity of the provider it built at Init, which wool stamps on +// every log line from this process. It is empty when the SDK has not been +// initialised. The SDK exposes no accessor of its own, and the environment +// carrier it reads this from is not product code's to read, so this asks the +// provider. +func codeflyServiceName(ctx context.Context) string { + provider, ok := ctx.Value(wool.ProviderKey).(*wool.Provider) + if !ok || provider == nil { + return "" + } + source := provider.Get(ctx).Source() + if source == nil { + return "" + } + // An identity with no module reads `/service`; with neither, `/`. + return strings.Trim(strings.TrimSpace(source.Unique), "/") +} diff --git a/module/services/accounts/code/telemetry_metrics.go b/module/services/accounts/code/telemetry_metrics.go index 61de19705..b6f672284 100644 --- a/module/services/accounts/code/telemetry_metrics.go +++ b/module/services/accounts/code/telemetry_metrics.go @@ -2,54 +2,50 @@ package main import ( "context" - "net/http" runtimemetrics "runtime/metrics" "time" - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" otelruntime "go.opentelemetry.io/contrib/instrumentation/runtime" "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc" - otelprometheus "go.opentelemetry.io/otel/exporters/prometheus" otelmetric "go.opentelemetry.io/otel/metric" "go.opentelemetry.io/otel/sdk/metric" - "go.opentelemetry.io/otel/sdk/resource" ) +// otelMetrics owns the process's MeterProvider. Metrics leave by one path only: +// an OTLP push to the collector OTEL_EXPORTER_OTLP_ENDPOINT names, which is the +// record for traces and metrics alike. There is no scrape endpoint to mount, +// expose or exempt. type otelMetrics struct { provider *metric.MeterProvider - handler http.Handler } -func enableOTELMetrics(ctx context.Context, serviceName, endpoint string) (*otelMetrics, error) { - exporter, err := otlpmetricgrpc.New( - ctx, - otlpmetricgrpc.WithEndpoint(endpoint), - otlpmetricgrpc.WithInsecure(), - ) - if err != nil { +// enableOTELMetrics starts the process's OTLP metrics push to the collector the +// environment names. It is only for a process that was given one: with no +// OTEL_EXPORTER_OTLP_ENDPOINT it refuses rather than build an exporter, because +// an otlpmetricgrpc exporter with nothing configured dials its own default, +// localhost:4317 over TLS, and a process that exports to nowhere reports success. +func enableOTELMetrics(ctx context.Context) (*otelMetrics, error) { + if err := requireMetricsEndpoint(); err != nil { return nil, err } - registry := prometheus.NewRegistry() - scrapeExporter, err := newScrapeExporter(registry) + // The resource comes before the exporter, so a failure here leaves nothing + // to shut down: an exporter holds a client connection that only Shutdown + // releases. + res, err := currentTelemetryResource(ctx) if err != nil { return nil, err } - res, err := resource.New( - ctx, - resource.WithService(), - resource.WithFromEnv(), - resource.WithAttributes(attribute.String("service.name", serviceName)), - ) + // The exporter reads the collector from OTEL_EXPORTER_OTLP_ENDPOINT itself and + // takes transport from the URL scheme: the operations intake uses https:// + // with TLS; http:// remains available for local plaintext receivers. + exporter, err := otlpmetricgrpc.New(ctx) if err != nil { return nil, err } provider := metric.NewMeterProvider( metric.WithResource(res), metric.WithReader(metric.NewPeriodicReader(exporter, metric.WithInterval(30*time.Second))), - metric.WithReader(scrapeExporter), ) if err := otelruntime.Start(otelruntime.WithMeterProvider(provider)); err != nil { _ = provider.Shutdown(ctx) @@ -60,21 +56,7 @@ func enableOTELMetrics(ctx context.Context, serviceName, endpoint string) (*otel return nil, err } otel.SetMeterProvider(provider) - return &otelMetrics{ - provider: provider, - handler: promhttp.HandlerFor(registry, promhttp.HandlerOpts{}), - }, nil -} - -// newScrapeExporter is the exporter the /metrics scrape serves: the names it -// gives instruments are the ones the alert pack's expressions read. -func newScrapeExporter(registry prometheus.Registerer) (*otelprometheus.Exporter, error) { - return otelprometheus.New( - otelprometheus.WithRegisterer(registry), - otelprometheus.WithResourceAsConstantLabels( - attribute.NewAllowKeysFilter("service.name"), - ), - ) + return &otelMetrics{provider: provider}, nil } func startGCActivityMetrics(provider *metric.MeterProvider) error { @@ -108,10 +90,6 @@ func startGCActivityMetrics(provider *metric.MeterProvider) error { return err } -func (m *otelMetrics) Handler() http.Handler { - return m.handler -} - func (m *otelMetrics) Shutdown(ctx context.Context) error { return m.provider.Shutdown(ctx) } diff --git a/module/services/accounts/code/telemetry_metrics_test.go b/module/services/accounts/code/telemetry_metrics_test.go index 0429dcd90..b4ecb1275 100644 --- a/module/services/accounts/code/telemetry_metrics_test.go +++ b/module/services/accounts/code/telemetry_metrics_test.go @@ -60,6 +60,7 @@ func (versionConnectHandler) Version( func TestTelemetryMetricsExportRuntimeAndUnsampledRED(t *testing.T) { endpoint, capture := startMetricCapture(t) t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "service.instance.id=test-instance") + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") previousMeterProvider := otel.GetMeterProvider() previousTracerProvider := otel.GetTracerProvider() @@ -70,7 +71,8 @@ func TestTelemetryMetricsExportRuntimeAndUnsampledRED(t *testing.T) { ) otel.SetTracerProvider(tracerProvider) - metrics, err := enableOTELMetrics(t.Context(), "test-service", endpoint) + collectorAt(t, "http", endpoint) + metrics, err := enableOTELMetrics(t.Context()) require.NoError(t, err) require.NotNil(t, metrics) t.Cleanup(func() { @@ -89,6 +91,7 @@ func TestTelemetryMetricsExportRuntimeAndUnsampledRED(t *testing.T) { require.NoError(t, metrics.provider.ForceFlush(t.Context())) request := receiveMetrics(t, capture.requests) require.Equal(t, "test-instance", resourceAttribute(request, "service.instance.id")) + require.Equal(t, "named-by-the-platform", resourceAttribute(request, "service.name")) metricNames := exportedMetricNames(request) require.Contains(t, metricNames, "go.goroutine.count") require.Contains(t, metricNames, "go.memory.allocated") @@ -107,24 +110,75 @@ func TestTelemetryMetricsExportRuntimeAndUnsampledRED(t *testing.T) { require.Equal(t, uint64(1), connectCounts["permission_denied"]) require.Empty(t, spanExporter.GetSpans()) +} - recorder := httptest.NewRecorder() - metrics.Handler().ServeHTTP( - recorder, - httptest.NewRequest(http.MethodGet, "http://accounts.internal/metrics", nil), - ) - require.Equal(t, http.StatusOK, recorder.Code) - require.Contains(t, recorder.Header().Get("Content-Type"), "text/plain") - body := recorder.Body.String() - require.Contains(t, body, "go_goroutine_count") - require.Contains(t, body, "go_memory_allocated") - require.Contains(t, body, "go_memory_gc_goal") - require.Contains(t, body, "go_gc_cycle_count") - require.Contains(t, body, "go_gc_pause_cpu_time") - require.Contains(t, body, "rpc_server_call_duration_seconds_count") - require.Contains(t, body, `rpc_method="grpc.health.v1.Health/Check"`) - require.Contains(t, body, `rpc_response_status_code="OK"`) - require.Contains(t, body, `service_name="test-service"`) +// The exporter reads its transport off the URL's scheme, so an https:// URL is +// TLS and cannot talk to a plaintext receiver. A constant WithInsecure() would +// have let it. +func TestEnableOTELMetricsDerivesTransportFromTheURLScheme(t *testing.T) { + endpoint, capture := startMetricCapture(t) + previousMeterProvider := otel.GetMeterProvider() + collectorAt(t, "https", endpoint) + metrics, err := enableOTELMetrics(t.Context()) + require.NoError(t, err) + require.NotNil(t, metrics) + t.Cleanup(func() { + shutdownCtx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + _ = metrics.Shutdown(shutdownCtx) // the flush it attempts is the failure under test + otel.SetMeterProvider(previousMeterProvider) + }) + + flushCtx, cancel := context.WithTimeout(t.Context(), time.Second) + defer cancel() + require.Error(t, metrics.provider.ForceFlush(flushCtx), "a TLS client cannot export to a plaintext receiver") + select { + case <-capture.requests: + t.Fatal("an https:// destination reached a plaintext receiver") + default: + } +} + +// A process with no collector configured builds no exporter: an +// otlpmetricgrpc exporter with nothing configured dials its own default, +// localhost:4317, so the call refuses instead. +func TestEnableOTELMetricsRefusesWithoutAnEndpoint(t *testing.T) { + for name, value := range map[string]string{"unset": "", "blank": " "} { + t.Run(name, func(t *testing.T) { + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, value) + metrics, err := enableOTELMetrics(t.Context()) + require.ErrorContains(t, err, "no "+otlpEndpointVariable) + require.Nil(t, metrics) + }) + } +} + +// The exporter reads the variable only as an http:// or https:// URL. Any other +// form keeps its default, localhost:4317, without a word, so it is refused here, +// and the refusal does not echo the value, which may carry credentials. +func TestEnableOTELMetricsRefusesAnEndpointTheExporterCannotRead(t *testing.T) { + for _, value := range []string{"collector.example:4317", "ftp://collector.example:4317", "://bad", "secret-user:secret-pass@collector.example:4317"} { + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, value) + metrics, err := enableOTELMetrics(t.Context()) + require.ErrorContains(t, err, "http:// or https://", value) + require.NotContains(t, err.Error(), "secret-pass") + require.Nil(t, metrics) + } +} + +// A resource that cannot be built fails the whole call, and the call builds it +// before it creates the exporter: there is no exporter, and so no client +// connection, left running behind the failure. +func TestEnableOTELMetricsFailsWhenTheResourceCannotBeBuilt(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "an-attribute-with-no-value") + endpoint, _ := startMetricCapture(t) + collectorAt(t, "http", endpoint) + metrics, err := enableOTELMetrics(t.Context()) + require.Error(t, err) + require.Nil(t, metrics) } func startMetricCapture(t *testing.T) (string, *metricCapture) { diff --git a/module/services/accounts/code/telemetry_test.go b/module/services/accounts/code/telemetry_test.go new file mode 100644 index 000000000..905a868d6 --- /dev/null +++ b/module/services/accounts/code/telemetry_test.go @@ -0,0 +1,168 @@ +package main + +import ( + "context" + "net" + "os" + "os/exec" + "strings" + "testing" + "time" + + "github.com/codefly-dev/core/resources" + codefly "github.com/codefly-dev/sdk-go" + "go.opentelemetry.io/otel" + collectortracev1 "go.opentelemetry.io/proto/otlp/collector/trace/v1" + "google.golang.org/grpc" + + "github.com/codefly-dev/core/wool" + "github.com/stretchr/testify/require" +) + +// isolateOTLPEnvironment clears every OTLP variable the exporters read, so what a +// test sets is the whole configuration and nothing leaks in from the machine +// running it. +func isolateOTLPEnvironment(t *testing.T) { + t.Helper() + for _, name := range []string{ + "OTEL_EXPORTER_OTLP_ENDPOINT", + "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", + "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT", + "OTEL_EXPORTER_OTLP_CERTIFICATE", + "OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE", + "OTEL_EXPORTER_OTLP_METRICS_CERTIFICATE", + "OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE", + "OTEL_EXPORTER_OTLP_CLIENT_KEY", + "OTEL_EXPORTER_OTLP_INSECURE", + "OTEL_EXPORTER_OTLP_HEADERS", + } { + t.Setenv(name, "") + } +} + +// collectorAt configures the endpoint carrier for a loopback export test. +func collectorAt(t *testing.T, scheme, hostPort string) { + t.Helper() + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, scheme+"://"+hostPort) +} + +func TestOTLPEndpointConfigured(t *testing.T) { + isolateOTLPEnvironment(t) + require.False(t, otlpEndpointConfigured(), "unset") + t.Setenv(otlpEndpointVariable, " ") + require.False(t, otlpEndpointConfigured(), "a blank value is no endpoint: the metrics exporter trims it to unset") + t.Setenv(otlpEndpointVariable, "https://telemetry.example.com:443") + require.True(t, otlpEndpointConfigured()) +} + +// service.name is never a literal typed here: the environment wins, then the +// service's own Codefly identity. +func TestTelemetryResourceServiceName(t *testing.T) { + resolve := func(t *testing.T, identity string) string { + t.Helper() + res, err := telemetryResource(t.Context(), identity) + require.NoError(t, err) + value, _ := res.Set().Value("service.name") + return value.AsString() + } + + t.Run("falls back to the Codefly service identity", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.Equal(t, "saas-starter/accounts", resolve(t, "saas-starter/accounts")) + }) + t.Run("OTEL_SERVICE_NAME wins over the identity", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.Equal(t, "named-by-the-platform", resolve(t, "saas-starter/accounts")) + }) + t.Run("OTEL_RESOURCE_ATTRIBUTES can set it", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "service.name=attributed-by-the-platform,deployment.environment=test") + require.Equal(t, "attributed-by-the-platform", resolve(t, "saas-starter/accounts")) + }) + t.Run("without either, the SDK default rather than a literal", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.True(t, strings.HasPrefix(resolve(t, ""), "unknown_service"), resolve(t, "")) + }) +} + +// The identity is the one the SDK built its provider with, the same one wool +// stamps on this process's log lines — read from the provider, not from the +// runtime's environment carrier. +func TestCodeflyServiceNameIsTheSDKProviderIdentity(t *testing.T) { + carrying := func(unique string) context.Context { + return wool.New(t.Context(), &wool.Resource{Kind: "service", Unique: unique}).Inject(t.Context()) + } + require.Equal(t, "saas-starter/accounts", codeflyServiceName(carrying("saas-starter/accounts"))) + require.Equal(t, "accounts", codeflyServiceName(carrying("/accounts")), "an identity with no module") + require.Empty(t, codeflyServiceName(carrying("/")), "an identity with neither") + require.Empty(t, codeflyServiceName(t.Context()), "no provider: the SDK was not initialised") +} + +// SDK Init must deliver a real span with the supplied service identity; its +// public shutdown flushes the batch before the process exits. +func TestSDKExportsTracesAndFlushesOnShutdown(t *testing.T) { + // SDK Init snapshots runtime configuration and consumer identity globally. + // Run the real SDK lifecycle in a child, as the diagnostic tests do, so + // later configuration tests can still vary their own process environment. + const childMarker = "SDK_TELEMETRY_TEST_CHILD" + if os.Getenv(childMarker) != "1" { + child := exec.CommandContext(t.Context(), os.Args[0], "-test.run=^TestSDKExportsTracesAndFlushesOnShutdown$", "-test.timeout=15s") + child.Env = append(os.Environ(), childMarker+"=1") + output, err := child.CombinedOutput() + require.NoError(t, err, "SDK trace exporter child: %s", output) + return + } + isolateOTLPEnvironment(t) + t.Setenv(resources.ModulePrefix, "saas-starter") + t.Setenv(resources.ServicePrefix, "accounts") + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") + previousTracer := otel.GetTracerProvider() + previousTelemetry := wool.GetTelemetry() + t.Cleanup(func() { + _ = codefly.ShutdownTelemetry(context.Background()) + otel.SetTracerProvider(previousTracer) + wool.RegisterTelemetry(previousTelemetry) + }) + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + capture := &sdkTraceCapture{requests: make(chan *collectortracev1.ExportTraceServiceRequest, 1)} + server := grpc.NewServer() + collectortracev1.RegisterTraceServiceServer(server, capture) + go func() { _ = server.Serve(listener) }() + t.Cleanup(func() { server.Stop(); _ = listener.Close() }) + t.Setenv(otlpEndpointVariable, "http://"+listener.Addr().String()) + _, err = codefly.Init(t.Context()) + require.NoError(t, err) + _, span := otel.Tracer("test").Start(t.Context(), "operation") + span.End() + shutdownCtx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + require.NoError(t, codefly.ShutdownTelemetry(shutdownCtx)) + select { + case request := <-capture.requests: + require.Len(t, request.ResourceSpans, 1) + for _, attribute := range request.ResourceSpans[0].Resource.Attributes { + if attribute.Key == "service.name" { + require.Equal(t, "named-by-the-platform", attribute.Value.GetStringValue()) + return + } + } + t.Fatal("exported trace has no service.name") + case <-time.After(5 * time.Second): + t.Fatal("SDK shutdown did not deliver the span") + } +} + +type sdkTraceCapture struct { + collectortracev1.UnimplementedTraceServiceServer + requests chan *collectortracev1.ExportTraceServiceRequest +} + +func (capture *sdkTraceCapture) Export(_ context.Context, request *collectortracev1.ExportTraceServiceRequest) (*collectortracev1.ExportTraceServiceResponse, error) { + capture.requests <- request + return &collectortracev1.ExportTraceServiceResponse{}, nil +} diff --git a/module/services/accounts/code/work.go b/module/services/accounts/code/work.go index 8a23534f8..2eedf0c5a 100644 --- a/module/services/accounts/code/work.go +++ b/module/services/accounts/code/work.go @@ -41,7 +41,6 @@ import ( "github.com/codefly-dev/core/resources" "github.com/codefly-dev/core/wool" - wooltel "github.com/codefly-dev/core/wool/otel" "github.com/codefly-dev/core/workloadidentity/native" "github.com/codefly-dev/module-saas-starter/module/frontendpayload" codefly "github.com/codefly-dev/sdk-go" @@ -54,7 +53,20 @@ func init() { WithWork(doWork) } -func doWork(ctx context.Context) (Clean, error) { +func doWork(ctx context.Context) (clean Clean, err error) { + var otelMetricProvider *otelMetrics + // SDK Init owns tracing. Stop telemetry exporters on startup errors too, + // before a normal cleanup function can be returned. + defer func() { + if err != nil { + shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if otelMetricProvider != nil { + _ = otelMetricProvider.Shutdown(shutdownCtx) + } + _ = codefly.ShutdownTelemetry(shutdownCtx) + } + }() // `app diagnose …` answers and exits; it never starts the service. if code, ran := runRequestedDiagnostic(os.Args[1:]); ran { os.Exit(code) @@ -92,42 +104,13 @@ func doWork(ctx context.Context) (Clean, error) { return nil, err } - // When external observability is configured, OpenTelemetry always targets - // the in-graph collector. Codefly owns its host and port; the accounts - // process never reads or hardcodes a collector address. - var otelProvider *wooltel.Provider - var otelMetricProvider *otelMetrics - if observabilityEnabled() { - collectorNetwork, err := codefly.For(ctx). - Service("telemetry"). - Endpoint("grpc"). - API("grpc"). - ResolveNetworkInstance() - if err != nil { - return nil, fmt.Errorf("resolve telemetry collector through Codefly: %w", err) - } - p, oerr := wooltel.Enable( - wooltel.WithServiceName("saas-starter-api"), - wooltel.WithEndpoint(collectorNetwork.Host), - wooltel.WithInsecure(), - ) - if oerr != nil { - return nil, fmt.Errorf("configure OTEL tracing: %w", oerr) - } - otelProvider = p - w.Info("OTEL enabled", - wool.Field("endpoint", collectorNetwork.Host), - wool.Field("service.name", "saas-starter-api")) - metricProvider, oerr := enableOTELMetrics(ctx, "saas-starter-api", collectorNetwork.Host) + if otlpEndpointConfigured() { + metricProvider, oerr := enableOTELMetrics(ctx) if oerr != nil { - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) - _ = otelProvider.Shutdown(shutdownCtx) - cancel() - return nil, fmt.Errorf("configure OTEL metrics: %w", oerr) + w.Warn("OTEL metrics exporter could not be built; service continues without metrics export", wool.ErrField(oerr)) + } else { + otelMetricProvider = metricProvider } - otelMetricProvider = metricProvider - // codefly:gateway-route-exempt the OTEL scrape endpoint; the collector reaches it over the mesh and it must never be public - adapters.RegisterHTTPRoute("/metrics", otelMetricProvider.Handler()) } store, err := infra.NewPostgresStore(ctx) @@ -202,10 +185,10 @@ func doWork(ctx context.Context) (Clean, error) { } jobStore := infra.NewPostgresJobStore(jobWorkerPool) // Durable job-operations metrics are enabled only when OTEL metrics are, i.e. - // otelMetricProvider != nil (observabilityEnabled()). With observability off - // the global meter is a no-op, so building the monitor would poll the job - // store every interval to feed instruments nothing can read; keep it nil and - // let the Start/Shutdown nil-checks below skip it entirely. + // otelMetricProvider != nil (the cell has a collector). Without one the global + // meter is a no-op, so building the monitor would poll the job store every + // interval to feed instruments nothing can read; keep it nil and let the + // Start/Shutdown nil-checks below skip it entirely. jobOperationsMonitor, err := newDurableJobMetricsMonitor(otelMetricProvider != nil, jobStore) if err != nil { return nil, fmt.Errorf("configure durable job metrics: %w", err) @@ -1457,15 +1440,10 @@ func doWork(ctx context.Context) (Clean, error) { sw.Info("closing store") store.Close() sw.Info("store closed") - if otelProvider != nil { - sw.Info("flushing OTEL provider") - // Bounded shutdown — exporter has its own flush window - // (otlptrace's batcher). 5s is a sensible cap. - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) - defer cancel() - if err := otelProvider.Shutdown(shutdownCtx); err != nil { - sw.Warn("OTEL shutdown failed", wool.ErrField(err)) - } + traceShutdownCtx, cancelTraceShutdown := context.WithTimeout(context.Background(), 5*time.Second) + defer cancelTraceShutdown() + if err := codefly.ShutdownTelemetry(traceShutdownCtx); err != nil { + sw.Warn("OTEL shutdown failed", wool.ErrField(err)) } if otelMetricProvider != nil { sw.Info("flushing OTEL metrics provider") @@ -2079,13 +2057,9 @@ func workspaceEnv(configuration, key string) string { return os.Getenv(key) } -func observabilityEnabled() bool { - return strings.TrimSpace(workspaceEnv("observability", "OTEL_EXPORTER_OTLP_ENDPOINT")) != "" -} - // newDurableJobMetricsMonitor builds the durable job-operations metrics monitor // only when OTEL metrics are enabled (metricsEnabled mirrors a non-nil -// otelMetricProvider, i.e. observabilityEnabled()). When metrics are disabled the +// otelMetricProvider, i.e. the cell has a collector). When metrics are disabled the // global meter is a no-op, so a monitor would poll the job store every interval // only to record into instruments nothing can read; returning a nil monitor lets // the caller's Start/Shutdown nil-checks skip that background work entirely. diff --git a/module/services/accounts/code/work_auth_test.go b/module/services/accounts/code/work_auth_test.go index b16663f39..cf47a5cea 100644 --- a/module/services/accounts/code/work_auth_test.go +++ b/module/services/accounts/code/work_auth_test.go @@ -97,25 +97,6 @@ func TestWorkspaceEnvPreservesExactCodeflyConfigurationName(t *testing.T) { require.Equal(t, "legacy-normalized", workspaceEnv("internal-auth", key)) } -func TestObservabilityEnabledRequiresExternalOTLPEndpoint(t *testing.T) { - const key = "OTEL_EXPORTER_OTLP_ENDPOINT" - configurationKey := "CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__" + key - secretKey := "CODEFLY__WORKSPACE_SECRET_CONFIGURATION__OBSERVABILITY__" + key - t.Setenv(key, "") - t.Setenv(configurationKey, "") - t.Setenv(secretKey, "") - require.False(t, observabilityEnabled()) - - t.Setenv(key, " ") - require.False(t, observabilityEnabled()) - t.Setenv(key, "https://otel.example") - require.True(t, observabilityEnabled()) - - t.Setenv(key, "") - t.Setenv(configurationKey, "https://workspace-otel.example") - require.True(t, observabilityEnabled()) -} - func TestConfiguredMFAStepUpMaxAge(t *testing.T) { t.Setenv("CODEFLY__WORKSPACE_CONFIGURATION__SECURITY__MFA_STEP_UP_MAX_AGE", "") t.Setenv("CODEFLY__WORKSPACE_SECRET_CONFIGURATION__SECURITY__MFA_STEP_UP_MAX_AGE", "") diff --git a/module/services/accounts/service.codefly.yaml b/module/services/accounts/service.codefly.yaml index 4432f16f0..dbaf96c9e 100644 --- a/module/services/accounts/service.codefly.yaml +++ b/module/services/accounts/service.codefly.yaml @@ -22,9 +22,6 @@ service-dependencies: - name: store endpoints: - name: tcp - - name: telemetry - endpoints: - - name: grpc - name: vault endpoints: - name: http @@ -44,7 +41,6 @@ workspace-configuration-dependencies: - internal-auth - internal-transport - key-service - - observability - product-analytics - security - vault diff --git a/module/services/auth-gateway/code/configuration.go b/module/services/auth-gateway/code/configuration.go index 45b601c36..b557942fe 100644 --- a/module/services/auth-gateway/code/configuration.go +++ b/module/services/auth-gateway/code/configuration.go @@ -2,15 +2,10 @@ package main import ( "os" - "strings" codefly "github.com/codefly-dev/sdk-go" ) -func observabilityEnabled() bool { - return strings.TrimSpace(workspaceEnv("observability", "OTEL_EXPORTER_OTLP_ENDPOINT")) != "" -} - // workspaceEnv resolves Codefly workspace configuration and secret values // before falling back to a raw environment variable. Services must use this // boundary for declared workspace-configuration-dependencies; Codefly keeps diff --git a/module/services/auth-gateway/code/configuration_test.go b/module/services/auth-gateway/code/configuration_test.go index 8c66bd328..513f79245 100644 --- a/module/services/auth-gateway/code/configuration_test.go +++ b/module/services/auth-gateway/code/configuration_test.go @@ -27,22 +27,3 @@ func TestWorkspaceEnvResolvesConfigurationSecretAndFallback(t *testing.T) { t.Setenv(configurationKey, "configuration") require.Equal(t, "configuration", workspaceEnv("internal-auth", key)) } - -func TestObservabilityEnabledRequiresExternalOTLPEndpoint(t *testing.T) { - const key = "OTEL_EXPORTER_OTLP_ENDPOINT" - configurationKey := "CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__" + key - secretKey := "CODEFLY__WORKSPACE_SECRET_CONFIGURATION__OBSERVABILITY__" + key - t.Setenv(key, "") - t.Setenv(configurationKey, "") - t.Setenv(secretKey, "") - require.False(t, observabilityEnabled()) - - t.Setenv(key, " ") - require.False(t, observabilityEnabled()) - t.Setenv(key, "https://otel.example") - require.True(t, observabilityEnabled()) - - t.Setenv(key, "") - t.Setenv(configurationKey, "https://workspace-otel.example") - require.True(t, observabilityEnabled()) -} diff --git a/module/services/auth-gateway/code/go.mod b/module/services/auth-gateway/code/go.mod index 001480627..48f936d53 100644 --- a/module/services/auth-gateway/code/go.mod +++ b/module/services/auth-gateway/code/go.mod @@ -12,14 +12,12 @@ require ( github.com/golang-jwt/jwt/v5 v5.3.1 github.com/google/uuid v1.6.0 github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 - github.com/prometheus/client_golang v1.24.1 github.com/redis/go-redis/v9 v9.22.0 github.com/stretchr/testify v1.12.1 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 go.opentelemetry.io/contrib/instrumentation/runtime v0.71.0 go.opentelemetry.io/otel v1.47.0 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.46.0 - go.opentelemetry.io/otel/exporters/prometheus v0.68.0 go.opentelemetry.io/otel/metric v1.47.0 go.opentelemetry.io/otel/sdk v1.47.0 go.opentelemetry.io/otel/sdk/metric v1.47.0 @@ -39,7 +37,6 @@ require ( github.com/Masterminds/semver/v3 v3.5.0 // indirect github.com/VividCortex/ewma v1.2.0 // indirect github.com/antlr4-go/antlr/v4 v4.13.1 // indirect - github.com/beorn7/perks v1.0.1 // indirect github.com/blang/semver v3.5.1+incompatible // indirect github.com/cenkalti/backoff/v5 v5.0.3 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect @@ -78,17 +75,12 @@ require ( github.com/mattn/go-isatty v0.0.24 // indirect github.com/mattn/go-runewidth v0.0.27 // indirect github.com/mitchellh/mapstructure v1.5.0 // indirect - github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/oklog/ulid/v2 v2.1.2 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.1.1 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect - github.com/prometheus/client_model v0.6.3 // indirect - github.com/prometheus/common v0.71.0 // indirect - github.com/prometheus/otlptranslator v1.0.0 // indirect - github.com/prometheus/procfs v0.22.0 // indirect github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 // indirect github.com/shirou/gopsutil/v3 v3.24.5 // indirect github.com/shoenig/go-m1cpu v0.2.1 // indirect diff --git a/module/services/auth-gateway/code/go.sum b/module/services/auth-gateway/code/go.sum index 4f1cb276c..9c9a03077 100644 --- a/module/services/auth-gateway/code/go.sum +++ b/module/services/auth-gateway/code/go.sum @@ -20,8 +20,6 @@ github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1o github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4= github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= -github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= -github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ= github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= @@ -176,16 +174,12 @@ github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOl github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo= github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY= github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M= -github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk= -github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE= github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= -github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e h1:Q6MvJtQK/iRcRtzAscm/zF23XxJlbECiGPyRicsX+Ak= github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg= github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY= @@ -202,8 +196,6 @@ github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 h1:n6/2gBQ3RWajuToeY6ZtZTIKv2v7ThUy5KKusIT0yc0= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/oklog/ulid/v2 v2.1.2 h1:IEclFb9JNvzYA6MW2SCxbLzcHTVsfqm3PrqGQJH5zec= github.com/oklog/ulid/v2 v2.1.2/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= @@ -219,16 +211,6 @@ github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgm github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8= github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU= github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= -github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU= -github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE= -github.com/prometheus/client_model v0.6.3 h1:O0jaTVAYNxTHYInEPFJt5I3+sN8zqBtVMPTB1qyxiEo= -github.com/prometheus/client_model v0.6.3/go.mod h1:gpN5P9S7Rr6Yr92PiQ+Ixvhf6JZEkF1dnxsYL2aPBEM= -github.com/prometheus/common v0.71.0 h1:9KDAKb7Mj3HEVKyFCK6Dc/HIwlBzZIN2l7/lrHl3KK8= -github.com/prometheus/common v0.71.0/go.mod h1:CLJ5H8TEsGX8bl31BdMkfhIZ+QmZ9tBPPotUxUbfcmk= -github.com/prometheus/otlptranslator v1.0.0 h1:s0LJW/iN9dkIH+EnhiD3BlkkP5QVIUVEoIwkU+A6qos= -github.com/prometheus/otlptranslator v1.0.0/go.mod h1:vRYWnXvI6aWGpsdY/mOT/cbeVRBlPWtBNDb7kGR3uKM= -github.com/prometheus/procfs v0.22.0 h1:6q9+/JL9IKAPbCmBrv9n5O5Ty3NKnciV5X7YGw0oics= -github.com/prometheus/procfs v0.22.0/go.mod h1:CvmFr/GVhIjIvWJZW3tgkODBQMRIf0EyWMQLHCHab58= github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0= github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4= github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8= @@ -279,8 +261,6 @@ go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 h1:julhjPeUH/q/7hinbSd go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0/go.mod h1:Ao2mz688LH/tFf0yMAenidq6k2YNSx6SIY2q6jDACck= go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 h1:UFpxOpYPmMNUtOWhdb+nC1WELIgVf8z9higURrbzYU4= go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0/go.mod h1:YBGjxe3lt0jtXoEXxGrGhv/jM7oUBUDVK7zhAvNyEjc= -go.opentelemetry.io/otel/exporters/prometheus v0.68.0 h1:QOf2IftqQwITVRJpnn0M7M9ZCbgWfxz4P7i9C9yc2N4= -go.opentelemetry.io/otel/exporters/prometheus v0.68.0/go.mod h1:bgSvqu2TWGXiz7yr5UTMfObH8oqxJWHTnubQ3ef9BO4= go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0 h1:N3YQCxjxQ/bMjyc3heladfRm9t9RTksGQH8z4w6yU/0= go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0/go.mod h1:Mp8HOFqcaUyypCuGv9IhDdTHnJ56lSudSHMd+pVSCEA= go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lqCcI= diff --git a/module/services/auth-gateway/code/main.go b/module/services/auth-gateway/code/main.go index d90b1fd2a..96bbd6bc9 100644 --- a/module/services/auth-gateway/code/main.go +++ b/module/services/auth-gateway/code/main.go @@ -34,6 +34,14 @@ func main() { panic(fmt.Sprintf("codefly init failed: %v", err)) } + defer func() { + shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if err := codefly.ShutdownTelemetry(shutdownCtx); err != nil { + log.Printf("shutdown: OTEL tracing: %v", err) + } + }() + // Refuse before anything is served: a deployed gateway holding a shipped // placeholder, or a perimeter credential too short to be one, decides // perimeter membership on a published value. @@ -63,31 +71,11 @@ func main() { panic(fmt.Sprintf("Codefly workload endpoint is unavailable: %v", err)) } workloadPort := workloadNet.Port - var otelProvider *wooltel.Provider var otelMetricProvider *otelMetrics - if observabilityEnabled() { - collectorNetwork, err := codefly.For(ctx). - Service("telemetry"). - Endpoint("grpc"). - API("grpc"). - ResolveNetworkInstance() + if otlpEndpointConfigured() { + otelMetricProvider, err = enableOTELMetrics(ctx) if err != nil { - panic(fmt.Sprintf("resolve telemetry collector through Codefly: %v", err)) - } - otelProvider, err = wooltel.Enable( - wooltel.WithServiceName("saas-starter-auth-gateway"), - wooltel.WithEndpoint(collectorNetwork.Host), - wooltel.WithInsecure(), - ) - if err != nil { - panic(fmt.Sprintf("configure OTEL tracing: %v", err)) - } - otelMetricProvider, err = enableOTELMetrics(ctx, "saas-starter-auth-gateway", collectorNetwork.Host) - if err != nil { - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) - _ = otelProvider.Shutdown(shutdownCtx) - cancel() - panic(fmt.Sprintf("configure OTEL metrics: %v", err)) + log.Printf("OTEL metrics exporter could not be built; service continues without metrics export: %v", err) } } defer func() { @@ -98,11 +86,6 @@ func main() { log.Printf("shutdown: OTEL metrics: %v", err) } } - if otelProvider != nil { - if err := otelProvider.Shutdown(shutdownCtx); err != nil { - log.Printf("shutdown: OTEL tracing: %v", err) - } - } }() // Upstream HTTP URLs used by the gateway listener. diff --git a/module/services/auth-gateway/code/telemetry.go b/module/services/auth-gateway/code/telemetry.go new file mode 100644 index 000000000..74ba1ed51 --- /dev/null +++ b/module/services/auth-gateway/code/telemetry.go @@ -0,0 +1,87 @@ +package main + +import ( + "context" + "errors" + "net/url" + "os" + "strings" + + "github.com/codefly-dev/core/wool" + codefly "github.com/codefly-dev/sdk-go" + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/sdk/resource" +) + +// otlpEndpointVariable is the whole telemetry contract: OpenTelemetry's own name +// for the collector's address. The platform delivers it when the cell has a +// collector this workload may reach, and delivers nothing when it does not. +// +// Deliver it and the process exports traces and metrics over OTLP/gRPC; deliver +// none and it exports nothing over OTLP. Nothing here reads a state, a reason or +// a protocol: core's wool/otel reads this variable itself, tells a URL from a +// host:port, takes the transport from the scheme, validates the URL and refuses a +// portless one, and the metrics exporter reads it from the same environment. +const otlpEndpointVariable = "OTEL_EXPORTER_OTLP_ENDPOINT" + +// otlpEndpointConfigured reports whether the process was given a collector. A +// whitespace-only value is none: the OpenTelemetry metrics exporter trims what it +// reads and treats an empty result as unset, and an exporter with nothing +// configured dials its own default, localhost:4317. +func otlpEndpointConfigured() bool { + return strings.TrimSpace(os.Getenv(otlpEndpointVariable)) != "" +} + +// requireMetricsEndpoint refuses what the metrics exporter cannot read. It reads +// the variable only as an http:// or https:// URL, and on anything else keeps its +// default and dials localhost:4317 without saying so, so a value of another shape +// is rejected here instead of creating a metrics exporter at a default address. +func requireMetricsEndpoint() error { + if !otlpEndpointConfigured() { + return errors.New("no " + otlpEndpointVariable + " to export metrics to") + } + parsed, err := url.Parse(strings.TrimSpace(os.Getenv(otlpEndpointVariable))) + if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") { + return errors.New(otlpEndpointVariable + " must be an http:// or https:// URL for metrics to be exported; the exporter ignores any other form and would fall back to its own default address") + } + return nil +} + +// telemetryResource builds the metrics resource. service.name is +// whatever the environment says — OTEL_SERVICE_NAME, or service.name in +// OTEL_RESOURCE_ATTRIBUTES — and otherwise the service identity the Codefly SDK +// holds. No name is typed here: a literal would override the environment, so the +// platform's identity could never reach the backend. +func telemetryResource(ctx context.Context, identity string) (*resource.Resource, error) { + options := []resource.Option{resource.WithService()} + if identity != "" { + options = append(options, resource.WithAttributes(attribute.String("service.name", identity))) + } + // Last, so what the environment declares wins over the identity above. + options = append(options, resource.WithFromEnv()) + return resource.New(ctx, options...) +} + +// currentTelemetryResource is telemetryResource for this process. +func currentTelemetryResource(ctx context.Context) (*resource.Resource, error) { + return telemetryResource(ctx, codeflyServiceName(codefly.Context())) +} + +// codeflyServiceName is the service identity the Codefly SDK holds: the +// `module/service` identity of the provider it built at Init, which wool stamps on +// every log line from this process. It is empty when the SDK has not been +// initialised. The SDK exposes no accessor of its own, and the environment +// carrier it reads this from is not product code's to read, so this asks the +// provider. +func codeflyServiceName(ctx context.Context) string { + provider, ok := ctx.Value(wool.ProviderKey).(*wool.Provider) + if !ok || provider == nil { + return "" + } + source := provider.Get(ctx).Source() + if source == nil { + return "" + } + // An identity with no module reads `/service`; with neither, `/`. + return strings.Trim(strings.TrimSpace(source.Unique), "/") +} diff --git a/module/services/auth-gateway/code/telemetry_metrics.go b/module/services/auth-gateway/code/telemetry_metrics.go index ad2e7f8e0..63610a816 100644 --- a/module/services/auth-gateway/code/telemetry_metrics.go +++ b/module/services/auth-gateway/code/telemetry_metrics.go @@ -6,56 +6,48 @@ import ( runtimemetrics "runtime/metrics" "time" - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" otelruntime "go.opentelemetry.io/contrib/instrumentation/runtime" "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc" - otelprometheus "go.opentelemetry.io/otel/exporters/prometheus" otelmetric "go.opentelemetry.io/otel/metric" "go.opentelemetry.io/otel/sdk/metric" - "go.opentelemetry.io/otel/sdk/resource" ) +// otelMetrics owns the process's MeterProvider. Metrics leave by one path only: +// an OTLP push to the collector OTEL_EXPORTER_OTLP_ENDPOINT names, which is the +// record for traces and metrics alike. There is no scrape endpoint to mount, +// expose or exempt. type otelMetrics struct { provider *metric.MeterProvider - handler http.Handler } -func enableOTELMetrics(ctx context.Context, serviceName, endpoint string) (*otelMetrics, error) { - exporter, err := otlpmetricgrpc.New( - ctx, - otlpmetricgrpc.WithEndpoint(endpoint), - otlpmetricgrpc.WithInsecure(), - ) - if err != nil { +// enableOTELMetrics starts the process's OTLP metrics push to the collector the +// environment names. It is only for a process that was given one: with no +// OTEL_EXPORTER_OTLP_ENDPOINT it refuses rather than build an exporter, because +// an otlpmetricgrpc exporter with nothing configured dials its own default, +// localhost:4317 over TLS, and a process that exports to nowhere reports success. +func enableOTELMetrics(ctx context.Context) (*otelMetrics, error) { + if err := requireMetricsEndpoint(); err != nil { return nil, err } - registry := prometheus.NewRegistry() - scrapeExporter, err := otelprometheus.New( - otelprometheus.WithRegisterer(registry), - otelprometheus.WithResourceAsConstantLabels( - attribute.NewAllowKeysFilter("service.name"), - ), - ) + // The resource comes before the exporter, so a failure here leaves nothing + // to shut down: an exporter holds a client connection that only Shutdown + // releases. + res, err := currentTelemetryResource(ctx) if err != nil { return nil, err } - res, err := resource.New( - ctx, - resource.WithService(), - resource.WithFromEnv(), - resource.WithAttributes(attribute.String("service.name", serviceName)), - ) + // The exporter reads the collector from OTEL_EXPORTER_OTLP_ENDPOINT itself and + // takes transport from the URL scheme: the operations intake uses https:// + // with TLS; http:// remains available for local plaintext receivers. + exporter, err := otlpmetricgrpc.New(ctx) if err != nil { return nil, err } provider := metric.NewMeterProvider( metric.WithResource(res), metric.WithReader(metric.NewPeriodicReader(exporter, metric.WithInterval(30*time.Second))), - metric.WithReader(scrapeExporter), ) if err := otelruntime.Start(otelruntime.WithMeterProvider(provider)); err != nil { _ = provider.Shutdown(ctx) @@ -66,10 +58,7 @@ func enableOTELMetrics(ctx context.Context, serviceName, endpoint string) (*otel return nil, err } otel.SetMeterProvider(provider) - return &otelMetrics{ - provider: provider, - handler: promhttp.HandlerFor(registry, promhttp.HandlerOpts{}), - }, nil + return &otelMetrics{provider: provider}, nil } func startGCActivityMetrics(provider *metric.MeterProvider) error { @@ -103,18 +92,14 @@ func startGCActivityMetrics(provider *metric.MeterProvider) error { return err } +// newGatewayHTTPHandler instruments the gateway's HTTP listener when metrics are +// on. It adds no route of its own: the listener serves the gateway's routes and +// nothing else, with or without telemetry. func newGatewayHTTPHandler(gateway http.Handler, metrics *otelMetrics) http.Handler { if metrics == nil { return gateway } - mux := http.NewServeMux() - mux.Handle("GET /metrics", metrics.Handler()) - mux.Handle("/", otelhttp.NewHandler(gateway, "auth-gateway.gateway")) - return mux -} - -func (m *otelMetrics) Handler() http.Handler { - return m.handler + return otelhttp.NewHandler(gateway, "auth-gateway.gateway") } func (m *otelMetrics) Shutdown(ctx context.Context) error { diff --git a/module/services/auth-gateway/code/telemetry_metrics_test.go b/module/services/auth-gateway/code/telemetry_metrics_test.go index 6e461dae4..5269bf211 100644 --- a/module/services/auth-gateway/code/telemetry_metrics_test.go +++ b/module/services/auth-gateway/code/telemetry_metrics_test.go @@ -39,8 +39,10 @@ func (c *authMetricCapture) Export( func TestAuthGatewayTelemetryExportsGatewayAndGRPCRED(t *testing.T) { endpoint, capture := startAuthMetricCapture(t) t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "service.instance.id=auth-test-instance") + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") previousMeterProvider := otel.GetMeterProvider() - metrics, err := enableOTELMetrics(t.Context(), "test-auth-gateway", endpoint) + collectorAt(t, "http", endpoint) + metrics, err := enableOTELMetrics(t.Context()) require.NoError(t, err) require.NotNil(t, metrics) t.Cleanup(func() { @@ -57,19 +59,10 @@ func TestAuthGatewayTelemetryExportsGatewayAndGRPCRED(t *testing.T) { require.Equal(t, http.StatusNotFound, getStatus(t, httpServer.URL+"/missing")) runtime.GC() - response, err := http.Get(httpServer.URL + "/metrics") - require.NoError(t, err) - t.Cleanup(func() { _ = response.Body.Close() }) - require.Equal(t, http.StatusOK, response.StatusCode) - prometheusBody, err := io.ReadAll(response.Body) - require.NoError(t, err) - require.Contains(t, string(prometheusBody), "http_server_request_duration_seconds_count") - require.Contains(t, string(prometheusBody), `http_route="/health"`) - require.Contains(t, string(prometheusBody), `service_name="test-auth-gateway"`) - require.NoError(t, metrics.provider.ForceFlush(t.Context())) request := receiveAuthMetrics(t, capture.requests) require.Equal(t, "auth-test-instance", authResourceAttribute(request, "service.instance.id")) + require.Equal(t, "named-by-the-platform", authResourceAttribute(request, "service.name")) metricNames := authExportedMetricNames(request) require.Contains(t, metricNames, "go.goroutine.count") require.Contains(t, metricNames, "go.gc.cycle.count") @@ -88,19 +81,26 @@ func TestAuthGatewayTelemetryExportsGatewayAndGRPCRED(t *testing.T) { require.Equal(t, uint64(2), httpRoutes["/health"]) } -func TestGatewayHandlerHasNoMetricsRouteWhenTelemetryIsDisabled(t *testing.T) { +// Metrics leave by OTLP push alone: the gateway's listener serves its own routes +// and no scrape endpoint, whether or not telemetry is on. +func TestGatewayHandlerHasNoMetricsRoute(t *testing.T) { matcher := NewRouteMatcher([]*RouteEntry{{ Service: "self", Method: http.MethodGet, Path: "/health", }}, nil) gateway := NewGateway(nil, matcher, nil, nil, newFakeSolutionRegistry(), newFakeClientRegistry()) - request := httptest.NewRequest(http.MethodGet, "/metrics", nil) - recorder := httptest.NewRecorder() - - newGatewayHTTPHandler(gateway, nil).ServeHTTP(recorder, request) - require.Equal(t, http.StatusNotFound, recorder.Code) + for name, metrics := range map[string]*otelMetrics{ + "telemetry disabled": nil, + "telemetry enabled": {}, + } { + t.Run(name, func(t *testing.T) { + recorder := httptest.NewRecorder() + newGatewayHTTPHandler(gateway, metrics).ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/metrics", nil)) + require.Equal(t, http.StatusNotFound, recorder.Code) + }) + } } func authInt64SumValue(request *collectormetricsv1.ExportMetricsServiceRequest, name string) int64 { @@ -135,6 +135,75 @@ func authFloat64SumValue(request *collectormetricsv1.ExportMetricsServiceRequest return value } +// The exporter reads its transport off the URL's scheme, so an https:// URL is +// TLS and cannot talk to a plaintext receiver. A constant WithInsecure() would +// have let it. +func TestEnableOTELMetricsDerivesTransportFromTheURLScheme(t *testing.T) { + endpoint, capture := startAuthMetricCapture(t) + previousMeterProvider := otel.GetMeterProvider() + collectorAt(t, "https", endpoint) + metrics, err := enableOTELMetrics(t.Context()) + require.NoError(t, err) + require.NotNil(t, metrics) + t.Cleanup(func() { + shutdownCtx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + _ = metrics.Shutdown(shutdownCtx) // the flush it attempts is the failure under test + otel.SetMeterProvider(previousMeterProvider) + }) + + flushCtx, cancel := context.WithTimeout(t.Context(), time.Second) + defer cancel() + require.Error(t, metrics.provider.ForceFlush(flushCtx), "a TLS client cannot export to a plaintext receiver") + select { + case <-capture.requests: + t.Fatal("an https:// destination reached a plaintext receiver") + default: + } +} + +// A process with no collector configured builds no exporter: an +// otlpmetricgrpc exporter with nothing configured dials its own default, +// localhost:4317, so the call refuses instead. +func TestEnableOTELMetricsRefusesWithoutAnEndpoint(t *testing.T) { + for name, value := range map[string]string{"unset": "", "blank": " "} { + t.Run(name, func(t *testing.T) { + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, value) + metrics, err := enableOTELMetrics(t.Context()) + require.ErrorContains(t, err, "no "+otlpEndpointVariable) + require.Nil(t, metrics) + }) + } +} + +// The exporter reads the variable only as an http:// or https:// URL. Any other +// form keeps its default, localhost:4317, without a word, so it is refused here, +// and the refusal does not echo the value, which may carry credentials. +func TestEnableOTELMetricsRefusesAnEndpointTheExporterCannotRead(t *testing.T) { + for _, value := range []string{"collector.example:4317", "ftp://collector.example:4317", "://bad", "secret-user:secret-pass@collector.example:4317"} { + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, value) + metrics, err := enableOTELMetrics(t.Context()) + require.ErrorContains(t, err, "http:// or https://", value) + require.NotContains(t, err.Error(), "secret-pass") + require.Nil(t, metrics) + } +} + +// A resource that cannot be built fails the whole call, and the call builds it +// before it creates the exporter: there is no exporter, and so no client +// connection, left running behind the failure. +func TestEnableOTELMetricsFailsWhenTheResourceCannotBeBuilt(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "an-attribute-with-no-value") + endpoint, _ := startAuthMetricCapture(t) + collectorAt(t, "http", endpoint) + metrics, err := enableOTELMetrics(t.Context()) + require.Error(t, err) + require.Nil(t, metrics) +} + func startAuthMetricCapture(t *testing.T) (string, *authMetricCapture) { t.Helper() listener, err := net.Listen("tcp", "127.0.0.1:0") diff --git a/module/services/auth-gateway/code/telemetry_test.go b/module/services/auth-gateway/code/telemetry_test.go new file mode 100644 index 000000000..d79b341e1 --- /dev/null +++ b/module/services/auth-gateway/code/telemetry_test.go @@ -0,0 +1,168 @@ +package main + +import ( + "context" + "net" + "os" + "os/exec" + "strings" + "testing" + "time" + + "github.com/codefly-dev/core/resources" + codefly "github.com/codefly-dev/sdk-go" + "go.opentelemetry.io/otel" + collectortracev1 "go.opentelemetry.io/proto/otlp/collector/trace/v1" + "google.golang.org/grpc" + + "github.com/codefly-dev/core/wool" + "github.com/stretchr/testify/require" +) + +// isolateOTLPEnvironment clears every OTLP variable the exporters read, so what a +// test sets is the whole configuration and nothing leaks in from the machine +// running it. +func isolateOTLPEnvironment(t *testing.T) { + t.Helper() + for _, name := range []string{ + "OTEL_EXPORTER_OTLP_ENDPOINT", + "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", + "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT", + "OTEL_EXPORTER_OTLP_CERTIFICATE", + "OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE", + "OTEL_EXPORTER_OTLP_METRICS_CERTIFICATE", + "OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE", + "OTEL_EXPORTER_OTLP_CLIENT_KEY", + "OTEL_EXPORTER_OTLP_INSECURE", + "OTEL_EXPORTER_OTLP_HEADERS", + } { + t.Setenv(name, "") + } +} + +// collectorAt configures the endpoint carrier for a loopback export test. +func collectorAt(t *testing.T, scheme, hostPort string) { + t.Helper() + isolateOTLPEnvironment(t) + t.Setenv(otlpEndpointVariable, scheme+"://"+hostPort) +} + +func TestOTLPEndpointConfigured(t *testing.T) { + isolateOTLPEnvironment(t) + require.False(t, otlpEndpointConfigured(), "unset") + t.Setenv(otlpEndpointVariable, " ") + require.False(t, otlpEndpointConfigured(), "a blank value is no endpoint: the metrics exporter trims it to unset") + t.Setenv(otlpEndpointVariable, "https://telemetry.example.com:443") + require.True(t, otlpEndpointConfigured()) +} + +// service.name is never a literal typed here: the environment wins, then the +// service's own Codefly identity. +func TestTelemetryResourceServiceName(t *testing.T) { + resolve := func(t *testing.T, identity string) string { + t.Helper() + res, err := telemetryResource(t.Context(), identity) + require.NoError(t, err) + value, _ := res.Set().Value("service.name") + return value.AsString() + } + + t.Run("falls back to the Codefly service identity", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.Equal(t, "saas-starter/auth-gateway", resolve(t, "saas-starter/auth-gateway")) + }) + t.Run("OTEL_SERVICE_NAME wins over the identity", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.Equal(t, "named-by-the-platform", resolve(t, "saas-starter/auth-gateway")) + }) + t.Run("OTEL_RESOURCE_ATTRIBUTES can set it", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "service.name=attributed-by-the-platform,deployment.environment=test") + require.Equal(t, "attributed-by-the-platform", resolve(t, "saas-starter/auth-gateway")) + }) + t.Run("without either, the SDK default rather than a literal", func(t *testing.T) { + t.Setenv("OTEL_SERVICE_NAME", "") + t.Setenv("OTEL_RESOURCE_ATTRIBUTES", "") + require.True(t, strings.HasPrefix(resolve(t, ""), "unknown_service"), resolve(t, "")) + }) +} + +// The identity is the one the SDK built its provider with, the same one wool +// stamps on this process's log lines — read from the provider, not from the +// runtime's environment carrier. +func TestCodeflyServiceNameIsTheSDKProviderIdentity(t *testing.T) { + carrying := func(unique string) context.Context { + return wool.New(t.Context(), &wool.Resource{Kind: "service", Unique: unique}).Inject(t.Context()) + } + require.Equal(t, "saas-starter/auth-gateway", codeflyServiceName(carrying("saas-starter/auth-gateway"))) + require.Equal(t, "auth-gateway", codeflyServiceName(carrying("/auth-gateway")), "an identity with no module") + require.Empty(t, codeflyServiceName(carrying("/")), "an identity with neither") + require.Empty(t, codeflyServiceName(t.Context()), "no provider: the SDK was not initialised") +} + +// SDK Init must deliver a real span with the supplied service identity; its +// public shutdown flushes the batch before the process exits. +func TestSDKExportsTracesAndFlushesOnShutdown(t *testing.T) { + // SDK Init snapshots runtime configuration and consumer identity globally. + // Run the real SDK lifecycle in a child, as the diagnostic tests do, so + // later configuration tests can still vary their own process environment. + const childMarker = "SDK_TELEMETRY_TEST_CHILD" + if os.Getenv(childMarker) != "1" { + child := exec.CommandContext(t.Context(), os.Args[0], "-test.run=^TestSDKExportsTracesAndFlushesOnShutdown$", "-test.timeout=15s") + child.Env = append(os.Environ(), childMarker+"=1") + output, err := child.CombinedOutput() + require.NoError(t, err, "SDK trace exporter child: %s", output) + return + } + isolateOTLPEnvironment(t) + t.Setenv(resources.ModulePrefix, "saas-starter") + t.Setenv(resources.ServicePrefix, "auth-gateway") + t.Setenv("OTEL_SERVICE_NAME", "named-by-the-platform") + previousTracer := otel.GetTracerProvider() + previousTelemetry := wool.GetTelemetry() + t.Cleanup(func() { + _ = codefly.ShutdownTelemetry(context.Background()) + otel.SetTracerProvider(previousTracer) + wool.RegisterTelemetry(previousTelemetry) + }) + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + capture := &sdkTraceCapture{requests: make(chan *collectortracev1.ExportTraceServiceRequest, 1)} + server := grpc.NewServer() + collectortracev1.RegisterTraceServiceServer(server, capture) + go func() { _ = server.Serve(listener) }() + t.Cleanup(func() { server.Stop(); _ = listener.Close() }) + t.Setenv(otlpEndpointVariable, "http://"+listener.Addr().String()) + _, err = codefly.Init(t.Context()) + require.NoError(t, err) + _, span := otel.Tracer("test").Start(t.Context(), "operation") + span.End() + shutdownCtx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + require.NoError(t, codefly.ShutdownTelemetry(shutdownCtx)) + select { + case request := <-capture.requests: + require.Len(t, request.ResourceSpans, 1) + for _, attribute := range request.ResourceSpans[0].Resource.Attributes { + if attribute.Key == "service.name" { + require.Equal(t, "named-by-the-platform", attribute.Value.GetStringValue()) + return + } + } + t.Fatal("exported trace has no service.name") + case <-time.After(5 * time.Second): + t.Fatal("SDK shutdown did not deliver the span") + } +} + +type sdkTraceCapture struct { + collectortracev1.UnimplementedTraceServiceServer + requests chan *collectortracev1.ExportTraceServiceRequest +} + +func (capture *sdkTraceCapture) Export(_ context.Context, request *collectortracev1.ExportTraceServiceRequest) (*collectortracev1.ExportTraceServiceResponse, error) { + capture.requests <- request + return &collectortracev1.ExportTraceServiceResponse{}, nil +} diff --git a/module/services/auth-gateway/service.codefly.yaml b/module/services/auth-gateway/service.codefly.yaml index 753fd327d..24c187049 100644 --- a/module/services/auth-gateway/service.codefly.yaml +++ b/module/services/auth-gateway/service.codefly.yaml @@ -23,9 +23,6 @@ service-dependencies: - name: cache endpoints: - name: write - - name: telemetry - endpoints: - - name: grpc workspace-configuration-dependencies: # APP_BASE_URL: this host's own public origin. The gateway publishes it in # the RFC 9728 document for a solution's MCP endpoint and requires a token's @@ -37,7 +34,6 @@ workspace-configuration-dependencies: - gateway - gateway-trust - internal-auth - - observability - security endpoints: - name: grpc diff --git a/module/services/frontend/code/next-env.d.ts b/module/services/frontend/code/next-env.d.ts index a419cbe4e..ce4e94a6b 100644 --- a/module/services/frontend/code/next-env.d.ts +++ b/module/services/frontend/code/next-env.d.ts @@ -1,7 +1,7 @@ /// /// -import "./.next/dev/types/routes.d.ts"; -import "./.next/dev/types/root-params.d.ts"; +import "./.next/types/routes.d.ts"; +import "./.next/types/root-params.d.ts"; // NOTE: This file should not be edited // see https://nextjs.org/docs/app/api-reference/config/typescript for more information. diff --git a/module/services/frontend/code/package-lock.json b/module/services/frontend/code/package-lock.json index b8ba2435b..95b7b307c 100644 --- a/module/services/frontend/code/package-lock.json +++ b/module/services/frontend/code/package-lock.json @@ -30,7 +30,7 @@ "@fontsource/source-serif-4": "^5.3.0", "@hookform/resolvers": "^3.9.0", "@module-federation/runtime": "^2.8.2", - "@sentry/nextjs": "^10.50.0", + "@sentry/nextjs": "10.75.2", "@simplewebauthn/browser": "^13.3.0", "@tanstack/react-query": "^5.60.0", "@tanstack/react-table": "^8.21.3", @@ -38,7 +38,7 @@ "codefly": "https://github.com/codefly-dev/sdk-js/archive/9f3bb78682a0c2b9a70631d35c2ae0094c281ac5.tar.gz", "js-yaml": "^4.3.2", "lucide-react": "^1.8.0", - "next": "^16.2.12", + "next": "16.4.1", "react": "^19.2.8", "react-dom": "^19.2.8", "react-hook-form": "^7.54.0", @@ -64,7 +64,7 @@ "ajv": "^8.17.1", "babel-plugin-react-compiler": "1.0.0", "eslint": "^9.39.5", - "eslint-config-next": "^16.2.12", + "eslint-config-next": "16.4.1", "happy-dom": "^20.8.9", "jsdom": "^29.0.2", "msw": "^2.7.0", @@ -2569,15 +2569,15 @@ } }, "node_modules/@next/env": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/env/-/env-16.3.8.tgz", - "integrity": "sha512-Al9zqHVV7TJv0eFuOU4U7Lvv74PTih4Ch63sk2xCIpSTkE3udFnaOcnzP2lQVymiL7yS9Cj2iClUXlR3EQ5sEw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/env/-/env-16.4.1.tgz", + "integrity": "sha512-5sUUrrSvs/2/2LnLoesybMGx12EtPB46fdZ6RhAw0XDGSPxU2LFyQBB7yuEiOFFJPKmBY6+q2jTr1/AMbnp0/g==", "license": "MIT" }, "node_modules/@next/eslint-plugin-next": { - "version": "16.3.4", - "resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.3.4.tgz", - "integrity": "sha512-szW9y2Aumu4z88YXfTzcFsgUAg2k64uzbtcO5L9f1AKS4w/GUKJcbFllRflROVyNPgJtGOnvNxiyp3v6b+prIA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.4.1.tgz", + "integrity": "sha512-2io6M+wAqe7nT5mt2gEVy6FOzye2DmwhjaYCpSvHfrIKL11SXNTwSARxpPHJacRmfgSBSDnNmnEXSp0DaJ23BQ==", "dev": true, "license": "MIT", "dependencies": { @@ -2618,9 +2618,9 @@ } }, "node_modules/@next/swc-darwin-arm64": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.3.8.tgz", - "integrity": "sha512-2JPRMh2nmQG5CiL7cXGL9AGwnPWJQ//cTtAUCT+w511QHk79SYz3LGv/pc5X643B/WEO0rvu3Yww0hqwt3kgeA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.4.1.tgz", + "integrity": "sha512-U+hQ5Gj3fpzf4fveTIF3dpUFYhaSW8vHhdAnbiIomzXzg7boreYFgBqprepXFtW4sw+aTttiNmg0gew/iFNhbg==", "cpu": [ "arm64" ], @@ -2634,9 +2634,9 @@ } }, "node_modules/@next/swc-darwin-x64": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.3.8.tgz", - "integrity": "sha512-GZtCCOBKJ4leVIT/Th0llWKhD1ca92lzbQiS5R5ON9QkoiFnilFsebDae1JU2a3HWoKMEmEZWGs1AGLavVM72Q==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.4.1.tgz", + "integrity": "sha512-/hqC3V1XLUjhMASf/WridKdO9FXoEgN3HBWmtKbj00MPgnltutfraWJ3bYTpCRIgEIpbjJyOVLpYgAhh/BkdcQ==", "cpu": [ "x64" ], @@ -2650,9 +2650,9 @@ } }, "node_modules/@next/swc-linux-arm64-gnu": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.3.8.tgz", - "integrity": "sha512-O659ygeQYqneJ1fBKMpFxIFqYkYswu8IAS1OCKK/4f3ZgJJm1dRz4fVJZRi/kLLWjnBKnebOePA4WNv+sV1pVA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.4.1.tgz", + "integrity": "sha512-GHzncHbcaS9g95kcvTNtKQW3aCLoR+xc3Jm7qrkpk1R+D2BjMHIPyvQGRPKCdMX95BsH/421jBUK06q9tEwzjw==", "cpu": [ "arm64" ], @@ -2669,9 +2669,9 @@ } }, "node_modules/@next/swc-linux-arm64-musl": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.3.8.tgz", - "integrity": "sha512-dSjKSyWpzxoO1d3DIZZcP4XJcNaKeLmxQMFOiYl5vuBRMmweIqnAhty8tAmRsvTss779cK1FtYnDMj40e4TQlg==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.4.1.tgz", + "integrity": "sha512-1SuJyDxW/PR9dHJat/ZRgcYLhuSZ+1psP4DlKb2tKIuXbjygqpLrJWRSq+onZQBv8LcmU67EIrmb6c3XTaJLjw==", "cpu": [ "arm64" ], @@ -2688,9 +2688,9 @@ } }, "node_modules/@next/swc-linux-x64-gnu": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.3.8.tgz", - "integrity": "sha512-lbqOuz3RPRcv+o9msNsJw5x4+Y1ZwPTs6vmL6DCf7i0fZfvng/F59wyeDwqHIvV0mK//RBy/jJkZ+nCKsSMXjQ==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.4.1.tgz", + "integrity": "sha512-Istng1hMGp9TGO4lUqeKLd70MPN/OOkucVTP2Nmj+gLkwBgmPqUw8CJYBCC7nbl/B/23bBCcjCI5Hsh6hazC1A==", "cpu": [ "x64" ], @@ -2707,9 +2707,9 @@ } }, "node_modules/@next/swc-linux-x64-musl": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.3.8.tgz", - "integrity": "sha512-+316WswI8ScVgZeUd+1KGaXkHhaYQzCjvH/05TZSpJ8zBizb1a4G7DtO7F12jcBIqMOtsz9ji1t48fmKtzqsGA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.4.1.tgz", + "integrity": "sha512-aSC9wMNiZRRFVpPkPWBzRwLQxIGm/CiQcXyFF1XNcgodb+uMxdk7aSSGGpDXCbnvZdf4itjHxpOd5h7i7zaiYw==", "cpu": [ "x64" ], @@ -2726,9 +2726,9 @@ } }, "node_modules/@next/swc-win32-arm64-msvc": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.3.8.tgz", - "integrity": "sha512-ji0gd4kMYUxO+1fJBIbiBVRCjzG/lloiyCccnlebvb1ZJ5qXCPZqYg4Jl1DrrixnWNMKylzgpmMWx0yNDYXlzw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.4.1.tgz", + "integrity": "sha512-t/myvwFDlypMUjRvbBf8qLPyuaB0iXYYdBtwMt6DM6f+60wtGr4bSYholdPqNeV5VUXZhrpMwfza829kmhS3iA==", "cpu": [ "arm64" ], @@ -2742,9 +2742,9 @@ } }, "node_modules/@next/swc-win32-x64-msvc": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.3.8.tgz", - "integrity": "sha512-WcTlaKt/TWkh5kUjdJcUmB1XgZ+1c6fz4Y9fDHL73YNSdGaUWjceeWrrlwF0nv19iABYWC4iAq1oX1w4Bn0vfg==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.4.1.tgz", + "integrity": "sha512-6ICqPo9awCkrtYNPhwkx1uU7rlFWrZRrIboDHS2LU+jv088UBYQtmJ25h6pCp27/+FZjF6BQ4Te4ErugyMeuBg==", "cpu": [ "x64" ], @@ -3522,30 +3522,54 @@ } }, "node_modules/@sentry/browser": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.73.0.tgz", - "integrity": "sha512-HqTe1S5RrWLufhX2LaFP3yNoMxfNDroh120bq1zdGHZfFDBMJQ0CDXxHO+L4UJfQ5dWdCCzWbXIAiZuWGa/DFQ==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-10.75.2.tgz", + "integrity": "sha512-2Q8Bd6aTIHmqWL/rGDZdr1JTGW2fYPXf+lMyhHp4CX6OE7gMaaI5ZUwQWgVelqUOvKkOZHHk0wo6K8+ExdwawQ==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.73.0", + "@sentry/browser-utils": "10.75.2", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0", - "@sentry/feedback": "10.73.0", - "@sentry/replay": "10.73.0", - "@sentry/replay-canvas": "10.73.0" + "@sentry/core": "10.75.2", + "@sentry/feedback": "10.75.2", + "@sentry/replay": "10.75.2", + "@sentry/replay-canvas": "10.75.2" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/browser-utils": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.73.0.tgz", - "integrity": "sha512-qQygxJZ+RV779+iL1+lrJ4f4sZLgbgW0/JWPNp0YlcEAE62yCsdKbqoTEjB/EugdS4mSjBMX0chZC6rblu2Ycw==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/browser-utils/-/browser-utils-10.75.2.tgz", + "integrity": "sha512-U9RIJE0vZHWEnfEpJAa/SQ60MsNSUaxww4MVymNLwJ26BrLFz9jLhVIW9bXcx0i1iwK+KQbIv6ebPDOnCfYUPQ==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/browser-utils/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/browser/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" @@ -3798,34 +3822,46 @@ } }, "node_modules/@sentry/feedback": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.73.0.tgz", - "integrity": "sha512-D6nSngX+e46Mae2/oh2bxBvxNK1z2NERbuMAhB5sx9x4xMBWIyGnYYTECehvEqV9+AqGAgxxhOZoYIG3AmRwww==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/feedback/-/feedback-10.75.2.tgz", + "integrity": "sha512-/nrtMVSxOqBAigMWxIQS+Y8OqQ27ty+isMh6+X96kfAp/SaGrNZ425SZxKwq0mwHS0WnVkF+rE5dJnbbi56XSA==", "license": "MIT", "dependencies": { - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/feedback/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/nextjs": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.73.0.tgz", - "integrity": "sha512-lZFlQkafIHnYKTo0QCY+VqFC7rEt/SRAVrTCKyM+r1yIAw+iIfU1qM2QT7xn71+t0U520I0xBJ2O+YzqLYXJIQ==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/nextjs/-/nextjs-10.75.2.tgz", + "integrity": "sha512-ilzb3E7Kaskx/RQ355kCkxLxrHBtYS67pe6KO+F/O6s53f5t4sVwz4gE2Mz4ScghDXeKM+ccyB5GMq9dlkQ35A==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@rollup/plugin-commonjs": "28.0.1", - "@sentry/browser-utils": "10.73.0", + "@sentry/browser-utils": "10.75.2", "@sentry/bundler-plugin-core": "^5.3.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0", - "@sentry/node": "10.73.0", - "@sentry/opentelemetry": "10.73.0", - "@sentry/react": "10.73.0", - "@sentry/server-utils": "10.73.0", - "@sentry/vercel-edge": "10.73.0", + "@sentry/core": "10.75.2", + "@sentry/node": "10.75.2", + "@sentry/opentelemetry": "10.75.2", + "@sentry/react": "10.75.2", + "@sentry/server-utils": "10.75.2", + "@sentry/vercel-edge": "10.75.2", "@sentry/webpack-plugin": "^5.3.0", "rollup": "^4.60.3", "stacktrace-parser": "^0.1.11" @@ -3837,20 +3873,32 @@ "next": "^13.2.0 || ^14.0 || ^15.0.0-rc.0 || ^16.0.0-0" } }, + "node_modules/@sentry/nextjs/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@sentry/node": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.73.0.tgz", - "integrity": "sha512-jiMJ6GgXDw6UMGzJY+o0c8OoeA9OfHqZ/xEpHfDqy75hn+9CEkRkbNGCIdMvsc7wW/Se1Os394hHfTpU+YEskg==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/node/-/node-10.75.2.tgz", + "integrity": "sha512-ffdx1i3Fx9Q3Ofdaone2I+HWOj9+Wcu268xgV+Bn8WJYABm8BZIBNvS+PORKBPesWILhQCThWOs+E/8vcV1HuQ==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", "@opentelemetry/instrumentation": "^0.220.0", "@opentelemetry/sdk-trace-base": "^2.9.0", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0", - "@sentry/node-core": "10.73.0", - "@sentry/opentelemetry": "10.73.0", - "@sentry/server-utils": "10.73.0", + "@sentry/core": "10.75.2", + "@sentry/node-core": "10.75.2", + "@sentry/opentelemetry": "10.75.2", + "@sentry/server-utils": "10.75.2", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -3858,14 +3906,14 @@ } }, "node_modules/@sentry/node-core": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.73.0.tgz", - "integrity": "sha512-GHAGUmZPmm6FKfxfv2maVVJ/A99YAmd7oFOuKMDmITI6O/Msl6JB3vPTEpopVAHLW0LYJQBOjddL9C7o+Jt44g==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/node-core/-/node-core-10.75.2.tgz", + "integrity": "sha512-6sFhrvWVLAPRTOYkwrOOLwgsWINe6Y75oX9xKj11zZcKd2e1K2Y3bcx3uzDU06+cxfVbUQUBulsCr/TD7yx/vw==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0", - "@sentry/opentelemetry": "10.73.0", + "@sentry/core": "10.75.2", + "@sentry/opentelemetry": "10.75.2", "import-in-the-middle": "^3.0.0" }, "engines": { @@ -3896,14 +3944,38 @@ } } }, + "node_modules/@sentry/node-core/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/node/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@sentry/opentelemetry": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.73.0.tgz", - "integrity": "sha512-fQouPQKsH0CQrw6oAn1k0Z2I+tgyochCovifr5qNS69i0OzjknLa03WJyiZ/IuzXc4AVa5jAKfOeE9slABz8Qw==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/opentelemetry/-/opentelemetry-10.75.2.tgz", + "integrity": "sha512-qoJIdrXiFPPYs7AmdkktB5O4vhHYKzk6EulpQlvROe+ZZeG4/S0yeJApAbg3ThDob/2zEAxYEPEvaHv2epipgA==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" }, "engines": { "node": ">=18" @@ -3914,15 +3986,27 @@ "@opentelemetry/sdk-trace-base": "^1.30.1 || ^2.1.0" } }, + "node_modules/@sentry/opentelemetry/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@sentry/react": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.73.0.tgz", - "integrity": "sha512-wJrzS98ddPvhGS/MKNHZyE8X7ecd4KwKdz7fHino2qkqBBrF4cxWr+q/uLTIk5PYWMkeJ4oKMjHAjOqmzGR4tg==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/react/-/react-10.75.2.tgz", + "integrity": "sha512-4MtB1tSmMHzwTwY7m30UzRbbUx9cP84UK7WIDL9g7RvoGd/aZwxbp6IkUJ46VBNItVoSF+1GM9wtqzkLma252g==", "license": "MIT", "dependencies": { - "@sentry/browser": "10.73.0", + "@sentry/browser": "10.75.2", "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" }, "engines": { "node": ">=18" @@ -3931,53 +4015,113 @@ "react": "^16.14.0 || 17.x || 18.x || 19.x" } }, + "node_modules/@sentry/react/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@sentry/replay": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.73.0.tgz", - "integrity": "sha512-nN2wjN/Y0J5BOJV5hqRHUEBfxwUsipp1PKjcDHh6Fpxnrtfldu3Y99E8cQInseo5heFdzEvrOHBBrqWZXOHVKQ==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-10.75.2.tgz", + "integrity": "sha512-cCfdPt892S+rrAFOuUvbwR318/ZPFfBFvVkMlL3goktAnbECAGy6comAxLQEalyp22Ee67bEqYQN7RdNuRHIJw==", "license": "MIT", "dependencies": { - "@sentry/browser-utils": "10.73.0", - "@sentry/core": "10.73.0" + "@sentry/browser-utils": "10.75.2", + "@sentry/core": "10.75.2" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/replay-canvas": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.73.0.tgz", - "integrity": "sha512-sxa2lKkHPfF/j5xFpW7gocthWXRqyoHz8KCPy6yGc8plT477nl57iGSKhQDYsx1Ny10TLjs7YkIuPP1GY/Ax2Q==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/replay-canvas/-/replay-canvas-10.75.2.tgz", + "integrity": "sha512-4GVhCkn+/0QEpXdY8UfXahg8qjF2QTH1+qej/CLketHGfRqxLhdanzHU9ILesplWX5GeuTkPn5ZTmd5CJf+KXg==", "license": "MIT", "dependencies": { - "@sentry/core": "10.73.0", - "@sentry/replay": "10.73.0" + "@sentry/core": "10.75.2", + "@sentry/replay": "10.75.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/replay-canvas/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/replay/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/server-utils": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.73.0.tgz", - "integrity": "sha512-QskripdKFbM/+gipC6mpa2crLwL7+VbkX84IpHg2z9UlYQ1kNKd3aMT+Qk9NLRSw/zu1rSIAvlbWfx4D3rgNAA==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/server-utils/-/server-utils-10.75.2.tgz", + "integrity": "sha512-ep8Ntj/vsLHbql1jUMFiRYrleuQ3dENb8oD4Nkjv7hou2j0UKLct1eLbx+TckeVOhYdWmFeO7WJWjRio/a4TOg==", "license": "MIT", "dependencies": { "@sentry/conventions": "^0.16.0", - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/server-utils/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" } }, "node_modules/@sentry/vercel-edge": { - "version": "10.73.0", - "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.73.0.tgz", - "integrity": "sha512-DJLqVzd3ng4IwR+ux/vK1Wth/CSwUrsiwIbNsyneHw79cyhfjYuovFTw1xrjn9zTJoVSE8KkdajLSLwmLbnoTw==", + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/vercel-edge/-/vercel-edge-10.75.2.tgz", + "integrity": "sha512-Ol2kQfob60s4qAeitUt2XXG9of/jUCqhQ3loMx5nASJn9Y1C86XZOqzbVuzBDIoMQVBPV2JGCD8HgWHiN5HPbw==", "license": "MIT", "dependencies": { "@opentelemetry/api": "^1.9.1", - "@sentry/core": "10.73.0" + "@sentry/core": "10.75.2" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@sentry/vercel-edge/node_modules/@sentry/core": { + "version": "10.75.2", + "resolved": "https://registry.npmjs.org/@sentry/core/-/core-10.75.2.tgz", + "integrity": "sha512-KXzNf2c430cJIOMv84+MKpkXgave1DBNMNNwn073olVD773NME5IhDv/9VQyiFM2wIU+6bm4FEBVEVSuzqgjJw==", + "license": "MIT", + "dependencies": { + "@sentry/conventions": "^0.16.0" }, "engines": { "node": ">=18" @@ -7147,9 +7291,9 @@ } }, "node_modules/cjs-module-lexer": { - "version": "2.2.1", - "resolved": "https://registry.npmjs.org/cjs-module-lexer/-/cjs-module-lexer-2.2.1.tgz", - "integrity": "sha512-Ca8swihM+/4yKecYHY52kgJd300hi2lADU/a1RxNTRe+RJ9jvqQlESpbz9DnG9mowez8qwXHB8qYdIUw9e+F5Q==", + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/cjs-module-lexer/-/cjs-module-lexer-2.3.0.tgz", + "integrity": "sha512-lmNyBzi6iYiqyrExTctfmjd8tInx+dHu9PBomTfH5RXS53c27JMFJgzNv5TyAnm3PgGf+MLy0+Ws41/HlVxdKQ==", "license": "MIT" }, "node_modules/class-variance-authority": { @@ -7620,9 +7764,9 @@ "peer": true }, "node_modules/dompurify": { - "version": "3.4.16", - "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.4.16.tgz", - "integrity": "sha512-sqo+pNp3qRhCIpbgRi1y8Tgk27Bo2Ry7w0dC1NBeNTdZChWjz9Xb/KOoZbRP/R6pQZ80Qw8YhXw13hWWBbMRnQ==", + "version": "3.4.17", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-3.4.17.tgz", + "integrity": "sha512-B/pSlYVHQ58kqISD/yDjuJ1yfzI34m1gF4lAod2bwSRhiN4K5Cl3H8Af7hqB2PmTL911tdEudnnXn5FnhRoz0w==", "license": "(MPL-2.0 OR Apache-2.0)", "optionalDependencies": { "@types/trusted-types": "^2.0.7" @@ -7840,7 +7984,8 @@ "version": "2.3.2", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.3.2.tgz", "integrity": "sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==", - "license": "MIT" + "license": "MIT", + "peer": true }, "node_modules/es-object-atoms": { "version": "1.1.2", @@ -8029,21 +8174,21 @@ } }, "node_modules/eslint-config-next": { - "version": "16.3.4", - "resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.3.4.tgz", - "integrity": "sha512-35/8RM10huEL9vlr8hUZMERMENHBrnyHN3ZZkF9efSgzGaqK34jIqry44A956//zriUhUAUW0XSkcolhrryqAA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.4.1.tgz", + "integrity": "sha512-6AsojbCfRdpWkP6BIUnMBv3MH4OvtfmH9p4hSfLh8ZgsW7nfhu7DpUMNBB7leisDyyMz+YYxTc+NAdBOQfovlA==", "dev": true, "license": "MIT", "dependencies": { - "@next/eslint-plugin-next": "16.3.4", + "@next/eslint-plugin-next": "16.4.1", "eslint-import-resolver-node": "^0.3.6", "eslint-import-resolver-typescript": "^3.5.2", "eslint-plugin-import": "^2.32.0", "eslint-plugin-jsx-a11y": "^6.10.0", "eslint-plugin-react": "^7.37.0", - "eslint-plugin-react-hooks": "^7.0.0", + "eslint-plugin-react-hooks": "^7.1.0", "globals": "16.4.0", - "typescript-eslint": "^8.46.0" + "typescript-eslint": "^8.56.0" }, "peerDependencies": { "eslint": ">=9.0.0", @@ -9300,19 +9445,25 @@ } }, "node_modules/import-in-the-middle": { - "version": "3.4.0", - "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-3.4.0.tgz", - "integrity": "sha512-Xfjwfarhe+LGmoaof+sexeNo3sGRysb5x56WrZLwHtmqT0OilSKtVWkv0lrCcMgSKyKP9oBogBAisHZvtJH0aw==", + "version": "3.5.2", + "resolved": "https://registry.npmjs.org/import-in-the-middle/-/import-in-the-middle-3.5.2.tgz", + "integrity": "sha512-WseIA/4o56+GYO3RmlOPvejYi771OFeJEeK/ejROsr9bdjW+E5NXTOD1Di8TL40Qgg3JJX9aa97Hf8iiSVIKZw==", "license": "Apache-2.0", "dependencies": { "cjs-module-lexer": "^2.2.0", - "es-module-lexer": "^2.2.0", + "es-module-lexer": "^3.0.2", "module-details-from-path": "^1.0.4" }, "engines": { "node": ">=18" } }, + "node_modules/import-in-the-middle/node_modules/es-module-lexer": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-3.0.4.tgz", + "integrity": "sha512-BldNY5mwkVu+cQ5Ymn3e6hB9Sx8XHx4ZevWbiUyZbfKxJxYO27Hf0wfzkxIaD5CAJMaGl3YspKELj0ff/7nCMA==", + "license": "MIT" + }, "node_modules/imurmurhash": { "version": "0.1.4", "resolved": "https://registry.npmjs.org/imurmurhash/-/imurmurhash-0.1.4.tgz", @@ -11742,12 +11893,12 @@ } }, "node_modules/next": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/next/-/next-16.3.8.tgz", - "integrity": "sha512-U7QEZaTini6wKrb8A8hqLLqYQyCetegKjCpJOyxk642vWoMoU1x5PyZCJFvgYgiptA8xc5j/9xYlZFO7w9Sjmw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/next/-/next-16.4.1.tgz", + "integrity": "sha512-uc657V62E34fOI6W5NtG9IzwsPcms0Z60grz2MJGtglgD+pTFnNt4Yw2PzaU622tfX4vhrDgLZzicE2YlhI20Q==", "license": "MIT", "dependencies": { - "@next/env": "16.3.8", + "@next/env": "16.4.1", "@swc/helpers": "0.5.23", "baseline-browser-mapping": "^2.9.19", "caniuse-lite": "^1.0.30001579", @@ -11761,15 +11912,15 @@ "node": ">=20.9.0" }, "optionalDependencies": { - "@next/swc-darwin-arm64": "16.3.8", - "@next/swc-darwin-x64": "16.3.8", - "@next/swc-linux-arm64-gnu": "16.3.8", - "@next/swc-linux-arm64-musl": "16.3.8", - "@next/swc-linux-x64-gnu": "16.3.8", - "@next/swc-linux-x64-musl": "16.3.8", - "@next/swc-win32-arm64-msvc": "16.3.8", - "@next/swc-win32-x64-msvc": "16.3.8", - "sharp": "^0.35.4" + "@next/swc-darwin-arm64": "16.4.1", + "@next/swc-darwin-x64": "16.4.1", + "@next/swc-linux-arm64-gnu": "16.4.1", + "@next/swc-linux-arm64-musl": "16.4.1", + "@next/swc-linux-x64-gnu": "16.4.1", + "@next/swc-linux-x64-musl": "16.4.1", + "@next/swc-win32-arm64-msvc": "16.4.1", + "@next/swc-win32-x64-msvc": "16.4.1", + "sharp": "^0.35.5" }, "peerDependencies": { "@opentelemetry/api": "^1.1.0", diff --git a/module/services/frontend/code/package.json b/module/services/frontend/code/package.json index f3277d9e8..15b6e0eb6 100644 --- a/module/services/frontend/code/package.json +++ b/module/services/frontend/code/package.json @@ -54,7 +54,7 @@ "@fontsource/source-serif-4": "^5.3.0", "@hookform/resolvers": "^3.9.0", "@module-federation/runtime": "^2.8.2", - "@sentry/nextjs": "^10.50.0", + "@sentry/nextjs": "10.75.2", "@simplewebauthn/browser": "^13.3.0", "@tanstack/react-query": "^5.60.0", "@tanstack/react-table": "^8.21.3", @@ -62,7 +62,7 @@ "codefly": "https://github.com/codefly-dev/sdk-js/archive/9f3bb78682a0c2b9a70631d35c2ae0094c281ac5.tar.gz", "js-yaml": "^4.3.2", "lucide-react": "^1.8.0", - "next": "^16.2.12", + "next": "16.4.1", "react": "^19.2.8", "react-dom": "^19.2.8", "react-hook-form": "^7.54.0", @@ -88,7 +88,7 @@ "ajv": "^8.17.1", "babel-plugin-react-compiler": "1.0.0", "eslint": "^9.39.5", - "eslint-config-next": "^16.2.12", + "eslint-config-next": "16.4.1", "happy-dom": "^20.8.9", "jsdom": "^29.0.2", "msw": "^2.7.0", diff --git a/module/services/frontend/code/src/test/pipeline-setup.ts b/module/services/frontend/code/src/test/pipeline-setup.ts index 08746fad9..85d2f0d0f 100644 --- a/module/services/frontend/code/src/test/pipeline-setup.ts +++ b/module/services/frontend/code/src/test/pipeline-setup.ts @@ -15,8 +15,8 @@ // workers, so starting the graph per file would start one graph per file. // // withDependencies passes --exclude-root: it brings up the frontend's -// dependencies (auth-gateway, Accounts, Postgres, Vault, Redis, storage, -// telemetry) without the Next server the frontend itself would run. The +// dependencies (auth-gateway, Accounts, Postgres, Vault, Redis, storage) +// without the Next server the frontend itself would run. The // pipeline tier addresses the gateway directly, so the root service is dead // weight — and leaving it out means these tests can run alongside a // `codefly run service` dev stack instead of fighting it for the Next dev lock. @@ -47,7 +47,7 @@ export default async function setup(): Promise<(() => Promise) | void> { readyService: "auth-gateway", scope, fixture: "dev-admin", - silents: ["store", "cache", "telemetry"], + silents: ["store", "cache"], echo: process.env.CODEFLY_TEST_ECHO === "1", }); } catch (error) { diff --git a/module/services/marketing/code/package-lock.json b/module/services/marketing/code/package-lock.json index 2bc7701a4..e6c4817b8 100644 --- a/module/services/marketing/code/package-lock.json +++ b/module/services/marketing/code/package-lock.json @@ -8,7 +8,7 @@ "name": "@codefly/marketing", "version": "0.0.1", "dependencies": { - "next": "^16.2.12", + "next": "16.4.1", "react": "^19.2.8", "react-dom": "^19.2.8", "yaml": "^2.8.1", @@ -1024,15 +1024,15 @@ } }, "node_modules/@next/env": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/env/-/env-16.3.8.tgz", - "integrity": "sha512-Al9zqHVV7TJv0eFuOU4U7Lvv74PTih4Ch63sk2xCIpSTkE3udFnaOcnzP2lQVymiL7yS9Cj2iClUXlR3EQ5sEw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/env/-/env-16.4.1.tgz", + "integrity": "sha512-5sUUrrSvs/2/2LnLoesybMGx12EtPB46fdZ6RhAw0XDGSPxU2LFyQBB7yuEiOFFJPKmBY6+q2jTr1/AMbnp0/g==", "license": "MIT" }, "node_modules/@next/swc-darwin-arm64": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.3.8.tgz", - "integrity": "sha512-2JPRMh2nmQG5CiL7cXGL9AGwnPWJQ//cTtAUCT+w511QHk79SYz3LGv/pc5X643B/WEO0rvu3Yww0hqwt3kgeA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.4.1.tgz", + "integrity": "sha512-U+hQ5Gj3fpzf4fveTIF3dpUFYhaSW8vHhdAnbiIomzXzg7boreYFgBqprepXFtW4sw+aTttiNmg0gew/iFNhbg==", "cpu": [ "arm64" ], @@ -1046,9 +1046,9 @@ } }, "node_modules/@next/swc-darwin-x64": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.3.8.tgz", - "integrity": "sha512-GZtCCOBKJ4leVIT/Th0llWKhD1ca92lzbQiS5R5ON9QkoiFnilFsebDae1JU2a3HWoKMEmEZWGs1AGLavVM72Q==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.4.1.tgz", + "integrity": "sha512-/hqC3V1XLUjhMASf/WridKdO9FXoEgN3HBWmtKbj00MPgnltutfraWJ3bYTpCRIgEIpbjJyOVLpYgAhh/BkdcQ==", "cpu": [ "x64" ], @@ -1062,9 +1062,9 @@ } }, "node_modules/@next/swc-linux-arm64-gnu": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.3.8.tgz", - "integrity": "sha512-O659ygeQYqneJ1fBKMpFxIFqYkYswu8IAS1OCKK/4f3ZgJJm1dRz4fVJZRi/kLLWjnBKnebOePA4WNv+sV1pVA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.4.1.tgz", + "integrity": "sha512-GHzncHbcaS9g95kcvTNtKQW3aCLoR+xc3Jm7qrkpk1R+D2BjMHIPyvQGRPKCdMX95BsH/421jBUK06q9tEwzjw==", "cpu": [ "arm64" ], @@ -1081,9 +1081,9 @@ } }, "node_modules/@next/swc-linux-arm64-musl": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.3.8.tgz", - "integrity": "sha512-dSjKSyWpzxoO1d3DIZZcP4XJcNaKeLmxQMFOiYl5vuBRMmweIqnAhty8tAmRsvTss779cK1FtYnDMj40e4TQlg==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.4.1.tgz", + "integrity": "sha512-1SuJyDxW/PR9dHJat/ZRgcYLhuSZ+1psP4DlKb2tKIuXbjygqpLrJWRSq+onZQBv8LcmU67EIrmb6c3XTaJLjw==", "cpu": [ "arm64" ], @@ -1100,9 +1100,9 @@ } }, "node_modules/@next/swc-linux-x64-gnu": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.3.8.tgz", - "integrity": "sha512-lbqOuz3RPRcv+o9msNsJw5x4+Y1ZwPTs6vmL6DCf7i0fZfvng/F59wyeDwqHIvV0mK//RBy/jJkZ+nCKsSMXjQ==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.4.1.tgz", + "integrity": "sha512-Istng1hMGp9TGO4lUqeKLd70MPN/OOkucVTP2Nmj+gLkwBgmPqUw8CJYBCC7nbl/B/23bBCcjCI5Hsh6hazC1A==", "cpu": [ "x64" ], @@ -1119,9 +1119,9 @@ } }, "node_modules/@next/swc-linux-x64-musl": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.3.8.tgz", - "integrity": "sha512-+316WswI8ScVgZeUd+1KGaXkHhaYQzCjvH/05TZSpJ8zBizb1a4G7DtO7F12jcBIqMOtsz9ji1t48fmKtzqsGA==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.4.1.tgz", + "integrity": "sha512-aSC9wMNiZRRFVpPkPWBzRwLQxIGm/CiQcXyFF1XNcgodb+uMxdk7aSSGGpDXCbnvZdf4itjHxpOd5h7i7zaiYw==", "cpu": [ "x64" ], @@ -1138,9 +1138,9 @@ } }, "node_modules/@next/swc-win32-arm64-msvc": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.3.8.tgz", - "integrity": "sha512-ji0gd4kMYUxO+1fJBIbiBVRCjzG/lloiyCccnlebvb1ZJ5qXCPZqYg4Jl1DrrixnWNMKylzgpmMWx0yNDYXlzw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.4.1.tgz", + "integrity": "sha512-t/myvwFDlypMUjRvbBf8qLPyuaB0iXYYdBtwMt6DM6f+60wtGr4bSYholdPqNeV5VUXZhrpMwfza829kmhS3iA==", "cpu": [ "arm64" ], @@ -1154,9 +1154,9 @@ } }, "node_modules/@next/swc-win32-x64-msvc": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.3.8.tgz", - "integrity": "sha512-WcTlaKt/TWkh5kUjdJcUmB1XgZ+1c6fz4Y9fDHL73YNSdGaUWjceeWrrlwF0nv19iABYWC4iAq1oX1w4Bn0vfg==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.4.1.tgz", + "integrity": "sha512-6ICqPo9awCkrtYNPhwkx1uU7rlFWrZRrIboDHS2LU+jv088UBYQtmJ25h6pCp27/+FZjF6BQ4Te4ErugyMeuBg==", "cpu": [ "x64" ], @@ -1339,12 +1339,12 @@ } }, "node_modules/next": { - "version": "16.3.8", - "resolved": "https://registry.npmjs.org/next/-/next-16.3.8.tgz", - "integrity": "sha512-U7QEZaTini6wKrb8A8hqLLqYQyCetegKjCpJOyxk642vWoMoU1x5PyZCJFvgYgiptA8xc5j/9xYlZFO7w9Sjmw==", + "version": "16.4.1", + "resolved": "https://registry.npmjs.org/next/-/next-16.4.1.tgz", + "integrity": "sha512-uc657V62E34fOI6W5NtG9IzwsPcms0Z60grz2MJGtglgD+pTFnNt4Yw2PzaU622tfX4vhrDgLZzicE2YlhI20Q==", "license": "MIT", "dependencies": { - "@next/env": "16.3.8", + "@next/env": "16.4.1", "@swc/helpers": "0.5.23", "baseline-browser-mapping": "^2.9.19", "caniuse-lite": "^1.0.30001579", @@ -1358,15 +1358,15 @@ "node": ">=20.9.0" }, "optionalDependencies": { - "@next/swc-darwin-arm64": "16.3.8", - "@next/swc-darwin-x64": "16.3.8", - "@next/swc-linux-arm64-gnu": "16.3.8", - "@next/swc-linux-arm64-musl": "16.3.8", - "@next/swc-linux-x64-gnu": "16.3.8", - "@next/swc-linux-x64-musl": "16.3.8", - "@next/swc-win32-arm64-msvc": "16.3.8", - "@next/swc-win32-x64-msvc": "16.3.8", - "sharp": "^0.35.4" + "@next/swc-darwin-arm64": "16.4.1", + "@next/swc-darwin-x64": "16.4.1", + "@next/swc-linux-arm64-gnu": "16.4.1", + "@next/swc-linux-arm64-musl": "16.4.1", + "@next/swc-linux-x64-gnu": "16.4.1", + "@next/swc-linux-x64-musl": "16.4.1", + "@next/swc-win32-arm64-msvc": "16.4.1", + "@next/swc-win32-x64-msvc": "16.4.1", + "sharp": "^0.35.5" }, "peerDependencies": { "@opentelemetry/api": "^1.1.0", diff --git a/module/services/marketing/code/package.json b/module/services/marketing/code/package.json index c15dbbc38..523d07aa4 100644 --- a/module/services/marketing/code/package.json +++ b/module/services/marketing/code/package.json @@ -18,7 +18,7 @@ "test:smoke": "node scripts/smoke.mjs" }, "dependencies": { - "next": "^16.2.12", + "next": "16.4.1", "react": "^19.2.8", "react-dom": "^19.2.8", "yaml": "^2.8.1", diff --git a/module/services/telemetry/builder/Dockerfile b/module/services/telemetry/builder/Dockerfile deleted file mode 100644 index 8022f031b..000000000 --- a/module/services/telemetry/builder/Dockerfile +++ /dev/null @@ -1,103 +0,0 @@ -# syntax=docker/dockerfile:1@sha256:ecfaec9ed6d810b56388c508f4121597bfbba70d41a6dfeee4d8cad5f295fc32 -# The Go modules the recipe declares it downloads (go_module_downloads), fetched -# by the caller on its own machine before any image build and supplied here as -# the build context of this name, in the Go module proxy layout. Without a -# caller supplying it, the stage is empty and the download below uses the -# network as it always did. -FROM scratch AS gomodproxy - -# Build stage -FROM golang:1.27.3-alpine3.23@sha256:895e5feab11a4b3774d24ee3c9579028456b82ad37f42a9eb6f82adaf776a201 AS builder - -ARG TARGETARCH - -# Private Go modules. A service built from its own directory declares its -# module's downloads in the recipe, and the caller fetches them before the build -# and supplies them as the gomodproxy context above: the build reads them from -# there and needs no credential. GOPRIVATE, a build argument the CLI sets from -# the host's own, only matters when nothing was supplied — a workspace build, or -# a plain `docker build` — where those paths bypass the public proxy and are -# fetched with git; the optional netrc secret mount on those download steps is -# for callers that predate the declaration and still pass a credential. -ARG GOPRIVATE - -# Install ca-certificates for HTTPS requests and git for private repos if needed. -# CGO services additionally need a native compiler and libc development files -# in the build stage; none of that toolchain reaches the runtime image. -RUN apk add --no-cache ca-certificates git - -ENV CGO_ENABLED=0 -ENV GOTOOLCHAIN=local - -# Set working directory -WORKDIR /app -# Copy the source code - -COPY service.codefly.yaml service.codefly.yaml - -COPY code code - - -# Download dependencies and build the binary -COPY code/go.mod code/go.sum code/ -# With the caller's prefetched modules, they are the only module source: no -# credential is needed or mounted, a module missing from them fails here rather -# than being fetched, and go.sum alone verifies them (the caller's fetch was -# verified against the checksum database on its side). GOPRIVATE is cleared so -# private paths are read from the proxy like any other. -RUN --mount=type=bind,from=gomodproxy,target=/gomodproxy \ - if [ -n "$(ls -A /gomodproxy)" ]; then export GOPROXY=file:///gomodproxy GOSUMDB=off GOPRIVATE= GONOPROXY= GONOSUMDB=; fi && \ - cd code && sha256sum go.mod go.sum > /tmp/locks.sha256 && go mod download && go list -mod=readonly -m -json all > /tmp/modules.json && go mod verify && sha256sum -c /tmp/locks.sha256 -COPY code code -RUN find . -type f -print0 | sort -z | xargs -0 sha256sum > /tmp/sources.sha256 -RUN --network=none cd code && \ - GOOS=linux GOARCH=${TARGETARCH} go build -mod=readonly -ldflags='-w -s -extldflags "-static"' -o /app/app . - -# Retain the locked graph and metadata of the executables actually compiled. -# These are build evidence, not a scan, approval, publication or installation. -RUN --network=none cd code && \ - sha256sum -c /tmp/locks.sha256 && \ - mkdir -p /app/build-evidence && \ - cp go.mod go.sum /app/build-evidence/ && \ - cp /tmp/sources.sha256 /app/build-evidence/sources.sha256 && \ - cp /tmp/modules.json /app/build-evidence/modules.json && \ - go version -m /app/app > /app/build-evidence/binaries.txt && \ - sha256sum /app/app > /app/build-evidence/binaries.sha256 - -# Final stage -FROM alpine:3.23.5@sha256:fd791d74b68913cbb027c6546007b3f0d3bc45125f797758156952bc2d6daf40 - -# Runtime only needs the public CA bundle. Keep build tooling out of the image. -RUN apk add --no-cache ca-certificates - -# Run from the Go source directory, mirroring the dev runner (which starts the -# binary with its working directory set to the source dir). This is what makes a -# service's runtime path — e.g. "../routing/rest", anchored at the source dir — -# resolve to the same file it does in dev. -WORKDIR /app/code - -# Set environment variables - - -# Create a non-root user -RUN adduser -D appuser - -# Copy the binary from the builder stage -COPY --chown=appuser --from=builder /app/app . -COPY --from=builder /app/build-evidence /usr/share/codefly/build - -# Copy declared runtime assets from the build context, reproducing the source -# tree under /app so a source-relative reference resolves the same as in dev. -# Services that read config or data files at startup rely on a source-relative -# fallback in dev; these COPY lines are what carry those files into the image. -# Inventory the final bytes at their real runtime locations, including assets. -RUN find /app -type f -print0 | sort -z | xargs -0 sha256sum > /usr/share/codefly/build/runtime.sha256 - -# Use the non-root user -USER appuser - -# Expose ports -EXPOSE 8080 9090 - -# Run the binary -CMD ["./app"] diff --git a/module/services/telemetry/builder/dockerignore b/module/services/telemetry/builder/dockerignore deleted file mode 100644 index d7b6bd156..000000000 --- a/module/services/telemetry/builder/dockerignore +++ /dev/null @@ -1,2 +0,0 @@ -code/go.work -code/go.work.sum diff --git a/module/services/telemetry/code/collector/collector.go b/module/services/telemetry/code/collector/collector.go deleted file mode 100644 index ed9196499..000000000 --- a/module/services/telemetry/code/collector/collector.go +++ /dev/null @@ -1,218 +0,0 @@ -package collector - -import ( - "bytes" - "context" - "errors" - "fmt" - "io" - "log/slog" - "net/http" - "net/url" - "strings" - "time" - - collectorlogsv1 "go.opentelemetry.io/proto/otlp/collector/logs/v1" - collectormetricsv1 "go.opentelemetry.io/proto/otlp/collector/metrics/v1" - collectortracev1 "go.opentelemetry.io/proto/otlp/collector/trace/v1" - "google.golang.org/protobuf/proto" -) - -type Config struct { - Exporter string - Endpoint string - Headers string - HTTPClient *http.Client - Logger *slog.Logger -} - -type Collector struct { - exporter string - endpoint string - headers http.Header - client *http.Client - logger *slog.Logger -} - -type TraceService struct { - collectortracev1.UnimplementedTraceServiceServer - collector *Collector -} - -type MetricsService struct { - collectormetricsv1.UnimplementedMetricsServiceServer - collector *Collector -} - -type LogsService struct { - collectorlogsv1.UnimplementedLogsServiceServer - collector *Collector -} - -func (c *Collector) TraceService() *TraceService { return &TraceService{collector: c} } - -func (c *Collector) MetricsService() *MetricsService { - return &MetricsService{collector: c} -} - -func (c *Collector) LogsService() *LogsService { return &LogsService{collector: c} } - -func New(config Config) (*Collector, error) { - exporter := strings.ToLower(strings.TrimSpace(config.Exporter)) - // Fail closed on an empty exporter instead of defaulting to "debug". - // An empty value never meant "the operator chose debug" — every committed - // observability.env sets this key explicitly, and so does - // configurations/local-dogfood/observability.env.example — it meant the configuration never reached the process. - // Defaulting it to debug turned that into SILENT trace loss: the collector - // logged "received OTLP signal" and dropped every span while its own - // ConfigMap said otlphttp. Refusing to start surfaces the same fault in one - // line of pod logs. - if exporter == "" { - return nil, errors.New( - "telemetry: OBSERVABILITY_EXPORTER is required and must be debug or otlphttp; " + - "the observability workspace configuration did not reach this process " + - "(on the local-dogfood profile, copy configurations/local-dogfood/observability.env.example to observability.env)") - } - switch exporter { - case "debug": - if strings.TrimSpace(config.Endpoint) != "" || strings.TrimSpace(config.Headers) != "" { - return nil, errors.New("telemetry: debug exporter cannot have an external endpoint or headers") - } - case "otlphttp": - endpoint, err := url.Parse(strings.TrimRight(strings.TrimSpace(config.Endpoint), "/")) - if err != nil || endpoint.Scheme == "" || endpoint.Host == "" { - return nil, errors.New("telemetry: OTLP/HTTP endpoint must be absolute") - } - // Plaintext is allowed only to loopback, which never leaves the pod and so - // is governed by no network policy. Everything else must be HTTPS, because - // that is the only egress this module actually grants telemetry: - // services/telemetry/service.codefly.yaml declares spec.deployment.public-egress-ports - // [443], which renders allow-telemetry-public-egress — TCP 443 to public IP - // space with 10/8, 172.16/12, 192.168/16 and fc00::/7 excepted — on top of a - // namespace-wide default-deny that has no allow-intra-namespace rule. - // - // A cluster-internal plaintext address (a *.svc name on 4318) is therefore - // denied at the network layer no matter what this check says; accepting it - // here would only move the failure from a startup error to a 10s export - // timeout per batch. Reaching an in-cluster collector is a topology change — - // a declared dependency edge with its regenerated NetworkPolicy — not a - // transport exemption in application code. - local := endpoint.Hostname() == "localhost" || endpoint.Hostname() == "127.0.0.1" - if endpoint.Scheme != "https" && (endpoint.Scheme != "http" || !local) { - return nil, errors.New("telemetry: OTLP/HTTP endpoint must use HTTPS") - } - config.Endpoint = endpoint.String() - default: - return nil, errors.New("telemetry: OBSERVABILITY_EXPORTER must be debug or otlphttp") - } - headers, err := parseHeaders(config.Headers) - if err != nil { - return nil, err - } - client := config.HTTPClient - if client == nil { - client = &http.Client{Timeout: 10 * time.Second} - } - logger := config.Logger - if logger == nil { - logger = slog.Default() - } - return &Collector{ - exporter: exporter, - endpoint: strings.TrimRight(config.Endpoint, "/"), - headers: headers, - client: client, - logger: logger, - }, nil -} - -func (s *TraceService) Export( - ctx context.Context, - request *collectortracev1.ExportTraceServiceRequest, -) (*collectortracev1.ExportTraceServiceResponse, error) { - if err := s.collector.deliver(ctx, "traces", request, len(request.GetResourceSpans())); err != nil { - return nil, err - } - return &collectortracev1.ExportTraceServiceResponse{}, nil -} - -func (s *MetricsService) Export( - ctx context.Context, - request *collectormetricsv1.ExportMetricsServiceRequest, -) (*collectormetricsv1.ExportMetricsServiceResponse, error) { - if err := s.collector.deliver(ctx, "metrics", request, len(request.GetResourceMetrics())); err != nil { - return nil, err - } - return &collectormetricsv1.ExportMetricsServiceResponse{}, nil -} - -func (s *LogsService) Export( - ctx context.Context, - request *collectorlogsv1.ExportLogsServiceRequest, -) (*collectorlogsv1.ExportLogsServiceResponse, error) { - if err := s.collector.deliver(ctx, "logs", request, len(request.GetResourceLogs())); err != nil { - return nil, err - } - return &collectorlogsv1.ExportLogsServiceResponse{}, nil -} - -func (c *Collector) deliver(ctx context.Context, signal string, message proto.Message, resources int) error { - if c.exporter == "debug" { - c.logger.Info("received OTLP signal", "signal", signal, "resources", resources) - return nil - } - body, err := proto.Marshal(message) - if err != nil { - return fmt.Errorf("telemetry: encode %s: %w", signal, err) - } - request, err := http.NewRequestWithContext( - ctx, - http.MethodPost, - c.endpoint+"/v1/"+signal, - bytes.NewReader(body), - ) - if err != nil { - return fmt.Errorf("telemetry: create %s export: %w", signal, err) - } - for name, values := range c.headers { - for _, value := range values { - request.Header.Add(name, value) - } - } - request.Header.Set("Content-Type", "application/x-protobuf") - response, err := c.client.Do(request) - if err != nil { - return fmt.Errorf("telemetry: export %s: %w", signal, err) - } - defer func() { _ = response.Body.Close() }() - _, _ = io.Copy(io.Discard, io.LimitReader(response.Body, 64*1024)) - if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices { - return fmt.Errorf("telemetry: export %s returned HTTP %d", signal, response.StatusCode) - } - return nil -} - -func parseHeaders(raw string) (http.Header, error) { - headers := make(http.Header) - raw = strings.TrimSpace(raw) - if raw == "" { - return headers, nil - } - values, err := url.ParseQuery(strings.ReplaceAll(raw, ",", "&")) - if err != nil { - return nil, errors.New("telemetry: OTLP headers must use URL-encoded key=value pairs") - } - for name, entries := range values { - name = strings.TrimSpace(name) - if name == "" { - return nil, errors.New("telemetry: OTLP header name is required") - } - for _, value := range entries { - if strings.ContainsAny(value, "\r\n") { - return nil, errors.New("telemetry: OTLP header value contains a newline") - } - headers.Add(name, value) - } - } - return headers, nil -} diff --git a/module/services/telemetry/code/collector/collector_test.go b/module/services/telemetry/code/collector/collector_test.go deleted file mode 100644 index dc121fbb4..000000000 --- a/module/services/telemetry/code/collector/collector_test.go +++ /dev/null @@ -1,134 +0,0 @@ -package collector_test - -import ( - "context" - "io" - "net" - "net/http" - "net/http/httptest" - "sync" - "testing" - - "telemetry/collector" - - collectortracev1 "go.opentelemetry.io/proto/otlp/collector/trace/v1" - tracev1 "go.opentelemetry.io/proto/otlp/trace/v1" - "google.golang.org/grpc" - "google.golang.org/grpc/credentials/insecure" - "google.golang.org/grpc/test/bufconn" - "google.golang.org/protobuf/proto" - - "github.com/stretchr/testify/require" -) - -func TestCollectorReceivesGRPCAndForwardsOTLPHTTP(t *testing.T) { - var mu sync.Mutex - var forwarded collectortracev1.ExportTraceServiceRequest - upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - require.Equal(t, "/v1/traces", r.URL.Path) - require.Equal(t, "Bearer test", r.Header.Get("Authorization")) - body, err := io.ReadAll(r.Body) - require.NoError(t, err) - mu.Lock() - defer mu.Unlock() - require.NoError(t, proto.Unmarshal(body, &forwarded)) - w.WriteHeader(http.StatusOK) - })) - defer upstream.Close() - - sink, err := collector.New(collector.Config{ - Exporter: "otlphttp", - Endpoint: upstream.URL, - Headers: "Authorization=Bearer+test", - }) - require.NoError(t, err) - listener := bufconn.Listen(1024 * 1024) - server := grpc.NewServer() - collectortracev1.RegisterTraceServiceServer(server, sink.TraceService()) - go func() { _ = server.Serve(listener) }() - t.Cleanup(server.Stop) - - connection, err := grpc.NewClient( - "passthrough:///bufconn", - grpc.WithTransportCredentials(insecure.NewCredentials()), - grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) { - return listener.Dial() - }), - ) - require.NoError(t, err) - defer func() { _ = connection.Close() }() - client := collectortracev1.NewTraceServiceClient(connection) - _, err = client.Export(t.Context(), &collectortracev1.ExportTraceServiceRequest{ - ResourceSpans: []*tracev1.ResourceSpans{{}}, - }) - require.NoError(t, err) - mu.Lock() - defer mu.Unlock() - require.Len(t, forwarded.GetResourceSpans(), 1) -} - -func TestCollectorConfigurationFailsClosed(t *testing.T) { - _, err := collector.New(collector.Config{ - Exporter: "debug", Endpoint: "https://collector.example", - }) - require.ErrorContains(t, err, "cannot have") - _, err = collector.New(collector.Config{ - Exporter: "otlphttp", Endpoint: "http://collector.example", - }) - require.ErrorContains(t, err, "HTTPS") -} - -// TestCollectorRequiresExplicitExporter pins the fail-closed behaviour: an empty -// OBSERVABILITY_EXPORTER used to fall back to "debug", which turned a broken -// configuration carrier into silent trace loss instead of a startup failure. -func TestCollectorRequiresExplicitExporter(t *testing.T) { - _, err := collector.New(collector.Config{}) - require.ErrorContains(t, err, "OBSERVABILITY_EXPORTER is required") - - _, err = collector.New(collector.Config{Exporter: " "}) - require.ErrorContains(t, err, "OBSERVABILITY_EXPORTER is required") -} - -// TestCollectorEndpointTransportPolicy pins which OTLP/HTTP destinations may be -// reached in plaintext: loopback only, because loopback never leaves the pod. -// -// The cluster-internal cases are the regression. Exempting *.svc / -// *.svc.cluster.local from the HTTPS requirement looks safe — those names -// resolve only inside the cluster — but this module grants telemetry exactly one -// egress path: public_egress_ports [443] in -// services/telemetry/service.codefly.yaml (spec.deployment), rendered as -// allow-telemetry-public-egress (TCP 443 to public IP space, private ranges -// excepted) over a namespace-wide default-deny with no allow-intra-namespace -// rule. A ClusterIP on 4318 is denied there no matter what this package accepts, -// so accepting it only converts a startup error into a 10s export timeout per -// batch. Reaching an in-cluster collector is a topology change, not a transport -// exemption in application code. -func TestCollectorEndpointTransportPolicy(t *testing.T) { - for _, testCase := range []struct { - name string - endpoint string - accepted bool - }{ - {"loopback name", "http://localhost:4318", true}, - {"loopback address", "http://127.0.0.1:4318", true}, - {"external TLS", "https://example.com", true}, - {"in-cluster collector FQDN is not reachable and not exempt", - "http://otel-collector.otel-collector.svc.cluster.local:4318", false}, - {"in-cluster collector short form", "http://otel-collector.otel-collector.svc:4318", false}, - {"external plaintext", "http://example.com:4318", false}, - {"external plaintext subdomain", "http://otel.internal.ops.example.com:4318", false}, - {"host merely containing svc", "http://svc.example.com:4318", false}, - } { - t.Run(testCase.name, func(t *testing.T) { - _, err := collector.New(collector.Config{ - Exporter: "otlphttp", - Endpoint: testCase.endpoint, - }) - if testCase.accepted { - require.NoError(t, err) - return - } - require.ErrorContains(t, err, "HTTPS") - }) - } -} diff --git a/module/services/telemetry/code/go.mod b/module/services/telemetry/code/go.mod deleted file mode 100644 index ee9cb654a..000000000 --- a/module/services/telemetry/code/go.mod +++ /dev/null @@ -1,69 +0,0 @@ -module telemetry - -go 1.27.0 - -require ( - github.com/codefly-dev/sdk-go v0.4.1 - github.com/stretchr/testify v1.12.1 - go.opentelemetry.io/proto/otlp v1.11.0 - google.golang.org/grpc v1.84.0 - google.golang.org/protobuf v1.36.12 -) - -require ( - buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.12-20260825204119-511051f7f437.2 // indirect - buf.build/go/protovalidate v1.4.0 // indirect - cel.dev/cel-go v0.32.0 // indirect - cel.dev/expr v0.25.3 // indirect - github.com/Masterminds/semver v1.5.0 // indirect - github.com/Masterminds/semver/v3 v3.5.0 // indirect - github.com/antlr4-go/antlr/v4 v4.13.1 // indirect - github.com/cenkalti/backoff/v5 v5.0.3 // indirect - github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/codefly-dev/core v0.19.6-0.20261018051341-b81d92e6feda // indirect - github.com/go-logr/logr v1.4.4 // indirect - github.com/go-logr/stdr v1.2.2 // indirect - github.com/go-openapi/analysis v1.0.0 // indirect - github.com/go-openapi/errors v0.22.9 // indirect - github.com/go-openapi/jsonpointer v1.0.1 // indirect - github.com/go-openapi/jsonreference v1.0.2 // indirect - github.com/go-openapi/loads v0.25.3 // indirect - github.com/go-openapi/spec v1.0.1 // indirect - github.com/go-openapi/strfmt v0.27.2 // indirect - github.com/go-openapi/swag/conv v0.29.2 // indirect - github.com/go-openapi/swag/jsonutils v0.29.2 // indirect - github.com/go-openapi/swag/loading v0.29.2 // indirect - github.com/go-openapi/swag/mangling v0.29.2 // indirect - github.com/go-openapi/swag/pools v0.29.2 // indirect - github.com/go-openapi/swag/stringutils v0.29.2 // indirect - github.com/go-openapi/swag/typeutils v0.29.2 // indirect - github.com/go-openapi/swag/yamlutils v0.29.2 // indirect - github.com/go-viper/mapstructure/v2 v2.5.0 // indirect - github.com/google/uuid v1.6.0 // indirect - github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 // indirect - github.com/hashicorp/errwrap v1.1.0 // indirect - github.com/hashicorp/go-multierror v1.1.1 // indirect - github.com/mitchellh/mapstructure v1.5.0 // indirect - github.com/oklog/ulid/v2 v2.1.2 // indirect - github.com/pkg/errors v0.9.1 // indirect - github.com/yoheimuta/go-protoparser/v4 v4.14.2 // indirect - go.opentelemetry.io/auto/sdk v1.2.1 // indirect - go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0 // indirect - go.opentelemetry.io/otel v1.47.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 // indirect - go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0 // indirect - go.opentelemetry.io/otel/log v1.47.0 // indirect - go.opentelemetry.io/otel/metric v1.47.0 // indirect - go.opentelemetry.io/otel/sdk v1.47.0 // indirect - go.opentelemetry.io/otel/trace v1.47.0 // indirect - go.yaml.in/yaml/v3 v3.0.5 // indirect - golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba // indirect - golang.org/x/net v0.60.0 // indirect - golang.org/x/sys v0.49.0 // indirect - golang.org/x/term v0.46.0 // indirect - golang.org/x/text v0.42.0 // indirect - google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20260928230214-8a89bd6388cc // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect -) diff --git a/module/services/telemetry/code/go.sum b/module/services/telemetry/code/go.sum deleted file mode 100644 index 3824e9bcc..000000000 --- a/module/services/telemetry/code/go.sum +++ /dev/null @@ -1,256 +0,0 @@ -buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.12-20260825204119-511051f7f437.2 h1:NbnlmV26O7oZ1iM5tsCI+GEx+3ZSdrhvnKQ/eWSrLiY= -buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.12-20260825204119-511051f7f437.2/go.mod h1:TCt1lluMFnctISJXvkIQ4x3ABrPuUKCWKyjKdkJNBpw= -buf.build/go/protovalidate v1.4.0 h1:UjLrYbt5VX7+TMOs2+pG5FhZhIG1mSfK4EIopbb4LcM= -buf.build/go/protovalidate v1.4.0/go.mod h1:8vJfzNT6NIG2qm3uFsJDXMlRmG+bQJzbcIn1Aa0vPGs= -cel.dev/cel-go v0.32.0 h1:irvpFKr5EuGPyxeME03ERh0rii1TX+BDAnB9eL3IvNk= -cel.dev/cel-go v0.32.0/go.mod h1:DnVip7tpJSsgZymwfT+m1tnEVy3ivAjSMXPx12YrMkU= -cel.dev/expr v0.25.3 h1:A2jO8jwOugrrovveCWfj0KEZOfqiLgAcwjpHPhzIGw0= -cel.dev/expr v0.25.3/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4= -github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww= -github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y= -github.com/Masterminds/semver/v3 v3.5.0 h1:kQceYJfbupGfZOKZQg0kou0DgAKhzDg2NZPAwZ/2OOE= -github.com/Masterminds/semver/v3 v3.5.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM= -github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= -github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= -github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow= -github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4= -github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ= -github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw= -github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ= -github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= -github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= -github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= -github.com/brianvoe/gofakeit/v6 v6.28.0 h1:Xib46XXuQfmlLS2EXRuJpqcw8St6qSZz75OUo0tgAW4= -github.com/brianvoe/gofakeit/v6 v6.28.0/go.mod h1:Xj58BMSnFqcn/fAQeSK+/PLtC5kSb7FJIq4JyGa8vEs= -github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM= -github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw= -github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= -github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cheggaaa/pb/v3 v3.2.1 h1:aprZbFRG+B7+ug76S8QZ6Y1PW168UHzOmbC3wa+aU6I= -github.com/cheggaaa/pb/v3 v3.2.1/go.mod h1:U9hSVxoKqJrZIE3PkFG1xXXNaK/Ilzg+EF/scpXzEdw= -github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk= -github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM= -github.com/codefly-dev/core v0.19.6-0.20261018051341-b81d92e6feda h1:ytTihtIEfVhQ7llQl+ls5swIL8JgJu89X0wlDMeqM+o= -github.com/codefly-dev/core v0.19.6-0.20261018051341-b81d92e6feda/go.mod h1:0uX/7lTBVaGV0RI2Mb3mpEr/PPdx/jZqXHym/U8FVDI= -github.com/codefly-dev/gortk v0.2.0 h1:7bOlS5valYz2zil+fZctQNcPCYBcPj86abcw9N8h1hQ= -github.com/codefly-dev/gortk v0.2.0/go.mod h1:dDWUMFgAP063OCGhTEpV7FFUj9+zTcxxO/nV80VKEwg= -github.com/codefly-dev/sdk-go v0.4.1 h1:WUKYWxXfojTr88SkLtq0T8aqpV0o5pR/muUUcsUUlzk= -github.com/codefly-dev/sdk-go v0.4.1/go.mod h1:zr9ts3kkWNv2XEFUXZB+ydfW6x5b6/NbUvBm4KVYNqs= -github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= -github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= -github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= -github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= -github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= -github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc= -github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= -github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= -github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= -github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w= -github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE= -github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc= -github.com/felixge/httpsnoop v1.1.0/go.mod h1:Zqxgdd+1Rkcz8euOqdr7lqgCRJztwr5hp9vDSi5UZCE= -github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho= -github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo= -github.com/go-errors/errors v1.5.1 h1:ZwEMSLRCapFLflTpT7NKaAc7ukJ8ZPEjzlxt8rPN8bk= -github.com/go-errors/errors v1.5.1/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= -github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8= -github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= -github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= -github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= -github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE= -github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78= -github.com/go-openapi/analysis v1.0.0 h1:sNvbAGCJqUTqIAodr9IVqJMmuZas3YS9ms1dGK9yiJ4= -github.com/go-openapi/analysis v1.0.0/go.mod h1:NhYjJ57fnE+bcE7UwrJyMkhWA3Dfz7TdiBfTVAnos4Y= -github.com/go-openapi/errors v0.22.9 h1:HI9+SyVYiRzyeBQGv0CbWbQa+u2S0nny7G7AhY+UMiM= -github.com/go-openapi/errors v0.22.9/go.mod h1:R73q66sXP2smzAGgFOI4nqtusRK2xN5pYCjsij+KX58= -github.com/go-openapi/jsonpointer v1.0.1 h1:2KxywRmNwJkT/FMBa3iRNHEaAxSJvjqoufQZy3au1Mg= -github.com/go-openapi/jsonpointer v1.0.1/go.mod h1:wI7ZYsFmbIi9nBXOZqgDaS/bqOchRGZjqxFli7FBYxY= -github.com/go-openapi/jsonreference v1.0.2 h1:oS4et8FOf3p3UQxEo4Xt0esijmBUM+F259Xl72OSZsc= -github.com/go-openapi/jsonreference v1.0.2/go.mod h1:TbUNSOo+fcorZjFaNoiSDSoaNnnZtqJtLGR1PuvE/Cs= -github.com/go-openapi/loads v0.25.3 h1:V+jKy/thXWdLJUuYC8sZX2dICyAa8M3DokF70jj34P0= -github.com/go-openapi/loads v0.25.3/go.mod h1:LgLyCSOLBL2Qnj0Ps1oo2YH8jZoEKsMeVZCc+ALIxFo= -github.com/go-openapi/spec v1.0.1 h1:lj2vdGpNDcVgwRc6qXdw6qt/KQpCtSa9tnUH6vpDPDk= -github.com/go-openapi/spec v1.0.1/go.mod h1:M//GWQGtDUAjnP37gE6fInLgaczB+FatoipV3H1fYw8= -github.com/go-openapi/strfmt v0.27.2 h1:SG32SlbwNy92s0KJiVxt2joJeFdqIYHvwrA0OU6HqzQ= -github.com/go-openapi/strfmt v0.27.2/go.mod h1:M4CKsMO0Fb8qR10+1Ra75wCKNNquy+Vj+4LWZrhTo2E= -github.com/go-openapi/swag v0.29.2 h1:9n8frkcsuQRA0INU31VbxNxt1zBaHxrHYGEf5BUjFi4= -github.com/go-openapi/swag v0.29.2/go.mod h1:RkCiX1gCVXWgLOavDXE3ALJrQ/VO2ziiIzOUs3z10xo= -github.com/go-openapi/swag/cmdutils v0.29.2 h1:cXEzX/nWCODon251f+1HgL75/R4XXML5IZxgLCWMoro= -github.com/go-openapi/swag/cmdutils v0.29.2/go.mod h1:XziaSVzYqkDvpWXzASfWajw2YvJrLscq610O2XiyqJw= -github.com/go-openapi/swag/conv v0.29.2 h1:8c9shoB8l0QRSR6ymq1llHdBWDqpIgJfjTGHx3Vuhm0= -github.com/go-openapi/swag/conv v0.29.2/go.mod h1:AZS0YigTNf8qNtD6WqJz/N4RUNmpr76SyjFGkq4+p6Q= -github.com/go-openapi/swag/fileutils v0.29.2 h1:mdUL+Vw5ah1fO1AvFCoHeIyv7YZsCz2MN3KNqd/lLVI= -github.com/go-openapi/swag/fileutils v0.29.2/go.mod h1:7QKmmodjAebaq61lGVJa8ldLdGeMF4DSgYv8tXOGGo0= -github.com/go-openapi/swag/jsonutils v0.29.2 h1:uZNSD2/rJDYAfvsLYkykTzXuyxM3QpDKV9jhRHrMu6k= -github.com/go-openapi/swag/jsonutils v0.29.2/go.mod h1:ONTdNvj3Y+IXRzfa17E+Rgt2ns/qiSOYjIo2K7v2yDs= -github.com/go-openapi/swag/jsonutils/fixtures_test v0.29.2 h1:w+Fd6EBOMGlC74GYGHqCLGyb3Bpams8TtNrgM/SG4jo= -github.com/go-openapi/swag/jsonutils/fixtures_test v0.29.2/go.mod h1:HC2egPORFzbj/gf05Zrfgz8Mgplx3oW7hgGi7pjZVhM= -github.com/go-openapi/swag/loading v0.29.2 h1:QU1ry24e6r6Shoxe380Nx0X5iKkpRO+hJffsyJ2dVlY= -github.com/go-openapi/swag/loading v0.29.2/go.mod h1:DqilDjuiJKETecsS4TRopWG1acUDsfqN39ZmQhPu6uI= -github.com/go-openapi/swag/mangling v0.29.2 h1:ltdo3K0tTP4P6zWhVVWDU9D37/YYBuMJRdZxiS2Dtog= -github.com/go-openapi/swag/mangling v0.29.2/go.mod h1:RjDi4TnItAenS3cgCXSpIl1kKbjG7il6pnmQVl/8bp8= -github.com/go-openapi/swag/netutils v0.29.2 h1:dBli6jyUa93sDS/XYWShJzf4+nnVfFKfvY9SLUaUzB8= -github.com/go-openapi/swag/netutils v0.29.2/go.mod h1:1pM6Xg/Um8E1eps3umWl8Hme3ByWXOtHsvFC0CULHUE= -github.com/go-openapi/swag/pools v0.29.2 h1:PlGoDRF8WtSyXkedZZkpW3f9wDhFf3u8KtaBcmgmh8c= -github.com/go-openapi/swag/pools v0.29.2/go.mod h1:V3lhxVT4qDYRqc2MRSSbLsTYIYV/2HlHafhyEkmzLIc= -github.com/go-openapi/swag/stringutils v0.29.2 h1:lcnBxwAaysT3bMUVBfn9V/PkfVqkurpqufKU6rTUMGk= -github.com/go-openapi/swag/stringutils v0.29.2/go.mod h1:i9cdh7sGaa0nm3CqIvH5IM5h2QClk1wns2ktKeILvRI= -github.com/go-openapi/swag/typeutils v0.29.2 h1:O7aVvkTs3pXwikgtrPLenigEMdQFgONKRooQA9pgf2I= -github.com/go-openapi/swag/typeutils v0.29.2/go.mod h1:7+GDG+uz9Ke+eVt4rClZg7wklSDp8hT/mKNh/pC26TE= -github.com/go-openapi/swag/yamlutils v0.29.2 h1:IFKFFeDnuIwzfsuWRQU+rI8iL3g4XyAYjV/RlnlxPLM= -github.com/go-openapi/swag/yamlutils v0.29.2/go.mod h1:7MGqtcrK73sxQ4ceiyIf8qiXS/s09JLMdR5esK5euYQ= -github.com/go-openapi/testify/enable/yaml/v2 v2.7.0 h1:wPW6YRgx3+SID1yUy/Xwa17L8kFEaEKod2VRbJDZNUs= -github.com/go-openapi/testify/enable/yaml/v2 v2.7.0/go.mod h1:mI1M88etYbc3PhgHsWQK2kwvNwW5aGFqMPbmib+SGIs= -github.com/go-openapi/testify/v2 v2.8.0 h1:19QDx5b57p8KjO2E4sgs/woL4Akp+FGLRlVm2xn0Rho= -github.com/go-openapi/testify/v2 v2.8.0/go.mod h1:4gUN8jC+RqOE2qjW/vDmfyTvl5BCht4qL7oca0odrak= -github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= -github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= -github.com/gofrs/flock v0.13.1 h1:jjREztyBeSKBZYAC+mgc1laB+xsgy4kYMf3FbKF2UBo= -github.com/gofrs/flock v0.13.1/go.mod h1:sf4BFiHwnvgxa25DlQoDqXQnwRMEOwqxRq37P6MzzmE= -github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= -github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= -github.com/google/gnostic-models v0.7.1 h1:SisTfuFKJSKM5CPZkffwi6coztzzeYUhc3v4yxLWH8c= -github.com/google/gnostic-models v0.7.1/go.mod h1:whL5G0m6dmc5cPxKc5bdKdEN3UjI7OUGxBlw57miDrQ= -github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= -github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= -github.com/google/go-github/v89 v89.0.0 h1:35bEK5XoEcF3PZrlVbl9XN63f5BcJRA/UGkxeC9xPg0= -github.com/google/go-github/v89 v89.0.0/go.mod h1:QLcbU0ipeAqQuR5KSg8c2lql4Qk1EwJ2dWz/0rP4Nho= -github.com/google/go-querystring v1.2.0 h1:yhqkPbu2/OH+V9BfpCVPZkNmUXhb2gBxJArfhIxNtP0= -github.com/google/go-querystring v1.2.0/go.mod h1:8IFJqpSRITyJ8QhQ13bmbeMBDfmeEJZD5A0egEOmkqU= -github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= -github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 h1:Bd7KaOxzULLxtZ/K5s1aLbWhR0+5RToO65TXHsf3bqQ= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0/go.mod h1:nN7ts3dFXKtCZWc//yfkpcQNKJABg16/uDVAZpLDalo= -github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= -github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= -github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= -github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= -github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= -github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= -github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e h1:Q6MvJtQK/iRcRtzAscm/zF23XxJlbECiGPyRicsX+Ak= -github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg= -github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY= -github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= -github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI= -github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= -github.com/mattn/go-runewidth v0.0.27 h1:Feg/Oou5zI/wnpgDF6omIU0OokC9GxLC/WRknhVlIR0= -github.com/mattn/go-runewidth v0.0.27/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8= -github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY= -github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= -github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= -github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= -github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U= -github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= -github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 h1:n6/2gBQ3RWajuToeY6ZtZTIKv2v7ThUy5KKusIT0yc0= -github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= -github.com/oklog/ulid/v2 v2.1.2 h1:IEclFb9JNvzYA6MW2SCxbLzcHTVsfqm3PrqGQJH5zec= -github.com/oklog/ulid/v2 v2.1.2/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ= -github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= -github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= -github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= -github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= -github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o= -github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= -github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU= -github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= -github.com/rodaine/protogofakeit v0.1.1 h1:ZKouljuRM3A+TArppfBqnH8tGZHOwM/pjvtXe9DaXH8= -github.com/rodaine/protogofakeit v0.1.1/go.mod h1:pXn/AstBYMaSfc1/RqH3N82pBuxtWgejz1AlYpY1mI0= -github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= -github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 h1:1EYB5IzjZawrrnELUi78f9fPu57HuXjmddZPjrls/28= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.3/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU= -github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI= -github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk= -github.com/shoenig/go-m1cpu v0.2.1 h1:yqRB4fvOge2+FyRXFkXqsyMoqPazv14Yyy+iyccT2E4= -github.com/shoenig/go-m1cpu v0.2.1/go.mod h1:KkDOw6m3ZJQAPHbrzkZki4hnx+pDRR1Lo+ldA56wD5w= -github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= -github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= -github.com/tklauser/go-sysconf v0.4.0 h1:7H0uAN+7RkwWRaxhYXDLqa5V3LPrJeV8wmD9dRUgPQU= -github.com/tklauser/go-sysconf v0.4.0/go.mod h1:8mTNWyog7H+MpKijp4VmKJAd2bbYQ2zuUwkYRbUArPI= -github.com/tklauser/numcpus v0.12.0 h1:NR85qdvHA9pFse3x3weVZ0r0ST8R6l5RHbZrlRaqob4= -github.com/tklauser/numcpus v0.12.0/go.mod h1:ABHeXzJnr/qqwguhClkZKT1/8VABcYrsyUiUGobwWJg= -github.com/xlab/treeprint v1.2.0 h1:HzHnuAF1plUN2zGlAFHbSQP2qJ0ZAD3XF5XD7OesXRQ= -github.com/xlab/treeprint v1.2.0/go.mod h1:gj5Gd3gPdKtR1ikdDK6fnFLdmIS0X30kTTuNd/WEJu0= -github.com/yoheimuta/go-protoparser/v4 v4.14.2 h1:/P/LlX1CF9NaTWEltGcIZVvNlPbhABuAnBtAWpb3+74= -github.com/yoheimuta/go-protoparser/v4 v4.14.2/go.mod h1:AHNNnSWnb0UoL4QgHPiOAg2BniQceFscPI5X/BZNHl8= -github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0= -github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0= -go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= -go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0 h1:Tq+E65HaNbZTeqcLw6QUhsV+/EIWdkfUH2CQzcrlE5M= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.72.0/go.mod h1:A+EVhDakAj1waJBPhmTszZaSr08ww8MRRKy1nB1rqck= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 h1:3g7B90UzBltIDKq1/5mrTGxTnOFDV0ICOhLoxiZ8jlg= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0/go.mod h1:Ef8SuTh59BT7+ofpDxN9z+yOlc4t2GjLmKDgYNJL/NU= -go.opentelemetry.io/otel v1.47.0 h1:j7ALJ/zgkS7Z6aeJW09p8VC9804bC+PpeTfCD4XPnOM= -go.opentelemetry.io/otel v1.47.0/go.mod h1:8wS9O2qfXrYrzp6hIF/HOYJJf/wIhFPhR2xLuP+iXQU= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 h1:julhjPeUH/q/7hinbSdDdqt5h7Zw9YWmRlWRhI0jd54= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0/go.mod h1:Ao2mz688LH/tFf0yMAenidq6k2YNSx6SIY2q6jDACck= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 h1:UFpxOpYPmMNUtOWhdb+nC1WELIgVf8z9higURrbzYU4= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0/go.mod h1:YBGjxe3lt0jtXoEXxGrGhv/jM7oUBUDVK7zhAvNyEjc= -go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0 h1:N3YQCxjxQ/bMjyc3heladfRm9t9RTksGQH8z4w6yU/0= -go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.47.0/go.mod h1:Mp8HOFqcaUyypCuGv9IhDdTHnJ56lSudSHMd+pVSCEA= -go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lqCcI= -go.opentelemetry.io/otel/log v1.47.0/go.mod h1:9byitSQ5pLC6PpqwGXjqdMKya6ZTswHRZh2vvXT33nw= -go.opentelemetry.io/otel/metric v1.47.0 h1:4PptaldXx3Eat1XjMZ68pPJEs5wrhlemctZE9a3UdWY= -go.opentelemetry.io/otel/metric v1.47.0/go.mod h1:ADGSXxRrXM6bjbvLo535EstVFlPpPYZm4LBKixjDHwU= -go.opentelemetry.io/otel/sdk v1.47.0 h1:zWXEr4j2lFefG87TU6Yg8a7ngfohIKFZHKp0Hf5hC6I= -go.opentelemetry.io/otel/sdk v1.47.0/go.mod h1:VUc24kiOeoGsxG8G9ULx3fWKvB7jMhnGE8Oi607lgR0= -go.opentelemetry.io/otel/sdk/metric v1.47.0 h1:lfISg2j93VT6yqdk9OfUaZmw/GfcZqCCV3jdXtsPnKw= -go.opentelemetry.io/otel/sdk/metric v1.47.0/go.mod h1:ypLp+mW1Nt2x+Szt3b5/i1syodyts49lMOwxpDI3VGw= -go.opentelemetry.io/otel/trace v1.47.0 h1:JOjX/Oci8K94QHddo+bbfya/Ai/nf6/dt9ZfrFNWSrM= -go.opentelemetry.io/otel/trace v1.47.0/go.mod h1:jNaSLa2PZEYFG6fRjJABAu+bw4FS08uDmPg28lTghu0= -go.opentelemetry.io/proto/otlp v1.11.0 h1:5rrYs0Ykyj50sdU/JU0x8etU+LubXWb+gED6TbEdMIk= -go.opentelemetry.io/proto/otlp v1.11.0/go.mod h1:SmVizdCOAm3XBtG1g1NnOdhW6jtddT72hLMhv8VwA8E= -go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= -go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ= -go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ= -go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= -go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= -golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba h1:Ck8QetSgk912qxWLMCKxd0in+aiyBQyDSMae6e/xmpU= -golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba/go.mod h1:50RgIsmK7OwqzTTeqcSXQW8SswW0o8fRcDxmqGluJ8E= -golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg= -golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs= -golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU= -golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg= -golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA= -golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= -golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE= -golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc= -golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI= -golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E= -gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4= -gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E= -google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc h1:rZHRz0ogq4Pid1IKtA/ivHi8XONVF5tqUQXA2TtgpkE= -google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc/go.mod h1:K/n0XLazCJHjD73YuMyNJsN38BPPy1kfFnKxUXaThFU= -google.golang.org/genproto/googleapis/rpc v0.0.0-20260928230214-8a89bd6388cc h1:4bNTbnb44EqGVy9HaQxSY2jnafSUdgV3qHtedvNpDKg= -google.golang.org/genproto/googleapis/rpc v0.0.0-20260928230214-8a89bd6388cc/go.mod h1:OaIUM3+LpYcK2GXM4FTmhWoIq371Owdr+Cc7/BsYHHc= -google.golang.org/grpc v1.84.0 h1:soMyaPJ8pAak5PIQ0DGBUir0XRo2fRoMqhNWMLlLxO0= -google.golang.org/grpc v1.84.0/go.mod h1:ljCht0DrxQrXBDRTZp52Qxh3Ffk8CdYm2sj4O2QN2C0= -google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc= -google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/evanphx/json-patch.v4 v4.13.0 h1:czT3CmqEaQ1aanPc5SdlgQrrEIb8w/wwCvWWnfEbYzo= -gopkg.in/evanphx/json-patch.v4 v4.13.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M= -gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= -gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -k8s.io/kube-openapi v0.0.0-20260721132016-d427ff9ee9ad h1:oXImqH8mQNk7PmvzKhmN3ddJoY6OnyM225MXwGHPm0A= -k8s.io/kube-openapi v0.0.0-20260721132016-d427ff9ee9ad/go.mod h1:0/mqHCVhlumdJ3BhCfnjSZQE037nAhNodh1/hK0T8/I= -sigs.k8s.io/kustomize/api v0.21.1 h1:lzqbzvz2CSvsjIUZUBNFKtIMsEw7hVLJp0JeSIVmuJs= -sigs.k8s.io/kustomize/api v0.21.1/go.mod h1:f3wkKByTrgpgltLgySCntrYoq5d3q7aaxveSagwTlwI= -sigs.k8s.io/kustomize/kyaml v0.21.1 h1:IVlbmhC076nf6foyL6Taw4BkrLuEsXUXNpsE+ScX7fI= -sigs.k8s.io/kustomize/kyaml v0.21.1/go.mod h1:hmxADesM3yUN2vbA5z1/YTBnzLJ1dajdqpQonwBL1FQ= -sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs= -sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4= diff --git a/module/services/telemetry/code/main.go b/module/services/telemetry/code/main.go deleted file mode 100644 index c13fda0ed..000000000 --- a/module/services/telemetry/code/main.go +++ /dev/null @@ -1,129 +0,0 @@ -package main - -import ( - "context" - "fmt" - "log" - "net" - "os" - "os/signal" - "strconv" - "syscall" - - "telemetry/collector" - - codefly "github.com/codefly-dev/sdk-go" - collectorlogsv1 "go.opentelemetry.io/proto/otlp/collector/logs/v1" - collectormetricsv1 "go.opentelemetry.io/proto/otlp/collector/metrics/v1" - collectortracev1 "go.opentelemetry.io/proto/otlp/collector/trace/v1" - "google.golang.org/grpc" -) - -// observabilityConfiguration is the workspace configuration group that carries -// the collector's upstream-exporter settings. Values arrive from either half of -// the group — configurations//observability.env or its .secret.env -// sibling, which is where OTEL_EXPORTER_OTLP_HEADERS lives — because -// WorkspaceValue consults the public namespace first and then the secret one. -const observabilityConfiguration = "observability" - -// exporterKeys are the three settings that make up one exporter configuration. -// They are resolved together, never individually; see resolveExporterConfig. -var exporterKeys = struct{ exporter, endpoint, headers string }{ - exporter: "OBSERVABILITY_EXPORTER", - endpoint: "OTEL_EXPORTER_OTLP_ENDPOINT", - headers: "OTEL_EXPORTER_OTLP_HEADERS", -} - -// workspaceExporterConfig reads all three settings from the observability -// workspace configuration group, reporting whether the group delivered any of -// them. -// -// Reading the group is load-bearing, not cosmetic. telemetry declares -// `observability` as a workspace-configuration-dependency, and a declared group -// reaches a deployed pod ONLY under prefixed names — -// CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__, via envFrom a -// ConfigMap — while codefly.Init builds an in-process map without ever mutating -// the process environment. A bare os.Getenv("OBSERVABILITY_EXPORTER") therefore -// resolved to "" wherever the configuration arrives that way; collector.New -// defaulted that empty value to "debug", and the collector logged and dropped -// every span it received while its own ConfigMap said otlphttp. -func workspaceExporterConfig(ctx context.Context) (collector.Config, bool) { - value := func(key string) string { - resolved, err := codefly.For(ctx).WorkspaceValue(observabilityConfiguration, key) - if err != nil { - return "" - } - return resolved - } - config := collector.Config{ - Exporter: value(exporterKeys.exporter), - Endpoint: value(exporterKeys.endpoint), - Headers: value(exporterKeys.headers), - } - return config, config.Exporter != "" || config.Endpoint != "" || config.Headers != "" -} - -// resolveExporterConfig picks ONE authority for the whole exporter -// configuration: the workspace group if it delivered anything at all, otherwise -// the bare process environment, so a local or non-Codefly run still works. -// -// The choice is deliberately made for the tuple, not per key. collector.New -// cross-validates these three values (debug rejects an endpoint; otlphttp -// requires one), so mixing sources produces configurations no operator wrote. -// Resolving key by key meant a developer who exported OBSERVABILITY_EXPORTER -// and OTEL_EXPORTER_OTLP_ENDPOINT in their shell could no -// longer start the collector: the committed local group supplied -// OBSERVABILITY_EXPORTER=debug while its empty endpoint fell through to the -// shell's, and debug+endpoint is refused. The same split let an ambient -// OTEL_EXPORTER_OTLP_HEADERS — the credential, and a name the OpenTelemetry -// spec makes ubiquitous — attach itself to a workspace-configured endpoint it -// was never issued for, whenever the observability.secret.env half was not -// provisioned. One authority for all three makes both states unrepresentable. -func resolveExporterConfig(ctx context.Context) collector.Config { - if config, delivered := workspaceExporterConfig(ctx); delivered { - return config - } - return collector.Config{ - Exporter: os.Getenv(exporterKeys.exporter), - Endpoint: os.Getenv(exporterKeys.endpoint), - Headers: os.Getenv(exporterKeys.headers), - } -} - -func main() { - ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) - defer stop() - provider, err := codefly.Init(ctx) - if err != nil { - log.Fatal(err) - } - ctx = provider.Inject(ctx) - defer codefly.CatchPanic(ctx) - - grpcNetwork, err := codefly.For(ctx).API("grpc").ResolveNetworkInstance() - if err != nil || grpcNetwork == nil || grpcNetwork.Port == 0 { - log.Fatalf("telemetry: Codefly collector gRPC endpoint is unavailable: %v", err) - } - port := grpcNetwork.Port - sink, err := collector.New(resolveExporterConfig(ctx)) - if err != nil { - log.Fatal(err) - } - listener, err := net.Listen("tcp", net.JoinHostPort("", strconv.Itoa(int(port)))) - if err != nil { - log.Fatal(err) - } - server := grpc.NewServer() - collectortracev1.RegisterTraceServiceServer(server, sink.TraceService()) - collectormetricsv1.RegisterMetricsServiceServer(server, sink.MetricsService()) - collectorlogsv1.RegisterLogsServiceServer(server, sink.LogsService()) - go func() { - if err := server.Serve(listener); err != nil { - log.Printf("telemetry: serve: %v", err) - stop() - } - }() - fmt.Printf("OpenTelemetry collector listening on Codefly gRPC endpoint %d\n", port) - <-ctx.Done() - server.GracefulStop() -} diff --git a/module/services/telemetry/code/main_test.go b/module/services/telemetry/code/main_test.go deleted file mode 100644 index d7002508f..000000000 --- a/module/services/telemetry/code/main_test.go +++ /dev/null @@ -1,98 +0,0 @@ -package main - -import ( - "context" - "testing" - - "telemetry/collector" - - codefly "github.com/codefly-dev/sdk-go" - "github.com/stretchr/testify/require" -) - -// clearWorkspaceObservability removes every carrier the observability group can -// arrive under, so a test that means "no workspace configuration" actually gets -// it. Without this a test passes only because a neighbouring test happened to -// restore these first, and fails wherever they are genuinely exported — which is -// any process started by `codefly run`. -func clearWorkspaceObservability(t *testing.T) { - t.Helper() - for _, prefix := range []string{ - "CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__", - "CODEFLY__WORKSPACE_SECRET_CONFIGURATION__OBSERVABILITY__", - } { - for _, key := range []string{ - "OBSERVABILITY_EXPORTER", - "OTEL_EXPORTER_OTLP_ENDPOINT", - "OTEL_EXPORTER_OTLP_HEADERS", - } { - t.Setenv(prefix+key, "") - } - } -} - -// TestResolveExporterConfigPrefersWorkspaceValue is the regression test for the -// boundary that broke in deployment: workspace configuration reaches the pod -// under prefixed names only, so reading the bare name returned "" and the -// collector silently downgraded itself to the debug exporter. -func TestResolveExporterConfigPrefersWorkspaceValue(t *testing.T) { - clearWorkspaceObservability(t) - t.Setenv("CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__OBSERVABILITY_EXPORTER", "otlphttp") - t.Setenv("CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__OTEL_EXPORTER_OTLP_ENDPOINT", - "https://collector.example") - t.Setenv("CODEFLY__WORKSPACE_SECRET_CONFIGURATION__OBSERVABILITY__OTEL_EXPORTER_OTLP_HEADERS", - "Authorization=Bearer+workspace") - // The bare names are what main.go used to read; they must lose to the - // prefixed workspace values above — including the header, which is the value - // that carries the credential. - t.Setenv("OBSERVABILITY_EXPORTER", "debug") - t.Setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://wrong-backend.example") - t.Setenv("OTEL_EXPORTER_OTLP_HEADERS", "Authorization=Bearer+ambient") - require.NoError(t, codefly.LoadEnvironmentVariables()) - - config := resolveExporterConfig(context.Background()) - require.Equal(t, "otlphttp", config.Exporter) - require.Equal(t, "https://collector.example", config.Endpoint) - require.Equal(t, "Authorization=Bearer+workspace", config.Headers) -} - -// TestResolveExporterConfigTakesWholeTupleFromOneAuthority is the regression -// test for per-key resolution. The workspace group selects debug and says -// nothing about an endpoint or headers; the process environment carries both. -// Resolving key by key produced debug+endpoint — a combination collector.New -// refuses — so a developer who had exported these in their shell could no -// longer start the collector at all. It also -// stopped an ambient OTEL_EXPORTER_OTLP_HEADERS credential from being attached -// to an endpoint it was never issued for. -func TestResolveExporterConfigTakesWholeTupleFromOneAuthority(t *testing.T) { - clearWorkspaceObservability(t) - t.Setenv("CODEFLY__WORKSPACE_CONFIGURATION__OBSERVABILITY__OBSERVABILITY_EXPORTER", "debug") - t.Setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://shell-backend.example") - t.Setenv("OTEL_EXPORTER_OTLP_HEADERS", "Authorization=Bearer+ambient") - require.NoError(t, codefly.LoadEnvironmentVariables()) - - config := resolveExporterConfig(context.Background()) - require.Equal(t, "debug", config.Exporter) - require.Empty(t, config.Endpoint, "endpoint must not be mixed in from the process environment") - require.Empty(t, config.Headers, "an ambient credential must not reach a workspace-selected exporter") - - // The consequence the mixing produced: the collector refused to start. - _, err := collector.New(config) - require.NoError(t, err) -} - -// TestResolveExporterConfigFallsBackToProcessEnvironment keeps a local or -// non-Codefly run working: with no workspace configuration injected, the bare -// process variables are still honoured — as a complete tuple. -func TestResolveExporterConfigFallsBackToProcessEnvironment(t *testing.T) { - clearWorkspaceObservability(t) - t.Setenv("OBSERVABILITY_EXPORTER", "otlphttp") - t.Setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "https://collector.example") - t.Setenv("OTEL_EXPORTER_OTLP_HEADERS", "Authorization=Bearer+ambient") - require.NoError(t, codefly.LoadEnvironmentVariables()) - - config := resolveExporterConfig(context.Background()) - require.Equal(t, "otlphttp", config.Exporter) - require.Equal(t, "https://collector.example", config.Endpoint) - require.Equal(t, "Authorization=Bearer+ambient", config.Headers) -} diff --git a/module/services/telemetry/code/pkg/gen/saas/telemetry/v1/schema_marker.pb.go b/module/services/telemetry/code/pkg/gen/saas/telemetry/v1/schema_marker.pb.go deleted file mode 100644 index 6821690f7..000000000 --- a/module/services/telemetry/code/pkg/gen/saas/telemetry/v1/schema_marker.pb.go +++ /dev/null @@ -1,117 +0,0 @@ -// Code generated by protoc-gen-go. DO NOT EDIT. -// versions: -// protoc-gen-go v1.36.11 -// protoc (unknown) -// source: saas/telemetry/v1/schema_marker.proto - -package telemetryv1 - -import ( - reflect "reflect" - sync "sync" - unsafe "unsafe" - - protoreflect "google.golang.org/protobuf/reflect/protoreflect" - protoimpl "google.golang.org/protobuf/runtime/protoimpl" -) - -const ( - // Verify that this generated code is sufficiently up-to-date. - _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) - // Verify that runtime/protoimpl is sufficiently up-to-date. - _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) -) - -// SchemaMarker keeps the Codefly protobuf sync input valid without duplicating -// the standard OpenTelemetry Protocol service definitions. -type SchemaMarker struct { - state protoimpl.MessageState `protogen:"open.v1"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache -} - -func (x *SchemaMarker) Reset() { - *x = SchemaMarker{} - mi := &file_saas_telemetry_v1_schema_marker_proto_msgTypes[0] - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - ms.StoreMessageInfo(mi) -} - -func (x *SchemaMarker) String() string { - return protoimpl.X.MessageStringOf(x) -} - -func (*SchemaMarker) ProtoMessage() {} - -func (x *SchemaMarker) ProtoReflect() protoreflect.Message { - mi := &file_saas_telemetry_v1_schema_marker_proto_msgTypes[0] - if x != nil { - ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) - if ms.LoadMessageInfo() == nil { - ms.StoreMessageInfo(mi) - } - return ms - } - return mi.MessageOf(x) -} - -// Deprecated: Use SchemaMarker.ProtoReflect.Descriptor instead. -func (*SchemaMarker) Descriptor() ([]byte, []int) { - return file_saas_telemetry_v1_schema_marker_proto_rawDescGZIP(), []int{0} -} - -var File_saas_telemetry_v1_schema_marker_proto protoreflect.FileDescriptor - -const file_saas_telemetry_v1_schema_marker_proto_rawDesc = "" + - "\n" + - "%saas/telemetry/v1/schema_marker.proto\x12\x11saas.telemetry.v1\"\x0e\n" + - "\fSchemaMarkerB\xc1\x01\n" + - "\x15com.saas.telemetry.v1B\x11SchemaMarkerProtoP\x01Z/telemetry/pkg/gen/saas/telemetry/v1;telemetryv1\xa2\x02\x03STX\xaa\x02\x11Saas.Telemetry.V1\xca\x02\x11Saas\\Telemetry\\V1\xe2\x02\x1dSaas\\Telemetry\\V1\\GPBMetadata\xea\x02\x13Saas::Telemetry::V1b\x06proto3" - -var ( - file_saas_telemetry_v1_schema_marker_proto_rawDescOnce sync.Once - file_saas_telemetry_v1_schema_marker_proto_rawDescData []byte -) - -func file_saas_telemetry_v1_schema_marker_proto_rawDescGZIP() []byte { - file_saas_telemetry_v1_schema_marker_proto_rawDescOnce.Do(func() { - file_saas_telemetry_v1_schema_marker_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_saas_telemetry_v1_schema_marker_proto_rawDesc), len(file_saas_telemetry_v1_schema_marker_proto_rawDesc))) - }) - return file_saas_telemetry_v1_schema_marker_proto_rawDescData -} - -var file_saas_telemetry_v1_schema_marker_proto_msgTypes = make([]protoimpl.MessageInfo, 1) -var file_saas_telemetry_v1_schema_marker_proto_goTypes = []any{ - (*SchemaMarker)(nil), // 0: saas.telemetry.v1.SchemaMarker -} -var file_saas_telemetry_v1_schema_marker_proto_depIdxs = []int32{ - 0, // [0:0] is the sub-list for method output_type - 0, // [0:0] is the sub-list for method input_type - 0, // [0:0] is the sub-list for extension type_name - 0, // [0:0] is the sub-list for extension extendee - 0, // [0:0] is the sub-list for field type_name -} - -func init() { file_saas_telemetry_v1_schema_marker_proto_init() } -func file_saas_telemetry_v1_schema_marker_proto_init() { - if File_saas_telemetry_v1_schema_marker_proto != nil { - return - } - type x struct{} - out := protoimpl.TypeBuilder{ - File: protoimpl.DescBuilder{ - GoPackagePath: reflect.TypeOf(x{}).PkgPath(), - RawDescriptor: unsafe.Slice(unsafe.StringData(file_saas_telemetry_v1_schema_marker_proto_rawDesc), len(file_saas_telemetry_v1_schema_marker_proto_rawDesc)), - NumEnums: 0, - NumMessages: 1, - NumExtensions: 0, - NumServices: 0, - }, - GoTypes: file_saas_telemetry_v1_schema_marker_proto_goTypes, - DependencyIndexes: file_saas_telemetry_v1_schema_marker_proto_depIdxs, - MessageInfos: file_saas_telemetry_v1_schema_marker_proto_msgTypes, - }.Build() - File_saas_telemetry_v1_schema_marker_proto = out.File - file_saas_telemetry_v1_schema_marker_proto_goTypes = nil - file_saas_telemetry_v1_schema_marker_proto_depIdxs = nil -} diff --git a/module/services/telemetry/proto/README.md b/module/services/telemetry/proto/README.md deleted file mode 100644 index e70d7c6dc..000000000 --- a/module/services/telemetry/proto/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Telemetry protobuf input - -The telemetry gateway implements the standard OpenTelemetry Protocol services -from `go.opentelemetry.io/proto/otlp`; it does not duplicate those upstream -service definitions. - -The local `SchemaMarker` is deliberately code-generation-free. It keeps the -Codefly Go gRPC agent's required protobuf sync input valid for a service that -depends exclusively on upstream wire schemas. diff --git a/module/services/telemetry/proto/buf.gen.yaml b/module/services/telemetry/proto/buf.gen.yaml deleted file mode 100644 index 63afd5b44..000000000 --- a/module/services/telemetry/proto/buf.gen.yaml +++ /dev/null @@ -1,11 +0,0 @@ -version: v2 -clean: true -managed: - enabled: true - override: - - file_option: go_package_prefix - value: telemetry/pkg/gen -plugins: - - local: protoc-gen-go - out: ../code/pkg/gen - opt: paths=source_relative diff --git a/module/services/telemetry/proto/buf.yaml b/module/services/telemetry/proto/buf.yaml deleted file mode 100644 index cb3b14b9f..000000000 --- a/module/services/telemetry/proto/buf.yaml +++ /dev/null @@ -1,6 +0,0 @@ -version: v2 -modules: - - path: . -lint: - use: - - STANDARD diff --git a/module/services/telemetry/proto/saas/telemetry/v1/schema_marker.proto b/module/services/telemetry/proto/saas/telemetry/v1/schema_marker.proto deleted file mode 100644 index ad48f2919..000000000 --- a/module/services/telemetry/proto/saas/telemetry/v1/schema_marker.proto +++ /dev/null @@ -1,7 +0,0 @@ -syntax = "proto3"; - -package saas.telemetry.v1; - -// SchemaMarker keeps the Codefly protobuf sync input valid without duplicating -// the standard OpenTelemetry Protocol service definitions. -message SchemaMarker {} diff --git a/module/services/telemetry/service.codefly.yaml b/module/services/telemetry/service.codefly.yaml deleted file mode 100644 index 3c0893a0b..000000000 --- a/module/services/telemetry/service.codefly.yaml +++ /dev/null @@ -1,26 +0,0 @@ -name: telemetry -version: 0.0.0 -description: | - In-graph OTLP gateway. Accounts resolves this endpoint exclusively - through the Codefly SDK; the collector either logs locally or forwards - trace, metric, and log protobufs to a configured OTLP/HTTP destination. -agent: - kind: codefly:service - name: go-grpc - version: 0.1.58 - publisher: codefly.dev -workspace-configuration-dependencies: - - observability -endpoints: - - name: grpc -spec: - hot-reload: true - protocol-output-dirs: - - code/pkg/gen - deployment: - # No endpoint-ports: go-grpc binds every endpoint on the port Codefly - # allocates for it, so each pod port is its deployed port, which the - # render takes from Codefly. endpoint-ports is only for a port the - # process binds at a fixed number the allocation does not give it. - public-egress-ports: - - 443 diff --git a/module/services/vault/service.codefly.yaml b/module/services/vault/service.codefly.yaml index f1c7b717b..ea310e411 100644 --- a/module/services/vault/service.codefly.yaml +++ b/module/services/vault/service.codefly.yaml @@ -11,7 +11,7 @@ secret-service-configurations: - key: vault_token endpoints: - name: http - visibility: internal + visibility: private spec: transit-key: api-keys deployment: diff --git a/module/tools/composition/composition_test.go b/module/tools/composition/composition_test.go index cf7e16a9f..7f99bf294 100644 --- a/module/tools/composition/composition_test.go +++ b/module/tools/composition/composition_test.go @@ -329,11 +329,18 @@ permissions: func writeCoreInput(t *testing.T, fixture *coreCompositionFixture) { t.Helper() + // The input names the package release it was made for, which the generator + // holds to the package it runs inside: the version is the manifest's, not a + // literal that every release would have to chase. + manifest, err := corecomposition.LoadPackageManifest(findModuleRoot(t)) + if err != nil { + t.Fatalf("load package manifest: %v", err) + } input := corecomposition.CompositionInput{ Schema: "codefly/composition-input/v2", Module: fixture.descriptor.Name, Package: "codefly/saas-starter", - Version: "0.1.0", + Version: manifest.Version, ConsumerRoot: fixture.consumerRoot, Projection: fixture.projection, Contracts: map[string]string{ diff --git a/module/tools/modulepackage/modulepackage_test.go b/module/tools/modulepackage/modulepackage_test.go index a76274bb0..6ffb21567 100644 --- a/module/tools/modulepackage/modulepackage_test.go +++ b/module/tools/modulepackage/modulepackage_test.go @@ -23,7 +23,7 @@ func TestStarterManifestIsTheCoreV2Contract(t *testing.T) { if err != nil { t.Fatal(err) } - if manifest.ID != "codefly/saas-starter" || manifest.Version != "0.1.0" { + if manifest.ID != "codefly/saas-starter" || manifest.Version != "0.2.0" { t.Fatalf("unexpected package identity: %s@%s", manifest.ID, manifest.Version) } // THE SERVICE SET IS DERIVED, NOT COUNTED. This pinned the number 8, and a diff --git a/scripts/ci/ci-performance.test.mjs b/scripts/ci/ci-performance.test.mjs index 10d32ec14..3fcbbb93e 100644 --- a/scripts/ci/ci-performance.test.mjs +++ b/scripts/ci/ci-performance.test.mjs @@ -65,7 +65,8 @@ test('the CLI installer retries resets, verifies downloads, and fails after exha // places, and it is what caught this branch moving the installer to 0.1.179 // while this constant still said 0.1.171. Deriving it from the installer would // make the second assertion below tautological and lose that. -const DEFAULT_CODEFLY_VERSION = 'source-5234ce248b092edf1138a16ec4140dfbc8745522'; +const DEFAULT_CODEFLY_VERSION = '0.1.197'; +const HISTORICAL_SOURCE_VERSION = 'source-5234ce248b092edf1138a16ec4140dfbc8745522'; test('source qualification installs the exact owner commit and never falls back or releases it', () => { const dir = mkdtempSync(join(tmpdir(), 'ci-cli-source-')); @@ -73,14 +74,14 @@ test('source qualification installs the exact owner commit and never falls back writeFileSync(join(dir, 'uname'), '#!/bin/sh\nif [ "$1" = -s ]; then echo Linux; else echo x86_64; fi\n', { mode: 0o755 }); writeFileSync(join(dir, 'go'), '#!/bin/sh\nprintf "%s\\n" "$@" > "$RUNNER_TEMP/go-args"\nprintf "%s\\n" "$GOWORK" "$GOBIN" > "$RUNNER_TEMP/go-env"\nexit "${TEST_GO_EXIT:-0}"\n', { mode: 0o755 }); const pathFile = join(dir, 'github-path'); - const env = { ...process.env, PATH: `${dir}:${process.env.PATH}`, CODEFLY_VERSION: DEFAULT_CODEFLY_VERSION, + const env = { ...process.env, PATH: `${dir}:${process.env.PATH}`, CODEFLY_VERSION: HISTORICAL_SOURCE_VERSION, RUNNER_TEMP: dir, GITHUB_PATH: pathFile, GITHUB_REF: 'refs/pull/1/merge' }; const run = overrides => spawnSync('bash', [join(root, 'scripts/ci/install-codefly.sh')], { encoding: 'utf8', env: { ...env, ...overrides }, }); const success = run({}); assert.equal(success.status, 0, success.stderr); - assert.ok(readFileSync(join(dir, 'go-args'), 'utf8').includes(`github.com/codefly-dev/cli/cmd/codefly@${DEFAULT_CODEFLY_VERSION.slice(7)}\n`)); + assert.ok(readFileSync(join(dir, 'go-args'), 'utf8').includes(`github.com/codefly-dev/cli/cmd/codefly@${HISTORICAL_SOURCE_VERSION.slice(7)}\n`)); assert.equal(readFileSync(join(dir, 'go-env'), 'utf8'), `off\n${dir}/codefly-bin\n`); assert.equal(readFileSync(pathFile, 'utf8'), `${dir}/codefly-bin\n`); writeFileSync(pathFile, ''); diff --git a/scripts/ci/install-codefly.sh b/scripts/ci/install-codefly.sh index 06e379c59..95995c614 100644 --- a/scripts/ci/install-codefly.sh +++ b/scripts/ci/install-codefly.sh @@ -2,7 +2,7 @@ # The hosted CI runner is Linux x64. Keep the release and archive digest paired. set -euo pipefail -version="${CODEFLY_VERSION:-source-5234ce248b092edf1138a16ec4140dfbc8745522}" +version="${CODEFLY_VERSION:-0.1.197}" source_revision="" case "${version}" in source-5234ce248b092edf1138a16ec4140dfbc8745522) @@ -63,6 +63,11 @@ case "${version}" in # Previous generator pin; retained for explicit historical qualifications. checksum=6e7666ef095815a77c10fb1644c69e17b0e8869476c18713bfaa496971b5c45f ;; + 0.1.197) + # Includes the merged lifecycle/artifact-cache changes (cli#994) and + # environment-wide standard OTEL delivery (cli#1097). + checksum=f426ca8ad8c13be4a8aba63c3618a866cb35462a9eb08def2a59b4d9b8290827 + ;; 0.1.185) # Released Core v0.18.0 and cli#947: scope-slot delivery, endpoint-served # Runnable derivation, and CLI-owned module contracts.codefly.yaml. diff --git a/topology_declarations_test.go b/topology_declarations_test.go new file mode 100644 index 000000000..cac1bc8a0 --- /dev/null +++ b/topology_declarations_test.go @@ -0,0 +1,58 @@ +package main + +import ( + "errors" + "strings" + "testing" + + "github.com/codefly-dev/core/resources" + "gopkg.in/yaml.v3" +) + +func TestTopologyConsumerDerivedInternalReach(t *testing.T) { + for _, exported := range []bool{false, true} { + visibility, err := endpointVisibility("internal", "", exported) + if err != nil || visibility != "module" { + t.Fatalf("internal exported=%v: %q, %v", exported, visibility, err) + } + } + visibility, err := endpointVisibility("", "", true) + if err != nil || visibility != "module" { + t.Fatalf("default interface reach: %q, %v", visibility, err) + } + visibility, err = endpointVisibility("", "", false) + if err != nil || visibility != "private" { + t.Fatalf("default service reach: %q, %v", visibility, err) + } +} + +func TestTopologyRefusesAuthoredConsumersByKeyPresence(t *testing.T) { + for _, key := range []string{"allow-modules", "allow_modules", "allowModules", "Allow Modules"} { + for _, value := range []string{"[\"*\"]", "[example]", "[]", "null"} { + document := []byte("visibility: internal\n" + key + ": " + value + "\n") + for _, endpoint := range []any{&topologyManifestEndpoint{}, &topologyInterface{}} { + err := yaml.Unmarshal(document, endpoint) + if !errors.Is(err, resources.ErrInvalidEndpointDeclaration) { + t.Fatalf("%T %s=%s: %v", endpoint, key, value, err) + } + } + } + } +} + +func TestBootstrapPoliciesSelectImmutableJobByServiceLabel(t *testing.T) { + // Postgres appends a digest to the immutable Job name. Kubernetes copies + // that name into job-name; the agent's service label remains stable. + for _, policy := range []kubeObject{ + bootstrapJobIngressPolicy("example", nil, "store", "store", []uint32{5432}), + bootstrapJobEgressPolicy("example", nil, "store", "store", []uint32{5432}), + } { + document, err := yaml.Marshal(policy) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(document), "job-name:") || !strings.Contains(string(document), "codefly.dev/bootstrap-service: store") { + t.Fatalf("policy cannot select a content-addressed bootstrap Job:\n%s", document) + } + } +} diff --git a/topology_manifests.go b/topology_manifests.go index 797cbc150..d6d98007f 100644 --- a/topology_manifests.go +++ b/topology_manifests.go @@ -12,6 +12,7 @@ import ( basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" "github.com/codefly-dev/core/network" + "github.com/codefly-dev/core/resources" "gopkg.in/yaml.v3" ) @@ -54,14 +55,8 @@ type serviceTopologyManifest struct { } `yaml:"service-dependencies"` WorkspaceConfigurationDependencies []string `yaml:"workspace-configuration-dependencies"` SecretServiceConfigurations []topologySecretServiceConfiguration `yaml:"secret-service-configurations"` - Endpoints []struct { - Name string `yaml:"name"` - Visibility string `yaml:"visibility"` - API string `yaml:"api"` - AllowModules []string `yaml:"allow-modules"` - Location string `yaml:"location"` - } `yaml:"endpoints"` - Spec map[string]any `yaml:"spec"` + Endpoints []topologyManifestEndpoint `yaml:"endpoints"` + Spec map[string]any `yaml:"spec"` } // deploymentSpecManifest is the spec.deployment block of a service manifest, @@ -108,11 +103,11 @@ func assembleDeploymentTopology(moduleDir, moduleName string, services []service } for i := range topology.Interface { exported := &topology.Interface[i] - visibility, err := endpointVisibility(exported.Visibility, "", exported.AllowModules) + visibility, err := endpointVisibility(exported.Visibility, "", true) if err != nil { return deploymentTopology{}, fmt.Errorf("interface %s/%s: %w", exported.Service, exported.Endpoint, err) } - exported.Visibility, exported.AllowModules = visibility, nil + exported.Visibility = visibility } ordered := append([]serviceDefinition(nil), services...) sort.Slice(ordered, func(i, j int) bool { return ordered[i].name < ordered[j].name }) @@ -171,7 +166,7 @@ func loadServiceTopology(moduleName string, service serviceDefinition) (topology // declares none and takes the Service port. coreEndpoints := make([]*basev0.Endpoint, 0, len(manifest.Endpoints)) for _, endpoint := range manifest.Endpoints { - visibility, err := endpointVisibility(endpoint.Visibility, endpoint.Location, endpoint.AllowModules) + visibility, err := endpointVisibility(endpoint.Visibility, endpoint.Location, false) if err != nil { return topologyService{}, fmt.Errorf("service %q endpoint %q: %w", service.name, endpoint.Name, err) } @@ -186,7 +181,7 @@ func loadServiceTopology(moduleName string, service serviceDefinition) (topology seen := make(map[string]bool, len(manifest.Endpoints)) for _, endpoint := range manifest.Endpoints { api := endpointAPI(endpoint.Name, endpoint.API) - visibility, err := endpointVisibility(endpoint.Visibility, endpoint.Location, endpoint.AllowModules) + visibility, err := endpointVisibility(endpoint.Visibility, endpoint.Location, false) if err != nil { return topologyService{}, err } @@ -231,48 +226,26 @@ func endpointAPI(name, api string) string { return api } -// endpointVisibility projects the authored Codefly axes into the host's -// deployment catalog categories. Its MODULE category means all composed -// modules, so a narrower allow-list cannot be represented and must refuse. -// -// THIS IS THE SECOND COPY OF THIS PROJECTION. The shipped generator holds the -// other, as manifestCatalogVisibility in -// module/services/accounts/code/pkg/cataloggen/deployment_model.go, and the two -// are in separate Go modules so neither can import the other. They must agree, -// and when they did not the symptom was a CI red nobody expected: this copy still -// required `allow-modules: ["*"]` after the manifests stopped authoring it, so -// TestShippedModuleAuthorityPortIsAllocatedNotDeclared failed on a module -// manifest the other copy accepted. TestEndpointCatalogProjectionPreservesOrRefusesPolicy -// and the accounts-side table are the pin; a change here needs the same change -// there. -// -// `internal` with NO allow-list is the v0.14.0 spelling of what `internal` plus -// the wildcard said, and maps to the same category: core refuses that key on an -// export by name, so the two spellings mean one thing and which modules reach the -// endpoint is decided by their own declared dependencies either way. The wildcard -// stays accepted for older heads of this repository; a NARROWER list still -// refuses, because this catalog cannot express "some modules" and admitting it -// would silently widen the generated policy to every module. -func endpointVisibility(visibility, location string, allowed []string) (string, error) { +// endpointVisibility projects the authored reach into the catalog's legacy MODULE +// category. It does not grant callers: only declared dependency edges render +// network access, and the composing Core derives cross-module reach from asks. +func endpointVisibility(visibility, location string, exported bool) (string, error) { if visibility == "" { visibility = "private" + if exported { + visibility = "internal" + } } if location != "" { - if location != "external" || visibility != "private" || len(allowed) != 0 { + if location != "external" || visibility != "private" { return "", fmt.Errorf("unsupported external endpoint policy") } return "external", nil } switch visibility { case "private", "public": - if len(allowed) != 0 { - return "", fmt.Errorf("allow-modules requires internal visibility") - } return visibility, nil case "internal": - if len(allowed) != 0 && (len(allowed) != 1 || allowed[0] != "*") { - return "", fmt.Errorf("deployment catalog accepts internal visibility with no allow-modules, or the legacy [*]; a narrower export needs an allow-list-aware catalog") - } return "module", nil default: return "", fmt.Errorf("unsupported authored endpoint visibility %q", visibility) @@ -321,3 +294,14 @@ func loadDeployJobs(moduleDir string) ([]topologyDeployJob, error) { } return manifest.DeployJobs, nil } + +type topologyManifestEndpoint = resources.Endpoint + +func (endpoint *topologyInterface) UnmarshalYAML(node *yaml.Node) error { + var declaration resources.InterfaceEndpoint + if err := node.Decode(&declaration); err != nil { + return err + } + endpoint.Service, endpoint.Endpoint, endpoint.Visibility = declaration.Service, declaration.Endpoint, declaration.Visibility + return nil +} diff --git a/topology_ports_test.go b/topology_ports_test.go index 63d3ca478..dc3aad2b6 100644 --- a/topology_ports_test.go +++ b/topology_ports_test.go @@ -23,10 +23,8 @@ endpoints: - name: authority api: grpc visibility: internal - allow-modules: ["*"] - name: connect visibility: internal - allow-modules: ["*"] - name: grpc - name: rest spec: diff --git a/topology_visibility_test.go b/topology_visibility_test.go index e299d9e8b..21188a15a 100644 --- a/topology_visibility_test.go +++ b/topology_visibility_test.go @@ -13,29 +13,20 @@ import "testing" func TestEndpointCatalogProjectionPreservesOrRefusesPolicy(t *testing.T) { for _, tc := range []struct { visibility, location string - allowed []string + exported bool want string }{ - // `internal` with NO allow-list is what the manifests author at core - // v0.14.0, and it maps to the same category the legacy wildcard did. - {"internal", "", nil, "module"}, - {"internal", "", []string{}, "module"}, - {"internal", "", []string{"*"}, "module"}, - {"private", "external", nil, "external"}, - {"public", "", nil, "public"}, - {"", "", nil, "private"}, - // A NARROWER list still refuses: this catalog cannot express "some - // modules", so admitting one would silently widen the generated policy to - // every module. - {"internal", "", []string{"example"}, ""}, - {"internal", "", []string{"*", "example"}, ""}, - {"public", "", []string{"example"}, ""}, - {"module", "", nil, ""}, - {"external", "", nil, ""}, - {"private", "elsewhere", nil, ""}, - {"internal", "external", []string{"example"}, ""}, + {"internal", "", false, "module"}, + {"private", "external", false, "external"}, + {"public", "", false, "public"}, + {"", "", false, "private"}, + {"internal", "", true, "module"}, + {"module", "", false, ""}, + {"external", "", false, ""}, + {"private", "elsewhere", false, ""}, + {"internal", "external", false, ""}, } { - got, err := endpointVisibility(tc.visibility, tc.location, tc.allowed) + got, err := endpointVisibility(tc.visibility, tc.location, tc.exported) if tc.want == "" { if err == nil { t.Errorf("accepted unsupported policy %#v as %q", tc, got) diff --git a/vault_binding_test.go b/vault_binding_test.go index 405f57259..6098250d1 100644 --- a/vault_binding_test.go +++ b/vault_binding_test.go @@ -6,6 +6,8 @@ import ( "strings" "testing" + basev0 "github.com/codefly-dev/core/generated/go/codefly/base/v0" + "github.com/codefly-dev/core/resources" "gopkg.in/yaml.v3" ) @@ -31,6 +33,47 @@ const ( meshGroupDefaults = "module/configurations/local/internal-transport.env" ) +// Vault is a dependency of accounts within this module, never an exported +// endpoint. Declare that boundary on the service too: older supported CLIs +// make an omitted module-interface endpoint private without removing its +// service's allow-list, which newer SDKs correctly reject as invalid. +func TestVaultEndpointIsPrivateAndResolvesOnlyWithinItsModule(t *testing.T) { + t.Parallel() + data, err := os.ReadFile(vaultServiceManifest) + if err != nil { + t.Fatal(err) + } + var service resources.Service + if err := yaml.Unmarshal(data, &service); err != nil { + t.Fatal(err) + } + if len(service.Endpoints) != 1 || service.Endpoints[0].Name != "http" { + t.Fatalf("unexpected vault endpoints: %#v", service.Endpoints) + } + endpoint := service.Endpoints[0] + if endpoint.Visibility != resources.VisibilityPrivate || len(endpoint.AllowModules) != 0 { + t.Fatalf("vault/http must declare private visibility without allow-modules; got %q, %v", endpoint.Visibility, endpoint.AllowModules) + } + endpoint.Module, endpoint.Service = "saas-starter", "vault" + declaration, err := endpoint.Proto() + if err != nil { + t.Fatal(err) + } + dependencies := []*resources.ServiceDependency{{ + Module: "saas-starter", Name: "vault", + Endpoints: []*resources.EndpointReference{{Name: "http"}}, + }} + mappings := []*basev0.NetworkMapping{{Endpoint: declaration}} + composition := &resources.Workspace{Name: "example-workspace", Modules: []*resources.ModuleReference{{Name: "saas-starter"}, {Name: "example"}}} + resolved, err := resources.ResolveDependencyNetworkMappings(composition, "saas-starter", dependencies, mappings) + if err != nil || len(resolved) != 1 { + t.Fatalf("same-module vault dependency: resolved %d mappings, error %v", len(resolved), err) + } + if _, err := resources.ResolveDependencyNetworkMappings(composition, "example", dependencies, mappings); err == nil { + t.Fatal("vault/http was handed to a different module") + } +} + // vaultDurableRenderFloor is the first vault agent release whose deployed render // is durable: `vault server` with integrated raft storage on a retained // PersistentVolumeClaim, auto-unsealed by the seal the environment supplies,