From 99d07532dd1df3f416ade3231b668da230510bb7 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 10:09:10 +0900 Subject: [PATCH 01/16] docs: design verify-only external DNS provider --- docs/design/README.md | 1 + docs/design/controllers-webhooks.md | 31 ++- docs/design/core-api.md | 18 +- docs/design/external-provider.md | 356 ++++++++++++++++++++++++++++ docs/design/open-decisions.md | 3 + docs/design/overview.md | 8 +- docs/design/provider-extension.md | 7 +- docs/design/test-policy.md | 6 +- docs/design/ui.md | 10 +- 9 files changed, 417 insertions(+), 23 deletions(-) create mode 100644 docs/design/external-provider.md diff --git a/docs/design/README.md b/docs/design/README.md index 018e533..5e51522 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -11,6 +11,7 @@ - [Controllers and Webhooks](./controllers-webhooks.md): controller responsibilities, modules, acceptance boundary, and admission webhook. - [Cloudflare Provider](./cloudflare-provider.md): Cloudflare identity, schema, ZoneClass parameters, provider data, reconciliation, deletion, and record mapping. - [Route 53 Provider](./route53-provider.md): Route 53 identity, provider settings, and record-set provider schema. +- [External Provider](./external-provider.md): verify-only provider for customer-managed DNS: resolver policy, exact-set verification, conditions, scheduling bounds, deletion semantics, and conversion decision. - [Gateway API Integration](./gateway-api-integration.md): `HTTPRoute` / `Gateway` DNS report, Zone selection, and generated `RecordSet` behavior. - [UI](./ui.md): reusable DNS API UI package, screens, workflows, components, and mockup design. - [Headlamp Desktop Plugin](./headlamp-plugin.md): Headlamp Desktop plugin package, distribution, and Desktop-specific behavior. diff --git a/docs/design/controllers-webhooks.md b/docs/design/controllers-webhooks.md index f119122..2036cae 100644 --- a/docs/design/controllers-webhooks.md +++ b/docs/design/controllers-webhooks.md @@ -25,8 +25,31 @@ Admission validation must not reject `Zone`, `RecordSet`, or `ZoneUnit` solely b All claim conditions include `observedGeneration`. If a resource moves out of a provider controller's scope, that controller stops updating `ZoneUnit.status`. Existing claim conditions can be recognized as old through `observedGeneration`. +When projecting a current provider condition from `ZoneUnit.status.zone` or `ZoneUnit.status.recordSets[]`, core preserves `Status`, `Reason`, and `Message`, including an empty Message. It must not replace a provider diagnostic with its reason string. This applies to both Accepted and Programmed on Zone and RecordSet and does not weaken UID/generation checks, composition acceptance, or the use of core-generated messages for stale/missing provider results. Providers must supply bounded, non-secret messages suitable for public claim status. + Provider implementation differences are controlled by common conditions, reasons, status ownership, support levels, and conformance tests. The core does not hide differences by providing one shared DNS controller implementation; providers give a consistent user experience by satisfying the same contract. +## Provider Enablement (required contract; not implemented at 87aa1dc) + +Provider enablement is a runtime and permission boundary, not merely a switch for rendering Provider manifests. The required chart contract passes explicit boolean manager flags. + +At the current revision only `--gateway-endpoint-controller-enabled` and `--service-endpoint-controller-enabled` exist as enable flags; `providers.*.enabled` gate manifests only, and identity controllers, identity webhooks, the Route 53 conversion handler, identity RBAC, and Secret read permission are always on (`app/operator/cmd/manager/main.go`, `deploy/charts/dns-api/templates/rbac.yaml`). The flags below are introduced with the external provider. + +| Chart value | Manager flag (new) | Default | Enabled scope | +| --- | --- | --- | --- | +| `providers.route53.enabled` | `--route53-controller-enabled` | true | Route53Identity, Route 53 ZoneClass/ZoneUnit controllers, identity webhook, provider identity RBAC, and the Route 53 Provider manifest. | +| `providers.cloudflare.enabled` | `--cloudflare-controller-enabled` | true | CloudflareIdentity, Cloudflare ZoneClass/ZoneUnit controllers, identity webhook, identity/Secret reads and watches, and the Cloudflare Provider manifest. | +| `providers.external.enabled` | `--external-controller-enabled` | false | External ZoneClass/ZoneUnit controllers, Provider, endpoint conversion handler/capability/APIService, and conversion-create RBAC. No identity controller, identity webhook, or credential reads. | +| `providers.route53.endpoint.conversionAPI.enabled` | `--route53-endpoint-conversion-enabled` | true | Route 53 conversion handler/capability/APIService and conversion-create RBAC, only when Route 53 is enabled. | + +The manager must not call disabled providers' controller or identity-webhook setup functions, register their conversion handlers, construct credential clients, or create their identity/Secret informers. Scheme/CRD types may remain available for reading existing objects. Provider-only RBAC and the corresponding ValidatingWebhookConfiguration entries are gated with registrations; leaving an unserved identity webhook in admission is not disabling a provider. Route 53 conversion requires both Route 53 flags. Cloudflare has no endpoint conversion handler in the current implementation; enablement must not invent an unserved capability. + +Namespace reads and core/endpoint RBAC remain independent. Secret get/list/watch is granted only for an enabled credential consumer (Cloudflare in the built-in chart), not to the external provider or an all-disabled deployment. Mounted serving TLS certificates do not require Kubernetes Secret API read permission. Helm and Kustomize wiring must provide equivalent admission coverage, including ZoneClass status and enabled provider identity spec/status handlers. Existing `gatewayEndpoint.enabled` / `--gateway-endpoint-controller-enabled` and `serviceEndpoint.enabled` / `--service-endpoint-controller-enabled` continue to gate their own registrations and source-only permissions. + +Core composition, endpoint translation, core CRDs, and core schema/status admission are retained when providers are disabled. External enablement requires its conversion API to be reachable through the existing webhook server with verified serving TLS and CA wiring; reject incompatible chart settings instead of publishing an unusable capability or using `insecureSkipTLSVerify`. Provider/controller-name/version flags select runtime bindings and cannot override a disabled flag. Changing flags does not complete cleanup of existing claims; operators must not disable a provider while relying on its finalizers or continuing verification. + +These gates are the required design contract; manifest-only enable switches do not satisfy it. + ## Implementation Modules Implementation modules are controller ownership boundaries. One Kubernetes controller has one owning module, and one owning module owns at most one controller. The owning module contains the reconciler entrypoint, watch mapping, finalizer writes, status writes, and controller-specific planning. Shared helpers used by more than one controller must live in a non-controller module and must not own watches, finalizers, or status writes. @@ -86,7 +109,7 @@ The webhook server runs as part of the controller-runtime manager and listens on Webhook TLS certificates are issued by cert-manager. Tilt installs cert-manager as a Helm chart. dns-api manifests apply `Certificate`, `Issuer`, and `ValidatingWebhookConfiguration`. cert-manager CA injection sets the webhook `caBundle`. -Core validating webhook covers at least spec create/update for `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`, and status update for `Zone`, `RecordSet`, and `ZoneUnit`. Provider-specific validating webhooks cover provider-specific resources such as `Route53Identity` and `CloudflareIdentity`. The core webhook must not import provider API packages such as `api/route53` or `api/cloudflare`. +Core validating webhook covers at least spec create/update for `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`, and status update for `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`. Provider-specific validating webhooks cover enabled provider-specific resources such as `Route53Identity` and `CloudflareIdentity`; external has no identity admission handler. The core webhook must not import provider API packages such as `api/route53` or `api/cloudflare`. Provider controllers do not fully trust the webhook. The webhook is the primary user feedback boundary, but controllers keep minimal safety checks. Controller-side validation failure is returned as `Accepted=False` or `Programmed=False`. @@ -139,7 +162,11 @@ validate ZoneClass: provider, providerVersion = resolve ProviderVersion(spec.provider) - validate spec.identityRef shape by providerVersion.identity.resource + if providerVersion.identity is present: + require spec.identityRef with nonempty name + validate spec.identityRef shape by providerVersion.identity.resource + else: + reject spec.identityRef validate spec.parameters by providerVersion.zoneClass.schemas.parameters validate providerVersion.zoneClass.validationRules validate spec.allowedZones diff --git a/docs/design/core-api.md b/docs/design/core-api.md index 7f3a899..43aaa2d 100644 --- a/docs/design/core-api.md +++ b/docs/design/core-api.md @@ -9,13 +9,13 @@ - `provider.name`: `Provider.metadata.name`. Immutable. Because it references cluster-scoped `Provider`, it does not include namespace or kind. - `provider.version`: `Provider.spec.versions[].name`. Immutable. - `controllerName`: provider controller instance that reconciles `ZoneUnit` resources reachable through this class. Required and immutable. -- `identityRef`: provider identity reference. Required. `identityRef.name` references the provider identity resource declared by the selected Provider version in the same namespace as the `ZoneClass`. +- `identityRef`: optional provider identity reference. Required with a nonempty `name` if and only if the selected Provider version declares `identity`; forbidden otherwise. When present, it references the declared identity resource in the same namespace as the `ZoneClass`. - `parameters`: provider-specific inline settings. Validated by the referenced Provider version `zoneClass.schemas.parameters`. - `allowedZones`: namespace conditions for `Zone` resources that may use this `ZoneClass`. `ZoneClass.spec.controllerName` uses the Gateway API-style domain-prefixed path form. It must include both a DNS subdomain prefix and a path part after `/`. Valid examples are `route53.dns.appthrust.io/controller` and `cloudflare.dns.appthrust.io/prod-a`; `route53.dns.appthrust.io` is invalid. -`ZoneClass.spec.identityRef` contains only `name`. It does not contain namespace, group, or kind. Cross-namespace identity references are not part of the initial design; identity resources live with the `ZoneClass` in the same platform namespace. The referenced Provider version declares the identity `group` and `kind`, so repeating them in every `ZoneClass` would create an unnecessary mismatch surface. +`ZoneClass.spec.identityRef`, when present, contains only `name`. It does not contain namespace, group, or kind. Cross-namespace identity references are not part of the initial design; identity resources live with the `ZoneClass` in the same platform namespace. The referenced Provider version declares the identity `group` and `kind`, so repeating them in every `ZoneClass` would create an unnecessary mismatch surface. Credentialless providers omit the field, rather than using an empty or sentinel reference. The Go field is `IdentityRef *LocalObjectReference` with JSON tag `identityRef,omitempty`; the CRD does not structurally require it, and core admission enforces the Provider-dependent requirement. Both generated CRD distributions, Go callers (`internal/go/providers/{route53,cloudflare}/controllers/{zoneclass,zoneunit}`), and the UI ZoneClass form are updated: the form omits `identityRef` and the identity picker step when the selected Provider version declares no `identity`. Route 53 and Cloudflare still require an identity. `ZoneClass.status.conditions` contains `Accepted`. `Accepted=True` means `provider`, `identityRef`, `parameters`, and `allowedZones` are statically acceptable. `Accepted=False` reasons are `InvalidProvider`, `InvalidIdentityRef`, `InvalidParameters`, and `DeniedByPolicy`. `Accepted=Unknown` reasons are `IdentityNotResolved`, `ProviderStorageVersionNotResolved`, and `ReconcileError`. @@ -53,19 +53,19 @@ spec: `ZoneClass` is not reconciled by a core controller. The core validating webhook reads the referenced Provider version and validates `spec.identityRef`, `spec.parameters`, and `spec.allowedZones`. -Provider controllers reconcile only `ZoneClass` resources whose `spec.controllerName` matches their own controller name. If it does not match, they do not update status. They interpret `identityRef` according to the Provider version identity resource declaration, interpret provider-specific `parameters`, and set `ZoneClass.status.conditions[Accepted]` to `True` when the class is statically acceptable. +Provider controllers reconcile only `ZoneClass` resources whose `spec.controllerName` matches their own controller name. If it does not match, they do not update status. They interpret `identityRef` only when the Provider version declares an identity resource, interpret provider-specific `parameters`, and set `ZoneClass.status.conditions[Accepted]` to `True` when the class is statically acceptable. Credentialless providers perform no identity lookup. ## Zone -`Zone` is a namespace-scoped resource. It is the management unit corresponding to a hosted zone / zone on the DNS provider, connecting Kubernetes ownership boundaries with the provider-side object. +`Zone` is a namespace-scoped resource connecting Kubernetes ownership boundaries with a provider's zone scope. For DNS-writing providers it corresponds to a hosted zone / zone on the DNS provider. For the [external verification provider](external-provider.md#zone-semantics), it is a logical DNS suffix and need not be a separately hosted or delegated zone. `Zone.spec` fields: -- `domainName`: zone apex. Only canonical ASCII values are accepted. Immutable. +- `domainName`: zone apex or logical suffix, according to the Provider contract. Only canonical ASCII values are accepted. Immutable. - `zoneClassRef`: selected `ZoneClass`. If namespace is omitted, it defaults to the `Zone` namespace. Immutable. - `provider.name`: `Provider.metadata.name`. Required and immutable. - `provider.version`: `Provider.spec.versions[].name`. Required and immutable. -- `adoption`: specified when adopting an existing external zone. Omitted for creation. Mutable, but provider controllers stop changes that point to a different external zone after management starts. +- `adoption`: specified when adopting an existing external zone under a writing provider; omitted for creation. The external verification provider does not adopt DNS objects. Mutable, but provider controllers stop changes that point to a different external zone after management starts. - `adoption`: provider-specific opaque object validated by the referenced Provider version `zone.schemas.adoption`. - `allowedRecordSets`: conditions under which `RecordSet` resources from other namespaces may attach. If omitted, only same-namespace `RecordSet` resources are allowed. Each rule combines a namespace selector and record policy. @@ -269,9 +269,9 @@ status: - `IdentityNotResolved`: provider-specific identity object or its accepted state is not resolved enough to decide. - `ProviderConversionUnavailable`: Provider conversion webhook is temporarily unavailable, timed out, or returned a retryable failure. -`Programmed` means the desired state has been reflected to the provider. +`Programmed` means the provider-specific completion criteria for the accepted desired state are satisfied. -`Programmed=True` does not mean DNS is resolvable. It means the last provider-side state observed by the controller matches the accepted desired state and there is no pending provider change. For Route 53, `Programmed=True` means no `pendingHostedZoneChange` exists in `ZoneUnit.status.provider.state` and the observed hosted zone matches `ZoneUnit.spec.zone` and `ZoneClass.spec.parameters`. Parent-zone NS delegation, public DNS resolvability, and resolver reachability are not guaranteed. +`Zone Programmed=True` is not a generic guarantee that records resolve. For Route 53, it means no `pendingHostedZoneChange` exists in `ZoneUnit.status.provider.state` and the observed hosted zone matches `ZoneUnit.spec.zone` and `ZoneClass.spec.parameters`; parent-zone delegation and DNS reachability are not guaranteed. For the external provider, it means the verifier is configured and any requested nameserver constraint is verified, independently of child record success. External RecordSet success requires the separate resolution predicate in [External Provider](external-provider.md#recordset-verification). Core performs no DNS queries. `Programmed=False` reasons: @@ -826,7 +826,7 @@ If a pre-applied `ZoneUnit` contains record-set ownership state whose owner clai - `IdentityNotResolved`: provider-specific identity object or its accepted state is not resolved enough to decide. - `ProviderConversionUnavailable`: Provider conversion webhook is temporarily unavailable, timed out, or returned a retryable failure. -`Programmed` means the desired state has been reflected to the provider. Provider-specific completion criteria are defined in provider sections. For Route 53, `Programmed=True` means the last observed RRSet matches the `ZoneUnit.spec.recordSets[]` item and there is no `pendingRecordSetChange` affecting that record identity. +`Programmed` means the provider-specific completion criteria for the accepted desired state are satisfied. For Route 53, `Programmed=True` means the last observed RRSet matches the `ZoneUnit.spec.recordSets[]` item and there is no `pendingRecordSetChange` affecting that record identity. For the external provider, it means the latest completed verification matches the expected records in every required resolver view, without DNS writes; it does not imply global resolver convergence or verified withdrawal after deletion. `Programmed=False` reasons: diff --git a/docs/design/external-provider.md b/docs/design/external-provider.md new file mode 100644 index 0000000..a2a8339 --- /dev/null +++ b/docs/design/external-provider.md @@ -0,0 +1,356 @@ +# External Provider + +## Purpose and Scope + +The external provider verifies customer-managed DNS. The customer owns authoritative zones, records, delegation, and TTLs. dns-api sends DNS queries and writes Kubernetes reconciliation status; it never creates, updates, adopts, or deletes DNS data. There is no DNS UPDATE, cloud DNS client, credential chain, or fallback DNS writer. + +This document defines the stage-1 design, not evidence that the implementation or a deployment has passed verification. Stage 1 preserves the normal EndpointRecordSet → generated RecordSet → ZoneUnit → Accepted/Programmed path. The platform's Phase 4 acceptance is a remote-member HTTPPublish becoming Ready through real resolution and HTTPS, not merely creating a Provider or reporting a configured Zone. Platform wiring and that acceptance proof are outside this provider's implementation boundary. + +Verified-withdrawal evidence before retired VIP reuse is not part of this provider. Until a `Withdrawn` receipt contract exists, member replacement and deletion under the platform's `external` mode are unsupported. Kubernetes claim deletion described below is supported, but does not make member retirement or address reuse safe. + +## Provider Declaration + +The Core Provider is `external.dns.appthrust.io/v1alpha1`: `Provider.metadata.name=external.dns.appthrust.io`, with one served and storage version named `v1alpha1`. The Provider object itself uses `apiVersion: dns.appthrust.io/v1alpha1`. The default controller name is `external.dns.appthrust.io/controller`. + +Provider declaration fields: + +| Field | Contract | +| --- | --- | +| `spec.display.name` | `External DNS` | +| `spec.display.description` | `Verifies customer-managed DNS without changing DNS records.` | +| `spec.versions[].identity` | Omitted. There is no ExternalIdentity resource. | +| `spec.versions[].zoneClass.schemas.parameters` | The parameter schema below. | +| `spec.versions[].zone.schemas.adoption` | Empty-object schema: `type: object`, `additionalProperties: false`, `properties: {}`. | +| `spec.versions[].recordSet.supportedTypes` | Exactly `A`, `AAAA`, and `CNAME`. | +| `spec.versions[].recordSet.schemas.options` and `.adoption` | The same empty-object schema. Omitted or empty payload is acceptable; nonempty payload is rejected. | +| `spec.versions[].zone.schemas.statusProviderData` and `.recordSet.schemas.statusProviderData` | Public verification data described below, with field descriptions and bounded lists. | +| `spec.versions[].zone.additionalConditionReasons` and `.recordSet.additionalConditionReasons` | The target-specific reasons in the condition tables below. | + +External does not disable core record validations. Standard type/body exclusivity, required TTL, address-family checks, ownership arbitration, CNAME exclusivity, and apex CNAME rejection remain effective. A/AAAA describe the terminal address sets to verify, even when customer DNS uses a CNAME chain. CNAME describes an actual CNAME target to verify, not a provider alias. + +Provider-specific Go types are inline payload/status types, including `ExternalZoneClassParameters`; there are no external Zone or RecordSet CRDs. The provider's sole storage version needs no Core provider-payload version conversion. That is separate from Endpoint App conversion below. + +## ZoneClass Parameters + +`ZoneClass.spec.identityRef` is omitted. Core admission requires a nonempty reference if and only if the selected Provider version declares `identity`, and forbids the field otherwise. Empty or sentinel identity objects do not represent credentialless operation. Route 53 and Cloudflare retain their required identity references. + +`ZoneClass.spec.parameters` fields: + +| Field | Description and bounds | +| --- | --- | +| `resolver.mode` | `Cluster` or `Recursive`. Omitted resolver or mode means `Cluster`. | +| `resolver.servers` | Recursive upstreams as canonical `IPv4:port` or `[IPv6]:port` strings. Required and nonempty for `Recursive`; omitted or empty for `Cluster`. At most 3 distinct endpoints. Port is a decimal integer from 1 through 65535 with no leading zeros. No hostname, URL, scoped IPv6 address, or implicit port. | +| `expectedNameservers` | Optional, at most 16 distinct canonical lowercase ASCII DNS names, without trailing root dots, IP literals, wildcards, or underscores. Omitted/empty means skip delegation and direct-authority checks. This is not a list of recursive resolvers. | + +Use a separate ZoneClass for each domain whose expected authority differs. The platform projects its domain's `expectedNameservers` into this field; dns-api does not read PlatformDomain. Changing resolver policy invalidates prior probe success even if RecordSet generation is unchanged. + +### Schema and CEL + +The `zoneClass.schemas.parameters.openAPIV3Schema` fragment is: + +```yaml +type: object +description: DNS resolver policy for verification without DNS writes. +additionalProperties: false +properties: + resolver: + type: object + description: Resolver view used for every verification query. + additionalProperties: false + properties: + mode: + type: string + description: Cluster uses pod DNS; Recursive uses every configured server. + enum: [Cluster, Recursive] + servers: + type: array + description: Explicit recursive DNS IP endpoints; never authoritative NS names. + maxItems: 3 + uniqueItems: true + items: + type: string + minLength: 3 + maxLength: 64 + pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' + expectedNameservers: + type: array + description: Expected authoritative nameservers for the exact Zone domain. + maxItems: 16 + uniqueItems: true + items: + type: string + minLength: 1 + maxLength: 253 + pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' +``` + +Omitted-field interpretation supplies Cluster; admission does not mutate the manifest. The following `zoneClass.validationRules` complement the schema in the existing KRO CEL environment (`self` is the ZoneClass). The schema first constrains endpoint separators and decimal ports; CEL checks real IP syntax, canonical addresses, port range, and mode/server exclusivity. No provider-specific webhook or new CEL dialect is introduced. + +```yaml +- rule: >- + has(self.spec.parameters.resolver) && + has(self.spec.parameters.resolver.mode) && + self.spec.parameters.resolver.mode == 'Recursive' + ? (has(self.spec.parameters.resolver.servers) && + size(self.spec.parameters.resolver.servers) > 0) + : (!has(self.spec.parameters.resolver) || + !has(self.spec.parameters.resolver.servers) || + size(self.spec.parameters.resolver.servers) == 0) + message: resolver.servers must be nonempty for Recursive and empty for Cluster +- rule: >- + !has(self.spec.parameters.resolver) || + !has(self.spec.parameters.resolver.servers) || + self.spec.parameters.resolver.servers.all(s, + cel.bind(host, s.startsWith('[') + ? s.split(']')[0].substring(1) : s.split(':')[0], + cel.bind(port, s.startsWith('[') + ? s.split(']:')[1] : s.split(':')[1], + isIP(host) && ip.isCanonical(host) && + (s.startsWith('[') ? ip(host).family() == 6 : ip(host).family() == 4) && + int(port) >= 1 && int(port) <= 65535))) + message: resolver.servers must contain canonical IP addresses and ports from 1 through 65535 +- rule: >- + !has(self.spec.parameters.expectedNameservers) || + self.spec.parameters.expectedNameservers.all(n, !isIP(n)) + message: expectedNameservers must contain DNS names, not IP addresses +``` + +Canonical server strings and names make `uniqueItems` a semantic duplicate check. DNS answers are normalized before comparison; manifests with noncanonical expectations are rejected rather than silently rewritten. Provider admission compiles these expressions as boolean rules and implementation tests exercise both omitted-field and malformed-input cases through the actual pinned CEL runtime. + +Example: + +```yaml +apiVersion: dns.appthrust.io/v1alpha1 +kind: ZoneClass +metadata: + namespace: platform + name: customer-example +spec: + provider: + name: external.dns.appthrust.io + version: v1alpha1 + controllerName: external.dns.appthrust.io/controller + parameters: + resolver: + mode: Recursive + servers: + - 192.0.2.53:53 + - '[2001:db8::53]:53' + expectedNameservers: + - ns1.example.net + - ns2.example.net + allowedZones: + namespaces: + from: Same +``` + +These documentation addresses are not public resolver defaults. + +### Controller Responsibility + +The external ZoneClass controller matches controllerName and Provider binding, resolves the storage version, validates converted parameters, and sets `Accepted`. It makes no DNS queries and resolves no identity. Invalid configuration uses the existing `InvalidParameters`, `InvalidIdentityRef`, or `InvalidProvider` reasons; missing storage-version material follows the existing Unknown contract. A present `identityRef` on an external ZoneClass, or an absent one on a Route 53/Cloudflare class after a Provider change, is reported controller-side as `Accepted=False/InvalidIdentityRef`. + +The external ZoneUnit controller watches ZoneUnit, matching ZoneClass, Provider, and namespace-policy changes. It reconciles only the configured provider/storage version through the matching class. Core remains the sole composer and public Zone/RecordSet status writer. The provider respects `Allowed`, `DeletionRequested`, claim UID, and claim observedGeneration; it does not probe denied ledger items or use old deletion receipts for a new incarnation. + +The local read-only test/runtime boundary is `Resolver.Probe(ctx context.Context, request ProbeRequest) (ProbeResult, error)`, with injected clock and jitter. Requests contain checked names, expected RRsets, and effective resolver policy; results contain per-view RCODE, answers, AA, TTL/SOA, and authority evidence. There are no DNS Create/Update/Delete methods. The provider does not use a generic cloud provider interface or call a cloud API. + +## Zone Semantics + +An external Zone is an authorized logical suffix. It need not be a separately hosted or delegated zone. Without `expectedNameservers`, a customer may verify records under a suffix of an existing authority. No hosted-zone ID, adoption target, creation policy, or deletion policy is needed. + +`Zone Accepted=True` confirms the binding and policy, not DNS reachability. `Zone Programmed=True` means the verifier is configured and, if requested, the exact Zone's nameserver constraint has passed. It is independent of record success: a missing application record must not make the Zone unconfigured or create a Zone ↔ RecordSet readiness cycle. Conversely, failed or unknown required nameserver checks prevent child RecordSets becoming Programmed=True, even if their recursive answers match. + +With `expectedNameservers`, the exact `Zone.spec.domainName` must be a zone apex in the selected view. A parent authority's NS set cannot be relabelled as a successful delegation check for a nondelegated suffix. `Zone.status.nameServers` contains only a completed observed NS set, not copied expected values; omit it if no set was observed. Conditions and probe times distinguish last observation from current success. + +## RecordSet Verification + +For every active allowed name, collect its A and AAAA desired ledger items together. Normalize addresses to their IP values, normalize DNS answer names to lowercase without the root dot, deduplicate, and sort for deterministic diagnostics. Probe **both** address families. The observed A set must equal the expected A set, and the observed AAAA set must equal the expected AAAA set in every required view. A missing desired family means an empty expected set, not permission to ignore that family's answers. NODATA for an empty expected family is acceptable; NXDOMAIN for a name with expected data is not. + +Set equality is exact: missing, unexpected, or additional-family addresses prevent success. Do not accept subsets, any matching VIP, an address in an allocated pool, or a union assembled from disagreeing resolvers. The two address-family claims share the combined result, so an A-only success cannot hide an unexpected AAAA answer. Adding/removing a sibling family changes the combined expectation and invalidates the surviving claim's earlier receipt. + +For IP intents, a customer CNAME chain is allowed only when terminal address sets satisfy the same predicate. Follow at most 8 CNAME links total, detect repeated canonical names, and reject loops. A chain limit or loop is not an address match. Preserve relevant chain TTLs for scheduling. A CNAME to a dual-stack target for an IPv4-only intent is a mismatch by design; the customer must point at an address set equal to the published one. + +For CNAME intent, the first canonical CNAME target at the checked owner must equal `spec.cname.target`. Another name with the same terminal addresses is not equivalent. Follow the expected target within the same 8-link/loop bounds and require usable terminal A or AAAA resolution in every required recursive view. Both families are inspected; address equality is not asserted against an IP intent that does not exist. Required views must agree on the observed chain and terminal sets. A CNAME returned together with incompatible same-owner data is not successful verification. CNAME at `name: "@"` remains rejected by core. + +### Wildcard Witness + +A wildcard RecordSet is verified by a concrete witness, not a query for the literal `*`. Replace the leftmost wildcard label with a stable DNS-safe label derived from the Zone identity and the sorted active claims' UID/generation tuple. All A/AAAA items for that wildcard use the same witness, including when they were created in different orders. Repeated timer checks of unchanged intent reuse it; recreation or changed desired intent changes it. + +Expose `checkedName` and `sampledWildcard=true` in public data. This proves only the sampled name. Existing explicit owners or closer enclosers can override wildcard synthesis, so every exact application hostname is also independently probed. It does not prove all descendant names or TLS wildcard coverage. + +For the platform's dedicated-member topology, customer records are `B` and `*.B` for the PMC public edge and `*..B` for that Project's actual serving Gateway addresses. A wildcard pointing at the union of reserved member VIPs cannot encode Project placement and must fail when it differs from the intended targets. A single wildcard is adequate only when all covered names deliberately share the same serving address set. Cross-namespace wildcard claims require an explicit `allowedRecordSets.records` grant on the Zone ([Core API](core-api.md#allowedrecordsets)); the suffix match alone does not authorize them. + +## Resolver Policy + +| Policy | Required observation | Scope and failure behavior | +| --- | --- | --- | +| `Cluster` (default) | Read pod `/etc/resolv.conf` nameservers and query each independently using DNS wire queries with RD=1. Use absolute FQDNs, without search-path expansion. Apply the same maximum of 3 distinct upstreams; empty, invalid, or oversized configuration fails closed. | Proves only the controller/PMC resolver view. Do not treat resolv.conf order as failover or silently discard an upstream. | +| `Recursive` | Query every configured IP:port endpoint independently with RD=1; all must match. | Supports operator-selected private/split-horizon or public views. No fallback to cluster DNS or hardcoded public resolvers after failure. | +| Nonempty `expectedNameservers` | Additionally query the exact Zone apex NS and SOA through every selected recursive view, compare the normalized NS set exactly, and require the SOA owner to identify that apex. Resolve every expected NS host through those views, then query every discovered authority address with RD=0. Require AA and the expected apex NS/SOA and in-zone record data. | Every required authority must agree with expected data; a failed authority is not ignored. At most 2 distinct addresses per expected NS after deduplication across views; an excess is an explicit probe-limit failure, not successful sampling. When a CNAME leaves the asserted zone, verify its in-zone authoritative link and follow the remaining chain through the chosen recursive view, not by demanding that the original authority serve the foreign zone. | +| Empty `expectedNameservers` | Recursive view only. | No delegation or direct-authority verification claim. | + +Direct-authority checks also fail closed for an in-suffix delegation whose data the expected authority does not authoritatively serve; the provider does not infer and trust an additional child authority. Configure a separate Zone/Class for a separately delegated child. + +Use a DNS wire client exposing RCODE, AA, TTL, SOA, and TC (for example `miekg/dns`), rather than `net.Resolver.LookupIPAddr` alone. Send QUERY only, UDP first and TCP on truncation. Resolve NS host addresses through configured resolvers, not an implicit OS fallback. No DNSSEC validation, root-to-parent iterative delegation proof, or public/global reachability guarantee is claimed: the recursive view and network are trusted inputs. + +Use the minimum relevant positive TTL across the completed chain; negative cache TTL is `min(SOA TTL, SOA.MINIMUM)` for NXDOMAIN/NODATA when an SOA is present. Expose observed TTLs/negative expiry for diagnostics. A matching authority plus stale recursive NXDOMAIN remains unverified until the recursive view also matches. Do not flush shared caches, change customer TTLs, install an additional process-wide answer cache, or interpret a polling cap as cache expiry. + +## Conditions and Public Data + +A valid claim keeps `Accepted=True` while DNS is wrong or unavailable. Only `Programmed` reflects probe observations. Each receipt uses the exact claim UID and observedGeneration. No provider-owned `Published` or Endpoint `Ready` condition is added; the existing endpoint controller aggregates nonempty generated RecordSets with current Accepted/Programmed evidence. + +RecordSet `Programmed` reason and stable message templates: + +| Status | Reason | Message | +| --- | --- | --- | +| True | `ResolutionVerified` | `DNS matches expected targets in every required resolver view; no DNS changes were made.` | +| False | `DNSNameNotFound` | `DNS returned NXDOMAIN from ; expected .` | +| False | `DNSNoData` | `DNS returned NOERROR with no answers; expected .` | +| False | `ResolutionPartial` | `DNS is missing expected targets ; observed .` | +| False | `ResolutionMismatch` | `DNS targets differ: missing ; unexpected .` | +| False | `ResolutionInconsistent` | `DNS differs between required resolver/authority views .` | +| False | `NameserverMismatch` | `DNS nameservers differ: expected ; observed .` | +| Unknown | `DNSProbeFailed` | `DNS verification unavailable for from : .` | + +`ResolutionPartial` means a nonempty proper subset with no unexpected values. Empty required-type answers use DNSNoData; any unexpected value uses ResolutionMismatch. `` includes typed values (A, AAAA, or CNAME). CNAME loops, incompatible data, and a wrong first target are ResolutionMismatch, with the observed chain included in bounded public data. `` is one of `timeout`, `SERVFAIL`, `REFUSED`, `transport`, `invalid-response`, `resolver-config`, `chain-limit`, or `probe-limit`; raw errors/packets are not embedded in messages. + +A required incomplete/failed probe is Unknown/DNSProbeFailed, never last-known success. Transient `Unknown/DNSProbeFailed` immediately drops EndpointRecordSet `Ready` through the existing generated-RecordSet aggregation; there is no grace window. Among completed observations, differing required views take precedence as ResolutionInconsistent; a consistently wrong expected NS set is NameserverMismatch; otherwise use the negative/partial/mismatch result. Normalize recursive and authoritative observations to the checked owner and terminal data before comparing, so normal AA/TTL differences are not disagreements. All-empty-family NODATA is not an inconsistency. Select the first failing view in canonical order for single-view message templates. + +Zone `Programmed` reasons: + +| Status | Reason | Message | +| --- | --- | --- | +| True | `VerificationConfigured` | `DNS verifier configured for ; authoritative nameserver checks were not requested.` | +| True | `NameserversVerified` | `DNS nameservers match expectations in every required view; no DNS changes were made.` | +| False | `NameserverMismatch` | The nameserver mismatch template above; includes a non-apex NS expectation. | +| False | `ResolutionInconsistent` | The inconsistent-view template with the Zone domain as ``. | +| Unknown | `DNSProbeFailed` | The probe-failure template with the Zone domain as ``. | + +Register exactly these extra status/reason combinations in the Provider's respective Zone and RecordSet `additionalConditionReasons`. Propagate a required Zone authority failure/unknown to affected RecordSets. Before any completed observation, use the existing Unknown/Reconciling condition. Unit-level Programmed uses only the fixed allowlist (`validator.go` ZoneUnit status): True/`Programmed` when every active observation succeeds; False/`ExternalResourceNotFound` when the first failing target is `DNSNameNotFound`; False/`ExternalResourceMismatch` for every other completed mismatch (`DNSNoData`, `ResolutionPartial`, `ResolutionMismatch`, `ResolutionInconsistent`, `NameserverMismatch`); Unknown/`ProviderUnavailable` for `DNSProbeFailed`. The provider implements the same precedence as the Route 53 unit aggregate (`zone_status.go`): any False target wins; Unknown only when no target is False. Target reasons are mapped, never copied, into unit-level status. + +Core projection preserves the selected current provider condition's `Message` along with Status/Reason for both Zone and RecordSet, including an empty message; it must not replace diagnostics with the reason string. Core-generated waiting/rejection conditions retain their own messages. Existing UID/generation guards and acceptance precedence remain in force. + +`Zone.status.provider.data` and `RecordSet.status.provider.data` expose: + +- `resolverMode`, `checkedName`, and, for RecordSets, `sampledWildcard`. +- Sorted typed `actual`, `missing`, and `extra` values and bounded CNAME-chain/view diagnostics. Each list is capped at 16 entries with total counts and an explicit truncation flag; equality is evaluated on complete observations, never truncated status. +- `observedTTLSeconds`, `negativeTTLSeconds`, and `negativeExpiryTime` when the corresponding evidence exists; no invented zero TTL for missing SOA. +- `lastProbeTime` (last completed attempt, successful or failed) and `nextProbeTime` (approximate next planned check). + +Messages are capped at 2048 bytes, use deterministic sorted list previews with omitted counts, and contain no credentials or raw packets. Successful timestamps/next-probe updates do not emit repeated success Events or change LastTransitionTime when condition status is unchanged. Events report transitions only; status remains the source of truth. + +Private `ZoneUnit.status.provider.state` holds advisory due times, fair-resumption cursor, backoff, and the resolver-policy/desired-name fingerprints. A probe receipt is bound to the claim tuple, all sibling-family desired inputs, the Zone binding, and effective normalized Class/Provider/resolver configuration. Before committing an asynchronous result, recheck that those inputs still match; otherwise discard it and queue current intent. Class/Provider policy changes and resolv.conf changes invalidate old success. Restart reconstructs scheduling from current claims and does not trust obsolete private state. + +## Scheduling and Resource Bounds + +DNS edits and recursive cache expiry have no Kubernetes watch. A timer is unavoidable: a customer may fix DNS without changing any Kubernetes object. Probe immediately on new/changed intent or effective policy and keep rechecking successful claims to detect drift. Kubernetes watches complement rather than replace timers. A later failed probe removes Programmed=True. + +| Outcome | Base next-check interval | +| --- | --- | +| Pending, partial, or mismatch | 30 seconds | +| Success | Minimum relevant positive TTL, clamped to 30–300 seconds | +| NXDOMAIN/NODATA | Negative TTL, clamped to 30–300 seconds | +| Missing relevant TTL | 30 seconds | +| Unavailable probe | Backoff 30 → 60 → 120 → 300 seconds; reset after a completed non-error observation or changed inputs | + +Apply ±10% injected jitter, then clamp again to 30–300 seconds. These are polling bounds, not an external propagation deadline or a wall-clock freshness promise. Customer caches can outlive the polling cap. A stopped controller cannot withdraw a previously published condition; expose probe times and monitor controller health. Consumers that need freshness treat a claim as stale when `now > nextProbeTime + 300 s`; conditions alone do not encode this. Existing EndpointRecordSet aggregation reads conditions, not that timestamp threshold; it immediately drops Ready on an observed Unknown/DNSProbeFailed, without a grace window. + +Every exchange has a 2-second timeout. Each name has a 10-second deadline that starts when its first exchange is admitted by the limiter; limiter wait time is not charged to the name. Zone-apex NS/SOA queries and expected-NS host resolution are performed once per Zone per probe cycle and shared by all names in that cycle, keyed by the resolver-policy fingerprint; only in-zone A/AAAA/CNAME queries are per name. The manager-wide limiter is sized so that 4 concurrent names with the maximum authority fan-out (16 NS × 2 addresses × 2 qtypes) complete inside the name deadline (default 40 QPS, burst 80), and is a manager flag. Exhausted deadlines or limits are Unknown, never success from a partial sample. + +The manager admits at most 4 concurrent names. The limiter flags are `--external-dns-qps` (default `40`) and `--external-dns-burst` (default `80`); they apply across all ZoneUnits. The per-cycle Zone evidence is not a process-wide answer cache and is invalidated by a resolver-policy change. Limiter queue delay remains observable separately from admitted probe duration. + +Process at most 32 due names per ZoneUnit reconcile. Resume fairly through private state and use a 1-second continuation while previously unprocessed due work remains; this exception schedules other due names, not another probe of a completed name before its 30-second floor. Recompute the next scheduled reconcile from the earliest due time when the backlog is drained. Queue saturation may exceed the nominal cadence and must be visible in queue-delay metrics. The provider's own unchanged status events must not trigger fresh probes; keep the core watch that projects genuine status updates. Patch status/receipts, not claim spec or arbitrary timestamps on every watch event. + +## Deletion + +External owns no DNS object. For a matching-UID ledger item with `DeletionRequested=true`, stop probes and mark `DeletionCompleted=true` after Kubernetes bookkeeping. Complete ZoneUnit finalization without a DNS delete call and without waiting for the customer to remove a wildcard. Existing core claim/endpoint finalizers still wait for the appropriate Kubernetes receipts and generated-child disappearance. Old-incarnation deletion completion must not finalize a replacement claim. + +**Deletion completion is not verified DNS withdrawal.** Neither claim disappearance, observedGeneration, a completed verifier finalizer, nor a timer establishes that retired targets have disappeared from DNS or caches. A separate `Withdrawn` condition/receipt contract is required before retired VIP reuse; its schema, persistence, consumers, and cache-safety protocol are outside this provider's scope. The provider does not emit or accept `Withdrawn`. Platform member replacement/deletion under external remains unsupported until that contract exists; allocations must not be released based on this bookkeeping-only completion. + +## Admission and Security + +Admission performs no network/DNS queries. Reuse Provider OpenAPI/CEL and core webhook rules for shape, immutable references, supported types/bodies, ordinary TTL range, and object-local namespace-policy declarations. Cross-object grants and ownership remain core composition checks. Reject external identityRef and nonempty options/adoption, including Route 53 alias and Cloudflare proxy fields. Keep all ordinary CNAME/record ownership protections. + +Only operators may create/update Providers and resolver-bearing ZoneClasses. Tenants cannot choose arbitrary DNS destinations by creating a RecordSet. Explicit private/RFC1918 resolvers are allowed for on-premises use; DNS egress controls must allow the intended recursive and optional authority endpoints, not blanket-ban private IPs or silently change the view. Resolver-address errors fail closed. + +External requires no Secret get/list/watch, identity-resource access, IAM/STS calls, token access, or credential environment configuration. Kubernetes API authentication and mounted webhook TLS serving certificates are not customer DNS credentials; they do not require the controller to read the Kubernetes Secret API. An external-only chart deployment must remove cloud provider RBAC and Secret informers, not just omit cloud Provider manifests. Existing core/endpoint resources, namespace reads, status/finalizers, Events, leader-election leases, and the enabled conversion API's create permission remain. + +## Endpoint Conversion Decision + +**Decided for stage 1: retain the provider-specific aggregated Endpoint RecordSet Conversion API; reject schema-equality-inferred built-in identity conversion.** No Core Provider or EndpointProviderCapability API field is added for an inferred bypass. + +Grounding at source revision `87aa1dc`: + +- `internal/go/apps/endpoint/controllers/endpointrecordset/controller.go:298-374` always creates the capability's conversion GVR (apart from an injected test function), requires a REST config, checks response UID/result/nonempty fragments, and rejects changed names or unsupported output types. +- `pkg/go/api/endpoint/v1alpha1/endpointprovider_capability_types.go:21-41` requires `spec.conversion` with group/resource and an optional version. There is no built-in or identity mode. +- `internal/go/providers/route53/conversion/handler.go:81-186` classifies targets, splits IPv4/IPv6, chooses TTL, rejects unsupported mixtures, and maps a hostname to aliases unless CNAME is explicitly requested. Its standard bodies do not establish identical conversion policy. +- Core Provider storage conversion operates on `{options, adoption}` (and other provider-owned payloads), not a full endpoint target list or RecordSetSpecFragment. Same storage version already avoids Core payload conversion, but does not transform endpoint intent. + +Schema equality therefore cannot prove semantic identity: input targets and output fragments have different shapes/cardinalities, and optional alias policy can differ despite shared standard record bodies. A future in-process standard-record converter is technically possible, but would need an explicit Endpoint App capability discriminator and a versioned shaping contract; it belongs to the App, not Core. It cannot be safely inferred from missing identity, A/AAAA/CNAME support, or schema similarity. That additional API is not needed for stage 1. The chosen tradeoff retains APIService/TLS operational cost and the existing discovery/authentication/RBAC/UID validation path instead of creating a second implicit convention. + +The exact capability uses the existing API unchanged: + +```yaml +apiVersion: endpoint.dns.appthrust.io/v1alpha1 +kind: EndpointProviderCapability +metadata: + name: external-v1alpha1 +spec: + provider: + name: external.dns.appthrust.io + version: v1alpha1 + conversion: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + resource: endpointrecordsetconversions +``` + +The matching APIService is `v1alpha1.endpoint.external.dns.appthrust.io`. A stateless handler on the existing manager webhook server serves discovery and POST-only resource creation at `/apis/endpoint.external.dns.appthrust.io/v1alpha1/endpointrecordsetconversions`; it does not persist conversion objects. Reuse the existing request/result/UID framing, never the Route 53 alias behavior. Conversion performs no DNS queries and has no credentials. + +For IP-only targets, emit nonempty A and/or AAAA fragments in that order after canonicalization, deduplication, and sorting. For exactly one distinct hostname target, emit CNAME; reject apex CNAME, mixed IP/hostname targets, and multiple distinct hostnames. Preserve `input.name`, including wildcard form. An absent preferredRecordType or CNAME preference is permitted for hostname targets; a preference incompatible with target kinds is rejected. Every emitted standard fragment has `ttl: 300`, no options/adoption, and its matching core body, including CNAME, unlike the Route 53 handler's 60-second CNAME TTL. The TTL satisfies the existing schema; external does not set customer TTL or require observed TTL equality. The endpoint controller currently derives `preferredRecordType` only from the `endpoint.dns.appthrust.io/route53-record-type` annotation; external accepts the forwarded value but does not introduce a second annotation. Renaming that annotation is an Endpoint App change outside this document. + +Conversion failures use `InvalidInput` for malformed input and `UnsupportedTarget` for unsupported target/preference combinations, with `retryable=false`; success uses `Converted`. Mixed targets use `External endpoint conversion does not support mixed hostname and IP targets`; multiple hostname targets use `External endpoint conversion requires exactly one hostname target`; apex uses `External endpoint conversion does not support CNAME at the zone apex`. UID, nonempty-fragment, name, and type checks remain mandatory in the endpoint controller. Failure never produces an empty ready endpoint. + +## Packaging and Metrics + +`providers.external.enabled` defaults to false. `--external-controller-enabled` gates external ZoneClass/ZoneUnit registration and its conversion handler. Provider/controller-name/version flags and environment variables follow existing provider bindings: `--external-controller-name` / `EXTERNAL_CONTROLLER_NAME`, `--external-provider-name` / `EXTERNAL_PROVIDER_NAME`, and `--external-provider-version` / `EXTERNAL_PROVIDER_VERSION`. + +Provider values must actually gate registrations and provider-only privileges as specified in [Controllers and Webhooks](controllers-webhooks.md#provider-enablement-required-contract-not-implemented-at-87aa1dc). Enabling external installs its Provider, capability, APIService, conversion-create RBAC, and handler together. Do not render an external capability that cannot be served. External enablement requires webhook TLS with a valid CA bundle and certificate; the chart `fail`s rendering when `providers.external.enabled=true` and either `webhook.enabled=false` or `certManager.enabled=false`; a user-supplied `webhook.caBundle` is not provided in this revision. Do not copy an `insecureSkipTLSVerify` fallback. No external conversion-disable sub-option is needed. Core admission remains available regardless of which providers are enabled. + +Synchronize generated API types/deepcopy, both CRD distributions, Go identity-using callers, `docs/design/ui.md`, and `pkg/ts/ui/src/views/platform/ZoneClassForm.tsx` when making identityRef optional. The UI supports external ZoneClass creation in stage 1: it selects the Provider without an identity, omits `identityRef` and the identity picker, and exposes external resolver parameters. A new chart with an old ZoneClass CRD still rejects credentialless classes; separate CRD carriers must be upgraded too. Helm and Kustomize admission wiring must agree, including ZoneClass status and enabled identity webhooks. Existing Gateway/Service watcher switches and the single controller ServiceAccount binding remain independent. + +Use the existing manager metrics registry/server: + +| Metric | Type and labels | +| --- | --- | +| `dns_external_probes_total{outcome,qtype}` | Counter; qtype is A, AAAA, CNAME, NS, or SOA; outcome is verified, mismatch, negative, or failed. | +| `dns_external_probe_duration_seconds{outcome}` | Histogram of completed probe durations. | +| `dns_external_pending_records{reason}` | Gauge of active unverified record claims, using only the closed reason set above plus Reconciling. | +| `dns_external_probe_queue_delay_seconds` | Histogram of delay from due time to attempt start. | + +No hostname, namespace, IP, UID, or resolver-address labels. Do not expose packet/credential content in logs or metrics. + +## Tests + +Follow [Test Policy](test-policy.md): exhaustive deterministic unit/envtest coverage, representative black-box success paths only for kest/conformance. The external success path is `tests/e2e/external/external-a-record-lifecycle.test.ts`, using an in-kind authoritative CoreDNS `file` fixture whose zone the harness edits to model the customer; it asserts mismatch→match→drift without public DNS. No public DNS, cloud credentials, or real customer resources are used in external-provider CI. Controlled local DNS fixtures may be changed by the test harness to model the customer; the provider itself must remain read-only. + +### Unit and Local Transport + +Inject a fake Resolver, clock, and jitter. Cover exact sets, order/duplicates, canonical IPv4/IPv6, unexpected opposite family, independent per-host outcomes, CNAME first-target identity and terminal resolution, eight-link boundary/loops, shared wildcard witness and explicit-owner overrides. Cover NXDOMAIN, empty expected-family NODATA, partial/mismatch, disagreeing views/authorities, nondelegated suffix, missing authoritative AA, foreign CNAME targets, and every failure mapping. + +Use fake time for positive/negative SOA TTLs, missing TTL, clamping/jitter, 30/60/120/300 backoff, policy-change reset, mismatch→match without Kubernetes writes, and drift after success. Cover rate/concurrency limits, limiter-admission deadline start, per-Zone NS/SOA and NS-host work sharing/invalidation, finite per-pass work, fair continuation, observer status events not becoming probe loops, and no success from incomplete fan-out. Verify UID/generation/sibling/policy fingerprints, same-name recreation, stale results, Allowed=false, and incarnation-matched deletion completion. + +Run transport tests against a loopback UDP/TCP fixture for RCODE/AA/SOA parsing, truncation fallback, malformed replies, and cancellation. Assert only QUERY packets, never UPDATE, and no implicit resolver fallback or cloud client calls. Test conversion HTTP framing, discovery, UID, typed fragments, unsupported targets/preferences, and TTL compatibility without DNS access. + +### Envtest + +Install the real regenerated CRDs and Provider manifest, enable core admission and external Class/Unit and endpoint controllers, and inject a controlled resolver. Create Class → Zone → EndpointRecordSet and assert generated RecordSets and exact current conditions through mismatch → match → drift without spec updates. A configured Zone must remain independent of child failures; failed nameserver checks must block otherwise-matching children. Assert diagnostic Message projection for both Zone and RecordSet, zero/stale-child readiness rejection, immediate Endpoint Ready loss on Unknown/DNSProbeFailed without a grace window, cross-namespace wildcard grants, finalizers, mapped unit-level reasons, and deletion bookkeeping without DNS mutation. + +Exercise external identityRef rejection, Route 53/Cloudflare missing-identity rejection, controller-side InvalidIdentityRef after Provider identity changes, canonical parameter/CEL validation, invalid status reasons, storage-version handling, and old-schema incompatibility. Tests must use real CRD/admission paths, not just direct helpers. Separately configure envtest aggregation, request-header authentication, serving TLS/CA, and RBAC to exercise the real APIService conversion path including denied/unavailable/malformed cases. An injected conversion function does not prove that wiring. + +### Chart and Registration Contracts + +Cover external-only, each cloud-provider enable combination, all-disabled, and independent Gateway/Service source switches. Assert flags, actual controller/identity-webhook/conversion registrations, rendered Provider/capability/APIService agreement, valid CA wiring, rendering rejection when webhook or cert-manager is disabled for external, and Helm/Kustomize admission parity. External-only must have no cloud identity or Secret RBAC/informers; disabled source controllers must not restore their unrelated permissions. Retain the single intended controller ServiceAccount binding and CRD-copy synchronization contracts. + +Platform acceptance uses an explicitly authorized run with real customer-controlled DNS and remote-member HTTPPublish Ready, including wrong-target/wrong-NS recovery. Forced-IP HTTP, TLS verification bypass, borrowed synthesis domains, public DNS in CI, and unexecuted test plans are not verification evidence. + diff --git a/docs/design/open-decisions.md b/docs/design/open-decisions.md index ec9ed1c..c438812 100644 --- a/docs/design/open-decisions.md +++ b/docs/design/open-decisions.md @@ -5,3 +5,6 @@ This section tracks unresolved design decisions. When an item is resolved, refle - [ ] Define conformance test distribution, execution method, configuration file schema, and support level representation. - [ ] Define shared ownership of hosted zones by multiple platforms and ownership tag naming. - [ ] Define non-Go devbox tool versions. +- [ ] Define the operator-controlled recursive view authoritative for each external deployment's clients and members. Cluster mode proves only the PMC view; dns-api does not infer a shared split-horizon view. +- [ ] Validate the external provider's 16-NS/2-address fan-out bounds and committed defaults (10-second name deadline from limiter admission; manager-wide 40 QPS / burst 80, with per-Zone NS/SOA and NS-host work amortized per probe cycle) against expected Zone sizes and DNS topologies. Larger configurations fail closed or incur queue delay, never relax exact-set verification. +- [ ] Assign coordinated CRD-carrier, admission/TLS, and chart upgrade ownership in consuming installations. External requires cert-manager-enabled TLS and synchronized optional-identity UI/CRDs; Provider enablement alone does not establish readiness. diff --git a/docs/design/overview.md b/docs/design/overview.md index 867575d..1db5bc3 100644 --- a/docs/design/overview.md +++ b/docs/design/overview.md @@ -19,7 +19,7 @@ The `Provider` resource is the Core Provider discovery contract. Core reads Core ## Initial Scope -The initial implementation targets Route 53 public hosted zones. dns-api handles Public DNS only. Private DNS, VPC association, split-horizon DNS, and DNS propagation checks are out of scope initially. The API is not Route 53-specific and must allow future providers such as Google Cloud DNS, Cloudflare, and Azure DNS. +The initial Route 53 milestone targets public hosted zones. Private DNS, VPC association, split-horizon DNS, and DNS propagation checks are outside that milestone. The external verification provider adds DNS-query observation of customer-managed records in operator-selected public or private resolver views under the existing `Programmed` condition, without new condition types. The API is not Route 53-specific; provider contracts define their supported scope. The initial milestone is the ability to use a development AWS profile to create public hosted zones, standard A records, standard AAAA records, standard TXT records, standard CNAME records, standard MX records, standard CAA records, delegated NS records, and Route 53 ALIAS records from `Provider`, `ZoneClass`, `Route53Identity`, `Zone`, and `RecordSet`. DNS propagation checks are not part of this milestone. @@ -29,11 +29,11 @@ Core API resources: - `ZoneClass`: a DNS zone creation menu. - `Provider`: provider capability, schema, validation, and display metadata. -- `Zone`: a management unit corresponding to a hosted zone / zone on the DNS provider. +- `Zone`: a management unit corresponding to a hosted zone / zone on a writing DNS provider, or a logical suffix for external verification. - `RecordSet`: a DNS record set. - `ZoneUnit`: the zone-scoped desired-state and ownership ledger built from accepted `Zone` and `RecordSet` claims. - Provider-specific identity CRDs: which provider account, IAM role, or service account is used for operations. -Provider identity details live in provider-specific identity CRDs. `ZoneClass.spec.identityRef` selects a provider identity by name, and `ZoneClass.spec.parameters` holds provider-specific policy as an inline object. The core API has no provider-specific account or credential schema. The provider version declares the identity resource kind so UI, documentation, and generic reference display can discover which identity kind a `ZoneClass` uses. +Provider identity details live in provider-specific identity CRDs. `ZoneClass.spec.identityRef` selects a provider identity by name when the Provider version declares one; credentialless providers omit it. `ZoneClass.spec.parameters` holds provider-specific policy as an inline object. The core API has no provider-specific account or credential schema. When an identity is declared, the provider version supplies its resource kind so UI, documentation, and generic reference display can discover which identity kind a `ZoneClass` uses. -Standard condition types are `Accepted` and `Programmed`. `Accepted` is the result of composition acceptance and provider acceptance. `Programmed` describes whether the external provider's observed state matches desired state. Conditions such as `Propagated` or `Resolvable` that require DNS-query observation are not part of the initial core API. +Standard condition types are `Accepted` and `Programmed`. `Accepted` is the result of composition acceptance and provider acceptance. `Programmed` describes whether the external provider's observed state matches desired state. Conditions such as `Propagated` or `Resolvable` that require DNS-query observation are not part of the initial core API. DNS-query observation is a provider-specific `Programmed` criterion, not a core condition. diff --git a/docs/design/provider-extension.md b/docs/design/provider-extension.md index 976eb63..a5d1126 100644 --- a/docs/design/provider-extension.md +++ b/docs/design/provider-extension.md @@ -16,6 +16,8 @@ The manager exposes startup flags and matching environment variables for the ini `route53-controller-name` and `cloudflare-controller-name` configure the controller name compared with `ZoneClass.spec.controllerName`. Defaults are `route53.dns.appthrust.io/controller` and `cloudflare.dns.appthrust.io/controller`. Provider controllers must not hard-code controller name as their only supported runtime binding; forks and multi-instance deployments can use these settings to claim a different `ZoneClass.spec.controllerName` while still using the same Provider contract. +The external provider uses the same binding convention with `--external-controller-name` / `EXTERNAL_CONTROLLER_NAME`, `--external-provider-name` / `EXTERNAL_PROVIDER_NAME`, and `--external-provider-version` / `EXTERNAL_PROVIDER_VERSION`, defaulting to `external.dns.appthrust.io/controller`, `external.dns.appthrust.io`, and `v1alpha1`. Provider binding flags select a contract; they are not enable switches. Manager registration, chart enable values, identity admission, conversion handlers, and provider-only RBAC must be gated together as defined in [Provider Enablement](controllers-webhooks.md#provider-enablement-required-contract-not-implemented-at-87aa1dc). + `Provider.spec` fields: - `display.name`: provider display name. Required. @@ -26,6 +28,7 @@ The manager exposes startup flags and matching environment variables for the ini - `versions[].served`: whether the version can be referenced on create or spec update. - `versions[].storage`: whether the version is the provider storage version used in `ZoneUnit`. - `versions[].deprecated`: whether the UI and validation messages should treat the version as deprecated. +- `versions[].identity`: optional identity declaration. Omitted for credentialless providers; an empty identity object is not valid. - `versions[].identity.resource.group`: API group of the provider identity resource selected by `ZoneClass.spec.identityRef`. - `versions[].identity.resource.kind`: kind of the provider identity resource selected by `ZoneClass.spec.identityRef`. - `versions[].identity.resource.scope`: identity resource scope. Initial supported value: `Namespaced`. @@ -137,7 +140,7 @@ Provider admission error messages identify the failing field path and state that Provider admission validates conversion CEL expressions separately. A conversion CEL expression must compile in the Provider conversion environment, where `self` is the provider-owned payload object and the expression result must be an object that can be validated against the storage target schema after defaulting. Conversion CEL expressions do not return `bool`. -`versions[].identity.resource` is required for provider versions that use provider identity CRDs. The initial dns-api providers use identity CRDs, so Route 53 and Cloudflare both declare it. The declaration is discovery and contract metadata: the core API can validate the `identityRef.name` shape and UI can link to the correct identity kind, while the provider controller remains responsible for resolving the identity object, checking provider-specific readiness, and applying provider-specific credential rules. +`versions[].identity.resource` is required when `versions[].identity` is present. Route 53 and Cloudflare declare it; the external verification provider omits it. The declaration is discovery and contract metadata: core admission requires a nonempty `ZoneClass.spec.identityRef.name` if and only if the selected version declares identity, and rejects `identityRef` otherwise. It does not manufacture an identity for credentialless providers. UI can link to the declared identity kind, while the provider controller remains responsible for resolving that object, checking provider-specific readiness, and applying credential rules. Provider admission validates `versions[].recordSet.supportedTypes` independently of any `RecordSet`. It rejects unknown types and types that do not have a core `RecordSet.spec` body in the current API version. A provider package must not claim `SRV` or another future type until the core API adds the matching body field, validation rules, examples, and provider schema support. This keeps `RecordSet.spec.type`, the required body field, and Provider capability metadata consistent. @@ -200,7 +203,7 @@ Provider runtime interfaces are local to each provider controller package. A pro - Route 53 uses a hosted zone and record-set change-batch model. Its runtime interface may combine hosted zone operations, change polling, tagging, record-set listing, and record-set batch changes. - Cloudflare uses separate zone and DNS record APIs. Its runtime interfaces may separate zone operations from DNS record operations. -Provider factories are also provider-specific. They resolve provider identity resources, credentials, SDK clients, and provider-specific readiness requirements before returning the local runtime interface. The core API only requires that `ZoneClass.spec.identityRef.name` points to the identity kind declared by the selected Provider version; the provider controller owns runtime identity resolution. +Provider factories are also provider-specific. When an identity is declared, they resolve provider identity resources, credentials, SDK clients, and provider-specific readiness requirements before returning the local runtime interface. Core requires `ZoneClass.spec.identityRef.name` to select the identity kind declared by the selected Provider version and forbids the reference when no identity is declared; the provider controller owns runtime identity resolution. A credentialless verifying provider instead uses a local read-only resolver interface, with no identity factory or DNS mutation methods. Fake providers are test boundaries, not public extension points. Unit tests and envtest-style controller tests use fake implementations of the provider package interfaces to verify finalizers, conditions, events, status provider data, adoption checks, provider error mapping, and retry behavior without calling real cloud APIs. These fakes should stay close to the controller package unless repeated cross-provider test logic proves that a shared helper removes real duplication. diff --git a/docs/design/test-policy.md b/docs/design/test-policy.md index 33d004c..81f2c4c 100644 --- a/docs/design/test-policy.md +++ b/docs/design/test-policy.md @@ -8,6 +8,8 @@ Conformance test distribution, execution method, configuration file schema, and envtest is used as integration testing with Kubernetes API server, etcd, CRDs, admission webhook, and reconciler. envtest does not operate real Route 53 or other external resources. Provider calls are replaced with fake or mock implementations. It checks Kubernetes resources, conditions, events, and finalizers. For TXT records, envtest or provider unit tests cover `spec.txt.values` validation, `_` in record names, core rejection of newline, tab, control-character, and non-ASCII TXT values through `forbid-txt-non-printable-ascii`, and Route 53 quote / escape / 255-octet chunk comparison as logical TXT values. For CNAME records, envtest or provider unit tests cover zone apex rejection, target syntax validation, standard-body exclusivity, and CNAME same-name conflict with other record types. For MX records, envtest or provider unit tests cover preference range validation, exchange syntax validation, duplicate pair rejection, Null MX validation, standard-body exclusivity, and Route 53 value parsing as unordered preference / exchange pairs. For CAA records, envtest or provider unit tests cover flags range validation, tag grammar validation, value presence, duplicate tuple rejection, standard-body exclusivity, and Route 53 quote / escape parsing as unordered flags / tag / value tuples. For delegated NS records, envtest or provider unit tests cover apex rejection, wildcard rejection, name server syntax validation, duplicate rejection, standard-body exclusivity, and Route 53 name server value comparison as unordered DNS name sets. +External provider: unit tests inject a fake Resolver/clock; a loopback UDP/TCP DNS fixture covers wire parsing; envtest injects controlled answers. kest `tests/e2e/external/external-a-record-lifecycle.test.ts` runs against an in-kind authoritative fixture (CoreDNS `file` plugin, not cluster service discovery) whose zone the harness edits to model the customer; it asserts mismatch→match→drift and never queries public DNS. + Provider validation tests cover the `forbid-txt-non-printable-ascii` disable switch as a core named validation. The built-in Route 53 and Cloudflare Provider manifests do not disable it. A provider test fixture may disable it to prove that the core webhook can be relaxed by a future provider version after that provider defines escape, unescape, display, and comparison behavior. Route 53 provider unit tests keep a defensive controller-side rejection path for non-printable ASCII TXT values even though the built-in webhook path rejects them earlier. envtest or webhook unit tests cover status subresource validation. For `RecordSet/status`, tests reject `status.observedGeneration` values greater than the `RecordSet.metadata.generation`, reject `status.zone.ref` namespace or name values that differ from `spec.zoneRef` after namespace defaulting, and reject any `status.zone.controllerName` field. Tests also accept a valid `status.zone.ref` as the controller-resolved `Zone` reference. CRD schema or generated-type checks must show that `RecordSet.status.zone.controllerName` is absent from the API surface. @@ -141,7 +143,7 @@ kest assertions are limited to API-contract behavior observable by users. Primar `ZoneUnit` ownership keys, stable record-set map behavior, finalizers, and controller internal intermediate states are tested in conformance tests or envtest. Initial kest success paths do not directly inspect internal `ZoneUnit` ownership details. -Failure paths, DNS propagation checks, adoption, conflict, drift, external AWS resource alias targets, alias target discovery, and long TXT value chunking are not part of initial kest scenarios. TXT quote, escape, and 255-octet chunking details are covered by envtest and provider implementation unit tests. +Failure paths, DNS propagation checks, adoption, conflict, drift, external AWS resource alias targets, alias target discovery, and long TXT value chunking are not part of the initial writing-provider kest scenarios. TXT quote, escape, and 255-octet chunking details are covered by envtest and provider implementation unit tests. The external provider's representative success path explicitly includes mismatch→match→drift because verification without DNS writes is its public contract; exhaustive DNS failures remain unit/envtest coverage. ## kest Implementation Rules @@ -165,7 +167,7 @@ Local Cloudflare kest reads `CF_ACCOUNT_ID` and `CF_API_TOKEN` from the environm Route 53 `INSYNC` checks wait up to 5 minutes. Short retry intervals are handled by kest action options; fixed sleeps are not used. -kest uses `ZoneClass.spec.parameters.zoneDeletionPolicy=Delete`. Hosted zones created by kest receive standard ownership tags plus `appthrust.io/test-scope=kest` and `appthrust.io/test-id=` from `ZoneClass.spec.parameters.tags`. The controller has no kest-specific behavior; it only uses normal tag support. +Writing-provider kest uses `ZoneClass.spec.parameters.zoneDeletionPolicy=Delete`. Route 53 hosted zones created by kest receive standard ownership tags plus `appthrust.io/test-scope=kest` and `appthrust.io/test-id=` from `ZoneClass.spec.parameters.tags`. The controller has no kest-specific behavior; it only uses normal tag support. External kest has no zone deletion policy or provider-owned DNS resources: the harness owns and cleans up its dedicated authoritative fixture, and deleting Kubernetes claims does not delete the fixture's DNS data. Leftover external resources from failures are not covered by the kest scenario itself. If `KEST_PRESERVE_ON_FAILURE=1` stops cleanup or a failure leaves a hosted zone, manual cleanup uses `appthrust.io/test-scope` and `appthrust.io/test-id`. diff --git a/docs/design/ui.md b/docs/design/ui.md index 1221d67..266492c 100644 --- a/docs/design/ui.md +++ b/docs/design/ui.md @@ -434,11 +434,13 @@ Delete confirmation shows that provider identities still referenced by `ZoneClas ## Zone Classes -`ZoneClass` is a provider-backed DNS zone capability published as platform policy to application namespaces. The initial UI handles only Public DNS. It does not show private hosted zone or VPC selection. +`ZoneClass` is a provider-backed DNS zone capability published as platform policy to application namespaces. The UI does not expose private hosted zone or VPC selection for writing providers. The external provider supports operator-selected public or private resolver views without creating hosted zones. -Creation starts with selecting a provider identity. Provider identities are grouped by provider and shown as cards. The `ZoneClass` is created in the same namespace as the selected provider identity. +For a Provider version that declares `identity`, creation starts with selecting a provider identity. Provider identities are grouped by provider and shown as cards. The `ZoneClass` is created in the same namespace as the selected provider identity. For a Provider version without `identity`, ZoneClass creation starts from the provider picker instead of the provider identity picker, omits `identityRef`, and the `Provider identity` vitals field is omitted. The user selects an authorized ZoneClass namespace in this credentialless flow. External ZoneClass creation is supported in stage 1. -The `ZoneClass` form handles `Name`, `Description`, `provider.name`, `provider.version`, `controllerName`, `identityRef`, provider policy parameters such as `zoneCreationPolicy`, `zoneDeletionPolicy`, `sameNameZonePolicy`, and `allowedZones`. `provider` and `identityRef.name` are derived from the selected provider identity and Provider version identity resource declaration. `provider` is shown as read-only provider name and version fields. `controllerName` is selected from the provider controller instance for the selected provider and is required. `identityRef` is shown as the selected provider identity, not as a raw group / kind / namespace editor. Hosted zone type is not selectable. +The `ZoneClass` form handles `Name`, `Description`, `provider.name`, `provider.version`, `controllerName`, optional `identityRef`, provider policy parameters, and `allowedZones`. For identity-backed providers, `provider` and `identityRef.name` are derived from the selected provider identity and Provider version identity resource declaration; for credentialless providers, `provider` comes from the selected Provider version and `identityRef` is absent from both the manifest preview and submitted object. `provider` is shown as read-only provider name and version fields after selection. `controllerName` is selected from the provider controller instance for the selected provider and is required. When declared, `identityRef` is shown as the selected provider identity, not as a raw group / kind / namespace editor. Hosted zone type is not selectable. + +External parameters are `resolver.mode`, `resolver.servers`, and `expectedNameservers`, using the schema and validation in [External Provider](external-provider.md#zoneclass-parameters). External does not show or submit the writing-provider parameters `zoneCreationPolicy`, `zoneDeletionPolicy`, or `sameNameZonePolicy`, and does not offer identity creation. The synchronized form implementation is `pkg/ts/ui/src/views/platform/ZoneClassForm.tsx`. `sameNameZonePolicy` is shown only when `zoneCreationPolicy=Create`. The UI label is `Duplicate hosted zone policy`. When `zoneCreationPolicy=Deny`, this policy is hidden. @@ -450,7 +452,7 @@ Route 53 `ZoneClass.spec.parameters.tags` is edited as a YAML mapping of `key: v `ZoneClass` detail follows the shared resource detail pattern. The header actions are `Edit` and the resource actions menu. `Delete ZoneClass` is inside the resource actions menu and opens the delete confirmation screen; it is not a standalone header button. -The `ZoneClass` detail vitals strip shows `Status`, `Used by`, `Namespace`, `Name`, `Provider`, and `Provider identity`. `Status` is derived from `ZoneClass` conditions. `Used by` is the count of `Zone` resources referencing the `ZoneClass`. `Provider identity` links to the same-namespace provider identity detail page when the referenced identity is visible; otherwise it shows the known identity name as plain text. The vitals strip is the canonical place for these fields, so the body does not repeat `Provider` and `Provider Identity` in a separate basic information grid. +The `ZoneClass` detail vitals strip shows `Status`, `Used by`, `Namespace`, `Name`, and `Provider`, plus `Provider identity` only when the selected Provider version declares `identity`. `Status` is derived from `ZoneClass` conditions. `Used by` is the count of `Zone` resources referencing the `ZoneClass`. When present, `Provider identity` links to the same-namespace provider identity detail page when the referenced identity is visible; otherwise it shows the known identity name as plain text. The vitals strip is the canonical place for these fields, so the body does not repeat `Provider` and `Provider Identity` in a separate basic information grid. `ZoneClass` detail tabs are ordered `Zones`, `Status`, `Policy`, and `Manifest`. From d039c48a05adfff6547ec5005a3fa3d3fe660ae0 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 10:40:19 +0900 Subject: [PATCH 02/16] feat(core): support credentialless DNS providers Make ZoneClass identityRef optional with Provider-version admission, preserve projected condition messages, gate cloud registrations and privileges, and support credentialless ZoneClass UI creation. Regenerate deepcopy and both CRD distributions; add deterministic admission, projection, chart and UI regressions. Contract: dns-api PR #15 / 99d0753 docs/design/external-provider.md (ZoneClass Parameters, Conditions and Public Data, Packaging), core-api.md, provider-extension.md, controllers-webhooks.md (Provider Enablement), ui.md (Zone Classes), and test-policy.md. Platform PR #5331 / cd995cf0ed design/platform/onprem/README.md sections 4.3, 4.4 step 4, 4.7, and Phase 4. Codegen: devbox run -- task generate manifests controller-chart:sync-crds completed. Tests, builds, linters and UI typecheck intentionally deferred to the primary integration agent. --- app/operator/cmd/manager/main.go | 156 +++++++++++------- app/operator/cmd/manager/main_test.go | 23 +++ .../bases/dns.appthrust.io_zoneclasses.yaml | 6 +- .../crds/dns.appthrust.io_zoneclasses.yaml | 6 +- deploy/charts/dns-api/rbac_contract_test.go | 151 +++++++++++++++++ .../charts/dns-api/templates/deployment.yaml | 3 + .../endpoint-capability-route53.yaml | 2 +- deploy/charts/dns-api/templates/rbac.yaml | 15 ++ .../validating-webhook-configuration.yaml | 47 ++++++ docs/design/controllers-webhooks.md | 24 ++- docs/design/core-api.md | 6 +- docs/design/ui.md | 10 +- .../zoneunit/zoneunit_controller_test.go | 2 +- .../controllers/zoneunit/zoneunit_status.go | 21 +-- .../zoneunit_status_freshness_test.go | 107 +++++++++++- internal/go/core/webhook/validator.go | 8 +- internal/go/core/webhook/validator_test.go | 43 ++++- .../controllers/zoneclass/controller.go | 4 +- .../controllers/zoneunit/recordset.go | 2 +- .../cloudflare/controllers/zoneunit/zone.go | 4 +- .../controllers/zoneunit/zone_test.go | 25 ++- .../controllers/zoneclass/controller.go | 4 +- .../controllers/zoneclass/controller_test.go | 26 +++ .../zoneclass/test_helpers_test.go | 2 +- .../route53/controllers/zoneunit/zone.go | 4 +- .../zoneunit/zone_test_fixtures_test.go | 2 +- pkg/go/api/dns/v1alpha1/scheme_test.go | 25 +++ pkg/go/api/dns/v1alpha1/zoneclass_types.go | 4 +- .../api/dns/v1alpha1/zz_generated.deepcopy.go | 6 +- .../platform/PlatformZoneClassDetailPage.tsx | 20 ++- .../PlatformZoneClassIdentitySelectPage.tsx | 21 ++- .../platform/PlatformZoneClassesPage.tsx | 9 +- .../src/views/platform/ZoneClassForm.test.ts | 65 ++++++++ .../ui/src/views/platform/ZoneClassForm.tsx | 129 ++++++++++++--- 34 files changed, 838 insertions(+), 144 deletions(-) create mode 100644 app/operator/cmd/manager/main_test.go create mode 100644 pkg/ts/ui/src/views/platform/ZoneClassForm.test.ts diff --git a/app/operator/cmd/manager/main.go b/app/operator/cmd/manager/main.go index 6857aa9..fcfb8b8 100644 --- a/app/operator/cmd/manager/main.go +++ b/app/operator/cmd/manager/main.go @@ -3,6 +3,7 @@ package main import ( "flag" "os" + "strconv" endpointrecordsetcontroller "github.com/appthrust/dns-api/internal/go/apps/endpoint/controllers/endpointrecordset" gatewayendpointcontroller "github.com/appthrust/dns-api/internal/go/apps/gatewayendpoint/controllers/gateway" @@ -82,6 +83,9 @@ func main() { var leaderElection bool var gatewayEndpointControllerEnabled bool var serviceEndpointControllerEnabled bool + route53ControllerEnabled := envBoolOrDefault("ROUTE53_CONTROLLER_ENABLED", true) + cloudflareControllerEnabled := envBoolOrDefault("CLOUDFLARE_CONTROLLER_ENABLED", true) + route53EndpointConversionEnabled := envBoolOrDefault("ROUTE53_ENDPOINT_CONVERSION_ENABLED", true) endpointRecordSetNamespace := envOrDefault("ENDPOINT_RECORDSET_NAMESPACE", "") route53ControllerName := envOrDefault("ROUTE53_CONTROLLER_NAME", route53zoneunit.DefaultControllerName) route53ProviderName := envOrDefault("ROUTE53_PROVIDER_NAME", route53v1alpha1.ProviderName) @@ -96,6 +100,9 @@ func main() { flag.BoolVar(&gatewayEndpointControllerEnabled, "gateway-endpoint-controller-enabled", true, "Watch Gateway API resources and synthesize EndpointRecordSets from HTTPRoutes.") flag.BoolVar(&serviceEndpointControllerEnabled, "service-endpoint-controller-enabled", true, "Watch Services and synthesize EndpointRecordSets from their endpoint annotations.") flag.StringVar(&endpointRecordSetNamespace, "endpoint-recordset-namespace", endpointRecordSetNamespace, "Namespace where generated EndpointRecordSet and RecordSet resources are stored. Defaults to the source namespace.") + flag.BoolVar(&route53ControllerEnabled, "route53-controller-enabled", route53ControllerEnabled, "Enable Route 53 controllers and identity admission.") + flag.BoolVar(&cloudflareControllerEnabled, "cloudflare-controller-enabled", cloudflareControllerEnabled, "Enable Cloudflare controllers and identity admission.") + flag.BoolVar(&route53EndpointConversionEnabled, "route53-endpoint-conversion-enabled", route53EndpointConversionEnabled, "Serve endpoint conversion when the Route 53 provider is enabled.") flag.StringVar(&route53ControllerName, "route53-controller-name", route53ControllerName, "ZoneClass.spec.controllerName handled by the Route 53 controller.") flag.StringVar(&route53ProviderName, "route53-provider-name", route53ProviderName, "Provider.metadata.name handled by the Route 53 controller.") flag.StringVar(&route53ProviderVersion, "route53-provider-version", route53ProviderVersion, "Provider version handled by the Route 53 controller.") @@ -123,68 +130,72 @@ func main() { os.Exit(1) } - if err := (&route53identity.IdentityReconciler{ - Client: mgr.GetClient(), - Resolver: route53identity.NewAWSIdentityResolver(), - Recorder: mgr.GetEventRecorderFor("route53-identity-controller"), - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Route 53 Identity controller") - os.Exit(1) - } + if route53ControllerEnabled { + if err := (&route53identity.IdentityReconciler{ + Client: mgr.GetClient(), + Resolver: route53identity.NewAWSIdentityResolver(), + Recorder: mgr.GetEventRecorderFor("route53-identity-controller"), + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Route 53 Identity controller") + os.Exit(1) + } - if err := (&cloudflareidentity.IdentityReconciler{ - Client: mgr.GetClient(), - Resolver: cloudflareidentity.NewCloudflareAPIClient(), - Recorder: mgr.GetEventRecorderFor("cloudflare-identity-controller"), - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Cloudflare Identity controller") - os.Exit(1) - } + if err := (&route53zoneclass.ZoneClassReconciler{ + ControllerName: route53ControllerName, + Client: mgr.GetClient(), + ProviderName: route53ProviderName, + ProviderVersion: route53ProviderVersion, + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up ZoneClass controller") + os.Exit(1) + } - if err := (&route53zoneclass.ZoneClassReconciler{ - ControllerName: route53ControllerName, - Client: mgr.GetClient(), - ProviderName: route53ProviderName, - ProviderVersion: route53ProviderVersion, - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up ZoneClass controller") - os.Exit(1) + if err := (&route53zoneunit.ZoneReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + ProviderFactory: route53zoneunit.NewAWSProviderFactory(), + ControllerName: route53ControllerName, + ProviderName: route53ProviderName, + ProviderVersion: route53ProviderVersion, + Recorder: mgr.GetEventRecorderFor("route53-zoneunit-controller"), + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Route 53 ZoneUnit controller") + os.Exit(1) + } } - if err := (&cloudflarezoneclass.ZoneClassReconciler{ - ControllerName: cloudflareControllerName, - Client: mgr.GetClient(), - ProviderName: cloudflareProviderName, - ProviderVersion: cloudflareProviderVersion, - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Cloudflare ZoneClass controller") - os.Exit(1) - } + if cloudflareControllerEnabled { + if err := (&cloudflareidentity.IdentityReconciler{ + Client: mgr.GetClient(), + Resolver: cloudflareidentity.NewCloudflareAPIClient(), + Recorder: mgr.GetEventRecorderFor("cloudflare-identity-controller"), + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Cloudflare Identity controller") + os.Exit(1) + } - if err := (&route53zoneunit.ZoneReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - ProviderFactory: route53zoneunit.NewAWSProviderFactory(), - ControllerName: route53ControllerName, - ProviderName: route53ProviderName, - ProviderVersion: route53ProviderVersion, - Recorder: mgr.GetEventRecorderFor("route53-zoneunit-controller"), - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Route 53 ZoneUnit controller") - os.Exit(1) - } + if err := (&cloudflarezoneclass.ZoneClassReconciler{ + ControllerName: cloudflareControllerName, + Client: mgr.GetClient(), + ProviderName: cloudflareProviderName, + ProviderVersion: cloudflareProviderVersion, + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Cloudflare ZoneClass controller") + os.Exit(1) + } - if err := (&cloudflarezoneunit.ZoneReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - ProviderFactory: cloudflarezoneunit.NewCloudflareAPIClient(), - ControllerName: cloudflareControllerName, - ProviderName: cloudflareProviderName, - ProviderVersion: cloudflareProviderVersion, - Recorder: mgr.GetEventRecorderFor("cloudflare-zoneunit-controller"), - }).SetupWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Cloudflare ZoneUnit controller") - os.Exit(1) + if err := (&cloudflarezoneunit.ZoneReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + ProviderFactory: cloudflarezoneunit.NewCloudflareAPIClient(), + ControllerName: cloudflareControllerName, + ProviderName: cloudflareProviderName, + ProviderVersion: cloudflareProviderVersion, + Recorder: mgr.GetEventRecorderFor("cloudflare-zoneunit-controller"), + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Cloudflare ZoneUnit controller") + os.Exit(1) + } } if err := (&zoneunitcontroller.ZoneUnitCompositionReconciler{ @@ -230,15 +241,21 @@ func main() { ctrl.Log.Error(err, "unable to set up core validation webhook") os.Exit(1) } - if err := route53webhook.SetupValidationWebhookWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Route 53 validation webhook") - os.Exit(1) + if route53ControllerEnabled { + if err := route53webhook.SetupValidationWebhookWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Route 53 validation webhook") + os.Exit(1) + } + if route53EndpointConversionEnabled { + route53conversion.NewHandler().Register(mgr.GetWebhookServer()) + } } - if err := cloudflarewebhook.SetupValidationWebhookWithManager(mgr); err != nil { - ctrl.Log.Error(err, "unable to set up Cloudflare validation webhook") - os.Exit(1) + if cloudflareControllerEnabled { + if err := cloudflarewebhook.SetupValidationWebhookWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up Cloudflare validation webhook") + os.Exit(1) + } } - route53conversion.NewHandler().Register(mgr.GetWebhookServer()) if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { ctrl.Log.Error(err, "unable to set up health check") @@ -262,3 +279,16 @@ func envOrDefault(name, fallback string) string { } return fallback } + +func envBoolOrDefault(name string, fallback bool) bool { + value := os.Getenv(name) + if value == "" { + return fallback + } + enabled, err := strconv.ParseBool(value) + if err != nil { + ctrl.Log.Error(err, "invalid boolean environment variable", "name", name) + os.Exit(1) + } + return enabled +} diff --git a/app/operator/cmd/manager/main_test.go b/app/operator/cmd/manager/main_test.go new file mode 100644 index 0000000..2c21b9b --- /dev/null +++ b/app/operator/cmd/manager/main_test.go @@ -0,0 +1,23 @@ +package main + +import "testing" + +func TestProviderEnablementEnvironmentDefaults(t *testing.T) { + for _, name := range []string{"ROUTE53_CONTROLLER_ENABLED", "CLOUDFLARE_CONTROLLER_ENABLED", "ROUTE53_ENDPOINT_CONVERSION_ENABLED"} { + t.Run(name, func(t *testing.T) { + for _, test := range []struct { + value string + want bool + }{ + {value: "", want: true}, + {value: "false", want: false}, + {value: "true", want: true}, + } { + t.Setenv(name, test.value) + if got := envBoolOrDefault(name, true); got != test.want { + t.Fatalf("envBoolOrDefault(%q) with %q = %t, want %t", name, test.value, got, test.want) + } + } + }) + } +} diff --git a/app/operator/config/crd/bases/dns.appthrust.io_zoneclasses.yaml b/app/operator/config/crd/bases/dns.appthrust.io_zoneclasses.yaml index bd3c826..35f5b0e 100644 --- a/app/operator/config/crd/bases/dns.appthrust.io_zoneclasses.yaml +++ b/app/operator/config/crd/bases/dns.appthrust.io_zoneclasses.yaml @@ -121,8 +121,9 @@ spec: - message: controllerName is immutable rule: self == oldSelf identityRef: - description: IdentityRef selects the provider identity resource in - the ZoneClass namespace. + description: |- + IdentityRef selects the provider identity resource in the ZoneClass namespace. + Required when the selected Provider version declares identity; forbidden otherwise. properties: name: description: Name is the referenced object name in the same namespace @@ -154,7 +155,6 @@ spec: required: - allowedZones - controllerName - - identityRef - parameters - provider type: object diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_zoneclasses.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_zoneclasses.yaml index bd3c826..35f5b0e 100644 --- a/deploy/charts/dns-api/crds/dns.appthrust.io_zoneclasses.yaml +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_zoneclasses.yaml @@ -121,8 +121,9 @@ spec: - message: controllerName is immutable rule: self == oldSelf identityRef: - description: IdentityRef selects the provider identity resource in - the ZoneClass namespace. + description: |- + IdentityRef selects the provider identity resource in the ZoneClass namespace. + Required when the selected Provider version declares identity; forbidden otherwise. properties: name: description: Name is the referenced object name in the same namespace @@ -154,7 +155,6 @@ spec: required: - allowedZones - controllerName - - identityRef - parameters - provider type: object diff --git a/deploy/charts/dns-api/rbac_contract_test.go b/deploy/charts/dns-api/rbac_contract_test.go index f8f606b..9bc04be 100644 --- a/deploy/charts/dns-api/rbac_contract_test.go +++ b/deploy/charts/dns-api/rbac_contract_test.go @@ -1,11 +1,21 @@ package dnsapichart import ( + "bytes" + "fmt" + "os" "os/exec" "path/filepath" + "reflect" "runtime" + "slices" + "strconv" "strings" "testing" + + admissionv1 "k8s.io/api/admissionregistration/v1" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + "sigs.k8s.io/yaml" ) // The dns-api chart owns only its controller identity. Cloud's central @@ -50,6 +60,147 @@ func TestEndpointRecordSetRBACBindsOnlyControllerServiceAccount(t *testing.T) { } } +func TestProviderEnablementGatesRuntimePermissionsAndAdmission(t *testing.T) { + for _, route53 := range []bool{false, true} { + for _, cloudflare := range []bool{false, true} { + for _, conversion := range []bool{false, true} { + t.Run(fmt.Sprintf("route53=%t/cloudflare=%t/conversion=%t", route53, cloudflare, conversion), func(t *testing.T) { + rendered := renderChart(t, + "--set", "providers.route53.enabled="+strconv.FormatBool(route53), + "--set", "providers.cloudflare.enabled="+strconv.FormatBool(cloudflare), + "--set", "providers.route53.endpoint.conversionAPI.enabled="+strconv.FormatBool(conversion), + "--set", "gatewayEndpoint.enabled=false", + "--set", "serviceEndpoint.enabled=false", + ) + role := renderedDocument(t, rendered, "rbac.authorization.k8s.io/v1", "ClusterRole") + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + webhooks := renderedDocument(t, rendered, "admissionregistration.k8s.io/v1", "ValidatingWebhookConfiguration") + for flag, enabled := range map[string]bool{ + "route53-controller-enabled": route53, + "cloudflare-controller-enabled": cloudflare, + "route53-endpoint-conversion-enabled": route53 && conversion, + } { + if !strings.Contains(deployment, "--"+flag+"="+strconv.FormatBool(enabled)) { + t.Fatalf("Deployment does not pass %s=%t:\n%s", flag, enabled, deployment) + } + } + for permission, enabled := range map[string]bool{ + "- route53.dns.appthrust.io\n": route53, + "- cloudflare.dns.appthrust.io\n": cloudflare, + "- secrets\n": cloudflare, + "- endpoint.route53.dns.appthrust.io\n": route53 && conversion, + "- gateway.networking.k8s.io\n": false, + "- services\n": false, + "- namespaces\n": true, + "- dns.appthrust.io\n": true, + "- endpoint.dns.appthrust.io\n": true, + } { + if strings.Contains(role, permission) != enabled { + t.Fatalf("permission %q presence does not equal %t:\n%s", permission, enabled, role) + } + } + for resource, enabled := range map[string]bool{ + "route53identities": route53, + "cloudflareidentities": cloudflare, + "zoneclasses": true, + } { + for _, suffix := range []string{"", "/status"} { + if strings.Contains(webhooks, "- "+resource+suffix+"\n") != enabled { + t.Fatalf("admission for %s%s does not equal %t:\n%s", resource, suffix, enabled, webhooks) + } + } + } + for declaration, enabled := range map[string]bool{ + "name: route53.dns.appthrust.io\n": route53, + "name: cloudflare.dns.appthrust.io\n": cloudflare, + "kind: EndpointProviderCapability\n": route53 && conversion, + "kind: APIService\n": route53 && conversion, + } { + if strings.Contains(rendered, declaration) != enabled { + t.Fatalf("declaration %q presence does not equal %t", declaration, enabled) + } + } + if strings.Count(rendered, "kind: ClusterRoleBinding\n") != 1 { + t.Fatal("expected exactly one controller ClusterRoleBinding") + } + }) + } + } + } +} + +func TestRoute53ConversionRequiresWebhookService(t *testing.T) { + rendered := renderChart(t, "--set", "webhook.enabled=false") + for _, forbidden := range []string{"kind: APIService\n", "kind: EndpointProviderCapability\n", "- endpoint.route53.dns.appthrust.io\n"} { + if strings.Contains(rendered, forbidden) { + t.Fatalf("webhook-disabled render contains %q", forbidden) + } + } + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + if !strings.Contains(deployment, "--route53-endpoint-conversion-enabled=false") { + t.Fatal("webhook-disabled deployment must disable Route 53 conversion") + } +} + +func TestHelmAndKustomizeAdmissionCoverageAgree(t *testing.T) { + rendered := renderedDocument(t, renderChart(t), "admissionregistration.k8s.io/v1", "ValidatingWebhookConfiguration") + var helmConfig admissionv1.ValidatingWebhookConfiguration + if err := yaml.Unmarshal([]byte(rendered), &helmConfig); err != nil { + t.Fatalf("decode Helm admission: %v", err) + } + _, file, _, _ := runtime.Caller(0) + source, err := os.ReadFile(filepath.Join(filepath.Dir(file), "../../../app/operator/config/webhook/validating_webhook_configuration.yaml")) + if err != nil { + t.Fatalf("read Kustomize admission: %v", err) + } + var kustomizeConfig admissionv1.ValidatingWebhookConfiguration + if err := yaml.Unmarshal(source, &kustomizeConfig); err != nil { + t.Fatalf("decode Kustomize admission: %v", err) + } + if len(helmConfig.Webhooks) != len(kustomizeConfig.Webhooks) { + t.Fatalf("Helm has %d webhooks, Kustomize has %d", len(helmConfig.Webhooks), len(kustomizeConfig.Webhooks)) + } + for index, expected := range kustomizeConfig.Webhooks { + actual := helmConfig.Webhooks[index] + // Release namespace and service names differ; admission coverage must not. + actual.ClientConfig.Service.Namespace = expected.ClientConfig.Service.Namespace + actual.ClientConfig.Service.Name = expected.ClientConfig.Service.Name + if !reflect.DeepEqual(actual, expected) { + t.Fatalf("admission differs for %s:\nHelm: %#v\nKustomize: %#v", expected.Name, actual, expected) + } + } +} + +func TestZoneClassCRDDistributionsAgreeOnOptionalIdentity(t *testing.T) { + _, file, _, _ := runtime.Caller(0) + chartDir := filepath.Dir(file) + chartCRD, err := os.ReadFile(filepath.Join(chartDir, "crds/dns.appthrust.io_zoneclasses.yaml")) + if err != nil { + t.Fatalf("read chart CRD: %v", err) + } + operatorCRD, err := os.ReadFile(filepath.Join(chartDir, "../../../app/operator/config/crd/bases/dns.appthrust.io_zoneclasses.yaml")) + if err != nil { + t.Fatalf("read operator CRD: %v", err) + } + if !bytes.Equal(chartCRD, operatorCRD) { + t.Fatal("ZoneClass CRD distributions differ; run task controller-chart:sync-crds") + } + var crd apiextensionsv1.CustomResourceDefinition + if err := yaml.Unmarshal(chartCRD, &crd); err != nil { + t.Fatalf("decode ZoneClass CRD: %v", err) + } + for _, version := range crd.Spec.Versions { + spec := version.Schema.OpenAPIV3Schema.Properties["spec"] + if slices.Contains(spec.Required, "identityRef") { + t.Fatalf("%s structurally requires identityRef", version.Name) + } + identity := spec.Properties["identityRef"] + if identity.Type != "object" || !slices.Contains(identity.Required, "name") { + t.Fatalf("%s does not require name when identityRef is present: %#v", version.Name, identity) + } + } +} + func ruleForAPIGroup(t *testing.T, role, apiGroup string) string { t.Helper() start := strings.Index(role, " - apiGroups:\n - "+apiGroup+"\n") diff --git a/deploy/charts/dns-api/templates/deployment.yaml b/deploy/charts/dns-api/templates/deployment.yaml index a2dea37..a9ee01d 100644 --- a/deploy/charts/dns-api/templates/deployment.yaml +++ b/deploy/charts/dns-api/templates/deployment.yaml @@ -41,6 +41,9 @@ spec: - --endpoint-recordset-namespace={{ .Values.endpoint.recordSetNamespace }} - --gateway-endpoint-controller-enabled={{ .Values.gatewayEndpoint.enabled }} - --service-endpoint-controller-enabled={{ .Values.serviceEndpoint.enabled }} + - --route53-controller-enabled={{ .Values.providers.route53.enabled }} + - --cloudflare-controller-enabled={{ .Values.providers.cloudflare.enabled }} + - --route53-endpoint-conversion-enabled={{ and .Values.providers.route53.enabled .Values.providers.route53.endpoint.conversionAPI.enabled .Values.webhook.enabled }} {{- range .Values.extraArgs }} - {{ . | quote }} {{- end }} diff --git a/deploy/charts/dns-api/templates/endpoint-capability-route53.yaml b/deploy/charts/dns-api/templates/endpoint-capability-route53.yaml index fa6460b..2d1538b 100644 --- a/deploy/charts/dns-api/templates/endpoint-capability-route53.yaml +++ b/deploy/charts/dns-api/templates/endpoint-capability-route53.yaml @@ -1,4 +1,4 @@ -{{- if and .Values.providers.route53.enabled .Values.providers.route53.endpoint.conversionAPI.enabled }} +{{- if and .Values.webhook.enabled .Values.providers.route53.enabled .Values.providers.route53.endpoint.conversionAPI.enabled }} apiVersion: endpoint.dns.appthrust.io/v1alpha1 kind: EndpointProviderCapability metadata: diff --git a/deploy/charts/dns-api/templates/rbac.yaml b/deploy/charts/dns-api/templates/rbac.yaml index 349c6aa..3777fb7 100644 --- a/deploy/charts/dns-api/templates/rbac.yaml +++ b/deploy/charts/dns-api/templates/rbac.yaml @@ -17,11 +17,20 @@ rules: - "" resources: - namespaces + verbs: + - get + - list + - watch + {{- if .Values.providers.cloudflare.enabled }} + - apiGroups: + - "" + resources: - secrets verbs: - get - list - watch + {{- end }} {{- if .Values.serviceEndpoint.enabled }} - apiGroups: - "" @@ -39,6 +48,7 @@ rules: verbs: - update {{- end }} + {{- if .Values.providers.cloudflare.enabled }} - apiGroups: - cloudflare.dns.appthrust.io resources: @@ -57,6 +67,7 @@ rules: - get - patch - update + {{- end }} - apiGroups: - coordination.k8s.io resources: @@ -153,12 +164,14 @@ rules: - get - patch - update + {{- if and .Values.providers.route53.enabled .Values.providers.route53.endpoint.conversionAPI.enabled .Values.webhook.enabled }} - apiGroups: - endpoint.route53.dns.appthrust.io resources: - endpointrecordsetconversions verbs: - create + {{- end }} {{- if .Values.gatewayEndpoint.enabled }} - apiGroups: - gateway.networking.k8s.io @@ -184,6 +197,7 @@ rules: verbs: - update {{- end }} + {{- if .Values.providers.route53.enabled }} - apiGroups: - route53.dns.appthrust.io resources: @@ -202,6 +216,7 @@ rules: - get - patch - update + {{- end }} --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding diff --git a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml index 3c80b38..c809831 100644 --- a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml +++ b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml @@ -50,6 +50,53 @@ webhooks: - UPDATE resources: - zoneclasses + - zoneclasses/status + {{- if .Values.providers.route53.enabled }} + - name: route53identities.route53.dns.appthrust.io + admissionReviewVersions: + - v1 + sideEffects: None + failurePolicy: {{ .Values.webhook.failurePolicy }} + clientConfig: + service: + namespace: {{ .Release.Namespace }} + name: {{ include "dns-api.webhookServiceName" . }} + path: /validate-route53-dns-appthrust-io-v1alpha1-route53identity + rules: + - apiGroups: + - route53.dns.appthrust.io + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - route53identities + - route53identities/status + {{- end }} + {{- if .Values.providers.cloudflare.enabled }} + - name: cloudflareidentities.cloudflare.dns.appthrust.io + admissionReviewVersions: + - v1 + sideEffects: None + failurePolicy: {{ .Values.webhook.failurePolicy }} + clientConfig: + service: + namespace: {{ .Release.Namespace }} + name: {{ include "dns-api.webhookServiceName" . }} + path: /validate-cloudflare-dns-appthrust-io-v1alpha1-cloudflareidentity + rules: + - apiGroups: + - cloudflare.dns.appthrust.io + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cloudflareidentities + - cloudflareidentities/status + {{- end }} - name: zones.dns.appthrust.io admissionReviewVersions: - v1 diff --git a/docs/design/controllers-webhooks.md b/docs/design/controllers-webhooks.md index f119122..ae34b8f 100644 --- a/docs/design/controllers-webhooks.md +++ b/docs/design/controllers-webhooks.md @@ -25,8 +25,24 @@ Admission validation must not reject `Zone`, `RecordSet`, or `ZoneUnit` solely b All claim conditions include `observedGeneration`. If a resource moves out of a provider controller's scope, that controller stops updating `ZoneUnit.status`. Existing claim conditions can be recognized as old through `observedGeneration`. +When projecting a current provider condition from `ZoneUnit.status.zone` or `ZoneUnit.status.recordSets[]`, core preserves `Status`, `Reason`, and `Message`, including an empty Message. This applies to Accepted and Programmed on both Zone and RecordSet, without changing UID/generation fences or acceptance precedence. Core-generated waiting/rejection conditions retain their own messages. + Provider implementation differences are controlled by common conditions, reasons, status ownership, support levels, and conformance tests. The core does not hide differences by providing one shared DNS controller implementation; providers give a consistent user experience by satisfying the same contract. +## Provider Enablement + +Provider enablement gates runtime registrations and provider-only privileges, not just Provider manifests. The manager accepts these flags and environment defaults; explicit flags take precedence: + +| Chart value | Manager flag | Environment variable | Default | +| --- | --- | --- | --- | +| `providers.route53.enabled` | `--route53-controller-enabled` | `ROUTE53_CONTROLLER_ENABLED` | true | +| `providers.cloudflare.enabled` | `--cloudflare-controller-enabled` | `CLOUDFLARE_CONTROLLER_ENABLED` | true | +| `providers.route53.endpoint.conversionAPI.enabled` | `--route53-endpoint-conversion-enabled` | `ROUTE53_ENDPOINT_CONVERSION_ENABLED` | true | + +Each provider switch gates its identity, ZoneClass, and ZoneUnit controllers, credential-client construction, and identity spec/status admission handler. The chart gates matching identity RBAC and webhook entries. Secret get/list/watch belongs only to enabled Cloudflare; mounted serving certificates do not require Secret API reads. Route 53 conversion requires both Route 53 flags. Its chart flag, capability, APIService, and conversion-create permission additionally require the webhook Service to be enabled. + +Core composition, endpoint translation, namespace reads, and core schema/status admission remain enabled independently. Gateway/Service source switches retain their existing registration and permission gates. Scheme/CRD types remain available when a provider is disabled. Helm's default admission coverage matches Kustomize, including ZoneClass status and both identity spec/status endpoints. Kustomize's default installation enables both providers; custom disable overlays must remove the corresponding admission entries and privileges as well as setting manager flags. Disabling a provider does not complete existing finalizers. + ## Implementation Modules Implementation modules are controller ownership boundaries. One Kubernetes controller has one owning module, and one owning module owns at most one controller. The owning module contains the reconciler entrypoint, watch mapping, finalizer writes, status writes, and controller-specific planning. Shared helpers used by more than one controller must live in a non-controller module and must not own watches, finalizers, or status writes. @@ -86,7 +102,7 @@ The webhook server runs as part of the controller-runtime manager and listens on Webhook TLS certificates are issued by cert-manager. Tilt installs cert-manager as a Helm chart. dns-api manifests apply `Certificate`, `Issuer`, and `ValidatingWebhookConfiguration`. cert-manager CA injection sets the webhook `caBundle`. -Core validating webhook covers at least spec create/update for `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`, and status update for `Zone`, `RecordSet`, and `ZoneUnit`. Provider-specific validating webhooks cover provider-specific resources such as `Route53Identity` and `CloudflareIdentity`. The core webhook must not import provider API packages such as `api/route53` or `api/cloudflare`. +Core validating webhook covers at least spec create/update for `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`, and status update for `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`. Provider-specific validating webhooks cover enabled provider-specific resources such as `Route53Identity` and `CloudflareIdentity`. The core webhook must not import provider API packages such as `api/route53` or `api/cloudflare`. Provider controllers do not fully trust the webhook. The webhook is the primary user feedback boundary, but controllers keep minimal safety checks. Controller-side validation failure is returned as `Accepted=False` or `Programmed=False`. @@ -139,7 +155,11 @@ validate ZoneClass: provider, providerVersion = resolve ProviderVersion(spec.provider) - validate spec.identityRef shape by providerVersion.identity.resource + if providerVersion.identity is present: + require spec.identityRef with nonempty name + validate spec.identityRef shape by providerVersion.identity.resource + else: + reject spec.identityRef validate spec.parameters by providerVersion.zoneClass.schemas.parameters validate providerVersion.zoneClass.validationRules validate spec.allowedZones diff --git a/docs/design/core-api.md b/docs/design/core-api.md index 7f3a899..8c90e00 100644 --- a/docs/design/core-api.md +++ b/docs/design/core-api.md @@ -9,13 +9,13 @@ - `provider.name`: `Provider.metadata.name`. Immutable. Because it references cluster-scoped `Provider`, it does not include namespace or kind. - `provider.version`: `Provider.spec.versions[].name`. Immutable. - `controllerName`: provider controller instance that reconciles `ZoneUnit` resources reachable through this class. Required and immutable. -- `identityRef`: provider identity reference. Required. `identityRef.name` references the provider identity resource declared by the selected Provider version in the same namespace as the `ZoneClass`. +- `identityRef`: optional provider identity reference. Required with a nonempty `name` if and only if the selected Provider version declares `identity`; forbidden otherwise. When present, it references the declared identity resource in the same namespace as the `ZoneClass`. - `parameters`: provider-specific inline settings. Validated by the referenced Provider version `zoneClass.schemas.parameters`. - `allowedZones`: namespace conditions for `Zone` resources that may use this `ZoneClass`. `ZoneClass.spec.controllerName` uses the Gateway API-style domain-prefixed path form. It must include both a DNS subdomain prefix and a path part after `/`. Valid examples are `route53.dns.appthrust.io/controller` and `cloudflare.dns.appthrust.io/prod-a`; `route53.dns.appthrust.io` is invalid. -`ZoneClass.spec.identityRef` contains only `name`. It does not contain namespace, group, or kind. Cross-namespace identity references are not part of the initial design; identity resources live with the `ZoneClass` in the same platform namespace. The referenced Provider version declares the identity `group` and `kind`, so repeating them in every `ZoneClass` would create an unnecessary mismatch surface. +`ZoneClass.spec.identityRef`, when present, contains only `name`. It does not contain namespace, group, or kind. Cross-namespace identity references are not part of the initial design; identity resources live with the `ZoneClass` in the same platform namespace. The referenced Provider version declares the identity `group` and `kind`, so repeating them in every `ZoneClass` would create an unnecessary mismatch surface. Credentialless providers omit the field rather than using an empty or sentinel reference. The Go field is `IdentityRef *LocalObjectReference` with JSON tag `identityRef,omitempty`; both CRD distributions omit the structural requirement, while core admission enforces the Provider-dependent requirement. Upgrade separately managed CRD carriers with the controller; Helm does not upgrade existing CRDs during a normal chart upgrade. `ZoneClass.status.conditions` contains `Accepted`. `Accepted=True` means `provider`, `identityRef`, `parameters`, and `allowedZones` are statically acceptable. `Accepted=False` reasons are `InvalidProvider`, `InvalidIdentityRef`, `InvalidParameters`, and `DeniedByPolicy`. `Accepted=Unknown` reasons are `IdentityNotResolved`, `ProviderStorageVersionNotResolved`, and `ReconcileError`. @@ -53,7 +53,7 @@ spec: `ZoneClass` is not reconciled by a core controller. The core validating webhook reads the referenced Provider version and validates `spec.identityRef`, `spec.parameters`, and `spec.allowedZones`. -Provider controllers reconcile only `ZoneClass` resources whose `spec.controllerName` matches their own controller name. If it does not match, they do not update status. They interpret `identityRef` according to the Provider version identity resource declaration, interpret provider-specific `parameters`, and set `ZoneClass.status.conditions[Accepted]` to `True` when the class is statically acceptable. +Provider controllers reconcile only `ZoneClass` resources whose `spec.controllerName` matches their own controller name. If it does not match, they do not update status. They interpret `identityRef` only when the Provider version declares an identity resource, interpret provider-specific `parameters`, and set `ZoneClass.status.conditions[Accepted]` to `True` when the class is statically acceptable. Credentialless providers perform no identity lookup. Route 53 and Cloudflare still reject absent or empty references controller-side as `Accepted=False/InvalidIdentityRef`. ## Zone diff --git a/docs/design/ui.md b/docs/design/ui.md index 1221d67..9fb481e 100644 --- a/docs/design/ui.md +++ b/docs/design/ui.md @@ -434,11 +434,13 @@ Delete confirmation shows that provider identities still referenced by `ZoneClas ## Zone Classes -`ZoneClass` is a provider-backed DNS zone capability published as platform policy to application namespaces. The initial UI handles only Public DNS. It does not show private hosted zone or VPC selection. +`ZoneClass` is a provider-backed DNS zone capability published as platform policy to application namespaces. Credentialless providers can expose resolver policy without creating or adopting hosted zones. -Creation starts with selecting a provider identity. Provider identities are grouped by provider and shown as cards. The `ZoneClass` is created in the same namespace as the selected provider identity. +Creation selects a provider identity for versions that declare `identity`, or the Provider itself for credentialless versions. Identity-backed classes share the selected identity's namespace. Credentialless creation skips identity selection, offers a namespace selector, and checks create permission in the selected namespace. -The `ZoneClass` form handles `Name`, `Description`, `provider.name`, `provider.version`, `controllerName`, `identityRef`, provider policy parameters such as `zoneCreationPolicy`, `zoneDeletionPolicy`, `sameNameZonePolicy`, and `allowedZones`. `provider` and `identityRef.name` are derived from the selected provider identity and Provider version identity resource declaration. `provider` is shown as read-only provider name and version fields. `controllerName` is selected from the provider controller instance for the selected provider and is required. `identityRef` is shown as the selected provider identity, not as a raw group / kind / namespace editor. Hosted zone type is not selectable. +The `ZoneClass` form handles `Name`, `Description`, `provider.name`, `provider.version`, `controllerName`, optional `identityRef`, provider policy parameters, and `allowedZones`. Provider and controller binding remain read-only after selection. The selected Provider version determines whether the form shows identity information and includes `identityRef` in the preview/submitted manifest; credentialless versions omit it, even if an old identity name remains in form state. Route 53 and Cloudflare retain their identity selection and policy controls. + +Credentialless provider parameters use the existing JSON-object editor fallback. For External DNS this exposes `resolver.mode`, `resolver.servers`, and `expectedNameservers`; `{}` uses the Provider's omitted-field defaults. The form neither shows nor injects writing-provider creation/deletion policy or tags in this flow. Admission remains authoritative for the selected Provider's schema and CEL. `sameNameZonePolicy` is shown only when `zoneCreationPolicy=Create`. The UI label is `Duplicate hosted zone policy`. When `zoneCreationPolicy=Deny`, this policy is hidden. @@ -450,7 +452,7 @@ Route 53 `ZoneClass.spec.parameters.tags` is edited as a YAML mapping of `key: v `ZoneClass` detail follows the shared resource detail pattern. The header actions are `Edit` and the resource actions menu. `Delete ZoneClass` is inside the resource actions menu and opens the delete confirmation screen; it is not a standalone header button. -The `ZoneClass` detail vitals strip shows `Status`, `Used by`, `Namespace`, `Name`, `Provider`, and `Provider identity`. `Status` is derived from `ZoneClass` conditions. `Used by` is the count of `Zone` resources referencing the `ZoneClass`. `Provider identity` links to the same-namespace provider identity detail page when the referenced identity is visible; otherwise it shows the known identity name as plain text. The vitals strip is the canonical place for these fields, so the body does not repeat `Provider` and `Provider Identity` in a separate basic information grid. +The `ZoneClass` detail vitals strip shows `Status`, `Used by`, `Namespace`, `Name`, and `Provider`, plus `Provider identity` only when the selected Provider version declares `identity`. Credentialless classes omit the identity health row. `Status` is derived from `ZoneClass` conditions. `Used by` is the count of referencing `Zone` resources. When present, `Provider identity` links to the same-namespace identity detail page when visible; otherwise it shows the known name as plain text. `ZoneClass` detail tabs are ordered `Zones`, `Status`, `Policy`, and `Manifest`. diff --git a/internal/go/core/controllers/zoneunit/zoneunit_controller_test.go b/internal/go/core/controllers/zoneunit/zoneunit_controller_test.go index 697cae6..20791f6 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_controller_test.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_controller_test.go @@ -610,7 +610,7 @@ func route53ZoneClassForZoneUnitTest() *dnsv1alpha1.ZoneClass { Spec: dnsv1alpha1.ZoneClassSpec{ Provider: dnsv1alpha1.ProviderReference{Name: "route53.dns.appthrust.io", Version: "v1alpha1"}, ControllerName: "route53.dns.appthrust.io/controller", - IdentityRef: dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, + IdentityRef: &dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, AllowedZones: dnsv1alpha1.ZoneClassAllowedZones{ Namespaces: dnsv1alpha1.NamespacePolicy{From: dnsv1alpha1.NamespacesFromAll}, }, diff --git a/internal/go/core/controllers/zoneunit/zoneunit_status.go b/internal/go/core/controllers/zoneunit/zoneunit_status.go index f20e0e8..8e74113 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_status.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_status.go @@ -15,7 +15,7 @@ func (r *ZoneUnitCompositionReconciler) projectZoneUnitStatus(ctx context.Contex if unit.Status.Zone != nil { accepted := projectedAccepted(unit.Spec.Zone.ObservedGeneration, zone.Status.Conditions, unit.Status.Zone.Conditions) programmed := projectedProgrammed(unit.Spec.Zone.ObservedGeneration, accepted.Status, unit.Status.Zone.Conditions) - if err := r.setZoneClaimStatus(ctx, zone, accepted.Status, accepted.Reason, programmed.Status, programmed.Reason, unit.Status.Zone); err != nil { + if err := r.setZoneClaimStatusWithMessages(ctx, zone, accepted.Status, accepted.Reason, accepted.Message, programmed.Status, programmed.Reason, programmed.Message, unit.Status.Zone); err != nil { return err } } @@ -60,7 +60,7 @@ func (r *ZoneUnitCompositionReconciler) projectZoneUnitStatus(ctx context.Contex } accepted := projectedAccepted(desiredItem.ObservedGeneration, recordSet.Status.Conditions, providerConditions) programmed := projectedProgrammed(desiredItem.ObservedGeneration, accepted.Status, providerConditions) - if err := r.setRecordSetClaimStatusWithZone(ctx, recordSet, accepted.Status, accepted.Reason, programmed.Status, programmed.Reason, recordSetZoneStatus(zone, zoneClass), &providerStatus); err != nil { + if err := r.setRecordSetClaimStatusWithZoneAndMessages(ctx, recordSet, accepted.Status, accepted.Reason, accepted.Message, programmed.Status, programmed.Reason, programmed.Message, recordSetZoneStatus(zone, zoneClass), &providerStatus); err != nil { return err } } @@ -76,31 +76,32 @@ func retainedNotAllowedRecordSetItem(item dnsv1alpha1.ZoneUnitRecordSetSpec) boo } type projectedCondition struct { - Status metav1.ConditionStatus - Reason string + Status metav1.ConditionStatus + Reason string + Message string } func projectedAccepted(observedGeneration int64, compositionConditions, providerConditions []metav1.Condition) projectedCondition { provider := conditionAtGeneration(providerConditions, string(dnsv1alpha1.ConditionAccepted), observedGeneration) if provider != nil { - return projectedCondition{Status: provider.Status, Reason: provider.Reason} + return projectedCondition{Status: provider.Status, Reason: provider.Reason, Message: provider.Message} } composition := conditionAtGeneration(compositionConditions, string(dnsv1alpha1.ConditionAccepted), observedGeneration) if composition != nil && composition.Status != metav1.ConditionTrue { - return projectedCondition{Status: composition.Status, Reason: composition.Reason} + return projectedCondition{Status: composition.Status, Reason: composition.Reason, Message: composition.Message} } - return projectedCondition{Status: metav1.ConditionUnknown, Reason: "OwnerStateNotResolved"} + return projectedCondition{Status: metav1.ConditionUnknown, Reason: "OwnerStateNotResolved", Message: "OwnerStateNotResolved"} } func projectedProgrammed(observedGeneration int64, accepted metav1.ConditionStatus, providerConditions []metav1.Condition) projectedCondition { if accepted != metav1.ConditionTrue { - return projectedCondition{Status: metav1.ConditionUnknown, Reason: "Reconciling"} + return projectedCondition{Status: metav1.ConditionUnknown, Reason: "Reconciling", Message: "Reconciling"} } provider := conditionAtGeneration(providerConditions, string(dnsv1alpha1.ConditionProgrammed), observedGeneration) if provider == nil { - return projectedCondition{Status: metav1.ConditionUnknown, Reason: "Reconciling"} + return projectedCondition{Status: metav1.ConditionUnknown, Reason: "Reconciling", Message: "Reconciling"} } - return projectedCondition{Status: provider.Status, Reason: provider.Reason} + return projectedCondition{Status: provider.Status, Reason: provider.Reason, Message: provider.Message} } func conditionAtGeneration(conditions []metav1.Condition, conditionType string, observedGeneration int64) *metav1.Condition { diff --git a/internal/go/core/controllers/zoneunit/zoneunit_status_freshness_test.go b/internal/go/core/controllers/zoneunit/zoneunit_status_freshness_test.go index 3a152a3..c7d0504 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_status_freshness_test.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_status_freshness_test.go @@ -40,8 +40,8 @@ func TestZoneUnitCompositionReconcilerDoesNotRebadgeStaleSameUIDProviderStatus(t ObservedGeneration: 1, Zone: &dnsv1alpha1.ZoneUnitZoneStatus{ Conditions: []metav1.Condition{ - {Type: string(dnsv1alpha1.ConditionAccepted), Status: metav1.ConditionTrue, Reason: "Accepted", ObservedGeneration: 1}, - {Type: string(dnsv1alpha1.ConditionProgrammed), Status: metav1.ConditionTrue, Reason: "Programmed", ObservedGeneration: 1}, + {Type: string(dnsv1alpha1.ConditionAccepted), Status: metav1.ConditionTrue, Reason: "Accepted", Message: "stale acceptance", ObservedGeneration: 1}, + {Type: string(dnsv1alpha1.ConditionProgrammed), Status: metav1.ConditionTrue, Reason: "Programmed", Message: "stale programming", ObservedGeneration: 1}, }, }, RecordSets: []dnsv1alpha1.ZoneUnitRecordSetStatus{{ @@ -51,8 +51,8 @@ func TestZoneUnitCompositionReconcilerDoesNotRebadgeStaleSameUIDProviderStatus(t ObservedGeneration: 1, Provider: &dnsv1alpha1.ProviderStatus{Data: runtime.RawExtension{Raw: []byte(`{"recordID":"current"}`)}}, Conditions: []metav1.Condition{ - {Type: string(dnsv1alpha1.ConditionAccepted), Status: metav1.ConditionTrue, Reason: "Accepted", ObservedGeneration: 1}, - {Type: string(dnsv1alpha1.ConditionProgrammed), Status: metav1.ConditionTrue, Reason: "Programmed", ObservedGeneration: 1}, + {Type: string(dnsv1alpha1.ConditionAccepted), Status: metav1.ConditionTrue, Reason: "Accepted", Message: "stale acceptance", ObservedGeneration: 1}, + {Type: string(dnsv1alpha1.ConditionProgrammed), Status: metav1.ConditionTrue, Reason: "Programmed", Message: "stale programming", ObservedGeneration: 1}, }, }}, }, @@ -86,6 +86,13 @@ func TestZoneUnitCompositionReconcilerDoesNotRebadgeStaleSameUIDProviderStatus(t } assertConditionAtGeneration(t, staleZoneProjection.Status.Conditions, string(dnsv1alpha1.ConditionAccepted), metav1.ConditionUnknown, "OwnerStateNotResolved", zone.Generation) assertConditionAtGeneration(t, staleZoneProjection.Status.Conditions, string(dnsv1alpha1.ConditionProgrammed), metav1.ConditionUnknown, "Reconciling", zone.Generation) + for _, conditions := range [][]metav1.Condition{staleProjection.Status.Conditions, staleZoneProjection.Status.Conditions} { + for _, condition := range conditions { + if condition.Message != condition.Reason { + t.Fatalf("stale provider diagnostic leaked into current condition: %#v", condition) + } + } + } var currentUnit dnsv1alpha1.ZoneUnit if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(unit), ¤tUnit); err != nil { @@ -156,6 +163,98 @@ func TestZoneUnitCompositionReconcilerDoesNotRebadgeStaleSameUIDProviderStatus(t assertRawEqual(t, recovered.Status.Provider.Data, map[string]any{"recordID": "current"}) } +func TestZoneUnitProjectionPreservesCurrentProviderMessages(t *testing.T) { + for _, test := range []struct { + name string + accepted metav1.ConditionStatus + programmed metav1.ConditionStatus + message string + }{ + {name: "verified", accepted: metav1.ConditionTrue, programmed: metav1.ConditionTrue, message: "DNS matches every required view; no DNS changes were made."}, + {name: "mismatch", accepted: metav1.ConditionTrue, programmed: metav1.ConditionFalse, message: "DNS targets differ: unexpected AAAA."}, + {name: "unavailable", accepted: metav1.ConditionTrue, programmed: metav1.ConditionUnknown, message: "DNS verification unavailable: timeout."}, + {name: "empty", accepted: metav1.ConditionTrue, programmed: metav1.ConditionTrue}, + {name: "rejected", accepted: metav1.ConditionFalse, programmed: metav1.ConditionTrue, message: "Provider identity is not usable."}, + {name: "unresolved", accepted: metav1.ConditionUnknown, programmed: metav1.ConditionTrue, message: "Provider identity is not resolved."}, + } { + t.Run(test.name, func(t *testing.T) { + ctx := context.Background() + zone := zone("app", "example-com", "example.com") + zone.Generation = 2 + recordSet := aRecordSet("app", "www", "app", "example-com", "www") + recordSet.Generation = 2 + recordSet.UID = types.UID("current") + oldConditions := []metav1.Condition{ + {Type: "Accepted", Status: metav1.ConditionTrue, Reason: "Accepted", Message: "old acceptance", ObservedGeneration: 2}, + {Type: "Programmed", Status: metav1.ConditionTrue, Reason: "Programmed", Message: "old success", ObservedGeneration: 2}, + } + zone.Status.Conditions = append([]metav1.Condition(nil), oldConditions...) + recordSet.Status.Conditions = append([]metav1.Condition(nil), oldConditions...) + acceptedReason := "Accepted" + if test.accepted == metav1.ConditionFalse { + acceptedReason = "InvalidIdentityRef" + } else if test.accepted == metav1.ConditionUnknown { + acceptedReason = "IdentityNotResolved" + } + programmedReason := "Programmed" + if test.programmed == metav1.ConditionFalse { + programmedReason = "ExternalResourceMismatch" + } else if test.programmed == metav1.ConditionUnknown { + programmedReason = "ProviderUnavailable" + } + conditions := []metav1.Condition{ + {Type: "Accepted", Status: test.accepted, Reason: acceptedReason, Message: test.message, ObservedGeneration: 2}, + {Type: "Programmed", Status: test.programmed, Reason: programmedReason, Message: test.message, ObservedGeneration: 2}, + } + unit := &dnsv1alpha1.ZoneUnit{ + Spec: dnsv1alpha1.ZoneUnitSpec{ + Zone: dnsv1alpha1.ZoneUnitZoneSpec{ObservedGeneration: 2}, + RecordSets: []dnsv1alpha1.ZoneUnitRecordSetSpec{zoneUnitRecordSetItem(recordSet, providercontract.Payload{})}, + }, + Status: dnsv1alpha1.ZoneUnitStatus{ + Zone: &dnsv1alpha1.ZoneUnitZoneStatus{Conditions: conditions}, + RecordSets: []dnsv1alpha1.ZoneUnitRecordSetStatus{{ + RecordSetNamespace: recordSet.Namespace, + RecordSetName: recordSet.Name, + RecordSetUID: recordSet.UID, + ObservedGeneration: 2, + Conditions: conditions, + }}, + }, + } + k8sClient := fake.NewClientBuilder(). + WithScheme(testScheme(t)). + WithObjects(zone, recordSet). + WithStatusSubresource(&dnsv1alpha1.Zone{}, &dnsv1alpha1.RecordSet{}). + Build() + reconciler := &ZoneUnitCompositionReconciler{Client: k8sClient} + if err := reconciler.projectZoneUnitStatus(ctx, zone, nil, []dnsv1alpha1.RecordSet{*recordSet}, unit); err != nil { + t.Fatalf("projectZoneUnitStatus: %v", err) + } + if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(zone), zone); err != nil { + t.Fatalf("get Zone: %v", err) + } + if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(recordSet), recordSet); err != nil { + t.Fatalf("get RecordSet: %v", err) + } + for _, projected := range [][]metav1.Condition{zone.Status.Conditions, recordSet.Status.Conditions} { + accepted := assertCondition(t, projected, "Accepted", test.accepted, acceptedReason) + if accepted.Message != test.message { + t.Fatalf("Accepted message = %q, want %q", accepted.Message, test.message) + } + programmedStatus, reason, message := test.programmed, programmedReason, test.message + if test.accepted != metav1.ConditionTrue { + programmedStatus, reason, message = metav1.ConditionUnknown, "Reconciling", "Reconciling" + } + programmed := assertCondition(t, projected, "Programmed", programmedStatus, reason) + if programmed.Message != message { + t.Fatalf("Programmed message = %q, want %q", programmed.Message, message) + } + } + }) + } +} + func assertConditionAtGeneration(t *testing.T, conditions []metav1.Condition, conditionType string, status metav1.ConditionStatus, reason string, generation int64) { t.Helper() condition := assertCondition(t, conditions, conditionType, status, reason) diff --git a/internal/go/core/webhook/validator.go b/internal/go/core/webhook/validator.go index 2beae9e..0fea100 100644 --- a/internal/go/core/webhook/validator.go +++ b/internal/go/core/webhook/validator.go @@ -337,13 +337,15 @@ func (v *CoreValidator) validateZoneClass(ctx context.Context, zoneClass *dnsv1a if err != nil { return err } - if strings.TrimSpace(zoneClass.Spec.IdentityRef.Name) == "" { - return errors.New("spec.identityRef.name is required") - } if providerVersion.Identity != nil { + if zoneClass.Spec.IdentityRef == nil || strings.TrimSpace(zoneClass.Spec.IdentityRef.Name) == "" { + return errors.New("spec.identityRef.name is required") + } if err := validateProviderIdentityResource(providerVersion.Identity.Resource, "Provider.identity.resource"); err != nil { return err } + } else if zoneClass.Spec.IdentityRef != nil { + return errors.New("spec.identityRef is forbidden when the Provider version does not declare identity") } if err := validateNamespacePolicy(zoneClass.Spec.AllowedZones.Namespaces, "spec.allowedZones.namespaces"); err != nil { return err diff --git a/internal/go/core/webhook/validator_test.go b/internal/go/core/webhook/validator_test.go index 4f2ebe5..d422d74 100644 --- a/internal/go/core/webhook/validator_test.go +++ b/internal/go/core/webhook/validator_test.go @@ -1437,6 +1437,47 @@ func TestCloudflareProviderIDSchemaValidation(t *testing.T) { }) } +func TestZoneClassValidationRequiresIdentityOnlyWhenDeclared(t *testing.T) { + for _, test := range []struct { + name string + hasIdentity bool + identityRef *dnsv1alpha1.LocalObjectReference + wantErr string + }{ + {name: "route53 missing", hasIdentity: true, wantErr: "spec.identityRef.name is required"}, + {name: "route53 empty", hasIdentity: true, identityRef: &dnsv1alpha1.LocalObjectReference{}, wantErr: "spec.identityRef.name is required"}, + {name: "route53 blank", hasIdentity: true, identityRef: &dnsv1alpha1.LocalObjectReference{Name: " "}, wantErr: "spec.identityRef.name is required"}, + {name: "route53 present", hasIdentity: true, identityRef: &dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}}, + {name: "credentialless absent"}, + {name: "credentialless empty", identityRef: &dnsv1alpha1.LocalObjectReference{}, wantErr: "spec.identityRef is forbidden when the Provider version does not declare identity"}, + {name: "credentialless present", identityRef: &dnsv1alpha1.LocalObjectReference{Name: "sentinel"}, wantErr: "spec.identityRef is forbidden when the Provider version does not declare identity"}, + } { + t.Run(test.name, func(t *testing.T) { + provider := route53Provider(t) + zoneClass := route53ZoneClass(t) + if !test.hasIdentity { + // Select a credentialless served version beside the identity-backed version. + version := *provider.Spec.Versions[0].DeepCopy() + version.Name = "v1credentialless" + version.Storage = false + version.Identity = nil + provider.Spec.Versions = append(provider.Spec.Versions, version) + zoneClass.Spec.Provider.Version = version.Name + } + zoneClass.Spec.IdentityRef = test.identityRef + validator := newTestValidator(t, provider) + err := validator.validateZoneClass(context.Background(), zoneClass) + if test.wantErr == "" { + if err != nil { + t.Fatalf("validateZoneClass returned error: %v", err) + } + } else if err == nil || err.Error() != test.wantErr { + t.Fatalf("validateZoneClass error = %v, want %q", err, test.wantErr) + } + }) + } +} + func TestZoneClassValidationAppliesExtensionCELRules(t *testing.T) { ctx := context.Background() extension := route53Provider(t) @@ -2136,7 +2177,7 @@ func route53ZoneClass(t *testing.T) *dnsv1alpha1.ZoneClass { Spec: dnsv1alpha1.ZoneClassSpec{ Provider: route53v1alpha1.ProviderRef, ControllerName: "route53.dns.appthrust.io/controller", - IdentityRef: dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, + IdentityRef: &dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, Parameters: raw(t, map[string]any{}), AllowedZones: dnsv1alpha1.ZoneClassAllowedZones{ Namespaces: dnsv1alpha1.NamespacePolicy{From: dnsv1alpha1.NamespacesFromSame}, diff --git a/internal/go/providers/cloudflare/controllers/zoneclass/controller.go b/internal/go/providers/cloudflare/controllers/zoneclass/controller.go index 2b76f74..28e5aa3 100644 --- a/internal/go/providers/cloudflare/controllers/zoneclass/controller.go +++ b/internal/go/providers/cloudflare/controllers/zoneclass/controller.go @@ -80,7 +80,7 @@ func (r *ZoneClassReconciler) acceptZoneClass(ctx context.Context, zoneClass *dn if zoneDeletionPolicy(params) != cloudflarev1alpha1.ZoneDeletionPolicyDelete && zoneDeletionPolicy(params) != cloudflarev1alpha1.ZoneDeletionPolicyRetain { return metav1.ConditionFalse, "InvalidParameters", "zoneDeletionPolicy must be Delete or Retain", nil } - if zoneClass.Spec.IdentityRef.Name == "" { + if zoneClass.Spec.IdentityRef == nil || zoneClass.Spec.IdentityRef.Name == "" { return metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef.name is required", nil } @@ -147,7 +147,7 @@ func (r *ZoneClassReconciler) mapIdentityToZoneClasses(ctx context.Context, obj if err != nil || zoneClass.Spec.ControllerName != r.controllerName() || provider == nil || version == nil || provider.Name != r.providerReference().Name { continue } - if zoneClass.Spec.IdentityRef.Name == identity.Name { + if zoneClass.Spec.IdentityRef != nil && zoneClass.Spec.IdentityRef.Name == identity.Name { requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(&zoneClass)}) } } diff --git a/internal/go/providers/cloudflare/controllers/zoneunit/recordset.go b/internal/go/providers/cloudflare/controllers/zoneunit/recordset.go index 71dd6bf..0bdb9a9 100644 --- a/internal/go/providers/cloudflare/controllers/zoneunit/recordset.go +++ b/internal/go/providers/cloudflare/controllers/zoneunit/recordset.go @@ -627,7 +627,7 @@ func cloudflareRecordValueKey(record CloudflareDNSRecord) string { } func (r *recordSetReconciler) identityForZoneClass(ctx context.Context, recordSet *dnsv1alpha1.RecordSet, zoneClass *dnsv1alpha1.ZoneClass) (*cloudflarev1alpha1.CloudflareIdentity, bool, error) { - if zoneClass.Spec.IdentityRef.Name == "" { + if zoneClass.Spec.IdentityRef == nil || zoneClass.Spec.IdentityRef.Name == "" { return nil, false, r.setAccepted(ctx, recordSet, metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef.name is required") } var identity cloudflarev1alpha1.CloudflareIdentity diff --git a/internal/go/providers/cloudflare/controllers/zoneunit/zone.go b/internal/go/providers/cloudflare/controllers/zoneunit/zone.go index 6ed3e8a..7dbc4c5 100644 --- a/internal/go/providers/cloudflare/controllers/zoneunit/zone.go +++ b/internal/go/providers/cloudflare/controllers/zoneunit/zone.go @@ -249,7 +249,7 @@ func (r *ZoneReconciler) acceptZone(ctx context.Context, zone *dnsv1alpha1.Zone) } func (r *ZoneReconciler) acceptIdentityRef(ctx context.Context, zone *dnsv1alpha1.Zone, zoneClass *dnsv1alpha1.ZoneClass) (*cloudflarev1alpha1.CloudflareIdentity, bool, error) { - if zoneClass.Spec.IdentityRef.Name == "" { + if zoneClass.Spec.IdentityRef == nil || zoneClass.Spec.IdentityRef.Name == "" { return nil, false, r.setAccepted(ctx, zone, metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef.name is required") } var identity cloudflarev1alpha1.CloudflareIdentity @@ -849,7 +849,7 @@ func (r *ZoneReconciler) mapIdentityToZoneUnits(ctx context.Context, obj client. if err != nil || zoneClass.Spec.ControllerName != r.controllerName() || !cloudflareProviderVersionMatches(provider, version, r.providerReference()) { continue } - if zoneClass.Spec.IdentityRef.Name == identity.Name { + if zoneClass.Spec.IdentityRef != nil && zoneClass.Spec.IdentityRef.Name == identity.Name { requests = appendUniqueRequests(requests, r.zoneUnitRequestsForZoneClass(ctx, zoneClass.Namespace, zoneClass.Name)...) } } diff --git a/internal/go/providers/cloudflare/controllers/zoneunit/zone_test.go b/internal/go/providers/cloudflare/controllers/zoneunit/zone_test.go index 2b2dd5b..6963897 100644 --- a/internal/go/providers/cloudflare/controllers/zoneunit/zone_test.go +++ b/internal/go/providers/cloudflare/controllers/zoneunit/zone_test.go @@ -39,6 +39,29 @@ func TestCloudflareZoneClassReconcilerAcceptsRootIdentityRef(t *testing.T) { assertCloudflareCondition(t, got.Status.Conditions, string(dnsv1alpha1.ConditionAccepted), metav1.ConditionTrue, "Accepted") } +func TestCloudflareZoneClassReconcilerRejectsMissingIdentityRef(t *testing.T) { + ctx := context.Background() + zoneClass := cloudflareZoneClass("platform", "cloudflare-public") + zoneClass.Spec.IdentityRef = nil + k8sClient := fake.NewClientBuilder(). + WithScheme(cloudflareTestScheme(t)). + WithObjects(cloudflareProvider(), zoneClass). + WithStatusSubresource(&dnsv1alpha1.ZoneClass{}). + Build() + reconciler := &zoneclass.ZoneClassReconciler{Client: k8sClient} + if _, err := reconciler.Reconcile(ctx, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(zoneClass)}); err != nil { + t.Fatalf("Reconcile returned error: %v", err) + } + var got dnsv1alpha1.ZoneClass + if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(zoneClass), &got); err != nil { + t.Fatalf("get ZoneClass: %v", err) + } + assertCloudflareCondition(t, got.Status.Conditions, string(dnsv1alpha1.ConditionAccepted), metav1.ConditionFalse, "InvalidIdentityRef") + if message := got.Status.Conditions[0].Message; message != "spec.identityRef.name is required" { + t.Fatalf("Accepted message = %q", message) + } +} + func TestCloudflareZoneReconcilerIgnoresRoute53ZoneUnit(t *testing.T) { ctx := context.Background() unit := cloudflareZoneUnit("app", "apps-example-com") @@ -516,7 +539,7 @@ func cloudflareZoneClass(namespace, name string) *dnsv1alpha1.ZoneClass { Spec: dnsv1alpha1.ZoneClassSpec{ Provider: cloudflarev1alpha1.ProviderRef, ControllerName: DefaultControllerName, - IdentityRef: dnsv1alpha1.LocalObjectReference{Name: "cloudflare-ci"}, + IdentityRef: &dnsv1alpha1.LocalObjectReference{Name: "cloudflare-ci"}, Parameters: runtime.RawExtension{Raw: raw}, AllowedZones: dnsv1alpha1.ZoneClassAllowedZones{ Namespaces: dnsv1alpha1.NamespacePolicy{From: dnsv1alpha1.NamespacesFromAll}, diff --git a/internal/go/providers/route53/controllers/zoneclass/controller.go b/internal/go/providers/route53/controllers/zoneclass/controller.go index 1ff95f3..2791044 100644 --- a/internal/go/providers/route53/controllers/zoneclass/controller.go +++ b/internal/go/providers/route53/controllers/zoneclass/controller.go @@ -81,7 +81,7 @@ func (r *ZoneClassReconciler) acceptZoneClass(ctx context.Context, zoneClass *dn return metav1.ConditionFalse, "InvalidParameters", "zoneType Private requires zoneDeletionPolicy Retain", nil } } - if zoneClass.Spec.IdentityRef.Name == "" { + if zoneClass.Spec.IdentityRef == nil || zoneClass.Spec.IdentityRef.Name == "" { return metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef.name is required", nil } @@ -155,7 +155,7 @@ func (r *ZoneClassReconciler) mapIdentityToZoneClasses(ctx context.Context, obj if err != nil || zoneClass.Spec.ControllerName != r.controllerName() || provider == nil || version == nil || provider.Name != r.providerReference().Name { continue } - if zoneClass.Spec.IdentityRef.Name == identity.Name { + if zoneClass.Spec.IdentityRef != nil && zoneClass.Spec.IdentityRef.Name == identity.Name { requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(&zoneClass)}) } } diff --git a/internal/go/providers/route53/controllers/zoneclass/controller_test.go b/internal/go/providers/route53/controllers/zoneclass/controller_test.go index aa287a8..4c9b9a2 100644 --- a/internal/go/providers/route53/controllers/zoneclass/controller_test.go +++ b/internal/go/providers/route53/controllers/zoneclass/controller_test.go @@ -36,6 +36,32 @@ func TestZoneClassReconcilerAcceptsRoute53Parameters(t *testing.T) { assertCondition(t, got.Status.Conditions, string(dnsv1alpha1.ConditionAccepted), metav1.ConditionTrue, "Accepted") } +func TestZoneClassReconcilerRejectsMissingIdentityRef(t *testing.T) { + ctx := context.Background() + zoneClass := route53ZoneClass("platform", "route53-public", nil) + zoneClass.Spec.IdentityRef = nil + k8sClient := fake.NewClientBuilder(). + WithScheme(testScheme(t)). + WithObjects(route53Provider(), zoneClass). + WithStatusSubresource(&dnsv1alpha1.ZoneClass{}). + Build() + reconciler := &ZoneClassReconciler{Client: k8sClient} + if _, err := reconciler.Reconcile(ctx, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(zoneClass)}); err != nil { + t.Fatalf("Reconcile returned error: %v", err) + } + var got dnsv1alpha1.ZoneClass + if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(zoneClass), &got); err != nil { + t.Fatalf("get ZoneClass: %v", err) + } + condition := assertCondition(t, got.Status.Conditions, string(dnsv1alpha1.ConditionAccepted), metav1.ConditionFalse, "InvalidIdentityRef") + if condition.Message != "spec.identityRef.name is required" { + t.Fatalf("Accepted message = %q", condition.Message) + } + if requests := reconciler.mapIdentityToZoneClasses(ctx, acceptedRoute53Identity("platform", "route53-dev")); len(requests) != 0 { + t.Fatalf("identity watch mapped credentialless class: %#v", requests) + } +} + func TestZoneClassReconcilerValidatesPrivateZonePolicies(t *testing.T) { for _, test := range []struct { name string diff --git a/internal/go/providers/route53/controllers/zoneclass/test_helpers_test.go b/internal/go/providers/route53/controllers/zoneclass/test_helpers_test.go index 33dd110..ea375ad 100644 --- a/internal/go/providers/route53/controllers/zoneclass/test_helpers_test.go +++ b/internal/go/providers/route53/controllers/zoneclass/test_helpers_test.go @@ -37,7 +37,7 @@ func route53ZoneClass(namespace, name string, tags map[string]string) *dnsv1alph Spec: dnsv1alpha1.ZoneClassSpec{ ControllerName: DefaultControllerName, Provider: route53v1alpha1.ProviderRef, - IdentityRef: dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, + IdentityRef: &dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, Parameters: runtime.RawExtension{Raw: raw}, }, } diff --git a/internal/go/providers/route53/controllers/zoneunit/zone.go b/internal/go/providers/route53/controllers/zoneunit/zone.go index f5d4f3d..37e0567 100644 --- a/internal/go/providers/route53/controllers/zoneunit/zone.go +++ b/internal/go/providers/route53/controllers/zoneunit/zone.go @@ -271,7 +271,7 @@ func (r *ZoneReconciler) mapIdentityToZoneUnits(ctx context.Context, obj client. if err != nil || zoneClass.Spec.ControllerName != r.controllerName() || !route53ProviderVersionMatches(provider, version, r.providerReference()) { continue } - if zoneClass.Spec.IdentityRef.Name == identity.Name { + if zoneClass.Spec.IdentityRef != nil && zoneClass.Spec.IdentityRef.Name == identity.Name { requests = appendUniqueRequests(requests, r.zoneUnitRequestsForZoneClass(ctx, zoneClass.Namespace, zoneClass.Name)...) } } @@ -380,7 +380,7 @@ func (r *ZoneReconciler) acceptZone(ctx context.Context, zone *dnsv1alpha1.Zone) } func (r *ZoneReconciler) acceptIdentityRef(ctx context.Context, zone *dnsv1alpha1.Zone, zoneClass *dnsv1alpha1.ZoneClass) (*route53v1alpha1.Route53Identity, bool, error) { - if zoneClass.Spec.IdentityRef.Name == "" { + if zoneClass.Spec.IdentityRef == nil || zoneClass.Spec.IdentityRef.Name == "" { return nil, false, r.setAccepted(ctx, zone, metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef.name is required") } diff --git a/internal/go/providers/route53/controllers/zoneunit/zone_test_fixtures_test.go b/internal/go/providers/route53/controllers/zoneunit/zone_test_fixtures_test.go index 5cf9dd1..ab3d361 100644 --- a/internal/go/providers/route53/controllers/zoneunit/zone_test_fixtures_test.go +++ b/internal/go/providers/route53/controllers/zoneunit/zone_test_fixtures_test.go @@ -67,7 +67,7 @@ func route53ZoneClass(namespace, name string, tags map[string]string) *dnsv1alph Spec: dnsv1alpha1.ZoneClassSpec{ Provider: route53v1alpha1.ProviderRef, ControllerName: DefaultControllerName, - IdentityRef: dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, + IdentityRef: &dnsv1alpha1.LocalObjectReference{Name: "route53-dev"}, Parameters: runtime.RawExtension{Raw: raw}, AllowedZones: dnsv1alpha1.ZoneClassAllowedZones{ Namespaces: dnsv1alpha1.NamespacePolicy{From: dnsv1alpha1.NamespacesFromSame}, diff --git a/pkg/go/api/dns/v1alpha1/scheme_test.go b/pkg/go/api/dns/v1alpha1/scheme_test.go index 7190c97..21c8cef 100644 --- a/pkg/go/api/dns/v1alpha1/scheme_test.go +++ b/pkg/go/api/dns/v1alpha1/scheme_test.go @@ -1,6 +1,7 @@ package v1alpha1_test import ( + "encoding/json" "testing" dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" @@ -38,3 +39,27 @@ func TestAddToSchemeRegistersCoreTypes(t *testing.T) { } } } + +func TestZoneClassOptionalIdentityWireAndDeepCopy(t *testing.T) { + zoneClass := &dnsv1alpha1.ZoneClass{} + payload, err := json.Marshal(zoneClass.Spec) + if err != nil { + t.Fatalf("marshal ZoneClass spec: %v", err) + } + var fields map[string]json.RawMessage + if err := json.Unmarshal(payload, &fields); err != nil { + t.Fatalf("decode ZoneClass spec: %v", err) + } + if _, present := fields["identityRef"]; present { + t.Fatalf("credentialless ZoneClass serialized identityRef: %s", payload) + } + if copied := zoneClass.DeepCopy(); copied.Spec.IdentityRef != nil { + t.Fatalf("DeepCopy manufactured identityRef: %#v", copied.Spec.IdentityRef) + } + zoneClass.Spec.IdentityRef = &dnsv1alpha1.LocalObjectReference{Name: "original"} + copied := zoneClass.DeepCopy() + copied.Spec.IdentityRef.Name = "changed" + if zoneClass.Spec.IdentityRef.Name != "original" { + t.Fatal("DeepCopy aliased the original identityRef") + } +} diff --git a/pkg/go/api/dns/v1alpha1/zoneclass_types.go b/pkg/go/api/dns/v1alpha1/zoneclass_types.go index 38bdddc..781dd8d 100644 --- a/pkg/go/api/dns/v1alpha1/zoneclass_types.go +++ b/pkg/go/api/dns/v1alpha1/zoneclass_types.go @@ -30,7 +30,9 @@ type ZoneClassSpec struct { ControllerName string `json:"controllerName"` // IdentityRef selects the provider identity resource in the ZoneClass namespace. - IdentityRef LocalObjectReference `json:"identityRef"` + // Required when the selected Provider version declares identity; forbidden otherwise. + // +optional + IdentityRef *LocalObjectReference `json:"identityRef,omitempty"` // Parameters contains provider-specific ZoneClass settings. // +kubebuilder:pruning:PreserveUnknownFields diff --git a/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go b/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go index 9c7ad64..b1164db 100644 --- a/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go +++ b/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go @@ -1154,7 +1154,11 @@ func (in *ZoneClassReference) DeepCopy() *ZoneClassReference { func (in *ZoneClassSpec) DeepCopyInto(out *ZoneClassSpec) { *out = *in out.Provider = in.Provider - out.IdentityRef = in.IdentityRef + if in.IdentityRef != nil { + in, out := &in.IdentityRef, &out.IdentityRef + *out = new(LocalObjectReference) + **out = **in + } in.Parameters.DeepCopyInto(&out.Parameters) in.AllowedZones.DeepCopyInto(&out.AllowedZones) } diff --git a/pkg/ts/ui/src/views/platform/PlatformZoneClassDetailPage.tsx b/pkg/ts/ui/src/views/platform/PlatformZoneClassDetailPage.tsx index fd52533..9433032 100644 --- a/pkg/ts/ui/src/views/platform/PlatformZoneClassDetailPage.tsx +++ b/pkg/ts/ui/src/views/platform/PlatformZoneClassDetailPage.tsx @@ -86,7 +86,13 @@ export function PlatformZoneClassDetailPage() { ); } - const identityName = zoneClassIdentityName(selectedZoneClass); + const providerVersion = data.providers.items + .find(item => nameOf(item) === selectedZoneClass.spec.provider.name) + ?.spec.versions.find(item => item.name === selectedZoneClass.spec.provider.version); + const requiresIdentity = providerVersion + ? Boolean(providerVersion.identity) + : Boolean(selectedZoneClass.spec.identityRef); + const identityName = requiresIdentity ? zoneClassIdentityName(selectedZoneClass) : ''; const identity = identityName ? findIdentity(data.identities.items, namespaceOf(selectedZoneClass), identityName) : undefined; @@ -133,6 +139,7 @@ export function PlatformZoneClassDetailPage() { health={health} identity={identity} identityName={identityName} + requiresIdentity={requiresIdentity} provider={provider} usedBy={referencingZones.length} zoneClass={selectedZoneClass} @@ -162,6 +169,7 @@ export function PlatformZoneClassDetailPage() { zoneClass={selectedZoneClass} identity={identity} identityName={identityName} + requiresIdentity={requiresIdentity} events={events} eventsError={data.events.error} onOpenIdentity={ @@ -304,6 +312,7 @@ function ZoneClassVitalsStrip({ health, identity, identityName, + requiresIdentity, provider, usedBy, zoneClass, @@ -313,6 +322,7 @@ function ZoneClassVitalsStrip({ health: Health; identity?: ProviderIdentity; identityName?: string; + requiresIdentity: boolean; provider: string; usedBy: number; zoneClass: ZoneClass; @@ -364,6 +374,7 @@ function ZoneClassVitalsStrip({ {`${zoneClass.spec.provider.name}/${zoneClass.spec.provider.version}`} + {requiresIdentity ? ( {identity ? ( @@ -383,6 +394,7 @@ function ZoneClassVitalsStrip({ )} + ) : null} @@ -500,6 +512,7 @@ function StatusTab({ zoneClass, identity, identityName, + requiresIdentity, events, eventsError, onOpenIdentity, @@ -507,6 +520,7 @@ function StatusTab({ zoneClass: ZoneClass; identity?: ProviderIdentity; identityName?: string; + requiresIdentity: boolean; events: KubeEvent[]; eventsError: unknown; onOpenIdentity?: () => void; @@ -524,8 +538,9 @@ function StatusTab({ label="ZoneClass" health={zoneClassHealth} reason={conditionSummary(zoneClass.status?.conditions)} - divided + divided={requiresIdentity} /> + {requiresIdentity ? ( + ) : null} diff --git a/pkg/ts/ui/src/views/platform/PlatformZoneClassIdentitySelectPage.tsx b/pkg/ts/ui/src/views/platform/PlatformZoneClassIdentitySelectPage.tsx index f384345..4ba45e0 100644 --- a/pkg/ts/ui/src/views/platform/PlatformZoneClassIdentitySelectPage.tsx +++ b/pkg/ts/ui/src/views/platform/PlatformZoneClassIdentitySelectPage.tsx @@ -1,6 +1,8 @@ /** @jsxRuntime classic */ import React from 'react'; import { useDnsData } from '../../api/dns'; +import type { Provider } from '../../types/resources'; +import { platformNamespaces } from './resources'; import { PLATFORM_ZONE_CLASSES_PATH, PlatformRouteFrame, @@ -8,16 +10,33 @@ import { ZONE_CLASSES_BREADCRUMB, zoneClassNewPath, } from './routes'; -import { ZoneClassIdentitySelectPage } from './ZoneClassForm'; +import { ZoneClassFormPage, ZoneClassIdentitySelectPage } from './ZoneClassForm'; export function PlatformZoneClassIdentitySelectPage() { const navigation = usePlatformNavigation(); const data = useDnsData(); + const [provider, setProvider] = React.useState(); + if (provider) { + return ( + + setProvider(undefined)} + onSaved={() => navigation.push(PLATFORM_ZONE_CLASSES_PATH)} + /> + + ); + } return ( navigation.push(PLATFORM_ZONE_CLASSES_PATH)} onSelectIdentity={identity => navigation.push(zoneClassNewPath(identity))} /> diff --git a/pkg/ts/ui/src/views/platform/PlatformZoneClassesPage.tsx b/pkg/ts/ui/src/views/platform/PlatformZoneClassesPage.tsx index 949b383..a6cb060 100644 --- a/pkg/ts/ui/src/views/platform/PlatformZoneClassesPage.tsx +++ b/pkg/ts/ui/src/views/platform/PlatformZoneClassesPage.tsx @@ -31,7 +31,10 @@ export function PlatformZoneClassesPage() { const navigation = usePlatformNavigation(); const data = useDnsData(); const { canCreateZoneClass } = useZoneClassAccess(); - const hasIdentities = data.identities.items.length > 0; + const hasProviderSelection = data.identities.items.length > 0 || data.providers.items.some(provider => { + const version = provider.spec.versions.find(item => item.served); + return version && !version.identity; + }); return ( @@ -39,7 +42,7 @@ export function PlatformZoneClassesPage() { title="Zone Classes" description="ZoneClasses publish provider-backed DNS zone capability to selected namespaces." action={ - hasIdentities ? ( + hasProviderSelection ? ( - {!hasIdentities ? ( + {!hasProviderSelection ? ( [0] { + return { + namespace: 'platform', + name: 'customer-dns', + description: '', + provider: `fixture.dns.example/${version}`, + controllerName: 'fixture.dns.example/controller', + identityName: 'stale-selection', + allowedFrom: 'Same', + allowedLabelsText: '', + tags: '', + parametersJSON: '{"resolver":{"mode":"Recursive","servers":["192.0.2.53:53"]},"expectedNameservers":["ns.example.net"]}', + zoneType: 'Public', + zoneCreationPolicy: 'Create', + zoneDeletionPolicy: 'Retain', + sameNameZonePolicy: 'Deny', + }; +} + +describe('ZoneClass provider identity contract', () => { + it('omits identity and writer parameters for the selected credentialless version', () => { + const manifest = buildZoneClassManifest(form('v1credentialless'), provider); + expect(manifest.spec.provider.version).toBe('v1credentialless'); + expect('identityRef' in manifest.spec).toBe(false); + expect(manifest.spec.parameters).toEqual({ + resolver: { mode: 'Recursive', servers: ['192.0.2.53:53'] }, + expectedNameservers: ['ns.example.net'], + }); + }); + + it('retains identity and writer parameters for an identity-backed version', () => { + const manifest = buildZoneClassManifest(form('v1identity'), provider); + expect(manifest.spec.identityRef).toEqual({ name: 'stale-selection' }); + expect(manifest.spec.parameters).toEqual({ zoneCreationPolicy: 'Create', zoneDeletionPolicy: 'Retain' }); + }); + + it('does not guess credentialless operation when the Provider version is unavailable', () => { + expect(() => buildZoneClassManifest(form('v1missing'), provider)).toThrow('Provider version is required'); + }); + + it('rejects non-object credentialless parameters', () => { + expect(() => buildZoneClassManifest({ ...form('v1credentialless'), parametersJSON: '[]' }, provider)).toThrow(); + }); +}); diff --git a/pkg/ts/ui/src/views/platform/ZoneClassForm.tsx b/pkg/ts/ui/src/views/platform/ZoneClassForm.tsx index 58d3d6d..f29b0ba 100644 --- a/pkg/ts/ui/src/views/platform/ZoneClassForm.tsx +++ b/pkg/ts/ui/src/views/platform/ZoneClassForm.tsx @@ -9,8 +9,8 @@ import { Stack } from '../../components/primitives'; import { Typography } from '../../components/primitives'; import React from 'react'; import { z } from 'zod'; -import { createZoneClass, updateZoneClass } from '../../api/dns'; -import { nameOf, namespaceOf, resourceKey, zoneClassIdentityName } from '../../resources'; +import { createZoneClass, updateZoneClass, useAccess } from '../../api/dns'; +import { nameOf, namespaceOf, resourceKey, zoneClassIdentityName, ZoneClassResource } from '../../resources'; import type { Provider, ProviderIdentity, ZoneClass } from '../../types/resources'; import { type BreadcrumbItem, @@ -23,6 +23,7 @@ import { IdentitySummaryCard, namespaceLabelsYaml, parseNamespaceLabelsYaml, + readJSON, ToolbarButton, ui, useDnsFormState, @@ -41,6 +42,7 @@ type ZoneClassForm = { allowedFrom: 'Same' | 'Selector' | 'All'; allowedLabelsText: string; tags: string; + parametersJSON: string; zoneType: 'Public' | 'Private'; zoneCreationPolicy: 'Create' | 'Deny'; zoneDeletionPolicy: 'Delete' | 'Retain'; @@ -62,6 +64,7 @@ const zoneClassFormSchema = z.object({ allowedFrom: z.enum(['Same', 'Selector', 'All']), allowedLabelsText: z.string(), tags: z.string(), + parametersJSON: z.string(), zoneType: z.enum(['Public', 'Private']), zoneCreationPolicy: z.enum(['Create', 'Deny']), zoneDeletionPolicy: z.enum(['Delete', 'Retain']), @@ -116,11 +119,13 @@ function zoneClassParametersFromForm( function buildZoneClassDesiredManifest({ form: rawForm, provider, + providerDefinition, baseParameters, validateSelector = true, }: { form: ZoneClassForm; provider: string; + providerDefinition?: Provider; baseParameters?: ZoneClass['spec']['parameters']; validateSelector?: boolean; }): ZoneClass { @@ -132,6 +137,12 @@ function buildZoneClassDesiredManifest({ throw new Error('Controller name is required'); } const providerObject = providerRefObject(provider); + const version = nameOf(providerDefinition) === providerObject.name + ? providerDefinition?.spec.versions.find(item => item.name === providerObject.version) + : undefined; + if (!version) { + throw new Error('Provider version is required'); + } return { apiVersion: 'dns.appthrust.io/v1alpha1', kind: 'ZoneClass', @@ -139,27 +150,31 @@ function buildZoneClassDesiredManifest({ spec: { provider: providerObject, controllerName: form.controllerName, - identityRef: { name: form.identityName }, + ...(version.identity ? { identityRef: { name: form.identityName } } : {}), allowedZones: allowedZonesFromForm(form, { validateSelector }), - parameters: zoneClassParametersFromForm(form, baseParameters), + parameters: version.identity + ? zoneClassParametersFromForm(form, baseParameters) + : z.record(z.string(), z.unknown()).parse(readJSON(form.parametersJSON, {}, 'Provider parameters')), }, }; } -function buildZoneClassManifest( +export function buildZoneClassManifest( form: ZoneClassForm, provider?: Provider ): ZoneClass { return buildZoneClassDesiredManifest({ form, - provider: providerRef(provider), + provider: form.provider, + providerDefinition: provider, }); } -function buildUpdatedZoneClassManifest(zoneClass: ZoneClass, form: ZoneClassForm): ZoneClass { +function buildUpdatedZoneClassManifest(zoneClass: ZoneClass, form: ZoneClassForm, provider?: Provider): ZoneClass { const desiredManifest = buildZoneClassDesiredManifest({ form, provider: providerRefString(zoneClass.spec.provider), + providerDefinition: provider, baseParameters: zoneClass.spec.parameters, }); const annotations = { ...(zoneClass.metadata?.annotations ?? {}) }; @@ -255,7 +270,8 @@ function zoneClassPreview( try { return buildZoneClassDesiredManifest({ form, - provider: providerRef(provider), + provider: form.provider, + providerDefinition: provider, validateSelector: false, }); } catch (error) { @@ -281,6 +297,7 @@ function zoneClassFormFromResource(zoneClass: ZoneClass): ZoneClassForm { allowedFrom, allowedLabelsText: namespaceLabelsYaml(selector?.matchLabels), tags: namespaceLabelsYaml(zoneClass.spec.parameters.tags), + parametersJSON: JSON.stringify(zoneClass.spec.parameters, null, 2), zoneType: zoneClass.spec.parameters.zoneType ?? 'Public', zoneCreationPolicy: zoneClass.spec.parameters.zoneCreationPolicy ?? 'Create', zoneDeletionPolicy: zoneClass.spec.parameters.zoneDeletionPolicy ?? 'Retain', @@ -290,12 +307,14 @@ function zoneClassFormFromResource(zoneClass: ZoneClass): ZoneClassForm { function updatedZoneClassPreview( zoneClass: ZoneClass, - form: ZoneClassForm + form: ZoneClassForm, + provider?: Provider ): ZoneClass | { error: string } { try { return buildZoneClassDesiredManifest({ form, provider: providerRefString(zoneClass.spec.provider), + providerDefinition: provider, baseParameters: zoneClass.spec.parameters, validateSelector: false, }); @@ -306,25 +325,43 @@ function updatedZoneClassPreview( export function ZoneClassIdentitySelectPage({ identities, + providers, + onSelectProvider, onCancel, onSelectIdentity, breadcrumb, }: { identities: ProviderIdentity[]; + providers: Provider[]; + onSelectProvider: (provider: Provider) => void; onCancel: () => void; onSelectIdentity: (identity: ProviderIdentity) => void; breadcrumb?: BreadcrumbItem[]; }) { + const credentiallessProviders = providers.filter(provider => { + const version = provider.spec.versions.find(item => item.served); + return version && !version.identity; + }); return ( } /> + {credentiallessProviders.map(provider => ( + + ))} {identities.length ? ( {identities.map(identity => ( @@ -346,7 +383,7 @@ export function ZoneClassIdentitySelectPage({ ))} - ) : ( + ) : credentiallessProviders.length ? null : ( void; onSaved: () => void; breadcrumb?: BreadcrumbItem[]; }) { const { showSuccess, showError, snackbar } = useNotice(); const isEdit = Boolean(zoneClass); - const selectedIdentityProvider = providerForIdentity(providers, identity); + const selectedIdentityProvider = provider ?? providerForIdentity(providers, identity); const initialForm = React.useMemo( () => zoneClass @@ -418,6 +459,7 @@ export function ZoneClassFormPage({ allowedFrom: 'Selector', allowedLabelsText: '', tags: '', + parametersJSON: '{}', zoneType: 'Public', zoneCreationPolicy: 'Create', zoneDeletionPolicy: 'Delete', @@ -428,14 +470,19 @@ export function ZoneClassFormPage({ const [form, setForm] = useDnsFormState(initialForm); const [errors, setErrors] = React.useState({}); - const selectedProvider = - providers.find( - provider => providerRef(provider) === form.provider - ) ?? selectedIdentityProvider; const providerObject = providerRefObject(form.provider); + const selectedProvider = providers.find(item => nameOf(item) === providerObject.name); + const selectedVersion = selectedProvider?.spec.versions.find(item => item.name === providerObject.version); + const requiresIdentity = Boolean(selectedVersion?.identity); + const canSave = useAccess(ZoneClassResource, isEdit ? 'update' : 'create', { + group: 'dns.appthrust.io', + version: 'v1alpha1', + resource: 'zoneclasses', + namespace: form.namespace, + }); const isPrivateZone = isRoute53ProviderRef(form.provider) && form.zoneType === 'Private'; const preview = zoneClass - ? updatedZoneClassPreview(zoneClass, form) + ? updatedZoneClassPreview(zoneClass, form, selectedProvider) : selectedProvider ? zoneClassPreview(form, selectedProvider) : { error: 'Provider is required' }; @@ -458,7 +505,7 @@ export function ZoneClassFormPage({ setErrors({}); try { if (zoneClass) { - await updateZoneClass(buildUpdatedZoneClassManifest(zoneClass, form)); + await updateZoneClass(buildUpdatedZoneClassManifest(zoneClass, form, selectedProvider)); showSuccess('ZoneClass updated'); } else { await createZoneClass(buildZoneClassManifest(form, selectedProvider)); @@ -485,16 +532,17 @@ export function ZoneClassFormPage({ } /> } > - {identity ? ( + {requiresIdentity ? identity ? ( ) : ( @@ -502,13 +550,28 @@ export function ZoneClassFormPage({ ? `Provider Identity ${form.namespace}/${form.identityName} is not visible.` : 'No Provider Identity selected.'} - )} + ) : null} Namespace - - - The ZoneClass is created in the same namespace as the selected Provider Identity. - + {!requiresIdentity && !isEdit ? ( + + + + ) : ( + + )} + {requiresIdentity ? ( + + The ZoneClass is created in the same namespace as the selected Provider Identity. + + ) : null} {isEdit ? ( @@ -546,6 +609,18 @@ export function ZoneClassFormPage({ Provider controller instance that reconciles ZoneUnits for this ZoneClass. + {!requiresIdentity ? ( + setForm({ ...form, parametersJSON: event.target.value })} + multiline + minRows={6} + fullWidth + /> + ) : ( + <> {isRoute53ProviderRef(form.provider) ? ( Zone type @@ -650,6 +725,8 @@ export function ZoneClassFormPage({ fullWidth /> ) : null} + + )} Date: Sun, 18 Oct 2026 11:01:01 +0900 Subject: [PATCH 03/16] feat(external): verify-only DNS provider Provider external.dns.appthrust.io/v1alpha1 (A/AAAA/CNAME, no identity), ZoneClass resolver policy and expectedNameservers, exact-set A/AAAA verification with bounded CNAME chains and wildcard witnesses, Cluster/Recursive views plus RD=0 authority checks, bounded scheduling (30-300s, jitter, backoff, 32 names/pass, 40 QPS/80 burst), UID/generation/policy-bound receipts, bookkeeping-only deletion, endpoint conversion handler, metrics. Design: docs/design/external-provider.md. --- docs/design/provider-extension.md | 17 +- docs/design/test-policy.md | 4 + go.mod | 7 +- go.sum | 2 + .../go/providers/external/config/config.go | 105 +++++ .../controllers/zoneclass/controller.go | 111 +++++ .../controllers/zoneclass/controller_test.go | 98 +++++ .../controllers/zoneunit/controller.go | 384 ++++++++++++++++ .../controllers/zoneunit/controller_test.go | 414 ++++++++++++++++++ .../external/controllers/zoneunit/model.go | 173 ++++++++ .../external/controllers/zoneunit/status.go | 156 +++++++ .../external/controllers/zoneunit/watches.go | 74 ++++ .../providers/external/conversion/handler.go | 141 ++++++ .../external/conversion/handler_test.go | 91 ++++ .../external/manifest/capability.yaml | 12 + .../providers/external/manifest/manifest.go | 23 + .../external/manifest/manifest_test.go | 89 ++++ .../providers/external/manifest/provider.yaml | 159 +++++++ .../go/providers/external/probe/evaluate.go | 300 +++++++++++++ .../providers/external/probe/evaluate_test.go | 118 +++++ .../providers/external/probe/limits_test.go | 163 +++++++ .../go/providers/external/probe/metrics.go | 47 ++ internal/go/providers/external/probe/types.go | 114 +++++ internal/go/providers/external/probe/wire.go | 388 ++++++++++++++++ .../go/providers/external/probe/wire_test.go | 158 +++++++ pkg/go/api/external/v1alpha1/doc.go | 4 + pkg/go/api/external/v1alpha1/register.go | 19 + pkg/go/api/external/v1alpha1/types.go | 53 +++ pkg/go/api/external/v1alpha1/types_test.go | 26 ++ .../v1alpha1/zz_generated.deepcopy.go | 110 +++++ 30 files changed, 3556 insertions(+), 4 deletions(-) create mode 100644 internal/go/providers/external/config/config.go create mode 100644 internal/go/providers/external/controllers/zoneclass/controller.go create mode 100644 internal/go/providers/external/controllers/zoneclass/controller_test.go create mode 100644 internal/go/providers/external/controllers/zoneunit/controller.go create mode 100644 internal/go/providers/external/controllers/zoneunit/controller_test.go create mode 100644 internal/go/providers/external/controllers/zoneunit/model.go create mode 100644 internal/go/providers/external/controllers/zoneunit/status.go create mode 100644 internal/go/providers/external/controllers/zoneunit/watches.go create mode 100644 internal/go/providers/external/conversion/handler.go create mode 100644 internal/go/providers/external/conversion/handler_test.go create mode 100644 internal/go/providers/external/manifest/capability.yaml create mode 100644 internal/go/providers/external/manifest/manifest.go create mode 100644 internal/go/providers/external/manifest/manifest_test.go create mode 100644 internal/go/providers/external/manifest/provider.yaml create mode 100644 internal/go/providers/external/probe/evaluate.go create mode 100644 internal/go/providers/external/probe/evaluate_test.go create mode 100644 internal/go/providers/external/probe/limits_test.go create mode 100644 internal/go/providers/external/probe/metrics.go create mode 100644 internal/go/providers/external/probe/types.go create mode 100644 internal/go/providers/external/probe/wire.go create mode 100644 internal/go/providers/external/probe/wire_test.go create mode 100644 pkg/go/api/external/v1alpha1/doc.go create mode 100644 pkg/go/api/external/v1alpha1/register.go create mode 100644 pkg/go/api/external/v1alpha1/types.go create mode 100644 pkg/go/api/external/v1alpha1/types_test.go create mode 100644 pkg/go/api/external/v1alpha1/zz_generated.deepcopy.go diff --git a/docs/design/provider-extension.md b/docs/design/provider-extension.md index 976eb63..80fc320 100644 --- a/docs/design/provider-extension.md +++ b/docs/design/provider-extension.md @@ -200,13 +200,26 @@ Provider runtime interfaces are local to each provider controller package. A pro - Route 53 uses a hosted zone and record-set change-batch model. Its runtime interface may combine hosted zone operations, change polling, tagging, record-set listing, and record-set batch changes. - Cloudflare uses separate zone and DNS record APIs. Its runtime interfaces may separate zone operations from DNS record operations. -Provider factories are also provider-specific. They resolve provider identity resources, credentials, SDK clients, and provider-specific readiness requirements before returning the local runtime interface. The core API only requires that `ZoneClass.spec.identityRef.name` points to the identity kind declared by the selected Provider version; the provider controller owns runtime identity resolution. +Provider factories are also provider-specific. When a provider declares an identity, its factory resolves identity resources, credentials, SDK clients, and readiness requirements. Credentialless providers omit `identity` and `ZoneClass.spec.identityRef`; they do not manufacture an empty identity or credential factory. Fake providers are test boundaries, not public extension points. Unit tests and envtest-style controller tests use fake implementations of the provider package interfaces to verify finalizers, conditions, events, status provider data, adoption checks, provider error mapping, and retry behavior without calling real cloud APIs. These fakes should stay close to the controller package unless repeated cross-provider test logic proves that a shared helper removes real duplication. Common code may be shared for provider reference parsing, Provider version resolution, condition reason naming, status provider data encoding, and validation helpers. It must not force Route 53, Cloudflare, or future providers into a single CRUD-shaped DNS provider abstraction. -Example: +### External Verify-Only Runtime + +`internal/go/providers/external` implements the credentialless `external.dns.appthrust.io/v1alpha1` contract. Its manifest advertises exactly A, AAAA, and CNAME. ZoneClass parameters choose Cluster pod DNS or up to three explicit Recursive IP:port views and optionally up to sixteen expected authoritative nameservers. All configured views must agree; there is no OS or public-resolver fallback. Provider OpenAPI/CEL rejects noncanonical endpoints, noncanonical nameservers, nonempty adoption/options, and incompatible resolver settings. + +The read-only boundary is `probe.Resolver.Probe(context.Context, probe.ProbeRequest) (probe.ProbeResult, error)`. The wire implementation sends QUERY only, uses UDP then TCP on truncation, follows at most eight CNAME links, and checks both address families against complete exact sets. A wildcard uses a stable concrete witness derived from the ZoneUnit identity, Zone reference, and sorted claim UID/generation tuple; it is sampled evidence, not a guarantee about every descendant name. The provider consumes the core ledger, never writes claim specs/status, and never reads DNS credentials. + +Manager wiring uses `zoneclass.ZoneClassReconciler.SetupWithManager`, `zoneunit.NewReconciler(client.Client, zoneunit.Options)` followed by `SetupWithManager`, and `conversion.NewHandler().Register(webhookServer)`. `Options` carries binding strings, optional injected Resolver/Clock/Jitter/EffectivePolicy, and QPS/Burst (defaults 40/80). The controller admits at most four names concurrently across units and thirty-two due names per pass. One cycle-local Zone probe shares NS/SOA and NS-host work across its names. It persists advisory due times/fingerprints and resumes a backlog after one second; completed names retain a 30–300 second polling floor/cap, ±10% jitter, and failure backoff of 30/60/120/300 seconds. + +Public verification data includes bounded typed actual/missing/extra previews, chain/views, observed TTL and optional SOA negative expiry, checkedName, sampledWildcard, and last/next probe times. A later failed observation clears success. ZoneUnit aggregate reasons map to core's fixed allowlist, with False taking precedence over Unknown. Deletion completes only UID-bound Kubernetes bookkeeping: it never removes customer DNS and is not proof that a retired VIP is safe to reuse. + +The endpoint conversion group is `endpoint.external.dns.appthrust.io/v1alpha1`, resource `endpointrecordsetconversions`. It preserves request UID and name, emits sorted canonical A then AAAA fragments or one CNAME fragment, and uses TTL 300 without modifying customer TTLs. Mixed IP/hostname targets, multiple distinct hostname targets, incompatible preferences, and apex CNAME are rejected. Provider and capability declarations are in `internal/go/providers/external/manifest`; manager/chart registration is separate from these runtime packages. + + +### Provider Declaration Example ```yaml apiVersion: dns.appthrust.io/v1alpha1 diff --git a/docs/design/test-policy.md b/docs/design/test-policy.md index 33d004c..dd858a0 100644 --- a/docs/design/test-policy.md +++ b/docs/design/test-policy.md @@ -8,6 +8,10 @@ Conformance test distribution, execution method, configuration file schema, and envtest is used as integration testing with Kubernetes API server, etcd, CRDs, admission webhook, and reconciler. envtest does not operate real Route 53 or other external resources. Provider calls are replaced with fake or mock implementations. It checks Kubernetes resources, conditions, events, and finalizers. For TXT records, envtest or provider unit tests cover `spec.txt.values` validation, `_` in record names, core rejection of newline, tab, control-character, and non-ASCII TXT values through `forbid-txt-non-printable-ascii`, and Route 53 quote / escape / 255-octet chunk comparison as logical TXT values. For CNAME records, envtest or provider unit tests cover zone apex rejection, target syntax validation, standard-body exclusivity, and CNAME same-name conflict with other record types. For MX records, envtest or provider unit tests cover preference range validation, exchange syntax validation, duplicate pair rejection, Null MX validation, standard-body exclusivity, and Route 53 value parsing as unordered preference / exchange pairs. For CAA records, envtest or provider unit tests cover flags range validation, tag grammar validation, value presence, duplicate tuple rejection, standard-body exclusivity, and Route 53 quote / escape parsing as unordered flags / tag / value tuples. For delegated NS records, envtest or provider unit tests cover apex rejection, wildcard rejection, name server syntax validation, duplicate rejection, standard-body exclusivity, and Route 53 name server value comparison as unordered DNS name sets. +External provider unit tests live in `internal/go/providers/external` and `pkg/go/api/external`. They use an injected Resolver, clock, and jitter for exact sets, sibling-family and policy fingerprints, wildcard witnesses, bounded passes, backoff, mismatch/recovery/drift, stale-result rejection, and UID-bound deletion. Loopback UDP/TCP fixtures exercise DNS wire parsing, truncation fallback, SOA negative TTLs, RD/AA, cancellation, and QUERY-only behavior. No public DNS, cloud credentials, or customer DNS resources are used. Manifest tests compile and evaluate the declared rules through the pinned KRO CEL environment; conversion tests cover HTTP framing, UID, typed fragments, TTL 300, and failures. + +After integrating the optional-identity core API, the focused command is `go test ./internal/go/providers/external/... ./pkg/go/api/external/...`. The runtime packages alone do not establish manager/chart registration, aggregated APIService authentication/TLS/RBAC, or end-to-end EndpointRecordSet readiness. Those remain separate integration acceptance checks, not claims inferred from fake-resolver tests. + Provider validation tests cover the `forbid-txt-non-printable-ascii` disable switch as a core named validation. The built-in Route 53 and Cloudflare Provider manifests do not disable it. A provider test fixture may disable it to prove that the core webhook can be relaxed by a future provider version after that provider defines escape, unescape, display, and comparison behavior. Route 53 provider unit tests keep a defensive controller-side rejection path for non-printable ASCII TXT values even though the built-in webhook path rejects them earlier. envtest or webhook unit tests cover status subresource validation. For `RecordSet/status`, tests reject `status.observedGeneration` values greater than the `RecordSet.metadata.generation`, reject `status.zone.ref` namespace or name values that differ from `spec.zoneRef` after namespace defaulting, and reject any `status.zone.controllerName` field. Tests also accept a valid `status.zone.ref` as the controller-resolved `Zone` reference. CRD schema or generated-type checks must show that `RecordSet.status.zone.controllerName` is absent from the API surface. diff --git a/go.mod b/go.mod index cf873d2..cb72b87 100644 --- a/go.mod +++ b/go.mod @@ -11,6 +11,9 @@ require ( github.com/aws/smithy-go v1.25.1 github.com/google/cel-go v0.27.0 github.com/kubernetes-sigs/kro v0.9.2 + github.com/miekg/dns v1.1.68 + github.com/prometheus/client_golang v1.23.2 + golang.org/x/time v0.14.0 k8s.io/api v0.35.0 k8s.io/apiextensions-apiserver v0.35.0 k8s.io/apimachinery v0.35.0 @@ -64,7 +67,6 @@ require ( github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect - github.com/prometheus/client_golang v1.23.2 // indirect github.com/prometheus/client_model v0.6.2 // indirect github.com/prometheus/common v0.66.1 // indirect github.com/prometheus/procfs v0.17.0 // indirect @@ -78,13 +80,14 @@ require ( go.yaml.in/yaml/v2 v2.4.3 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect golang.org/x/exp v0.0.0-20240823005443-9b4947da3948 // indirect + golang.org/x/mod v0.30.0 // indirect golang.org/x/net v0.47.0 // indirect golang.org/x/oauth2 v0.30.0 // indirect golang.org/x/sync v0.19.0 // indirect golang.org/x/sys v0.38.0 // indirect golang.org/x/term v0.37.0 // indirect golang.org/x/text v0.31.0 // indirect - golang.org/x/time v0.14.0 // indirect + golang.org/x/tools v0.39.0 // indirect gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 // indirect diff --git a/go.sum b/go.sum index 5945ad8..2ee97df 100644 --- a/go.sum +++ b/go.sum @@ -139,6 +139,8 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/mailru/easyjson v0.9.0 h1:PrnmzHw7262yW8sTBwxi1PdJA3Iw/EKBa8psRf7d9a4= github.com/mailru/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU= +github.com/miekg/dns v1.1.68 h1:jsSRkNozw7G/mnmXULynzMNIsgY2dHC8LO6U6Ij2JEA= +github.com/miekg/dns v1.1.68/go.mod h1:fujopn7TB3Pu3JM69XaawiU0wqjpL9/8xGop5UrTPps= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= diff --git a/internal/go/providers/external/config/config.go b/internal/go/providers/external/config/config.go new file mode 100644 index 0000000..531a98e --- /dev/null +++ b/internal/go/providers/external/config/config.go @@ -0,0 +1,105 @@ +// Package config implements static external-provider configuration, without DNS I/O. +package config + +import ( + "bytes" + "encoding/json" + "fmt" + "net/netip" + "regexp" + "slices" + "strings" + + "github.com/appthrust/dns-api/internal/go/core/providercontract" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "k8s.io/apimachinery/pkg/runtime" +) + +var namePattern = regexp.MustCompile(`^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$`) + +func ValidName(name string) bool { + _, ipErr := netip.ParseAddr(name) + return len(name) > 0 && len(name) <= 253 && namePattern.MatchString(name) && ipErr != nil +} + +func Binding(name, version string) dnsv1alpha1.ProviderReference { + if name == "" { + name = externalv1alpha1.ProviderName + } + if version == "" { + version = externalv1alpha1.ProviderVersion + } + return dnsv1alpha1.ProviderReference{Name: name, Version: version} +} + +func ControllerName(name string) string { + if name == "" { + return externalv1alpha1.ControllerName + } + return name +} + +// HasIdentity never dereferences IdentityRef. Only the optional reference's nil +// value is absent; a pre-cutover value reference or an empty object is present. +func HasIdentity(class *dnsv1alpha1.ZoneClass) bool { + return any(class.Spec.IdentityRef) != (*dnsv1alpha1.LocalObjectReference)(nil) +} + +func Parameters(provider *dnsv1alpha1.Provider, version *dnsv1alpha1.ProviderVersion, class *dnsv1alpha1.ZoneClass) (externalv1alpha1.ExternalZoneClassParameters, error) { + raw, err := providercontract.ConvertZoneClassParametersToStorage(provider, version, class) + if err != nil { + return externalv1alpha1.ExternalZoneClassParameters{}, err + } + return Decode(raw) +} + +func Decode(raw runtime.RawExtension) (externalv1alpha1.ExternalZoneClassParameters, error) { + var params externalv1alpha1.ExternalZoneClassParameters + if len(raw.Raw) != 0 { + decoder := json.NewDecoder(bytes.NewReader(raw.Raw)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(¶ms); err != nil { + return params, fmt.Errorf("invalid external parameters: %w", err) + } + } + if params.Resolver.Mode == "" { + params.Resolver.Mode = externalv1alpha1.ResolverModeCluster + } + switch params.Resolver.Mode { + case externalv1alpha1.ResolverModeCluster: + if len(params.Resolver.Servers) != 0 { + return params, fmt.Errorf("resolver.servers must be empty for Cluster") + } + case externalv1alpha1.ResolverModeRecursive: + if err := ValidateServers(params.Resolver.Servers); err != nil { + return params, err + } + default: + return params, fmt.Errorf("resolver.mode must be Cluster or Recursive") + } + if len(params.ExpectedNameservers) > 16 { + return params, fmt.Errorf("expectedNameservers must contain at most 16 names") + } + for i, name := range params.ExpectedNameservers { + if !ValidName(name) || slices.Contains(params.ExpectedNameservers[:i], name) { + return params, fmt.Errorf("expectedNameservers must contain distinct canonical DNS names") + } + } + slices.Sort(params.Resolver.Servers) + slices.Sort(params.ExpectedNameservers) + return params, nil +} + +func ValidateServers(servers []string) error { + if len(servers) == 0 || len(servers) > 3 { + return fmt.Errorf("resolver.servers must contain 1 through 3 endpoints") + } + for i, server := range servers { + address, err := netip.ParseAddrPort(server) + if err != nil || address.Port() == 0 || address.Addr().Zone() != "" || address.String() != server || strings.Contains(server, "%") || slices.Contains(servers[:i], server) { + return fmt.Errorf("resolver.servers must contain distinct canonical IP addresses and ports from 1 through 65535") + } + } + return nil +} diff --git a/internal/go/providers/external/controllers/zoneclass/controller.go b/internal/go/providers/external/controllers/zoneclass/controller.go new file mode 100644 index 0000000..a4f1858 --- /dev/null +++ b/internal/go/providers/external/controllers/zoneclass/controller.go @@ -0,0 +1,111 @@ +package zoneclass + +import ( + "context" + "errors" + + "github.com/appthrust/dns-api/internal/go/core/providercontract" + "github.com/appthrust/dns-api/internal/go/providers/external/config" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "k8s.io/apimachinery/pkg/api/equality" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/builder" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/predicate" + "sigs.k8s.io/controller-runtime/pkg/reconcile" +) + +type ZoneClassReconciler struct { + client.Client + // Reader provides real pagination for Provider dependency fan-out. + Reader client.Reader + ControllerName string + ProviderName string + ProviderVersion string +} + +func (r *ZoneClassReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + var class dnsv1alpha1.ZoneClass + if err := r.Get(ctx, req.NamespacedName, &class); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + if class.Spec.ControllerName != config.ControllerName(r.ControllerName) || !class.DeletionTimestamp.IsZero() { + return ctrl.Result{}, nil + } + var provider dnsv1alpha1.Provider + if err := r.Get(ctx, client.ObjectKey{Name: class.Spec.Provider.Name}, &provider); err != nil && !apierrors.IsNotFound(err) { + return ctrl.Result{}, err + } + status, reason, message := r.accept(&class, &provider) + base := class.DeepCopy() + meta.SetStatusCondition(&class.Status.Conditions, metav1.Condition{Type: "Accepted", Status: status, Reason: reason, Message: message, ObservedGeneration: class.Generation}) + if equality.Semantic.DeepEqual(base.Status, class.Status) { + return ctrl.Result{}, nil + } + return ctrl.Result{}, r.Status().Patch(ctx, &class, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) +} + +func (r *ZoneClassReconciler) accept(class *dnsv1alpha1.ZoneClass, provider *dnsv1alpha1.Provider) (metav1.ConditionStatus, string, string) { + binding := config.Binding(r.ProviderName, r.ProviderVersion) + version, served := providercontract.ServedVersion(provider, class.Spec.Provider.Version) + if provider.Name != binding.Name || !served { + return metav1.ConditionFalse, "InvalidProvider", "ZoneClass provider is not handled by external controller" + } + storage, resolved := providercontract.StorageVersion(provider) + if !resolved { + return metav1.ConditionUnknown, "ProviderStorageVersionNotResolved", "provider storage version is not resolved" + } + if storage.Name != binding.Version || version.Identity != nil || storage.Identity != nil { + return metav1.ConditionFalse, "InvalidProvider", "external provider requires its configured credentialless storage version" + } + if config.HasIdentity(class) { + return metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef must be omitted for external DNS" + } + if _, err := config.Parameters(provider, version, class); err != nil { + var payloadErr *providercontract.PayloadError + if errors.As(err, &payloadErr) && payloadErr.Kind == providercontract.ProviderContractNotReady { + return metav1.ConditionUnknown, "ProviderStorageVersionNotResolved", "provider storage version is not resolved" + } + return metav1.ConditionFalse, "InvalidParameters", err.Error() + } + return metav1.ConditionTrue, "Accepted", "ZoneClass is accepted by external controller" +} + +func (r *ZoneClassReconciler) SetupWithManager(mgr ctrl.Manager) error { + if r.Client == nil { + r.Client = mgr.GetClient() + } + r.Reader = mgr.GetAPIReader() + return ctrl.NewControllerManagedBy(mgr).Named("external-zoneclass"). + For(&dnsv1alpha1.ZoneClass{}, builder.WithPredicates(predicate.GenerationChangedPredicate{})). + Watches(&dnsv1alpha1.Provider{}, handler.EnqueueRequestsFromMapFunc(r.mapProvider), builder.WithPredicates(predicate.GenerationChangedPredicate{})).Complete(r) +} + +func (r *ZoneClassReconciler) mapProvider(ctx context.Context, obj client.Object) []reconcile.Request { + reader := r.Reader + if reader == nil { + reader = r.Client + } + var requests []reconcile.Request + continuation := "" + for { + var classes dnsv1alpha1.ZoneClassList + if err := reader.List(ctx, &classes, client.Limit(256), client.Continue(continuation)); err != nil { + return nil + } + for i := range classes.Items { + class := &classes.Items[i] + if class.Spec.Provider.Name == obj.GetName() && class.Spec.ControllerName == config.ControllerName(r.ControllerName) { + requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(class)}) + } + } + continuation = classes.Continue + if continuation == "" { + return requests + } + } +} diff --git a/internal/go/providers/external/controllers/zoneclass/controller_test.go b/internal/go/providers/external/controllers/zoneclass/controller_test.go new file mode 100644 index 0000000..602763b --- /dev/null +++ b/internal/go/providers/external/controllers/zoneclass/controller_test.go @@ -0,0 +1,98 @@ +package zoneclass + +import ( + "context" + "encoding/json" + "testing" + + "github.com/appthrust/dns-api/internal/go/providers/external/manifest" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" +) + +func classFixture() *dnsv1alpha1.ZoneClass { + return &dnsv1alpha1.ZoneClass{ObjectMeta: metav1.ObjectMeta{Namespace: "platform", Name: "external", Generation: 1}, Spec: dnsv1alpha1.ZoneClassSpec{Provider: dnsv1alpha1.ProviderReference{Name: externalv1alpha1.ProviderName, Version: externalv1alpha1.ProviderVersion}, ControllerName: externalv1alpha1.ControllerName, Parameters: runtime.RawExtension{Raw: []byte(`{}`)}}} +} + +func TestClassAcceptanceAndIdentityRejection(t *testing.T) { + provider, err := manifest.Provider() + if err != nil { + t.Fatal(err) + } + r := &ZoneClassReconciler{} + class := classFixture() + status, reason, message := r.accept(class, provider) + if status != metav1.ConditionTrue || reason != "Accepted" { + t.Fatalf("accept=%s/%s %s (requires optional identityRef core cutover)", status, reason, message) + } + for _, identity := range []string{`{}`, `{"name":"forbidden"}`} { + // JSON keeps this test source compatible with both sides of the optional-pointer cutover. + if err := json.Unmarshal([]byte(`{"spec":{"identityRef":`+identity+`}}`), class); err != nil { + t.Fatal(err) + } + status, reason, _ = r.accept(class, provider) + if status != metav1.ConditionFalse || reason != "InvalidIdentityRef" { + t.Fatalf("identity=%s accept=%s/%s", identity, status, reason) + } + } + class = classFixture() + class.Spec.Parameters.Raw = []byte(`{"resolver":{"mode":"Recursive"}}`) + if status, reason, _ := r.accept(class, provider); status != metav1.ConditionFalse || reason != "InvalidParameters" { + t.Fatalf("accept=%s/%s", status, reason) + } + provider.Spec.Versions[0].Storage = false + if status, reason, _ := r.accept(class, provider); status != metav1.ConditionUnknown || reason != "ProviderStorageVersionNotResolved" { + t.Fatalf("accept=%s/%s", status, reason) + } +} + +func TestClassScopeAndUnchangedStatus(t *testing.T) { + scheme := runtime.NewScheme() + if err := dnsv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + provider, err := manifest.Provider() + if err != nil { + t.Fatal(err) + } + class := classFixture() + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, class).WithStatusSubresource(&dnsv1alpha1.ZoneClass{}).Build() + r := &ZoneClassReconciler{Client: c} + request := ctrl.Request{NamespacedName: client.ObjectKeyFromObject(class)} + if _, err := r.Reconcile(context.Background(), request); err != nil { + t.Fatal(err) + } + if err := c.Get(context.Background(), request.NamespacedName, class); err != nil { + t.Fatal(err) + } + condition := meta.FindStatusCondition(class.Status.Conditions, "Accepted") + if condition == nil || condition.Status != metav1.ConditionTrue || condition.ObservedGeneration != 1 { + t.Fatalf("condition=%#v", condition) + } + version := class.ResourceVersion + if _, err := r.Reconcile(context.Background(), request); err != nil { + t.Fatal(err) + } + if err := c.Get(context.Background(), request.NamespacedName, class); err != nil { + t.Fatal(err) + } + if class.ResourceVersion != version { + t.Fatal("unchanged class status was patched") + } + r.ControllerName = "other.example/controller" + if _, err := r.Reconcile(context.Background(), request); err != nil { + t.Fatal(err) + } + if err := c.Get(context.Background(), request.NamespacedName, class); err != nil { + t.Fatal(err) + } + if class.ResourceVersion != version { + t.Fatal("foreign controller changed class") + } +} diff --git a/internal/go/providers/external/controllers/zoneunit/controller.go b/internal/go/providers/external/controllers/zoneunit/controller.go new file mode 100644 index 0000000..b027983 --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/controller.go @@ -0,0 +1,384 @@ +package zoneunit + +import ( + "context" + "encoding/json" + "errors" + "math/rand/v2" + "slices" + "time" + + "github.com/appthrust/dns-api/internal/go/core/declaration" + "github.com/appthrust/dns-api/internal/go/core/providercontract" + "github.com/appthrust/dns-api/internal/go/providers/external/config" + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/equality" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +type Options struct { + ControllerName string + ProviderName string + ProviderVersion string + QPS float64 + Burst int + Resolver probe.Resolver + Clock func() time.Time + // Jitter returns a value in [-1,1], scaled to +/-10 percent. + Jitter func() float64 + // EffectivePolicy resolves pod DNS without using the OS DNS lookup fallback. + EffectivePolicy func(externalv1alpha1.ExternalZoneClassParameters) (externalv1alpha1.ExternalZoneClassParameters, error) +} + +type ZoneReconciler struct { + client.Client + Options Options + // Reader is a live read boundary for rejecting results whose inputs changed during DNS I/O. + Reader client.Reader +} + +func NewReconciler(k8sClient client.Client, options Options) (*ZoneReconciler, error) { + if options.Resolver == nil { + wire, err := probe.NewWireResolver(probe.WireOptions{QPS: options.QPS, Burst: options.Burst}) + if err != nil { + return nil, err + } + options.Resolver = wire + if options.EffectivePolicy == nil { + options.EffectivePolicy = wire.EffectivePolicy + } + } + if options.Clock == nil { + options.Clock = time.Now + } + if options.Jitter == nil { + options.Jitter = func() float64 { return rand.Float64()*2 - 1 } + } + return &ZoneReconciler{Client: k8sClient, Reader: k8sClient, Options: options}, nil +} + +type environment struct { + class dnsv1alpha1.ZoneClass + provider dnsv1alpha1.Provider + policy externalv1alpha1.ExternalZoneClassParameters + fingerprint string + policyError error +} + +type acceptance struct { + Status metav1.ConditionStatus + Reason, Message string +} + +func (r *ZoneReconciler) observe(ctx context.Context, c client.Client, unit *dnsv1alpha1.ZoneUnit) (*environment, *acceptance, error) { + env := &environment{} + ref := unit.Spec.Zone.ZoneClassRef + if err := c.Get(ctx, client.ObjectKey{Namespace: ref.Namespace, Name: ref.Name}, &env.class); err != nil { + if apierrors.IsNotFound(err) { + return nil, &acceptance{metav1.ConditionFalse, "InvalidZoneClassRef", "referenced ZoneClass was not found"}, nil + } + return nil, nil, err + } + if env.class.Spec.ControllerName != config.ControllerName(r.Options.ControllerName) { + return nil, nil, nil + } + binding := config.Binding(r.Options.ProviderName, r.Options.ProviderVersion) + if unit.Spec.Provider != binding || env.class.Spec.Provider.Name != binding.Name { + return nil, &acceptance{metav1.ConditionFalse, "InvalidProvider", "ZoneUnit provider is not handled by external controller"}, nil + } + if err := c.Get(ctx, client.ObjectKey{Name: binding.Name}, &env.provider); err != nil { + if apierrors.IsNotFound(err) { + return nil, &acceptance{metav1.ConditionFalse, "InvalidProvider", "referenced Provider was not found"}, nil + } + return nil, nil, err + } + version, served := providercontract.ServedVersion(&env.provider, env.class.Spec.Provider.Version) + storage, resolved := providercontract.StorageVersion(&env.provider) + if !resolved { + return nil, &acceptance{metav1.ConditionUnknown, "ProviderStorageVersionNotResolved", "provider storage version is not resolved"}, nil + } + if !served || storage.Name != binding.Version || version.Identity != nil || storage.Identity != nil { + return nil, &acceptance{metav1.ConditionFalse, "InvalidProvider", "external provider requires its configured credentialless storage version"}, nil + } + if config.HasIdentity(&env.class) { + return nil, &acceptance{metav1.ConditionFalse, "InvalidIdentityRef", "spec.identityRef must be omitted for external DNS"}, nil + } + condition := meta.FindStatusCondition(env.class.Status.Conditions, "Accepted") + if !env.class.DeletionTimestamp.IsZero() || condition == nil || condition.Status != metav1.ConditionTrue || condition.ObservedGeneration != env.class.Generation { + return nil, &acceptance{metav1.ConditionFalse, "ZoneClassNotAccepted", "referenced ZoneClass is not accepted"}, nil + } + params, err := config.Parameters(&env.provider, version, &env.class) + if err != nil { + return nil, &acceptance{metav1.ConditionFalse, "InvalidProviderPayload", "external resolver parameters are invalid"}, nil + } + if allowed, err := declaration.ZoneAllowedByClass(ctx, c, &env.class, unit.Spec.Zone.Ref.Namespace); err != nil { + return nil, nil, err + } else if !allowed { + return nil, &acceptance{metav1.ConditionFalse, "NotAllowedByZoneClass", "Zone namespace is not allowed by ZoneClass"}, nil + } + if !emptyObject(unit.Spec.Zone.Adoption) { + return nil, &acceptance{metav1.ConditionFalse, "InvalidProviderPayload", "external DNS does not support adoption"}, nil + } + env.policy = params + if r.Options.EffectivePolicy != nil { + env.policy, env.policyError = r.Options.EffectivePolicy(params) + } + namespaceNames := []string{unit.Spec.Zone.Ref.Namespace} + for _, item := range unit.Spec.RecordSets { + namespaceNames = append(namespaceNames, item.RecordSetNamespace) + } + slices.Sort(namespaceNames) + namespaceNames = slices.Compact(namespaceNames) + namespacePolicies := make(map[string]map[string]string, len(namespaceNames)) + for _, name := range namespaceNames { + var namespace corev1.Namespace + if err := c.Get(ctx, client.ObjectKey{Name: name}, &namespace); err != nil && !apierrors.IsNotFound(err) { + return nil, nil, err + } + namespacePolicies[name] = namespace.Labels + } + env.fingerprint = probe.Fingerprint(struct { + ClassUID string + ClassSpec dnsv1alpha1.ZoneClassSpec + ProviderUID string + ProviderSpec dnsv1alpha1.ProviderSpec + UnitUID string + Zone dnsv1alpha1.ZoneUnitZoneSpec + Policy externalv1alpha1.ExternalZoneClassParameters + NamespacePolicies map[string]map[string]string + }{string(env.class.UID), env.class.Spec, string(env.provider.UID), env.provider.Spec, string(unit.UID), unit.Spec.Zone, env.policy, namespacePolicies}) + return env, nil, nil +} + +func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (ctrl.Result, error) { + var unit dnsv1alpha1.ZoneUnit + if err := r.Get(ctx, request.NamespacedName, &unit); err != nil { + if apierrors.IsNotFound(err) { + probe.SetPending(request.NamespacedName.String(), nil) + } + return ctrl.Result{}, client.IgnoreNotFound(err) + } + if unit.Spec.Provider.Name != config.Binding(r.Options.ProviderName, r.Options.ProviderVersion).Name { + return ctrl.Result{}, nil + } + env, rejected, err := r.observe(ctx, r.Client, &unit) + if err != nil { + return ctrl.Result{}, err + } + if env == nil && rejected == nil { + return ctrl.Result{}, nil + } + base := unit.DeepCopy() + prepareReceipts(&unit) + if !unit.DeletionTimestamp.IsZero() { + // No provider finalizer is necessary: there is no external object to delete. + for i := range unit.Status.RecordSets { + unit.Status.RecordSets[i].DeletionCompleted = unit.Spec.RecordSets[i].RecordSetUID != "" + } + probe.SetPending(request.NamespacedName.String(), nil) + return ctrl.Result{}, r.patch(ctx, base, &unit) + } + if rejected != nil { + r.reject(&unit, *rejected) + return ctrl.Result{RequeueAfter: 30 * time.Second}, r.patch(ctx, base, &unit) + } + now := r.Options.Clock() + state := loadState(&unit, env.fingerprint) + groups := groupNames(&unit, env.fingerprint) + active := make(map[string]bool, len(groups)) + for _, group := range groups { + active[group.Name] = true + currentReceipt := true + for _, index := range group.Items { + item := &unit.Status.RecordSets[index] + condition := meta.FindStatusCondition(item.Conditions, "Programmed") + if condition == nil || condition.ObservedGeneration != item.ObservedGeneration { + currentReceipt = false + } + if meta.FindStatusCondition(item.Conditions, "Accepted") == nil { + setCondition(&item.Conditions, "Accepted", metav1.ConditionTrue, "Accepted", "External DNS record intent is accepted.", item.ObservedGeneration, now) + } + } + if !currentReceipt || state.Names[group.Name].Fingerprint != group.Fingerprint { + delete(state.Names, group.Name) + for _, index := range group.Items { + resetProgrammed(&unit.Status.RecordSets[index], now) + } + } + } + for name := range state.Names { + if !active[name] { + delete(state.Names, name) + } + } + due := selectDue(groups, state, now) + zoneCondition := meta.FindStatusCondition(unit.Status.Zone.Conditions, "Programmed") + zoneDue := state.Zone.Fingerprint != env.fingerprint || !state.Zone.Next.After(now) || zoneCondition == nil || zoneCondition.ObservedGeneration != unit.Spec.Zone.ObservedGeneration + var evidence *probe.ZoneEvidence + zoneDecision := currentDecision(unit.Status.Zone.Conditions) + zoneProbed := zoneDue || (len(due) != 0 && zoneDecision.Status == metav1.ConditionTrue) + if zoneProbed { + zoneRequest := probe.ProbeRequest{Zone: unit.Spec.Zone.DomainName, Name: unit.Spec.Zone.DomainName, Policy: env.policy, ZoneOnly: true} + switch { + case env.policyError != nil: + zoneDecision = probe.Failed(zoneRequest.Zone, "configured-resolvers", "resolver-config") + case len(env.policy.ExpectedNameservers) == 0: + zoneDecision = probe.EvaluateZone(zoneRequest, nil) + default: + observed, probeErr := r.Options.Resolver.Probe(ctx, zoneRequest) + evidence = observed.Zone + zoneDecision = probe.EvaluateZone(zoneRequest, evidence) + if probeErr != nil { + zoneDecision = probe.Failed(zoneRequest.Zone, "required-views", failureCode(probeErr)) + } + } + completed := r.Options.Clock() + state.Zone = advance(state.Zone, env.fingerprint, zoneDecision, completed, r.Options.Jitter()) + setZoneResult(&unit, env.policy, zoneDecision, completed, state.Zone.Next) + } + invalidGroups := map[string]bool{} + for _, groupIndex := range due { + group := groups[groupIndex] + old := state.Names[group.Name] + dueTime := old.Next + if dueTime.IsZero() { + dueTime = now + } + valid, reason, message := validateGroup(&unit, group) + decision := zoneDecision + if !valid { + invalidGroups[group.Name] = true + for _, index := range group.Items { + setCondition(&unit.Status.RecordSets[index].Conditions, "Accepted", metav1.ConditionFalse, reason, message, unit.Spec.RecordSets[index].ObservedGeneration, now) + resetProgrammed(&unit.Status.RecordSets[index], now) + } + decision = probe.Decision{Status: metav1.ConditionUnknown, Reason: "Reconciling", Message: message} + } else if zoneDecision.Status == metav1.ConditionTrue { + probeRequest := probe.ProbeRequest{Name: group.CheckedName, Zone: unit.Spec.Zone.DomainName, Expected: group.Expected, Policy: env.policy, Evidence: evidence, DueTime: dueTime} + observed, probeErr := r.Options.Resolver.Probe(ctx, probeRequest) + decision = probe.Evaluate(probeRequest, observed) + if probeErr != nil { + decision = probe.Failed(group.CheckedName, "required-views", failureCode(probeErr)) + } + } + completed := r.Options.Clock() + scheduled := advance(old, group.Fingerprint, decision, completed, r.Options.Jitter()) + state.Names[group.Name] = scheduled + if valid { + setGroupResult(&unit, group, env.policy, decision, completed, scheduled.Next) + } + state.Cursor = group.Name + } + if zoneProbed && zoneDecision.Status != metav1.ConditionTrue { + for _, group := range groups { + if invalidGroups[group.Name] { + continue + } + // Withdraw every old success immediately, including names outside this pass. + for _, index := range group.Items { + item := &unit.Status.RecordSets[index] + item.Provider = publicData(env.policy, group.CheckedName, group.SampledWildcard, zoneDecision, r.Options.Clock(), state.Zone.Next) + setCondition(&item.Conditions, "Programmed", zoneDecision.Status, zoneDecision.Reason, zoneDecision.Message, item.ObservedGeneration, r.Options.Clock()) + } + } + } + // DNS I/O is an asynchronous observation boundary. Live inputs must still agree. + reader := readerClient{Client: r.Client, reader: r.Reader} + var current dnsv1alpha1.ZoneUnit + if err := reader.Get(ctx, request.NamespacedName, ¤t); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + if current.UID != unit.UID || current.Generation != unit.Generation || !equality.Semantic.DeepEqual(current.Spec, unit.Spec) || !current.DeletionTimestamp.IsZero() { + return ctrl.Result{RequeueAfter: time.Second}, nil + } + fresh, blocked, err := r.observe(ctx, reader, ¤t) + if err != nil { + return ctrl.Result{}, err + } + if blocked != nil || fresh == nil || fresh.fingerprint != env.fingerprint || (fresh.policyError != nil) != (env.policyError != nil) { + return ctrl.Result{RequeueAfter: time.Second}, nil + } + stateBytes, err := json.Marshal(state) + if err != nil { + return ctrl.Result{}, err + } + unit.Status.Provider = &dnsv1alpha1.ProviderStatus{State: runtime.RawExtension{Raw: stateBytes}} + setAggregate(&unit, r.Options.Clock()) + r.updatePending(&unit) + return ctrl.Result{RequeueAfter: nextRequeue(groups, state, r.Options.Clock())}, r.patch(ctx, base, &unit) +} + +func validateGroup(unit *dnsv1alpha1.ZoneUnit, group nameGroup) (bool, string, string) { + if group.Expected.CNAME != "" && len(group.Expected.A)+len(group.Expected.AAAA) != 0 { + return false, "InvalidProviderPayload", "CNAME must not coexist with address records" + } + for _, index := range group.Items { + item := unit.Spec.RecordSets[index] + if !item.IsAllowed() || item.DeletionRequested || item.RecordSetUID == "" || item.ObservedGeneration <= 0 { + return false, "DeniedByPolicy", "record claim is not allowed or not bound to a current incarnation" + } + if !emptyObject(item.Options) || !emptyObject(item.Adoption) || item.TTL == nil || *item.TTL < 1 { + return false, "InvalidProviderPayload", "external DNS requires a standard TTL and no options or adoption" + } + valid := false + switch item.Type { + case dnsv1alpha1.RecordTypeA: + valid = item.A != nil && len(item.A.Addresses) != 0 && item.AAAA == nil && item.CNAME == nil + case dnsv1alpha1.RecordTypeAAAA: + valid = item.AAAA != nil && len(item.AAAA.Addresses) != 0 && item.A == nil && item.CNAME == nil + case dnsv1alpha1.RecordTypeCNAME: + valid = item.CNAME != nil && item.CNAME.Target != "" && item.Name != "@" && item.A == nil && item.AAAA == nil + } + if !valid || item.TXT != nil || item.MX != nil || item.CAA != nil || item.NS != nil { + return false, "InvalidProviderPayload", "external DNS supports only standard A, AAAA and CNAME bodies" + } + } + return true, "", "" +} + +func emptyObject(raw runtime.RawExtension) bool { + if len(raw.Raw) == 0 { + return true + } + var value map[string]json.RawMessage + return json.Unmarshal(raw.Raw, &value) == nil && len(value) == 0 +} + +func (r *ZoneReconciler) patch(ctx context.Context, base, unit *dnsv1alpha1.ZoneUnit) error { + unit.Status.ObservedGeneration = unit.Generation + if equality.Semantic.DeepEqual(base.Status, unit.Status) { + return nil + } + return r.Status().Patch(ctx, unit, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) +} + +func failureCode(err error) string { + if errors.Is(err, context.DeadlineExceeded) { + return "timeout" + } + switch err.Error() { + case "resolver-config", "invalid-response", "probe-limit", "chain-limit": + return err.Error() + } + return "transport" +} + +type readerClient struct { + client.Client + reader client.Reader +} + +func (c readerClient) Get(ctx context.Context, key client.ObjectKey, obj client.Object, opts ...client.GetOption) error { + if c.reader == nil { + return c.Client.Get(ctx, key, obj, opts...) + } + return c.reader.Get(ctx, key, obj, opts...) +} diff --git a/internal/go/providers/external/controllers/zoneunit/controller_test.go b/internal/go/providers/external/controllers/zoneunit/controller_test.go new file mode 100644 index 0000000..bb43fa7 --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/controller_test.go @@ -0,0 +1,414 @@ +package zoneunit + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "slices" + "testing" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/manifest" + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" +) + +type fakeResolver struct { + requests []probe.ProbeRequest + answer func(probe.ProbeRequest) (probe.ProbeResult, error) +} + +func (r *fakeResolver) Probe(_ context.Context, request probe.ProbeRequest) (probe.ProbeResult, error) { + r.requests = append(r.requests, request) + return r.answer(request) +} + +func matching(request probe.ProbeRequest) (probe.ProbeResult, error) { + ttl := uint32(120) + view := probe.View{ID: "recursive/192.0.2.53:53", A: request.Expected.A, AAAA: request.Expected.AAAA, TTL: &ttl} + if request.Expected.CNAME != "" { + view.Chain = []string{request.Expected.CNAME} + view.A = []string{"192.0.2.10"} + } + return probe.ProbeResult{Views: []probe.View{view}}, nil +} + +func fixtureUnit(count int) *dnsv1alpha1.ZoneUnit { + unit := &dnsv1alpha1.ZoneUnit{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "example", UID: "unit-1", Generation: 1}, Spec: dnsv1alpha1.ZoneUnitSpec{ + Provider: dnsv1alpha1.ProviderReference{Name: externalv1alpha1.ProviderName, Version: externalv1alpha1.ProviderVersion}, + Zone: dnsv1alpha1.ZoneUnitZoneSpec{Ref: dnsv1alpha1.ObjectReference{Namespace: "app", Name: "example"}, ObservedGeneration: 1, DomainName: "example.test", ZoneClassRef: dnsv1alpha1.ObjectReference{Namespace: "app", Name: "external"}}, + }} + ttl := int32(300) + for i := range count { + name := fmt.Sprintf("host-%03d", i) + unit.Spec.RecordSets = append(unit.Spec.RecordSets, dnsv1alpha1.ZoneUnitRecordSetSpec{RecordSetNamespace: "app", RecordSetName: name, RecordSetUID: types.UID(name), ObservedGeneration: 1, Name: name, Type: dnsv1alpha1.RecordTypeA, TTL: &ttl, A: &dnsv1alpha1.ARecordSet{Addresses: []string{"192.0.2.10"}}}) + } + return unit +} + +func fixtureClass() *dnsv1alpha1.ZoneClass { + return &dnsv1alpha1.ZoneClass{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "external", UID: "class-1", Generation: 1}, Spec: dnsv1alpha1.ZoneClassSpec{ + Provider: dnsv1alpha1.ProviderReference{Name: externalv1alpha1.ProviderName, Version: externalv1alpha1.ProviderVersion}, ControllerName: externalv1alpha1.ControllerName, + Parameters: runtime.RawExtension{Raw: []byte(`{"resolver":{"mode":"Recursive","servers":["192.0.2.53:53"]}}`)}, + }, Status: dnsv1alpha1.ZoneClassStatus{Conditions: []metav1.Condition{{Type: "Accepted", Status: metav1.ConditionTrue, Reason: "Accepted", ObservedGeneration: 1, LastTransitionTime: metav1.NewTime(time.Unix(1, 0))}}}} +} + +func newFixture(t *testing.T, count int, resolver *fakeResolver) (*ZoneReconciler, client.Client, *time.Time, ctrl.Request) { + t.Helper() + scheme := runtime.NewScheme() + if err := dnsv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + if err := corev1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + provider, err := manifest.Provider() + if err != nil { + t.Fatal(err) + } + provider.UID = "provider-1" + unit := fixtureUnit(count) + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, fixtureClass(), unit, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "app"}}).WithStatusSubresource(&dnsv1alpha1.ZoneUnit{}, &dnsv1alpha1.ZoneClass{}).Build() + now := time.Date(2026, 10, 18, 0, 0, 0, 0, time.UTC) + reconciler, err := NewReconciler(c, Options{Resolver: resolver, Clock: func() time.Time { return now }, Jitter: func() float64 { return 0 }}) + if err != nil { + t.Fatal(err) + } + return reconciler, c, &now, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(unit)} +} + +func reconcile(t *testing.T, r *ZoneReconciler, request ctrl.Request) (ctrl.Result, dnsv1alpha1.ZoneUnit) { + t.Helper() + result, err := r.Reconcile(context.Background(), request) + if err != nil { + t.Fatal(err) + } + var unit dnsv1alpha1.ZoneUnit + if err := r.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + return result, unit +} + +func assertProgrammed(t *testing.T, conditions []metav1.Condition, status metav1.ConditionStatus, reason string) *metav1.Condition { + t.Helper() + condition := meta.FindStatusCondition(conditions, "Programmed") + if condition == nil || condition.Status != status || condition.Reason != reason { + t.Fatalf("Programmed = %#v, want %s/%s", condition, status, reason) + } + return condition +} + +func TestMismatchRecoveryDriftAndNoStatusLoop(t *testing.T) { + resolver := &fakeResolver{answer: func(request probe.ProbeRequest) (probe.ProbeResult, error) { + result, err := matching(request) + result.Views[0].A = []string{"192.0.2.99"} + return result, err + }} + r, _, now, request := newFixture(t, 1, resolver) + _, first := reconcile(t, r, request) + assertProgrammed(t, first.Status.Zone.Conditions, metav1.ConditionTrue, "VerificationConfigured") + assertProgrammed(t, first.Status.RecordSets[0].Conditions, metav1.ConditionFalse, "ResolutionMismatch") + _, unchanged := reconcile(t, r, request) + if len(resolver.requests) != 1 || unchanged.ResourceVersion != first.ResourceVersion { + t.Fatal("unchanged status triggered work or a status patch") + } + resolver.answer = matching + *now = now.Add(30 * time.Second) + _, ready := reconcile(t, r, request) + condition := assertProgrammed(t, ready.Status.RecordSets[0].Conditions, metav1.ConditionTrue, "ResolutionVerified") + transition := condition.LastTransitionTime + *now = now.Add(120 * time.Second) + _, readyAgain := reconcile(t, r, request) + if !assertProgrammed(t, readyAgain.Status.RecordSets[0].Conditions, metav1.ConditionTrue, "ResolutionVerified").LastTransitionTime.Equal(&transition) { + t.Fatal("success timestamp changed transition time") + } + resolver.answer = func(probe.ProbeRequest) (probe.ProbeResult, error) { + return probe.ProbeResult{}, errors.New("private raw transport details") + } + *now = now.Add(120 * time.Second) + _, failed := reconcile(t, r, request) + assertProgrammed(t, failed.Status.RecordSets[0].Conditions, metav1.ConditionUnknown, "DNSProbeFailed") + assertProgrammed(t, failed.Status.Conditions, metav1.ConditionUnknown, "ProviderUnavailable") + if failed.Status.RecordSets[0].RecordSetUID != "host-000" || failed.Status.RecordSets[0].ObservedGeneration != 1 { + t.Fatal("unbound receipt") + } +} + +func TestPolicyAndSiblingInvalidation(t *testing.T) { + resolver := &fakeResolver{answer: matching} + r, c, _, request := newFixture(t, 1, resolver) + reconcile(t, r, request) + var class dnsv1alpha1.ZoneClass + if err := c.Get(context.Background(), client.ObjectKey{Namespace: "app", Name: "external"}, &class); err != nil { + t.Fatal(err) + } + class.Spec.Parameters.Raw = []byte(`{"resolver":{"mode":"Recursive","servers":["192.0.2.54:53"]}}`) + if err := c.Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + resolver.answer = func(request probe.ProbeRequest) (probe.ProbeResult, error) { + result, err := matching(request) + result.Views[0].ID = "recursive/192.0.2.54:53" + return result, err + } + _, unit := reconcile(t, r, request) + if len(resolver.requests) != 2 { + t.Fatal("policy change retained previous success") + } + family := unit.Spec.RecordSets[0] + family.RecordSetName, family.RecordSetUID, family.Type, family.A, family.AAAA = "v6", "v6-uid", dnsv1alpha1.RecordTypeAAAA, nil, &dnsv1alpha1.AAAARecordSet{Addresses: []string{"2001:db8::1"}} + unit.Spec.RecordSets = append(unit.Spec.RecordSets, family) + unit.Generation++ + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, got := reconcile(t, r, request) + if len(resolver.requests) != 3 || len(resolver.requests[2].Expected.AAAA) != 1 { + t.Fatal("sibling family did not invalidate combined observation") + } + for _, receipt := range got.Status.RecordSets { + assertProgrammed(t, receipt.Conditions, metav1.ConditionTrue, "ResolutionVerified") + } +} + +func TestBoundedFairPassAndDeletion(t *testing.T) { + resolver := &fakeResolver{answer: matching} + r, c, now, request := newFixture(t, 65, resolver) + result, unit := reconcile(t, r, request) + if len(resolver.requests) != 32 || result.RequeueAfter != time.Second { + t.Fatalf("requests=%d result=%#v", len(resolver.requests), result) + } + assertProgrammed(t, unit.Status.RecordSets[64].Conditions, metav1.ConditionUnknown, "Reconciling") + *now = now.Add(time.Second) + reconcile(t, r, request) + *now = now.Add(time.Second) + _, unit = reconcile(t, r, request) + if len(resolver.requests) != 65 { + t.Fatalf("requests=%d, want each name once", len(resolver.requests)) + } + unit.Spec.RecordSets[0].DeletionRequested = true + denied := false + unit.Spec.RecordSets[1].Allowed = &denied + unit.Generation++ + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, got := reconcile(t, r, request) + if !got.Status.RecordSets[0].DeletionCompleted || got.Status.RecordSets[0].RecordSetUID != unit.Spec.RecordSets[0].RecordSetUID { + t.Fatal("deletion receipt missing incarnation") + } + if len(resolver.requests) != 65 { + t.Fatal("denied/deleted names were probed") + } + unit = got + unit.Spec.RecordSets[0].RecordSetUID = "replacement" + unit.Spec.RecordSets[0].DeletionRequested = false + unit.Generation++ + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, got = reconcile(t, r, request) + if got.Status.RecordSets[0].DeletionCompleted || got.Status.RecordSets[0].RecordSetUID != "replacement" { + t.Fatal("old deletion finalized replacement") + } +} + +func TestStaleAsynchronousResultsDiscarded(t *testing.T) { + resolver := &fakeResolver{} + r, c, _, request := newFixture(t, 1, resolver) + resolver.answer = func(input probe.ProbeRequest) (probe.ProbeResult, error) { + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + unit.Spec.RecordSets[0].ObservedGeneration++ + unit.Spec.RecordSets[0].A.Addresses = []string{"192.0.2.20"} + unit.Generation++ + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + return matching(input) + } + result, unit := reconcile(t, r, request) + if result.RequeueAfter != time.Second || len(unit.Status.RecordSets) != 0 { + t.Fatalf("stale result committed: %#v", unit.Status) + } +} + +func TestWildcardWitnessAndBackoff(t *testing.T) { + unit := fixtureUnit(2) + unit.Spec.RecordSets[0].Name, unit.Spec.RecordSets[1].Name = "*.project", "*.project" + first := groupNames(unit, "policy-a")[0] + slices.Reverse(unit.Spec.RecordSets) + second := groupNames(unit, "policy-b")[0] + if first.CheckedName != second.CheckedName || !first.SampledWildcard || first.Fingerprint == second.Fingerprint { + t.Fatal("unstable witness or policy fingerprint") + } + unit.Spec.RecordSets[0].ObservedGeneration++ + if groupNames(unit, "policy-b")[0].CheckedName == first.CheckedName { + t.Fatal("changed intent reused witness") + } + now := time.Unix(100, 0) + state := schedule{} + failure := probe.Failed("www.example", "view", "timeout") + for _, want := range []time.Duration{30, 60, 120, 300, 300} { + state = advance(state, "same", failure, now, 0) + if state.Next.Sub(now) != want*time.Second { + t.Fatalf("backoff=%v want=%v", state.Next.Sub(now), want) + } + } + state = advance(state, "changed", failure, now, 0) + if state.Failures != 1 || state.Next.Sub(now) != 30*time.Second { + t.Fatal("changed input did not reset backoff") + } + zero, high, minute := uint32(0), uint32(3600), uint32(60) + for _, tc := range []struct { + decision probe.Decision + jitter float64 + want time.Duration + }{ + {probe.Decision{Status: metav1.ConditionTrue, TTL: &zero}, -1, 30 * time.Second}, + {probe.Decision{Status: metav1.ConditionTrue, TTL: &high}, 1, 300 * time.Second}, + {probe.Decision{Status: metav1.ConditionFalse, Reason: "DNSNoData", NegativeTTL: &minute}, 0, time.Minute}, + {probe.Decision{Status: metav1.ConditionTrue, TTL: &minute}, 1, 66 * time.Second}, + } { + if got := nextInterval(tc.decision, 0, tc.jitter); got != tc.want { + t.Fatalf("interval=%v want=%v", got, tc.want) + } + } +} + +func TestAggregateAllowlistAndFalsePrecedence(t *testing.T) { + unit := fixtureUnit(2) + prepareReceipts(unit) + now := time.Unix(100, 0) + setCondition(&unit.Status.Zone.Conditions, "Programmed", metav1.ConditionTrue, "VerificationConfigured", "", 1, now) + setCondition(&unit.Status.RecordSets[0].Conditions, "Programmed", metav1.ConditionUnknown, "DNSProbeFailed", "", 1, now) + setCondition(&unit.Status.RecordSets[1].Conditions, "Programmed", metav1.ConditionFalse, "DNSNameNotFound", "", 1, now) + setAggregate(unit, now) + assertProgrammed(t, unit.Status.Conditions, metav1.ConditionFalse, "ExternalResourceNotFound") + for _, reason := range []string{"DNSNoData", "ResolutionPartial", "ResolutionMismatch", "ResolutionInconsistent", "NameserverMismatch"} { + status, mapped := UnitReason(probe.Decision{Status: metav1.ConditionFalse, Reason: reason}) + if status != metav1.ConditionFalse || mapped != "ExternalResourceMismatch" { + t.Fatalf("mapping=%s/%s", status, mapped) + } + } + data := publicData(externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Mode: externalv1alpha1.ResolverModeRecursive}}, "www.example", false, probe.Decision{}, now, now.Add(time.Minute)) + var public externalv1alpha1.ExternalVerificationData + if err := json.Unmarshal(data.Data.Raw, &public); err != nil { + t.Fatal(err) + } + if public.NegativeTTLSeconds != nil || public.NegativeExpiryTime != nil { + t.Fatal("invented negative TTL") + } +} + +func TestZoneAuthorityEvidenceSharedAndBlocksRecords(t *testing.T) { + resolver := &fakeResolver{} + r, c, now, request := newFixture(t, 2, resolver) + var class dnsv1alpha1.ZoneClass + if err := c.Get(context.Background(), client.ObjectKey{Namespace: "app", Name: "external"}, &class); err != nil { + t.Fatal(err) + } + class.Spec.Parameters.Raw = []byte(`{"resolver":{"mode":"Recursive","servers":["192.0.2.53:53"]},"expectedNameservers":["ns.example.test"]}`) + if err := c.Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + wrong := false + zoneCalls := 0 + resolver.answer = func(input probe.ProbeRequest) (probe.ProbeResult, error) { + if input.ZoneOnly { + zoneCalls++ + ns := "ns.example.test" + if wrong { + ns = "wrong.example.test" + } + return probe.ProbeResult{Zone: &probe.ZoneEvidence{ + Zone: input.Zone, PolicyFingerprint: probe.Fingerprint(input.Policy), Authorities: []string{"192.0.2.54:53"}, + Views: []probe.AuthorityObservation{ + {ID: "recursive/192.0.2.53:53", Nameservers: []string{ns}, SOAOwner: input.Zone}, + {ID: "authority/192.0.2.54:53", Nameservers: []string{ns}, SOAOwner: input.Zone, Authoritative: true}, + }, + }}, nil + } + if input.Evidence == nil { + t.Fatal("per-Zone evidence was not shared") + } + result, err := matching(input) + authority := result.Views[0] + authority.ID, authority.Authoritative = "authority/192.0.2.54:53/via/192.0.2.53:53", true + result.Views = append(result.Views, authority) + return result, err + } + _, unit := reconcile(t, r, request) + if zoneCalls != 1 || len(resolver.requests) != 3 { + t.Fatalf("zoneCalls=%d probes=%d", zoneCalls, len(resolver.requests)) + } + assertProgrammed(t, unit.Status.Zone.Conditions, metav1.ConditionTrue, "NameserversVerified") + for _, receipt := range unit.Status.RecordSets { + assertProgrammed(t, receipt.Conditions, metav1.ConditionTrue, "ResolutionVerified") + } + wrong = true + *now = now.Add(30 * time.Second) + _, unit = reconcile(t, r, request) + if zoneCalls != 2 || len(resolver.requests) != 4 { + t.Fatal("failed authority check still probed child records") + } + for _, receipt := range unit.Status.RecordSets { + assertProgrammed(t, receipt.Conditions, metav1.ConditionFalse, "NameserverMismatch") + } + if len(unit.Status.Zone.NameServers) != 1 || unit.Status.Zone.NameServers[0] != "wrong.example.test" { + t.Fatal("expected NS was published as observed evidence") + } +} + +func TestNamespacePolicyChangeInvalidatesSuccess(t *testing.T) { + resolver := &fakeResolver{answer: matching} + r, c, _, request := newFixture(t, 1, resolver) + reconcile(t, r, request) + var namespace corev1.Namespace + if err := c.Get(context.Background(), client.ObjectKey{Name: "app"}, &namespace); err != nil { + t.Fatal(err) + } + namespace.Labels = map[string]string{"policy": "changed"} + if err := c.Update(context.Background(), &namespace); err != nil { + t.Fatal(err) + } + reconcile(t, r, request) + if len(resolver.requests) != 2 { + t.Fatal("namespace-policy fingerprint was not invalidated") + } +} + +func TestStaleConditionGenerationCannotUseScheduledSuccess(t *testing.T) { + resolver := &fakeResolver{answer: matching} + r, c, _, request := newFixture(t, 1, resolver) + _, unit := reconcile(t, r, request) + for i := range unit.Status.RecordSets[0].Conditions { + if unit.Status.RecordSets[0].Conditions[i].Type == "Programmed" { + unit.Status.RecordSets[0].Conditions[i].ObservedGeneration = 0 + } + } + if err := c.Status().Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, current := reconcile(t, r, request) + if len(resolver.requests) != 2 { + t.Fatal("stale condition reused a future scheduling receipt") + } + condition := assertProgrammed(t, current.Status.RecordSets[0].Conditions, metav1.ConditionTrue, "ResolutionVerified") + if condition.ObservedGeneration != 1 { + t.Fatal("stale result was rebadged instead of reprobed") + } +} diff --git a/internal/go/providers/external/controllers/zoneunit/model.go b/internal/go/providers/external/controllers/zoneunit/model.go new file mode 100644 index 0000000..a0a81eb --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/model.go @@ -0,0 +1,173 @@ +package zoneunit + +import ( + "encoding/json" + "slices" + "strings" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +const MaxNamesPerPass = 32 + +type schedule struct { + Fingerprint string `json:"fingerprint"` + Next time.Time `json:"next"` + Failures int `json:"failures,omitempty"` +} + +type schedulerState struct { + Policy string `json:"policy"` + Cursor string `json:"cursor,omitempty"` + Zone schedule `json:"zone"` + Names map[string]schedule `json:"names"` +} + +type nameGroup struct { + Name string + CheckedName string + SampledWildcard bool + Items []int + Expected probe.RRsets + Fingerprint string +} + +func groupNames(unit *dnsv1alpha1.ZoneUnit, policy string) []nameGroup { + byName := map[string][]int{} + for i, item := range unit.Spec.RecordSets { + if item.IsAllowed() && !item.DeletionRequested && item.RecordSetUID != "" { + byName[item.Name] = append(byName[item.Name], i) + } + } + groups := make([]nameGroup, 0, len(byName)) + for name, indices := range byName { + slices.SortFunc(indices, func(a, b int) int { + return strings.Compare(itemKey(unit.Spec.RecordSets[a]), itemKey(unit.Spec.RecordSets[b])) + }) + items := make([]dnsv1alpha1.ZoneUnitRecordSetSpec, 0, len(indices)) + group := nameGroup{Name: name, Items: indices} + for _, index := range indices { + item := unit.Spec.RecordSets[index] + items = append(items, item) + if item.A != nil { + group.Expected.A = append(group.Expected.A, item.A.Addresses...) + } + if item.AAAA != nil { + group.Expected.AAAA = append(group.Expected.AAAA, item.AAAA.Addresses...) + } + if item.CNAME != nil { + group.Expected.CNAME = probe.CanonicalName(item.CNAME.Target) + } + } + group.Expected.A, group.Expected.AAAA = probe.Addresses(group.Expected.A), probe.Addresses(group.Expected.AAAA) + group.Fingerprint = probe.Fingerprint(struct { + Policy string + Items []dnsv1alpha1.ZoneUnitRecordSetSpec + }{policy, items}) + group.CheckedName = unit.Spec.Zone.DomainName + if name != "@" { + group.CheckedName = name + "." + group.CheckedName + } + if name == "*" || strings.HasPrefix(name, "*.") { + group.SampledWildcard = true + // Resolver policy does not alter the witness: only the Zone and sorted claim tuple do. + tuple := make([]string, 0, len(items)) + for _, item := range items { + tuple = append(tuple, probe.Fingerprint(struct { + UID string + Generation int64 + }{string(item.RecordSetUID), item.ObservedGeneration})) + } + slices.Sort(tuple) + witness := probe.Fingerprint(struct { + UnitUID string + Zone dnsv1alpha1.ObjectReference + Claims []string + }{string(unit.UID), unit.Spec.Zone.Ref, tuple}) + group.CheckedName = "dns-verify-" + witness[:24] + strings.TrimPrefix(group.CheckedName, "*") + } + groups = append(groups, group) + } + slices.SortFunc(groups, func(a, b nameGroup) int { return strings.Compare(a.Name, b.Name) }) + return groups +} + +func itemKey(item dnsv1alpha1.ZoneUnitRecordSetSpec) string { + return item.RecordSetNamespace + "/" + item.RecordSetName +} + +func loadState(unit *dnsv1alpha1.ZoneUnit, policy string) schedulerState { + state := schedulerState{} + if unit.Status.Provider != nil { + _ = json.Unmarshal(unit.Status.Provider.State.Raw, &state) + } + if state.Policy != policy { + state = schedulerState{Policy: policy} + } + if state.Names == nil { + state.Names = map[string]schedule{} + } + return state +} + +func nextInterval(decision probe.Decision, failures int, jitter float64) time.Duration { + seconds := 30.0 + switch { + case decision.Status == metav1.ConditionUnknown: + steps := [...]float64{30, 60, 120, 300} + seconds = steps[min(max(failures-1, 0), len(steps)-1)] + case decision.Status == metav1.ConditionTrue && decision.TTL != nil: + seconds = float64(*decision.TTL) + case (decision.Reason == "DNSNameNotFound" || decision.Reason == "DNSNoData") && decision.NegativeTTL != nil: + seconds = float64(*decision.NegativeTTL) + } + seconds = max(30, min(300, seconds)) + seconds *= 1 + max(-1, min(1, jitter))*0.1 + return time.Duration(max(30, min(300, seconds)) * float64(time.Second)) +} + +func advance(previous schedule, fingerprint string, decision probe.Decision, now time.Time, jitter float64) schedule { + failures := 0 + if decision.Status == metav1.ConditionUnknown { + if previous.Fingerprint == fingerprint { + failures = previous.Failures + } + failures++ + } + return schedule{Fingerprint: fingerprint, Failures: failures, Next: now.Add(nextInterval(decision, failures, jitter))} +} + +func selectDue(groups []nameGroup, state schedulerState, now time.Time) []int { + indices := make([]int, 0, MaxNamesPerPass) + start := 0 + for start < len(groups) && groups[start].Name <= state.Cursor { + start++ + } + for offset := 0; offset < len(groups) && len(indices) < MaxNamesPerPass; offset++ { + index := (start + offset) % len(groups) + group := groups[index] + scheduled := state.Names[group.Name] + if scheduled.Fingerprint != group.Fingerprint || !scheduled.Next.After(now) { + indices = append(indices, index) + } + } + return indices +} + +func nextRequeue(groups []nameGroup, state schedulerState, now time.Time) time.Duration { + next := 300 * time.Second + if !state.Zone.Next.IsZero() { + next = min(next, state.Zone.Next.Sub(now)) + } + for _, group := range groups { + scheduled := state.Names[group.Name] + if scheduled.Fingerprint != group.Fingerprint || !scheduled.Next.After(now) { + return time.Second + } + next = min(next, scheduled.Next.Sub(now)) + } + return max(time.Second, next) +} diff --git a/internal/go/providers/external/controllers/zoneunit/status.go b/internal/go/providers/external/controllers/zoneunit/status.go new file mode 100644 index 0000000..8c0de7c --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/status.go @@ -0,0 +1,156 @@ +package zoneunit + +import ( + "encoding/json" + "slices" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +func setCondition(conditions *[]metav1.Condition, kind string, status metav1.ConditionStatus, reason, message string, generation int64, now time.Time) { + meta.SetStatusCondition(conditions, metav1.Condition{Type: kind, Status: status, Reason: reason, Message: message, ObservedGeneration: generation, LastTransitionTime: metav1.NewTime(now)}) +} + +func prepareReceipts(unit *dnsv1alpha1.ZoneUnit) { + old := make(map[string]dnsv1alpha1.ZoneUnitRecordSetStatus, len(unit.Status.RecordSets)) + for _, item := range unit.Status.RecordSets { + old[item.RecordSetNamespace+"/"+item.RecordSetName] = item + } + unit.Status.RecordSets = make([]dnsv1alpha1.ZoneUnitRecordSetStatus, len(unit.Spec.RecordSets)) + for i, item := range unit.Spec.RecordSets { + status := old[itemKey(item)] + if status.RecordSetUID != item.RecordSetUID || status.ObservedGeneration != item.ObservedGeneration { + status = dnsv1alpha1.ZoneUnitRecordSetStatus{} + } + status.RecordSetNamespace, status.RecordSetName, status.RecordSetUID, status.ObservedGeneration = item.RecordSetNamespace, item.RecordSetName, item.RecordSetUID, item.ObservedGeneration + status.DeletionCompleted = item.DeletionRequested && item.RecordSetUID != "" + if item.DeletionRequested || !item.IsAllowed() || item.RecordSetUID == "" { + status.Provider = nil + status.Conditions = nil + } + unit.Status.RecordSets[i] = status + } + if unit.Status.Zone == nil { + unit.Status.Zone = &dnsv1alpha1.ZoneUnitZoneStatus{} + } +} + +func resetProgrammed(status *dnsv1alpha1.ZoneUnitRecordSetStatus, now time.Time) { + status.Provider = nil + setCondition(&status.Conditions, "Programmed", metav1.ConditionUnknown, "Reconciling", "DNS verification is pending current inputs.", status.ObservedGeneration, now) +} + +func currentDecision(conditions []metav1.Condition) probe.Decision { + condition := meta.FindStatusCondition(conditions, "Programmed") + if condition == nil { + return probe.Decision{Status: metav1.ConditionUnknown, Reason: "Reconciling", Message: "DNS verification is pending."} + } + return probe.Decision{Status: condition.Status, Reason: condition.Reason, Message: condition.Message} +} + +func publicData(policy externalv1alpha1.ExternalZoneClassParameters, name string, sampled bool, decision probe.Decision, now, next time.Time) *dnsv1alpha1.ProviderStatus { + data := externalv1alpha1.ExternalVerificationData{ + ResolverMode: policy.Resolver.Mode, CheckedName: name, SampledWildcard: sampled, + Actual: probe.Preview(decision.Actual), Missing: probe.Preview(decision.Missing), Extra: probe.Preview(decision.Extra), + CNAMEChain: probe.Preview(decision.Chain), Views: probe.Preview(decision.Views), + ObservedTTLSeconds: decision.TTL, NegativeTTLSeconds: decision.NegativeTTL, + LastProbeTime: timePointer(now), NextProbeTime: timePointer(next), + } + if decision.NegativeTTL != nil { + data.NegativeExpiryTime = timePointer(now.Add(time.Duration(*decision.NegativeTTL) * time.Second)) + } + encoded, _ := json.Marshal(data) + return &dnsv1alpha1.ProviderStatus{Data: runtime.RawExtension{Raw: encoded}} +} + +func timePointer(value time.Time) *metav1.Time { result := metav1.NewTime(value); return &result } + +func setZoneResult(unit *dnsv1alpha1.ZoneUnit, policy externalv1alpha1.ExternalZoneClassParameters, decision probe.Decision, now, next time.Time) { + zone := unit.Status.Zone + zone.NameServers = slices.Clone(decision.Nameservers) + zone.Provider = publicData(policy, unit.Spec.Zone.DomainName, false, decision, now, next) + setCondition(&zone.Conditions, "Accepted", metav1.ConditionTrue, "Accepted", "External DNS Zone binding is accepted.", unit.Spec.Zone.ObservedGeneration, now) + setCondition(&zone.Conditions, "Programmed", decision.Status, decision.Reason, decision.Message, unit.Spec.Zone.ObservedGeneration, now) +} + +func setGroupResult(unit *dnsv1alpha1.ZoneUnit, group nameGroup, policy externalv1alpha1.ExternalZoneClassParameters, decision probe.Decision, now, next time.Time) { + data := publicData(policy, group.CheckedName, group.SampledWildcard, decision, now, next) + for _, index := range group.Items { + item := &unit.Status.RecordSets[index] + item.Provider = data + setCondition(&item.Conditions, "Accepted", metav1.ConditionTrue, "Accepted", "External DNS record intent is accepted.", item.ObservedGeneration, now) + setCondition(&item.Conditions, "Programmed", decision.Status, decision.Reason, decision.Message, item.ObservedGeneration, now) + } +} + +// UnitReason is the explicit boundary to core's fixed ZoneUnit reason allowlist. +func UnitReason(decision probe.Decision) (metav1.ConditionStatus, string) { + if decision.Status == metav1.ConditionTrue { + return metav1.ConditionTrue, "Programmed" + } + if decision.Reason == "DNSProbeFailed" { + return metav1.ConditionUnknown, "ProviderUnavailable" + } + if decision.Status == metav1.ConditionUnknown { + return metav1.ConditionUnknown, "Reconciling" + } + if decision.Reason == "DNSNameNotFound" { + return metav1.ConditionFalse, "ExternalResourceNotFound" + } + return metav1.ConditionFalse, "ExternalResourceMismatch" +} + +func setAggregate(unit *dnsv1alpha1.ZoneUnit, now time.Time) { + decision := currentDecision(unit.Status.Zone.Conditions) + for i, status := range unit.Status.RecordSets { + item := unit.Spec.RecordSets[i] + if !item.IsAllowed() || item.DeletionRequested { + continue + } + candidate := currentDecision(status.Conditions) + if decision.Status != metav1.ConditionFalse && (candidate.Status == metav1.ConditionFalse || (candidate.Status == metav1.ConditionUnknown && decision.Status == metav1.ConditionTrue)) { + decision = candidate + } + } + status, reason := UnitReason(decision) + setCondition(&unit.Status.Conditions, "Programmed", status, reason, decision.Message, unit.Generation, now) +} + +func (r *ZoneReconciler) reject(unit *dnsv1alpha1.ZoneUnit, rejected acceptance) { + now := r.Options.Clock() + unit.Status.Provider = nil + unit.Status.Zone.Provider = nil + unit.Status.Zone.NameServers = nil + setCondition(&unit.Status.Zone.Conditions, "Accepted", rejected.Status, rejected.Reason, rejected.Message, unit.Spec.Zone.ObservedGeneration, now) + setCondition(&unit.Status.Zone.Conditions, "Programmed", metav1.ConditionUnknown, "Reconciling", rejected.Message, unit.Spec.Zone.ObservedGeneration, now) + for i := range unit.Status.RecordSets { + if unit.Spec.RecordSets[i].DeletionRequested { + continue + } + setCondition(&unit.Status.RecordSets[i].Conditions, "Accepted", metav1.ConditionFalse, "ZoneNotAccepted", rejected.Message, unit.Status.RecordSets[i].ObservedGeneration, now) + resetProgrammed(&unit.Status.RecordSets[i], now) + } + setAggregate(unit, now) + r.updatePending(unit) +} + +func (r *ZoneReconciler) updatePending(unit *dnsv1alpha1.ZoneUnit) { + counts := map[string]int{} + for i, status := range unit.Status.RecordSets { + item := unit.Spec.RecordSets[i] + if !item.IsAllowed() || item.DeletionRequested { + continue + } + decision := currentDecision(status.Conditions) + if decision.Status != metav1.ConditionTrue { + counts[decision.Reason]++ + } + } + probe.SetPending(unit.Namespace+"/"+unit.Name, counts) +} diff --git a/internal/go/providers/external/controllers/zoneunit/watches.go b/internal/go/providers/external/controllers/zoneunit/watches.go new file mode 100644 index 0000000..288941b --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/watches.go @@ -0,0 +1,74 @@ +package zoneunit + +import ( + "context" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + corev1 "k8s.io/api/core/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/builder" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/predicate" + "sigs.k8s.io/controller-runtime/pkg/reconcile" +) + +func (r *ZoneReconciler) SetupWithManager(mgr ctrl.Manager) error { + r.Reader = mgr.GetAPIReader() + return ctrl.NewControllerManagedBy(mgr).Named("external-zoneunit"). + WithOptions(controller.Options{MaxConcurrentReconciles: 4}). + For(&dnsv1alpha1.ZoneUnit{}, builder.WithPredicates(predicate.Funcs{UpdateFunc: func(e event.UpdateEvent) bool { + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() || !e.ObjectOld.GetDeletionTimestamp().Equal(e.ObjectNew.GetDeletionTimestamp()) + }})). + Watches(&dnsv1alpha1.ZoneClass{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []reconcile.Request { + return r.mapUnits(ctx, func(unit *dnsv1alpha1.ZoneUnit) bool { + ref := unit.Spec.Zone.ZoneClassRef + return ref.Namespace == obj.GetNamespace() && ref.Name == obj.GetName() + }) + })). + Watches(&dnsv1alpha1.Provider{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []reconcile.Request { + return r.mapUnits(ctx, func(unit *dnsv1alpha1.ZoneUnit) bool { return unit.Spec.Provider.Name == obj.GetName() }) + }), builder.WithPredicates(predicate.GenerationChangedPredicate{})). + Watches(&corev1.Namespace{}, handler.EnqueueRequestsFromMapFunc(func(ctx context.Context, obj client.Object) []reconcile.Request { + return r.mapUnits(ctx, func(unit *dnsv1alpha1.ZoneUnit) bool { + if unit.Spec.Zone.Ref.Namespace == obj.GetName() { + return true + } + for _, item := range unit.Spec.RecordSets { + if item.RecordSetNamespace == obj.GetName() { + return true + } + } + return false + }) + }), builder.WithPredicates(predicate.LabelChangedPredicate{})). + Complete(r) +} + +func (r *ZoneReconciler) mapUnits(ctx context.Context, matches func(*dnsv1alpha1.ZoneUnit) bool) []reconcile.Request { + // Real API pagination bounds each dependency read; cache List does not provide + // continuation tokens and custom informer indexes are not API field selectors. + reader := r.Reader + if reader == nil { + reader = r.Client + } + var requests []reconcile.Request + continuation := "" + for { + var units dnsv1alpha1.ZoneUnitList + if err := reader.List(ctx, &units, client.Limit(256), client.Continue(continuation)); err != nil { + return nil + } + for i := range units.Items { + if matches(&units.Items[i]) { + requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(&units.Items[i])}) + } + } + continuation = units.Continue + if continuation == "" { + return requests + } + } +} diff --git a/internal/go/providers/external/conversion/handler.go b/internal/go/providers/external/conversion/handler.go new file mode 100644 index 0000000..9c1ec6b --- /dev/null +++ b/internal/go/providers/external/conversion/handler.go @@ -0,0 +1,141 @@ +package conversion + +import ( + "encoding/json" + "fmt" + "net/http" + "net/netip" + "slices" + "strings" + + "github.com/appthrust/dns-api/internal/go/providers/external/config" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + conversionv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/conversion/v1alpha1" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +const ( + GroupName = "endpoint.external.dns.appthrust.io" + Version = "v1alpha1" + Resource = "endpointrecordsetconversions" +) + +type Handler struct{} + +func NewHandler() *Handler { return &Handler{} } + +func (h *Handler) Register(mux interface{ Register(string, http.Handler) }) { + mux.Register("/apis/"+GroupName+"/"+Version, h) + mux.Register("/apis/"+GroupName+"/"+Version+"/"+Resource, h) +} + +func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { + path := "/apis/" + GroupName + "/" + Version + switch { + case r.Method == http.MethodGet && r.URL.Path == path: + writeJSON(w, http.StatusOK, metav1.APIResourceList{ + TypeMeta: metav1.TypeMeta{APIVersion: "v1", Kind: "APIResourceList"}, + GroupVersion: GroupName + "/" + Version, + APIResources: []metav1.APIResource{{Name: Resource, Kind: "EndpointRecordSetConversion", Verbs: []string{"create"}}}, + }) + case r.Method == http.MethodPost && r.URL.Path == path+"/"+Resource: + defer r.Body.Close() + var request conversionv1alpha1.EndpointRecordSetConversion + response := conversionv1alpha1.EndpointRecordSetConversion{TypeMeta: metav1.TypeMeta{APIVersion: GroupName + "/" + Version, Kind: "EndpointRecordSetConversion"}} + if err := json.NewDecoder(r.Body).Decode(&request); err != nil { + response.Status.Result = failure("InvalidInput", "request body is malformed") + writeJSON(w, http.StatusBadRequest, response) + return + } + response.ObjectMeta, response.Spec, response.Status.UID = request.ObjectMeta, request.Spec, request.Spec.UID + fragments, result := convertEndpoint(request.Spec.Input) + response.Status.Result = result + if len(fragments) > 0 { + response.Status.Output = &endpointv1alpha1.EndpointRecordSetConversionOutput{Fragments: fragments} + } + writeJSON(w, http.StatusCreated, response) + default: + http.NotFound(w, r) + } +} + +func convertEndpoint(input endpointv1alpha1.EndpointRecordSetConversionInput) ([]endpointv1alpha1.RecordSetSpecFragment, conversionv1alpha1.EndpointRecordSetConversionResult) { + if input.Name == "" { + return nil, failure("InvalidInput", "input.name is required") + } + if len(input.Targets) == 0 { + return nil, failure("InvalidInput", "input.targets must not be empty") + } + var ipv4, ipv6, hosts []string + for _, target := range input.Targets { + switch target.Type { + case endpointv1alpha1.EndpointTargetTypeIPAddress: + ip, err := netip.ParseAddr(target.Value) + if err != nil || ip.Zone() != "" { + return nil, failure("InvalidInput", fmt.Sprintf("target %q is not a valid IP address", target.Value)) + } + ip = ip.Unmap() + if ip.Is4() { + ipv4 = append(ipv4, ip.String()) + } else { + ipv6 = append(ipv6, ip.String()) + } + case endpointv1alpha1.EndpointTargetTypeHostname: + name := strings.TrimSuffix(strings.ToLower(target.Value), ".") + if !config.ValidName(name) { + return nil, failure("InvalidInput", "hostname target must be a valid DNS name") + } + hosts = append(hosts, name) + default: + return nil, failure("InvalidInput", "unsupported target type "+string(target.Type)) + } + } + slices.Sort(ipv4) + ipv4 = slices.Compact(ipv4) + slices.Sort(ipv6) + ipv6 = slices.Compact(ipv6) + slices.Sort(hosts) + hosts = slices.Compact(hosts) + if len(hosts) > 0 && len(ipv4)+len(ipv6) > 0 { + return nil, failure("UnsupportedTarget", "External endpoint conversion does not support mixed hostname and IP targets") + } + if len(hosts) > 1 { + return nil, failure("UnsupportedTarget", "External endpoint conversion requires exactly one hostname target") + } + ttl := int32(300) + if len(hosts) == 1 { + if input.Name == "@" { + return nil, failure("UnsupportedTarget", "External endpoint conversion does not support CNAME at the zone apex") + } + if input.PreferredRecordType != "" && input.PreferredRecordType != endpointv1alpha1.EndpointRecordSetTypeCNAME { + return nil, failure("UnsupportedTarget", "preferredRecordType must be CNAME for a hostname target") + } + return []endpointv1alpha1.RecordSetSpecFragment{{Name: input.Name, Type: endpointv1alpha1.EndpointRecordSetTypeCNAME, TTL: &ttl, CNAME: &dnsv1alpha1.CNAMERecordSet{Target: hosts[0]}}}, success() + } + if input.PreferredRecordType != "" && !((input.PreferredRecordType == endpointv1alpha1.EndpointRecordSetTypeA && len(ipv6) == 0) || (input.PreferredRecordType == endpointv1alpha1.EndpointRecordSetTypeAAAA && len(ipv4) == 0)) { + return nil, failure("UnsupportedTarget", "preferredRecordType is incompatible with IP targets") + } + fragments := make([]endpointv1alpha1.RecordSetSpecFragment, 0, 2) + if len(ipv4) > 0 { + fragments = append(fragments, endpointv1alpha1.RecordSetSpecFragment{Name: input.Name, Type: endpointv1alpha1.EndpointRecordSetTypeA, TTL: &ttl, A: &dnsv1alpha1.ARecordSet{Addresses: ipv4}}) + } + if len(ipv6) > 0 { + fragments = append(fragments, endpointv1alpha1.RecordSetSpecFragment{Name: input.Name, Type: endpointv1alpha1.EndpointRecordSetTypeAAAA, TTL: &ttl, AAAA: &dnsv1alpha1.AAAARecordSet{Addresses: ipv6}}) + } + return fragments, success() +} + +func failure(reason, message string) conversionv1alpha1.EndpointRecordSetConversionResult { + return conversionv1alpha1.EndpointRecordSetConversionResult{Status: "Failure", Reason: reason, Message: message, Retryable: false} +} + +func success() conversionv1alpha1.EndpointRecordSetConversionResult { + return conversionv1alpha1.EndpointRecordSetConversionResult{Status: "Success", Reason: "Converted", Message: "converted endpoint targets to external DNS record sets"} +} + +func writeJSON(w http.ResponseWriter, status int, value any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(value) +} diff --git a/internal/go/providers/external/conversion/handler_test.go b/internal/go/providers/external/conversion/handler_test.go new file mode 100644 index 0000000..6ae6a8c --- /dev/null +++ b/internal/go/providers/external/conversion/handler_test.go @@ -0,0 +1,91 @@ +package conversion + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + conversionv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/conversion/v1alpha1" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" +) + +func TestHandlerFramingAndCanonicalTargets(t *testing.T) { + input := endpointv1alpha1.EndpointRecordSetConversionInput{Name: "*.project", Targets: []endpointv1alpha1.EndpointTarget{ + {Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "2001:0db8:0:0::1"}, + {Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.2"}, + {Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.1"}, + {Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "2001:db8::1"}, + }} + review := conversionv1alpha1.EndpointRecordSetConversion{Spec: conversionv1alpha1.EndpointRecordSetConversionSpec{UID: "request-1", Input: input}} + body, _ := json.Marshal(review) + response := httptest.NewRecorder() + NewHandler().ServeHTTP(response, httptest.NewRequest(http.MethodPost, "/apis/"+GroupName+"/"+Version+"/"+Resource, bytes.NewReader(body))) + var got conversionv1alpha1.EndpointRecordSetConversion + if err := json.Unmarshal(response.Body.Bytes(), &got); err != nil { + t.Fatal(err) + } + if response.Code != http.StatusCreated || got.Status.UID != "request-1" || got.Status.Result.Reason != "Converted" || got.Status.Output == nil { + t.Fatalf("response = %d %#v", response.Code, got) + } + fragments := got.Status.Output.Fragments + if len(fragments) != 2 || fragments[0].A.Addresses[0] != "192.0.2.1" || len(fragments[1].AAAA.Addresses) != 1 || fragments[1].AAAA.Addresses[0] != "2001:db8::1" { + t.Fatalf("fragments = %#v", fragments) + } + for _, fragment := range fragments { + if fragment.Name != input.Name || fragment.TTL == nil || *fragment.TTL != 300 || len(fragment.Options.Raw) != 0 { + t.Fatalf("fragment = %#v", fragment) + } + } +} + +func TestHostnameConversionAndFailures(t *testing.T) { + host := endpointv1alpha1.EndpointTarget{Type: endpointv1alpha1.EndpointTargetTypeHostname, Value: "TARGET.EXAMPLE."} + ip := endpointv1alpha1.EndpointTarget{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.1"} + cases := []struct { + name string + input endpointv1alpha1.EndpointRecordSetConversionInput + reason, message string + }{ + {"hostname", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "www", Targets: []endpointv1alpha1.EndpointTarget{host, host}}, "Converted", ""}, + {"apex", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "@", Targets: []endpointv1alpha1.EndpointTarget{host}}, "UnsupportedTarget", "External endpoint conversion does not support CNAME at the zone apex"}, + {"mixed", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "www", Targets: []endpointv1alpha1.EndpointTarget{host, ip}}, "UnsupportedTarget", "External endpoint conversion does not support mixed hostname and IP targets"}, + {"multiple", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "www", Targets: []endpointv1alpha1.EndpointTarget{host, {Type: endpointv1alpha1.EndpointTargetTypeHostname, Value: "other.example"}}}, "UnsupportedTarget", "External endpoint conversion requires exactly one hostname target"}, + {"invalid address", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "www", Targets: []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "bad"}}}, "InvalidInput", ""}, + {"preference", endpointv1alpha1.EndpointRecordSetConversionInput{Name: "www", Targets: []endpointv1alpha1.EndpointTarget{ip}, PreferredRecordType: endpointv1alpha1.EndpointRecordSetTypeCNAME}, "UnsupportedTarget", ""}, + {"missing", endpointv1alpha1.EndpointRecordSetConversionInput{}, "InvalidInput", ""}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + fragments, result := convertEndpoint(tc.input) + if result.Reason != tc.reason || result.Retryable || (tc.message != "" && result.Message != tc.message) { + t.Fatalf("result = %#v", result) + } + if result.Status == "Success" { + if len(fragments) != 1 || fragments[0].CNAME.Target != "target.example" || *fragments[0].TTL != 300 { + t.Fatalf("fragments = %#v", fragments) + } + } else if len(fragments) != 0 { + t.Fatal("failure emitted fragments") + } + }) + } +} + +func TestDiscoveryAndMalformedRequest(t *testing.T) { + for _, tc := range []struct { + method, suffix, body string + code int + }{ + {http.MethodGet, "", "", http.StatusOK}, + {http.MethodGet, "/" + Resource, "", http.StatusNotFound}, + {http.MethodPost, "/" + Resource, "{", http.StatusBadRequest}, + } { + response := httptest.NewRecorder() + NewHandler().ServeHTTP(response, httptest.NewRequest(tc.method, "/apis/"+GroupName+"/"+Version+tc.suffix, bytes.NewBufferString(tc.body))) + if response.Code != tc.code { + t.Fatalf("status = %d want %d", response.Code, tc.code) + } + } +} diff --git a/internal/go/providers/external/manifest/capability.yaml b/internal/go/providers/external/manifest/capability.yaml new file mode 100644 index 0000000..9d07fd3 --- /dev/null +++ b/internal/go/providers/external/manifest/capability.yaml @@ -0,0 +1,12 @@ +apiVersion: endpoint.dns.appthrust.io/v1alpha1 +kind: EndpointProviderCapability +metadata: + name: external-v1alpha1 +spec: + provider: + name: external.dns.appthrust.io + version: v1alpha1 + conversion: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + resource: endpointrecordsetconversions diff --git a/internal/go/providers/external/manifest/manifest.go b/internal/go/providers/external/manifest/manifest.go new file mode 100644 index 0000000..a462f9e --- /dev/null +++ b/internal/go/providers/external/manifest/manifest.go @@ -0,0 +1,23 @@ +// Package manifest contains the external Provider and endpoint capability declarations. +package manifest + +import ( + _ "embed" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "sigs.k8s.io/yaml" +) + +//go:embed provider.yaml +var ProviderYAML []byte + +//go:embed capability.yaml +var CapabilityYAML []byte + +func Provider() (*dnsv1alpha1.Provider, error) { + var provider dnsv1alpha1.Provider + if err := yaml.UnmarshalStrict(ProviderYAML, &provider); err != nil { + return nil, err + } + return &provider, nil +} diff --git a/internal/go/providers/external/manifest/manifest_test.go b/internal/go/providers/external/manifest/manifest_test.go new file mode 100644 index 0000000..5c77755 --- /dev/null +++ b/internal/go/providers/external/manifest/manifest_test.go @@ -0,0 +1,89 @@ +package manifest + +import ( + "encoding/json" + "testing" + + "github.com/appthrust/dns-api/internal/go/core/celruntime" + "github.com/appthrust/dns-api/internal/go/core/providercontract" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "github.com/google/cel-go/cel" + "k8s.io/apimachinery/pkg/runtime" +) + +func TestExternalDeclarationAndCEL(t *testing.T) { + provider, err := Provider() + if err != nil { + t.Fatal(err) + } + version := &provider.Spec.Versions[0] + if len(provider.Spec.Versions) != 1 || version.Identity != nil || !version.Storage || !version.Served || len(version.RecordSet.SupportedTypes) != 3 { + t.Fatalf("provider=%#v", provider) + } + if len(version.Zone.AdditionalConditionReasons) != 5 || len(version.RecordSet.AdditionalConditionReasons) != 8 { + t.Fatal("condition reason tables differ from design") + } + for _, rule := range version.ZoneClass.ValidationRules { + if err := celruntime.CompileBool(rule.Rule, cel.Variable("self", cel.DynType)); err != nil { + t.Fatalf("compile %q: %v", rule.Rule, err) + } + } + cases := []struct { + name, parameters string + valid bool + }{ + {"default", `{}`, true}, + {"empty resolver", `{"resolver":{}}`, true}, + {"recursive", `{"resolver":{"mode":"Recursive","servers":["192.0.2.53:53","[2001:db8::53]:5353"]},"expectedNameservers":["ns.example.test"]}`, true}, + {"missing servers", `{"resolver":{"mode":"Recursive"}}`, false}, + {"cluster servers", `{"resolver":{"mode":"Cluster","servers":["192.0.2.53:53"]}}`, false}, + {"bad IP", `{"resolver":{"mode":"Recursive","servers":["999.1.1.1:53"]}}`, false}, + {"port overflow", `{"resolver":{"mode":"Recursive","servers":["192.0.2.1:65536"]}}`, false}, + {"leading zero port", `{"resolver":{"mode":"Recursive","servers":["192.0.2.1:053"]}}`, false}, + {"duplicate", `{"resolver":{"mode":"Recursive","servers":["192.0.2.1:53","192.0.2.1:53"]}}`, false}, + {"NS IP", `{"expectedNameservers":["192.0.2.1"]}`, false}, + {"noncanonical NS", `{"expectedNameservers":["NS.example.test."]}`, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + class := &dnsv1alpha1.ZoneClass{Spec: dnsv1alpha1.ZoneClassSpec{Parameters: runtime.RawExtension{Raw: []byte(tc.parameters)}}} + _, schemaErr := providercontract.ConvertZoneClassParametersToStorage(provider, version, class) + valid := schemaErr == nil + if valid { + var params map[string]interface{} + if err := json.Unmarshal([]byte(tc.parameters), ¶ms); err != nil { + t.Fatal(err) + } + self := map[string]interface{}{"spec": map[string]interface{}{"parameters": params}} + for _, rule := range version.ZoneClass.ValidationRules { + ok, err := celruntime.EvalBool(rule.Rule, map[string]interface{}{"self": self}, cel.Variable("self", cel.DynType)) + if err != nil || !ok { + valid = false + break + } + } + } + if valid != tc.valid { + t.Fatalf("valid=%v want=%v schemaError=%v", valid, tc.valid, schemaErr) + } + }) + } +} + +func TestNonemptyProviderPayloadsRejected(t *testing.T) { + provider, err := Provider() + if err != nil { + t.Fatal(err) + } + version := &provider.Spec.Versions[0] + for _, target := range []providercontract.PayloadTarget{providercontract.TargetZone, providercontract.TargetRecordSet} { + input := providercontract.ConversionInput{Provider: provider, FromVersion: version, ToVersion: version, Target: target, Payload: providercontract.Payload{Adoption: runtime.RawExtension{Raw: []byte(`{"enabled":true}`)}}} + if _, err := providercontract.ConvertProviderPayloadToStorage(input); err == nil { + t.Fatalf("nonempty adoption accepted for %s", target) + } + } + input := providercontract.ConversionInput{Provider: provider, FromVersion: version, ToVersion: version, Target: providercontract.TargetRecordSet, Payload: providercontract.Payload{Options: runtime.RawExtension{Raw: []byte(`{"alias":{}}`)}}} + if _, err := providercontract.ConvertProviderPayloadToStorage(input); err == nil { + t.Fatal("nonempty options accepted") + } +} diff --git a/internal/go/providers/external/manifest/provider.yaml b/internal/go/providers/external/manifest/provider.yaml new file mode 100644 index 0000000..a143394 --- /dev/null +++ b/internal/go/providers/external/manifest/provider.yaml @@ -0,0 +1,159 @@ +apiVersion: dns.appthrust.io/v1alpha1 +kind: Provider +metadata: + name: external.dns.appthrust.io +spec: + display: + name: External DNS + description: Verifies customer-managed DNS without changing DNS records. + versions: + - name: v1alpha1 + served: true + storage: true + zoneClass: + schemas: + parameters: + openAPIV3Schema: + type: object + description: DNS resolver policy for verification without DNS writes. + additionalProperties: false + properties: + resolver: + type: object + description: Resolver view used for every verification query. + additionalProperties: false + properties: + mode: + type: string + description: Cluster uses pod DNS; Recursive uses every configured server. + enum: [Cluster, Recursive] + servers: + type: array + description: Explicit recursive DNS IP endpoints; never authoritative NS names. + maxItems: 3 + uniqueItems: true + items: + type: string + minLength: 3 + maxLength: 64 + pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' + expectedNameservers: + type: array + description: Expected authoritative nameservers for the exact Zone domain. + maxItems: 16 + uniqueItems: true + items: + type: string + minLength: 1 + maxLength: 253 + pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' + validationRules: + - rule: >- + has(self.spec.parameters.resolver) && + has(self.spec.parameters.resolver.mode) && + self.spec.parameters.resolver.mode == 'Recursive' + ? (has(self.spec.parameters.resolver.servers) && size(self.spec.parameters.resolver.servers) > 0) + : (!has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || size(self.spec.parameters.resolver.servers) == 0) + message: resolver.servers must be nonempty for Recursive and empty for Cluster + - rule: >- + !has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || + self.spec.parameters.resolver.servers.all(s, + cel.bind(host, s.startsWith('[') ? s.split(']')[0].substring(1) : s.split(':')[0], + cel.bind(port, s.startsWith('[') ? s.split(']:')[1] : s.split(':')[1], + isIP(host) && ip.isCanonical(host) && + (s.startsWith('[') ? ip(host).family() == 6 : ip(host).family() == 4) && + int(port) >= 1 && int(port) <= 65535))) + message: resolver.servers must contain canonical IP addresses and ports from 1 through 65535 + - rule: >- + !has(self.spec.parameters.expectedNameservers) || + self.spec.parameters.expectedNameservers.all(n, !isIP(n)) + message: expectedNameservers must contain DNS names, not IP addresses + zone: + schemas: + adoption: &empty + openAPIV3Schema: + type: object + additionalProperties: false + properties: {} + statusProviderData: &status + openAPIV3Schema: + type: object + description: Bounded read-only DNS verification evidence; not DNS withdrawal proof. + additionalProperties: false + properties: + resolverMode: + type: string + description: Selected recursive resolver view policy. + enum: [Cluster, Recursive] + checkedName: + type: string + description: Absolute DNS owner checked, without the root dot. + maxLength: 253 + sampledWildcard: + type: boolean + description: Whether checkedName is a stable concrete wildcard witness. + actual: &preview + type: object + description: Sorted bounded diagnostic preview; equality uses the complete observation. + additionalProperties: false + required: [total, truncated] + properties: + values: + type: array + description: Up to sixteen typed values or diagnostic entries. + maxItems: 16 + items: + type: string + maxLength: 512 + total: + type: integer + description: Total entries before truncation. + minimum: 0 + truncated: + type: boolean + description: Whether values omits entries from the complete observation. + missing: *preview + extra: *preview + cnameChain: *preview + views: *preview + observedTTLSeconds: + type: integer + description: Minimum relevant positive TTL in seconds, when observed. + minimum: 0 + negativeTTLSeconds: + type: integer + description: Minimum of negative SOA TTL and SOA MINIMUM, when observed. + minimum: 0 + negativeExpiryTime: + type: string + format: date-time + description: Negative cache expiry suggested by the observed SOA, not the polling cap. + lastProbeTime: + type: string + format: date-time + description: Last completed attempt, including failures. + nextProbeTime: + type: string + format: date-time + description: Approximate next planned attempt; queue load can delay it. + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: VerificationConfigured, description: Verifier configured without authority checks.} + - {conditionType: Programmed, status: 'True', reason: NameserversVerified, description: Every required view verified the exact apex and nameservers.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Observed apex or nameservers differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordSet: + supportedTypes: [A, AAAA, CNAME] + schemas: + options: *empty + adoption: *empty + statusProviderData: *status + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: ResolutionVerified, description: Exact targets verified in every required view.} + - {conditionType: Programmed, status: 'False', reason: DNSNameNotFound, description: Required view returned NXDOMAIN.} + - {conditionType: Programmed, status: 'False', reason: DNSNoData, description: Required record type has no answers.} + - {conditionType: Programmed, status: 'False', reason: ResolutionPartial, description: A nonempty proper subset of targets was observed.} + - {conditionType: Programmed, status: 'False', reason: ResolutionMismatch, description: Targets or CNAME chain differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} diff --git a/internal/go/providers/external/probe/evaluate.go b/internal/go/providers/external/probe/evaluate.go new file mode 100644 index 0000000..21778c8 --- /dev/null +++ b/internal/go/providers/external/probe/evaluate.go @@ -0,0 +1,300 @@ +package probe + +import ( + "fmt" + "slices" + "strings" + + "github.com/appthrust/dns-api/internal/go/providers/external/config" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "github.com/miekg/dns" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type Decision struct { + Status metav1.ConditionStatus + Reason string + Message string + Actual []string + Missing []string + Extra []string + Chain []string + Views []string + Nameservers []string + TTL *uint32 + NegativeTTL *uint32 +} + +func (d Decision) Outcome() string { + if d.Status == metav1.ConditionUnknown { + return "failed" + } + if d.Status == metav1.ConditionTrue { + return "verified" + } + if d.Reason == "DNSNameNotFound" || d.Reason == "DNSNoData" { + return "negative" + } + return "mismatch" +} + +func Failed(name, view, failure string) Decision { + switch failure { + case "timeout", "SERVFAIL", "REFUSED", "transport", "invalid-response", "resolver-config", "chain-limit", "probe-limit": + default: + failure = "transport" + } + return Decision{Status: metav1.ConditionUnknown, Reason: "DNSProbeFailed", Message: bounded(fmt.Sprintf("DNS verification unavailable for %s from %s: %s.", name, view, failure))} +} + +func EvaluateZone(request ProbeRequest, evidence *ZoneEvidence) Decision { + return evaluateZone(request, evidence, true) +} + +func evaluateZone(request ProbeRequest, evidence *ZoneEvidence, requireAuthorities bool) Decision { + if len(request.Policy.ExpectedNameservers) == 0 { + return Decision{Status: metav1.ConditionTrue, Reason: "VerificationConfigured", Message: bounded(fmt.Sprintf("DNS verifier configured for %s; authoritative nameserver checks were not requested.", request.Zone))} + } + if evidence == nil || len(evidence.Views) < len(request.Policy.Resolver.Servers) || len(evidence.Views) == 0 { + return Failed(request.Zone, "required-views", "invalid-response") + } + for _, server := range request.Policy.Resolver.Servers { + if !slices.ContainsFunc(evidence.Views, func(view AuthorityObservation) bool { return view.ID == "recursive/"+server }) { + return Failed(request.Zone, "recursive/"+server, "invalid-response") + } + } + views := slices.Clone(evidence.Views) + slices.SortFunc(views, func(a, b AuthorityObservation) int { return strings.Compare(a.ID, b.ID) }) + decision := Decision{Status: metav1.ConditionTrue, Reason: "NameserversVerified"} + for _, view := range views { + decision.Views = append(decision.Views, view.ID) + if view.Failure != "" { + return Failed(request.Zone, view.ID, view.Failure) + } + for _, name := range view.Nameservers { + if !config.ValidName(CanonicalName(name)) { + return Failed(request.Zone, view.ID, "invalid-response") + } + } + decision.TTL = minTTL(decision.TTL, view.TTL) + decision.NegativeTTL = minTTL(decision.NegativeTTL, view.NegativeTTL) + } + first := views[0] + decision.Nameservers = Names(first.Nameservers) + decision.Actual = typed("NS", decision.Nameservers) + for _, view := range views[1:] { + if !slices.Equal(decision.Nameservers, Names(view.Nameservers)) || CanonicalName(view.SOAOwner) != CanonicalName(first.SOAOwner) { + return inconsistent(request.Zone, decision) + } + } + if CanonicalName(first.SOAOwner) != request.Zone || !slices.Equal(decision.Nameservers, Names(request.Policy.ExpectedNameservers)) { + decision.Status, decision.Reason = metav1.ConditionFalse, "NameserverMismatch" + decision.Missing, decision.Extra = difference(typed("NS", Names(request.Policy.ExpectedNameservers)), decision.Actual) + decision.Message = bounded(fmt.Sprintf("DNS %s nameservers differ: expected %s; observed %s.", request.Zone, preview(Names(request.Policy.ExpectedNameservers)), preview(decision.Nameservers))) + return decision + } + if requireAuthorities && len(evidence.Authorities) == 0 { + return Failed(request.Zone, "required-authorities", "invalid-response") + } + if requireAuthorities { + for _, authority := range evidence.Authorities { + found := false + for _, view := range views { + if view.ID == "authority/"+authority && view.Authoritative { + found = true + break + } + } + if !found { + return Failed(request.Zone, "authority/"+authority, "invalid-response") + } + } + } + decision.Message = bounded(fmt.Sprintf("DNS %s nameservers match expectations in every required view; no DNS changes were made.", request.Zone)) + return decision +} + +func Evaluate(request ProbeRequest, result ProbeResult) Decision { + if len(result.Views) == 0 || len(result.Views) < len(request.Policy.Resolver.Servers) { + return Failed(request.Name, "required-views", "invalid-response") + } + for _, server := range request.Policy.Resolver.Servers { + if !slices.ContainsFunc(result.Views, func(view View) bool { return view.ID == "recursive/"+server }) { + return Failed(request.Name, "recursive/"+server, "invalid-response") + } + } + if len(request.Policy.ExpectedNameservers) != 0 { + if request.Evidence == nil || len(request.Evidence.Authorities) == 0 { + return Failed(request.Name, "required-authorities", "invalid-response") + } + for _, authority := range request.Evidence.Authorities { + if !slices.ContainsFunc(result.Views, func(view View) bool { + return strings.HasPrefix(view.ID, "authority/"+authority+"/via/") && view.Authoritative + }) { + return Failed(request.Name, "authority/"+authority, "invalid-response") + } + } + } + views := slices.Clone(result.Views) + slices.SortFunc(views, func(a, b View) int { return strings.Compare(a.ID, b.ID) }) + decision := Decision{Status: metav1.ConditionTrue, Reason: "ResolutionVerified"} + for i := range views { + view := &views[i] + view.A, view.AAAA = Addresses(view.A), Addresses(view.AAAA) + view.Chain = slices.Clone(view.Chain) + seen := map[string]bool{CanonicalName(request.Name): true} + for j, name := range view.Chain { + name = CanonicalName(name) + view.Chain[j] = name + if seen[name] { + view.Mismatch = true + } + seen[name] = true + } + if len(view.Chain) > MaxCNAMEs { + view.Failure = "chain-limit" + } + if view.Failure != "" { + return Failed(request.Name, view.ID, view.Failure) + } + if view.RCode != dns.RcodeSuccess && view.RCode != dns.RcodeNameError { + failure := "invalid-response" + if view.RCode == dns.RcodeServerFailure { + failure = "SERVFAIL" + } + if view.RCode == dns.RcodeRefused { + failure = "REFUSED" + } + return Failed(request.Name, view.ID, failure) + } + decision.Views = append(decision.Views, view.ID) + decision.TTL = minTTL(decision.TTL, view.TTL) + decision.NegativeTTL = minTTL(decision.NegativeTTL, view.NegativeTTL) + } + if request.Evidence != nil { + for _, view := range request.Evidence.Views { + decision.TTL = minTTL(decision.TTL, view.TTL) + } + } + first := views[0] + decision.Actual = append(typed("A", first.A), typed("AAAA", first.AAAA)...) + decision.Chain = first.Chain + for _, view := range views[1:] { + if view.RCode != first.RCode || view.Mismatch != first.Mismatch || !slices.Equal(view.A, first.A) || !slices.Equal(view.AAAA, first.AAAA) || !slices.Equal(view.Chain, first.Chain) { + return inconsistent(request.Name, decision) + } + } + expected := append(typed("A", Addresses(request.Expected.A)), typed("AAAA", Addresses(request.Expected.AAAA))...) + if request.Expected.CNAME != "" { + expected = typed("CNAME", []string{CanonicalName(request.Expected.CNAME)}) + decision.Actual = nil + if len(first.Chain) > 0 { + decision.Actual = typed("CNAME", first.Chain[:1]) + } + } + decision.Missing, decision.Extra = difference(expected, decision.Actual) + if first.Mismatch { + return mismatch(request.Name, decision) + } + if first.RCode == dns.RcodeNameError { + decision.Status, decision.Reason = metav1.ConditionFalse, "DNSNameNotFound" + decision.Message = bounded(fmt.Sprintf("DNS %s returned NXDOMAIN from %s; expected %s.", request.Name, first.ID, preview(expected))) + return decision + } + if len(decision.Extra) != 0 { + return mismatch(request.Name, decision) + } + if request.Expected.CNAME != "" && len(decision.Missing) != 0 { + return mismatch(request.Name, decision) + } + qtype := "" + if len(request.Expected.A) > 0 && len(first.A) == 0 { + qtype = "A" + } + if qtype == "" && len(request.Expected.AAAA) > 0 && len(first.AAAA) == 0 { + qtype = "AAAA" + } + if request.Expected.CNAME != "" && len(first.A)+len(first.AAAA) == 0 { + qtype = "A or AAAA" + } + if qtype != "" { + decision.Status, decision.Reason = metav1.ConditionFalse, "DNSNoData" + decision.Message = bounded(fmt.Sprintf("DNS %s returned NOERROR with no %s answers; expected %s.", request.Name, qtype, preview(expected))) + return decision + } + if len(decision.Missing) != 0 { + decision.Status, decision.Reason = metav1.ConditionFalse, "ResolutionPartial" + decision.Message = bounded(fmt.Sprintf("DNS %s is missing expected targets %s; observed %s.", request.Name, preview(decision.Missing), preview(decision.Actual))) + return decision + } + decision.Message = bounded(fmt.Sprintf("DNS %s matches expected targets in every required resolver view; no DNS changes were made.", request.Name)) + return decision +} + +func inconsistent(name string, decision Decision) Decision { + decision.Status, decision.Reason = metav1.ConditionFalse, "ResolutionInconsistent" + decision.Message = bounded(fmt.Sprintf("DNS %s differs between required resolver/authority views %s.", name, preview(decision.Views))) + return decision +} + +func mismatch(name string, decision Decision) Decision { + decision.Status, decision.Reason = metav1.ConditionFalse, "ResolutionMismatch" + decision.Message = bounded(fmt.Sprintf("DNS %s targets differ: missing %s; unexpected %s.", name, preview(decision.Missing), preview(decision.Extra))) + return decision +} + +func typed(kind string, values []string) []string { + out := make([]string, len(values)) + for i, value := range values { + out[i] = kind + " " + value + } + return out +} + +func difference(expected, actual []string) (missing, extra []string) { + for _, value := range expected { + if !slices.Contains(actual, value) { + missing = append(missing, value) + } + } + for _, value := range actual { + if !slices.Contains(expected, value) { + extra = append(extra, value) + } + } + slices.Sort(missing) + slices.Sort(extra) + return missing, extra +} + +func Preview(values []string) externalv1alpha1.VerificationValues { + result := externalv1alpha1.VerificationValues{Total: int32(len(values)), Truncated: len(values) > 16} + result.Values = slices.Clone(values[:min(len(values), 16)]) + for i, value := range result.Values { + if len(value) > 512 { + result.Values[i] = value[:512] + result.Truncated = true + } + } + return result +} + +func preview(values []string) string { + // The message preview is smaller than public data so two previews plus a name fit 2048 bytes. + shown := values[:min(len(values), 4)] + result := "[" + strings.Join(shown, ", ") + "]" + if len(values) > len(shown) { + result += fmt.Sprintf(" (+%d omitted)", len(values)-len(shown)) + } + if len(result) > 700 { + result = result[:680] + " (truncated)" + } + return result +} + +func bounded(value string) string { + if len(value) > 2048 { + return value[:2032] + " (truncated)" + } + return value +} diff --git a/internal/go/providers/external/probe/evaluate_test.go b/internal/go/providers/external/probe/evaluate_test.go new file mode 100644 index 0000000..1535a3c --- /dev/null +++ b/internal/go/providers/external/probe/evaluate_test.go @@ -0,0 +1,118 @@ +package probe + +import ( + "fmt" + "testing" + + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "github.com/miekg/dns" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func TestExactVerification(t *testing.T) { + cases := []struct { + name string + expected RRsets + views []View + reason string + }{ + {"exact", RRsets{A: []string{"192.0.2.1"}}, []View{{A: []string{"192.0.2.1"}}}, "ResolutionVerified"}, + {"normalized duplicate IPv6", RRsets{AAAA: []string{"2001:db8::1"}}, []View{{AAAA: []string{"2001:0DB8:0:0::1", "2001:db8::1"}}}, "ResolutionVerified"}, + {"unexpected family", RRsets{A: []string{"192.0.2.1"}}, []View{{A: []string{"192.0.2.1"}, AAAA: []string{"2001:db8::1"}}}, "ResolutionMismatch"}, + {"partial", RRsets{A: []string{"192.0.2.1", "192.0.2.2"}}, []View{{A: []string{"192.0.2.1"}}}, "ResolutionPartial"}, + {"wrong", RRsets{A: []string{"192.0.2.1"}}, []View{{A: []string{"192.0.2.2"}}}, "ResolutionMismatch"}, + {"NXDOMAIN", RRsets{A: []string{"192.0.2.1"}}, []View{{RCode: dns.RcodeNameError}}, "DNSNameNotFound"}, + {"NODATA", RRsets{A: []string{"192.0.2.1"}}, []View{{}}, "DNSNoData"}, + {"inconsistent", RRsets{A: []string{"192.0.2.1"}}, []View{{ID: "a", A: []string{"192.0.2.1"}}, {ID: "b", RCode: dns.RcodeNameError}}, "ResolutionInconsistent"}, + {"failure beats mismatch", RRsets{A: []string{"192.0.2.1"}}, []View{{ID: "a", A: []string{"192.0.2.2"}}, {ID: "b", Failure: "timeout"}}, "DNSProbeFailed"}, + {"SERVFAIL", RRsets{A: []string{"192.0.2.1"}}, []View{{RCode: dns.RcodeServerFailure}}, "DNSProbeFailed"}, + {"CNAME to IP", RRsets{A: []string{"192.0.2.1"}}, []View{{Chain: []string{"TARGET.EXAMPLE."}, A: []string{"192.0.2.1"}}}, "ResolutionVerified"}, + {"CNAME identity", RRsets{CNAME: "target.example"}, []View{{Chain: []string{"TARGET.EXAMPLE."}, A: []string{"192.0.2.1"}}}, "ResolutionVerified"}, + {"wrong CNAME same IP", RRsets{CNAME: "target.example"}, []View{{Chain: []string{"other.example"}, A: []string{"192.0.2.1"}}}, "ResolutionMismatch"}, + {"unusable CNAME", RRsets{CNAME: "target.example"}, []View{{Chain: []string{"target.example"}}}, "DNSNoData"}, + {"CNAME loop", RRsets{A: []string{"192.0.2.1"}}, []View{{Chain: []string{"target.example", "www.example"}, A: []string{"192.0.2.1"}}}, "ResolutionMismatch"}, + {"incompatible CNAME", RRsets{CNAME: "target.example"}, []View{{Chain: []string{"target.example"}, A: []string{"192.0.2.1"}, Mismatch: true}}, "ResolutionMismatch"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + decision := Evaluate(ProbeRequest{Name: "www.example", Expected: tc.expected}, ProbeResult{Views: tc.views}) + if decision.Reason != tc.reason { + t.Fatalf("decision = %#v, want %s", decision, tc.reason) + } + if len(decision.Message) > 2048 { + t.Fatal("unbounded message") + } + }) + } +} + +func TestCNAMEChainBoundary(t *testing.T) { + for _, links := range []int{8, 9} { + chain := make([]string, links) + for i := range chain { + chain[i] = fmt.Sprintf("link-%d.example", i) + } + decision := Evaluate(ProbeRequest{Name: "www.example", Expected: RRsets{A: []string{"192.0.2.1"}}}, ProbeResult{Views: []View{{Chain: chain, A: []string{"192.0.2.1"}}}}) + want := "ResolutionVerified" + if links == 9 { + want = "DNSProbeFailed" + } + if decision.Reason != want { + t.Fatalf("links=%d: %#v", links, decision) + } + } +} + +func TestNameserverEvidence(t *testing.T) { + request := ProbeRequest{Zone: "example.test", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{"192.0.2.53:53"}}, ExpectedNameservers: []string{"ns.example.test"}}} + good := AuthorityObservation{ID: "recursive/192.0.2.53:53", Nameservers: []string{"NS.EXAMPLE.TEST."}, SOAOwner: "example.test"} + authority := AuthorityObservation{ID: "authority/192.0.2.54:53", Nameservers: []string{"ns.example.test"}, SOAOwner: "example.test", Authoritative: true} + cases := []struct { + name string + mutate func(*ZoneEvidence) + reason string + }{ + {"verified", func(*ZoneEvidence) {}, "NameserversVerified"}, + {"non apex", func(e *ZoneEvidence) { e.Views[0].SOAOwner = "test"; e.Views[1].SOAOwner = "test" }, "NameserverMismatch"}, + {"NS mismatch", func(e *ZoneEvidence) { + for i := range e.Views { + e.Views[i].Nameservers = []string{"wrong.example.test"} + } + }, "NameserverMismatch"}, + {"inconsistent", func(e *ZoneEvidence) { e.Views[1].Nameservers = []string{"wrong.example.test"} }, "ResolutionInconsistent"}, + {"missing AA", func(e *ZoneEvidence) { e.Views[1].Authoritative = false }, "DNSProbeFailed"}, + {"failed authority", func(e *ZoneEvidence) { e.Views[1].Failure = "REFUSED" }, "DNSProbeFailed"}, + {"missing authority", func(e *ZoneEvidence) { e.Views = e.Views[:1] }, "DNSProbeFailed"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + evidence := &ZoneEvidence{Views: []AuthorityObservation{good, authority}, Authorities: []string{"192.0.2.54:53"}} + tc.mutate(evidence) + decision := EvaluateZone(request, evidence) + if decision.Reason != tc.reason { + t.Fatalf("decision = %#v, want %s", decision, tc.reason) + } + }) + } + request.Policy.ExpectedNameservers = nil + if decision := EvaluateZone(request, nil); decision.Status != metav1.ConditionTrue || len(decision.Nameservers) != 0 { + t.Fatalf("suffix = %#v", decision) + } +} + +func TestStatusPreviewDoesNotAffectEquality(t *testing.T) { + values := make([]string, 20) + for i := range values { + values[i] = fmt.Sprintf("192.0.2.%d", i+1) + } + observed := append([]string(nil), values...) + observed[19] = "198.51.100.1" + decision := Evaluate(ProbeRequest{Name: "www.example", Expected: RRsets{A: values}}, ProbeResult{Views: []View{{A: observed}}}) + if decision.Reason != "ResolutionMismatch" { + t.Fatalf("decision = %#v", decision) + } + preview := Preview(decision.Actual) + if preview.Total != 20 || !preview.Truncated || len(preview.Values) != 16 { + t.Fatalf("preview = %#v", preview) + } +} diff --git a/internal/go/providers/external/probe/limits_test.go b/internal/go/providers/external/probe/limits_test.go new file mode 100644 index 0000000..54079c6 --- /dev/null +++ b/internal/go/providers/external/probe/limits_test.go @@ -0,0 +1,163 @@ +package probe + +import ( + "context" + "fmt" + "math" + "net" + "strings" + "sync/atomic" + "testing" + + "github.com/miekg/dns" +) + +func TestWireChainBoundaries(t *testing.T) { + for _, tc := range []struct { + name string + links int + loop bool + reason string + }{ + {"eight", 8, false, "ResolutionVerified"}, + {"nine", 9, false, "DNSProbeFailed"}, + {"loop", 2, true, "ResolutionMismatch"}, + } { + t.Run(tc.name, func(t *testing.T) { + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + reply := new(dns.Msg) + reply.SetReply(request) + question := request.Question[0] + var link int + if _, err := fmt.Sscanf(question.Name, "link-%d.example.test.", &link); err != nil { + t.Error(err) + } + if link < tc.links { + next := link + 1 + if tc.loop && next == tc.links { + next = 0 + } + reply.Answer = []dns.RR{&dns.CNAME{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeCNAME, Class: dns.ClassINET, Ttl: 60}, Target: fmt.Sprintf("link-%d.example.test.", next)}} + } else if question.Qtype == dns.TypeA { + reply.Answer = []dns.RR{&dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 120}, A: net.ParseIP("192.0.2.1")}} + } + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + session := wireSession{resolver: resolver} + view := session.resolve(context.Background(), "link-0.example.test", server, "", "example.test") + decision := Evaluate(ProbeRequest{Name: "link-0.example.test", Expected: RRsets{A: []string{"192.0.2.1"}}}, ProbeResult{Views: []View{view}}) + if decision.Reason != tc.reason { + t.Fatalf("decision=%#v view=%#v", decision, view) + } + }) + } +} + +func TestForeignCNAMEUsesSelectedRecursiveView(t *testing.T) { + var authorityQueries, recursiveQueries atomic.Int32 + authority := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + authorityQueries.Add(1) + if request.RecursionDesired || request.Question[0].Name != "www.example.test." { + t.Errorf("unexpected authority query %#v", request) + } + reply := new(dns.Msg) + reply.SetReply(request) + reply.Authoritative = true + reply.Answer = []dns.RR{&dns.CNAME{Hdr: dns.RR_Header{Name: "www.example.test.", Rrtype: dns.TypeCNAME, Class: dns.ClassINET, Ttl: 60}, Target: "target.foreign.test."}} + _ = w.WriteMsg(reply) + }) + recursive := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + recursiveQueries.Add(1) + if !request.RecursionDesired || request.Question[0].Name != "target.foreign.test." { + t.Errorf("unexpected recursive query %#v", request) + } + reply := new(dns.Msg) + reply.SetReply(request) + if request.Question[0].Qtype == dns.TypeA { + reply.Answer = []dns.RR{&dns.A{Hdr: dns.RR_Header{Name: "target.foreign.test.", Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 120}, A: net.ParseIP("192.0.2.1")}} + } + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + session := wireSession{resolver: resolver, authorityAnswers: map[questionKey]*dns.Msg{}} + view := session.resolve(context.Background(), "www.example.test", recursive, authority, "example.test") + if view.Failure != "" || !view.Authoritative || len(view.A) != 1 || authorityQueries.Load() != 2 || recursiveQueries.Load() != 2 { + t.Fatalf("view=%#v authority=%d recursive=%d", view, authorityQueries.Load(), recursiveQueries.Load()) + } + // Reusing this name's in-zone evidence for another recursive suffix walk sends no duplicate authority exchanges. + session.resolve(context.Background(), "www.example.test", recursive, authority, "example.test") + if authorityQueries.Load() != 2 { + t.Fatal("in-zone authority work multiplied by recursive views") + } +} + +func TestMalformedWireResponseCannotVerify(t *testing.T) { + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + reply := new(dns.Msg) + reply.SetReply(request) + reply.Opcode = dns.OpcodeUpdate + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + session := wireSession{resolver: resolver} + _, failure := session.exchange(context.Background(), server, "www.example.test", dns.TypeA, true) + if failure == "" { + t.Fatal("invalid opcode was accepted") + } +} + +func TestLimiterDefaultsAndClosedFailureCodes(t *testing.T) { + resolver, err := NewWireResolver(WireOptions{}) + if err != nil { + t.Fatal(err) + } + if resolver.limiter.Limit() != 40 || resolver.limiter.Burst() != 80 || cap(resolver.slots) != 4 { + t.Fatal("manager-wide bounds differ from contract") + } + for _, qps := range []float64{-1, math.NaN(), math.Inf(1)} { + if _, err := NewWireResolver(WireOptions{QPS: qps}); err == nil { + t.Fatal("invalid limiter accepted") + } + } + decision := Failed("www.example.test", "view", "raw secret-containing diagnostic") + if strings.Contains(decision.Message, "secret") || !strings.HasSuffix(decision.Message, "transport.") { + t.Fatalf("message=%s", decision.Message) + } +} + +func TestNameserverAddressFanoutFailsClosed(t *testing.T) { + var queries atomic.Int32 + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + queries.Add(1) + if !request.RecursionDesired || request.Opcode != dns.OpcodeQuery { + t.Error("NS discovery must be recursive QUERY") + } + reply := new(dns.Msg) + reply.SetReply(request) + question := request.Question[0] + switch question.Qtype { + case dns.TypeNS: + reply.Answer = []dns.RR{&dns.NS{Hdr: dns.RR_Header{Name: "example.test.", Rrtype: dns.TypeNS, Class: dns.ClassINET, Ttl: 300}, Ns: "ns.example.test."}} + case dns.TypeSOA: + reply.Answer = []dns.RR{&dns.SOA{Hdr: dns.RR_Header{Name: "example.test.", Rrtype: dns.TypeSOA, Class: dns.ClassINET, Ttl: 300}, Ns: "ns.example.test.", Mbox: "hostmaster.example.test.", Minttl: 60}} + case dns.TypeA: + for _, address := range []string{"127.0.0.2", "127.0.0.3", "127.0.0.4"} { + reply.Answer = append(reply.Answer, &dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 300}, A: net.ParseIP(address)}) + } + } + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + request := ProbeRequest{Zone: "example.test", Name: "example.test", ZoneOnly: true} + request.Policy.Resolver.Servers = []string{server} + request.Policy.ExpectedNameservers = []string{"ns.example.test"} + result, err := resolver.Probe(context.Background(), request) + if err != nil { + t.Fatal(err) + } + decision := EvaluateZone(request, result.Zone) + if decision.Reason != "DNSProbeFailed" || !strings.Contains(decision.Message, "probe-limit") || queries.Load() != 4 { + t.Fatalf("decision=%#v queries=%d", decision, queries.Load()) + } +} diff --git a/internal/go/providers/external/probe/metrics.go b/internal/go/providers/external/probe/metrics.go new file mode 100644 index 0000000..937b3de --- /dev/null +++ b/internal/go/providers/external/probe/metrics.go @@ -0,0 +1,47 @@ +package probe + +import ( + "sync" + "time" + + "github.com/prometheus/client_golang/prometheus" + "sigs.k8s.io/controller-runtime/pkg/metrics" +) + +var ( + probes = prometheus.NewCounterVec(prometheus.CounterOpts{Name: "dns_external_probes_total", Help: "Completed external DNS observations."}, []string{"outcome", "qtype"}) + duration = prometheus.NewHistogramVec(prometheus.HistogramOpts{Name: "dns_external_probe_duration_seconds", Help: "Admitted external DNS probe duration."}, []string{"outcome"}) + queueDelay = prometheus.NewHistogram(prometheus.HistogramOpts{Name: "dns_external_probe_queue_delay_seconds", Help: "Scheduling and limiter admission delay for external DNS probes."}) + pending = prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: "dns_external_pending_records", Help: "Active unverified external record claims."}, []string{"reason"}) + pendingMu sync.Mutex + pendingUnits = map[string]map[string]int{} +) + +func init() { metrics.Registry.MustRegister(probes, duration, queueDelay, pending) } + +func ObserveQueueDelay(delay time.Duration) { queueDelay.Observe(max(0, delay.Seconds())) } + +// SetPending replaces a unit's contribution; identities are internal keys, never labels. +func SetPending(unit string, counts map[string]int) { + pendingMu.Lock() + defer pendingMu.Unlock() + for reason, count := range pendingUnits[unit] { + pending.WithLabelValues(reason).Sub(float64(count)) + } + delete(pendingUnits, unit) + next := map[string]int{} + for reason, count := range counts { + switch reason { + case "Reconciling", "DNSNameNotFound", "DNSNoData", "ResolutionPartial", "ResolutionMismatch", "ResolutionInconsistent", "NameserverMismatch", "DNSProbeFailed": + default: + reason = "Reconciling" + } + next[reason] += count + } + if len(next) != 0 { + pendingUnits[unit] = next + } + for reason, count := range next { + pending.WithLabelValues(reason).Add(float64(count)) + } +} diff --git a/internal/go/providers/external/probe/types.go b/internal/go/providers/external/probe/types.go new file mode 100644 index 0000000..3d110b2 --- /dev/null +++ b/internal/go/providers/external/probe/types.go @@ -0,0 +1,114 @@ +// Package probe observes DNS using QUERY only. It has no DNS mutation interface. +package probe + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "net/netip" + "slices" + "strings" + "time" + + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" +) + +type Resolver interface { + Probe(context.Context, ProbeRequest) (ProbeResult, error) +} + +type RRsets struct { + A []string `json:"a,omitempty"` + AAAA []string `json:"aaaa,omitempty"` + CNAME string `json:"cname,omitempty"` +} + +type ProbeRequest struct { + Name string + Zone string + Expected RRsets + Policy externalv1alpha1.ExternalZoneClassParameters + // DueTime lets the wire resolver measure scheduling plus limiter queue delay once. + DueTime time.Time + // ZoneOnly collects apex/NS-host work once for this Zone's probe cycle. + ZoneOnly bool + // Evidence is cycle-local, never a process-wide DNS answer cache. + Evidence *ZoneEvidence +} + +type ProbeResult struct { + Views []View + Zone *ZoneEvidence +} + +type View struct { + ID string + RCode int + A []string + AAAA []string + Chain []string + Authoritative bool + TTL *uint32 + NegativeTTL *uint32 + // Failure is a closed diagnostic code; raw transport errors never enter status. + Failure string + Mismatch bool +} + +type AuthorityObservation struct { + ID string + Nameservers []string + SOAOwner string + Authoritative bool + TTL *uint32 + NegativeTTL *uint32 + Failure string +} + +type ZoneEvidence struct { + PolicyFingerprint string + Zone string + Views []AuthorityObservation + Authorities []string +} + +func CanonicalName(name string) string { return strings.TrimSuffix(strings.ToLower(name), ".") } + +func Names(values []string) []string { + out := make([]string, 0, len(values)) + for _, value := range values { + out = append(out, CanonicalName(value)) + } + slices.Sort(out) + return slices.Compact(out) +} + +func Addresses(values []string) []string { + out := make([]string, 0, len(values)) + for _, value := range values { + if ip, err := netip.ParseAddr(value); err == nil { + value = ip.Unmap().String() + } + out = append(out, value) + } + slices.Sort(out) + return slices.Compact(out) +} + +func Fingerprint(value any) string { + data, _ := json.Marshal(value) + hash := sha256.Sum256(data) + return hex.EncodeToString(hash[:]) +} + +func minTTL(current *uint32, next *uint32) *uint32 { + if next == nil { + return current + } + if current == nil || *next < *current { + value := *next + return &value + } + return current +} diff --git a/internal/go/providers/external/probe/wire.go b/internal/go/providers/external/probe/wire.go new file mode 100644 index 0000000..ee4dd16 --- /dev/null +++ b/internal/go/providers/external/probe/wire.go @@ -0,0 +1,388 @@ +package probe + +import ( + "context" + "errors" + "fmt" + "math" + "net" + "net/netip" + "slices" + "strings" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/config" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "github.com/miekg/dns" + "golang.org/x/time/rate" +) + +const ( + DefaultQPS = 40 + DefaultBurst = 80 + MaxConcurrentNames = 4 + ExchangeTimeout = 2 * time.Second + NameTimeout = 10 * time.Second + MaxCNAMEs = 8 +) + +type WireOptions struct { + QPS float64 + Burst int + ResolvConf string +} + +type WireResolver struct { + limiter *rate.Limiter + slots chan struct{} + resolvConf string +} + +func NewWireResolver(options WireOptions) (*WireResolver, error) { + if options.QPS == 0 { + options.QPS = DefaultQPS + } + if options.Burst == 0 { + options.Burst = DefaultBurst + } + if options.QPS <= 0 || math.IsNaN(options.QPS) || math.IsInf(options.QPS, 0) || options.Burst < 1 { + return nil, fmt.Errorf("external DNS QPS and burst must be positive and finite") + } + if options.ResolvConf == "" { + options.ResolvConf = "/etc/resolv.conf" + } + return &WireResolver{limiter: rate.NewLimiter(rate.Limit(options.QPS), options.Burst), slots: make(chan struct{}, MaxConcurrentNames), resolvConf: options.ResolvConf}, nil +} + +// EffectivePolicy rereads resolv.conf each reconcile so changes invalidate receipts. +func (r *WireResolver) EffectivePolicy(policy externalv1alpha1.ExternalZoneClassParameters) (externalv1alpha1.ExternalZoneClassParameters, error) { + policy.Resolver.Servers = slices.Clone(policy.Resolver.Servers) + if policy.Resolver.Mode == "" { + policy.Resolver.Mode = externalv1alpha1.ResolverModeCluster + } + if policy.Resolver.Mode == externalv1alpha1.ResolverModeCluster { + conf, err := dns.ClientConfigFromFile(r.resolvConf) + if err != nil { + return policy, errors.New("resolver-config") + } + policy.Resolver.Servers = nil + for _, host := range conf.Servers { + ip, err := netip.ParseAddr(host) + if err != nil || ip.Zone() != "" { + return policy, errors.New("resolver-config") + } + policy.Resolver.Servers = append(policy.Resolver.Servers, net.JoinHostPort(ip.String(), conf.Port)) + } + slices.Sort(policy.Resolver.Servers) + policy.Resolver.Servers = slices.Compact(policy.Resolver.Servers) + } + if config.ValidateServers(policy.Resolver.Servers) != nil { + return policy, errors.New("resolver-config") + } + slices.Sort(policy.Resolver.Servers) + return policy, nil +} + +func (r *WireResolver) Probe(ctx context.Context, request ProbeRequest) (ProbeResult, error) { + if config.ValidateServers(request.Policy.Resolver.Servers) != nil { + return ProbeResult{}, errors.New("resolver-config") + } + queued := time.Now() + if !request.DueTime.IsZero() { + queued = request.DueTime + } + select { + case r.slots <- struct{}{}: + defer func() { <-r.slots }() + case <-ctx.Done(): + return ProbeResult{}, ctx.Err() + } + // Admit the first exchange before starting the name deadline. + if err := r.limiter.Wait(ctx); err != nil { + return ProbeResult{}, err + } + ObserveQueueDelay(time.Since(queued)) + ctx, cancel := context.WithTimeout(ctx, NameTimeout) + defer cancel() + start := time.Now() + session := wireSession{resolver: r, firstToken: true, authorityAnswers: map[questionKey]*dns.Msg{}, qtypes: map[uint16]int{}} + var result ProbeResult + if request.ZoneOnly { + result.Zone = session.zone(ctx, request) + } else { + for _, server := range request.Policy.Resolver.Servers { + result.Views = append(result.Views, session.resolve(ctx, request.Name, server, "", request.Zone)) + } + if len(request.Policy.ExpectedNameservers) != 0 { + if request.Evidence == nil || request.Evidence.Zone != request.Zone || request.Evidence.PolicyFingerprint != Fingerprint(request.Policy) || len(request.Evidence.Authorities) == 0 { + return ProbeResult{}, errors.New("invalid-response") + } + for _, authority := range request.Evidence.Authorities { + view := session.resolve(ctx, request.Name, request.Policy.Resolver.Servers[0], authority, request.Zone) + result.Views = append(result.Views, view) + foreign := false + for _, target := range view.Chain { + if target != request.Zone && !strings.HasSuffix(target, "."+request.Zone) { + foreign = true + break + } + } + if foreign { + for _, server := range request.Policy.Resolver.Servers[1:] { + result.Views = append(result.Views, session.resolve(ctx, request.Name, server, authority, request.Zone)) + } + } + } + } + } + decision := Evaluate(request, result) + if request.ZoneOnly { + decision = EvaluateZone(request, result.Zone) + } + outcome := decision.Outcome() + for qtype, count := range session.qtypes { + probes.WithLabelValues(outcome, dns.TypeToString[qtype]).Add(float64(count)) + } + duration.WithLabelValues(outcome).Observe(time.Since(start).Seconds()) + return result, nil +} + +type wireSession struct { + resolver *WireResolver + firstToken bool + authorityAnswers map[questionKey]*dns.Msg + qtypes map[uint16]int +} + +type questionKey struct { + server, name string + qtype uint16 +} + +func (s *wireSession) exchange(ctx context.Context, server, name string, qtype uint16, recursive bool) (*dns.Msg, string) { + key := questionKey{server, name, qtype} + if !recursive { + if reply := s.authorityAnswers[key]; reply != nil { + return reply, "" + } + } + if s.firstToken { + s.firstToken = false + } else if err := s.resolver.limiter.Wait(ctx); err != nil { + return nil, "timeout" + } + request := new(dns.Msg) + request.SetQuestion(dns.Fqdn(name), qtype) + request.RecursionDesired = recursive + request.Opcode = dns.OpcodeQuery + if s.qtypes != nil { + s.qtypes[qtype]++ + } + client := &dns.Client{Net: "udp", Timeout: ExchangeTimeout} + reply, _, err := client.ExchangeContext(ctx, request, server) + if err == nil && reply != nil && reply.Truncated { + if err = s.resolver.limiter.Wait(ctx); err == nil { + client.Net = "tcp" + reply, _, err = client.ExchangeContext(ctx, request, server) + } + } + failure := "" + if err != nil { + failure = "transport" + var netErr net.Error + if errors.Is(err, context.DeadlineExceeded) || (errors.As(err, &netErr) && netErr.Timeout()) { + failure = "timeout" + } + } else if reply == nil || reply.Truncated || !reply.Response || reply.Opcode != dns.OpcodeQuery || len(reply.Question) != 1 || reply.Question[0] != request.Question[0] { + failure = "invalid-response" + } else { + switch reply.Rcode { + case dns.RcodeSuccess, dns.RcodeNameError: + case dns.RcodeServerFailure: + failure = "SERVFAIL" + case dns.RcodeRefused: + failure = "REFUSED" + default: + failure = "invalid-response" + } + } + if failure == "" && !recursive && s.authorityAnswers != nil { + s.authorityAnswers[key] = reply + } + return reply, failure +} + +func (s *wireSession) resolve(ctx context.Context, name, recursive, authority, zone string) View { + view := View{ID: "recursive/" + recursive} + if authority != "" { + view.ID = "authority/" + authority + "/via/" + recursive + } + current := CanonicalName(name) + seen := map[string]bool{} + for { + if seen[current] { + view.Mismatch = true + return view + } + seen[current] = true + server, rd := recursive, true + if authority != "" && (current == zone || strings.HasSuffix(current, "."+zone)) { + server, rd = authority, false + } + var targets, a, aaaa []string + rcodes := [2]int{} + for i, qtype := range []uint16{dns.TypeA, dns.TypeAAAA} { + reply, failure := s.exchange(ctx, server, current, qtype, rd) + if failure != "" { + view.Failure = failure + return view + } + if !rd && !reply.Authoritative { + view.Failure = "invalid-response" + return view + } + if !rd { + view.Authoritative = true + } + rcodes[i] = reply.Rcode + ownerAnswers := 0 + for _, rr := range reply.Answer { + if CanonicalName(rr.Header().Name) != current { + continue + } + ttl := rr.Header().Ttl + switch value := rr.(type) { + case *dns.CNAME: + targets = append(targets, CanonicalName(value.Target)) + ownerAnswers++ + case *dns.A: + a = append(a, value.A.String()) + ownerAnswers++ + case *dns.AAAA: + aaaa = append(aaaa, value.AAAA.String()) + ownerAnswers++ + default: + continue + } + view.TTL = minTTL(view.TTL, &ttl) + } + if reply.Rcode == dns.RcodeNameError || ownerAnswers == 0 { + view.NegativeTTL = minTTL(view.NegativeTTL, negativeTTL(reply)) + } + } + if rcodes[0] != rcodes[1] { + view.Failure = "invalid-response" + return view + } + view.RCode = rcodes[0] + targets = Names(targets) + if len(targets) == 0 { + view.A, view.AAAA = Addresses(a), Addresses(aaaa) + return view + } + if len(targets) != 1 || len(a)+len(aaaa) != 0 { + view.Mismatch = true + view.Chain = append(view.Chain, targets...) + return view + } + view.Chain = append(view.Chain, targets[0]) + if len(view.Chain) > MaxCNAMEs { + view.Failure = "chain-limit" + return view + } + current = targets[0] + } +} + +func negativeTTL(reply *dns.Msg) *uint32 { + var ttl *uint32 + for _, rr := range reply.Ns { + if soa, ok := rr.(*dns.SOA); ok { + value := min(soa.Hdr.Ttl, soa.Minttl) + ttl = minTTL(ttl, &value) + } + } + return ttl +} + +func (s *wireSession) apex(ctx context.Context, server, zone string, rd bool) AuthorityObservation { + view := AuthorityObservation{ID: "recursive/" + server} + if !rd { + view.ID = "authority/" + server + } + for _, qtype := range []uint16{dns.TypeNS, dns.TypeSOA} { + reply, failure := s.exchange(ctx, server, zone, qtype, rd) + if failure != "" { + view.Failure = failure + return view + } + if !rd && !reply.Authoritative { + view.Failure = "invalid-response" + return view + } + view.Authoritative = reply.Authoritative + view.NegativeTTL = minTTL(view.NegativeTTL, negativeTTL(reply)) + for _, rr := range reply.Answer { + if CanonicalName(rr.Header().Name) != zone { + continue + } + ttl := rr.Header().Ttl + switch value := rr.(type) { + case *dns.NS: + view.Nameservers = append(view.Nameservers, CanonicalName(value.Ns)) + case *dns.SOA: + view.SOAOwner = CanonicalName(value.Hdr.Name) + default: + continue + } + view.TTL = minTTL(view.TTL, &ttl) + } + } + view.Nameservers = Names(view.Nameservers) + return view +} + +func (s *wireSession) zone(ctx context.Context, request ProbeRequest) *ZoneEvidence { + evidence := &ZoneEvidence{Zone: request.Zone, PolicyFingerprint: Fingerprint(request.Policy)} + if len(request.Policy.ExpectedNameservers) == 0 { + return evidence + } + for _, server := range request.Policy.Resolver.Servers { + evidence.Views = append(evidence.Views, s.apex(ctx, server, request.Zone, true)) + } + // A wrong delegation is completed evidence; never send to unverified authority destinations. + if evaluateZone(request, evidence, false).Status != "True" { + return evidence + } + for _, host := range request.Policy.ExpectedNameservers { + var addresses []string + for viewIndex, server := range request.Policy.Resolver.Servers { + view := s.resolve(ctx, host, server, "", "") + if view.Failure != "" || view.Mismatch || view.RCode != dns.RcodeSuccess || len(view.A)+len(view.AAAA) == 0 { + failure := view.Failure + if failure == "" { + failure = "invalid-response" + } + evidence.Views = append(evidence.Views, AuthorityObservation{ID: view.ID, Failure: failure}) + return evidence + } + evidence.Views[viewIndex].TTL = minTTL(evidence.Views[viewIndex].TTL, view.TTL) + addresses = append(addresses, view.A...) + addresses = append(addresses, view.AAAA...) + } + addresses = Addresses(addresses) + if len(addresses) > 2 { + evidence.Views = append(evidence.Views, AuthorityObservation{ID: "authority/" + host, Failure: "probe-limit"}) + return evidence + } + for _, address := range addresses { + evidence.Authorities = append(evidence.Authorities, net.JoinHostPort(address, "53")) + } + } + slices.Sort(evidence.Authorities) + evidence.Authorities = slices.Compact(evidence.Authorities) + for _, authority := range evidence.Authorities { + evidence.Views = append(evidence.Views, s.apex(ctx, authority, request.Zone, false)) + } + return evidence +} diff --git a/internal/go/providers/external/probe/wire_test.go b/internal/go/providers/external/probe/wire_test.go new file mode 100644 index 0000000..0bdc92e --- /dev/null +++ b/internal/go/providers/external/probe/wire_test.go @@ -0,0 +1,158 @@ +package probe + +import ( + "context" + "net" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "github.com/miekg/dns" +) + +func localDNS(t *testing.T, handler dns.HandlerFunc) string { + t.Helper() + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + packet, err := net.ListenPacket("udp", listener.Addr().String()) + if err != nil { + listener.Close() + t.Fatal(err) + } + udpReady, tcpReady := make(chan struct{}), make(chan struct{}) + udp := &dns.Server{PacketConn: packet, Handler: handler, NotifyStartedFunc: func() { close(udpReady) }} + tcp := &dns.Server{Listener: listener, Handler: handler, NotifyStartedFunc: func() { close(tcpReady) }} + go func() { _ = udp.ActivateAndServe() }() + go func() { _ = tcp.ActivateAndServe() }() + t.Cleanup(func() { _ = udp.Shutdown(); _ = tcp.Shutdown() }) + for _, ready := range []chan struct{}{udpReady, tcpReady} { + select { + case <-ready: + case <-time.After(2 * time.Second): + t.Fatal("fixture did not start") + } + } + return listener.Addr().String() +} + +func TestWireTruncationAndNegativeSOA(t *testing.T) { + var udpQueries, tcpQueries atomic.Int32 + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + if request.Opcode != dns.OpcodeQuery || !request.RecursionDesired || !strings.HasSuffix(request.Question[0].Name, ".") { + t.Errorf("not an absolute recursive QUERY: %#v", request) + } + reply := new(dns.Msg) + reply.SetReply(request) + if _, ok := w.RemoteAddr().(*net.UDPAddr); ok { + udpQueries.Add(1) + reply.Truncated = true + } else { + tcpQueries.Add(1) + reply.Rcode = dns.RcodeNameError + reply.Ns = []dns.RR{&dns.SOA{Hdr: dns.RR_Header{Name: "example.test.", Rrtype: dns.TypeSOA, Class: dns.ClassINET, Ttl: 600}, Ns: "ns.example.test.", Mbox: "hostmaster.example.test.", Minttl: 120}} + } + _ = w.WriteMsg(reply) + }) + resolver, err := NewWireResolver(WireOptions{}) + if err != nil { + t.Fatal(err) + } + request := ProbeRequest{Name: "absent.example.test", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Mode: externalv1alpha1.ResolverModeRecursive, Servers: []string{server}}}, Expected: RRsets{A: []string{"192.0.2.1"}}} + result, err := resolver.Probe(context.Background(), request) + if err != nil { + t.Fatal(err) + } + decision := Evaluate(request, result) + if decision.Reason != "DNSNameNotFound" || decision.NegativeTTL == nil || *decision.NegativeTTL != 120 { + t.Fatalf("decision = %#v", decision) + } + if udpQueries.Load() != 2 || tcpQueries.Load() != 2 { + t.Fatalf("UDP=%d TCP=%d", udpQueries.Load(), tcpQueries.Load()) + } +} + +func TestWireCNAMEAndAuthorityRD(t *testing.T) { + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + if request.Opcode != dns.OpcodeQuery { + t.Error("DNS mutation opcode") + } + if request.RecursionDesired { + t.Error("authority query requested recursion") + } + reply := new(dns.Msg) + reply.SetReply(request) + reply.Authoritative = true + question := request.Question[0] + if question.Name == "www.example.test." { + reply.Answer = []dns.RR{&dns.CNAME{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeCNAME, Class: dns.ClassINET, Ttl: 42}, Target: "target.example.test."}} + } else if question.Qtype == dns.TypeA { + reply.Answer = []dns.RR{&dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 300}, A: net.ParseIP("192.0.2.1")}} + } + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + session := wireSession{resolver: resolver} + view := session.resolve(context.Background(), "www.example.test", "192.0.2.53:53", server, "example.test") + if view.Failure != "" || len(view.Chain) != 1 || len(view.A) != 1 || !view.Authoritative || view.TTL == nil || *view.TTL != 42 { + t.Fatalf("view = %#v", view) + } +} + +func TestWireNoFallbackAndCancellation(t *testing.T) { + var queries atomic.Int32 + server := localDNS(t, func(w dns.ResponseWriter, request *dns.Msg) { + queries.Add(1) + reply := new(dns.Msg) + reply.SetReply(request) + reply.Rcode = dns.RcodeRefused + _ = w.WriteMsg(reply) + }) + resolver, _ := NewWireResolver(WireOptions{}) + request := ProbeRequest{Name: "www.example.test", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{server}}}} + result, err := resolver.Probe(context.Background(), request) + if err != nil { + t.Fatal(err) + } + if len(result.Views) != 1 || result.Views[0].Failure != "REFUSED" || queries.Load() != 1 { + t.Fatalf("result = %#v, queries=%d", result, queries.Load()) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if _, err := resolver.Probe(ctx, request); err == nil { + t.Fatal("canceled probe succeeded") + } + if queries.Load() != 1 { + t.Fatal("canceled probe sent DNS") + } +} + +func TestClusterPolicyRereadAndBounds(t *testing.T) { + path := filepath.Join(t.TempDir(), "resolv.conf") + if err := os.WriteFile(path, []byte("nameserver 192.0.2.53\nsearch must.not.expand\n"), 0600); err != nil { + t.Fatal(err) + } + resolver, _ := NewWireResolver(WireOptions{ResolvConf: path}) + first, err := resolver.EffectivePolicy(externalv1alpha1.ExternalZoneClassParameters{}) + if err != nil || len(first.Resolver.Servers) != 1 || first.Resolver.Servers[0] != "192.0.2.53:53" { + t.Fatalf("policy=%#v error=%v", first, err) + } + if err := os.WriteFile(path, []byte("nameserver 192.0.2.54\n"), 0600); err != nil { + t.Fatal(err) + } + second, err := resolver.EffectivePolicy(externalv1alpha1.ExternalZoneClassParameters{}) + if err != nil || Fingerprint(first) == Fingerprint(second) { + t.Fatal("resolv.conf change did not invalidate policy") + } + if err := os.WriteFile(path, []byte("nameserver 192.0.2.1\nnameserver 192.0.2.2\nnameserver 192.0.2.3\nnameserver 192.0.2.4\n"), 0600); err != nil { + t.Fatal(err) + } + if _, err := resolver.EffectivePolicy(externalv1alpha1.ExternalZoneClassParameters{}); err == nil { + t.Fatal("oversized resolver set accepted") + } +} diff --git a/pkg/go/api/external/v1alpha1/doc.go b/pkg/go/api/external/v1alpha1/doc.go new file mode 100644 index 0000000..bf66017 --- /dev/null +++ b/pkg/go/api/external/v1alpha1/doc.go @@ -0,0 +1,4 @@ +// Package v1alpha1 contains the external provider's inline verification payloads. +// +kubebuilder:object:generate=true +// +groupName=external.dns.appthrust.io +package v1alpha1 diff --git a/pkg/go/api/external/v1alpha1/register.go b/pkg/go/api/external/v1alpha1/register.go new file mode 100644 index 0000000..5bc8127 --- /dev/null +++ b/pkg/go/api/external/v1alpha1/register.go @@ -0,0 +1,19 @@ +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" +) + +const GroupName = ProviderName + +var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: ProviderVersion} +var SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes) +var AddToScheme = SchemeBuilder.AddToScheme + +// External has inline payload types only: no identity, Zone or RecordSet CRDs. +func addKnownTypes(scheme *runtime.Scheme) error { + metav1.AddToGroupVersion(scheme, SchemeGroupVersion) + return nil +} diff --git a/pkg/go/api/external/v1alpha1/types.go b/pkg/go/api/external/v1alpha1/types.go new file mode 100644 index 0000000..d5582da --- /dev/null +++ b/pkg/go/api/external/v1alpha1/types.go @@ -0,0 +1,53 @@ +package v1alpha1 + +import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + +const ( + ProviderName = "external.dns.appthrust.io" + ProviderVersion = "v1alpha1" + ControllerName = ProviderName + "/controller" +) + +type ResolverMode string + +const ( + ResolverModeCluster ResolverMode = "Cluster" + ResolverModeRecursive ResolverMode = "Recursive" +) + +// ExternalZoneClassParameters is an inline payload, not a Kubernetes resource. +type ExternalZoneClassParameters struct { + Resolver ResolverPolicy `json:"resolver,omitempty"` + ExpectedNameservers []string `json:"expectedNameservers,omitempty"` +} + +// ResolverPolicy selects independent recursive views; servers are canonical IP:port endpoints. +type ResolverPolicy struct { + Mode ResolverMode `json:"mode,omitempty"` + Servers []string `json:"servers,omitempty"` +} + +// VerificationValues is a bounded preview. Comparisons always use the complete observation. +type VerificationValues struct { + Values []string `json:"values,omitempty"` + Total int32 `json:"total"` + Truncated bool `json:"truncated"` +} + +// ExternalVerificationData is public, non-secret evidence shared by Zone and RecordSet status. +// Values are typed (for example "A 192.0.2.1"); chain and views are bounded diagnostics. +type ExternalVerificationData struct { + ResolverMode ResolverMode `json:"resolverMode"` + CheckedName string `json:"checkedName"` + SampledWildcard bool `json:"sampledWildcard,omitempty"` + Actual VerificationValues `json:"actual"` + Missing VerificationValues `json:"missing"` + Extra VerificationValues `json:"extra"` + CNAMEChain VerificationValues `json:"cnameChain"` + Views VerificationValues `json:"views"` + ObservedTTLSeconds *uint32 `json:"observedTTLSeconds,omitempty"` + NegativeTTLSeconds *uint32 `json:"negativeTTLSeconds,omitempty"` + NegativeExpiryTime *metav1.Time `json:"negativeExpiryTime,omitempty"` + LastProbeTime *metav1.Time `json:"lastProbeTime,omitempty"` + NextProbeTime *metav1.Time `json:"nextProbeTime,omitempty"` +} diff --git a/pkg/go/api/external/v1alpha1/types_test.go b/pkg/go/api/external/v1alpha1/types_test.go new file mode 100644 index 0000000..02a15fd --- /dev/null +++ b/pkg/go/api/external/v1alpha1/types_test.go @@ -0,0 +1,26 @@ +package v1alpha1 + +import ( + "testing" + + "k8s.io/apimachinery/pkg/runtime" +) + +func TestInlineOnlySchemeAndDeepcopy(t *testing.T) { + scheme := runtime.NewScheme() + if err := AddToScheme(scheme); err != nil { + t.Fatal(err) + } + for _, kind := range []string{"ExternalIdentity", "ExternalZone", "ExternalRecordSet"} { + if scheme.Recognizes(SchemeGroupVersion.WithKind(kind)) { + t.Fatalf("unexpected CRD kind %s", kind) + } + } + params := &ExternalZoneClassParameters{Resolver: ResolverPolicy{Mode: ResolverModeRecursive, Servers: []string{"192.0.2.53:53"}}, ExpectedNameservers: []string{"ns.example.test"}} + copy := params.DeepCopy() + copy.Resolver.Servers[0] = "192.0.2.54:53" + copy.ExpectedNameservers[0] = "other.example.test" + if params.Resolver.Servers[0] != "192.0.2.53:53" || params.ExpectedNameservers[0] != "ns.example.test" { + t.Fatal("deepcopy aliases source") + } +} diff --git a/pkg/go/api/external/v1alpha1/zz_generated.deepcopy.go b/pkg/go/api/external/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 0000000..d4b82ef --- /dev/null +++ b/pkg/go/api/external/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,110 @@ +//go:build !ignore_autogenerated + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import () + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ExternalVerificationData) DeepCopyInto(out *ExternalVerificationData) { + *out = *in + in.Actual.DeepCopyInto(&out.Actual) + in.Missing.DeepCopyInto(&out.Missing) + in.Extra.DeepCopyInto(&out.Extra) + in.CNAMEChain.DeepCopyInto(&out.CNAMEChain) + in.Views.DeepCopyInto(&out.Views) + if in.ObservedTTLSeconds != nil { + in, out := &in.ObservedTTLSeconds, &out.ObservedTTLSeconds + *out = new(uint32) + **out = **in + } + if in.NegativeTTLSeconds != nil { + in, out := &in.NegativeTTLSeconds, &out.NegativeTTLSeconds + *out = new(uint32) + **out = **in + } + if in.NegativeExpiryTime != nil { + in, out := &in.NegativeExpiryTime, &out.NegativeExpiryTime + *out = (*in).DeepCopy() + } + if in.LastProbeTime != nil { + in, out := &in.LastProbeTime, &out.LastProbeTime + *out = (*in).DeepCopy() + } + if in.NextProbeTime != nil { + in, out := &in.NextProbeTime, &out.NextProbeTime + *out = (*in).DeepCopy() + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExternalVerificationData. +func (in *ExternalVerificationData) DeepCopy() *ExternalVerificationData { + if in == nil { + return nil + } + out := new(ExternalVerificationData) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ExternalZoneClassParameters) DeepCopyInto(out *ExternalZoneClassParameters) { + *out = *in + in.Resolver.DeepCopyInto(&out.Resolver) + if in.ExpectedNameservers != nil { + in, out := &in.ExpectedNameservers, &out.ExpectedNameservers + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExternalZoneClassParameters. +func (in *ExternalZoneClassParameters) DeepCopy() *ExternalZoneClassParameters { + if in == nil { + return nil + } + out := new(ExternalZoneClassParameters) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ResolverPolicy) DeepCopyInto(out *ResolverPolicy) { + *out = *in + if in.Servers != nil { + in, out := &in.Servers, &out.Servers + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ResolverPolicy. +func (in *ResolverPolicy) DeepCopy() *ResolverPolicy { + if in == nil { + return nil + } + out := new(ResolverPolicy) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VerificationValues) DeepCopyInto(out *VerificationValues) { + *out = *in + if in.Values != nil { + in, out := &in.Values, &out.Values + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VerificationValues. +func (in *VerificationValues) DeepCopy() *VerificationValues { + if in == nil { + return nil + } + out := new(VerificationValues) + in.DeepCopyInto(out) + return out +} From b55ed6707d96d969ea244bd8b0a7b8454508a198 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 11:07:27 +0900 Subject: [PATCH 04/16] fix(external): avoid unchanged scheduler status patches --- docs/design/test-policy.md | 2 + .../controllers/zoneunit/controller.go | 15 +++- .../controllers/zoneunit/controller_test.go | 82 ++++++++++++++----- 3 files changed, 73 insertions(+), 26 deletions(-) diff --git a/docs/design/test-policy.md b/docs/design/test-policy.md index ef3e7cb..b5bd94b 100644 --- a/docs/design/test-policy.md +++ b/docs/design/test-policy.md @@ -12,6 +12,8 @@ External provider: unit tests inject a fake Resolver/clock; a loopback UDP/TCP D External provider unit tests live in `internal/go/providers/external` and `pkg/go/api/external`. They use an injected Resolver, clock, and jitter for exact sets, sibling-family and policy fingerprints, wildcard witnesses, bounded passes, backoff, mismatch/recovery/drift, stale-result rejection, and UID-bound deletion. Loopback UDP/TCP fixtures exercise DNS wire parsing, truncation fallback, SOA negative TTLs, RD/AA, cancellation, and QUERY-only behavior. No public DNS, cloud credentials, or customer DNS resources are used. Manifest tests compile and evaluate the declared rules through the pinned KRO CEL environment; conversion tests cover HTTP framing, UID, typed fragments, TTL 300, and failures. +External no-op reconciliation tests also round-trip the private scheduling JSON with reordered keys and changed formatting. Before a due time, unchanged typed receipts must retain their stored representation, issue no DNS probe, and perform no status patch; resourceVersion and probe counts are both asserted for successful and failed observations. + After integrating the optional-identity core API, the focused command is `go test ./internal/go/providers/external/... ./pkg/go/api/external/...`. The runtime packages alone do not establish manager/chart registration, aggregated APIService authentication/TLS/RBAC, or end-to-end EndpointRecordSet readiness. Those remain separate integration acceptance checks, not claims inferred from fake-resolver tests. Provider validation tests cover the `forbid-txt-non-printable-ascii` disable switch as a core named validation. The built-in Route 53 and Cloudflare Provider manifests do not disable it. A provider test fixture may disable it to prove that the core webhook can be relaxed by a future provider version after that provider defines escape, unescape, display, and comparison behavior. Route 53 provider unit tests keep a defensive controller-side rejection path for non-printable ASCII TXT values even though the built-in webhook path rejects them earlier. diff --git a/internal/go/providers/external/controllers/zoneunit/controller.go b/internal/go/providers/external/controllers/zoneunit/controller.go index b027983..2672fee 100644 --- a/internal/go/providers/external/controllers/zoneunit/controller.go +++ b/internal/go/providers/external/controllers/zoneunit/controller.go @@ -214,9 +214,11 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c } } } + prunedSchedule := false for name := range state.Names { if !active[name] { delete(state.Names, name) + prunedSchedule = true } } due := selectDue(groups, state, now) @@ -306,11 +308,16 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c if blocked != nil || fresh == nil || fresh.fingerprint != env.fingerprint || (fresh.policyError != nil) != (env.policyError != nil) { return ctrl.Result{RequeueAfter: time.Second}, nil } - stateBytes, err := json.Marshal(state) - if err != nil { - return ctrl.Result{}, err + // Preserve the stored RawExtension unless scheduling actually changed. + // Re-encoding unchanged state can reorder API-normalized JSON keys and + // make the status equality guard issue an otherwise empty patch. + if zoneProbed || len(due) != 0 || prunedSchedule { + stateBytes, err := json.Marshal(state) + if err != nil { + return ctrl.Result{}, err + } + unit.Status.Provider = &dnsv1alpha1.ProviderStatus{State: runtime.RawExtension{Raw: stateBytes}} } - unit.Status.Provider = &dnsv1alpha1.ProviderStatus{State: runtime.RawExtension{Raw: stateBytes}} setAggregate(&unit, r.Options.Clock()) r.updatePending(&unit) return ctrl.Result{RequeueAfter: nextRequeue(groups, state, r.Options.Clock())}, r.patch(ctx, base, &unit) diff --git a/internal/go/providers/external/controllers/zoneunit/controller_test.go b/internal/go/providers/external/controllers/zoneunit/controller_test.go index bb43fa7..902f845 100644 --- a/internal/go/providers/external/controllers/zoneunit/controller_test.go +++ b/internal/go/providers/external/controllers/zoneunit/controller_test.go @@ -87,7 +87,7 @@ func newFixture(t *testing.T, count int, resolver *fakeResolver) (*ZoneReconcile return reconciler, c, &now, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(unit)} } -func reconcile(t *testing.T, r *ZoneReconciler, request ctrl.Request) (ctrl.Result, dnsv1alpha1.ZoneUnit) { +func reconcileOnce(t *testing.T, r *ZoneReconciler, request ctrl.Request) (ctrl.Result, dnsv1alpha1.ZoneUnit) { t.Helper() result, err := r.Reconcile(context.Background(), request) if err != nil { @@ -116,20 +116,20 @@ func TestMismatchRecoveryDriftAndNoStatusLoop(t *testing.T) { return result, err }} r, _, now, request := newFixture(t, 1, resolver) - _, first := reconcile(t, r, request) + _, first := reconcileOnce(t, r, request) assertProgrammed(t, first.Status.Zone.Conditions, metav1.ConditionTrue, "VerificationConfigured") assertProgrammed(t, first.Status.RecordSets[0].Conditions, metav1.ConditionFalse, "ResolutionMismatch") - _, unchanged := reconcile(t, r, request) + _, unchanged := reconcileOnce(t, r, request) if len(resolver.requests) != 1 || unchanged.ResourceVersion != first.ResourceVersion { - t.Fatal("unchanged status triggered work or a status patch") + t.Fatalf("unchanged status triggered work or a status patch: probes=%d (want 1), resourceVersion=%q -> %q", len(resolver.requests), first.ResourceVersion, unchanged.ResourceVersion) } resolver.answer = matching *now = now.Add(30 * time.Second) - _, ready := reconcile(t, r, request) + _, ready := reconcileOnce(t, r, request) condition := assertProgrammed(t, ready.Status.RecordSets[0].Conditions, metav1.ConditionTrue, "ResolutionVerified") transition := condition.LastTransitionTime *now = now.Add(120 * time.Second) - _, readyAgain := reconcile(t, r, request) + _, readyAgain := reconcileOnce(t, r, request) if !assertProgrammed(t, readyAgain.Status.RecordSets[0].Conditions, metav1.ConditionTrue, "ResolutionVerified").LastTransitionTime.Equal(&transition) { t.Fatal("success timestamp changed transition time") } @@ -137,7 +137,7 @@ func TestMismatchRecoveryDriftAndNoStatusLoop(t *testing.T) { return probe.ProbeResult{}, errors.New("private raw transport details") } *now = now.Add(120 * time.Second) - _, failed := reconcile(t, r, request) + _, failed := reconcileOnce(t, r, request) assertProgrammed(t, failed.Status.RecordSets[0].Conditions, metav1.ConditionUnknown, "DNSProbeFailed") assertProgrammed(t, failed.Status.Conditions, metav1.ConditionUnknown, "ProviderUnavailable") if failed.Status.RecordSets[0].RecordSetUID != "host-000" || failed.Status.RecordSets[0].ObservedGeneration != 1 { @@ -148,7 +148,7 @@ func TestMismatchRecoveryDriftAndNoStatusLoop(t *testing.T) { func TestPolicyAndSiblingInvalidation(t *testing.T) { resolver := &fakeResolver{answer: matching} r, c, _, request := newFixture(t, 1, resolver) - reconcile(t, r, request) + reconcileOnce(t, r, request) var class dnsv1alpha1.ZoneClass if err := c.Get(context.Background(), client.ObjectKey{Namespace: "app", Name: "external"}, &class); err != nil { t.Fatal(err) @@ -162,7 +162,7 @@ func TestPolicyAndSiblingInvalidation(t *testing.T) { result.Views[0].ID = "recursive/192.0.2.54:53" return result, err } - _, unit := reconcile(t, r, request) + _, unit := reconcileOnce(t, r, request) if len(resolver.requests) != 2 { t.Fatal("policy change retained previous success") } @@ -173,7 +173,7 @@ func TestPolicyAndSiblingInvalidation(t *testing.T) { if err := c.Update(context.Background(), &unit); err != nil { t.Fatal(err) } - _, got := reconcile(t, r, request) + _, got := reconcileOnce(t, r, request) if len(resolver.requests) != 3 || len(resolver.requests[2].Expected.AAAA) != 1 { t.Fatal("sibling family did not invalidate combined observation") } @@ -185,15 +185,15 @@ func TestPolicyAndSiblingInvalidation(t *testing.T) { func TestBoundedFairPassAndDeletion(t *testing.T) { resolver := &fakeResolver{answer: matching} r, c, now, request := newFixture(t, 65, resolver) - result, unit := reconcile(t, r, request) + result, unit := reconcileOnce(t, r, request) if len(resolver.requests) != 32 || result.RequeueAfter != time.Second { t.Fatalf("requests=%d result=%#v", len(resolver.requests), result) } assertProgrammed(t, unit.Status.RecordSets[64].Conditions, metav1.ConditionUnknown, "Reconciling") *now = now.Add(time.Second) - reconcile(t, r, request) + reconcileOnce(t, r, request) *now = now.Add(time.Second) - _, unit = reconcile(t, r, request) + _, unit = reconcileOnce(t, r, request) if len(resolver.requests) != 65 { t.Fatalf("requests=%d, want each name once", len(resolver.requests)) } @@ -204,7 +204,7 @@ func TestBoundedFairPassAndDeletion(t *testing.T) { if err := c.Update(context.Background(), &unit); err != nil { t.Fatal(err) } - _, got := reconcile(t, r, request) + _, got := reconcileOnce(t, r, request) if !got.Status.RecordSets[0].DeletionCompleted || got.Status.RecordSets[0].RecordSetUID != unit.Spec.RecordSets[0].RecordSetUID { t.Fatal("deletion receipt missing incarnation") } @@ -218,7 +218,7 @@ func TestBoundedFairPassAndDeletion(t *testing.T) { if err := c.Update(context.Background(), &unit); err != nil { t.Fatal(err) } - _, got = reconcile(t, r, request) + _, got = reconcileOnce(t, r, request) if got.Status.RecordSets[0].DeletionCompleted || got.Status.RecordSets[0].RecordSetUID != "replacement" { t.Fatal("old deletion finalized replacement") } @@ -240,7 +240,7 @@ func TestStaleAsynchronousResultsDiscarded(t *testing.T) { } return matching(input) } - result, unit := reconcile(t, r, request) + result, unit := reconcileOnce(t, r, request) if result.RequeueAfter != time.Second || len(unit.Status.RecordSets) != 0 { t.Fatalf("stale result committed: %#v", unit.Status) } @@ -351,7 +351,7 @@ func TestZoneAuthorityEvidenceSharedAndBlocksRecords(t *testing.T) { result.Views = append(result.Views, authority) return result, err } - _, unit := reconcile(t, r, request) + _, unit := reconcileOnce(t, r, request) if zoneCalls != 1 || len(resolver.requests) != 3 { t.Fatalf("zoneCalls=%d probes=%d", zoneCalls, len(resolver.requests)) } @@ -361,7 +361,7 @@ func TestZoneAuthorityEvidenceSharedAndBlocksRecords(t *testing.T) { } wrong = true *now = now.Add(30 * time.Second) - _, unit = reconcile(t, r, request) + _, unit = reconcileOnce(t, r, request) if zoneCalls != 2 || len(resolver.requests) != 4 { t.Fatal("failed authority check still probed child records") } @@ -376,7 +376,7 @@ func TestZoneAuthorityEvidenceSharedAndBlocksRecords(t *testing.T) { func TestNamespacePolicyChangeInvalidatesSuccess(t *testing.T) { resolver := &fakeResolver{answer: matching} r, c, _, request := newFixture(t, 1, resolver) - reconcile(t, r, request) + reconcileOnce(t, r, request) var namespace corev1.Namespace if err := c.Get(context.Background(), client.ObjectKey{Name: "app"}, &namespace); err != nil { t.Fatal(err) @@ -385,7 +385,7 @@ func TestNamespacePolicyChangeInvalidatesSuccess(t *testing.T) { if err := c.Update(context.Background(), &namespace); err != nil { t.Fatal(err) } - reconcile(t, r, request) + reconcileOnce(t, r, request) if len(resolver.requests) != 2 { t.Fatal("namespace-policy fingerprint was not invalidated") } @@ -394,7 +394,7 @@ func TestNamespacePolicyChangeInvalidatesSuccess(t *testing.T) { func TestStaleConditionGenerationCannotUseScheduledSuccess(t *testing.T) { resolver := &fakeResolver{answer: matching} r, c, _, request := newFixture(t, 1, resolver) - _, unit := reconcile(t, r, request) + _, unit := reconcileOnce(t, r, request) for i := range unit.Status.RecordSets[0].Conditions { if unit.Status.RecordSets[0].Conditions[i].Type == "Programmed" { unit.Status.RecordSets[0].Conditions[i].ObservedGeneration = 0 @@ -403,7 +403,7 @@ func TestStaleConditionGenerationCannotUseScheduledSuccess(t *testing.T) { if err := c.Status().Update(context.Background(), &unit); err != nil { t.Fatal(err) } - _, current := reconcile(t, r, request) + _, current := reconcileOnce(t, r, request) if len(resolver.requests) != 2 { t.Fatal("stale condition reused a future scheduling receipt") } @@ -412,3 +412,41 @@ func TestStaleConditionGenerationCannotUseScheduledSuccess(t *testing.T) { t.Fatal("stale result was rebadged instead of reprobed") } } + +func TestUnchangedSchedulerJSONDoesNotPatch(t *testing.T) { + for _, probeFails := range []bool{false, true} { + t.Run(fmt.Sprintf("probeFails=%t", probeFails), func(t *testing.T) { + resolver := &fakeResolver{answer: matching} + if probeFails { + resolver.answer = func(probe.ProbeRequest) (probe.ProbeResult, error) { + return probe.ProbeResult{}, errors.New("transport") + } + } + r, c, now, request := newFixture(t, 1, resolver) + _, first := reconcileOnce(t, r, request) + + // Model an API round-trip that changes JSON formatting/key order, + // but neither desired inputs nor the persisted scheduling receipt. + var state map[string]any + if err := json.Unmarshal(first.Status.Provider.State.Raw, &state); err != nil { + t.Fatal(err) + } + encoded, err := json.MarshalIndent(state, "", " ") + if err != nil { + t.Fatal(err) + } + first.Status.Provider.State.Raw = encoded + if err := c.Status().Update(context.Background(), &first); err != nil { + t.Fatal(err) + } + version := first.ResourceVersion + for range 3 { + *now = now.Add(time.Second) + _, unchanged := reconcileOnce(t, r, request) + if len(resolver.requests) != 1 || unchanged.ResourceVersion != version { + t.Fatalf("unchanged scheduling receipt: probes=%d (want 1), resourceVersion=%q -> %q", len(resolver.requests), version, unchanged.ResourceVersion) + } + } + }) + } +} From 0800667fe222b9fa6f5ceb7e3ad1796ad957dd0a Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 11:08:22 +0900 Subject: [PATCH 05/16] docs: trim trailing blank line --- docs/design/external-provider.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/design/external-provider.md b/docs/design/external-provider.md index a2a8339..c5b4ade 100644 --- a/docs/design/external-provider.md +++ b/docs/design/external-provider.md @@ -353,4 +353,3 @@ Exercise external identityRef rejection, Route 53/Cloudflare missing-identity re Cover external-only, each cloud-provider enable combination, all-disabled, and independent Gateway/Service source switches. Assert flags, actual controller/identity-webhook/conversion registrations, rendered Provider/capability/APIService agreement, valid CA wiring, rendering rejection when webhook or cert-manager is disabled for external, and Helm/Kustomize admission parity. External-only must have no cloud identity or Secret RBAC/informers; disabled source controllers must not restore their unrelated permissions. Retain the single intended controller ServiceAccount binding and CRD-copy synchronization contracts. Platform acceptance uses an explicitly authorized run with real customer-controlled DNS and remote-member HTTPPublish Ready, including wrong-target/wrong-NS recovery. Forced-IP HTTP, TLS verification bypass, borrowed synthesis domains, public DNS in CI, and unexecuted test plans are not verification evidence. - From af97f8c3fa2ef5ac7680b8b7137b447b9d00b57f Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 11:16:33 +0900 Subject: [PATCH 06/16] Wire external DNS provider into manager and deployment packaging --- README.md | 67 ++++ app/operator/cmd/manager/main.go | 51 +++ app/operator/cmd/manager/main_test.go | 124 ++++++- .../external/endpoint_capabilities.yaml | 12 + ...ternal_endpoint_conversion_apiservice.yaml | 15 + .../config/external/external_extensions.yaml | 159 +++++++++ .../config/external/kustomization.yaml | 45 +++ .../charts/dns-api/external_contract_test.go | 309 ++++++++++++++++++ deploy/charts/dns-api/rbac_contract_test.go | 3 + ...iservice-external-endpoint-conversion.yaml | 22 ++ .../charts/dns-api/templates/deployment.yaml | 8 + .../endpointprovidercapability-external.yaml | 14 + .../dns-api/templates/provider-external.yaml | 161 +++++++++ deploy/charts/dns-api/templates/rbac.yaml | 8 + deploy/charts/dns-api/values.yaml | 7 + docs/design/controllers-webhooks.md | 2 +- 16 files changed, 1002 insertions(+), 5 deletions(-) create mode 100644 app/operator/config/external/endpoint_capabilities.yaml create mode 100644 app/operator/config/external/external_endpoint_conversion_apiservice.yaml create mode 100644 app/operator/config/external/external_extensions.yaml create mode 100644 app/operator/config/external/kustomization.yaml create mode 100644 deploy/charts/dns-api/external_contract_test.go create mode 100644 deploy/charts/dns-api/templates/apiservice-external-endpoint-conversion.yaml create mode 100644 deploy/charts/dns-api/templates/endpointprovidercapability-external.yaml create mode 100644 deploy/charts/dns-api/templates/provider-external.yaml diff --git a/README.md b/README.md index 4de046f..7cfa014 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,73 @@ ghcr.io/appthrust/dns-api: For local development or source installs, use the chart in [deploy/charts/dns-api](./deploy/charts/dns-api). +### Customer-managed DNS (external provider) + +The external provider is opt-in. It verifies customer-managed `A`, `AAAA`, and +`CNAME` records without creating, updating, or deleting DNS records and needs no +identity or provider credentials. Install cert-manager first, then save these +values as `external-values.yaml` and install from this checkout: + +```yaml +providers: + route53: + enabled: false + cloudflare: + enabled: false + external: + enabled: true + controllerName: external.dns.appthrust.io/controller + providerName: external.dns.appthrust.io + providerVersion: v1alpha1 + dnsQPS: 40 + dnsBurst: 80 +webhook: + enabled: true +certManager: + enabled: true +``` + +```sh +helm install dns-api ./deploy/charts/dns-api \ + --namespace dns-api-system \ + --create-namespace \ + --values external-values.yaml +``` + +Use a controller image built from the same revision (or a release containing the +external provider). The chart rejects external enablement without both webhook +and cert-manager support: endpoint conversion uses the webhook Service with +cert-manager CA injection, never a TLS-verification bypass. External-only installs +grant no cloud identity or Secret API permissions; serving TLS certificates are +mounted by Kubernetes. External ZoneClasses must omit `identityRef`. + +The chart values map to `--external-controller-enabled`, +`--external-controller-name`, `--external-provider-name`, +`--external-provider-version`, `--external-dns-qps`, and `--external-dns-burst`. +When running the manager directly, the first four accept environment defaults +`EXTERNAL_CONTROLLER_ENABLED`, `EXTERNAL_CONTROLLER_NAME`, +`EXTERNAL_PROVIDER_NAME`, and `EXTERNAL_PROVIDER_VERSION`; explicit flags win. +The defaults are the values above except enablement, which defaults to `false`. +DNS query limits apply per controller process. + +For Kustomize, `app/operator/config/default` remains external-disabled. +`app/operator/config/external` is the opt-in overlay that adds external controllers, +Provider, capability, TLS-backed APIService, and conversion-create permission to +the default Route 53/Cloudflare installation. Disabling cloud providers in a +custom overlay must also remove their identity admission entries and RBAC (and +Cloudflare's Secret permission). + +For a dedicated-member platform with base domain `B` and **M Projects**, customers +maintain **M+2 DNS record names**: `B` and `*.B` point at the PMC public edge, and +each `*..B` points at that Project's actual serving Gateway addresses. +Publish the required A/AAAA sets (or a supported CNAME) for each name; do not point +a shared wildcard at the union of reserved member VIPs. Verification requires +exact target sets, including the absence of unexpected address-family answers. +Wildcard sampling does not replace probing each exact application hostname. +Cross-namespace wildcard claims also require an explicit `allowedRecordSets.records` +grant. See the [external provider design](./docs/design/external-provider.md) for +resolver policy, readiness, and wildcard limits. + ## First Resources A typical setup has two sides. diff --git a/app/operator/cmd/manager/main.go b/app/operator/cmd/manager/main.go index fcfb8b8..365baa7 100644 --- a/app/operator/cmd/manager/main.go +++ b/app/operator/cmd/manager/main.go @@ -14,6 +14,10 @@ import ( cloudflarezoneclass "github.com/appthrust/dns-api/internal/go/providers/cloudflare/controllers/zoneclass" cloudflarezoneunit "github.com/appthrust/dns-api/internal/go/providers/cloudflare/controllers/zoneunit" cloudflarewebhook "github.com/appthrust/dns-api/internal/go/providers/cloudflare/webhook" + externalzoneclass "github.com/appthrust/dns-api/internal/go/providers/external/controllers/zoneclass" + externalzoneunit "github.com/appthrust/dns-api/internal/go/providers/external/controllers/zoneunit" + externalconversion "github.com/appthrust/dns-api/internal/go/providers/external/conversion" + externalprobe "github.com/appthrust/dns-api/internal/go/providers/external/probe" route53identity "github.com/appthrust/dns-api/internal/go/providers/route53/controllers/identity" route53zoneclass "github.com/appthrust/dns-api/internal/go/providers/route53/controllers/zoneclass" route53zoneunit "github.com/appthrust/dns-api/internal/go/providers/route53/controllers/zoneunit" @@ -23,6 +27,7 @@ import ( dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" endpointconversionv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/conversion/v1alpha1" endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" route53v1alpha1 "github.com/appthrust/dns-api/pkg/go/api/route53/v1alpha1" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" @@ -73,6 +78,7 @@ func init() { utilruntime.Must(dnsv1alpha1.AddToScheme(scheme)) utilruntime.Must(endpointv1alpha1.AddToScheme(scheme)) utilruntime.Must(endpointconversionv1alpha1.AddToScheme(scheme)) + utilruntime.Must(externalv1alpha1.AddToScheme(scheme)) utilruntime.Must(gatewayv1.Install(scheme)) utilruntime.Must(route53v1alpha1.AddToScheme(scheme)) } @@ -85,6 +91,7 @@ func main() { var serviceEndpointControllerEnabled bool route53ControllerEnabled := envBoolOrDefault("ROUTE53_CONTROLLER_ENABLED", true) cloudflareControllerEnabled := envBoolOrDefault("CLOUDFLARE_CONTROLLER_ENABLED", true) + externalControllerEnabled := envBoolOrDefault("EXTERNAL_CONTROLLER_ENABLED", false) route53EndpointConversionEnabled := envBoolOrDefault("ROUTE53_ENDPOINT_CONVERSION_ENABLED", true) endpointRecordSetNamespace := envOrDefault("ENDPOINT_RECORDSET_NAMESPACE", "") route53ControllerName := envOrDefault("ROUTE53_CONTROLLER_NAME", route53zoneunit.DefaultControllerName) @@ -93,6 +100,11 @@ func main() { cloudflareControllerName := envOrDefault("CLOUDFLARE_CONTROLLER_NAME", cloudflarezoneunit.DefaultControllerName) cloudflareProviderName := envOrDefault("CLOUDFLARE_PROVIDER_NAME", cloudflarev1alpha1.ProviderName) cloudflareProviderVersion := envOrDefault("CLOUDFLARE_PROVIDER_VERSION", cloudflarev1alpha1.ProviderVersion) + externalControllerName := envOrDefault("EXTERNAL_CONTROLLER_NAME", externalv1alpha1.ControllerName) + externalProviderName := envOrDefault("EXTERNAL_PROVIDER_NAME", externalv1alpha1.ProviderName) + externalProviderVersion := envOrDefault("EXTERNAL_PROVIDER_VERSION", externalv1alpha1.ProviderVersion) + var externalDNSQPS float64 + var externalDNSBurst int flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "The address the metric endpoint binds to.") flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.") @@ -102,6 +114,7 @@ func main() { flag.StringVar(&endpointRecordSetNamespace, "endpoint-recordset-namespace", endpointRecordSetNamespace, "Namespace where generated EndpointRecordSet and RecordSet resources are stored. Defaults to the source namespace.") flag.BoolVar(&route53ControllerEnabled, "route53-controller-enabled", route53ControllerEnabled, "Enable Route 53 controllers and identity admission.") flag.BoolVar(&cloudflareControllerEnabled, "cloudflare-controller-enabled", cloudflareControllerEnabled, "Enable Cloudflare controllers and identity admission.") + flag.BoolVar(&externalControllerEnabled, "external-controller-enabled", externalControllerEnabled, "Enable external DNS verification controllers and endpoint conversion.") flag.BoolVar(&route53EndpointConversionEnabled, "route53-endpoint-conversion-enabled", route53EndpointConversionEnabled, "Serve endpoint conversion when the Route 53 provider is enabled.") flag.StringVar(&route53ControllerName, "route53-controller-name", route53ControllerName, "ZoneClass.spec.controllerName handled by the Route 53 controller.") flag.StringVar(&route53ProviderName, "route53-provider-name", route53ProviderName, "Provider.metadata.name handled by the Route 53 controller.") @@ -109,6 +122,11 @@ func main() { flag.StringVar(&cloudflareControllerName, "cloudflare-controller-name", cloudflareControllerName, "ZoneClass.spec.controllerName handled by the Cloudflare controller.") flag.StringVar(&cloudflareProviderName, "cloudflare-provider-name", cloudflareProviderName, "Provider.metadata.name handled by the Cloudflare controller.") flag.StringVar(&cloudflareProviderVersion, "cloudflare-provider-version", cloudflareProviderVersion, "Provider version handled by the Cloudflare controller.") + flag.StringVar(&externalControllerName, "external-controller-name", externalControllerName, "ZoneClass.spec.controllerName handled by the external controller.") + flag.StringVar(&externalProviderName, "external-provider-name", externalProviderName, "Provider.metadata.name handled by the external controller.") + flag.StringVar(&externalProviderVersion, "external-provider-version", externalProviderVersion, "Provider version handled by the external controller.") + flag.Float64Var(&externalDNSQPS, "external-dns-qps", externalprobe.DefaultQPS, "Maximum external DNS query rate per controller process.") + flag.IntVar(&externalDNSBurst, "external-dns-burst", externalprobe.DefaultBurst, "Maximum external DNS query burst per controller process.") zapOpts := zap.Options{Development: true} zapOpts.BindFlags(flag.CommandLine) @@ -198,6 +216,36 @@ func main() { } } + if externalControllerEnabled { + if err := (&externalzoneclass.ZoneClassReconciler{ + Client: mgr.GetClient(), + ControllerName: externalControllerName, + ProviderName: externalProviderName, + ProviderVersion: externalProviderVersion, + }).SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up external ZoneClass controller") + os.Exit(1) + } + + // NewReconciler supplies the wire resolver, its effective pod DNS policy, + // time.Now, and random jitter when their injection hooks are unset. + externalReconciler, err := externalzoneunit.NewReconciler(mgr.GetClient(), externalzoneunit.Options{ + ControllerName: externalControllerName, + ProviderName: externalProviderName, + ProviderVersion: externalProviderVersion, + QPS: externalDNSQPS, + Burst: externalDNSBurst, + }) + if err != nil { + ctrl.Log.Error(err, "unable to configure external DNS resolver") + os.Exit(1) + } + if err := externalReconciler.SetupWithManager(mgr); err != nil { + ctrl.Log.Error(err, "unable to set up external ZoneUnit controller") + os.Exit(1) + } + } + if err := (&zoneunitcontroller.ZoneUnitCompositionReconciler{ Client: mgr.GetClient(), }).SetupWithManager(mgr); err != nil { @@ -256,6 +304,9 @@ func main() { os.Exit(1) } } + if externalControllerEnabled { + externalconversion.NewHandler().Register(mgr.GetWebhookServer()) + } if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { ctrl.Log.Error(err, "unable to set up health check") diff --git a/app/operator/cmd/manager/main_test.go b/app/operator/cmd/manager/main_test.go index 2c21b9b..26e9460 100644 --- a/app/operator/cmd/manager/main_test.go +++ b/app/operator/cmd/manager/main_test.go @@ -1,23 +1,139 @@ package main -import "testing" +import ( + "encoding/json" + "flag" + "os" + "os/exec" + "strings" + "testing" + + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" +) func TestProviderEnablementEnvironmentDefaults(t *testing.T) { - for _, name := range []string{"ROUTE53_CONTROLLER_ENABLED", "CLOUDFLARE_CONTROLLER_ENABLED", "ROUTE53_ENDPOINT_CONVERSION_ENABLED"} { + for name, fallback := range map[string]bool{ + "ROUTE53_CONTROLLER_ENABLED": true, + "CLOUDFLARE_CONTROLLER_ENABLED": true, + "ROUTE53_ENDPOINT_CONVERSION_ENABLED": true, + "EXTERNAL_CONTROLLER_ENABLED": false, + } { t.Run(name, func(t *testing.T) { for _, test := range []struct { value string want bool }{ - {value: "", want: true}, + {value: "", want: fallback}, {value: "false", want: false}, {value: "true", want: true}, } { t.Setenv(name, test.value) - if got := envBoolOrDefault(name, true); got != test.want { + if got := envBoolOrDefault(name, fallback); got != test.want { t.Fatalf("envBoolOrDefault(%q) with %q = %t, want %t", name, test.value, got, test.want) } } }) } } + +func TestExternalSchemeRegistered(t *testing.T) { + if !scheme.IsVersionRegistered(externalv1alpha1.SchemeGroupVersion) { + t.Fatal("external API scheme is not registered") + } +} + +// Run the real manager's flag binding in a subprocess. Help exits before any +// Kubernetes client, resolver, controller, or webhook is constructed. +func TestManagerFlagsProcess(t *testing.T) { + if os.Getenv("DNS_API_TEST_MANAGER_FLAGS") != "1" { + return + } + for i, arg := range os.Args { + if arg == "--" { + os.Args = append([]string{os.Args[0]}, os.Args[i+1:]...) + break + } + } + flag.CommandLine = flag.NewFlagSet("manager", flag.ExitOnError) + flag.CommandLine.Usage = func() { + values := map[string]string{} + flag.CommandLine.VisitAll(func(f *flag.Flag) { values[f.Name] = f.Value.String() }) + if err := json.NewEncoder(os.Stdout).Encode(values); err != nil { + os.Exit(1) + } + } + main() +} + +func TestManagerProviderFlags(t *testing.T) { + defaults := map[string]string{ + "route53-controller-enabled": "true", + "cloudflare-controller-enabled": "true", + "route53-endpoint-conversion-enabled": "true", + "external-controller-enabled": "false", + "route53-controller-name": "route53.dns.appthrust.io/controller", + "route53-provider-name": "route53.dns.appthrust.io", + "route53-provider-version": "v1alpha1", + "cloudflare-controller-name": "cloudflare.dns.appthrust.io/controller", + "cloudflare-provider-name": "cloudflare.dns.appthrust.io", + "cloudflare-provider-version": "v1alpha1", + "external-controller-name": "external.dns.appthrust.io/controller", + "external-provider-name": "external.dns.appthrust.io", + "external-provider-version": "v1alpha1", + "external-dns-qps": "40", + "external-dns-burst": "80", + } + for _, mode := range []string{"defaults", "environment", "flags-override-environment"} { + t.Run(mode, func(t *testing.T) { + env := append(os.Environ(), "DNS_API_TEST_MANAGER_FLAGS=1") + args := []string{"-test.run=^TestManagerFlagsProcess$", "--"} + expected := map[string]string{} + for name, fallback := range defaults { + value := "" + want := fallback + hasEnvironment := name != "external-dns-qps" && name != "external-dns-burst" + if mode != "defaults" && hasEnvironment { + if strings.HasSuffix(name, "-enabled") { + value = "false" + if fallback == "false" { + value = "true" + } + } else { + value = "environment-" + name + } + want = value + } + if hasEnvironment { + env = append(env, strings.ToUpper(strings.ReplaceAll(name, "-", "_"))+"="+value) + } + if mode == "flags-override-environment" { + want = fallback + if name == "external-dns-qps" { + want = "12.5" + } + if name == "external-dns-burst" { + want = "25" + } + args = append(args, "--"+name+"="+want) + } + expected[name] = want + } + args = append(args, "--help") + cmd := exec.Command(os.Args[0], args...) + cmd.Env = env + output, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("manager flag parsing failed: %v\n%s", err, output) + } + var actual map[string]string + if err := json.Unmarshal(output, &actual); err != nil { + t.Fatalf("decode manager flags: %v\n%s", err, output) + } + for name, want := range expected { + if actual[name] != want { + t.Errorf("--%s = %q, want %q", name, actual[name], want) + } + } + }) + } +} diff --git a/app/operator/config/external/endpoint_capabilities.yaml b/app/operator/config/external/endpoint_capabilities.yaml new file mode 100644 index 0000000..9d07fd3 --- /dev/null +++ b/app/operator/config/external/endpoint_capabilities.yaml @@ -0,0 +1,12 @@ +apiVersion: endpoint.dns.appthrust.io/v1alpha1 +kind: EndpointProviderCapability +metadata: + name: external-v1alpha1 +spec: + provider: + name: external.dns.appthrust.io + version: v1alpha1 + conversion: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + resource: endpointrecordsetconversions diff --git a/app/operator/config/external/external_endpoint_conversion_apiservice.yaml b/app/operator/config/external/external_endpoint_conversion_apiservice.yaml new file mode 100644 index 0000000..3e6924a --- /dev/null +++ b/app/operator/config/external/external_endpoint_conversion_apiservice.yaml @@ -0,0 +1,15 @@ +apiVersion: apiregistration.k8s.io/v1 +kind: APIService +metadata: + name: v1alpha1.endpoint.external.dns.appthrust.io + annotations: + cert-manager.io/inject-ca-from: dns-api-system/dns-api-webhook-serving-cert +spec: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + groupPriorityMinimum: 1000 + versionPriority: 15 + service: + name: dns-api-webhook-service + namespace: dns-api-system + port: 443 diff --git a/app/operator/config/external/external_extensions.yaml b/app/operator/config/external/external_extensions.yaml new file mode 100644 index 0000000..a143394 --- /dev/null +++ b/app/operator/config/external/external_extensions.yaml @@ -0,0 +1,159 @@ +apiVersion: dns.appthrust.io/v1alpha1 +kind: Provider +metadata: + name: external.dns.appthrust.io +spec: + display: + name: External DNS + description: Verifies customer-managed DNS without changing DNS records. + versions: + - name: v1alpha1 + served: true + storage: true + zoneClass: + schemas: + parameters: + openAPIV3Schema: + type: object + description: DNS resolver policy for verification without DNS writes. + additionalProperties: false + properties: + resolver: + type: object + description: Resolver view used for every verification query. + additionalProperties: false + properties: + mode: + type: string + description: Cluster uses pod DNS; Recursive uses every configured server. + enum: [Cluster, Recursive] + servers: + type: array + description: Explicit recursive DNS IP endpoints; never authoritative NS names. + maxItems: 3 + uniqueItems: true + items: + type: string + minLength: 3 + maxLength: 64 + pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' + expectedNameservers: + type: array + description: Expected authoritative nameservers for the exact Zone domain. + maxItems: 16 + uniqueItems: true + items: + type: string + minLength: 1 + maxLength: 253 + pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' + validationRules: + - rule: >- + has(self.spec.parameters.resolver) && + has(self.spec.parameters.resolver.mode) && + self.spec.parameters.resolver.mode == 'Recursive' + ? (has(self.spec.parameters.resolver.servers) && size(self.spec.parameters.resolver.servers) > 0) + : (!has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || size(self.spec.parameters.resolver.servers) == 0) + message: resolver.servers must be nonempty for Recursive and empty for Cluster + - rule: >- + !has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || + self.spec.parameters.resolver.servers.all(s, + cel.bind(host, s.startsWith('[') ? s.split(']')[0].substring(1) : s.split(':')[0], + cel.bind(port, s.startsWith('[') ? s.split(']:')[1] : s.split(':')[1], + isIP(host) && ip.isCanonical(host) && + (s.startsWith('[') ? ip(host).family() == 6 : ip(host).family() == 4) && + int(port) >= 1 && int(port) <= 65535))) + message: resolver.servers must contain canonical IP addresses and ports from 1 through 65535 + - rule: >- + !has(self.spec.parameters.expectedNameservers) || + self.spec.parameters.expectedNameservers.all(n, !isIP(n)) + message: expectedNameservers must contain DNS names, not IP addresses + zone: + schemas: + adoption: &empty + openAPIV3Schema: + type: object + additionalProperties: false + properties: {} + statusProviderData: &status + openAPIV3Schema: + type: object + description: Bounded read-only DNS verification evidence; not DNS withdrawal proof. + additionalProperties: false + properties: + resolverMode: + type: string + description: Selected recursive resolver view policy. + enum: [Cluster, Recursive] + checkedName: + type: string + description: Absolute DNS owner checked, without the root dot. + maxLength: 253 + sampledWildcard: + type: boolean + description: Whether checkedName is a stable concrete wildcard witness. + actual: &preview + type: object + description: Sorted bounded diagnostic preview; equality uses the complete observation. + additionalProperties: false + required: [total, truncated] + properties: + values: + type: array + description: Up to sixteen typed values or diagnostic entries. + maxItems: 16 + items: + type: string + maxLength: 512 + total: + type: integer + description: Total entries before truncation. + minimum: 0 + truncated: + type: boolean + description: Whether values omits entries from the complete observation. + missing: *preview + extra: *preview + cnameChain: *preview + views: *preview + observedTTLSeconds: + type: integer + description: Minimum relevant positive TTL in seconds, when observed. + minimum: 0 + negativeTTLSeconds: + type: integer + description: Minimum of negative SOA TTL and SOA MINIMUM, when observed. + minimum: 0 + negativeExpiryTime: + type: string + format: date-time + description: Negative cache expiry suggested by the observed SOA, not the polling cap. + lastProbeTime: + type: string + format: date-time + description: Last completed attempt, including failures. + nextProbeTime: + type: string + format: date-time + description: Approximate next planned attempt; queue load can delay it. + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: VerificationConfigured, description: Verifier configured without authority checks.} + - {conditionType: Programmed, status: 'True', reason: NameserversVerified, description: Every required view verified the exact apex and nameservers.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Observed apex or nameservers differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordSet: + supportedTypes: [A, AAAA, CNAME] + schemas: + options: *empty + adoption: *empty + statusProviderData: *status + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: ResolutionVerified, description: Exact targets verified in every required view.} + - {conditionType: Programmed, status: 'False', reason: DNSNameNotFound, description: Required view returned NXDOMAIN.} + - {conditionType: Programmed, status: 'False', reason: DNSNoData, description: Required record type has no answers.} + - {conditionType: Programmed, status: 'False', reason: ResolutionPartial, description: A nonempty proper subset of targets was observed.} + - {conditionType: Programmed, status: 'False', reason: ResolutionMismatch, description: Targets or CNAME chain differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} diff --git a/app/operator/config/external/kustomization.yaml b/app/operator/config/external/kustomization.yaml new file mode 100644 index 0000000..15e42db --- /dev/null +++ b/app/operator/config/external/kustomization.yaml @@ -0,0 +1,45 @@ +resources: + - ../default + - external_extensions.yaml + - endpoint_capabilities.yaml + - external_endpoint_conversion_apiservice.yaml +patches: + - target: + group: apps + version: v1 + kind: Deployment + name: dns-api-controller-manager + patch: |- + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-controller-enabled=true + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-controller-name=external.dns.appthrust.io/controller + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-provider-name=external.dns.appthrust.io + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-provider-version=v1alpha1 + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-dns-qps=40 + - op: add + path: /spec/template/spec/containers/0/args/- + value: --external-dns-burst=80 + - target: + group: rbac.authorization.k8s.io + version: v1 + kind: ClusterRole + name: dns-api-controller-manager-role + patch: |- + - op: add + path: /rules/- + value: + apiGroups: + - endpoint.external.dns.appthrust.io + resources: + - endpointrecordsetconversions + verbs: + - create diff --git a/deploy/charts/dns-api/external_contract_test.go b/deploy/charts/dns-api/external_contract_test.go new file mode 100644 index 0000000..7507f28 --- /dev/null +++ b/deploy/charts/dns-api/external_contract_test.go @@ -0,0 +1,309 @@ +package dnsapichart + +import ( + "fmt" + "os" + "os/exec" + "path/filepath" + "reflect" + "runtime" + "strconv" + "strings" + "testing" + + externalmanifest "github.com/appthrust/dns-api/internal/go/providers/external/manifest" + rbacv1 "k8s.io/api/rbac/v1" + "sigs.k8s.io/yaml" +) + +func TestExternalOnlyRender(t *testing.T) { + for _, gateway := range []bool{false, true} { + for _, service := range []bool{false, true} { + t.Run(fmt.Sprintf("gateway=%t/service=%t", gateway, service), func(t *testing.T) { + rendered := renderChart(t, + "--set", "providers.route53.enabled=false", + "--set", "providers.cloudflare.enabled=false", + "--set", "providers.external.enabled=true", + "--set", "gatewayEndpoint.enabled="+strconv.FormatBool(gateway), + "--set", "serviceEndpoint.enabled="+strconv.FormatBool(service), + ) + role := renderedDocument(t, rendered, "rbac.authorization.k8s.io/v1", "ClusterRole") + for _, forbidden := range []string{"secrets", "identities", "route53.dns.appthrust.io", "cloudflare.dns.appthrust.io"} { + if strings.Contains(role, forbidden) { + t.Errorf("external-only role contains %q:\n%s", forbidden, role) + } + } + for permission, enabled := range map[string]bool{ + "- gateway.networking.k8s.io\n": gateway, + "- services\n": service, + } { + if strings.Contains(role, permission) != enabled { + t.Errorf("permission %q presence does not equal %t", permission, enabled) + } + } + assertExternalConversionRule(t, role) + webhooks := renderedDocument(t, rendered, "admissionregistration.k8s.io/v1", "ValidatingWebhookConfiguration") + if strings.Contains(webhooks, "identities") { + t.Fatal("external-only installation includes identity admission") + } + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + for _, arg := range []string{ + "--route53-controller-enabled=false", + "--cloudflare-controller-enabled=false", + "--route53-endpoint-conversion-enabled=false", + "--external-controller-enabled=true", + "--external-controller-name=external.dns.appthrust.io/controller", + "--external-provider-name=external.dns.appthrust.io", + "--external-provider-version=v1alpha1", + "--external-dns-qps=40", + "--external-dns-burst=80", + } { + if !strings.Contains(deployment, arg) { + t.Errorf("external-only Deployment is missing %q", arg) + } + } + for _, kind := range []string{"Provider", "EndpointProviderCapability", "APIService", "ClusterRoleBinding"} { + if count := strings.Count(rendered, "kind: "+kind+"\n"); count != 1 { + t.Errorf("got %d %s declarations, want exactly one", count, kind) + } + } + assertManifestEqual(t, externalmanifest.ProviderYAML, renderedDocument(t, rendered, "dns.appthrust.io/v1alpha1", "Provider")) + assertManifestEqual(t, externalmanifest.CapabilityYAML, renderedDocument(t, rendered, "endpoint.dns.appthrust.io/v1alpha1", "EndpointProviderCapability")) + assertExternalTLS(t, rendered) + }) + } + } +} + +func TestExternalCanCoexistWithCloudProviders(t *testing.T) { + for _, route53 := range []bool{false, true} { + for _, cloudflare := range []bool{false, true} { + t.Run(fmt.Sprintf("route53=%t/cloudflare=%t", route53, cloudflare), func(t *testing.T) { + rendered := renderChart(t, + "--set", "providers.external.enabled=true", + "--set", "providers.route53.enabled="+strconv.FormatBool(route53), + "--set", "providers.cloudflare.enabled="+strconv.FormatBool(cloudflare), + ) + role := renderedDocument(t, rendered, "rbac.authorization.k8s.io/v1", "ClusterRole") + assertExternalConversionRule(t, role) + for permission, enabled := range map[string]bool{ + "- route53.dns.appthrust.io\n": route53, + "- endpoint.route53.dns.appthrust.io\n": route53, + "- cloudflare.dns.appthrust.io\n": cloudflare, + "- secrets\n": cloudflare, + } { + if strings.Contains(role, permission) != enabled { + t.Errorf("permission %q presence does not equal %t", permission, enabled) + } + } + }) + } + } +} + +func TestExternalRequiresWebhookAndCertManager(t *testing.T) { + for _, webhook := range []bool{false, true} { + for _, certManager := range []bool{false, true} { + if webhook && certManager { + continue + } + t.Run(fmt.Sprintf("webhook=%t/certManager=%t", webhook, certManager), func(t *testing.T) { + _, file, _, _ := runtime.Caller(0) + output, err := exec.Command("helm", "template", "dns-api", filepath.Dir(file), + "--set", "providers.external.enabled=true", + "--set", "webhook.enabled="+strconv.FormatBool(webhook), + "--set", "certManager.enabled="+strconv.FormatBool(certManager), + ).CombinedOutput() + if err == nil { + t.Fatal("external enablement without serving TLS succeeded") + } + if !strings.Contains(string(output), "providers.external.enabled requires webhook.enabled=true and certManager.enabled=true") { + t.Fatalf("unexpected render failure: %v\n%s", err, output) + } + }) + } + } +} + +func TestExternalDefaultsToDisabled(t *testing.T) { + rendered := renderChart(t) + if explicit := renderChart(t, "--set", "providers.external.enabled=false"); rendered != explicit { + t.Fatal("default render differs from explicitly disabled external provider") + } + for _, forbidden := range []string{"external.dns.appthrust.io", "--external-dns-", "--external-provider-"} { + if strings.Contains(rendered, forbidden) { + t.Errorf("default render contains external declaration %q", forbidden) + } + } + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + for _, arg := range []string{"--route53-controller-enabled=true", "--cloudflare-controller-enabled=true", "--route53-endpoint-conversion-enabled=true", "--external-controller-enabled=false"} { + if !strings.Contains(deployment, arg) { + t.Errorf("default Deployment is missing %q", arg) + } + } + // External's strict TLS requirement must not change existing disabled-provider renders. + renderChart(t, "--set", "webhook.enabled=false", "--set", "certManager.enabled=false") +} + +func TestExternalCustomBindings(t *testing.T) { + rendered := renderChart(t, + "--set", "providers.external.enabled=true", + "--set", "providers.route53.enabled=false", + "--set", "providers.cloudflare.enabled=false", + "--set", "providers.external.controllerName=custom.example/controller", + "--set", "providers.external.providerName=custom.example", + "--set", "providers.external.providerVersion=v1beta1", + "--set", "providers.external.dnsQPS=12.5", + "--set", "providers.external.dnsBurst=25", + "--set", "fullnameOverride=custom-dns", + "--set", "webhook.service.port=8443", + ) + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + for _, arg := range []string{"--external-controller-name=custom.example/controller", "--external-provider-name=custom.example", "--external-provider-version=v1beta1", "--external-dns-qps=12.5", "--external-dns-burst=25"} { + if !strings.Contains(deployment, arg) { + t.Errorf("custom Deployment is missing %q", arg) + } + } + provider := manifestMap(t, renderedDocument(t, rendered, "dns.appthrust.io/v1alpha1", "Provider")) + if provider["metadata"].(map[string]any)["name"] != "custom.example" || provider["spec"].(map[string]any)["versions"].([]any)[0].(map[string]any)["name"] != "v1beta1" { + t.Fatal("Provider does not match custom runtime binding") + } + capability := manifestMap(t, renderedDocument(t, rendered, "endpoint.dns.appthrust.io/v1alpha1", "EndpointProviderCapability")) + spec := capability["spec"].(map[string]any) + if !reflect.DeepEqual(spec["provider"], map[string]any{"name": "custom.example", "version": "v1beta1"}) { + t.Fatalf("capability does not match custom binding: %#v", spec) + } + // Provider versions are configurable; the stateless conversion protocol is not. + if !reflect.DeepEqual(spec["conversion"], map[string]any{"group": "endpoint.external.dns.appthrust.io", "version": "v1alpha1", "resource": "endpointrecordsetconversions"}) { + t.Fatalf("custom binding changed the conversion GVR: %#v", spec) + } + assertExternalTLS(t, rendered) +} + +func TestExternalKustomizeParity(t *testing.T) { + _, file, _, _ := runtime.Caller(0) + configDir := filepath.Join(filepath.Dir(file), "../../../app/operator/config") + for name, expected := range map[string][]byte{"external_extensions.yaml": externalmanifest.ProviderYAML, "endpoint_capabilities.yaml": externalmanifest.CapabilityYAML} { + actual, err := os.ReadFile(filepath.Join(configDir, "external", name)) + if err != nil { + t.Fatal(err) + } + assertManifestEqual(t, expected, string(actual)) + } + output, err := exec.Command("kustomize", "build", filepath.Join(configDir, "external")).CombinedOutput() + if err != nil { + t.Fatalf("external kustomize build failed: %v\n%s", err, output) + } + rendered := string(output) + assertExternalTLS(t, rendered) + role := renderedDocument(t, rendered, "rbac.authorization.k8s.io/v1", "ClusterRole") + assertExternalConversionRule(t, role) + deployment := renderedDocument(t, rendered, "apps/v1", "Deployment") + for _, arg := range []string{"--external-controller-enabled=true", "--external-controller-name=external.dns.appthrust.io/controller", "--external-provider-name=external.dns.appthrust.io", "--external-provider-version=v1alpha1", "--external-dns-qps=40", "--external-dns-burst=80"} { + if !strings.Contains(deployment, arg) { + t.Errorf("Kustomize Deployment is missing %q", arg) + } + } + base, err := exec.Command("kustomize", "build", filepath.Join(configDir, "default")).CombinedOutput() + if err != nil { + t.Fatalf("default kustomize build failed: %v\n%s", err, base) + } + if strings.Contains(string(base), "external.dns.appthrust.io") || strings.Contains(string(base), "--external-controller-enabled=true") { + t.Fatal("Kustomize default installation enables external DNS") + } + var baseRole, externalRole rbacv1.ClusterRole + if err := yaml.Unmarshal([]byte(renderedDocument(t, string(base), "rbac.authorization.k8s.io/v1", "ClusterRole")), &baseRole); err != nil { + t.Fatal(err) + } + if err := yaml.Unmarshal([]byte(role), &externalRole); err != nil { + t.Fatal(err) + } + var unchanged []rbacv1.PolicyRule + for _, rule := range externalRole.Rules { + if !reflect.DeepEqual(rule.APIGroups, []string{"endpoint.external.dns.appthrust.io"}) { + unchanged = append(unchanged, rule) + } + } + if !reflect.DeepEqual(baseRole.Rules, unchanged) { + t.Fatal("external overlay changes permissions beyond its conversion-create rule") + } +} + +func assertExternalConversionRule(t *testing.T, source string) { + t.Helper() + var role rbacv1.ClusterRole + if err := yaml.Unmarshal([]byte(source), &role); err != nil { + t.Fatal(err) + } + expected := rbacv1.PolicyRule{APIGroups: []string{"endpoint.external.dns.appthrust.io"}, Resources: []string{"endpointrecordsetconversions"}, Verbs: []string{"create"}} + count := 0 + for _, rule := range role.Rules { + for _, group := range rule.APIGroups { + if group == "endpoint.external.dns.appthrust.io" { + count++ + if !reflect.DeepEqual(rule, expected) { + t.Errorf("external conversion permission differs: %#v", rule) + } + } + } + } + if count != 1 { + t.Errorf("got %d external conversion rules, want exactly one", count) + } +} + +func assertExternalTLS(t *testing.T, rendered string) { + t.Helper() + var apiService map[string]any + for _, document := range strings.Split(rendered, "\n---") { + if strings.Contains(document, "kind: APIService\n") && strings.Contains(document, "name: v1alpha1.endpoint.external.dns.appthrust.io\n") { + apiService = manifestMap(t, document) + } + } + if apiService == nil { + t.Fatal("external conversion APIService is missing") + } + spec := apiService["spec"].(map[string]any) + if _, exists := spec["insecureSkipTLSVerify"]; exists { + t.Fatal("external APIService bypasses TLS verification") + } + if spec["group"] != "endpoint.external.dns.appthrust.io" || spec["version"] != "v1alpha1" { + t.Fatalf("external APIService GVR differs: %#v", spec) + } + certificate := manifestMap(t, renderedDocument(t, rendered, "cert-manager.io/v1", "Certificate")) + certMetadata := certificate["metadata"].(map[string]any) + annotation := apiService["metadata"].(map[string]any)["annotations"].(map[string]any)["cert-manager.io/inject-ca-from"] + if annotation != certMetadata["namespace"].(string)+"/"+certMetadata["name"].(string) { + t.Fatalf("CA injection does not reference serving Certificate: %v", annotation) + } + service := manifestMap(t, renderedDocument(t, rendered, "v1", "Service")) + serviceMetadata := service["metadata"].(map[string]any) + serviceRef := spec["service"].(map[string]any) + servicePort := service["spec"].(map[string]any)["ports"].([]any)[0].(map[string]any)["port"] + if serviceRef["namespace"] != serviceMetadata["namespace"] || serviceRef["name"] != serviceMetadata["name"] || serviceRef["port"] != servicePort { + t.Fatalf("APIService does not reference webhook Service: %#v", serviceRef) + } + wantDNSName := serviceRef["name"].(string) + "." + serviceRef["namespace"].(string) + ".svc" + for _, name := range certificate["spec"].(map[string]any)["dnsNames"].([]any) { + if name == wantDNSName { + return + } + } + t.Fatalf("Certificate does not cover webhook Service DNS name %s", wantDNSName) +} + +func assertManifestEqual(t *testing.T, expected []byte, actual string) { + t.Helper() + if !reflect.DeepEqual(manifestMap(t, string(expected)), manifestMap(t, actual)) { + t.Fatalf("packaged manifest differs from embedded source:\n%s", actual) + } +} + +func manifestMap(t *testing.T, source string) map[string]any { + t.Helper() + var result map[string]any + if err := yaml.Unmarshal([]byte(source), &result); err != nil { + t.Fatalf("decode manifest: %v", err) + } + return result +} diff --git a/deploy/charts/dns-api/rbac_contract_test.go b/deploy/charts/dns-api/rbac_contract_test.go index 9bc04be..9bad00d 100644 --- a/deploy/charts/dns-api/rbac_contract_test.go +++ b/deploy/charts/dns-api/rbac_contract_test.go @@ -79,6 +79,7 @@ func TestProviderEnablementGatesRuntimePermissionsAndAdmission(t *testing.T) { "route53-controller-enabled": route53, "cloudflare-controller-enabled": cloudflare, "route53-endpoint-conversion-enabled": route53 && conversion, + "external-controller-enabled": false, } { if !strings.Contains(deployment, "--"+flag+"="+strconv.FormatBool(enabled)) { t.Fatalf("Deployment does not pass %s=%t:\n%s", flag, enabled, deployment) @@ -89,6 +90,7 @@ func TestProviderEnablementGatesRuntimePermissionsAndAdmission(t *testing.T) { "- cloudflare.dns.appthrust.io\n": cloudflare, "- secrets\n": cloudflare, "- endpoint.route53.dns.appthrust.io\n": route53 && conversion, + "- endpoint.external.dns.appthrust.io\n": false, "- gateway.networking.k8s.io\n": false, "- services\n": false, "- namespaces\n": true, @@ -113,6 +115,7 @@ func TestProviderEnablementGatesRuntimePermissionsAndAdmission(t *testing.T) { for declaration, enabled := range map[string]bool{ "name: route53.dns.appthrust.io\n": route53, "name: cloudflare.dns.appthrust.io\n": cloudflare, + "name: external.dns.appthrust.io\n": false, "kind: EndpointProviderCapability\n": route53 && conversion, "kind: APIService\n": route53 && conversion, } { diff --git a/deploy/charts/dns-api/templates/apiservice-external-endpoint-conversion.yaml b/deploy/charts/dns-api/templates/apiservice-external-endpoint-conversion.yaml new file mode 100644 index 0000000..d9e9786 --- /dev/null +++ b/deploy/charts/dns-api/templates/apiservice-external-endpoint-conversion.yaml @@ -0,0 +1,22 @@ +{{- if .Values.providers.external.enabled }} +{{- if not (and .Values.webhook.enabled .Values.certManager.enabled) }} +{{- fail "providers.external.enabled requires webhook.enabled=true and certManager.enabled=true" }} +{{- end }} +apiVersion: apiregistration.k8s.io/v1 +kind: APIService +metadata: + name: v1alpha1.endpoint.external.dns.appthrust.io + labels: + {{- include "dns-api.labels" . | nindent 4 }} + annotations: + cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "dns-api.webhookCertificateName" . }} +spec: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + groupPriorityMinimum: 1000 + versionPriority: 15 + service: + name: {{ include "dns-api.webhookServiceName" . }} + namespace: {{ .Release.Namespace }} + port: {{ .Values.webhook.service.port }} +{{- end }} diff --git a/deploy/charts/dns-api/templates/deployment.yaml b/deploy/charts/dns-api/templates/deployment.yaml index a9ee01d..3eec920 100644 --- a/deploy/charts/dns-api/templates/deployment.yaml +++ b/deploy/charts/dns-api/templates/deployment.yaml @@ -44,6 +44,14 @@ spec: - --route53-controller-enabled={{ .Values.providers.route53.enabled }} - --cloudflare-controller-enabled={{ .Values.providers.cloudflare.enabled }} - --route53-endpoint-conversion-enabled={{ and .Values.providers.route53.enabled .Values.providers.route53.endpoint.conversionAPI.enabled .Values.webhook.enabled }} + - --external-controller-enabled={{ .Values.providers.external.enabled }} + {{- if .Values.providers.external.enabled }} + - {{ printf "--external-controller-name=%s" .Values.providers.external.controllerName | quote }} + - {{ printf "--external-provider-name=%s" .Values.providers.external.providerName | quote }} + - {{ printf "--external-provider-version=%s" .Values.providers.external.providerVersion | quote }} + - --external-dns-qps={{ .Values.providers.external.dnsQPS }} + - --external-dns-burst={{ .Values.providers.external.dnsBurst }} + {{- end }} {{- range .Values.extraArgs }} - {{ . | quote }} {{- end }} diff --git a/deploy/charts/dns-api/templates/endpointprovidercapability-external.yaml b/deploy/charts/dns-api/templates/endpointprovidercapability-external.yaml new file mode 100644 index 0000000..1990ffb --- /dev/null +++ b/deploy/charts/dns-api/templates/endpointprovidercapability-external.yaml @@ -0,0 +1,14 @@ +{{- if .Values.providers.external.enabled }} +apiVersion: endpoint.dns.appthrust.io/v1alpha1 +kind: EndpointProviderCapability +metadata: + name: {{ printf "external-%s" .Values.providers.external.providerVersion | quote }} +spec: + provider: + name: {{ .Values.providers.external.providerName | quote }} + version: {{ .Values.providers.external.providerVersion | quote }} + conversion: + group: endpoint.external.dns.appthrust.io + version: v1alpha1 + resource: endpointrecordsetconversions +{{- end }} diff --git a/deploy/charts/dns-api/templates/provider-external.yaml b/deploy/charts/dns-api/templates/provider-external.yaml new file mode 100644 index 0000000..6e1415d --- /dev/null +++ b/deploy/charts/dns-api/templates/provider-external.yaml @@ -0,0 +1,161 @@ +{{- if .Values.providers.external.enabled }} +apiVersion: dns.appthrust.io/v1alpha1 +kind: Provider +metadata: + name: {{ .Values.providers.external.providerName | quote }} +spec: + display: + name: External DNS + description: Verifies customer-managed DNS without changing DNS records. + versions: + - name: {{ .Values.providers.external.providerVersion | quote }} + served: true + storage: true + zoneClass: + schemas: + parameters: + openAPIV3Schema: + type: object + description: DNS resolver policy for verification without DNS writes. + additionalProperties: false + properties: + resolver: + type: object + description: Resolver view used for every verification query. + additionalProperties: false + properties: + mode: + type: string + description: Cluster uses pod DNS; Recursive uses every configured server. + enum: [Cluster, Recursive] + servers: + type: array + description: Explicit recursive DNS IP endpoints; never authoritative NS names. + maxItems: 3 + uniqueItems: true + items: + type: string + minLength: 3 + maxLength: 64 + pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' + expectedNameservers: + type: array + description: Expected authoritative nameservers for the exact Zone domain. + maxItems: 16 + uniqueItems: true + items: + type: string + minLength: 1 + maxLength: 253 + pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' + validationRules: + - rule: >- + has(self.spec.parameters.resolver) && + has(self.spec.parameters.resolver.mode) && + self.spec.parameters.resolver.mode == 'Recursive' + ? (has(self.spec.parameters.resolver.servers) && size(self.spec.parameters.resolver.servers) > 0) + : (!has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || size(self.spec.parameters.resolver.servers) == 0) + message: resolver.servers must be nonempty for Recursive and empty for Cluster + - rule: >- + !has(self.spec.parameters.resolver) || !has(self.spec.parameters.resolver.servers) || + self.spec.parameters.resolver.servers.all(s, + cel.bind(host, s.startsWith('[') ? s.split(']')[0].substring(1) : s.split(':')[0], + cel.bind(port, s.startsWith('[') ? s.split(']:')[1] : s.split(':')[1], + isIP(host) && ip.isCanonical(host) && + (s.startsWith('[') ? ip(host).family() == 6 : ip(host).family() == 4) && + int(port) >= 1 && int(port) <= 65535))) + message: resolver.servers must contain canonical IP addresses and ports from 1 through 65535 + - rule: >- + !has(self.spec.parameters.expectedNameservers) || + self.spec.parameters.expectedNameservers.all(n, !isIP(n)) + message: expectedNameservers must contain DNS names, not IP addresses + zone: + schemas: + adoption: &empty + openAPIV3Schema: + type: object + additionalProperties: false + properties: {} + statusProviderData: &status + openAPIV3Schema: + type: object + description: Bounded read-only DNS verification evidence; not DNS withdrawal proof. + additionalProperties: false + properties: + resolverMode: + type: string + description: Selected recursive resolver view policy. + enum: [Cluster, Recursive] + checkedName: + type: string + description: Absolute DNS owner checked, without the root dot. + maxLength: 253 + sampledWildcard: + type: boolean + description: Whether checkedName is a stable concrete wildcard witness. + actual: &preview + type: object + description: Sorted bounded diagnostic preview; equality uses the complete observation. + additionalProperties: false + required: [total, truncated] + properties: + values: + type: array + description: Up to sixteen typed values or diagnostic entries. + maxItems: 16 + items: + type: string + maxLength: 512 + total: + type: integer + description: Total entries before truncation. + minimum: 0 + truncated: + type: boolean + description: Whether values omits entries from the complete observation. + missing: *preview + extra: *preview + cnameChain: *preview + views: *preview + observedTTLSeconds: + type: integer + description: Minimum relevant positive TTL in seconds, when observed. + minimum: 0 + negativeTTLSeconds: + type: integer + description: Minimum of negative SOA TTL and SOA MINIMUM, when observed. + minimum: 0 + negativeExpiryTime: + type: string + format: date-time + description: Negative cache expiry suggested by the observed SOA, not the polling cap. + lastProbeTime: + type: string + format: date-time + description: Last completed attempt, including failures. + nextProbeTime: + type: string + format: date-time + description: Approximate next planned attempt; queue load can delay it. + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: VerificationConfigured, description: Verifier configured without authority checks.} + - {conditionType: Programmed, status: 'True', reason: NameserversVerified, description: Every required view verified the exact apex and nameservers.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Observed apex or nameservers differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordSet: + supportedTypes: [A, AAAA, CNAME] + schemas: + options: *empty + adoption: *empty + statusProviderData: *status + additionalConditionReasons: + - {conditionType: Programmed, status: 'True', reason: ResolutionVerified, description: Exact targets verified in every required view.} + - {conditionType: Programmed, status: 'False', reason: DNSNameNotFound, description: Required view returned NXDOMAIN.} + - {conditionType: Programmed, status: 'False', reason: DNSNoData, description: Required record type has no answers.} + - {conditionType: Programmed, status: 'False', reason: ResolutionPartial, description: A nonempty proper subset of targets was observed.} + - {conditionType: Programmed, status: 'False', reason: ResolutionMismatch, description: Targets or CNAME chain differ from expectations.} + - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} + - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} + - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} +{{- end }} diff --git a/deploy/charts/dns-api/templates/rbac.yaml b/deploy/charts/dns-api/templates/rbac.yaml index 3777fb7..0346733 100644 --- a/deploy/charts/dns-api/templates/rbac.yaml +++ b/deploy/charts/dns-api/templates/rbac.yaml @@ -172,6 +172,14 @@ rules: verbs: - create {{- end }} + {{- if .Values.providers.external.enabled }} + - apiGroups: + - endpoint.external.dns.appthrust.io + resources: + - endpointrecordsetconversions + verbs: + - create + {{- end }} {{- if .Values.gatewayEndpoint.enabled }} - apiGroups: - gateway.networking.k8s.io diff --git a/deploy/charts/dns-api/values.yaml b/deploy/charts/dns-api/values.yaml index 481736e..38a43f5 100644 --- a/deploy/charts/dns-api/values.yaml +++ b/deploy/charts/dns-api/values.yaml @@ -62,6 +62,13 @@ providers: enabled: true cloudflare: enabled: true + external: + enabled: false + controllerName: external.dns.appthrust.io/controller + providerName: external.dns.appthrust.io + providerVersion: v1alpha1 + dnsQPS: 40 + dnsBurst: 80 extraArgs: [] extraEnv: [] diff --git a/docs/design/controllers-webhooks.md b/docs/design/controllers-webhooks.md index 9d4b76b..2406b9a 100644 --- a/docs/design/controllers-webhooks.md +++ b/docs/design/controllers-webhooks.md @@ -42,7 +42,7 @@ Provider enablement gates runtime registrations and provider-only privileges, no Each provider switch gates its identity, ZoneClass, and ZoneUnit controllers, credential-client construction, and identity spec/status admission handler. The chart gates matching identity RBAC and webhook entries. Secret get/list/watch belongs only to enabled Cloudflare; mounted serving certificates do not require Secret API reads. Route 53 conversion requires both Route 53 flags. Its chart flag, capability, APIService, and conversion-create permission additionally require the webhook Service to be enabled. The external switch gates the external ZoneClass/ZoneUnit controllers, the external Provider and `EndpointProviderCapability`, its conversion handler/APIService, and conversion-create RBAC; external has no identity controller, identity webhook, or credential reads, and its conversion API requires the webhook Service with cert-manager-managed serving TLS ([External Provider](external-provider.md#packaging-and-metrics)). -Core composition, endpoint translation, namespace reads, and core schema/status admission remain enabled independently. Gateway/Service source switches retain their existing registration and permission gates. Scheme/CRD types remain available when a provider is disabled. Helm's default admission coverage matches Kustomize, including ZoneClass status and both identity spec/status endpoints. Kustomize's default installation enables both providers; custom disable overlays must remove the corresponding admission entries and privileges as well as setting manager flags. Disabling a provider does not complete existing finalizers. +Core composition, endpoint translation, namespace reads, and core schema/status admission remain enabled independently. Gateway/Service source switches retain their existing registration and permission gates. Scheme/CRD types remain available when a provider is disabled. Helm's default admission coverage matches Kustomize, including ZoneClass status and both identity spec/status endpoints. Kustomize's default installation enables Route 53 and Cloudflare, with external disabled. The opt-in `app/operator/config/external` overlay adds the external Provider, capability, cert-manager-backed APIService, manager arguments, and only its conversion-create permission; it does not add identity or Secret privileges. Custom disable overlays must remove the corresponding admission entries and privileges as well as setting manager flags. Disabling a provider does not complete existing finalizers. ## Implementation Modules From 00f98491d24f57fa5af66c805ccbb5c4fabd5aef Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 12:36:11 +0900 Subject: [PATCH 07/16] fix(external): describe every array item in the Provider schemas so core admission accepts the manifest The core webhook requires a description on every property and array item; the external Provider was rejected on install. The built-in-manifest validation test now covers external_extensions.yaml. --- app/operator/config/external/external_extensions.yaml | 3 +++ deploy/charts/dns-api/templates/provider-external.yaml | 3 +++ internal/go/core/webhook/validator_test.go | 4 ++-- internal/go/providers/external/manifest/provider.yaml | 3 +++ 4 files changed, 11 insertions(+), 2 deletions(-) diff --git a/app/operator/config/external/external_extensions.yaml b/app/operator/config/external/external_extensions.yaml index a143394..f24735f 100644 --- a/app/operator/config/external/external_extensions.yaml +++ b/app/operator/config/external/external_extensions.yaml @@ -34,6 +34,7 @@ spec: uniqueItems: true items: type: string + description: One recursive resolver endpoint as IPv4:port or [IPv6]:port. minLength: 3 maxLength: 64 pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' @@ -44,6 +45,7 @@ spec: uniqueItems: true items: type: string + description: One expected authoritative nameserver hostname, lowercase, without a trailing dot. minLength: 1 maxLength: 253 pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' @@ -104,6 +106,7 @@ spec: maxItems: 16 items: type: string + description: One bounded value or diagnostic entry. maxLength: 512 total: type: integer diff --git a/deploy/charts/dns-api/templates/provider-external.yaml b/deploy/charts/dns-api/templates/provider-external.yaml index 6e1415d..5ddfea7 100644 --- a/deploy/charts/dns-api/templates/provider-external.yaml +++ b/deploy/charts/dns-api/templates/provider-external.yaml @@ -35,6 +35,7 @@ spec: uniqueItems: true items: type: string + description: One recursive resolver endpoint as IPv4:port or [IPv6]:port. minLength: 3 maxLength: 64 pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' @@ -45,6 +46,7 @@ spec: uniqueItems: true items: type: string + description: One expected authoritative nameserver hostname, lowercase, without a trailing dot. minLength: 1 maxLength: 253 pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' @@ -105,6 +107,7 @@ spec: maxItems: 16 items: type: string + description: One bounded value or diagnostic entry. maxLength: 512 total: type: integer diff --git a/internal/go/core/webhook/validator_test.go b/internal/go/core/webhook/validator_test.go index d422d74..a1336b0 100644 --- a/internal/go/core/webhook/validator_test.go +++ b/internal/go/core/webhook/validator_test.go @@ -1832,8 +1832,8 @@ func TestProviderValidationChecksSchemaDescriptions(t *testing.T) { } func TestBuiltInProviderManifestsPassValidation(t *testing.T) { - for _, name := range []string{"route53_extensions.yaml", "cloudflare_extensions.yaml"} { - t.Run(name, func(t *testing.T) { + for _, name := range []string{"route53_extensions.yaml", "cloudflare_extensions.yaml", filepath.Join("..", "external", "external_extensions.yaml")} { + t.Run(filepath.Base(name), func(t *testing.T) { provider := builtInProviderManifest(t, name) if err := validateProvider(&provider); err != nil { t.Fatalf("validateProvider(%s) returned error: %v", name, err) diff --git a/internal/go/providers/external/manifest/provider.yaml b/internal/go/providers/external/manifest/provider.yaml index a143394..f24735f 100644 --- a/internal/go/providers/external/manifest/provider.yaml +++ b/internal/go/providers/external/manifest/provider.yaml @@ -34,6 +34,7 @@ spec: uniqueItems: true items: type: string + description: One recursive resolver endpoint as IPv4:port or [IPv6]:port. minLength: 3 maxLength: 64 pattern: '^(\[[0-9a-f:.]+\]|[0-9.]+):[1-9][0-9]{0,4}$' @@ -44,6 +45,7 @@ spec: uniqueItems: true items: type: string + description: One expected authoritative nameserver hostname, lowercase, without a trailing dot. minLength: 1 maxLength: 253 pattern: '^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$' @@ -104,6 +106,7 @@ spec: maxItems: 16 items: type: string + description: One bounded value or diagnostic entry. maxLength: 512 total: type: integer From cf7ea65e420ff9e29db061ad5371ed1d27228950 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:06:39 +0900 Subject: [PATCH 08/16] =?UTF-8?q?docs:=20DEP-0003=20RecordWithdrawal=20?= =?UTF-8?q?=E2=80=94=20verified=20DNS=20withdrawal=20before=20address=20re?= =?UTF-8?q?use=20(Draft)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- deps/README.md | 9 + deps/dep-0003-record-withdrawal.md | 257 +++++++++++++++++++++++++++++ deps/dep-template.md | 47 ++++++ 3 files changed, 313 insertions(+) create mode 100644 deps/README.md create mode 100644 deps/dep-0003-record-withdrawal.md create mode 100644 deps/dep-template.md diff --git a/deps/README.md b/deps/README.md new file mode 100644 index 0000000..44b55fe --- /dev/null +++ b/deps/README.md @@ -0,0 +1,9 @@ +# DNS Enhancement Proposals + +DEPs record API proposals, decisions, alternatives, and compatibility analysis. The [DEP process](../docs/design/common.md#dep-process) defines the layout, required sections, and statuses. [Current design](../docs/design/README.md) remains the specification; accepted DEPs are reflected there before implementation follows them. + +Use [the template](./dep-template.md) for a new proposal. Single Markdown files named `dep--.md` are the default. + +| DEP | Title | Status | +| --- | --- | --- | +| [0003](./dep-0003-record-withdrawal.md) | Verified record withdrawal before address reuse | Draft | diff --git a/deps/dep-0003-record-withdrawal.md b/deps/dep-0003-record-withdrawal.md new file mode 100644 index 0000000..25f96d9 --- /dev/null +++ b/deps/dep-0003-record-withdrawal.md @@ -0,0 +1,257 @@ +# DEP-0003: Verified record withdrawal before address reuse + +- Status: Draft +- Owner: dns-api maintainers (Core, Endpoint App, and external provider) +- Created: 2026-10-18 + +## Summary + +Introduce independent Core `RecordWithdrawal` and `RecordWithdrawalWaiver` claims, Core composition/projection through `ZoneUnit`, and external-provider exclusion verification. Endpoint sources preserve retired hostname/address pairs until the Endpoint App durably creates claims. Platform consumers use verified results before member retirement and address reuse; exceptional operator risk acceptance remains distinct from DNS verification. + +This is the dns-api part of the reviewed **Phase 4b: verified external-DNS withdrawal before VIP reuse** design, especially §§3.2–3.4, 4.1–4.3, 6–7, 8.2, and 9. It is a follow-up after PR #16, not additional stage-1 implementation scope. It is proposed behavior, not an implementation or runtime-verification report. Under the [DEP process](../docs/design/common.md#dep-process), committed `docs/design/` remains current specification until acceptance and reflection there. + +## Motivation + +The external provider observes customer-managed DNS; it neither owns nor deletes those DNS records. Ordinary generated-child deletion can finish while the customer's wildcard or exact record still resolves to a retired address. Reusing that address can route traffic for the former owner to a new allocation. + +The stage-1 baseline is [External Provider §Deletion](../docs/design/external-provider.md#deletion): matching-UID `DeletionRequested`/`DeletionCompleted` is Kubernetes bookkeeping, not proof of DNS withdrawal. Its deferred `Withdrawn` sentence explicitly leaves schema, persistence, consumers, and cache safety outside stage 1. On acceptance, this DEP supersedes that deferral, **not** ordinary RecordSet deletion semantics. + +A result tied only to an ERS or generated RecordSet disappears when that object is deleted. Provider-private scheduling state is reconstructible and pruned, not a durable public obligation inventory. Independent Core claims preserve the obligation across source deletion and keep App → Core → Provider ownership intact. + +### Source anchors + +Anchors below are from proposal source snapshot `00f98491d24f57fa5af66c805ccbb5c4fabd5aef`; line numbers describe that revision, not future edits: + +| Existing contract | Source anchor | +| --- | --- | +| Core composes and projects; provider reconciles Units | `docs/design/controllers-webhooks.md:5-28`; `docs/design/overview.md:13-18` | +| Unit ledger identities and deletion bookkeeping | `pkg/go/api/dns/v1alpha1/zoneunit_types.go:60-134,184-218`; `internal/go/core/controllers/zoneunit/zoneunit_controller.go:459-465,515-531` | +| External deletion and pruned scheduling state | `internal/go/providers/external/controllers/zoneunit/controller.go:179-187,217-223`; `model.go:16-27,102-113`; `docs/design/external-provider.md:264-268` | +| Source update/delete paths, including route cleanup | `internal/go/apps/gatewayendpoint/controllers/gateway/controller.go:219-245,345-402,406-420` | +| Endpoint finalizer installation and deletion | `internal/go/apps/endpoint/controllers/endpointrecordset/controller.go:63-84,536-559` | +| Nonempty A/AAAA validation | `pkg/go/api/dns/v1alpha1/recordset_types.go:87-97`; `internal/go/core/webhook/validator.go:450-481` | +| Exact-set evaluator and minimum-TTL summaries | `internal/go/providers/external/probe/evaluate.go:117-230`; `types.go:45-56`; `wire.go:267-270,297-305` | +| Zone child-claim deletion boundary | `internal/go/core/controllers/zoneunit/zoneunit_controller.go:217-255` | +| DEP process and reflection rules | `docs/design/common.md:7-21,41-49` | + +## Goals + +- Verify exclusion of one retired address while permitting healthy replacement addresses. +- Preserve obligations through concurrent source edits, direct deletion, and controller restarts. +- Keep authorization/composition/public status in Core and DNS verification in the provider. +- Expose identity-bound, cache-aware, reverified results and an independently authorized waiver. +- Preserve external stage-1 programming/deletion and non-external behavior. + +## Non-Goals + +- DNS mutation, customer credentials, a second address allocator, or automatic release after a timeout/attempt count. +- Empty public A/AAAA RecordSets, publication epochs, platform DNS-history inventories, or an ERS admission webhook. +- Enumeration of deleted-route history or proof that every historical customer hostname/cache is clear. +- IPv6 platform allocation: dns-api accepts IPv4 and IPv6 withdrawals, but the current platform member allocator remains IPv4-only. + +The supported proof covers the Project wildcard plus currently retained ERS hostnames, retirement entries, and their independent claims. Wildcards use stable concrete sampled witnesses, not exhaustive descendant enumeration. An explicit owner no longer represented by retained state is outside this proof. [INFERENCE] Configured resolvers/authorities are trusted and honor TTLs; arbitrary application caches, unconfigured views, and future customer edits are outside the guarantee. + +## API + +### RecordWithdrawal + +Namespaced GVK: `dns.appthrust.io/v1alpha1`, kind `RecordWithdrawal`. + +| Surface | Proposed contract | +| --- | --- | +| `spec.zoneRef` | Existing Core Zone reference shape; immutable. | +| `spec.name` | Canonical zone-relative owner: `@`, exact name, or leftmost wildcard; immutable. | +| `spec.address` | One canonical IPv4 or IPv6 address to exclude; immutable; no hostname target. | +| `metadata.name` | Deterministic creator key. Endpoint identity includes ERS UID, retirement generation, owner, and address; platform identity includes reservation UID, wildcard owner, and address. Collision handling checks complete immutable shape and source provenance. | +| `metadata.labels[endpoint.dns.appthrust.io/address]` | Required from both creators. IPv4 uses its canonical string; IPv6 uses lowercase unpadded base32 of the 16 normalized bytes. Validation checks label/spec consistency. | +| Source annotations | Creator/source namespace/name/UID/generation for diagnosis and collision checks, never authorization. Core does not interpret platform/Gateway kinds. | +| Ownership/lifecycle | No ownerReferences and no request-level finalizer. Operator-controlled DNS namespace; no age-based garbage collection. | + +Public status includes: + +- `observedGeneration`, resolved `zone` reference/UID and Unit identity; +- `policyFingerprint`, `requiredViews`, `checkedNames`, and `sampledWildcard`; +- `absentSince`, `notBefore`, `lastProbeTime`, `nextProbeTime`, and `verifiedAt`; +- `resultDigest`, covering claim UID/generation/spec, Unit/binding identity, policy, checked names/views, and observation window/result; +- bounded `views[]`/provider evidence and diagnostics, without using a truncated preview for decisions; +- `waiverRef` with exact accepted waiver name/UID/digest and reservation UID, when present; +- `Accepted`, `Withdrawn`, `Ready`, and optional `OverrideAccepted` conditions, each with observedGeneration. + +Creators supply neither resolver policy nor `spec.verifier`. Address-label selection is only discovery: consumers recheck canonical spec address, UID, expected source/wildcard identity, and result bindings. There is no reservation-UID label requirement, consumerRef propagation, Gateway metadata patch, or platform-kind lookup inside dns-api. Platform uses bounded live APIReader label queries; truncation/continuation or missing expected results holds release, never proves an empty inventory. + +### ZoneUnit and Provider contract + +`ZoneUnit.spec.withdrawals[]` is a map-list keyed by claim namespace/name. Each item contains nonempty claim namespace/name/UID, observedGeneration, relative name, address, Core authorization/Allowed state, and any Core-validated waiver reference and immutable scope. Core applies the same Zone namespace and record-pattern grants as for corresponding A/AAAA RecordSets; wildcard grants remain explicit. + +`ZoneUnit.status.withdrawals[]` mirrors claim identity and carries verifier conditions, effective policy fingerprint, checked names/views, cache window, resultDigest, and waiver binding. Core projects only exact UID/generation matches and preserves provider messages. Same-name recreation cannot inherit an old result; missing, stale, or disallowed items are not ready. + +Provider versions expose a `recordWithdrawal` capability containing `supported`, public status-provider-data schema, and `additionalConditionReasons`. External advertises support; Route53/Cloudflare do not advertise or reconcile it. Unsupported binding is a Core condition. The provider reconciles only matching ZoneUnits/ZoneClasses: it does not read App objects, withdrawal claims, or waivers, or duplicate grant authorization. Core is the sole public withdrawal/waiver status writer. + +Pending withdrawals do not make unrelated active RecordSet/Zone `Programmed` false. Zone deletion has a separate dependency gate. + +### RecordWithdrawalWaiver + +Namespaced GVK: `dns.appthrust.io/v1alpha1`, kind `RecordWithdrawalWaiver`, in its target's operator-controlled namespace. It is an immutable separate resource, not a privileged field on the withdrawal claim, and has no ownerReferences or request-level finalizer. + +Its spec contains exact withdrawal namespace/name/UID and immutable claim-spec digest; canonical address; exact receiving reservation namespace/name/UID; reason; ticket/change reference; and `acceptStaleDNSRisk: true`. + +Kubernetes **create RBAC is the authority**. The spec contains no client-supplied authorizedBy, username, group, authorization time, or claimed audit identity. Kubernetes API audit supplies authenticated identity/time/provenance; persisted UID/resourceVersion identifies the authorization. + +Core checks reference/UID/digest/address bindings and composes a validated waiver into the corresponding Unit item. The provider binds the exact waiver UID and emits `OverrideAccepted=True/OperatorAuthorizedRelease`; Core projects it. No provider claim reads or SubjectAccessReview are introduced. A waiver **never sets `Withdrawn=True`**; an old DNS answer remains visible as `Withdrawn=False/DNSStillResolves`. + +Every unresolved obligation needs its exact waiver. Another reservation incarnation or recreated/changed claim cannot consume it. Waivers bypass DNS absence/cache proof only, not active/uncaptured ERS targets, allocation ownership, workload/storage drain, or host cleanup. Deleting a waiver does not undo an already consumed release. Claims/waivers remain retained after reservation deletion. + +### Conditions and messages + +| Condition/status | Reason | Meaning | +| --- | --- | --- | +| `Accepted=True` | `Accepted` | Binding, grants, and capability accepted. | +| `Accepted=False` | `InvalidWithdrawal`, `NotAllowed`, `UnsupportedProvider` | Invalid shape/binding, denied grant, or unavailable capability; no release. | +| `Withdrawn=False`, `Ready=False` | `DNSStillResolves` | A required checked name/view/family still returns the retired address. | +| `Withdrawn=False`, `Ready=False` | `NegativeTTLNotElapsed` | Exclusion observed; cache horizon or separate confirming attempt remains. | +| `Withdrawn=Unknown`, `Ready=False` | `DNSProbeFailed` | Missing, failed, incomplete observation or required negative SOA evidence. | +| `Withdrawn=Unknown`, `Ready=False` | `PolicyChanged` | Binding/view-policy change cannot be equivalent or weaker success. | +| `Withdrawn=True`, `Ready=True` | `WithdrawalVerified` | Complete current all-view exclusion after horizon and confirming probe. | +| `OverrideAccepted=True` | `OperatorAuthorizedRelease` | Separately authorized exact waiver, not successful DNS observation. | + +Stable message templates: + +- `DNS still resolves to retired address
in ().` +- `Retired address
is absent in every required view; waiting until for cache safety and a confirming probe.` +- `Retired address
was verified absent from in every required view at .` +- `External DNS withdrawal cannot be verified for in : .` + +Use the existing 2048-byte message limit and bounded sorted diagnostics; include no packets, credentials, or unbounded address lists. Use **result**/`resultDigest`; existing ERS admission **receipt** semantics remain unchanged. The provider describes DNS, not whether ERS desires the address, and does not emit an ERS-derived `EndpointStillPublished` condition. + +## Behavior + +### Source capture and endpoint lifecycle + +The Gateway source remains provider-neutral: no Zone/provider selection, resolver inspection, or Core claim creation. It preserves removed pairs in two correlated ERS collections: + +- `spec.retiredTargets[]`: entry ID, retirement source generation, removed IP, and its old hostname snapshot. +- `spec.retiredHostnames[]`: entry ID, retirement source generation, removed hostname, and its old IP snapshot. + +Each capture key includes ERS UID/generation. Associations are snapshots, not independent sets later multiplied together: simultaneous removal of H1 and change V1→V2 retains `(H1,V1)`, never invented `(H1,V2)`. Duplicate pairs across collections share deterministic claim identity; a later re-add/remove generation has a new retirement identity. + +Both collections use `listType=map` keyed by entry ID. Limits are 64 target-retirement entries and 256 hostname-retirement entries, with snapshots capped at 256 hostnames and 64 IPs respectively. Overflow reports `WithdrawalCapturePending` and leaves old intent intact; it never truncates or drops obligations. + +1. The source reads current ERS, computes removed pairs, and atomically changes desired intent while appending snapshots, preserving existing retirement entries with resourceVersion optimistic concurrency. +2. Source writes never prune. The endpoint controller resolves the Zone/provider for each retired pair and creates external Core claims by deterministic create-or-get, checking complete immutable shape/provenance on collision. +3. Only after every associated claim is durably created/read back does endpoint prune exact captured entry IDs using an optimistic-lock patch. Current-generation status confirms captured entries. A create failure leaves retirement intent; a crash between create and prune is harmless. Source retries merge current state, never restore an old whole spec over concurrent entries. +4. Non-external pairs finish existing child cleanup and prune without an external DNS wait. Generated RecordSets/ERSs can finish ordinary deletion once independent claims exist; they do not wait for customer DNS changes. + +Missing/deleting Gateway, rejected admission, and every cleanup path, **including `cleanupForRoute`**, move remaining pairs to retired state rather than immediately delete the ERS. Source deletion waits for current-generation capture confirmation. + +The source installs endpoint-owned `generatedRecordSetsFinalizer` **at ERS create**, not first endpoint reconcile. Direct ERS DELETE runs `reconcileDeletion`, captures every live and retired pair, then removes that finalizer after normal child cleanup. Other enabled sources follow the same create/retire contract or cannot publish into the protected external member-address path. Controller outage cannot lose retained intent. + +A retire-only ERS may have no active hostname/target but nonempty retirement entries; CEL permits that transitional state. Endpoint reports `Ready=False/Retiring` and creates no empty RecordSet or conversion request. Active ERSs retain nonempty-intent validation. No ERS validating webhook, protection-label authorization, or new failurePolicy dependency is added; trusted source RBAC and monotone writes enforce capture. Manual stripping of finalizers, deletion of retained evidence, or unauthorized overwriting of protected state is outside the supported contract. + +### Exclusion verification and cache safety + +`EvaluateWithdrawal` is separate from existing exact-set `Evaluate`: + +```text +excluded(address) := address ∉ normalize(terminal A ∪ terminal AAAA) +``` + +For every required checked name and view, probe **both families**, even for IPv4 retirement. One old answer blocks; missing/failed/incomplete required evidence is Unknown. Replacement addresses may remain and need not be identical across views. No partial-observation union, subset, or resolver failover constitutes success. + +- Complete NOERROR with other addresses is exclusion evidence. +- Complete NODATA and NXDOMAIN are exclusion evidence only with required valid negative SOA evidence. Active `Evaluate` still returns `DNSNameNotFound` for NXDOMAIN. +- Follow existing CNAME eight-link/loop and foreign-chain boundaries and inspect both terminal families. Loops, limits, mixed invalid data, or unresolved chains are Unknown. +- Required views include every configured Recursive server or effective Cluster resolver and every required discovered authority endpoint when expectedNameservers is configured. Existing NS/SOA/AA/delegation checks remain. + +Retain complete per-view/per-qtype expiry evidence; existing minimum-TTL summaries alone are insufficient: + +1. First complete all-view exclusion starts `absentSince=t0`. +2. For each negative response, negative TTL is `min(SOA TTL, SOA.MINIMUM)`; missing required SOA is not invented zero. +3. `notBefore` is the maximum of t0, every `t0 + negativeTTL`, and retained known positive-answer/CNAME expiry horizons. Platform drain delay remains additional. +4. Identical negative answers do not slide t0 indefinitely. A changed bound may extend but never shorten the window. Reappearance, failed/incomplete observation, policy change, or stale scheduling invalidates/restarts success; old positive-expiry evidence remains conservative. +5. A **new complete successful probe at/after notBefore** is mandatory, even with zero negative TTL. Timer firing or restart without fresh evidence cannot set `Withdrawn=True`. +6. Restart preserves a window only with matching claim/binding identity, policy/view set, and fresh schedule; otherwise restart conservatively. A 3600-second TTL is not shortened to the polling cap. A replacement with no negative responses invents no negative TTL but still honors old-positive horizons and a separate confirming probe. + +Retained claims are reverified. Later failure drops readiness. Consumers reject future/missing timestamps and stale success at `now > nextProbeTime + 300s`, and require matching UID/generation/binding/policy/digest and `verifiedAt >= notBefore`. + +A live Class/policy change invalidates success/window. The prior required view set remains a floor: a weaker policy reports `PolicyChanged` until restored or explicitly waived. Missing historical policy evidence is Unknown, not proof of equivalence. Creators do not supply copied policy snapshots. + +For wildcard claims the verifier derives a stable concrete witness from claim UID and bound Zone identity, reports `sampledWildcard=true`, and reuses it across probes/restarts. Recreated claims get distinct witnesses. A literal `*` query alone is insufficient; retained exact hostnames get their own claims. Closer enclosers/explicit owners can override a witness, so the result remains sampled. + +Reuse [Scheduling and Resource Bounds](../docs/design/external-provider.md#scheduling-and-resource-bounds): shared limiter, bounded concurrency/deadlines, fair work, jitter, timers, stale-input checks, and QUERY-only transport. There is no maximum hold duration or release-after-N-attempts. Customer DNS continuing to answer the old IP holds indefinitely with `DNSStillResolves` and visible probe times. Finite sampling does not promise global cache eradication. + +### Core retention and Zone deletion + +Core retains outstanding withdrawals as dependent Zone claims. Zone/ZoneUnit deletion stays blocked while retained claims reference it, with a visible blocking dependency; the external provider continues verification instead of treating Unit deletion as successful withdrawal. Completed claims remain retained/reverified and can also pin deletion until an operator safely archives/removes them after no allocated address/reservation can consume them. There is no age-based GC. Missing claims, CRDs, bindings, or force-deleted Units never imply success. + +### Admission and RBAC + +Admission validates local shape, immutable spec/references, canonical name/address, address-label consistency, bounded schemas, condition/status shape, and existing namespace-policy declarations. Cross-object binding/grants/provider capability are Core reconciliation conditions. Admission performs no DNS/network query or field-level SubjectAccessReview. Ordinary nonempty A/AAAA and CNAME/ownership rules remain unchanged. + +| Actor | Permission boundary | +| --- | --- | +| Platform CapacityReservation | Create/get/list/watch withdrawals in configured DNS namespace; read/list/watch waiver/results and configured Zone binding; existing ERS reads. No public claim status writes, waiver create, or general RecordSet mutation. | +| Platform CRS | Read/list/watch claims/waivers and reservation status; no claim create or public status writes. | +| Gateway source | Existing ERS create/update/patch/delete; install finalizer and preserve retired entries. No withdrawal, ZoneClass, or resolver permissions. | +| Endpoint controller | Create/get/list/watch withdrawals, existing Zone resolution and ERS spec/status/finalizer permissions. No waiver create or withdrawal status writes. | +| Core | Read/watch both claims, compose Unit spec, write public claim statuses, retain Zone dependencies. | +| External provider | Matching Unit/Class reconciliation and Unit status only; no App/withdrawal/waiver reads or public claim status writes. | +| Operator release role | Namespace-scoped waiver create/get/list/watch; API audit retention records privileged authorization. | +| Tenant/Project administrator | No mutation of retained operator-namespace claims, statuses, or safety metadata. | +| Operator archive role | Explicit safe deletion of completed, nonconsumable evidence; never an automatic release mechanism. | + +Chart/Kustomize permissions and registrations agree. New Core CRDs, Unit/ERS schemas, generated bindings, admission/controller registrations, and provider capability ship together in both `app/operator/config/crd/bases/` and `deploy/charts/dns-api/crds/`. No customer Secret/cloud identity privilege is added. + +## Compatibility + +Stage-1 external RecordSet `DeletionRequested`/`DeletionCompleted` and ordinary finalizer bookkeeping stay unchanged and separate. The new withdrawal items survive source deletion and continue probing; their Zone deletion dependency is intentionally new. Existing active exact-set evaluation and admission receipt terminology remain unchanged. `none`/Route53 do not gain external DNS waiting; writing providers do not advertise the capability. + +Two claim creators are intentional: platform CapacityReservation creates the mandatory Project wildcard per retiring external allocation, even with zero routes/ERSs; endpoint creates retained ERS-pair claims. Both use the same Core contract, neither writes status. Platform gates both CRS retirement and terminal-reservation address clearing on current results; a waiver has a separate release reason. External has no automatic `NeverPublished` release because customer wildcard DNS can exist before any route. + +The dns-api Core follow-up release, reviewed platform import, and CRD delivery precede platform Phase 4b support. Missing CRD/watch/verifier, denied/truncated reads, missing/stale results, or a mode switch cannot erase held external obligations. Pre-4b retirements that already lost history are not reconstructed by epochs or synthetic success. + +## Documentation + +On acceptance, reflect this contract in present tense into: + +- `docs/design/external-provider.md` §Deletion, conditions/public data, scheduling, and admission/security, replacing the deferred `Withdrawn` sentence while preserving stage-1 deletion bookkeeping; +- `docs/design/core-api.md`: both claims, Unit lists, immutable fields, status, and Zone deletion dependency; +- `docs/design/controllers-webhooks.md`: Core admission/authorization/composition/projection and ownership; +- `docs/design/provider-extension.md`: capability, schema, and condition reasons; +- `docs/design/endpoint-api.md`: correlated bounded retirement state, retire-only validation, capture acknowledgment, and finalizer lifecycle; +- `docs/design/gateway-api-integration.md`: monotone writes, finalizer at create, and all cleanup paths. + +The document writer subsequently updates user-facing manuals from those committed design snapshots: proof limitations, indefinite holds, safe archival, exact waiver procedure, and audit retention. Draft publication does not modify current design, root user README, or manuals. + +## Alternatives + +- **Provider-private retained ZoneUnit ledger:** feasible with public result projection and Zone retention, but changes reconstructible scheduling state into primary durable claim inventory. It still needs monotone source capture and exclusion verification. Independent Core claims make identity, authorization, and lifecycle explicit without hiding safety inventory in provider JSON. +- **Platform-owned history and empty RecordSet:** rejected. A/AAAA and ERS/conversion fragments are nonempty today; relaxing admission changes the ordinary model. Exact empty-set equality rejects a healthy replacement IP, whereas exclusion permits it; active NXDOMAIN evaluation is also not withdrawal. It duplicates DNS binding/grant/lifecycle logic in platform and asynchronous lastPublished watches can miss overwritten pairs. +- **ERS admission webhook:** rejected. It introduces a new all-write availability/failurePolicy and authorization dependency. Trusted monotone source writers, create-time endpoint finalization, and create-before-prune retain intent without a second admission convention. +- **Publication epochs/history-recovery classes:** rejected. They do not repair already-lost history and add another protocol beyond allocation provenance and UID/generation-fenced claims. Mandatory zero-route wildcard claims and currently retained correlated ERS pairs define the supported proof; no claimed historical completeness is manufactured. + +## Tests + +These are implementation requirements, not executed results of this documentation change. + +### Unit and controller coverage + +- Separate exact-set/exclusion behavior; replacements allowed; NXDOMAIN differs for publication; NODATA/missing SOA, zero/nonzero TTL, both families, and IPv4-mapped normalization. +- Every recursive/authority view, opposite-family old answer, complete CNAME terminal lookup, loop/eight-link limits, partial fan-out, and failed required view. +- Maximum negative/known-positive horizons, 3600-second TTL not capped, repeated negatives not sliding forever, mandatory separate confirming probe, restart/clock reversal/stale scheduling/late-result invalidation, policy weakening, UID recreation, and old-IP reappearance. +- Stable sampled wildcard witness; explicit retained hostname blocks after wildcard removal; zero routes still has platform wildcard obligation; proof limitation remains visible. +- Correlated simultaneous hostname/target changes, duplicate pairs, re-add/remove generations, append/prune conflict, create-before-prune crash/failure, overflow hold, and endpoint outage. +- Every source cleanup including `cleanupForRoute`, create-time finalizer, and direct DELETE capture of live and retired pairs. +- Core grant denial/wildcard grant, unsupported capability, exact claim/Unit UID-generation projection, message preservation, provider ownership boundaries, and pending withdrawal not breaking active programming. +- Zone deletion remains blocked and probes continue; ordinary RecordSet deletion bookkeeping stays unchanged. +- Canonical IPv4/IPv6 label encoding and exact-spec validation; label-only match never grants success. +- Waiver role allowed; tenant/endpoint/platform denied; exact immutable claim/reservation UID/digest/address binding; DNS False persists under accepted override; audit identity is never client-controlled. + +### Cross-controller and controlled acceptance + +Use regenerated real CRDs, admission, Provider declaration, endpoint/Core/external controllers, role-based denials, and controlled DNS rather than injecting a projected success shortcut. Pause/interleave move-to-retired, create/readback, status capture, and prune: live readers see retained intent or persisted claims, never neither. Cover missing prerequisites, bounded/truncated lists, stale True, policy changes, retained-source disappearance, and exact waiver consumption. + +The Phase 4b controlled fixture has two eligible sshd-container hosts, one workload VIP, and Projects A/B. A's wildcard and retained exact names keep answering the VIP after source retirement; projected `DNSStillResolves` blocks CRS and allocation beyond the old local drain timer. B passes host selection but fails specifically with `LoadBalancerRangeExhausted`, not machine exhaustion. A verifier restart preserves the hold. + +Customer DNS removal leaves one required view answering the VIP first, then all views supply valid exclusion/SOA evidence. Assert no release before the maximum cache horizon, followed by a new confirming all-view probe, Core `WithdrawalVerified`, the additional platform drain delay, ordinary member cleanup, and automatic terminal-reservation release. Recreate B's failed deployment through the normal path, then prove actual VIP reuse and serving without overlapping allocations. Capture wildcard-owner and independent non-witness evidence as well as the stable sampled witness and retained exact names. Compare unrelated management DNS/allocations and clean disposable resources through ordinary lifecycles without deleting unresolved evidence. + +Additional cases cover zero-route wildcard publication, finished reservation/member already gone (hosts clean but VIP held), replacement address still serving, both-family/partial-address behavior, and a separate exceptional waiver run with DNS still False. Waiver acceptance is never substituted for the normal verified-withdrawal proof. diff --git a/deps/dep-template.md b/deps/dep-template.md new file mode 100644 index 0000000..047c7bd --- /dev/null +++ b/deps/dep-template.md @@ -0,0 +1,47 @@ +# DEP-NNNN: Title + +- Status: Draft +- Owner: Proposal owner +- Created: YYYY-MM-DD + +Follow the [DEP process](../docs/design/common.md#dep-process). Replace the prompts below with the proposal; include source anchors for existing contracts. + +## Summary + +Describe the change and its decision boundary. + +## Motivation + +Explain the problem and existing behavior. + +## Goals + +List the intended outcomes. + +## Non-Goals + +State exclusions and proof limits. + +## API + +Define resources, fields, validation, conditions, and status ownership. + +## Behavior + +Describe reconciliation, failure handling, lifecycle, and permissions. + +## Compatibility + +Explain preserved behavior, migration, and delivery dependencies. + +## Documentation + +List current-design sections to update on acceptance and user documentation needed for implementation. + +## Alternatives + +Compare alternatives and explain the decision. + +## Tests + +Specify required tests and acceptance evidence; distinguish requirements from executed results. From 9fc0ba8fd3ef8f49ee09b358738aae93dd4b6a46 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:09:30 +0900 Subject: [PATCH 09/16] docs: DEP-0003 ZoneUnit withdrawal items keyed by claimNamespace/claimName --- deps/dep-0003-record-withdrawal.md | 4 ++-- devbox.lock | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/deps/dep-0003-record-withdrawal.md b/deps/dep-0003-record-withdrawal.md index 25f96d9..e96fa08 100644 --- a/deps/dep-0003-record-withdrawal.md +++ b/deps/dep-0003-record-withdrawal.md @@ -81,9 +81,9 @@ Creators supply neither resolver policy nor `spec.verifier`. Address-label selec ### ZoneUnit and Provider contract -`ZoneUnit.spec.withdrawals[]` is a map-list keyed by claim namespace/name. Each item contains nonempty claim namespace/name/UID, observedGeneration, relative name, address, Core authorization/Allowed state, and any Core-validated waiver reference and immutable scope. Core applies the same Zone namespace and record-pattern grants as for corresponding A/AAAA RecordSets; wildcard grants remain explicit. +`ZoneUnit.spec.withdrawals[]` is a map-list bounded to 256 items and keyed by the scalar top-level fields `claimNamespace` and `claimName`. Each item also contains a nested `claim{namespace,name,uid}` object; CEL keeps its namespace/name consistent with the top-level keys, and its UID is nonempty. The item carries observedGeneration, relative name, address, Core authorization/Allowed state, and any Core-validated waiver reference and immutable scope. Core applies the same Zone namespace and record-pattern grants as for corresponding A/AAAA RecordSets; wildcard grants remain explicit. -`ZoneUnit.status.withdrawals[]` mirrors claim identity and carries verifier conditions, effective policy fingerprint, checked names/views, cache window, resultDigest, and waiver binding. Core projects only exact UID/generation matches and preserves provider messages. Same-name recreation cannot inherit an old result; missing, stale, or disallowed items are not ready. +`ZoneUnit.status.withdrawals[]` uses the same scalar map-list keys and 256-item bound, mirrors the nested claim identity with CEL-enforced namespace/name consistency, and carries verifier conditions, effective policy fingerprint, checked names/views, cache window, resultDigest, and waiver binding. Core projects only exact UID/generation matches using `claimNamespace`, `claimName`, and `claim.uid` for identity, and preserves provider messages. Same-name recreation cannot inherit an old result; missing, stale, or disallowed items are not ready. Provider versions expose a `recordWithdrawal` capability containing `supported`, public status-provider-data schema, and `additionalConditionReasons`. External advertises support; Route53/Cloudflare do not advertise or reconcile it. Unsupported binding is a Core condition. The provider reconciles only matching ZoneUnits/ZoneClasses: it does not read App objects, withdrawal claims, or waivers, or duplicate grant authorization. Core is the sole public withdrawal/waiver status writer. diff --git a/devbox.lock b/devbox.lock index 031c76d..99e2586 100644 --- a/devbox.lock +++ b/devbox.lock @@ -491,7 +491,7 @@ }, "nodejs@latest": { "last_modified": "2026-04-23T13:07:47Z", - "plugin_version": "0.0.3", + "plugin_version": "0.0.4", "resolved": "github:NixOS/nixpkgs/01fbdeef22b76df85ea168fbfe1bfd9e63681b30#nodejs_25", "source": "devbox-search", "version": "25.9.0", From 318195816ecb3c6f8e55e8b68bec3002d1758bc6 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:13:29 +0900 Subject: [PATCH 10/16] feat(api): define retained DNS withdrawal and retirement contracts --- pkg/go/api/dns/v1alpha1/extension_types.go | 20 +- .../dns/v1alpha1/recordwithdrawal_types.go | 271 +++++++++++ pkg/go/api/dns/v1alpha1/register.go | 4 + pkg/go/api/dns/v1alpha1/zoneunit_types.go | 16 + .../api/dns/v1alpha1/zz_generated.deepcopy.go | 441 ++++++++++++++++++ .../v1alpha1/endpointrecordset_types.go | 54 ++- .../v1alpha1/zz_generated.deepcopy.go | 54 +++ 7 files changed, 856 insertions(+), 4 deletions(-) create mode 100644 pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go diff --git a/pkg/go/api/dns/v1alpha1/extension_types.go b/pkg/go/api/dns/v1alpha1/extension_types.go index be0d085..43e609b 100644 --- a/pkg/go/api/dns/v1alpha1/extension_types.go +++ b/pkg/go/api/dns/v1alpha1/extension_types.go @@ -78,6 +78,10 @@ type ProviderVersion struct { // +optional RecordSet ProviderRecordSet `json:"recordSet,omitempty"` + // RecordWithdrawal declares support for independent exclusion verification. + // +optional + RecordWithdrawal *ProviderRecordWithdrawal `json:"recordWithdrawal,omitempty"` + // ZoneUnit defines provider-specific composition validations evaluated when core builds ZoneUnit. // +optional ZoneUnit ProviderZoneUnit `json:"zoneUnit,omitempty"` @@ -177,6 +181,20 @@ type ProviderRecordSet struct { Conversion ProviderConversion `json:"conversion,omitempty"` } +type ProviderRecordWithdrawal struct { + Supported bool `json:"supported"` + // +optional + Schemas ProviderRecordWithdrawalSchemas `json:"schemas,omitempty"` + // +optional + AdditionalConditionReasons []ProviderConditionReason `json:"additionalConditionReasons,omitempty"` +} + +type ProviderRecordWithdrawalSchemas struct { + // StatusProviderData validates public withdrawal provider evidence. + // +optional + StatusProviderData *ProviderOpenAPISchema `json:"statusProviderData,omitempty"` +} + type ProviderZoneUnit struct { // DisableValidations declares named core ZoneUnit composition validations disabled by a CEL condition. // +optional @@ -278,7 +296,7 @@ type ProviderValidationRule struct { type ProviderConditionReason struct { // ConditionType is the status condition type this reason applies to. - // +kubebuilder:validation:Enum=Accepted;Programmed + // +kubebuilder:validation:Enum=Accepted;Programmed;Withdrawn;Ready;OverrideAccepted ConditionType ConditionType `json:"conditionType"` // Status is the condition status this reason applies to. diff --git a/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go b/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go new file mode 100644 index 0000000..7677948 --- /dev/null +++ b/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go @@ -0,0 +1,271 @@ +package v1alpha1 + +import ( + "crypto/sha256" + "encoding/base32" + "encoding/hex" + "encoding/json" + "net/netip" + "strings" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" +) + +const ( + AddressLabelKey = "endpoint.dns.appthrust.io/address" + ConditionWithdrawn ConditionType = "Withdrawn" + ConditionOverrideAccepted ConditionType = "OverrideAccepted" +) + +// AddressLabelValue encodes a normalized address as a Kubernetes label value. +// Invalid addresses return an empty value; callers must validate before creation. +func AddressLabelValue(ip netip.Addr) string { + if !ip.IsValid() || ip.Zone() != "" { return "" } + ip = ip.Unmap() + if ip.Is4() { return ip.String() } + bytes := ip.As16() + return strings.ToLower(base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(bytes[:])) +} + +// WithdrawalSpecDigest binds the exact immutable wire spec. The digest is SHA-256 +// of encoding/json's struct encoding, prefixed with sha256:. +func WithdrawalSpecDigest(spec RecordWithdrawalSpec) string { + data, _ := json.Marshal(spec) + digest := sha256.Sum256(data) + return "sha256:" + hex.EncodeToString(digest[:]) +} + +// RecordWithdrawal is a retained request to verify exclusion of one DNS address. +// It deliberately has no owner reference or automatic cleanup finalizer. +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Address",type=string,JSONPath=`.spec.address` +// +kubebuilder:printcolumn:name="Withdrawn",type=string,JSONPath=`.status.conditions[?(@.type=="Withdrawn")].status` +type RecordWithdrawal struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Spec RecordWithdrawalSpec `json:"spec"` + // +optional + Status RecordWithdrawalStatus `json:"status,omitempty"` +} + +// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="spec is immutable" +type RecordWithdrawalSpec struct { + ZoneRef ZoneReference `json:"zoneRef"` + // Name is a canonical zone-relative owner, including @ or a leftmost wildcard. + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` + // Address is a canonical, unscoped and unmapped IP literal. + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=39 + Address string `json:"address"` +} + +// WithdrawalObjectReference binds one immutable Kubernetes object incarnation. +type WithdrawalObjectReference struct { + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=63 + Namespace string `json:"namespace"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=128 + UID types.UID `json:"uid"` +} + +type RecordWithdrawalStatus struct { + // +optional + ObservedGeneration int64 `json:"observedGeneration,omitempty"` + // Zone and Unit are the exact binding used to interpret the provider result. + // +optional + Zone *WithdrawalObjectReference `json:"zone,omitempty"` + // +optional + Unit *WithdrawalObjectReference `json:"unit,omitempty"` + // +optional + ProviderRef *ProviderReference `json:"providerRef,omitempty"` + // +optional + PolicyFingerprint string `json:"policyFingerprint,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=set + CheckedNames []string `json:"checkedNames,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=set + RequiredViews []string `json:"requiredViews,omitempty"` + // +optional + SampledWildcard bool `json:"sampledWildcard,omitempty"` + // +optional + AbsentSince *metav1.Time `json:"absentSince,omitempty"` + // +optional + NotBefore *metav1.Time `json:"notBefore,omitempty"` + // +optional + LastProbeTime *metav1.Time `json:"lastProbeTime,omitempty"` + // +optional + NextProbeTime *metav1.Time `json:"nextProbeTime,omitempty"` + // +optional + VerifiedAt *metav1.Time `json:"verifiedAt,omitempty"` + // +optional + ResultDigest string `json:"resultDigest,omitempty"` + // Provider contains bounded public evidence, never private provider state. + // +optional + Provider *ProviderStatus `json:"provider,omitempty"` + // +optional + WaiverRef *RecordWithdrawalWaiverReference `json:"waiverRef,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=4 + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +// +kubebuilder:object:root=true +type RecordWithdrawalList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []RecordWithdrawal `json:"items"` +} + +// RecordWithdrawalWaiver records separately RBAC-authorized acceptance of stale +// DNS risk. Kubernetes audit, not client-supplied identity fields, is authority. +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +type RecordWithdrawalWaiver struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Spec RecordWithdrawalWaiverSpec `json:"spec"` + // +optional + Status RecordWithdrawalWaiverStatus `json:"status,omitempty"` +} + +// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="spec is immutable" +type RecordWithdrawalWaiverSpec struct { + WithdrawalRef WithdrawalObjectReference `json:"withdrawalRef"` + // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` + ClaimSpecDigest string `json:"claimSpecDigest"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=39 + Address string `json:"address"` + ReservationRef WithdrawalObjectReference `json:"reservationRef"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=2048 + Reason string `json:"reason"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=1024 + TicketRef string `json:"ticketRef"` + // +kubebuilder:validation:XValidation:rule="self == true",message="acceptStaleDNSRisk must be true" + AcceptStaleDNSRisk bool `json:"acceptStaleDNSRisk"` +} + +func WithdrawalWaiverDigest(spec RecordWithdrawalWaiverSpec) string { + data, _ := json.Marshal(spec) + digest := sha256.Sum256(data) + return "sha256:" + hex.EncodeToString(digest[:]) +} + +type RecordWithdrawalWaiverStatus struct { + // +optional + ObservedGeneration int64 `json:"observedGeneration,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=1 + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +type RecordWithdrawalWaiverReference struct { + // +kubebuilder:validation:MinLength=1 + Name string `json:"name"` + // +kubebuilder:validation:MinLength=1 + UID types.UID `json:"uid"` + // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` + Digest string `json:"digest"` + ReservationRef WithdrawalObjectReference `json:"reservationRef"` +} + +// +kubebuilder:object:root=true +type RecordWithdrawalWaiverList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []RecordWithdrawalWaiver `json:"items"` +} + +// Scalar keys duplicate the nested identity because Kubernetes map-list keys +// cannot address nested fields. CEL prevents either representation diverging. +// +kubebuilder:validation:XValidation:rule="self.claimNamespace == self.claim.namespace && self.claimName == self.claim.name",message="scalar claim keys must match claim" +type ZoneUnitWithdrawal struct { + // +kubebuilder:validation:MinLength=1 + ClaimNamespace string `json:"claimNamespace"` + // +kubebuilder:validation:MinLength=1 + ClaimName string `json:"claimName"` + Claim WithdrawalObjectReference `json:"claim"` + // +kubebuilder:validation:Minimum=1 + ObservedGeneration int64 `json:"observedGeneration"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=39 + Address string `json:"address"` + Allowed bool `json:"allowed"` + // +optional + Waiver *ZoneUnitWithdrawalWaiver `json:"waiver,omitempty"` +} + +type ZoneUnitWithdrawalWaiver struct { + // +kubebuilder:validation:MinLength=1 + Name string `json:"name"` + // +kubebuilder:validation:MinLength=1 + UID types.UID `json:"uid"` + // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` + Digest string `json:"digest"` + // ReservationRef is immutable waiver scope, not platform lookup authority. + ReservationRef WithdrawalObjectReference `json:"reservationRef"` +} + +// +kubebuilder:validation:XValidation:rule="self.claimNamespace == self.claim.namespace && self.claimName == self.claim.name",message="scalar claim keys must match claim" +type ZoneUnitWithdrawalStatus struct { + // +kubebuilder:validation:MinLength=1 + ClaimNamespace string `json:"claimNamespace"` + // +kubebuilder:validation:MinLength=1 + ClaimName string `json:"claimName"` + Claim WithdrawalObjectReference `json:"claim"` + // +kubebuilder:validation:Minimum=1 + ObservedGeneration int64 `json:"observedGeneration"` + // +optional + PolicyFingerprint string `json:"policyFingerprint,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=set + CheckedNames []string `json:"checkedNames,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=set + RequiredViews []string `json:"requiredViews,omitempty"` + // +optional + SampledWildcard bool `json:"sampledWildcard,omitempty"` + // +optional + AbsentSince *metav1.Time `json:"absentSince,omitempty"` + // +optional + NotBefore *metav1.Time `json:"notBefore,omitempty"` + // +optional + LastProbeTime *metav1.Time `json:"lastProbeTime,omitempty"` + // +optional + NextProbeTime *metav1.Time `json:"nextProbeTime,omitempty"` + // +optional + VerifiedAt *metav1.Time `json:"verifiedAt,omitempty"` + // +optional + ResultDigest string `json:"resultDigest,omitempty"` + // +optional + Provider *ProviderStatus `json:"provider,omitempty"` + // +optional + WaiverRef *RecordWithdrawalWaiverReference `json:"waiverRef,omitempty"` + // +optional + // +kubebuilder:validation:MaxItems=4 + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} diff --git a/pkg/go/api/dns/v1alpha1/register.go b/pkg/go/api/dns/v1alpha1/register.go index 389425f..8d44a55 100644 --- a/pkg/go/api/dns/v1alpha1/register.go +++ b/pkg/go/api/dns/v1alpha1/register.go @@ -18,6 +18,10 @@ func addKnownTypes(scheme *runtime.Scheme) error { &ZoneUnitList{}, &RecordSet{}, &RecordSetList{}, + &RecordWithdrawal{}, + &RecordWithdrawalList{}, + &RecordWithdrawalWaiver{}, + &RecordWithdrawalWaiverList{}, ) metav1.AddToGroupVersion(scheme, SchemeGroupVersion) return nil diff --git a/pkg/go/api/dns/v1alpha1/zoneunit_types.go b/pkg/go/api/dns/v1alpha1/zoneunit_types.go index 1dc379a..601a4b5 100644 --- a/pkg/go/api/dns/v1alpha1/zoneunit_types.go +++ b/pkg/go/api/dns/v1alpha1/zoneunit_types.go @@ -36,6 +36,14 @@ type ZoneUnitSpec struct { // +listMapKey=recordSetNamespace // +listMapKey=recordSetName RecordSets []ZoneUnitRecordSetSpec `json:"recordSets,omitempty"` + + // Withdrawals retains independently authorized address-exclusion claims. + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=map + // +listMapKey=claimNamespace + // +listMapKey=claimName + Withdrawals []ZoneUnitWithdrawal `json:"withdrawals,omitempty"` } type ZoneUnitZoneSpec struct { @@ -163,6 +171,14 @@ type ZoneUnitStatus struct { // +listMapKey=recordSetNamespace // +listMapKey=recordSetName RecordSets []ZoneUnitRecordSetStatus `json:"recordSets,omitempty"` + + // Withdrawals contains identity-bound exclusion verification results. + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=map + // +listMapKey=claimNamespace + // +listMapKey=claimName + Withdrawals []ZoneUnitWithdrawalStatus `json:"withdrawals,omitempty"` } type ZoneUnitZoneStatus struct { diff --git a/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go b/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go index b1164db..618112a 100644 --- a/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go +++ b/pkg/go/api/dns/v1alpha1/zz_generated.deepcopy.go @@ -584,6 +584,47 @@ func (in *ProviderRecordSetSchemas) DeepCopy() *ProviderRecordSetSchemas { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ProviderRecordWithdrawal) DeepCopyInto(out *ProviderRecordWithdrawal) { + *out = *in + in.Schemas.DeepCopyInto(&out.Schemas) + if in.AdditionalConditionReasons != nil { + in, out := &in.AdditionalConditionReasons, &out.AdditionalConditionReasons + *out = make([]ProviderConditionReason, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ProviderRecordWithdrawal. +func (in *ProviderRecordWithdrawal) DeepCopy() *ProviderRecordWithdrawal { + if in == nil { + return nil + } + out := new(ProviderRecordWithdrawal) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ProviderRecordWithdrawalSchemas) DeepCopyInto(out *ProviderRecordWithdrawalSchemas) { + *out = *in + if in.StatusProviderData != nil { + in, out := &in.StatusProviderData, &out.StatusProviderData + *out = new(ProviderOpenAPISchema) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ProviderRecordWithdrawalSchemas. +func (in *ProviderRecordWithdrawalSchemas) DeepCopy() *ProviderRecordWithdrawalSchemas { + if in == nil { + return nil + } + out := new(ProviderRecordWithdrawalSchemas) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ProviderReference) DeepCopyInto(out *ProviderReference) { *out = *in @@ -665,6 +706,11 @@ func (in *ProviderVersion) DeepCopyInto(out *ProviderVersion) { in.ZoneClass.DeepCopyInto(&out.ZoneClass) in.Zone.DeepCopyInto(&out.Zone) in.RecordSet.DeepCopyInto(&out.RecordSet) + if in.RecordWithdrawal != nil { + in, out := &in.RecordWithdrawal, &out.RecordWithdrawal + *out = new(ProviderRecordWithdrawal) + (*in).DeepCopyInto(*out) + } in.ZoneUnit.DeepCopyInto(&out.ZoneUnit) } @@ -993,6 +1039,272 @@ func (in *RecordSetZoneStatus) DeepCopy() *RecordSetZoneStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawal) DeepCopyInto(out *RecordWithdrawal) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawal. +func (in *RecordWithdrawal) DeepCopy() *RecordWithdrawal { + if in == nil { + return nil + } + out := new(RecordWithdrawal) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *RecordWithdrawal) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalList) DeepCopyInto(out *RecordWithdrawalList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]RecordWithdrawal, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalList. +func (in *RecordWithdrawalList) DeepCopy() *RecordWithdrawalList { + if in == nil { + return nil + } + out := new(RecordWithdrawalList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *RecordWithdrawalList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalSpec) DeepCopyInto(out *RecordWithdrawalSpec) { + *out = *in + in.ZoneRef.DeepCopyInto(&out.ZoneRef) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalSpec. +func (in *RecordWithdrawalSpec) DeepCopy() *RecordWithdrawalSpec { + if in == nil { + return nil + } + out := new(RecordWithdrawalSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalStatus) DeepCopyInto(out *RecordWithdrawalStatus) { + *out = *in + if in.Zone != nil { + in, out := &in.Zone, &out.Zone + *out = new(WithdrawalObjectReference) + **out = **in + } + if in.Unit != nil { + in, out := &in.Unit, &out.Unit + *out = new(WithdrawalObjectReference) + **out = **in + } + if in.ProviderRef != nil { + in, out := &in.ProviderRef, &out.ProviderRef + *out = new(ProviderReference) + **out = **in + } + if in.CheckedNames != nil { + in, out := &in.CheckedNames, &out.CheckedNames + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.RequiredViews != nil { + in, out := &in.RequiredViews, &out.RequiredViews + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.AbsentSince != nil { + in, out := &in.AbsentSince, &out.AbsentSince + *out = (*in).DeepCopy() + } + if in.NotBefore != nil { + in, out := &in.NotBefore, &out.NotBefore + *out = (*in).DeepCopy() + } + if in.LastProbeTime != nil { + in, out := &in.LastProbeTime, &out.LastProbeTime + *out = (*in).DeepCopy() + } + if in.NextProbeTime != nil { + in, out := &in.NextProbeTime, &out.NextProbeTime + *out = (*in).DeepCopy() + } + if in.VerifiedAt != nil { + in, out := &in.VerifiedAt, &out.VerifiedAt + *out = (*in).DeepCopy() + } + if in.Provider != nil { + in, out := &in.Provider, &out.Provider + *out = new(ProviderStatus) + (*in).DeepCopyInto(*out) + } + if in.WaiverRef != nil { + in, out := &in.WaiverRef, &out.WaiverRef + *out = new(RecordWithdrawalWaiverReference) + **out = **in + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalStatus. +func (in *RecordWithdrawalStatus) DeepCopy() *RecordWithdrawalStatus { + if in == nil { + return nil + } + out := new(RecordWithdrawalStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalWaiver) DeepCopyInto(out *RecordWithdrawalWaiver) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalWaiver. +func (in *RecordWithdrawalWaiver) DeepCopy() *RecordWithdrawalWaiver { + if in == nil { + return nil + } + out := new(RecordWithdrawalWaiver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *RecordWithdrawalWaiver) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalWaiverList) DeepCopyInto(out *RecordWithdrawalWaiverList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]RecordWithdrawalWaiver, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalWaiverList. +func (in *RecordWithdrawalWaiverList) DeepCopy() *RecordWithdrawalWaiverList { + if in == nil { + return nil + } + out := new(RecordWithdrawalWaiverList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *RecordWithdrawalWaiverList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalWaiverReference) DeepCopyInto(out *RecordWithdrawalWaiverReference) { + *out = *in + out.ReservationRef = in.ReservationRef +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalWaiverReference. +func (in *RecordWithdrawalWaiverReference) DeepCopy() *RecordWithdrawalWaiverReference { + if in == nil { + return nil + } + out := new(RecordWithdrawalWaiverReference) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalWaiverSpec) DeepCopyInto(out *RecordWithdrawalWaiverSpec) { + *out = *in + out.WithdrawalRef = in.WithdrawalRef + out.ReservationRef = in.ReservationRef +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalWaiverSpec. +func (in *RecordWithdrawalWaiverSpec) DeepCopy() *RecordWithdrawalWaiverSpec { + if in == nil { + return nil + } + out := new(RecordWithdrawalWaiverSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RecordWithdrawalWaiverStatus) DeepCopyInto(out *RecordWithdrawalWaiverStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecordWithdrawalWaiverStatus. +func (in *RecordWithdrawalWaiverStatus) DeepCopy() *RecordWithdrawalWaiverStatus { + if in == nil { + return nil + } + out := new(RecordWithdrawalWaiverStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *TXTRecordSet) DeepCopyInto(out *TXTRecordSet) { *out = *in @@ -1028,6 +1340,21 @@ func (in *TypedLocalObjectReference) DeepCopy() *TypedLocalObjectReference { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WithdrawalObjectReference) DeepCopyInto(out *WithdrawalObjectReference) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WithdrawalObjectReference. +func (in *WithdrawalObjectReference) DeepCopy() *WithdrawalObjectReference { + if in == nil { + return nil + } + out := new(WithdrawalObjectReference) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *Zone) DeepCopyInto(out *Zone) { *out = *in @@ -1464,6 +1791,13 @@ func (in *ZoneUnitSpec) DeepCopyInto(out *ZoneUnitSpec) { (*in)[i].DeepCopyInto(&(*out)[i]) } } + if in.Withdrawals != nil { + in, out := &in.Withdrawals, &out.Withdrawals + *out = make([]ZoneUnitWithdrawal, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneUnitSpec. @@ -1503,6 +1837,13 @@ func (in *ZoneUnitStatus) DeepCopyInto(out *ZoneUnitStatus) { (*in)[i].DeepCopyInto(&(*out)[i]) } } + if in.Withdrawals != nil { + in, out := &in.Withdrawals, &out.Withdrawals + *out = make([]ZoneUnitWithdrawalStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneUnitStatus. @@ -1530,6 +1871,106 @@ func (in *ZoneUnitValidationToggle) DeepCopy() *ZoneUnitValidationToggle { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ZoneUnitWithdrawal) DeepCopyInto(out *ZoneUnitWithdrawal) { + *out = *in + out.Claim = in.Claim + if in.Waiver != nil { + in, out := &in.Waiver, &out.Waiver + *out = new(ZoneUnitWithdrawalWaiver) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneUnitWithdrawal. +func (in *ZoneUnitWithdrawal) DeepCopy() *ZoneUnitWithdrawal { + if in == nil { + return nil + } + out := new(ZoneUnitWithdrawal) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ZoneUnitWithdrawalStatus) DeepCopyInto(out *ZoneUnitWithdrawalStatus) { + *out = *in + out.Claim = in.Claim + if in.CheckedNames != nil { + in, out := &in.CheckedNames, &out.CheckedNames + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.RequiredViews != nil { + in, out := &in.RequiredViews, &out.RequiredViews + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.AbsentSince != nil { + in, out := &in.AbsentSince, &out.AbsentSince + *out = (*in).DeepCopy() + } + if in.NotBefore != nil { + in, out := &in.NotBefore, &out.NotBefore + *out = (*in).DeepCopy() + } + if in.LastProbeTime != nil { + in, out := &in.LastProbeTime, &out.LastProbeTime + *out = (*in).DeepCopy() + } + if in.NextProbeTime != nil { + in, out := &in.NextProbeTime, &out.NextProbeTime + *out = (*in).DeepCopy() + } + if in.VerifiedAt != nil { + in, out := &in.VerifiedAt, &out.VerifiedAt + *out = (*in).DeepCopy() + } + if in.Provider != nil { + in, out := &in.Provider, &out.Provider + *out = new(ProviderStatus) + (*in).DeepCopyInto(*out) + } + if in.WaiverRef != nil { + in, out := &in.WaiverRef, &out.WaiverRef + *out = new(RecordWithdrawalWaiverReference) + **out = **in + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneUnitWithdrawalStatus. +func (in *ZoneUnitWithdrawalStatus) DeepCopy() *ZoneUnitWithdrawalStatus { + if in == nil { + return nil + } + out := new(ZoneUnitWithdrawalStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ZoneUnitWithdrawalWaiver) DeepCopyInto(out *ZoneUnitWithdrawalWaiver) { + *out = *in + out.ReservationRef = in.ReservationRef +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneUnitWithdrawalWaiver. +func (in *ZoneUnitWithdrawalWaiver) DeepCopy() *ZoneUnitWithdrawalWaiver { + if in == nil { + return nil + } + out := new(ZoneUnitWithdrawalWaiver) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ZoneUnitZoneSpec) DeepCopyInto(out *ZoneUnitZoneSpec) { *out = *in diff --git a/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go b/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go index 50d185b..5b74d05 100644 --- a/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go +++ b/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go @@ -13,6 +13,9 @@ type EndpointTargetType string const ( EndpointTargetTypeHostname EndpointTargetType = "Hostname" EndpointTargetTypeIPAddress EndpointTargetType = "IPAddress" + GeneratedRecordSetsFinalizer = "endpoint.dns.appthrust.io/generated-recordsets" + ReasonRetiring = "Retiring" + ReasonWithdrawalCapturePending = "WithdrawalCapturePending" ) // EndpointTarget is one provider-neutral endpoint address. @@ -38,16 +41,61 @@ type EndpointRecordSet struct { Status EndpointRecordSetStatus `json:"status,omitempty"` } +// +kubebuilder:validation:XValidation:rule="(has(self.hostnames) && size(self.hostnames) > 0 && has(self.targets) && size(self.targets) > 0) || (has(self.retiredTargets) && size(self.retiredTargets) > 0) || (has(self.retiredHostnames) && size(self.retiredHostnames) > 0)",message="nonempty hostname and target intent or retirement entries are required" type EndpointRecordSetSpec struct { // Hostnames are fully qualified DNS hostnames to publish. + // +optional + // +listType=set + Hostnames []string `json:"hostnames,omitempty"` + + // Targets are endpoint addresses that the hostnames should publish. + // +optional + // +listType=atomic + Targets []EndpointTarget `json:"targets,omitempty"` + + // RetiredTargets preserves removed addresses with their old hostname snapshot. + // +optional + // +kubebuilder:validation:MaxItems=64 + // +listType=map + // +listMapKey=id + RetiredTargets []RetiredTarget `json:"retiredTargets,omitempty"` + + // RetiredHostnames preserves removed names with their old address snapshot. + // +optional + // +kubebuilder:validation:MaxItems=256 + // +listType=map + // +listMapKey=id + RetiredHostnames []RetiredHostname `json:"retiredHostnames,omitempty"` +} + +type RetiredTarget struct { + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=128 + ID string `json:"id"` + // +kubebuilder:validation:Minimum=1 + SourceGeneration int64 `json:"sourceGeneration"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=39 + Address string `json:"address"` // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=256 // +listType=set Hostnames []string `json:"hostnames"` +} - // Targets are endpoint addresses that the hostnames should publish. +type RetiredHostname struct { + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=128 + ID string `json:"id"` + // +kubebuilder:validation:Minimum=1 + SourceGeneration int64 `json:"sourceGeneration"` + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Hostname string `json:"hostname"` // +kubebuilder:validation:MinItems=1 - // +listType=atomic - Targets []EndpointTarget `json:"targets"` + // +kubebuilder:validation:MaxItems=64 + // +listType=set + Addresses []string `json:"addresses"` } type EndpointRecordSetStatus struct { diff --git a/pkg/go/api/endpoint/v1alpha1/zz_generated.deepcopy.go b/pkg/go/api/endpoint/v1alpha1/zz_generated.deepcopy.go index 1717560..44f5ff1 100644 --- a/pkg/go/api/endpoint/v1alpha1/zz_generated.deepcopy.go +++ b/pkg/go/api/endpoint/v1alpha1/zz_generated.deepcopy.go @@ -292,6 +292,20 @@ func (in *EndpointRecordSetSpec) DeepCopyInto(out *EndpointRecordSetSpec) { *out = make([]EndpointTarget, len(*in)) copy(*out, *in) } + if in.RetiredTargets != nil { + in, out := &in.RetiredTargets, &out.RetiredTargets + *out = make([]RetiredTarget, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.RetiredHostnames != nil { + in, out := &in.RetiredHostnames, &out.RetiredHostnames + *out = make([]RetiredHostname, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EndpointRecordSetSpec. @@ -400,3 +414,43 @@ func (in *RecordSetSpecFragment) DeepCopy() *RecordSetSpecFragment { in.DeepCopyInto(out) return out } + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RetiredHostname) DeepCopyInto(out *RetiredHostname) { + *out = *in + if in.Addresses != nil { + in, out := &in.Addresses, &out.Addresses + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RetiredHostname. +func (in *RetiredHostname) DeepCopy() *RetiredHostname { + if in == nil { + return nil + } + out := new(RetiredHostname) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RetiredTarget) DeepCopyInto(out *RetiredTarget) { + *out = *in + if in.Hostnames != nil { + in, out := &in.Hostnames, &out.Hostnames + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RetiredTarget. +func (in *RetiredTarget) DeepCopy() *RetiredTarget { + if in == nil { + return nil + } + out := new(RetiredTarget) + in.DeepCopyInto(out) + return out +} From 870b2d3d2eedb9749a374244bb29e415458e8a81 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:23:46 +0900 Subject: [PATCH 11/16] feat(api): fence withdrawal results to ZoneClass incarnation and generation --- pkg/go/api/dns/v1alpha1/common_types.go | 2 + .../dns/v1alpha1/recordwithdrawal_types.go | 46 ++++++++++++------- pkg/go/api/dns/v1alpha1/zoneunit_types.go | 9 ++++ .../v1alpha1/endpointrecordset_types.go | 10 ++-- 4 files changed, 45 insertions(+), 22 deletions(-) diff --git a/pkg/go/api/dns/v1alpha1/common_types.go b/pkg/go/api/dns/v1alpha1/common_types.go index cc59cfb..9631bf0 100644 --- a/pkg/go/api/dns/v1alpha1/common_types.go +++ b/pkg/go/api/dns/v1alpha1/common_types.go @@ -18,10 +18,12 @@ type ZoneClassReference struct { type ZoneReference struct { // Namespace defaults to the RecordSet namespace when omitted. // +optional + // +kubebuilder:validation:MaxLength=63 Namespace *string `json:"namespace,omitempty"` // Name is the Zone name. // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 Name string `json:"name"` } diff --git a/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go b/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go index 7677948..59d2c33 100644 --- a/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go +++ b/pkg/go/api/dns/v1alpha1/recordwithdrawal_types.go @@ -13,17 +13,21 @@ import ( ) const ( - AddressLabelKey = "endpoint.dns.appthrust.io/address" - ConditionWithdrawn ConditionType = "Withdrawn" + AddressLabelKey = "endpoint.dns.appthrust.io/address" + ConditionWithdrawn ConditionType = "Withdrawn" ConditionOverrideAccepted ConditionType = "OverrideAccepted" ) // AddressLabelValue encodes a normalized address as a Kubernetes label value. // Invalid addresses return an empty value; callers must validate before creation. func AddressLabelValue(ip netip.Addr) string { - if !ip.IsValid() || ip.Zone() != "" { return "" } + if !ip.IsValid() || ip.Zone() != "" { + return "" + } ip = ip.Unmap() - if ip.Is4() { return ip.String() } + if ip.Is4() { + return ip.String() + } bytes := ip.As16() return strings.ToLower(base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(bytes[:])) } @@ -43,9 +47,9 @@ func WithdrawalSpecDigest(spec RecordWithdrawalSpec) string { // +kubebuilder:printcolumn:name="Address",type=string,JSONPath=`.spec.address` // +kubebuilder:printcolumn:name="Withdrawn",type=string,JSONPath=`.status.conditions[?(@.type=="Withdrawn")].status` type RecordWithdrawal struct { - metav1.TypeMeta `json:",inline"` + metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` - Spec RecordWithdrawalSpec `json:"spec"` + Spec RecordWithdrawalSpec `json:"spec"` // +optional Status RecordWithdrawalStatus `json:"status,omitempty"` } @@ -73,6 +77,7 @@ type WithdrawalObjectReference struct { Name string `json:"name"` // +kubebuilder:validation:MinLength=1 // +kubebuilder:validation:MaxLength=128 + // +kubebuilder:validation:Type=string UID types.UID `json:"uid"` } @@ -126,7 +131,7 @@ type RecordWithdrawalStatus struct { type RecordWithdrawalList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` - Items []RecordWithdrawal `json:"items"` + Items []RecordWithdrawal `json:"items"` } // RecordWithdrawalWaiver records separately RBAC-authorized acceptance of stale @@ -134,9 +139,9 @@ type RecordWithdrawalList struct { // +kubebuilder:object:root=true // +kubebuilder:subresource:status type RecordWithdrawalWaiver struct { - metav1.TypeMeta `json:",inline"` + metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` - Spec RecordWithdrawalWaiverSpec `json:"spec"` + Spec RecordWithdrawalWaiverSpec `json:"spec"` // +optional Status RecordWithdrawalWaiverStatus `json:"status,omitempty"` } @@ -145,10 +150,11 @@ type RecordWithdrawalWaiver struct { type RecordWithdrawalWaiverSpec struct { WithdrawalRef WithdrawalObjectReference `json:"withdrawalRef"` // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` + // +kubebuilder:validation:MaxLength=71 ClaimSpecDigest string `json:"claimSpecDigest"` // +kubebuilder:validation:MinLength=1 // +kubebuilder:validation:MaxLength=39 - Address string `json:"address"` + Address string `json:"address"` ReservationRef WithdrawalObjectReference `json:"reservationRef"` // +kubebuilder:validation:MinLength=1 // +kubebuilder:validation:MaxLength=2048 @@ -180,9 +186,10 @@ type RecordWithdrawalWaiverReference struct { // +kubebuilder:validation:MinLength=1 Name string `json:"name"` // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:Type=string UID types.UID `json:"uid"` // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` - Digest string `json:"digest"` + Digest string `json:"digest"` ReservationRef WithdrawalObjectReference `json:"reservationRef"` } @@ -190,7 +197,7 @@ type RecordWithdrawalWaiverReference struct { type RecordWithdrawalWaiverList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` - Items []RecordWithdrawalWaiver `json:"items"` + Items []RecordWithdrawalWaiver `json:"items"` } // Scalar keys duplicate the nested identity because Kubernetes map-list keys @@ -198,10 +205,12 @@ type RecordWithdrawalWaiverList struct { // +kubebuilder:validation:XValidation:rule="self.claimNamespace == self.claim.namespace && self.claimName == self.claim.name",message="scalar claim keys must match claim" type ZoneUnitWithdrawal struct { // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=63 ClaimNamespace string `json:"claimNamespace"` // +kubebuilder:validation:MinLength=1 - ClaimName string `json:"claimName"` - Claim WithdrawalObjectReference `json:"claim"` + // +kubebuilder:validation:MaxLength=253 + ClaimName string `json:"claimName"` + Claim WithdrawalObjectReference `json:"claim"` // +kubebuilder:validation:Minimum=1 ObservedGeneration int64 `json:"observedGeneration"` // +kubebuilder:validation:MinLength=1 @@ -210,7 +219,7 @@ type ZoneUnitWithdrawal struct { // +kubebuilder:validation:MinLength=1 // +kubebuilder:validation:MaxLength=39 Address string `json:"address"` - Allowed bool `json:"allowed"` + Allowed bool `json:"allowed"` // +optional Waiver *ZoneUnitWithdrawalWaiver `json:"waiver,omitempty"` } @@ -219,6 +228,7 @@ type ZoneUnitWithdrawalWaiver struct { // +kubebuilder:validation:MinLength=1 Name string `json:"name"` // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:Type=string UID types.UID `json:"uid"` // +kubebuilder:validation:Pattern=`^sha256:[0-9a-f]{64}$` Digest string `json:"digest"` @@ -229,10 +239,12 @@ type ZoneUnitWithdrawalWaiver struct { // +kubebuilder:validation:XValidation:rule="self.claimNamespace == self.claim.namespace && self.claimName == self.claim.name",message="scalar claim keys must match claim" type ZoneUnitWithdrawalStatus struct { // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=63 ClaimNamespace string `json:"claimNamespace"` // +kubebuilder:validation:MinLength=1 - ClaimName string `json:"claimName"` - Claim WithdrawalObjectReference `json:"claim"` + // +kubebuilder:validation:MaxLength=253 + ClaimName string `json:"claimName"` + Claim WithdrawalObjectReference `json:"claim"` // +kubebuilder:validation:Minimum=1 ObservedGeneration int64 `json:"observedGeneration"` // +optional diff --git a/pkg/go/api/dns/v1alpha1/zoneunit_types.go b/pkg/go/api/dns/v1alpha1/zoneunit_types.go index 601a4b5..21e8fda 100644 --- a/pkg/go/api/dns/v1alpha1/zoneunit_types.go +++ b/pkg/go/api/dns/v1alpha1/zoneunit_types.go @@ -59,6 +59,15 @@ type ZoneUnitZoneSpec struct { // ZoneClassRef is the resolved ZoneClass reference. ZoneClassRef ObjectReference `json:"zoneClassRef"` + // ZoneClassUID and ZoneClassObservedGeneration fence policy changes before + // provider reconciliation. Missing legacy bindings cannot prove withdrawal. + // +optional + // +kubebuilder:validation:Type=string + ZoneClassUID types.UID `json:"zoneClassUID,omitempty"` + // +optional + // +kubebuilder:validation:Minimum=1 + ZoneClassObservedGeneration int64 `json:"zoneClassObservedGeneration,omitempty"` + // Adoption is provider-specific adoption payload in storage-version form. // +kubebuilder:pruning:PreserveUnknownFields // +optional diff --git a/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go b/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go index 5b74d05..bc9de83 100644 --- a/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go +++ b/pkg/go/api/endpoint/v1alpha1/endpointrecordset_types.go @@ -11,11 +11,11 @@ import ( type EndpointTargetType string const ( - EndpointTargetTypeHostname EndpointTargetType = "Hostname" - EndpointTargetTypeIPAddress EndpointTargetType = "IPAddress" - GeneratedRecordSetsFinalizer = "endpoint.dns.appthrust.io/generated-recordsets" - ReasonRetiring = "Retiring" - ReasonWithdrawalCapturePending = "WithdrawalCapturePending" + EndpointTargetTypeHostname EndpointTargetType = "Hostname" + EndpointTargetTypeIPAddress EndpointTargetType = "IPAddress" + GeneratedRecordSetsFinalizer = "endpoint.dns.appthrust.io/generated-recordsets" + ReasonRetiring = "Retiring" + ReasonWithdrawalCapturePending = "WithdrawalCapturePending" ) // EndpointTarget is one provider-neutral endpoint address. From ad91cc7f4583717facca9ba8016dbb641427b2ba Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:27:53 +0900 Subject: [PATCH 12/16] feat(core): compose and project verified DNS withdrawal claims --- .../crd/bases/dns.appthrust.io_providers.yaml | 67 ++++ .../bases/dns.appthrust.io_recordsets.yaml | 2 + .../dns.appthrust.io_recordwithdrawals.yaml | 313 ++++++++++++++++ ....appthrust.io_recordwithdrawalwaivers.yaml | 198 ++++++++++ .../crd/bases/dns.appthrust.io_zoneunits.yaml | 340 ++++++++++++++++++ ...t.dns.appthrust.io_endpointrecordsets.yaml | 82 ++++- app/operator/config/crd/kustomization.yaml | 2 + .../config/external/external_extensions.yaml | 22 ++ .../provider/cloudflare_extensions.yaml | 2 + .../config/provider/route53_extensions.yaml | 2 + .../validating_webhook_configuration.yaml | 28 ++ .../crds/dns.appthrust.io_providers.yaml | 67 ++++ .../crds/dns.appthrust.io_recordsets.yaml | 2 + .../dns.appthrust.io_recordwithdrawals.yaml | 313 ++++++++++++++++ ....appthrust.io_recordwithdrawalwaivers.yaml | 198 ++++++++++ .../crds/dns.appthrust.io_zoneunits.yaml | 340 ++++++++++++++++++ ...t.dns.appthrust.io_endpointrecordsets.yaml | 82 ++++- .../templates/provider-cloudflare.yaml | 2 + .../dns-api/templates/provider-external.yaml | 22 ++ .../dns-api/templates/provider-route53.yaml | 2 + deploy/charts/dns-api/templates/rbac.yaml | 4 + .../validating-webhook-configuration.yaml | 16 + .../zoneunit/withdrawal_projection_test.go | 56 +++ .../zoneunit/withdrawal_waivers.go | 65 ++++ .../core/controllers/zoneunit/withdrawals.go | 256 +++++++++++++ .../controllers/zoneunit/withdrawals_test.go | 266 ++++++++++++++ .../zoneunit/zoneunit_controller.go | 20 +- .../controllers/zoneunit/zoneunit_watches.go | 12 + internal/go/core/declaration/policy.go | 6 +- internal/go/core/declaration/withdrawal.go | 84 +++++ internal/go/core/webhook/validator.go | 35 +- internal/go/core/webhook/validator_test.go | 4 + internal/go/core/webhook/withdrawals.go | 244 +++++++++++++ internal/go/core/webhook/withdrawals_test.go | 173 +++++++++ .../external/manifest/manifest_test.go | 3 + .../providers/external/manifest/provider.yaml | 22 ++ .../api/dns/v1alpha1/recordwithdrawal_test.go | 65 ++++ 37 files changed, 3390 insertions(+), 27 deletions(-) create mode 100644 app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawals.yaml create mode 100644 app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawalwaivers.yaml create mode 100644 deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawals.yaml create mode 100644 deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawalwaivers.yaml create mode 100644 internal/go/core/controllers/zoneunit/withdrawal_projection_test.go create mode 100644 internal/go/core/controllers/zoneunit/withdrawal_waivers.go create mode 100644 internal/go/core/controllers/zoneunit/withdrawals.go create mode 100644 internal/go/core/controllers/zoneunit/withdrawals_test.go create mode 100644 internal/go/core/declaration/withdrawal.go create mode 100644 internal/go/core/webhook/withdrawals.go create mode 100644 internal/go/core/webhook/withdrawals_test.go create mode 100644 pkg/go/api/dns/v1alpha1/recordwithdrawal_test.go diff --git a/app/operator/config/crd/bases/dns.appthrust.io_providers.yaml b/app/operator/config/crd/bases/dns.appthrust.io_providers.yaml index 9ea89d1..04187a4 100644 --- a/app/operator/config/crd/bases/dns.appthrust.io_providers.yaml +++ b/app/operator/config/crd/bases/dns.appthrust.io_providers.yaml @@ -120,6 +120,9 @@ spec: enum: - Accepted - Programmed + - Withdrawn + - Ready + - OverrideAccepted type: string description: description: Description explains when the provider-specific @@ -304,6 +307,67 @@ spec: type: object type: array type: object + recordWithdrawal: + description: RecordWithdrawal declares support for independent + exclusion verification. + properties: + additionalConditionReasons: + items: + properties: + conditionType: + description: ConditionType is the status condition + type this reason applies to. + enum: + - Accepted + - Programmed + - Withdrawn + - Ready + - OverrideAccepted + type: string + description: + description: Description explains when the provider-specific + reason is used. + minLength: 1 + type: string + reason: + description: Reason is the provider-specific condition + reason stored in metav1.Condition.reason. + minLength: 1 + pattern: ^[A-Z][A-Za-z0-9]*$ + type: string + status: + description: Status is the condition status this reason + applies to. + enum: + - "True" + - "False" + - Unknown + type: string + required: + - conditionType + - description + - reason + - status + type: object + type: array + schemas: + properties: + statusProviderData: + description: StatusProviderData validates public withdrawal + provider evidence. + properties: + openAPIV3Schema: + description: OpenAPIV3Schema is the schema for the + corresponding inline object. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + type: object + supported: + type: boolean + required: + - supported + type: object served: description: Served controls whether new or updated specs may reference this version. @@ -326,6 +390,9 @@ spec: enum: - Accepted - Programmed + - Withdrawn + - Ready + - OverrideAccepted type: string description: description: Description explains when the provider-specific diff --git a/app/operator/config/crd/bases/dns.appthrust.io_recordsets.yaml b/app/operator/config/crd/bases/dns.appthrust.io_recordsets.yaml index c8fd206..af5b0d7 100644 --- a/app/operator/config/crd/bases/dns.appthrust.io_recordsets.yaml +++ b/app/operator/config/crd/bases/dns.appthrust.io_recordsets.yaml @@ -223,11 +223,13 @@ spec: properties: name: description: Name is the Zone name. + maxLength: 253 minLength: 1 type: string namespace: description: Namespace defaults to the RecordSet namespace when omitted. + maxLength: 63 type: string required: - name diff --git a/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawals.yaml b/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawals.yaml new file mode 100644 index 0000000..cbf8abe --- /dev/null +++ b/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawals.yaml @@ -0,0 +1,313 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: recordwithdrawals.dns.appthrust.io +spec: + group: dns.appthrust.io + names: + kind: RecordWithdrawal + listKind: RecordWithdrawalList + plural: recordwithdrawals + singular: recordwithdrawal + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.address + name: Address + type: string + - jsonPath: .status.conditions[?(@.type=="Withdrawn")].status + name: Withdrawn + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + RecordWithdrawal is a retained request to verify exclusion of one DNS address. + It deliberately has no owner reference or automatic cleanup finalizer. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + properties: + address: + description: Address is a canonical, unscoped and unmapped IP literal. + maxLength: 39 + minLength: 1 + type: string + name: + description: Name is a canonical zone-relative owner, including @ + or a leftmost wildcard. + maxLength: 253 + minLength: 1 + type: string + zoneRef: + properties: + name: + description: Name is the Zone name. + maxLength: 253 + minLength: 1 + type: string + namespace: + description: Namespace defaults to the RecordSet namespace when + omitted. + maxLength: 63 + type: string + required: + - name + type: object + required: + - address + - name + - zoneRef + type: object + x-kubernetes-validations: + - message: spec is immutable + rule: self == oldSelf + status: + properties: + absentSince: + format: date-time + type: string + checkedNames: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + conditions: + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 4 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lastProbeTime: + format: date-time + type: string + nextProbeTime: + format: date-time + type: string + notBefore: + format: date-time + type: string + observedGeneration: + format: int64 + type: integer + policyFingerprint: + type: string + provider: + description: Provider contains bounded public evidence, never private + provider state. + properties: + data: + description: Data contains public provider-specific structured + status. + type: object + x-kubernetes-preserve-unknown-fields: true + state: + description: State contains provider-controller internal state. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + providerRef: + properties: + name: + description: Name is the cluster-scoped Provider resource name. + minLength: 1 + type: string + version: + description: Version is the Provider version name. + minLength: 1 + type: string + required: + - name + - version + type: object + requiredViews: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + resultDigest: + type: string + sampledWildcard: + type: boolean + unit: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + verifiedAt: + format: date-time + type: string + waiverRef: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + zone: + description: Zone and Unit are the exact binding used to interpret + the provider result. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawalwaivers.yaml b/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawalwaivers.yaml new file mode 100644 index 0000000..9130362 --- /dev/null +++ b/app/operator/config/crd/bases/dns.appthrust.io_recordwithdrawalwaivers.yaml @@ -0,0 +1,198 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: recordwithdrawalwaivers.dns.appthrust.io +spec: + group: dns.appthrust.io + names: + kind: RecordWithdrawalWaiver + listKind: RecordWithdrawalWaiverList + plural: recordwithdrawalwaivers + singular: recordwithdrawalwaiver + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + RecordWithdrawalWaiver records separately RBAC-authorized acceptance of stale + DNS risk. Kubernetes audit, not client-supplied identity fields, is authority. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + properties: + acceptStaleDNSRisk: + type: boolean + x-kubernetes-validations: + - message: acceptStaleDNSRisk must be true + rule: self == true + address: + maxLength: 39 + minLength: 1 + type: string + claimSpecDigest: + maxLength: 71 + pattern: ^sha256:[0-9a-f]{64}$ + type: string + reason: + maxLength: 2048 + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + ticketRef: + maxLength: 1024 + minLength: 1 + type: string + withdrawalRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + required: + - acceptStaleDNSRisk + - address + - claimSpecDigest + - reason + - reservationRef + - ticketRef + - withdrawalRef + type: object + x-kubernetes-validations: + - message: spec is immutable + rule: self == oldSelf + status: + properties: + conditions: + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 1 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + observedGeneration: + format: int64 + type: integer + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/app/operator/config/crd/bases/dns.appthrust.io_zoneunits.yaml b/app/operator/config/crd/bases/dns.appthrust.io_zoneunits.yaml index 55811c1..98b73d7 100644 --- a/app/operator/config/crd/bases/dns.appthrust.io_zoneunits.yaml +++ b/app/operator/config/crd/bases/dns.appthrust.io_zoneunits.yaml @@ -255,6 +255,126 @@ spec: - recordSetNamespace - recordSetName x-kubernetes-list-type: map + withdrawals: + description: Withdrawals retains independently authorized address-exclusion + claims. + items: + description: |- + Scalar keys duplicate the nested identity because Kubernetes map-list keys + cannot address nested fields. CEL prevents either representation diverging. + properties: + address: + maxLength: 39 + minLength: 1 + type: string + allowed: + type: boolean + claim: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + claimName: + maxLength: 253 + minLength: 1 + type: string + claimNamespace: + maxLength: 63 + minLength: 1 + type: string + name: + maxLength: 253 + minLength: 1 + type: string + observedGeneration: + format: int64 + minimum: 1 + type: integer + waiver: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: ReservationRef is immutable waiver scope, not + platform lookup authority. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + required: + - address + - allowed + - claim + - claimName + - claimNamespace + - name + - observedGeneration + type: object + x-kubernetes-validations: + - message: scalar claim keys must match claim + rule: self.claimNamespace == self.claim.namespace && self.claimName + == self.claim.name + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - claimNamespace + - claimName + x-kubernetes-list-type: map zone: description: Zone is the accepted desired state for the owning Zone claim. @@ -287,6 +407,10 @@ spec: - name - namespace type: object + zoneClassObservedGeneration: + format: int64 + minimum: 1 + type: integer zoneClassRef: description: ZoneClassRef is the resolved ZoneClass reference. properties: @@ -302,6 +426,11 @@ spec: - name - namespace type: object + zoneClassUID: + description: |- + ZoneClassUID and ZoneClassObservedGeneration fence policy changes before + provider reconciliation. Missing legacy bindings cannot prove withdrawal. + type: string required: - domainName - ref @@ -514,6 +643,217 @@ spec: - recordSetNamespace - recordSetName x-kubernetes-list-type: map + withdrawals: + description: Withdrawals contains identity-bound exclusion verification + results. + items: + properties: + absentSince: + format: date-time + type: string + checkedNames: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + claim: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + claimName: + maxLength: 253 + minLength: 1 + type: string + claimNamespace: + maxLength: 63 + minLength: 1 + type: string + conditions: + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 4 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lastProbeTime: + format: date-time + type: string + nextProbeTime: + format: date-time + type: string + notBefore: + format: date-time + type: string + observedGeneration: + format: int64 + minimum: 1 + type: integer + policyFingerprint: + type: string + provider: + properties: + data: + description: Data contains public provider-specific structured + status. + type: object + x-kubernetes-preserve-unknown-fields: true + state: + description: State contains provider-controller internal + state. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + requiredViews: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + resultDigest: + type: string + sampledWildcard: + type: boolean + verifiedAt: + format: date-time + type: string + waiverRef: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable + Kubernetes object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + required: + - claim + - claimName + - claimNamespace + - observedGeneration + type: object + x-kubernetes-validations: + - message: scalar claim keys must match claim + rule: self.claimNamespace == self.claim.namespace && self.claimName + == self.claim.name + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - claimNamespace + - claimName + x-kubernetes-list-type: map zone: description: Zone contains provider result for the owning Zone claim. properties: diff --git a/app/operator/config/crd/bases/endpoint.dns.appthrust.io_endpointrecordsets.yaml b/app/operator/config/crd/bases/endpoint.dns.appthrust.io_endpointrecordsets.yaml index 9926a7c..ea1314f 100644 --- a/app/operator/config/crd/bases/endpoint.dns.appthrust.io_endpointrecordsets.yaml +++ b/app/operator/config/crd/bases/endpoint.dns.appthrust.io_endpointrecordsets.yaml @@ -49,9 +49,78 @@ spec: description: Hostnames are fully qualified DNS hostnames to publish. items: type: string - minItems: 1 type: array x-kubernetes-list-type: set + retiredHostnames: + description: RetiredHostnames preserves removed names with their old + address snapshot. + items: + properties: + addresses: + items: + type: string + maxItems: 64 + minItems: 1 + type: array + x-kubernetes-list-type: set + hostname: + maxLength: 253 + minLength: 1 + type: string + id: + maxLength: 128 + minLength: 1 + type: string + sourceGeneration: + format: int64 + minimum: 1 + type: integer + required: + - addresses + - hostname + - id + - sourceGeneration + type: object + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - id + x-kubernetes-list-type: map + retiredTargets: + description: RetiredTargets preserves removed addresses with their + old hostname snapshot. + items: + properties: + address: + maxLength: 39 + minLength: 1 + type: string + hostnames: + items: + type: string + maxItems: 256 + minItems: 1 + type: array + x-kubernetes-list-type: set + id: + maxLength: 128 + minLength: 1 + type: string + sourceGeneration: + format: int64 + minimum: 1 + type: integer + required: + - address + - hostnames + - id + - sourceGeneration + type: object + maxItems: 64 + type: array + x-kubernetes-list-map-keys: + - id + x-kubernetes-list-type: map targets: description: Targets are endpoint addresses that the hostnames should publish. @@ -72,13 +141,16 @@ spec: - type - value type: object - minItems: 1 type: array x-kubernetes-list-type: atomic - required: - - hostnames - - targets type: object + x-kubernetes-validations: + - message: nonempty hostname and target intent or retirement entries are + required + rule: (has(self.hostnames) && size(self.hostnames) > 0 && has(self.targets) + && size(self.targets) > 0) || (has(self.retiredTargets) && size(self.retiredTargets) + > 0) || (has(self.retiredHostnames) && size(self.retiredHostnames) + > 0) status: properties: conditions: diff --git a/app/operator/config/crd/kustomization.yaml b/app/operator/config/crd/kustomization.yaml index 8b8bcde..3986dc1 100644 --- a/app/operator/config/crd/kustomization.yaml +++ b/app/operator/config/crd/kustomization.yaml @@ -3,6 +3,8 @@ resources: - bases/dns.appthrust.io_zoneclasses.yaml - bases/dns.appthrust.io_zones.yaml - bases/dns.appthrust.io_recordsets.yaml + - bases/dns.appthrust.io_recordwithdrawals.yaml + - bases/dns.appthrust.io_recordwithdrawalwaivers.yaml - bases/dns.appthrust.io_zoneunits.yaml - bases/endpoint.dns.appthrust.io_endpointprovidercapabilities.yaml - bases/endpoint.dns.appthrust.io_endpointrecordsets.yaml diff --git a/app/operator/config/external/external_extensions.yaml b/app/operator/config/external/external_extensions.yaml index f24735f..9a7425e 100644 --- a/app/operator/config/external/external_extensions.yaml +++ b/app/operator/config/external/external_extensions.yaml @@ -160,3 +160,25 @@ spec: - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordWithdrawal: + supported: true + schemas: + statusProviderData: + openAPIV3Schema: + type: object + description: Bounded withdrawal diagnostics; complete typed evidence drives verification. + additionalProperties: false + properties: + views: *preview + additionalConditionReasons: + - {conditionType: Withdrawn, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Ready, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Withdrawn, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Ready, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Withdrawn, status: 'Unknown', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Ready, status: 'False', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Withdrawn, status: 'Unknown', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Ready, status: 'False', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Withdrawn, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: Ready, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: OverrideAccepted, status: 'True', reason: OperatorAuthorizedRelease, description: An exact separately authorized waiver accepts stale DNS risk.} diff --git a/app/operator/config/provider/cloudflare_extensions.yaml b/app/operator/config/provider/cloudflare_extensions.yaml index 2420e7c..7a4cf9a 100644 --- a/app/operator/config/provider/cloudflare_extensions.yaml +++ b/app/operator/config/provider/cloudflare_extensions.yaml @@ -13,6 +13,8 @@ spec: served: true storage: true deprecated: false + recordWithdrawal: + supported: false identity: resource: group: cloudflare.dns.appthrust.io diff --git a/app/operator/config/provider/route53_extensions.yaml b/app/operator/config/provider/route53_extensions.yaml index efe3f20..0aa412a 100644 --- a/app/operator/config/provider/route53_extensions.yaml +++ b/app/operator/config/provider/route53_extensions.yaml @@ -13,6 +13,8 @@ spec: served: true storage: true deprecated: false + recordWithdrawal: + supported: false identity: resource: group: route53.dns.appthrust.io diff --git a/app/operator/config/webhook/validating_webhook_configuration.yaml b/app/operator/config/webhook/validating_webhook_configuration.yaml index 2722a9b..8d6756a 100644 --- a/app/operator/config/webhook/validating_webhook_configuration.yaml +++ b/app/operator/config/webhook/validating_webhook_configuration.yaml @@ -152,3 +152,31 @@ webhooks: resources: - zoneunits - zoneunits/status + - name: recordwithdrawals.dns.appthrust.io + admissionReviewVersions: [v1] + sideEffects: None + failurePolicy: Fail + clientConfig: + service: + namespace: dns-api-system + name: dns-api-webhook-service + path: /validate-dns-appthrust-io-v1alpha1-recordwithdrawal + rules: + - apiGroups: [dns.appthrust.io] + apiVersions: [v1alpha1] + operations: [CREATE, UPDATE] + resources: [recordwithdrawals, recordwithdrawals/status] + - name: recordwithdrawalwaivers.dns.appthrust.io + admissionReviewVersions: [v1] + sideEffects: None + failurePolicy: Fail + clientConfig: + service: + namespace: dns-api-system + name: dns-api-webhook-service + path: /validate-dns-appthrust-io-v1alpha1-recordwithdrawalwaiver + rules: + - apiGroups: [dns.appthrust.io] + apiVersions: [v1alpha1] + operations: [CREATE, UPDATE] + resources: [recordwithdrawalwaivers, recordwithdrawalwaivers/status] diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_providers.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_providers.yaml index 9ea89d1..04187a4 100644 --- a/deploy/charts/dns-api/crds/dns.appthrust.io_providers.yaml +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_providers.yaml @@ -120,6 +120,9 @@ spec: enum: - Accepted - Programmed + - Withdrawn + - Ready + - OverrideAccepted type: string description: description: Description explains when the provider-specific @@ -304,6 +307,67 @@ spec: type: object type: array type: object + recordWithdrawal: + description: RecordWithdrawal declares support for independent + exclusion verification. + properties: + additionalConditionReasons: + items: + properties: + conditionType: + description: ConditionType is the status condition + type this reason applies to. + enum: + - Accepted + - Programmed + - Withdrawn + - Ready + - OverrideAccepted + type: string + description: + description: Description explains when the provider-specific + reason is used. + minLength: 1 + type: string + reason: + description: Reason is the provider-specific condition + reason stored in metav1.Condition.reason. + minLength: 1 + pattern: ^[A-Z][A-Za-z0-9]*$ + type: string + status: + description: Status is the condition status this reason + applies to. + enum: + - "True" + - "False" + - Unknown + type: string + required: + - conditionType + - description + - reason + - status + type: object + type: array + schemas: + properties: + statusProviderData: + description: StatusProviderData validates public withdrawal + provider evidence. + properties: + openAPIV3Schema: + description: OpenAPIV3Schema is the schema for the + corresponding inline object. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + type: object + supported: + type: boolean + required: + - supported + type: object served: description: Served controls whether new or updated specs may reference this version. @@ -326,6 +390,9 @@ spec: enum: - Accepted - Programmed + - Withdrawn + - Ready + - OverrideAccepted type: string description: description: Description explains when the provider-specific diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_recordsets.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_recordsets.yaml index c8fd206..af5b0d7 100644 --- a/deploy/charts/dns-api/crds/dns.appthrust.io_recordsets.yaml +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_recordsets.yaml @@ -223,11 +223,13 @@ spec: properties: name: description: Name is the Zone name. + maxLength: 253 minLength: 1 type: string namespace: description: Namespace defaults to the RecordSet namespace when omitted. + maxLength: 63 type: string required: - name diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawals.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawals.yaml new file mode 100644 index 0000000..cbf8abe --- /dev/null +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawals.yaml @@ -0,0 +1,313 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: recordwithdrawals.dns.appthrust.io +spec: + group: dns.appthrust.io + names: + kind: RecordWithdrawal + listKind: RecordWithdrawalList + plural: recordwithdrawals + singular: recordwithdrawal + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.address + name: Address + type: string + - jsonPath: .status.conditions[?(@.type=="Withdrawn")].status + name: Withdrawn + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + RecordWithdrawal is a retained request to verify exclusion of one DNS address. + It deliberately has no owner reference or automatic cleanup finalizer. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + properties: + address: + description: Address is a canonical, unscoped and unmapped IP literal. + maxLength: 39 + minLength: 1 + type: string + name: + description: Name is a canonical zone-relative owner, including @ + or a leftmost wildcard. + maxLength: 253 + minLength: 1 + type: string + zoneRef: + properties: + name: + description: Name is the Zone name. + maxLength: 253 + minLength: 1 + type: string + namespace: + description: Namespace defaults to the RecordSet namespace when + omitted. + maxLength: 63 + type: string + required: + - name + type: object + required: + - address + - name + - zoneRef + type: object + x-kubernetes-validations: + - message: spec is immutable + rule: self == oldSelf + status: + properties: + absentSince: + format: date-time + type: string + checkedNames: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + conditions: + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 4 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lastProbeTime: + format: date-time + type: string + nextProbeTime: + format: date-time + type: string + notBefore: + format: date-time + type: string + observedGeneration: + format: int64 + type: integer + policyFingerprint: + type: string + provider: + description: Provider contains bounded public evidence, never private + provider state. + properties: + data: + description: Data contains public provider-specific structured + status. + type: object + x-kubernetes-preserve-unknown-fields: true + state: + description: State contains provider-controller internal state. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + providerRef: + properties: + name: + description: Name is the cluster-scoped Provider resource name. + minLength: 1 + type: string + version: + description: Version is the Provider version name. + minLength: 1 + type: string + required: + - name + - version + type: object + requiredViews: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + resultDigest: + type: string + sampledWildcard: + type: boolean + unit: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + verifiedAt: + format: date-time + type: string + waiverRef: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + zone: + description: Zone and Unit are the exact binding used to interpret + the provider result. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawalwaivers.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawalwaivers.yaml new file mode 100644 index 0000000..9130362 --- /dev/null +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_recordwithdrawalwaivers.yaml @@ -0,0 +1,198 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.2 + name: recordwithdrawalwaivers.dns.appthrust.io +spec: + group: dns.appthrust.io + names: + kind: RecordWithdrawalWaiver + listKind: RecordWithdrawalWaiverList + plural: recordwithdrawalwaivers + singular: recordwithdrawalwaiver + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + RecordWithdrawalWaiver records separately RBAC-authorized acceptance of stale + DNS risk. Kubernetes audit, not client-supplied identity fields, is authority. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + properties: + acceptStaleDNSRisk: + type: boolean + x-kubernetes-validations: + - message: acceptStaleDNSRisk must be true + rule: self == true + address: + maxLength: 39 + minLength: 1 + type: string + claimSpecDigest: + maxLength: 71 + pattern: ^sha256:[0-9a-f]{64}$ + type: string + reason: + maxLength: 2048 + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + ticketRef: + maxLength: 1024 + minLength: 1 + type: string + withdrawalRef: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + required: + - acceptStaleDNSRisk + - address + - claimSpecDigest + - reason + - reservationRef + - ticketRef + - withdrawalRef + type: object + x-kubernetes-validations: + - message: spec is immutable + rule: self == oldSelf + status: + properties: + conditions: + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 1 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + observedGeneration: + format: int64 + type: integer + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/deploy/charts/dns-api/crds/dns.appthrust.io_zoneunits.yaml b/deploy/charts/dns-api/crds/dns.appthrust.io_zoneunits.yaml index 55811c1..98b73d7 100644 --- a/deploy/charts/dns-api/crds/dns.appthrust.io_zoneunits.yaml +++ b/deploy/charts/dns-api/crds/dns.appthrust.io_zoneunits.yaml @@ -255,6 +255,126 @@ spec: - recordSetNamespace - recordSetName x-kubernetes-list-type: map + withdrawals: + description: Withdrawals retains independently authorized address-exclusion + claims. + items: + description: |- + Scalar keys duplicate the nested identity because Kubernetes map-list keys + cannot address nested fields. CEL prevents either representation diverging. + properties: + address: + maxLength: 39 + minLength: 1 + type: string + allowed: + type: boolean + claim: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + claimName: + maxLength: 253 + minLength: 1 + type: string + claimNamespace: + maxLength: 63 + minLength: 1 + type: string + name: + maxLength: 253 + minLength: 1 + type: string + observedGeneration: + format: int64 + minimum: 1 + type: integer + waiver: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: ReservationRef is immutable waiver scope, not + platform lookup authority. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + required: + - address + - allowed + - claim + - claimName + - claimNamespace + - name + - observedGeneration + type: object + x-kubernetes-validations: + - message: scalar claim keys must match claim + rule: self.claimNamespace == self.claim.namespace && self.claimName + == self.claim.name + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - claimNamespace + - claimName + x-kubernetes-list-type: map zone: description: Zone is the accepted desired state for the owning Zone claim. @@ -287,6 +407,10 @@ spec: - name - namespace type: object + zoneClassObservedGeneration: + format: int64 + minimum: 1 + type: integer zoneClassRef: description: ZoneClassRef is the resolved ZoneClass reference. properties: @@ -302,6 +426,11 @@ spec: - name - namespace type: object + zoneClassUID: + description: |- + ZoneClassUID and ZoneClassObservedGeneration fence policy changes before + provider reconciliation. Missing legacy bindings cannot prove withdrawal. + type: string required: - domainName - ref @@ -514,6 +643,217 @@ spec: - recordSetNamespace - recordSetName x-kubernetes-list-type: map + withdrawals: + description: Withdrawals contains identity-bound exclusion verification + results. + items: + properties: + absentSince: + format: date-time + type: string + checkedNames: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + claim: + description: WithdrawalObjectReference binds one immutable Kubernetes + object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + claimName: + maxLength: 253 + minLength: 1 + type: string + claimNamespace: + maxLength: 63 + minLength: 1 + type: string + conditions: + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 4 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lastProbeTime: + format: date-time + type: string + nextProbeTime: + format: date-time + type: string + notBefore: + format: date-time + type: string + observedGeneration: + format: int64 + minimum: 1 + type: integer + policyFingerprint: + type: string + provider: + properties: + data: + description: Data contains public provider-specific structured + status. + type: object + x-kubernetes-preserve-unknown-fields: true + state: + description: State contains provider-controller internal + state. + type: object + x-kubernetes-preserve-unknown-fields: true + type: object + requiredViews: + items: + type: string + maxItems: 256 + type: array + x-kubernetes-list-type: set + resultDigest: + type: string + sampledWildcard: + type: boolean + verifiedAt: + format: date-time + type: string + waiverRef: + properties: + digest: + pattern: ^sha256:[0-9a-f]{64}$ + type: string + name: + minLength: 1 + type: string + reservationRef: + description: WithdrawalObjectReference binds one immutable + Kubernetes object incarnation. + properties: + name: + maxLength: 253 + minLength: 1 + type: string + namespace: + maxLength: 63 + minLength: 1 + type: string + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + maxLength: 128 + minLength: 1 + type: string + required: + - name + - namespace + - uid + type: object + uid: + description: |- + UID is a type that holds unique ID values, including UUIDs. Because we + don't ONLY use UUIDs, this is an alias to string. Being a type captures + intent and helps make sure that UIDs and names do not get conflated. + minLength: 1 + type: string + required: + - digest + - name + - reservationRef + - uid + type: object + required: + - claim + - claimName + - claimNamespace + - observedGeneration + type: object + x-kubernetes-validations: + - message: scalar claim keys must match claim + rule: self.claimNamespace == self.claim.namespace && self.claimName + == self.claim.name + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - claimNamespace + - claimName + x-kubernetes-list-type: map zone: description: Zone contains provider result for the owning Zone claim. properties: diff --git a/deploy/charts/dns-api/crds/endpoint.dns.appthrust.io_endpointrecordsets.yaml b/deploy/charts/dns-api/crds/endpoint.dns.appthrust.io_endpointrecordsets.yaml index 9926a7c..ea1314f 100644 --- a/deploy/charts/dns-api/crds/endpoint.dns.appthrust.io_endpointrecordsets.yaml +++ b/deploy/charts/dns-api/crds/endpoint.dns.appthrust.io_endpointrecordsets.yaml @@ -49,9 +49,78 @@ spec: description: Hostnames are fully qualified DNS hostnames to publish. items: type: string - minItems: 1 type: array x-kubernetes-list-type: set + retiredHostnames: + description: RetiredHostnames preserves removed names with their old + address snapshot. + items: + properties: + addresses: + items: + type: string + maxItems: 64 + minItems: 1 + type: array + x-kubernetes-list-type: set + hostname: + maxLength: 253 + minLength: 1 + type: string + id: + maxLength: 128 + minLength: 1 + type: string + sourceGeneration: + format: int64 + minimum: 1 + type: integer + required: + - addresses + - hostname + - id + - sourceGeneration + type: object + maxItems: 256 + type: array + x-kubernetes-list-map-keys: + - id + x-kubernetes-list-type: map + retiredTargets: + description: RetiredTargets preserves removed addresses with their + old hostname snapshot. + items: + properties: + address: + maxLength: 39 + minLength: 1 + type: string + hostnames: + items: + type: string + maxItems: 256 + minItems: 1 + type: array + x-kubernetes-list-type: set + id: + maxLength: 128 + minLength: 1 + type: string + sourceGeneration: + format: int64 + minimum: 1 + type: integer + required: + - address + - hostnames + - id + - sourceGeneration + type: object + maxItems: 64 + type: array + x-kubernetes-list-map-keys: + - id + x-kubernetes-list-type: map targets: description: Targets are endpoint addresses that the hostnames should publish. @@ -72,13 +141,16 @@ spec: - type - value type: object - minItems: 1 type: array x-kubernetes-list-type: atomic - required: - - hostnames - - targets type: object + x-kubernetes-validations: + - message: nonempty hostname and target intent or retirement entries are + required + rule: (has(self.hostnames) && size(self.hostnames) > 0 && has(self.targets) + && size(self.targets) > 0) || (has(self.retiredTargets) && size(self.retiredTargets) + > 0) || (has(self.retiredHostnames) && size(self.retiredHostnames) + > 0) status: properties: conditions: diff --git a/deploy/charts/dns-api/templates/provider-cloudflare.yaml b/deploy/charts/dns-api/templates/provider-cloudflare.yaml index 996a5bf..d311b3b 100644 --- a/deploy/charts/dns-api/templates/provider-cloudflare.yaml +++ b/deploy/charts/dns-api/templates/provider-cloudflare.yaml @@ -14,6 +14,8 @@ spec: served: true storage: true deprecated: false + recordWithdrawal: + supported: false identity: resource: group: cloudflare.dns.appthrust.io diff --git a/deploy/charts/dns-api/templates/provider-external.yaml b/deploy/charts/dns-api/templates/provider-external.yaml index 5ddfea7..6c8f84f 100644 --- a/deploy/charts/dns-api/templates/provider-external.yaml +++ b/deploy/charts/dns-api/templates/provider-external.yaml @@ -161,4 +161,26 @@ spec: - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordWithdrawal: + supported: true + schemas: + statusProviderData: + openAPIV3Schema: + type: object + description: Bounded withdrawal diagnostics; complete typed evidence drives verification. + additionalProperties: false + properties: + views: *preview + additionalConditionReasons: + - {conditionType: Withdrawn, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Ready, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Withdrawn, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Ready, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Withdrawn, status: 'Unknown', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Ready, status: 'False', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Withdrawn, status: 'Unknown', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Ready, status: 'False', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Withdrawn, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: Ready, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: OverrideAccepted, status: 'True', reason: OperatorAuthorizedRelease, description: An exact separately authorized waiver accepts stale DNS risk.} {{- end }} diff --git a/deploy/charts/dns-api/templates/provider-route53.yaml b/deploy/charts/dns-api/templates/provider-route53.yaml index 5b4c567..5bbc032 100644 --- a/deploy/charts/dns-api/templates/provider-route53.yaml +++ b/deploy/charts/dns-api/templates/provider-route53.yaml @@ -14,6 +14,8 @@ spec: served: true storage: true deprecated: false + recordWithdrawal: + supported: false identity: resource: group: route53.dns.appthrust.io diff --git a/deploy/charts/dns-api/templates/rbac.yaml b/deploy/charts/dns-api/templates/rbac.yaml index 0346733..7d3a9e4 100644 --- a/deploy/charts/dns-api/templates/rbac.yaml +++ b/deploy/charts/dns-api/templates/rbac.yaml @@ -91,6 +91,8 @@ rules: resources: - recordsets - zoneunits + - recordwithdrawals + - recordwithdrawalwaivers verbs: - create - delete @@ -113,6 +115,8 @@ rules: - recordsets/status - zoneclasses/status - zones/status + - recordwithdrawals/status + - recordwithdrawalwaivers/status verbs: - get - patch diff --git a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml index c809831..105abbc 100644 --- a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml +++ b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml @@ -139,6 +139,22 @@ webhooks: resources: - recordsets - recordsets/status + {{- range $kind := list "recordwithdrawal" "recordwithdrawalwaiver" }} + - name: {{ $kind }}s.dns.appthrust.io + admissionReviewVersions: [v1] + sideEffects: None + failurePolicy: {{ $.Values.webhook.failurePolicy }} + clientConfig: + service: + namespace: {{ $.Release.Namespace }} + name: {{ include "dns-api.webhookServiceName" $ }} + path: /validate-dns-appthrust-io-v1alpha1-{{ $kind }} + rules: + - apiGroups: [dns.appthrust.io] + apiVersions: [v1alpha1] + operations: [CREATE, UPDATE] + resources: [{{ $kind }}s, {{ $kind }}s/status] + {{- end }} - name: zoneunits.dns.appthrust.io admissionReviewVersions: - v1 diff --git a/internal/go/core/controllers/zoneunit/withdrawal_projection_test.go b/internal/go/core/controllers/zoneunit/withdrawal_projection_test.go new file mode 100644 index 0000000..fcd8e30 --- /dev/null +++ b/internal/go/core/controllers/zoneunit/withdrawal_projection_test.go @@ -0,0 +1,56 @@ +package zoneunit + +import ( + "context" + "testing" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" +) + +func TestWithdrawalProjectionRejectsStaleBindings(t *testing.T) { + _, _, _, claim := withdrawalFixture() + item := dnsv1alpha1.ZoneUnitWithdrawal{ClaimNamespace: claim.Namespace, ClaimName: claim.Name, Claim: withdrawalObjectRef(claim), ObservedGeneration: claim.Generation, Name: claim.Spec.Name, Address: claim.Spec.Address, Allowed: true} + for _, tc := range []struct { + name string + mutate func(*dnsv1alpha1.ZoneUnit) + want bool + }{ + {"current", func(*dnsv1alpha1.ZoneUnit) {}, true}, + {"claim UID", func(u *dnsv1alpha1.ZoneUnit) { u.Status.Withdrawals[0].Claim.UID = "other" }, false}, + {"claim generation", func(u *dnsv1alpha1.ZoneUnit) { u.Status.Withdrawals[0].ObservedGeneration++ }, false}, + {"condition generation", func(u *dnsv1alpha1.ZoneUnit) { u.Status.Withdrawals[0].Conditions[0].ObservedGeneration++ }, false}, + {"unit generation", func(u *dnsv1alpha1.ZoneUnit) { u.Status.ObservedGeneration-- }, false}, + {"missing class UID", func(u *dnsv1alpha1.ZoneUnit) { u.Spec.Zone.ZoneClassUID = "" }, false}, + {"missing class generation", func(u *dnsv1alpha1.ZoneUnit) { u.Spec.Zone.ZoneClassObservedGeneration = 0 }, false}, + {"authorization revoked", func(u *dnsv1alpha1.ZoneUnit) { u.Spec.Withdrawals[0].Allowed = false }, false}, + {"missing item", func(u *dnsv1alpha1.ZoneUnit) { u.Spec.Withdrawals = nil }, false}, + } { + t.Run(tc.name, func(t *testing.T) { + unit := &dnsv1alpha1.ZoneUnit{ObjectMeta: metav1.ObjectMeta{Generation: 10}, Spec: dnsv1alpha1.ZoneUnitSpec{Zone: dnsv1alpha1.ZoneUnitZoneSpec{ZoneClassUID: "class", ZoneClassObservedGeneration: 1}, Withdrawals: []dnsv1alpha1.ZoneUnitWithdrawal{item}}, Status: dnsv1alpha1.ZoneUnitStatus{ObservedGeneration: 10, Withdrawals: []dnsv1alpha1.ZoneUnitWithdrawalStatus{{ClaimNamespace: claim.Namespace, ClaimName: claim.Name, Claim: item.Claim, ObservedGeneration: claim.Generation, ResultDigest: "current", Conditions: []metav1.Condition{{Type: "Withdrawn", Status: metav1.ConditionTrue, Reason: "WithdrawalVerified", Message: "exact provider detail", ObservedGeneration: claim.Generation}, {Type: "Ready", Status: metav1.ConditionTrue, Reason: "WithdrawalVerified", ObservedGeneration: claim.Generation}}}}}} + tc.mutate(unit) + status := dnsv1alpha1.RecordWithdrawalStatus{} + projectWithdrawalResult(&status, claim, item, unit) + if (status.ResultDigest != "") == tc.want { + return + } + t.Fatalf("projected=%#v want=%v", status, tc.want) + }) + } +} + +func TestWithdrawalWaiverMissingClaimInvalidatesAcceptance(t *testing.T) { + waiver := &dnsv1alpha1.RecordWithdrawalWaiver{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "waiver", Generation: 1}, Spec: dnsv1alpha1.RecordWithdrawalWaiverSpec{WithdrawalRef: dnsv1alpha1.WithdrawalObjectReference{Namespace: "app", Name: "missing", UID: "gone"}}, Status: dnsv1alpha1.RecordWithdrawalWaiverStatus{ObservedGeneration: 1, Conditions: []metav1.Condition{{Type: "Accepted", Status: metav1.ConditionTrue, Reason: "Accepted", ObservedGeneration: 1}}}} + c := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(waiver).WithObjects(waiver).Build() + r := &RecordWithdrawalWaiverReconciler{Client: c} + if _, err := r.Reconcile(context.Background(), ctrl.Request{NamespacedName: client.ObjectKeyFromObject(waiver)}); err != nil { + t.Fatal(err) + } + if err := c.Get(context.Background(), client.ObjectKeyFromObject(waiver), waiver); err != nil { + t.Fatal(err) + } + assertCondition(t, waiver.Status.Conditions, "Accepted", metav1.ConditionFalse, "InvalidWithdrawal") +} diff --git a/internal/go/core/controllers/zoneunit/withdrawal_waivers.go b/internal/go/core/controllers/zoneunit/withdrawal_waivers.go new file mode 100644 index 0000000..b394222 --- /dev/null +++ b/internal/go/core/controllers/zoneunit/withdrawal_waivers.go @@ -0,0 +1,65 @@ +package zoneunit + +import ( + "context" + + "github.com/appthrust/dns-api/internal/go/core/declaration" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "k8s.io/apimachinery/pkg/api/equality" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" +) + +// RecordWithdrawalWaiverReconciler owns waiver status independently of Zone +// existence, so missing or archived claims cannot leave Accepted=True behind. +// Composition still validates exact bindings directly before attaching a waiver. +type RecordWithdrawalWaiverReconciler struct{ client.Client } + +func (r *RecordWithdrawalWaiverReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + var waiver dnsv1alpha1.RecordWithdrawalWaiver + if err := r.Get(ctx, req.NamespacedName, &waiver); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + var claim dnsv1alpha1.RecordWithdrawal + err := r.Get(ctx, client.ObjectKey{Namespace: waiver.Spec.WithdrawalRef.Namespace, Name: waiver.Spec.WithdrawalRef.Name}, &claim) + if err != nil && client.IgnoreNotFound(err) != nil { + return ctrl.Result{}, err + } + return ctrl.Result{}, r.setWaiverStatus(ctx, &waiver, err == nil && declaration.WithdrawalWaiverMatches(&waiver, &claim)) +} + +func (r *RecordWithdrawalWaiverReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr).Named("recordwithdrawal-waiver").For(&dnsv1alpha1.RecordWithdrawalWaiver{}).Watches(&dnsv1alpha1.RecordWithdrawal{}, handler.EnqueueRequestsFromMapFunc(r.mapClaimToWaivers)).Complete(r) +} + +func (r *RecordWithdrawalWaiverReconciler) mapClaimToWaivers(ctx context.Context, obj client.Object) []reconcile.Request { + var waivers dnsv1alpha1.RecordWithdrawalWaiverList + if err := r.List(ctx, &waivers, client.InNamespace(obj.GetNamespace())); err != nil { + return nil + } + requests := make([]reconcile.Request, 0) + for i := range waivers.Items { + waiver := &waivers.Items[i] + if waiver.Spec.WithdrawalRef.Namespace == obj.GetNamespace() && waiver.Spec.WithdrawalRef.Name == obj.GetName() { + requests = append(requests, reconcile.Request{NamespacedName: client.ObjectKeyFromObject(waiver)}) + } + } + return requests +} +func (r *RecordWithdrawalWaiverReconciler) setWaiverStatus(ctx context.Context, waiver *dnsv1alpha1.RecordWithdrawalWaiver, valid bool) error { + base := waiver.DeepCopy() + condition := metav1.Condition{Type: string(dnsv1alpha1.ConditionAccepted), Status: metav1.ConditionFalse, Reason: "InvalidWithdrawal", Message: "Waiver does not match the exact claim UID, immutable spec digest and address", ObservedGeneration: waiver.Generation} + if valid { + condition.Status, condition.Reason, condition.Message = metav1.ConditionTrue, "Accepted", "Waiver is bound to the exact claim and receiving reservation" + } + waiver.Status.ObservedGeneration = waiver.Generation + meta.SetStatusCondition(&waiver.Status.Conditions, condition) + if equality.Semantic.DeepEqual(base.Status, waiver.Status) { + return nil + } + return client.IgnoreNotFound(r.Status().Patch(ctx, waiver, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{}))) +} diff --git a/internal/go/core/controllers/zoneunit/withdrawals.go b/internal/go/core/controllers/zoneunit/withdrawals.go new file mode 100644 index 0000000..30166ee --- /dev/null +++ b/internal/go/core/controllers/zoneunit/withdrawals.go @@ -0,0 +1,256 @@ +package zoneunit + +import ( + "context" + "slices" + "strings" + + "github.com/appthrust/dns-api/internal/go/core/declaration" + "github.com/appthrust/dns-api/internal/go/core/providercontract" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "k8s.io/apimachinery/pkg/api/equality" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/reconcile" +) + +// +kubebuilder:rbac:groups=dns.appthrust.io,resources=recordwithdrawals;recordwithdrawalwaivers,verbs=create;delete;get;list;watch;patch;update +// +kubebuilder:rbac:groups=dns.appthrust.io,resources=recordwithdrawals/status;recordwithdrawalwaivers/status,verbs=get;patch;update +func (r *ZoneUnitCompositionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + claims, err := r.withdrawalsForZone(ctx, req.NamespacedName) + if err != nil { + return ctrl.Result{}, err + } + result, err := r.reconcileZone(ctx, req, claims) + if err != nil { + return result, err + } + return result, r.reconcileWithdrawals(ctx, req.NamespacedName, claims) +} + +func withdrawalZoneKey(claim *dnsv1alpha1.RecordWithdrawal) client.ObjectKey { + namespace := claim.Namespace + if claim.Spec.ZoneRef.Namespace != nil && *claim.Spec.ZoneRef.Namespace != "" { + namespace = *claim.Spec.ZoneRef.Namespace + } + return client.ObjectKey{Namespace: namespace, Name: claim.Spec.ZoneRef.Name} +} + +func (r *ZoneUnitCompositionReconciler) withdrawalsForZone(ctx context.Context, key client.ObjectKey) ([]dnsv1alpha1.RecordWithdrawal, error) { + var list dnsv1alpha1.RecordWithdrawalList + if err := r.List(ctx, &list); err != nil { + return nil, err + } + claims := make([]dnsv1alpha1.RecordWithdrawal, 0) + for _, claim := range list.Items { + if withdrawalZoneKey(&claim) == key { + claims = append(claims, claim) + } + } + slices.SortFunc(claims, func(a, b dnsv1alpha1.RecordWithdrawal) int { + return strings.Compare(recordSetClaimKey(a.Namespace, a.Name), recordSetClaimKey(b.Namespace, b.Name)) + }) + return claims, nil +} + +func (r *ZoneUnitCompositionReconciler) mapWithdrawalToZone(_ context.Context, obj client.Object) []reconcile.Request { + claim := obj.(*dnsv1alpha1.RecordWithdrawal) + return []reconcile.Request{{NamespacedName: withdrawalZoneKey(claim)}} +} + +func (r *ZoneUnitCompositionReconciler) mapWaiverToZone(ctx context.Context, obj client.Object) []reconcile.Request { + waiver := obj.(*dnsv1alpha1.RecordWithdrawalWaiver) + var claim dnsv1alpha1.RecordWithdrawal + if err := r.Get(ctx, client.ObjectKey{Namespace: waiver.Spec.WithdrawalRef.Namespace, Name: waiver.Spec.WithdrawalRef.Name}, &claim); err != nil { + return nil + } + return []reconcile.Request{{NamespacedName: withdrawalZoneKey(&claim)}} +} + +func (r *ZoneUnitCompositionReconciler) reconcileWithdrawals(ctx context.Context, key client.ObjectKey, claims []dnsv1alpha1.RecordWithdrawal) error { + var zone dnsv1alpha1.Zone + var unit dnsv1alpha1.ZoneUnit + zoneErr := r.Get(ctx, key, &zone) + if err := client.IgnoreNotFound(zoneErr); err != nil { + return err + } + unitErr := r.Get(ctx, key, &unit) + if err := client.IgnoreNotFound(unitErr); err != nil { + return err + } + var version *dnsv1alpha1.ProviderVersion + bindingValid := false + zoneAllowed := true + if zoneErr == nil { + class, found, err := r.zoneClassForZone(ctx, &zone) + if err != nil { + return err + } + if found && class.Spec.Provider == zone.Spec.Provider { + allowed, err := declaration.ZoneAllowedByClass(ctx, r.Client, class, zone.Namespace) + if err != nil { + return err + } + zoneAllowed = allowed + var provider dnsv1alpha1.Provider + if err := r.Get(ctx, client.ObjectKey{Name: class.Spec.Provider.Name}, &provider); err != nil { + if client.IgnoreNotFound(err) != nil { + return err + } + } else { + input, served := providercontract.ServedVersion(&provider, zone.Spec.Provider.Version) + storage, stored := providercontract.StorageVersion(&provider) + if served && stored && input.RecordWithdrawal != nil && input.RecordWithdrawal.Supported { + version = storage + } + bindingValid = allowed && served && stored && unitErr == nil && unit.DeletionTimestamp.IsZero() && + unit.Spec.Zone.Ref == (dnsv1alpha1.ObjectReference{Namespace: zone.Namespace, Name: zone.Name}) && + unit.Spec.Zone.ObservedGeneration == zone.Generation && unit.Spec.Zone.DomainName == zone.Spec.DomainName && + unit.Spec.Zone.ZoneClassRef == (dnsv1alpha1.ObjectReference{Namespace: class.Namespace, Name: class.Name}) && + unit.Spec.Zone.ZoneClassUID == class.UID && unit.Spec.Zone.ZoneClassObservedGeneration == class.Generation && + unit.Spec.Provider == (dnsv1alpha1.ProviderReference{Name: provider.Name, Version: storage.Name}) + } + } + } + var waivers dnsv1alpha1.RecordWithdrawalWaiverList + if len(claims) > 0 { + if err := r.List(ctx, &waivers); err != nil { + return err + } + slices.SortFunc(waivers.Items, func(a, b dnsv1alpha1.RecordWithdrawalWaiver) int { + return strings.Compare(recordSetClaimKey(a.Namespace, a.Name), recordSetClaimKey(b.Namespace, b.Name)) + }) + } + items := make([]dnsv1alpha1.ZoneUnitWithdrawal, 0, len(claims)) + if len(claims) > 256 { + // Refuse the new inventory as a whole, retaining previous identities and + // provider cache history instead of truncating obligations. + items = append(items, unit.Spec.Withdrawals...) + for i := range items { + items[i].Allowed = false + items[i].Waiver = nil + } + } + for i := range claims { + claim := &claims[i] + reason, message := "Accepted", "Withdrawal binding, grants and provider capability are accepted" + if !zoneAllowed { + reason, message = "NotAllowed", "Zone namespace is not allowed by the referenced ZoneClass" + } else if !bindingValid || len(claims) > 256 { + reason, message = "InvalidWithdrawal", "Withdrawal Zone binding is unavailable or exceeds the 256 claim limit" + } else if version == nil || version.RecordWithdrawal == nil || !version.RecordWithdrawal.Supported { + reason, message = "UnsupportedProvider", "The bound Provider version does not support record withdrawal" + } else if err := declaration.ValidateWithdrawal(claim); err != nil { + reason, message = "InvalidWithdrawal", err.Error() + } else if len(claim.Spec.Name)+len(zone.Spec.DomainName)+1 > 253 && claim.Spec.Name != "@" { + reason, message = "InvalidWithdrawal", "Withdrawal owner and Zone domain exceed the maximum DNS name length" + } else if allowed, detail := declaration.WithdrawalAllowedByZone(ctx, r.Client, claim, &zone); !allowed { + reason, message = "NotAllowed", detail + } + item := dnsv1alpha1.ZoneUnitWithdrawal{ClaimNamespace: claim.Namespace, ClaimName: claim.Name, Claim: withdrawalObjectRef(claim), ObservedGeneration: claim.Generation, Name: claim.Spec.Name, Address: claim.Spec.Address, Allowed: reason == "Accepted"} + for j := range waivers.Items { + waiver := &waivers.Items[j] + if waiver.Spec.WithdrawalRef.Namespace != claim.Namespace || waiver.Spec.WithdrawalRef.Name != claim.Name { + continue + } + valid := declaration.WithdrawalWaiverMatches(waiver, claim) + if valid && item.Allowed && item.Waiver == nil { + item.Waiver = &dnsv1alpha1.ZoneUnitWithdrawalWaiver{Name: waiver.Name, UID: waiver.UID, Digest: dnsv1alpha1.WithdrawalWaiverDigest(waiver.Spec), ReservationRef: waiver.Spec.ReservationRef} + } + } + if len(claims) <= 256 && declaration.ValidateWithdrawal(claim) == nil { + items = append(items, item) + } + status := dnsv1alpha1.RecordWithdrawalStatus{ObservedGeneration: claim.Generation} + if zoneErr == nil { + ref := withdrawalObjectRef(&zone) + status.Zone = &ref + } + if unitErr == nil { + ref := withdrawalObjectRef(&unit) + status.Unit = &ref + status.ProviderRef = &unit.Spec.Provider + } + accepted := metav1.ConditionFalse + if item.Allowed { + accepted = metav1.ConditionTrue + } + status.Conditions = append(status.Conditions, metav1.Condition{Type: string(dnsv1alpha1.ConditionAccepted), Status: accepted, Reason: reason, Message: message, ObservedGeneration: claim.Generation}) + if item.Allowed && unitErr == nil { + projectWithdrawalResult(&status, claim, item, &unit) + } + if meta.FindStatusCondition(status.Conditions, string(dnsv1alpha1.ConditionWithdrawn)) == nil { + status.Conditions = append(status.Conditions, + metav1.Condition{Type: string(dnsv1alpha1.ConditionWithdrawn), Status: metav1.ConditionUnknown, Reason: "DNSProbeFailed", Message: "No current identity-bound withdrawal observation is available", ObservedGeneration: claim.Generation}, + metav1.Condition{Type: string(dnsv1alpha1.ConditionReady), Status: metav1.ConditionFalse, Reason: "DNSProbeFailed", Message: "No current identity-bound withdrawal observation is available", ObservedGeneration: claim.Generation}) + } + if err := r.setWithdrawalStatus(ctx, claim, status); err != nil { + return err + } + } + if unitErr != nil || !unit.DeletionTimestamp.IsZero() || equality.Semantic.DeepEqual(unit.Spec.Withdrawals, items) || len(unit.Spec.Withdrawals) == 0 && len(items) == 0 { + return nil + } + base := unit.DeepCopy() + unit.Spec.Withdrawals = items + return r.Patch(ctx, &unit, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) +} + +func withdrawalObjectRef(obj client.Object) dnsv1alpha1.WithdrawalObjectReference { + return dnsv1alpha1.WithdrawalObjectReference{Namespace: obj.GetNamespace(), Name: obj.GetName(), UID: obj.GetUID()} +} + +func projectWithdrawalResult(status *dnsv1alpha1.RecordWithdrawalStatus, claim *dnsv1alpha1.RecordWithdrawal, desired dnsv1alpha1.ZoneUnitWithdrawal, unit *dnsv1alpha1.ZoneUnit) { + // Both spec and result must refer to this claim incarnation and current policy + // admission. A cached result may never cross a changed waiver or desired item. + if unit.Status.ObservedGeneration != unit.Generation || unit.Spec.Zone.ZoneClassUID == "" || unit.Spec.Zone.ZoneClassObservedGeneration < 1 { + return + } + if !slices.ContainsFunc(unit.Spec.Withdrawals, func(item dnsv1alpha1.ZoneUnitWithdrawal) bool { return equality.Semantic.DeepEqual(item, desired) }) { + return + } + for i := range unit.Status.Withdrawals { + result := &unit.Status.Withdrawals[i] + if result.Claim != desired.Claim || result.ClaimNamespace != claim.Namespace || result.ClaimName != claim.Name || result.ObservedGeneration != claim.Generation { + continue + } + withdrawn := conditionAtGeneration(result.Conditions, string(dnsv1alpha1.ConditionWithdrawn), claim.Generation) + ready := conditionAtGeneration(result.Conditions, string(dnsv1alpha1.ConditionReady), claim.Generation) + if withdrawn == nil || ready == nil { + return + } + status.PolicyFingerprint, status.CheckedNames, status.RequiredViews, status.SampledWildcard = result.PolicyFingerprint, slices.Clone(result.CheckedNames), slices.Clone(result.RequiredViews), result.SampledWildcard + status.AbsentSince, status.NotBefore, status.LastProbeTime, status.NextProbeTime, status.VerifiedAt = result.AbsentSince.DeepCopy(), result.NotBefore.DeepCopy(), result.LastProbeTime.DeepCopy(), result.NextProbeTime.DeepCopy(), result.VerifiedAt.DeepCopy() + status.ResultDigest = result.ResultDigest + if result.Provider != nil { + status.Provider = &dnsv1alpha1.ProviderStatus{Data: *result.Provider.Data.DeepCopy()} + } + status.Conditions = append(status.Conditions, *withdrawn, *ready) + if desired.Waiver != nil && result.WaiverRef != nil && result.WaiverRef.Name == desired.Waiver.Name && result.WaiverRef.UID == desired.Waiver.UID && result.WaiverRef.Digest == desired.Waiver.Digest && result.WaiverRef.ReservationRef == desired.Waiver.ReservationRef { + if override := conditionAtGeneration(result.Conditions, string(dnsv1alpha1.ConditionOverrideAccepted), claim.Generation); override != nil && override.Status == metav1.ConditionTrue && override.Reason == "OperatorAuthorizedRelease" { + status.WaiverRef = result.WaiverRef.DeepCopy() + status.Conditions = append(status.Conditions, *override) + } + } + return + } +} + +func (r *ZoneUnitCompositionReconciler) setWithdrawalStatus(ctx context.Context, claim *dnsv1alpha1.RecordWithdrawal, status dnsv1alpha1.RecordWithdrawalStatus) error { + base := claim.DeepCopy() + conditions := append([]metav1.Condition(nil), claim.Status.Conditions...) + for _, condition := range status.Conditions { + meta.SetStatusCondition(&conditions, condition) + } + if status.WaiverRef == nil { + meta.RemoveStatusCondition(&conditions, string(dnsv1alpha1.ConditionOverrideAccepted)) + } + status.Conditions = conditions + claim.Status = status + if equality.Semantic.DeepEqual(base.Status, status) { + return nil + } + return client.IgnoreNotFound(r.Status().Patch(ctx, claim, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{}))) +} diff --git a/internal/go/core/controllers/zoneunit/withdrawals_test.go b/internal/go/core/controllers/zoneunit/withdrawals_test.go new file mode 100644 index 0000000..1d45238 --- /dev/null +++ b/internal/go/core/controllers/zoneunit/withdrawals_test.go @@ -0,0 +1,266 @@ +package zoneunit + +import ( + "context" + "os" + "path/filepath" + "testing" + "time" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + "sigs.k8s.io/controller-runtime/pkg/envtest" +) + +func withdrawalFixture() (*dnsv1alpha1.Provider, *dnsv1alpha1.ZoneClass, *dnsv1alpha1.Zone, *dnsv1alpha1.RecordWithdrawal) { + provider := route53ProviderForZoneUnitTest() + provider.Spec.Versions[0].RecordWithdrawal = &dnsv1alpha1.ProviderRecordWithdrawal{Supported: true} + class := route53ZoneClassForZoneUnitTest() + class.Namespace = "app" + class.Spec.IdentityRef = nil + class.Spec.Parameters = runtime.RawExtension{Raw: []byte(`{}`)} + z := zone("app", "example", "example.com") + claim := &dnsv1alpha1.RecordWithdrawal{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "retired", UID: "claim-uid", Generation: 1, Labels: map[string]string{dnsv1alpha1.AddressLabelKey: "192.0.2.1"}}, Spec: dnsv1alpha1.RecordWithdrawalSpec{ZoneRef: dnsv1alpha1.ZoneReference{Name: z.Name}, Name: "www", Address: "192.0.2.1"}} + return provider, class, z, claim +} + +func TestWithdrawalCompositionAuthorization(t *testing.T) { + for _, tc := range []struct { + name string + mutate func(*dnsv1alpha1.Provider, *dnsv1alpha1.Zone, *dnsv1alpha1.RecordWithdrawal) + reason string + }{ + {"accepted", func(*dnsv1alpha1.Provider, *dnsv1alpha1.Zone, *dnsv1alpha1.RecordWithdrawal) {}, "Accepted"}, + {"unsupported", func(p *dnsv1alpha1.Provider, _ *dnsv1alpha1.Zone, _ *dnsv1alpha1.RecordWithdrawal) { + p.Spec.Versions[0].RecordWithdrawal = nil + }, "UnsupportedProvider"}, + {"wildcard requires grant", func(_ *dnsv1alpha1.Provider, _ *dnsv1alpha1.Zone, c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Name = "*" }, "NotAllowed"}, + {"invalid label", func(_ *dnsv1alpha1.Provider, _ *dnsv1alpha1.Zone, c *dnsv1alpha1.RecordWithdrawal) { c.Labels = nil }, "InvalidWithdrawal"}, + {"cross namespace denied", func(_ *dnsv1alpha1.Provider, _ *dnsv1alpha1.Zone, c *dnsv1alpha1.RecordWithdrawal) { + c.Namespace = "tenant" + c.Spec.ZoneRef.Namespace = ptr("app") + }, "NotAllowed"}, + {"wildcard grant", func(_ *dnsv1alpha1.Provider, z *dnsv1alpha1.Zone, c *dnsv1alpha1.RecordWithdrawal) { + c.Spec.Name = "*.apps" + z.Spec.AllowedRecordSets = withdrawalGrant() + }, "Accepted"}, + } { + t.Run(tc.name, func(t *testing.T) { + p, class, z, c := withdrawalFixture() + tc.mutate(p, z, c) + k8sClient := fake.NewClientBuilder().WithScheme(testScheme(t)).WithObjects(p, class, z, c, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "app", Labels: map[string]string{"dns": "allowed"}}}).WithStatusSubresource(&dnsv1alpha1.RecordWithdrawal{}, &dnsv1alpha1.Zone{}, &dnsv1alpha1.ZoneUnit{}, &dnsv1alpha1.RecordWithdrawalWaiver{}).Build() + r := &ZoneUnitCompositionReconciler{Client: k8sClient} + if _, err := r.Reconcile(context.Background(), ctrl.Request{NamespacedName: client.ObjectKeyFromObject(z)}); err != nil { + t.Fatal(err) + } + var got dnsv1alpha1.RecordWithdrawal + if err := k8sClient.Get(context.Background(), client.ObjectKeyFromObject(c), &got); err != nil { + t.Fatal(err) + } + status := metav1.ConditionFalse + if tc.reason == "Accepted" { + status = metav1.ConditionTrue + } + assertCondition(t, got.Status.Conditions, "Accepted", status, tc.reason) + }) + } +} + +func withdrawalGrant() []dnsv1alpha1.AllowedRecordSet { + return []dnsv1alpha1.AllowedRecordSet{{Namespaces: dnsv1alpha1.AllowedRecordSetNamespaces{Selector: metav1.LabelSelector{MatchLabels: map[string]string{"dns": "allowed"}}}, Records: []dnsv1alpha1.AllowedRecord{{Name: dnsv1alpha1.RecordNamePolicy{Pattern: `\*\.apps`}, Types: []dnsv1alpha1.RecordType{dnsv1alpha1.RecordTypeA}}}}} +} + +func TestWithdrawalEnvtestCompositionProjectionWaiverAndRetention(t *testing.T) { + assets := os.Getenv("KUBEBUILDER_ASSETS") + if assets == "" { + t.Skip("KUBEBUILDER_ASSETS is required for the real CRD integration test") + } + environment := &envtest.Environment{BinaryAssetsDirectory: assets, CRDDirectoryPaths: []string{filepath.Join("..", "..", "..", "..", "..", "app", "operator", "config", "crd", "bases")}, ErrorIfCRDPathMissing: true, ControlPlaneStartTimeout: 30 * time.Second, ControlPlaneStopTimeout: 15 * time.Second} + t.Cleanup(func() { + if err := environment.Stop(); err != nil { + t.Errorf("stop envtest: %v", err) + } + }) + cfg, err := environment.Start() + if err != nil { + t.Fatal(err) + } + scheme := testScheme(t) + if err := endpointv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + c, err := client.New(cfg, client.Options{Scheme: scheme}) + if err != nil { + t.Fatal(err) + } + ctx := context.Background() + p, class, z, claim := withdrawalFixture() + claim.UID = "" + claim.Generation = 0 + claim.Spec.Name = "*.apps" + z.Spec.AllowedRecordSets = withdrawalGrant() + for _, obj := range []client.Object{&corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "app", Labels: map[string]string{"dns": "allowed"}}}, p, class, z, claim} { + if err := c.Create(ctx, obj); err != nil { + t.Fatalf("create %T: %v", obj, err) + } + } + r := &ZoneUnitCompositionReconciler{Client: c} + key := client.ObjectKeyFromObject(z) + reconcile := func() { + t.Helper() + if _, err := r.Reconcile(ctx, ctrl.Request{NamespacedName: key}); err != nil { + t.Fatal(err) + } + } + readClaim := func() { + t.Helper() + if err := c.Get(ctx, client.ObjectKeyFromObject(claim), claim); err != nil { + t.Fatal(err) + } + } + readUnit := func() *dnsv1alpha1.ZoneUnit { + t.Helper() + u := &dnsv1alpha1.ZoneUnit{} + if err := c.Get(ctx, key, u); err != nil { + t.Fatal(err) + } + return u + } + reconcile() + readClaim() + assertCondition(t, claim.Status.Conditions, "Accepted", metav1.ConditionTrue, "Accepted") + unit := readUnit() + if len(unit.Spec.Withdrawals) != 1 || unit.Spec.Withdrawals[0].Claim.UID != claim.UID || !unit.Spec.Withdrawals[0].Allowed { + t.Fatalf("composition=%#v", unit.Spec.Withdrawals) + } + oldSpec := claim.DeepCopy() + claim.Spec.Address = "192.0.2.2" + if err := c.Update(ctx, claim); !apierrors.IsInvalid(err) { + t.Fatalf("mutable claim admitted: %v", err) + } + claim = oldSpec + invalidUnit := unit.DeepCopy() + invalidUnit.Spec.Withdrawals[0].ClaimName = "different" + if err := c.Update(ctx, invalidUnit); !apierrors.IsInvalid(err) { + t.Fatalf("inconsistent scalar key admitted: %v", err) + } + waiver := &dnsv1alpha1.RecordWithdrawalWaiver{ObjectMeta: metav1.ObjectMeta{Namespace: claim.Namespace, Name: "risk"}, Spec: dnsv1alpha1.RecordWithdrawalWaiverSpec{WithdrawalRef: withdrawalObjectRef(claim), ClaimSpecDigest: dnsv1alpha1.WithdrawalSpecDigest(claim.Spec), Address: claim.Spec.Address, ReservationRef: dnsv1alpha1.WithdrawalObjectReference{Namespace: "mgmt", Name: "reservation", UID: "reservation-uid"}, Reason: "explicit operator risk acceptance", TicketRef: "change-1", AcceptStaleDNSRisk: true}} + if err := c.Create(ctx, waiver); err != nil { + t.Fatal(err) + } + if _, err := (&RecordWithdrawalWaiverReconciler{Client: c}).Reconcile(ctx, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(waiver)}); err != nil { + t.Fatal(err) + } + if err := c.Get(ctx, client.ObjectKeyFromObject(waiver), waiver); err != nil { + t.Fatal(err) + } + assertCondition(t, waiver.Status.Conditions, "Accepted", metav1.ConditionTrue, "Accepted") + reconcile() + unit = readUnit() + if unit.Spec.Withdrawals[0].Waiver == nil { + t.Fatal("exact waiver not composed") + } + changedWaiver := waiver.DeepCopy() + changedWaiver.Spec.ReservationRef.UID = "replacement" + if err := c.Update(ctx, changedWaiver); !apierrors.IsInvalid(err) { + t.Fatalf("mutable waiver admitted: %v", err) + } + stamp := metav1.NewTime(time.Date(2026, 10, 18, 1, 0, 0, 0, time.UTC)) + item := unit.Spec.Withdrawals[0] + result := dnsv1alpha1.ZoneUnitWithdrawalStatus{ClaimNamespace: claim.Namespace, ClaimName: claim.Name, Claim: withdrawalObjectRef(claim), ObservedGeneration: claim.Generation, PolicyFingerprint: "policy", CheckedNames: []string{"witness.apps.example.com"}, RequiredViews: []string{"recursive"}, SampledWildcard: true, LastProbeTime: &stamp, NextProbeTime: &stamp, ResultDigest: "result", WaiverRef: &dnsv1alpha1.RecordWithdrawalWaiverReference{Name: waiver.Name, UID: waiver.UID, Digest: item.Waiver.Digest, ReservationRef: waiver.Spec.ReservationRef}, Conditions: []metav1.Condition{{Type: "Withdrawn", Status: metav1.ConditionFalse, Reason: "DNSStillResolves", Message: "DNS witness.apps.example.com still resolves to retired address 192.0.2.1 in recursive (A).", ObservedGeneration: claim.Generation, LastTransitionTime: stamp}, {Type: "Ready", Status: metav1.ConditionFalse, Reason: "DNSStillResolves", Message: "provider detail", ObservedGeneration: claim.Generation, LastTransitionTime: stamp}, {Type: "OverrideAccepted", Status: metav1.ConditionTrue, Reason: "OperatorAuthorizedRelease", Message: "explicit risk", ObservedGeneration: claim.Generation, LastTransitionTime: stamp}}} + unit.Status.ObservedGeneration = unit.Generation + unit.Status.Withdrawals = []dnsv1alpha1.ZoneUnitWithdrawalStatus{result} + unit.Status.Zone = &dnsv1alpha1.ZoneUnitZoneStatus{Conditions: []metav1.Condition{ + {Type: "Accepted", Status: metav1.ConditionTrue, Reason: "Accepted", ObservedGeneration: z.Generation, LastTransitionTime: stamp}, + {Type: "Programmed", Status: metav1.ConditionTrue, Reason: "Programmed", ObservedGeneration: z.Generation, LastTransitionTime: stamp}, + }} + if err := c.Status().Update(ctx, unit); err != nil { + t.Fatal(err) + } + reconcile() + readClaim() + condition := assertCondition(t, claim.Status.Conditions, "Withdrawn", metav1.ConditionFalse, "DNSStillResolves") + if condition.Message != result.Conditions[0].Message { + t.Fatalf("message lost: %s", condition.Message) + } + assertCondition(t, claim.Status.Conditions, "OverrideAccepted", metav1.ConditionTrue, "OperatorAuthorizedRelease") + if err := c.Get(ctx, key, z); err != nil { + t.Fatal(err) + } + assertCondition(t, z.Status.Conditions, "Programmed", metav1.ConditionTrue, "Programmed") + // A policy edit fences old observations before the verifier runs again. + class.Spec.Parameters = runtime.RawExtension{Raw: []byte(`{"policy":"changed"}`)} + if err := c.Update(ctx, class); err != nil { + t.Fatal(err) + } + reconcile() + readClaim() + assertCondition(t, claim.Status.Conditions, "Withdrawn", metav1.ConditionUnknown, "DNSProbeFailed") + unit = readUnit() + if unit.Spec.Zone.ZoneClassUID != class.UID || unit.Spec.Zone.ZoneClassObservedGeneration != class.Generation { + t.Fatal("Unit did not bind the new Class generation") + } + // Retained claims pin deletion without marking their provider Unit deleting. + if err := c.Get(ctx, key, z); err != nil { + t.Fatal(err) + } + if err := c.Delete(ctx, z); err != nil { + t.Fatal(err) + } + reconcile() + unit = readUnit() + if !unit.DeletionTimestamp.IsZero() { + t.Fatal("pending withdrawal allowed Unit deletion") + } + if err := c.Get(ctx, key, z); err != nil { + t.Fatal(err) + } + gate := assertCondition(t, z.Status.Conditions, "Programmed", metav1.ConditionFalse, "ProviderChangePending") + if gate.Message == "" { + t.Fatal("missing dependency detail") + } + // A same-name recreation cannot consume the old result or waiver. + if err := c.Delete(ctx, claim); err != nil { + t.Fatal(err) + } + replacement := oldSpec.DeepCopy() + replacement.ResourceVersion = "" + replacement.UID = "" + replacement.Generation = 0 + replacement.CreationTimestamp = metav1.Time{} + replacement.Status = dnsv1alpha1.RecordWithdrawalStatus{} + if err := c.Create(ctx, replacement); err != nil { + t.Fatal(err) + } + claim = replacement + reconcile() + readClaim() + assertCondition(t, claim.Status.Conditions, "Withdrawn", metav1.ConditionUnknown, "DNSProbeFailed") + unit = readUnit() + if unit.Spec.Withdrawals[0].Waiver != nil { + t.Fatal("old waiver attached to recreated claim") + } + if _, err := (&RecordWithdrawalWaiverReconciler{Client: c}).Reconcile(ctx, ctrl.Request{NamespacedName: client.ObjectKeyFromObject(waiver)}); err != nil { + t.Fatal(err) + } + if err := c.Get(ctx, client.ObjectKeyFromObject(waiver), waiver); err != nil { + t.Fatal(err) + } + assertCondition(t, waiver.Status.Conditions, "Accepted", metav1.ConditionFalse, "InvalidWithdrawal") + // CEL admits retire-only intent but never an empty unprotected ERS. + ers := &endpointv1alpha1.EndpointRecordSet{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "retire-only"}, Spec: endpointv1alpha1.EndpointRecordSetSpec{RetiredTargets: []endpointv1alpha1.RetiredTarget{{ID: "capture", SourceGeneration: 1, Address: "192.0.2.1", Hostnames: []string{"www.example.com"}}}}} + if err := c.Create(ctx, ers); err != nil { + t.Fatal(err) + } + ers.Spec.RetiredTargets = nil + if err := c.Update(ctx, ers); !apierrors.IsInvalid(err) { + t.Fatalf("empty ERS admitted: %v", err) + } +} diff --git a/internal/go/core/controllers/zoneunit/zoneunit_controller.go b/internal/go/core/controllers/zoneunit/zoneunit_controller.go index 47c4047..c91bfba 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_controller.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_controller.go @@ -29,7 +29,7 @@ const ( recordSetNotAcceptedProgrammedMessage = "RecordSet programming is waiting for Accepted=True" ) -// ZoneUnitCompositionReconciler builds ZoneUnit desired state from Zone and RecordSet claims. +// ZoneUnitCompositionReconciler composes Zone, RecordSet and retained withdrawal claims. type ZoneUnitCompositionReconciler struct { client.Client } @@ -46,7 +46,7 @@ type ZoneUnitCompositionReconciler struct { // +kubebuilder:rbac:groups=dns.appthrust.io,resources=zoneunits/finalizers,verbs=update // +kubebuilder:rbac:groups=dns.appthrust.io,resources=zoneunits/status,verbs=get;watch // +kubebuilder:rbac:groups="",resources=namespaces,verbs=get;list;watch -func (r *ZoneUnitCompositionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { +func (r *ZoneUnitCompositionReconciler) reconcileZone(ctx context.Context, req ctrl.Request, withdrawals []dnsv1alpha1.RecordWithdrawal) (ctrl.Result, error) { recordSets, err := r.recordSetsForZoneKey(ctx, req.NamespacedName) if err != nil { return ctrl.Result{}, err @@ -201,12 +201,16 @@ func (r *ZoneUnitCompositionReconciler) Reconcile(ctx context.Context, req ctrl. Spec: dnsv1alpha1.ZoneUnitSpec{ Provider: dnsv1alpha1.ProviderReference{Name: provider.Name, Version: storageVersion.Name}, Zone: dnsv1alpha1.ZoneUnitZoneSpec{ - Ref: dnsv1alpha1.ObjectReference{Namespace: zone.Namespace, Name: zone.Name}, - ObservedGeneration: zone.Generation, - DomainName: zone.Spec.DomainName, - ZoneClassRef: dnsv1alpha1.ObjectReference{Namespace: zoneClass.Namespace, Name: zoneClass.Name}, + Ref: dnsv1alpha1.ObjectReference{Namespace: zone.Namespace, Name: zone.Name}, + ObservedGeneration: zone.Generation, + DomainName: zone.Spec.DomainName, + ZoneClassRef: dnsv1alpha1.ObjectReference{Namespace: zoneClass.Namespace, Name: zoneClass.Name}, + ZoneClassUID: zoneClass.UID, + ZoneClassObservedGeneration: zoneClass.Generation, }, RecordSets: items, + // Preserve the independently converged withdrawal slice. + Withdrawals: existing.Spec.Withdrawals, }, } syncZoneUnitReconcileRequestAnnotation(&desired, &zone) @@ -215,7 +219,7 @@ func (r *ZoneUnitCompositionReconciler) Reconcile(ctx context.Context, req ctrl. } if !zone.DeletionTimestamp.IsZero() { - if len(recordSets) > 0 { + if len(recordSets) > 0 || len(withdrawals) > 0 { if !unitExists || existing.DeletionTimestamp.IsZero() { if err := r.applyZoneUnitDesired(ctx, &desired, !unitExists); err != nil { return ctrl.Result{}, err @@ -233,7 +237,7 @@ func (r *ZoneUnitCompositionReconciler) Reconcile(ctx context.Context, req ctrl. "Zone is accepted", metav1.ConditionFalse, "ProviderChangePending", - fmt.Sprintf("%d RecordSet resources still reference this Zone; delete or move them before Zone cleanup can continue", len(recordSets)), + fmt.Sprintf("%d RecordSet and %d RecordWithdrawal resources still reference this Zone; archive dependent claims before Zone cleanup can continue", len(recordSets), len(withdrawals)), providerStatus, ) } diff --git a/internal/go/core/controllers/zoneunit/zoneunit_watches.go b/internal/go/core/controllers/zoneunit/zoneunit_watches.go index 92365e2..9a018fe 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_watches.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_watches.go @@ -12,10 +12,15 @@ import ( ) func (r *ZoneUnitCompositionReconciler) SetupWithManager(mgr ctrl.Manager) error { + if err := (&RecordWithdrawalWaiverReconciler{Client: r.Client}).SetupWithManager(mgr); err != nil { + return err + } return ctrl.NewControllerManagedBy(mgr). Named("zoneunit-composition"). For(&dnsv1alpha1.Zone{}). Watches(&dnsv1alpha1.RecordSet{}, handler.EnqueueRequestsFromMapFunc(r.mapRecordSetToZone)). + Watches(&dnsv1alpha1.RecordWithdrawal{}, handler.EnqueueRequestsFromMapFunc(r.mapWithdrawalToZone)). + Watches(&dnsv1alpha1.RecordWithdrawalWaiver{}, handler.EnqueueRequestsFromMapFunc(r.mapWaiverToZone)). Watches(&dnsv1alpha1.ZoneUnit{}, handler.EnqueueRequestsFromMapFunc(r.mapZoneUnitToZone)). Watches(&dnsv1alpha1.ZoneClass{}, handler.EnqueueRequestsFromMapFunc(r.mapZoneClassToZones)). Watches(&dnsv1alpha1.Provider{}, handler.EnqueueRequestsFromMapFunc(r.mapProviderToZones)). @@ -116,6 +121,13 @@ func (r *ZoneUnitCompositionReconciler) mapNamespaceToZones(ctx context.Context, zoneNamespace, zoneName := recordSetZoneKey(&recordSet) appendRequest(client.ObjectKey{Namespace: zoneNamespace, Name: zoneName}) } + var withdrawals dnsv1alpha1.RecordWithdrawalList + if err := r.List(ctx, &withdrawals, client.InNamespace(namespace.Name)); err != nil { + return requests + } + for i := range withdrawals.Items { + appendRequest(withdrawalZoneKey(&withdrawals.Items[i])) + } return requests } diff --git a/internal/go/core/declaration/policy.go b/internal/go/core/declaration/policy.go index 2fee995..0c6740e 100644 --- a/internal/go/core/declaration/policy.go +++ b/internal/go/core/declaration/policy.go @@ -38,7 +38,11 @@ func ZoneAllowedByClass(ctx context.Context, k8sClient client.Client, zoneClass } func RecordSetAllowedByZone(ctx context.Context, k8sClient client.Client, recordSet *dnsv1alpha1.RecordSet, zone *dnsv1alpha1.Zone) (bool, string) { - if recordSet.Namespace == zone.Namespace { + return recordSetAllowedByZone(ctx, k8sClient, recordSet, zone, true) +} + +func recordSetAllowedByZone(ctx context.Context, k8sClient client.Client, recordSet *dnsv1alpha1.RecordSet, zone *dnsv1alpha1.Zone, allowSameNamespace bool) (bool, string) { + if allowSameNamespace && recordSet.Namespace == zone.Namespace { return true, "" } namespaceMatched := false diff --git a/internal/go/core/declaration/withdrawal.go b/internal/go/core/declaration/withdrawal.go new file mode 100644 index 0000000..01006f3 --- /dev/null +++ b/internal/go/core/declaration/withdrawal.go @@ -0,0 +1,84 @@ +package declaration + +import ( + "context" + "fmt" + "net/netip" + "strings" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/validation" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func ValidateWithdrawal(claim *dnsv1alpha1.RecordWithdrawal) error { + if len(claim.OwnerReferences) != 0 || len(claim.Finalizers) != 0 { + return fmt.Errorf("withdrawals must not have ownerReferences or finalizers") + } + if len(validation.IsDNS1123Subdomain(claim.Spec.ZoneRef.Name)) != 0 { + return fmt.Errorf("spec.zoneRef.name must be a valid object name") + } + if claim.Spec.ZoneRef.Namespace != nil && len(validation.IsDNS1123Label(*claim.Spec.ZoneRef.Namespace)) != 0 { + return fmt.Errorf("spec.zoneRef.namespace must be a valid namespace") + } + name := claim.Spec.Name + if name != "@" && name != "*" { + if strings.HasPrefix(name, "*.") { + name = strings.TrimPrefix(name, "*.") + } + if len(validation.IsDNS1123Subdomain(name)) != 0 || len(claim.Spec.Name) > 253 { + return fmt.Errorf("spec.name must be a canonical relative DNS owner") + } + } + address, err := netip.ParseAddr(claim.Spec.Address) + if err != nil || address.Zone() != "" || address.Is4In6() || address.String() != claim.Spec.Address { + return fmt.Errorf("spec.address must be a canonical unscoped and unmapped IP address") + } + if claim.Labels[dnsv1alpha1.AddressLabelKey] != dnsv1alpha1.AddressLabelValue(address) { + return fmt.Errorf("metadata.labels[%s] must match spec.address", dnsv1alpha1.AddressLabelKey) + } + return nil +} + +func ValidateWithdrawalWaiver(waiver *dnsv1alpha1.RecordWithdrawalWaiver) error { + if len(waiver.OwnerReferences) != 0 || len(waiver.Finalizers) != 0 { + return fmt.Errorf("waivers must not have ownerReferences or finalizers") + } + for _, ref := range []dnsv1alpha1.WithdrawalObjectReference{waiver.Spec.WithdrawalRef, waiver.Spec.ReservationRef} { + if len(validation.IsDNS1123Label(ref.Namespace)) != 0 || len(validation.IsDNS1123Subdomain(ref.Name)) != 0 || ref.UID == "" { + return fmt.Errorf("withdrawalRef and reservationRef require exact namespace, name and UID") + } + } + if waiver.Spec.WithdrawalRef.Namespace != waiver.Namespace { + return fmt.Errorf("spec.withdrawalRef.namespace must match waiver namespace") + } + ip, err := netip.ParseAddr(waiver.Spec.Address) + if err != nil || ip.Zone() != "" || ip.Is4In6() || ip.String() != waiver.Spec.Address { + return fmt.Errorf("spec.address must be a canonical unscoped and unmapped IP address") + } + if !waiver.Spec.AcceptStaleDNSRisk { + return fmt.Errorf("spec.acceptStaleDNSRisk must be true") + } + if strings.TrimSpace(waiver.Spec.Reason) == "" || len(waiver.Spec.Reason) > 2048 || strings.TrimSpace(waiver.Spec.TicketRef) == "" || len(waiver.Spec.TicketRef) > 1024 { + return fmt.Errorf("spec.reason and spec.ticketRef must be nonempty and bounded") + } + return nil +} + +func WithdrawalWaiverMatches(waiver *dnsv1alpha1.RecordWithdrawalWaiver, claim *dnsv1alpha1.RecordWithdrawal) bool { + return ValidateWithdrawalWaiver(waiver) == nil && waiver.DeletionTimestamp.IsZero() && waiver.UID != "" && waiver.Namespace == claim.Namespace && waiver.Spec.WithdrawalRef == (dnsv1alpha1.WithdrawalObjectReference{Namespace: claim.Namespace, Name: claim.Name, UID: claim.UID}) && waiver.Spec.ClaimSpecDigest == dnsv1alpha1.WithdrawalSpecDigest(claim.Spec) && waiver.Spec.Address == claim.Spec.Address +} + +func WithdrawalAllowedByZone(ctx context.Context, c client.Client, claim *dnsv1alpha1.RecordWithdrawal, zone *dnsv1alpha1.Zone) (bool, string) { + ip, err := netip.ParseAddr(claim.Spec.Address) + if err != nil { + return false, "Withdrawal address is invalid" + } + recordType := dnsv1alpha1.RecordTypeA + if ip.Is6() { + recordType = dnsv1alpha1.RecordTypeAAAA + } + record := dnsv1alpha1.RecordSet{ObjectMeta: metav1.ObjectMeta{Namespace: claim.Namespace}, Spec: dnsv1alpha1.RecordSetSpec{Name: claim.Spec.Name, Type: recordType}} + return recordSetAllowedByZone(ctx, c, &record, zone, !strings.HasPrefix(claim.Spec.Name, "*")) +} diff --git a/internal/go/core/webhook/validator.go b/internal/go/core/webhook/validator.go index 0fea100..937f3d8 100644 --- a/internal/go/core/webhook/validator.go +++ b/internal/go/core/webhook/validator.go @@ -66,6 +66,8 @@ func SetupCoreValidationWebhookWithManager(mgr ctrl.Manager) error { server.Register(zoneWebhookPath, &webhook.Admission{Handler: validator}) server.Register(recordSetWebhookPath, &webhook.Admission{Handler: validator}) server.Register(zoneUnitWebhookPath, &webhook.Admission{Handler: validator}) + server.Register(recordWithdrawalWebhookPath, &webhook.Admission{Handler: validator}) + server.Register(recordWithdrawalWaiverWebhookPath, &webhook.Admission{Handler: validator}) return nil } @@ -135,6 +137,10 @@ func (v *CoreValidator) Handle(ctx context.Context, req admission.Request) admis } else if err == nil { err = v.validateRecordSet(ctx, &recordSet) } + case "RecordWithdrawal": + err = v.validateWithdrawalRequest(ctx, req) + case "RecordWithdrawalWaiver": + err = v.validateWaiverRequest(req) case "ZoneUnit": var zoneUnit dnsv1alpha1.ZoneUnit err = v.decoder.Decode(req, &zoneUnit) @@ -243,6 +249,9 @@ func validateProvider(provider *dnsv1alpha1.Provider) error { if err := validateProviderConditionReasons(version.RecordSet.AdditionalConditionReasons, prefix+".recordSet.additionalConditionReasons", recordSetCoreConditionReasons()); err != nil { return err } + if err := validateProviderWithdrawal(version.RecordWithdrawal, prefix+".recordWithdrawal"); err != nil { + return err + } for index, toggle := range version.RecordSet.DisableValidations { path := fmt.Sprintf("%s.recordSet.disableValidations[%d].when", prefix, index) if err := compileObjectCELBoolRule(toggle.When, path); err != nil { @@ -752,10 +761,24 @@ func (v *CoreValidator) validateZoneUnit(ctx context.Context, zoneUnit *dnsv1alp } } } - return nil + return validateUnitWithdrawals(zoneUnit, oldZoneUnit) } func (v *CoreValidator) validateZoneUnitDelete(ctx context.Context, zoneUnit *dnsv1alpha1.ZoneUnit) error { + var claims dnsv1alpha1.RecordWithdrawalList + if err := v.List(ctx, &claims); err != nil { + return err + } + for i := range claims.Items { + claim := &claims.Items[i] + namespace := claim.Namespace + if claim.Spec.ZoneRef.Namespace != nil { + namespace = *claim.Spec.ZoneRef.Namespace + } + if namespace == zoneUnit.Spec.Zone.Ref.Namespace && claim.Spec.ZoneRef.Name == zoneUnit.Spec.Zone.Ref.Name { + return errors.New("ZoneUnit cannot be deleted while RecordWithdrawal resources reference its Zone") + } + } var zone dnsv1alpha1.Zone err := v.Client.Get(ctx, client.ObjectKey{Namespace: zoneUnit.Spec.Zone.Ref.Namespace, Name: zoneUnit.Spec.Zone.Ref.Name}, &zone) if apierrors.IsNotFound(err) { @@ -830,7 +853,7 @@ func (v *CoreValidator) validateZoneUnitStatus(ctx context.Context, zoneUnit *dn } } } - return nil + return v.validateUnitWithdrawalStatus(zoneUnit, providerVersion) } func (v *CoreValidator) zoneForRecordSet(ctx context.Context, recordSet *dnsv1alpha1.RecordSet) (*dnsv1alpha1.Zone, error) { @@ -1113,8 +1136,8 @@ func validateProviderConditionReasons(reasons []dnsv1alpha1.ProviderConditionRea seen := conditionReasonSet{} for index, reason := range reasons { itemPath := fmt.Sprintf("%s[%d]", path, index) - if !isKnownConditionType(reason.ConditionType) { - return fmt.Errorf("%s.conditionType must be Accepted or Programmed", itemPath) + if _, allowed := core[reason.ConditionType]; !allowed { + return fmt.Errorf("%s.conditionType is not allowed for this resource", itemPath) } if !isKnownConditionStatus(reason.Status) { return fmt.Errorf("%s.status must be True, False, or Unknown", itemPath) @@ -1373,10 +1396,6 @@ func recordSetCoreConditionReasons() conditionReasonSet { } } -func isKnownConditionType(conditionType dnsv1alpha1.ConditionType) bool { - return conditionType == dnsv1alpha1.ConditionAccepted || conditionType == dnsv1alpha1.ConditionProgrammed -} - func isKnownConditionStatus(status metav1.ConditionStatus) bool { return status == metav1.ConditionTrue || status == metav1.ConditionFalse || status == metav1.ConditionUnknown } diff --git a/internal/go/core/webhook/validator_test.go b/internal/go/core/webhook/validator_test.go index a1336b0..aebfc59 100644 --- a/internal/go/core/webhook/validator_test.go +++ b/internal/go/core/webhook/validator_test.go @@ -1838,6 +1838,10 @@ func TestBuiltInProviderManifestsPassValidation(t *testing.T) { if err := validateProvider(&provider); err != nil { t.Fatalf("validateProvider(%s) returned error: %v", name, err) } + capability := provider.Spec.Versions[0].RecordWithdrawal + if capability == nil || capability.Supported != (provider.Name == "external.dns.appthrust.io") { + t.Fatalf("provider %s has incorrect withdrawal capability: %#v", provider.Name, capability) + } }) } } diff --git a/internal/go/core/webhook/withdrawals.go b/internal/go/core/webhook/withdrawals.go new file mode 100644 index 0000000..bdcb92c --- /dev/null +++ b/internal/go/core/webhook/withdrawals.go @@ -0,0 +1,244 @@ +package webhook + +import ( + "context" + "encoding/json" + "fmt" + "net/netip" + "reflect" + "regexp" + + "github.com/appthrust/dns-api/internal/go/core/declaration" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + admissionv1 "k8s.io/api/admission/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +const ( + recordWithdrawalWebhookPath = "/validate-dns-appthrust-io-v1alpha1-recordwithdrawal" + recordWithdrawalWaiverWebhookPath = "/validate-dns-appthrust-io-v1alpha1-recordwithdrawalwaiver" +) + +var withdrawalDigestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`) + +func (v *CoreValidator) validateWithdrawalRequest(ctx context.Context, req admission.Request) error { + var claim dnsv1alpha1.RecordWithdrawal + if err := v.decoder.Decode(req, &claim); err != nil { + return err + } + if req.SubResource == "status" { + return v.validateWithdrawalStatus(ctx, &claim) + } + if err := declaration.ValidateWithdrawal(&claim); err != nil { + return err + } + if req.Operation == admissionv1.Update { + var old dnsv1alpha1.RecordWithdrawal + if err := json.Unmarshal(req.OldObject.Raw, &old); err != nil { + return fmt.Errorf("old RecordWithdrawal object could not be decoded: %w", err) + } + if !reflect.DeepEqual(old.Spec, claim.Spec) { + return fmt.Errorf("spec is immutable") + } + } + return nil +} + +func (v *CoreValidator) validateWaiverRequest(req admission.Request) error { + var waiver dnsv1alpha1.RecordWithdrawalWaiver + if err := v.decoder.Decode(req, &waiver); err != nil { + return err + } + if req.SubResource == "status" { + if err := validateObservedGeneration(waiver.Status.ObservedGeneration, waiver.Generation, "status.observedGeneration"); err != nil { + return err + } + if err := validateConditionObservedGenerations(waiver.Status.Conditions, waiver.Generation, "status.conditions"); err != nil { + return err + } + return validateStatusConditions(waiver.Status.Conditions, "status.conditions", withdrawalAcceptanceReasons()) + } + if err := declaration.ValidateWithdrawalWaiver(&waiver); err != nil { + return err + } + if !withdrawalDigestPattern.MatchString(waiver.Spec.ClaimSpecDigest) { + return fmt.Errorf("spec.claimSpecDigest must be a sha256 digest") + } + if req.Operation == admissionv1.Update { + var old dnsv1alpha1.RecordWithdrawalWaiver + if err := json.Unmarshal(req.OldObject.Raw, &old); err != nil { + return fmt.Errorf("old RecordWithdrawalWaiver object could not be decoded: %w", err) + } + if !reflect.DeepEqual(old.Spec, waiver.Spec) { + return fmt.Errorf("spec is immutable") + } + } + return nil +} + +func withdrawalAcceptanceReasons() conditionReasonSet { + return conditionReasonSet{dnsv1alpha1.ConditionAccepted: {metav1.ConditionTrue: {"Accepted": {}}, metav1.ConditionFalse: {"InvalidWithdrawal": {}, "NotAllowed": {}, "UnsupportedProvider": {}}}} +} + +func withdrawalConditionReasons() conditionReasonSet { + set := withdrawalAcceptanceReasons() + for _, reason := range []string{"DNSStillResolves", "NegativeTTLNotElapsed"} { + addConditionReason(set, dnsv1alpha1.ConditionWithdrawn, metav1.ConditionFalse, reason) + addConditionReason(set, dnsv1alpha1.ConditionReady, metav1.ConditionFalse, reason) + } + for _, reason := range []string{"DNSProbeFailed", "PolicyChanged"} { + addConditionReason(set, dnsv1alpha1.ConditionWithdrawn, metav1.ConditionUnknown, reason) + addConditionReason(set, dnsv1alpha1.ConditionReady, metav1.ConditionFalse, reason) + } + addConditionReason(set, dnsv1alpha1.ConditionWithdrawn, metav1.ConditionTrue, "WithdrawalVerified") + addConditionReason(set, dnsv1alpha1.ConditionReady, metav1.ConditionTrue, "WithdrawalVerified") + addConditionReason(set, dnsv1alpha1.ConditionOverrideAccepted, metav1.ConditionTrue, "OperatorAuthorizedRelease") + return set +} + +func validateWithdrawalConditions(conditions []metav1.Condition, generation int64, path string) error { + if err := validateStatusConditions(conditions, path, withdrawalConditionReasons()); err != nil { + return err + } + if err := validateConditionObservedGenerations(conditions, generation, path); err != nil { + return err + } + seen := make(map[string]bool, len(conditions)) + for _, c := range conditions { + if seen[c.Type] { + return fmt.Errorf("%s contains duplicate condition type %s", path, c.Type) + } + seen[c.Type] = true + if len(c.Message) > 2048 { + return fmt.Errorf("%s message exceeds 2048 bytes", path) + } + } + return nil +} + +func (v *CoreValidator) validateWithdrawalStatus(ctx context.Context, claim *dnsv1alpha1.RecordWithdrawal) error { + if err := validateObservedGeneration(claim.Status.ObservedGeneration, claim.Generation, "status.observedGeneration"); err != nil { + return err + } + if err := validateWithdrawalConditions(claim.Status.Conditions, claim.Generation, "status.conditions"); err != nil { + return err + } + if claim.Status.Zone != nil { + namespace := claim.Namespace + if claim.Spec.ZoneRef.Namespace != nil { + namespace = *claim.Spec.ZoneRef.Namespace + } + if claim.Status.Zone.Namespace != namespace || claim.Status.Zone.Name != claim.Spec.ZoneRef.Name || claim.Status.Zone.UID == "" { + return fmt.Errorf("status.zone must match spec.zoneRef with a nonempty UID") + } + if claim.Status.Unit != nil && (claim.Status.Unit.Namespace != namespace || claim.Status.Unit.Name != claim.Spec.ZoneRef.Name || claim.Status.Unit.UID == "") { + return fmt.Errorf("status.unit must match the resolved Zone with a nonempty UID") + } + } + if len(claim.Status.CheckedNames) > 256 || len(claim.Status.RequiredViews) > 256 { + return fmt.Errorf("withdrawal checkedNames and requiredViews must not exceed 256 items") + } + if claim.Status.Provider != nil { + if len(rawExtensionBytes(claim.Status.Provider.State)) > 0 { + return fmt.Errorf("status.provider.state is private to the ZoneUnit") + } + if len(rawExtensionBytes(claim.Status.Provider.Data)) > 0 { + if claim.Status.ProviderRef == nil { + return fmt.Errorf("status.providerRef is required for provider data") + } + _, version, err := v.resolveProviderVersion(ctx, *claim.Status.ProviderRef, false) + if err != nil { + return err + } + if version.RecordWithdrawal == nil || !version.RecordWithdrawal.Supported { + return fmt.Errorf("Provider does not support record withdrawal") + } + return validateRawExtensionBySchema(claim.Status.Provider.Data, version.RecordWithdrawal.Schemas.StatusProviderData, "status.provider.data") + } + } + return nil +} + +func validateProviderWithdrawal(capability *dnsv1alpha1.ProviderRecordWithdrawal, path string) error { + if capability == nil { + return nil + } + if err := validateExtensionOpenAPISchema(capability.Schemas.StatusProviderData, path+".schemas.statusProviderData.openAPIV3Schema"); err != nil { + return err + } + core := withdrawalAcceptanceReasons() + core[dnsv1alpha1.ConditionWithdrawn] = nil + core[dnsv1alpha1.ConditionReady] = nil + core[dnsv1alpha1.ConditionOverrideAccepted] = nil + if err := validateProviderConditionReasons(capability.AdditionalConditionReasons, path+".additionalConditionReasons", core); err != nil { + return err + } + for _, reason := range capability.AdditionalConditionReasons { + if !hasConditionReason(withdrawalConditionReasons(), reason.ConditionType, reason.Status, reason.Reason) { + return fmt.Errorf("%s condition reason %s is not part of the withdrawal contract", path, reason.Reason) + } + } + return nil +} + +func validateUnitWithdrawals(unit *dnsv1alpha1.ZoneUnit, old *dnsv1alpha1.ZoneUnit) error { + if len(unit.Spec.Withdrawals) > 256 { + return fmt.Errorf("spec.withdrawals exceeds 256 items") + } + seen := map[string]bool{} + for i, item := range unit.Spec.Withdrawals { + key := recordSetClaimKey(item.ClaimNamespace, item.ClaimName) + if item.ClaimNamespace == "" || item.ClaimName == "" || item.Claim.UID == "" || item.ClaimNamespace != item.Claim.Namespace || item.ClaimName != item.Claim.Name || item.ObservedGeneration < 1 || seen[key] { + return fmt.Errorf("spec.withdrawals[%d] has invalid or duplicate claim identity", i) + } + seen[key] = true + claim := dnsv1alpha1.RecordWithdrawal{ObjectMeta: metav1.ObjectMeta{Labels: map[string]string{dnsv1alpha1.AddressLabelKey: addressLabel(item.Address)}}, Spec: dnsv1alpha1.RecordWithdrawalSpec{ZoneRef: dnsv1alpha1.ZoneReference{Name: unit.Spec.Zone.Ref.Name}, Name: item.Name, Address: item.Address}} + if err := declaration.ValidateWithdrawal(&claim); err != nil { + return fmt.Errorf("spec.withdrawals[%d]: %w", i, err) + } + if item.Waiver != nil && (item.Waiver.Name == "" || item.Waiver.UID == "" || !withdrawalDigestPattern.MatchString(item.Waiver.Digest) || item.Waiver.ReservationRef.UID == "") { + return fmt.Errorf("spec.withdrawals[%d].waiver must include exact immutable scope", i) + } + if old != nil { + for _, previous := range old.Spec.Withdrawals { + if previous.Claim == item.Claim && (previous.Name != item.Name || previous.Address != item.Address || previous.ObservedGeneration != item.ObservedGeneration) { + return fmt.Errorf("spec.withdrawals[%d] immutable claim shape changed", i) + } + } + } + } + return nil +} + +func (v *CoreValidator) validateUnitWithdrawalStatus(unit *dnsv1alpha1.ZoneUnit, version *dnsv1alpha1.ProviderVersion) error { + if len(unit.Status.Withdrawals) > 256 { + return fmt.Errorf("status.withdrawals exceeds 256 items") + } + seen := map[string]bool{} + for i, item := range unit.Status.Withdrawals { + path := fmt.Sprintf("status.withdrawals[%d]", i) + key := recordSetClaimKey(item.ClaimNamespace, item.ClaimName) + if item.ClaimNamespace == "" || item.ClaimName == "" || item.Claim.UID == "" || item.ClaimNamespace != item.Claim.Namespace || item.ClaimName != item.Claim.Name || item.ObservedGeneration < 1 || seen[key] { + return fmt.Errorf("%s has invalid or duplicate claim identity", path) + } + seen[key] = true + if err := validateWithdrawalConditions(item.Conditions, item.ObservedGeneration, path+".conditions"); err != nil { + return err + } + if item.Provider != nil && len(rawExtensionBytes(item.Provider.Data)) > 0 { + if version.RecordWithdrawal == nil || !version.RecordWithdrawal.Supported { + return fmt.Errorf("Provider does not support record withdrawal") + } + if err := validateRawExtensionBySchema(item.Provider.Data, version.RecordWithdrawal.Schemas.StatusProviderData, path+".provider.data"); err != nil { + return err + } + } + } + return nil +} + +func addressLabel(address string) string { + ip, _ := netip.ParseAddr(address) + return dnsv1alpha1.AddressLabelValue(ip) +} diff --git a/internal/go/core/webhook/withdrawals_test.go b/internal/go/core/webhook/withdrawals_test.go new file mode 100644 index 0000000..d92dfae --- /dev/null +++ b/internal/go/core/webhook/withdrawals_test.go @@ -0,0 +1,173 @@ +package webhook + +import ( + "context" + "encoding/json" + "net/netip" + "strings" + "testing" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + admissionv1 "k8s.io/api/admission/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +func validWithdrawal() *dnsv1alpha1.RecordWithdrawal { + return &dnsv1alpha1.RecordWithdrawal{TypeMeta: metav1.TypeMeta{APIVersion: dnsv1alpha1.SchemeGroupVersion.String(), Kind: "RecordWithdrawal"}, ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "retired", UID: "claim-uid", Generation: 1, Labels: map[string]string{dnsv1alpha1.AddressLabelKey: "192.0.2.1"}}, Spec: dnsv1alpha1.RecordWithdrawalSpec{ZoneRef: dnsv1alpha1.ZoneReference{Name: "example"}, Name: "*.apps", Address: "192.0.2.1"}} +} + +func withdrawalRequest(t *testing.T, obj runtime.Object, old runtime.Object, kind, subresource string) admission.Request { + t.Helper() + data, err := json.Marshal(obj) + if err != nil { + t.Fatal(err) + } + req := admission.Request{AdmissionRequest: admissionv1.AdmissionRequest{Operation: admissionv1.Create, Kind: metav1.GroupVersionKind{Group: dnsv1alpha1.GroupName, Version: "v1alpha1", Kind: kind}, Object: runtime.RawExtension{Raw: data}, SubResource: subresource}} + if old != nil { + data, err = json.Marshal(old) + if err != nil { + t.Fatal(err) + } + req.Operation = admissionv1.Update + req.OldObject.Raw = data + } + return req +} + +func TestWithdrawalAdmissionShapeAndImmutability(t *testing.T) { + validator := newTestValidator(t) + cases := []struct { + name string + mutate func(*dnsv1alpha1.RecordWithdrawal) + want string + }{ + {"valid", func(*dnsv1alpha1.RecordWithdrawal) {}, ""}, + {"label mismatch", func(c *dnsv1alpha1.RecordWithdrawal) { c.Labels[dnsv1alpha1.AddressLabelKey] = "192.0.2.2" }, "must match spec.address"}, + {"hostname address", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Address = "vip.example.com" }, "canonical"}, + {"noncanonical IPv6", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Address = "2001:0db8::1" }, "canonical"}, + {"mapped IPv6", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Address = "::ffff:192.0.2.1" }, "canonical"}, + {"scoped IPv6", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Address = "fe80::1%eth0" }, "canonical"}, + {"IPv6", func(c *dnsv1alpha1.RecordWithdrawal) { + c.Spec.Address = "2001:db8::1" + c.Labels[dnsv1alpha1.AddressLabelKey] = dnsv1alpha1.AddressLabelValue(netip.MustParseAddr(c.Spec.Address)) + }, ""}, + {"wildcard position", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Name = "app.*" }, "canonical relative"}, + {"noncanonical owner", func(c *dnsv1alpha1.RecordWithdrawal) { c.Spec.Name = "APP." }, "canonical relative"}, + {"finalizer", func(c *dnsv1alpha1.RecordWithdrawal) { c.Finalizers = []string{"test"} }, "ownerReferences or finalizers"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + c := validWithdrawal() + tc.mutate(c) + response := validator.Handle(context.Background(), withdrawalRequest(t, c, nil, "RecordWithdrawal", "")) + if tc.want == "" { + if !response.Allowed { + t.Fatal(response.Result.Message) + } + } else if response.Allowed || !strings.Contains(response.Result.Message, tc.want) { + t.Fatalf("response=%#v want %s", response, tc.want) + } + }) + } + old := validWithdrawal() + changed := old.DeepCopy() + changed.Spec.Name = "other" + response := validator.Handle(context.Background(), withdrawalRequest(t, changed, old, "RecordWithdrawal", "")) + if response.Allowed || response.Result.Message != "spec is immutable" { + t.Fatalf("response=%#v", response) + } +} + +func validWaiver() *dnsv1alpha1.RecordWithdrawalWaiver { + claim := validWithdrawal() + return &dnsv1alpha1.RecordWithdrawalWaiver{TypeMeta: metav1.TypeMeta{APIVersion: dnsv1alpha1.SchemeGroupVersion.String(), Kind: "RecordWithdrawalWaiver"}, ObjectMeta: metav1.ObjectMeta{Namespace: claim.Namespace, Name: "override", UID: "waiver-uid", Generation: 1}, Spec: dnsv1alpha1.RecordWithdrawalWaiverSpec{WithdrawalRef: dnsv1alpha1.WithdrawalObjectReference{Namespace: claim.Namespace, Name: claim.Name, UID: claim.UID}, ClaimSpecDigest: dnsv1alpha1.WithdrawalSpecDigest(claim.Spec), Address: claim.Spec.Address, ReservationRef: dnsv1alpha1.WithdrawalObjectReference{Namespace: "mgmt", Name: "reservation", UID: "reservation-uid"}, Reason: "operator accepted risk", TicketRef: "change-123", AcceptStaleDNSRisk: true}} +} + +func TestWithdrawalWaiverAdmission(t *testing.T) { + validator := newTestValidator(t) + for _, tc := range []struct { + name string + mutate func(*dnsv1alpha1.RecordWithdrawalWaiver) + want string + }{ + {"valid", func(*dnsv1alpha1.RecordWithdrawalWaiver) {}, ""}, + {"missing UID", func(w *dnsv1alpha1.RecordWithdrawalWaiver) { w.Spec.ReservationRef.UID = "" }, "exact namespace, name and UID"}, + {"cross namespace", func(w *dnsv1alpha1.RecordWithdrawalWaiver) { w.Spec.WithdrawalRef.Namespace = "tenant" }, "must match waiver namespace"}, + {"missing acceptance", func(w *dnsv1alpha1.RecordWithdrawalWaiver) { w.Spec.AcceptStaleDNSRisk = false }, "must be true"}, + {"invalid digest", func(w *dnsv1alpha1.RecordWithdrawalWaiver) { w.Spec.ClaimSpecDigest = "bad" }, "sha256 digest"}, + } { + t.Run(tc.name, func(t *testing.T) { + w := validWaiver() + tc.mutate(w) + response := validator.Handle(context.Background(), withdrawalRequest(t, w, nil, "RecordWithdrawalWaiver", "")) + if tc.want == "" { + if !response.Allowed { + t.Fatal(response.Result.Message) + } + } else if response.Allowed || !strings.Contains(response.Result.Message, tc.want) { + t.Fatalf("response=%#v", response) + } + }) + } + old := validWaiver() + changed := old.DeepCopy() + changed.Spec.ReservationRef.UID = "recreated" + response := validator.Handle(context.Background(), withdrawalRequest(t, changed, old, "RecordWithdrawalWaiver", "")) + if response.Allowed || response.Result.Message != "spec is immutable" { + t.Fatalf("response=%#v", response) + } +} + +func TestWithdrawalStatusReasonsAndGeneration(t *testing.T) { + validator := newTestValidator(t) + for condition, statuses := range withdrawalConditionReasons() { + for status, reasons := range statuses { + for reason := range reasons { + t.Run(string(condition)+"/"+reason, func(t *testing.T) { + c := validWithdrawal() + c.Status.ObservedGeneration = 1 + c.Status.Conditions = []metav1.Condition{{Type: string(condition), Status: status, Reason: reason, Message: "provider message", ObservedGeneration: 1}} + response := validator.Handle(context.Background(), withdrawalRequest(t, c, nil, "RecordWithdrawal", "status")) + if !response.Allowed { + t.Fatal(response.Result.Message) + } + }) + } + } + } + for _, tc := range []struct { + name string + generation int64 + reason, message string + }{{"future", 2, "WithdrawalVerified", "ok"}, {"invalid reason", 1, "TimerElapsed", "ok"}, {"oversized message", 1, "WithdrawalVerified", strings.Repeat("x", 2049)}} { + t.Run(tc.name, func(t *testing.T) { + c := validWithdrawal() + c.Status.Conditions = []metav1.Condition{{Type: "Withdrawn", Status: metav1.ConditionTrue, Reason: tc.reason, Message: tc.message, ObservedGeneration: tc.generation}} + if response := validator.Handle(context.Background(), withdrawalRequest(t, c, nil, "RecordWithdrawal", "status")); response.Allowed { + t.Fatal("invalid status admitted") + } + }) + } +} + +func TestWithdrawalBlocksUnitDeletionEvenWhenZoneDeleting(t *testing.T) { + claim := validWithdrawal() + claim.Spec.ZoneRef.Name = "example-com" + unit := validZoneUnit() + claim.Namespace = unit.Spec.Zone.Ref.Namespace + claim.Spec.ZoneRef.Name = unit.Spec.Zone.Ref.Name + validator := newTestValidator(t, claim) + if err := validator.validateZoneUnitDelete(context.Background(), unit); err == nil || !strings.Contains(err.Error(), "RecordWithdrawal") { + t.Fatalf("delete error=%v", err) + } +} + +func TestWithdrawalProviderReasonsRemainResourceScoped(t *testing.T) { + provider := route53Provider(t) + provider.Spec.Versions[0].Zone.AdditionalConditionReasons = []dnsv1alpha1.ProviderConditionReason{{ConditionType: dnsv1alpha1.ConditionReady, Status: metav1.ConditionTrue, Reason: "WithdrawalVerified", Description: "Wrong resource."}} + if err := validateProvider(provider); err == nil || !strings.Contains(err.Error(), "conditionType") { + t.Fatalf("withdrawal condition leaked into Zone declarations: %v", err) + } +} diff --git a/internal/go/providers/external/manifest/manifest_test.go b/internal/go/providers/external/manifest/manifest_test.go index 5c77755..9d02403 100644 --- a/internal/go/providers/external/manifest/manifest_test.go +++ b/internal/go/providers/external/manifest/manifest_test.go @@ -23,6 +23,9 @@ func TestExternalDeclarationAndCEL(t *testing.T) { if len(version.Zone.AdditionalConditionReasons) != 5 || len(version.RecordSet.AdditionalConditionReasons) != 8 { t.Fatal("condition reason tables differ from design") } + if version.RecordWithdrawal == nil || !version.RecordWithdrawal.Supported || version.RecordWithdrawal.Schemas.StatusProviderData == nil || len(version.RecordWithdrawal.AdditionalConditionReasons) != 11 { + t.Fatal("withdrawal capability, schema and condition table must ship together") + } for _, rule := range version.ZoneClass.ValidationRules { if err := celruntime.CompileBool(rule.Rule, cel.Variable("self", cel.DynType)); err != nil { t.Fatalf("compile %q: %v", rule.Rule, err) diff --git a/internal/go/providers/external/manifest/provider.yaml b/internal/go/providers/external/manifest/provider.yaml index f24735f..9a7425e 100644 --- a/internal/go/providers/external/manifest/provider.yaml +++ b/internal/go/providers/external/manifest/provider.yaml @@ -160,3 +160,25 @@ spec: - {conditionType: Programmed, status: 'False', reason: ResolutionInconsistent, description: Required resolver or authority observations disagree.} - {conditionType: Programmed, status: 'False', reason: NameserverMismatch, description: Required zone nameserver verification failed.} - {conditionType: Programmed, status: Unknown, reason: DNSProbeFailed, description: A required DNS observation could not complete.} + recordWithdrawal: + supported: true + schemas: + statusProviderData: + openAPIV3Schema: + type: object + description: Bounded withdrawal diagnostics; complete typed evidence drives verification. + additionalProperties: false + properties: + views: *preview + additionalConditionReasons: + - {conditionType: Withdrawn, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Ready, status: 'False', reason: DNSStillResolves, description: A required observation still returns the retired address.} + - {conditionType: Withdrawn, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Ready, status: 'False', reason: NegativeTTLNotElapsed, description: Cache safety and a separate confirming attempt are pending.} + - {conditionType: Withdrawn, status: 'Unknown', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Ready, status: 'False', reason: DNSProbeFailed, description: Required exclusion evidence is incomplete.} + - {conditionType: Withdrawn, status: 'Unknown', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Ready, status: 'False', reason: PolicyChanged, description: The binding or required view policy changed.} + - {conditionType: Withdrawn, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: Ready, status: 'True', reason: WithdrawalVerified, description: Complete exclusion verified after the cache horizon.} + - {conditionType: OverrideAccepted, status: 'True', reason: OperatorAuthorizedRelease, description: An exact separately authorized waiver accepts stale DNS risk.} diff --git a/pkg/go/api/dns/v1alpha1/recordwithdrawal_test.go b/pkg/go/api/dns/v1alpha1/recordwithdrawal_test.go new file mode 100644 index 0000000..29bad01 --- /dev/null +++ b/pkg/go/api/dns/v1alpha1/recordwithdrawal_test.go @@ -0,0 +1,65 @@ +package v1alpha1_test + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "net/netip" + "testing" + + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "k8s.io/apimachinery/pkg/runtime" +) + +func TestWithdrawalAddressLabelsAndDigests(t *testing.T) { + for _, tc := range []struct{ address, want string }{ + {"192.0.2.1", "192.0.2.1"}, + {"::ffff:192.0.2.1", "192.0.2.1"}, + {"::1", "aaaaaaaaaaaaaaaaaaaaaaaaae"}, + {"2001:db8::1", "eaaq3oaaaaaaaaaaaaaaaaaaae"}, + {"fe80::1%eth0", ""}, + } { + t.Run(tc.address, func(t *testing.T) { + if got := dnsv1alpha1.AddressLabelValue(netip.MustParseAddr(tc.address)); got != tc.want { + t.Fatalf("label=%s want=%s", got, tc.want) + } + }) + } + if got := dnsv1alpha1.AddressLabelValue(netip.Addr{}); got != "" { + t.Fatalf("invalid label=%s", got) + } + spec := dnsv1alpha1.RecordWithdrawalSpec{ZoneRef: dnsv1alpha1.ZoneReference{Name: "example"}, Name: "*.apps", Address: "192.0.2.1"} + data, err := json.Marshal(spec) + if err != nil { + t.Fatal(err) + } + digest := sha256.Sum256(data) + if got := dnsv1alpha1.WithdrawalSpecDigest(spec); got != "sha256:"+hex.EncodeToString(digest[:]) { + t.Fatal("digest does not bind exact serialized spec") + } + before := dnsv1alpha1.WithdrawalSpecDigest(spec) + namespace := "app" + spec.ZoneRef.Namespace = &namespace + if before == dnsv1alpha1.WithdrawalSpecDigest(spec) { + t.Fatal("digest ignored explicit reference namespace") + } +} + +func TestWithdrawalKindsRegisteredAndDeepCopy(t *testing.T) { + scheme := runtime.NewScheme() + if err := dnsv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + for _, kind := range []string{"RecordWithdrawal", "RecordWithdrawalList", "RecordWithdrawalWaiver", "RecordWithdrawalWaiverList"} { + if _, err := scheme.New(dnsv1alpha1.SchemeGroupVersion.WithKind(kind)); err != nil { + t.Fatal(err) + } + } + claim := &dnsv1alpha1.RecordWithdrawal{Status: dnsv1alpha1.RecordWithdrawalStatus{RequiredViews: []string{"first"}, WaiverRef: &dnsv1alpha1.RecordWithdrawalWaiverReference{Name: "waiver"}}} + copied := claim.DeepCopy() + copied.Status.RequiredViews[0] = "changed" + copied.Status.WaiverRef.Name = "changed" + if claim.Status.RequiredViews[0] != "first" || claim.Status.WaiverRef.Name != "waiver" { + t.Fatal("deepcopy aliased public evidence") + } +} From 3068d634e994ea225864acb4935ccb7684e825f8 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:28:16 +0900 Subject: [PATCH 13/16] feat(external): verify DNS withdrawal before retired address reuse --- .../controllers/zoneunit/controller.go | 37 +- .../external/controllers/zoneunit/model.go | 23 +- .../external/controllers/zoneunit/status.go | 14 +- .../external/controllers/zoneunit/watches.go | 5 + .../controllers/zoneunit/withdrawal.go | 318 ++++++++++++ .../controllers/zoneunit/withdrawal_model.go | 257 ++++++++++ .../controllers/zoneunit/withdrawal_test.go | 464 ++++++++++++++++++ .../go/providers/external/probe/metrics.go | 41 +- internal/go/providers/external/probe/types.go | 5 + internal/go/providers/external/probe/wire.go | 5 +- .../go/providers/external/probe/withdrawal.go | 325 ++++++++++++ .../external/probe/withdrawal_test.go | 190 +++++++ 12 files changed, 1664 insertions(+), 20 deletions(-) create mode 100644 internal/go/providers/external/controllers/zoneunit/withdrawal.go create mode 100644 internal/go/providers/external/controllers/zoneunit/withdrawal_model.go create mode 100644 internal/go/providers/external/controllers/zoneunit/withdrawal_test.go create mode 100644 internal/go/providers/external/probe/withdrawal.go create mode 100644 internal/go/providers/external/probe/withdrawal_test.go diff --git a/internal/go/providers/external/controllers/zoneunit/controller.go b/internal/go/providers/external/controllers/zoneunit/controller.go index 2672fee..b3bfc9b 100644 --- a/internal/go/providers/external/controllers/zoneunit/controller.go +++ b/internal/go/providers/external/controllers/zoneunit/controller.go @@ -135,6 +135,9 @@ func (r *ZoneReconciler) observe(ctx context.Context, c client.Client, unit *dns for _, item := range unit.Spec.RecordSets { namespaceNames = append(namespaceNames, item.RecordSetNamespace) } + for _, item := range unit.Spec.Withdrawals { + namespaceNames = append(namespaceNames, item.ClaimNamespace) + } slices.Sort(namespaceNames) namespaceNames = slices.Compact(namespaceNames) namespacePolicies := make(map[string]map[string]string, len(namespaceNames)) @@ -163,6 +166,7 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c if err := r.Get(ctx, request.NamespacedName, &unit); err != nil { if apierrors.IsNotFound(err) { probe.SetPending(request.NamespacedName.String(), nil) + probe.SetWithdrawalsPending(request.NamespacedName.String(), nil) } return ctrl.Result{}, client.IgnoreNotFound(err) } @@ -184,7 +188,10 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c unit.Status.RecordSets[i].DeletionCompleted = unit.Spec.RecordSets[i].RecordSetUID != "" } probe.SetPending(request.NamespacedName.String(), nil) - return ctrl.Result{}, r.patch(ctx, base, &unit) + if len(unit.Spec.Withdrawals) == 0 { + probe.SetWithdrawalsPending(request.NamespacedName.String(), nil) + return ctrl.Result{}, r.patch(ctx, base, &unit) + } } if rejected != nil { r.reject(&unit, *rejected) @@ -193,6 +200,7 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c now := r.Options.Clock() state := loadState(&unit, env.fingerprint) groups := groupNames(&unit, env.fingerprint) + withdrawalsChanged := prepareWithdrawals(&unit, &state, env, now) active := make(map[string]bool, len(groups)) for _, group := range groups { active[group.Name] = true @@ -222,11 +230,17 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c } } due := selectDue(groups, state, now) + withdrawalDue := selectWithdrawalDue(&unit, state, env.fingerprint, now, MaxNamesPerPass) + if len(due) != 0 && len(withdrawalDue) != 0 { + // Reserve half of a busy pass for each class of work; both have fair cursors. + withdrawalDue = selectWithdrawalDue(&unit, state, env.fingerprint, now, MaxNamesPerPass/2) + due = due[:min(len(due), MaxNamesPerPass-withdrawalProbeCost(&unit, state, withdrawalDue))] + } zoneCondition := meta.FindStatusCondition(unit.Status.Zone.Conditions, "Programmed") zoneDue := state.Zone.Fingerprint != env.fingerprint || !state.Zone.Next.After(now) || zoneCondition == nil || zoneCondition.ObservedGeneration != unit.Spec.Zone.ObservedGeneration var evidence *probe.ZoneEvidence zoneDecision := currentDecision(unit.Status.Zone.Conditions) - zoneProbed := zoneDue || (len(due) != 0 && zoneDecision.Status == metav1.ConditionTrue) + zoneProbed := zoneDue || ((len(due) != 0 || len(withdrawalDue) != 0) && zoneDecision.Status == metav1.ConditionTrue) if zoneProbed { zoneRequest := probe.ProbeRequest{Zone: unit.Spec.Zone.DomainName, Name: unit.Spec.Zone.DomainName, Policy: env.policy, ZoneOnly: true} switch { @@ -267,6 +281,7 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c probeRequest := probe.ProbeRequest{Name: group.CheckedName, Zone: unit.Spec.Zone.DomainName, Expected: group.Expected, Policy: env.policy, Evidence: evidence, DueTime: dueTime} observed, probeErr := r.Options.Resolver.Probe(ctx, probeRequest) decision = probe.Evaluate(probeRequest, observed) + retainPublication(&state, group, probeRequest, observed, r.Options.Clock()) if probeErr != nil { decision = probe.Failed(group.CheckedName, "required-views", failureCode(probeErr)) } @@ -292,13 +307,24 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c } } } + r.probeWithdrawals(ctx, &unit, &state, env, evidence, zoneDecision, withdrawalDue) + if zoneProbed && zoneDecision.Status != metav1.ConditionTrue { + invalidateWithdrawalZone(&unit, &state, r.Options.Clock()) + withdrawalsChanged = true + } + for name, publication := range state.Publications { + if !active[name] && !publicationUnexpired(publication, r.Options.Clock()) { + delete(state.Publications, name) + prunedSchedule = true + } + } // DNS I/O is an asynchronous observation boundary. Live inputs must still agree. reader := readerClient{Client: r.Client, reader: r.Reader} var current dnsv1alpha1.ZoneUnit if err := reader.Get(ctx, request.NamespacedName, ¤t); err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } - if current.UID != unit.UID || current.Generation != unit.Generation || !equality.Semantic.DeepEqual(current.Spec, unit.Spec) || !current.DeletionTimestamp.IsZero() { + if current.UID != unit.UID || current.Generation != unit.Generation || !equality.Semantic.DeepEqual(current.Spec, unit.Spec) || !current.DeletionTimestamp.Equal(unit.DeletionTimestamp) { return ctrl.Result{RequeueAfter: time.Second}, nil } fresh, blocked, err := r.observe(ctx, reader, ¤t) @@ -311,7 +337,7 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c // Preserve the stored RawExtension unless scheduling actually changed. // Re-encoding unchanged state can reorder API-normalized JSON keys and // make the status equality guard issue an otherwise empty patch. - if zoneProbed || len(due) != 0 || prunedSchedule { + if zoneProbed || len(due) != 0 || len(withdrawalDue) != 0 || prunedSchedule || withdrawalsChanged { stateBytes, err := json.Marshal(state) if err != nil { return ctrl.Result{}, err @@ -320,7 +346,8 @@ func (r *ZoneReconciler) Reconcile(ctx context.Context, request ctrl.Request) (c } setAggregate(&unit, r.Options.Clock()) r.updatePending(&unit) - return ctrl.Result{RequeueAfter: nextRequeue(groups, state, r.Options.Clock())}, r.patch(ctx, base, &unit) + next := min(nextRequeue(groups, state, r.Options.Clock()), nextWithdrawalRequeue(&unit, state, env.fingerprint, r.Options.Clock())) + return ctrl.Result{RequeueAfter: next}, r.patch(ctx, base, &unit) } func validateGroup(unit *dnsv1alpha1.ZoneUnit, group nameGroup) (bool, string, string) { diff --git a/internal/go/providers/external/controllers/zoneunit/model.go b/internal/go/providers/external/controllers/zoneunit/model.go index a0a81eb..12672e0 100644 --- a/internal/go/providers/external/controllers/zoneunit/model.go +++ b/internal/go/providers/external/controllers/zoneunit/model.go @@ -20,10 +20,13 @@ type schedule struct { } type schedulerState struct { - Policy string `json:"policy"` - Cursor string `json:"cursor,omitempty"` - Zone schedule `json:"zone"` - Names map[string]schedule `json:"names"` + Policy string `json:"policy"` + Cursor string `json:"cursor,omitempty"` + Zone schedule `json:"zone"` + Names map[string]schedule `json:"names"` + Withdrawals map[string]withdrawalState `json:"withdrawals,omitempty"` + WithdrawalCursor string `json:"withdrawalCursor,omitempty"` + Publications map[string]publicationEvidence `json:"publications,omitempty"` } type nameGroup struct { @@ -38,7 +41,7 @@ type nameGroup struct { func groupNames(unit *dnsv1alpha1.ZoneUnit, policy string) []nameGroup { byName := map[string][]int{} for i, item := range unit.Spec.RecordSets { - if item.IsAllowed() && !item.DeletionRequested && item.RecordSetUID != "" { + if unit.DeletionTimestamp.IsZero() && item.IsAllowed() && !item.DeletionRequested && item.RecordSetUID != "" { byName[item.Name] = append(byName[item.Name], i) } } @@ -105,11 +108,19 @@ func loadState(unit *dnsv1alpha1.ZoneUnit, policy string) schedulerState { _ = json.Unmarshal(unit.Status.Provider.State.Raw, &state) } if state.Policy != policy { - state = schedulerState{Policy: policy} + state.Policy = policy + state.Zone = schedule{} + state.Names = nil } if state.Names == nil { state.Names = map[string]schedule{} } + if state.Withdrawals == nil { + state.Withdrawals = map[string]withdrawalState{} + } + if state.Publications == nil { + state.Publications = map[string]publicationEvidence{} + } return state } diff --git a/internal/go/providers/external/controllers/zoneunit/status.go b/internal/go/providers/external/controllers/zoneunit/status.go index 8c0de7c..21f129e 100644 --- a/internal/go/providers/external/controllers/zoneunit/status.go +++ b/internal/go/providers/external/controllers/zoneunit/status.go @@ -124,7 +124,6 @@ func setAggregate(unit *dnsv1alpha1.ZoneUnit, now time.Time) { func (r *ZoneReconciler) reject(unit *dnsv1alpha1.ZoneUnit, rejected acceptance) { now := r.Options.Clock() - unit.Status.Provider = nil unit.Status.Zone.Provider = nil unit.Status.Zone.NameServers = nil setCondition(&unit.Status.Zone.Conditions, "Accepted", rejected.Status, rejected.Reason, rejected.Message, unit.Spec.Zone.ObservedGeneration, now) @@ -136,6 +135,7 @@ func (r *ZoneReconciler) reject(unit *dnsv1alpha1.ZoneUnit, rejected acceptance) setCondition(&unit.Status.RecordSets[i].Conditions, "Accepted", metav1.ConditionFalse, "ZoneNotAccepted", rejected.Message, unit.Status.RecordSets[i].ObservedGeneration, now) resetProgrammed(&unit.Status.RecordSets[i], now) } + rejectWithdrawals(unit, now) setAggregate(unit, now) r.updatePending(unit) } @@ -153,4 +153,16 @@ func (r *ZoneReconciler) updatePending(unit *dnsv1alpha1.ZoneUnit) { } } probe.SetPending(unit.Namespace+"/"+unit.Name, counts) + withdrawals := map[string]int{} + for _, status := range unit.Status.Withdrawals { + condition := meta.FindStatusCondition(status.Conditions, "Withdrawn") + if accepted := meta.FindStatusCondition(status.Conditions, "Accepted"); accepted != nil && accepted.Status != metav1.ConditionTrue { + withdrawals[accepted.Reason]++ + } else if condition == nil { + withdrawals["DNSProbeFailed"]++ + } else if condition.Status != metav1.ConditionTrue { + withdrawals[condition.Reason]++ + } + } + probe.SetWithdrawalsPending(unit.Namespace+"/"+unit.Name, withdrawals) } diff --git a/internal/go/providers/external/controllers/zoneunit/watches.go b/internal/go/providers/external/controllers/zoneunit/watches.go index 288941b..e856ab2 100644 --- a/internal/go/providers/external/controllers/zoneunit/watches.go +++ b/internal/go/providers/external/controllers/zoneunit/watches.go @@ -41,6 +41,11 @@ func (r *ZoneReconciler) SetupWithManager(mgr ctrl.Manager) error { return true } } + for _, item := range unit.Spec.Withdrawals { + if item.ClaimNamespace == obj.GetName() { + return true + } + } return false }) }), builder.WithPredicates(predicate.LabelChangedPredicate{})). diff --git a/internal/go/providers/external/controllers/zoneunit/withdrawal.go b/internal/go/providers/external/controllers/zoneunit/withdrawal.go new file mode 100644 index 0000000..2fbff2a --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/withdrawal.go @@ -0,0 +1,318 @@ +package zoneunit + +import ( + "context" + "encoding/json" + "fmt" + "reflect" + "slices" + "strings" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +func prepareWithdrawals(unit *dnsv1alpha1.ZoneUnit, state *schedulerState, env *environment, now time.Time) bool { + old := make(map[string]dnsv1alpha1.ZoneUnitWithdrawalStatus, len(unit.Status.Withdrawals)) + for _, status := range unit.Status.Withdrawals { + old[status.ClaimNamespace+"/"+status.ClaimName] = status + } + var statuses []dnsv1alpha1.ZoneUnitWithdrawalStatus + active := make(map[string]bool, len(unit.Spec.Withdrawals)) + changed := false + for _, item := range unit.Spec.Withdrawals { + key := withdrawalKey(item) + active[key] = true + status := old[key] + if status.Claim != item.Claim || status.ObservedGeneration != item.ObservedGeneration { + status = dnsv1alpha1.ZoneUnitWithdrawalStatus{} + } + status.ClaimNamespace, status.ClaimName, status.Claim, status.ObservedGeneration = item.ClaimNamespace, item.ClaimName, item.Claim, item.ObservedGeneration + stored, exists := state.Withdrawals[key] + identity := withdrawalIdentity(unit, item) + if !exists || stored.Identity != identity { + publication := state.Publications[item.Name] + stored = withdrawalState{Identity: identity, CheckedNames: withdrawalNames(unit, item, publication), MissingHistory: status.LastProbeTime != nil} + if positive := publication.Positive[item.Address]; len(positive) != 0 { + stored.Positive = make(map[string]time.Time, len(positive)) + for key, expires := range positive { + stored.Positive[key] = expires + } + } + invalidateWithdrawal(&status, &stored) + changed = true + } + status.CheckedNames = slices.Clone(stored.CheckedNames) + status.SampledWildcard = item.Name == "*" || strings.HasPrefix(item.Name, "*.") + decision := probe.WithdrawalFailed(item.Name, "required-views", "invalid-response").Decision + switch { + case !item.Allowed || !validWithdrawal(unit, item): + invalidateWithdrawal(&status, &stored) + reason := "InvalidWithdrawal" + if !item.Allowed { + reason = "NotAllowed" + } + setCondition(&status.Conditions, "Accepted", metav1.ConditionFalse, reason, "External DNS withdrawal intent is not accepted.", item.ObservedGeneration, now) + setWithdrawalConditions(&status, decision, now) + status.WaiverRef = nil + meta.RemoveStatusCondition(&status.Conditions, "OverrideAccepted") + default: + if env != nil { + setCondition(&status.Conditions, "Accepted", metav1.ConditionTrue, "Accepted", "External DNS withdrawal intent is accepted.", item.ObservedGeneration, now) + } + if meta.FindStatusCondition(status.Conditions, "Withdrawn") == nil { + setWithdrawalConditions(&status, decision, now) + } + if env != nil && stored.Policy != "" && stored.Policy != env.fingerprint { + invalidateWithdrawal(&status, &stored) + setWithdrawalConditions(&status, policyChanged(item.Name), now) + changed = true + } + if status.LastProbeTime != nil && (now.Before(status.LastProbeTime.Time) || status.NextProbeTime == nil || now.After(status.NextProbeTime.Add(300*time.Second))) { + invalidateWithdrawal(&status, &stored) + stored.Schedule.Next = time.Time{} + setWithdrawalConditions(&status, decision, now) + changed = true + } + bindWithdrawalWaiver(&status, item, now) + if status.ResultDigest != "" { + status.ResultDigest = withdrawalResultDigest(unit, item, status) + } + } + state.Withdrawals[key] = stored + statuses = append(statuses, status) + } + for key := range state.Withdrawals { + if !active[key] { + delete(state.Withdrawals, key) + changed = true + } + } + unit.Status.Withdrawals = statuses + return changed +} + +func selectWithdrawalDue(unit *dnsv1alpha1.ZoneUnit, state schedulerState, policy string, now time.Time, budget int) []int { + indices := make([]int, 0, len(unit.Spec.Withdrawals)) + for i, item := range unit.Spec.Withdrawals { + if item.Allowed && validWithdrawal(unit, item) { + indices = append(indices, i) + } + } + slices.SortFunc(indices, func(a, b int) int { + return strings.Compare(withdrawalKey(unit.Spec.Withdrawals[a]), withdrawalKey(unit.Spec.Withdrawals[b])) + }) + start := 0 + for start < len(indices) && withdrawalKey(unit.Spec.Withdrawals[indices[start]]) <= state.WithdrawalCursor { + start++ + } + var due []int + for offset := range len(indices) { + index := indices[(start+offset)%len(indices)] + stored := state.Withdrawals[withdrawalKey(unit.Spec.Withdrawals[index])] + if stored.Policy == policy && stored.Schedule.Next.After(now) { + continue + } + cost := len(stored.CheckedNames) + if cost > budget { + break + } + budget -= cost + due = append(due, index) + } + return due +} + +func withdrawalProbeCost(unit *dnsv1alpha1.ZoneUnit, state schedulerState, indices []int) int { + cost := 0 + for _, i := range indices { + cost += len(state.Withdrawals[withdrawalKey(unit.Spec.Withdrawals[i])].CheckedNames) + } + return cost +} + +func (r *ZoneReconciler) probeWithdrawals(ctx context.Context, unit *dnsv1alpha1.ZoneUnit, state *schedulerState, env *environment, evidence *probe.ZoneEvidence, zoneDecision probe.Decision, due []int) { + for _, index := range due { + item := unit.Spec.Withdrawals[index] + key := withdrawalKey(item) + stored := state.Withdrawals[key] + status := &unit.Status.Withdrawals[index] + started := r.Options.Clock() + views := requiredWithdrawalViews(env.policy, evidence) + decision := probe.WithdrawalDecision{Decision: probe.Decision{Status: metav1.ConditionTrue, Reason: "WithdrawalVerified"}} + if zoneDecision.Status != metav1.ConditionTrue { + decision = probe.WithdrawalFailed(item.Name, "required-authorities", "invalid-response") + if env.policyError != nil { + decision = probe.WithdrawalFailed(item.Name, "configured-resolvers", "resolver-config") + } + } else { + for _, name := range stored.CheckedNames { + request := probe.ProbeRequest{Name: name, Zone: unit.Spec.Zone.DomainName, Policy: env.policy, Evidence: evidence, DueTime: stored.Schedule.Next, WithdrawalAddress: item.Address} + observed, err := r.Options.Resolver.Probe(ctx, request) + one := probe.EvaluateWithdrawal(request, observed) + retainPositive(&stored, name, one, r.Options.Clock()) + if err != nil { + one = probe.WithdrawalFailed(name, "required-views", failureCode(err)) + } + decision.Cache = append(decision.Cache, one.Cache...) + if one.Status == metav1.ConditionUnknown || (one.Status == metav1.ConditionFalse && decision.Status == metav1.ConditionTrue) { + decision.Decision = one.Decision + } + } + } + now := r.Options.Clock() + if now.Before(started) || now.Sub(started) > 300*time.Second { + decision.Decision = probe.WithdrawalFailed(item.Name, "required-views", "timeout").Decision + } + if status.LastProbeTime != nil && (now.Before(status.LastProbeTime.Time) || status.NextProbeTime == nil || now.After(status.NextProbeTime.Add(300*time.Second))) { + invalidateWithdrawal(status, &stored) + } + fingerprint := probe.Fingerprint(struct { + Policy string + Views []string + }{env.fingerprint, views}) + changed := stored.Policy != "" && (stored.Policy != env.fingerprint || !slices.Equal(status.RequiredViews, views)) + weakened := policyWeakened(stored, env.policy, views) + if changed || weakened { + invalidateWithdrawal(status, &stored) + decision.Decision = policyChanged(item.Name) + } + // A failed authority discovery cannot lower the persisted policy floor. + if zoneDecision.Status == metav1.ConditionTrue && !weakened { + addFloor(&stored, env.policy, views) + } + stored.Policy = env.fingerprint + status.PolicyFingerprint = fingerprint + status.RequiredViews = views + result := withdrawalWindow(status, &stored, decision, now) + if result.Reason == "NegativeTTLNotElapsed" { + result.Message = fmt.Sprintf("Retired address %s is absent in every required view; waiting until %s for cache safety and a confirming probe.", item.Address, status.NotBefore.UTC().Format(time.RFC3339)) + } + if result.Reason == "WithdrawalVerified" { + result.Message = fmt.Sprintf("Retired address %s was verified absent from %s in every required view at %s.", item.Address, item.Name, now.UTC().Format(time.RFC3339)) + } + stored.Schedule = advance(stored.Schedule, stored.Identity+fingerprint, result, now, r.Options.Jitter()) + status.LastProbeTime, status.NextProbeTime = timePointer(now), timePointer(stored.Schedule.Next) + setWithdrawalConditions(status, result, now) + data, _ := json.Marshal(struct { + Views externalv1alpha1.VerificationValues `json:"views"` + }{Views: probe.Preview(views)}) + status.Provider = &dnsv1alpha1.ProviderStatus{Data: runtime.RawExtension{Raw: data}} + bindWithdrawalWaiver(status, item, now) + status.ResultDigest = withdrawalResultDigest(unit, item, *status) + state.Withdrawals[key] = stored + state.WithdrawalCursor = key + } +} + +func policyChanged(name string) probe.Decision { + return probe.Decision{Status: metav1.ConditionUnknown, Reason: "PolicyChanged", Message: fmt.Sprintf("External DNS withdrawal policy changed for %s; the prior required view policy remains a floor.", name)} +} + +func setWithdrawalConditions(status *dnsv1alpha1.ZoneUnitWithdrawalStatus, decision probe.Decision, now time.Time) { + setCondition(&status.Conditions, "Withdrawn", decision.Status, decision.Reason, decision.Message, status.ObservedGeneration, now) + ready := metav1.ConditionFalse + if decision.Status == metav1.ConditionTrue { + ready = metav1.ConditionTrue + } + setCondition(&status.Conditions, "Ready", ready, decision.Reason, decision.Message, status.ObservedGeneration, now) +} + +func bindWithdrawalWaiver(status *dnsv1alpha1.ZoneUnitWithdrawalStatus, item dnsv1alpha1.ZoneUnitWithdrawal, now time.Time) { + status.WaiverRef = nil + waiver := item.Waiver + if waiver == nil || waiver.Name == "" || waiver.UID == "" || waiver.Digest == "" || waiver.ReservationRef.UID == "" { + meta.RemoveStatusCondition(&status.Conditions, "OverrideAccepted") + return + } + status.WaiverRef = &dnsv1alpha1.RecordWithdrawalWaiverReference{Name: waiver.Name, UID: waiver.UID, Digest: waiver.Digest, ReservationRef: waiver.ReservationRef} + setCondition(&status.Conditions, "OverrideAccepted", metav1.ConditionTrue, "OperatorAuthorizedRelease", "Core-validated withdrawal waiver is bound to its exact UID and immutable scope.", item.ObservedGeneration, now) +} + +func withdrawalResultDigest(unit *dnsv1alpha1.ZoneUnit, item dnsv1alpha1.ZoneUnitWithdrawal, status dnsv1alpha1.ZoneUnitWithdrawalStatus) string { + status.ResultDigest = "" + status.Provider = nil // Bounded diagnostic previews are never decision inputs. + // Transition timestamps do not alter the observation/result identity. + status.Conditions = slices.Clone(status.Conditions) + for i := range status.Conditions { + status.Conditions[i].LastTransitionTime = metav1.Time{} + } + slices.SortFunc(status.Conditions, func(a, b metav1.Condition) int { return strings.Compare(a.Type, b.Type) }) + return "sha256:" + probe.Fingerprint(struct { + Identity string + Provider dnsv1alpha1.ProviderReference + Zone dnsv1alpha1.ZoneUnitZoneSpec + Status dnsv1alpha1.ZoneUnitWithdrawalStatus + }{withdrawalIdentity(unit, item), unit.Spec.Provider, unit.Spec.Zone, status}) +} + +func nextWithdrawalRequeue(unit *dnsv1alpha1.ZoneUnit, state schedulerState, policy string, now time.Time) time.Duration { + next := 300 * time.Second + for _, item := range unit.Spec.Withdrawals { + if !item.Allowed || !validWithdrawal(unit, item) { + continue + } + stored := state.Withdrawals[withdrawalKey(item)] + if stored.Policy != policy || !stored.Schedule.Next.After(now) { + return time.Second + } + next = min(next, stored.Schedule.Next.Sub(now)) + } + return max(time.Second, next) +} + +func invalidateWithdrawalZone(unit *dnsv1alpha1.ZoneUnit, state *schedulerState, now time.Time) { + for i, item := range unit.Spec.Withdrawals { + if !item.Allowed { + continue + } + status := &unit.Status.Withdrawals[i] + stored := state.Withdrawals[withdrawalKey(item)] + invalidateWithdrawal(status, &stored) + if condition := meta.FindStatusCondition(status.Conditions, "Withdrawn"); condition == nil || condition.Reason != "PolicyChanged" { + setWithdrawalConditions(status, probe.WithdrawalFailed(item.Name, "required-authorities", "invalid-response").Decision, now) + } + state.Withdrawals[withdrawalKey(item)] = stored + } +} + +func rejectWithdrawals(unit *dnsv1alpha1.ZoneUnit, now time.Time) { + var prior schedulerState + if unit.Status.Provider != nil { + _ = json.Unmarshal(unit.Status.Provider.State.Raw, &prior) + } + state := loadState(unit, prior.Policy) + // Invalidate reconstructible active schedules, but never discard retained + // publication horizons or a withdrawal's prior policy floor on rejection. + state.Zone = schedule{} + state.Names = map[string]schedule{} + prepareWithdrawals(unit, &state, nil, now) + for i, item := range unit.Spec.Withdrawals { + status := &unit.Status.Withdrawals[i] + stored := state.Withdrawals[withdrawalKey(item)] + invalidateWithdrawal(status, &stored) + stored.Schedule.Next = time.Time{} + state.Withdrawals[withdrawalKey(item)] = stored + setWithdrawalConditions(status, probe.WithdrawalFailed(item.Name, "binding", "invalid-response").Decision, now) + setCondition(&status.Conditions, "Accepted", metav1.ConditionFalse, "InvalidWithdrawal", "External DNS withdrawal binding is not accepted.", item.ObservedGeneration, now) + status.WaiverRef = nil + meta.RemoveStatusCondition(&status.Conditions, "OverrideAccepted") + } + // These maps are omitted from private JSON when empty. Normalize them before + // comparing the decoded state, which contains native (not metav1) times. + if len(state.Publications) == 0 { + state.Publications = nil + } + if len(state.Withdrawals) == 0 { + state.Withdrawals = nil + } + if (unit.Status.Provider != nil || len(unit.Spec.Withdrawals) != 0) && !reflect.DeepEqual(prior, state) { + raw, _ := json.Marshal(state) + unit.Status.Provider = &dnsv1alpha1.ProviderStatus{State: runtime.RawExtension{Raw: raw}} + } +} diff --git a/internal/go/providers/external/controllers/zoneunit/withdrawal_model.go b/internal/go/providers/external/controllers/zoneunit/withdrawal_model.go new file mode 100644 index 0000000..197ee0d --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/withdrawal_model.go @@ -0,0 +1,257 @@ +package zoneunit + +import ( + "net/netip" + "slices" + "strings" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/config" + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// The independent claims are the obligation inventory. This private state only +// retains cache bounds, policy floors and witnesses for those current claims. +type withdrawalState struct { + Identity string `json:"identity"` + Policy string `json:"policy"` + Floor externalv1alpha1.ExternalZoneClassParameters `json:"floor"` + Views []string `json:"views,omitempty"` + CheckedNames []string `json:"checkedNames"` + Positive map[string]time.Time `json:"positive,omitempty"` + Negative map[string]uint32 `json:"negative,omitempty"` + // Preserve subsecond t0: metav1.Time's wire encoding has only second precision. + WindowStart time.Time `json:"windowStart,omitempty"` + Schedule schedule `json:"schedule"` + MissingHistory bool `json:"missingHistory,omitempty"` +} + +type publicationEvidence struct { + Witnesses map[string]string `json:"witnesses,omitempty"` + Positive map[string]map[string]time.Time `json:"positive,omitempty"` +} + +func withdrawalKey(item dnsv1alpha1.ZoneUnitWithdrawal) string { + return item.ClaimNamespace + "/" + item.ClaimName +} + +func withdrawalIdentity(unit *dnsv1alpha1.ZoneUnit, item dnsv1alpha1.ZoneUnitWithdrawal) string { + return probe.Fingerprint(struct { + UnitUID string + Zone dnsv1alpha1.ObjectReference + Claim dnsv1alpha1.WithdrawalObjectReference + Generation int64 + Name, Address string + }{string(unit.UID), unit.Spec.Zone.Ref, item.Claim, item.ObservedGeneration, item.Name, item.Address}) +} + +func withdrawalNames(unit *dnsv1alpha1.ZoneUnit, item dnsv1alpha1.ZoneUnitWithdrawal, publication publicationEvidence) []string { + name := unit.Spec.Zone.DomainName + if item.Name != "@" { + name = item.Name + "." + name + } + if item.Name != "*" && !strings.HasPrefix(item.Name, "*.") { + return []string{name} + } + witness := probe.Fingerprint(struct { + ClaimUID string + Zone dnsv1alpha1.ObjectReference + UnitUID string + }{string(item.Claim.UID), unit.Spec.Zone.Ref, string(unit.UID)}) + names := []string{"dns-retire-" + witness[:24] + strings.TrimPrefix(name, "*")} + if witness := publication.Witnesses[item.Address]; witness != "" { + names = append(names, witness) + } + slices.Sort(names) + return slices.Compact(names) +} + +func validWithdrawal(unit *dnsv1alpha1.ZoneUnit, item dnsv1alpha1.ZoneUnitWithdrawal) bool { + ip, err := netip.ParseAddr(item.Address) + if err != nil || ip.Zone() != "" || ip.Unmap().String() != item.Address || item.Claim.UID == "" || item.ClaimNamespace == "" || item.ClaimName == "" || item.Claim.Namespace != item.ClaimNamespace || item.Claim.Name != item.ClaimName || item.ObservedGeneration < 1 { + return false + } + name := strings.TrimPrefix(item.Name, "*.") + if item.Name == "@" || item.Name == "*" { + name = unit.Spec.Zone.DomainName + } else { + name += "." + unit.Spec.Zone.DomainName + } + return config.ValidName(name) +} + +func cacheKey(checked string, evidence probe.WithdrawalCacheEvidence) string { + // A chain's maximum is sufficient for each checked-name/view/family bound; + // retaining terminal owner keys would grow without bound as CNAMEs change. + return checked + "/" + evidence.View + "/" + evidence.QType +} + +func retainPositive(state *withdrawalState, checked string, decision probe.WithdrawalDecision, now time.Time) { + for _, item := range decision.Cache { + if item.PositiveTTL == nil { + continue + } + if state.Positive == nil { + state.Positive = map[string]time.Time{} + } + key := cacheKey(checked, item) + expires := now.Add(time.Duration(*item.PositiveTTL) * time.Second) + if expires.After(state.Positive[key]) { + state.Positive[key] = expires + } + } +} + +func retainPublication(state *schedulerState, group nameGroup, request probe.ProbeRequest, result probe.ProbeResult, now time.Time) { + old := state.Publications[group.Name] + if old.Witnesses == nil { + old.Witnesses = map[string]string{} + } + if old.Positive == nil { + old.Positive = map[string]map[string]time.Time{} + } + for address, horizons := range old.Positive { + for key, expires := range horizons { + if !expires.After(now) { + delete(horizons, key) + } + } + if len(horizons) == 0 { + delete(old.Positive, address) + delete(old.Witnesses, address) + } + } + addresses := append(slices.Clone(group.Expected.A), group.Expected.AAAA...) + for address := range old.Positive { + addresses = append(addresses, address) + } + slices.Sort(addresses) + // Retain any renewed observation of an already-known retired address, even + // while the active publication intent has moved to its replacement. + for _, address := range slices.Compact(addresses) { + request.WithdrawalAddress = address + decision := probe.EvaluateWithdrawal(request, result) + temporary := withdrawalState{Positive: old.Positive[address]} + retainPositive(&temporary, group.CheckedName, decision, now) + if slices.ContainsFunc(decision.Cache, func(evidence probe.WithdrawalCacheEvidence) bool { return evidence.PositiveTTL != nil }) { + old.Witnesses[address] = group.CheckedName + } + if len(temporary.Positive) != 0 { + old.Positive[address] = temporary.Positive + } + } + state.Publications[group.Name] = old +} + +func publicationUnexpired(publication publicationEvidence, now time.Time) bool { + for _, horizons := range publication.Positive { + for _, expires := range horizons { + if expires.After(now) { + return true + } + } + } + return false +} + +func requiredWithdrawalViews(policy externalv1alpha1.ExternalZoneClassParameters, evidence *probe.ZoneEvidence) []string { + var views []string + for _, server := range policy.Resolver.Servers { + views = append(views, "recursive/"+server) + } + if len(policy.ExpectedNameservers) != 0 && evidence != nil { + for _, authority := range evidence.Authorities { + for _, server := range policy.Resolver.Servers { + views = append(views, "authority/"+authority+"/via/"+server) + } + } + } + slices.Sort(views) + return slices.Compact(views) +} + +func containsAll(values, floor []string) bool { + for _, value := range floor { + if !slices.Contains(values, value) { + return false + } + } + return true +} + +func policyWeakened(state withdrawalState, policy externalv1alpha1.ExternalZoneClassParameters, views []string) bool { + return state.MissingHistory || !containsAll(policy.Resolver.Servers, state.Floor.Resolver.Servers) || !containsAll(policy.ExpectedNameservers, state.Floor.ExpectedNameservers) || !containsAll(views, state.Views) +} + +func addFloor(state *withdrawalState, policy externalv1alpha1.ExternalZoneClassParameters, views []string) { + state.Floor.Resolver.Servers = sortedUnion(state.Floor.Resolver.Servers, policy.Resolver.Servers) + state.Floor.ExpectedNameservers = sortedUnion(state.Floor.ExpectedNameservers, policy.ExpectedNameservers) + state.Views = sortedUnion(state.Views, views) +} + +func sortedUnion(a, b []string) []string { + out := append(slices.Clone(a), b...) + slices.Sort(out) + return slices.Compact(out) +} + +func invalidateWithdrawal(status *dnsv1alpha1.ZoneUnitWithdrawalStatus, state *withdrawalState) { + status.AbsentSince, status.NotBefore, status.VerifiedAt = nil, nil, nil + status.ResultDigest = "" + state.Negative = nil + state.WindowStart = time.Time{} +} + +func withdrawalWindow(status *dnsv1alpha1.ZoneUnitWithdrawalStatus, state *withdrawalState, decision probe.WithdrawalDecision, now time.Time) probe.Decision { + if decision.Status != metav1.ConditionTrue { + invalidateWithdrawal(status, state) + return decision.Decision + } + if status.AbsentSince != nil && state.WindowStart.IsZero() { + invalidateWithdrawal(status, state) + } + confirming := status.AbsentSince != nil && status.LastProbeTime != nil && now.After(status.LastProbeTime.Time) + if status.AbsentSince == nil { + status.AbsentSince = timePointer(now) + state.WindowStart = now + } + notBefore := state.WindowStart + if status.NotBefore != nil && status.NotBefore.After(notBefore) { + notBefore = status.NotBefore.Time + } + for _, expires := range state.Positive { + if expires.After(notBefore) { + notBefore = expires + } + } + if state.Negative == nil { + state.Negative = map[string]uint32{} + } + for _, item := range decision.Cache { + if item.NegativeTTL == nil { + continue + } + key := item.View + "/" + item.QType + state.Negative[key] = max(state.Negative[key], *item.NegativeTTL) + } + for _, ttl := range state.Negative { + expires := state.WindowStart.Add(time.Duration(ttl) * time.Second) + if expires.After(notBefore) { + notBefore = expires + } + } + // Round the public horizon up, never down across a status round-trip. + if notBefore.Nanosecond() != 0 { + notBefore = notBefore.Truncate(time.Second).Add(time.Second) + } + status.NotBefore = timePointer(notBefore) + if !confirming || now.Before(notBefore) { + status.VerifiedAt = nil + return probe.Decision{Status: metav1.ConditionFalse, Reason: "NegativeTTLNotElapsed"} + } + status.VerifiedAt = timePointer(now) + return probe.Decision{Status: metav1.ConditionTrue, Reason: "WithdrawalVerified"} +} diff --git a/internal/go/providers/external/controllers/zoneunit/withdrawal_test.go b/internal/go/providers/external/controllers/zoneunit/withdrawal_test.go new file mode 100644 index 0000000..0d151dc --- /dev/null +++ b/internal/go/providers/external/controllers/zoneunit/withdrawal_test.go @@ -0,0 +1,464 @@ +package zoneunit + +import ( + "context" + "errors" + "fmt" + "slices" + "testing" + "time" + + "github.com/appthrust/dns-api/internal/go/providers/external/probe" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func withdrawalItem(name, address string) dnsv1alpha1.ZoneUnitWithdrawal { + return dnsv1alpha1.ZoneUnitWithdrawal{ClaimNamespace: "app", ClaimName: "retired", Claim: dnsv1alpha1.WithdrawalObjectReference{Namespace: "app", Name: "retired", UID: "claim-1"}, ObservedGeneration: 1, Name: name, Address: address, Allowed: true} +} + +func withdrawalAnswer(request probe.ProbeRequest, address string, ttl uint32) (probe.ProbeResult, error) { + result := probe.ProbeResult{} + for _, server := range request.Policy.Resolver.Servers { + view := probe.View{ID: "recursive/" + server} + for _, family := range []string{"A", "AAAA"} { + answer := probe.AnswerEvidence{Name: request.Name, QType: family, NegativeTTL: &ttl} + if family == "A" && address != "" { + answer.Addresses = []string{address} + answer.PositiveTTL = &ttl + view.A = []string{address} + } + view.Answers = append(view.Answers, answer) + } + result.Views = append(result.Views, view) + } + return result, nil +} + +func addWithdrawal(t *testing.T, c client.Client, request ctrl.Request, item dnsv1alpha1.ZoneUnitWithdrawal) { + t.Helper() + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + unit.Spec.Withdrawals = append(unit.Spec.Withdrawals, item) + unit.Generation++ + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } +} + +func assertWithdrawal(t *testing.T, unit dnsv1alpha1.ZoneUnit, status metav1.ConditionStatus, reason string) dnsv1alpha1.ZoneUnitWithdrawalStatus { + t.Helper() + if len(unit.Status.Withdrawals) != 1 { + t.Fatalf("withdrawals=%#v", unit.Status.Withdrawals) + } + got := unit.Status.Withdrawals[0] + condition := meta.FindStatusCondition(got.Conditions, "Withdrawn") + if condition == nil || condition.Status != status || condition.Reason != reason { + t.Fatalf("Withdrawn=%#v, want %s/%s", condition, status, reason) + } + return got +} + +func TestWithdrawalHorizonConfirmationAndNoStatusLoop(t *testing.T) { + resolver := &fakeResolver{answer: func(request probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(request, "192.0.2.20", 60) + }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + _, first := reconcileOnce(t, r, request) + pending := assertWithdrawal(t, first, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if pending.AbsentSince == nil || !pending.AbsentSince.Equal(timePointer(*now)) || pending.NotBefore == nil || !pending.NotBefore.Equal(timePointer(now.Add(time.Minute))) || pending.VerifiedAt != nil { + t.Fatalf("window=%#v", pending) + } + _, same := reconcileOnce(t, r, request) + if same.ResourceVersion != first.ResourceVersion || len(resolver.requests) != 1 { + t.Fatal("unchanged status was patched or reprobed") + } + *now = now.Add(30 * time.Second) + _, waiting := reconcileOnce(t, r, request) + still := assertWithdrawal(t, waiting, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if !still.NotBefore.Equal(pending.NotBefore) { + t.Fatal("identical negatives slid the horizon") + } + // A new reconciler restores the still-fresh identity-bound window. + r2, err := NewReconciler(c, r.Options) + if err != nil { + t.Fatal(err) + } + *now = now.Add(30 * time.Second) + _, verified := reconcileOnce(t, r2, request) + good := assertWithdrawal(t, verified, metav1.ConditionTrue, "WithdrawalVerified") + if good.VerifiedAt == nil || good.VerifiedAt.Before(good.NotBefore) || good.ResultDigest == "" { + t.Fatalf("unbound success=%#v", good) + } + assertProgrammed(t, verified.Status.Conditions, metav1.ConditionTrue, "Programmed") + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(req, "192.0.2.10", 600) + } + *now = now.Add(30 * time.Second) + _, reappeared := reconcileOnce(t, r2, request) + old := assertWithdrawal(t, reappeared, metav1.ConditionFalse, "DNSStillResolves") + if old.AbsentSince != nil || old.VerifiedAt != nil { + t.Fatal("reappearance preserved success") + } + resolver.answer = func(probe.ProbeRequest) (probe.ProbeResult, error) { + return probe.ProbeResult{}, errors.New("secret transport diagnostic") + } + *now = now.Add(30 * time.Second) + _, failed := reconcileOnce(t, r2, request) + assertWithdrawal(t, failed, metav1.ConditionUnknown, "DNSProbeFailed") + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) } + *now = now.Add(30 * time.Second) + _, afterFailure := reconcileOnce(t, r2, request) + fresh := assertWithdrawal(t, afterFailure, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if !fresh.NotBefore.Equal(timePointer(old.LastProbeTime.Add(600 * time.Second))) { + t.Fatal("failure discarded known positive horizon") + } +} + +func TestWithdrawalPolicyFloorAndWaiver(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + reconcileOnce(t, r, request) + *now = now.Add(30 * time.Second) + reconcileOnce(t, r, request) + var class dnsv1alpha1.ZoneClass + if err := c.Get(context.Background(), client.ObjectKey{Namespace: "app", Name: "external"}, &class); err != nil { + t.Fatal(err) + } + class.Spec.Parameters.Raw = []byte(`{"resolver":{"mode":"Recursive","servers":["192.0.2.54:53"]}}`) + if err := c.Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + _, changed := reconcileOnce(t, r, request) + assertWithdrawal(t, changed, metav1.ConditionUnknown, "PolicyChanged") + *now = now.Add(30 * time.Second) + _, weaker := reconcileOnce(t, r, request) + assertWithdrawal(t, weaker, metav1.ConditionUnknown, "PolicyChanged") + class.Spec.Parameters.Raw = []byte(`{"resolver":{"mode":"Recursive","servers":["192.0.2.53:53"]}}`) + if err := c.Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + reconcileOnce(t, r, request) + *now = now.Add(30 * time.Second) + _, restored := reconcileOnce(t, r, request) + assertWithdrawal(t, restored, metav1.ConditionFalse, "NegativeTTLNotElapsed") + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(req, "192.0.2.10", 60) + } + *now = now.Add(30 * time.Second) + _, old := reconcileOnce(t, r, request) + old.Spec.Withdrawals[0].Waiver = &dnsv1alpha1.ZoneUnitWithdrawalWaiver{Name: "emergency", UID: "waiver-1", Digest: "sha256:scope", ReservationRef: dnsv1alpha1.WithdrawalObjectReference{Namespace: "mgmt", Name: "reservation", UID: "reservation-1"}} + if err := c.Update(context.Background(), &old); err != nil { + t.Fatal(err) + } + _, waived := reconcileOnce(t, r, request) + got := assertWithdrawal(t, waived, metav1.ConditionFalse, "DNSStillResolves") + if !meta.IsStatusConditionTrue(got.Conditions, "OverrideAccepted") || got.WaiverRef == nil || got.WaiverRef.UID != "waiver-1" { + t.Fatalf("waiver=%#v", got) + } + waived.Spec.Withdrawals[0].Waiver = nil + if err := c.Update(context.Background(), &waived); err != nil { + t.Fatal(err) + } + _, revoked := reconcileOnce(t, r, request) + if revoked.Status.Withdrawals[0].WaiverRef != nil || meta.IsStatusConditionTrue(revoked.Status.Withdrawals[0].Conditions, "OverrideAccepted") { + t.Fatal("deleted waiver retained") + } +} + +func TestWithdrawalWildcardRetainsPublicationWitness(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(req, "192.0.2.10", 3600) + }} + r, c, now, request := newFixture(t, 1, resolver) + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + unit.Spec.RecordSets[0].Name = "*.project" + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, published := reconcileOnce(t, r, request) + witness := resolver.requests[0].Name + published.Spec.RecordSets = nil + if err := c.Update(context.Background(), &published); err != nil { + t.Fatal(err) + } + reconcileOnce(t, r, request) // Core may remove the record before composing its independent claim. + addWithdrawal(t, c, request, withdrawalItem("*.project", "192.0.2.10")) + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { + if req.Name == witness { + return withdrawalAnswer(req, "192.0.2.10", 3600) + } + return withdrawalAnswer(req, "", 0) + } + _, retiring := reconcileOnce(t, r, request) + got := assertWithdrawal(t, retiring, metav1.ConditionFalse, "DNSStillResolves") + if !got.SampledWildcard || len(got.CheckedNames) != 2 || !slices.Contains(got.CheckedNames, witness) { + t.Fatalf("lost witness=%#v", got) + } + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) } + *now = now.Add(30 * time.Second) + _, absent := reconcileOnce(t, r, request) + pending := assertWithdrawal(t, absent, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if !pending.NotBefore.Equal(timePointer(got.LastProbeTime.Add(time.Hour))) { + t.Fatal("3600-second horizon was capped") + } + recreated := withdrawalItem("*.project", "192.0.2.10") + recreated.Claim.UID = "claim-2" + if slices.Equal(withdrawalNames(&unit, recreated, publicationEvidence{}), withdrawalNames(&unit, withdrawalItem("*.project", "192.0.2.10"), publicationEvidence{})) { + t.Fatal("recreated claim reused retirement witness") + } +} + +func TestWithdrawalFreshnessAndDigest(t *testing.T) { + for _, mode := range []string{"stale", "clock reversal", "missing private state", "UID recreation"} { + t.Run(mode, func(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + reconcileOnce(t, r, request) + *now = now.Add(30 * time.Second) + _, unit := reconcileOnce(t, r, request) + good := assertWithdrawal(t, unit, metav1.ConditionTrue, "WithdrawalVerified") + copy := good + copy.Conditions = slices.Clone(good.Conditions) + slices.Reverse(copy.Conditions) + for i := range copy.Conditions { + copy.Conditions[i].LastTransitionTime = metav1.Now() + } + if withdrawalResultDigest(&unit, unit.Spec.Withdrawals[0], copy) != good.ResultDigest { + t.Fatal("digest depends on condition order/transition times") + } + switch mode { + case "stale": + *now = now.Add(331 * time.Second) + case "clock reversal": + *now = now.Add(-time.Minute) + case "missing private state": + unit.Status.Provider = nil + if err := c.Status().Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + case "UID recreation": + unit.Spec.Withdrawals[0].Claim.UID = "claim-2" + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + } + _, got := reconcileOnce(t, r, request) + reason := "NegativeTTLNotElapsed" + status := metav1.ConditionFalse + if mode == "missing private state" { + reason = "PolicyChanged" + status = metav1.ConditionUnknown + } + assertWithdrawal(t, got, status, reason) + }) + } +} + +func TestWithdrawalFairBudgetAndDenied(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) }} + r, c, now, request := newFixture(t, 40, resolver) + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + for i := range 40 { + item := withdrawalItem("www", "192.0.2.10") + item.ClaimName = fmt.Sprintf("claim-%02d", i) + item.Claim.Name = item.ClaimName + item.Claim.UID = types.UID(item.ClaimName) + unit.Spec.Withdrawals = append(unit.Spec.Withdrawals, item) + } + unit.Spec.Withdrawals[0].Allowed = false + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, first := reconcileOnce(t, r, request) + if len(resolver.requests) != MaxNamesPerPass || first.Status.Withdrawals[0].LastProbeTime != nil { + t.Fatal("budget/denied boundary") + } + for range 2 { + *now = now.Add(time.Second) + reconcileOnce(t, r, request) + } + if len(resolver.requests) != 79 { + t.Fatalf("starved work: probes=%d", len(resolver.requests)) + } +} + +func TestWithdrawalLateObservationAndInputMutation(t *testing.T) { + for _, mode := range []string{"late", "changed"} { + t.Run(mode, func(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + reconcileOnce(t, r, request) + *now = now.Add(30 * time.Second) + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { + if mode == "late" { + *now = now.Add(301 * time.Second) + } else { + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + unit.Spec.Withdrawals[0].Claim.UID = "replacement" + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + } + return withdrawalAnswer(req, "", 0) + } + result, unit := reconcileOnce(t, r, request) + if mode == "late" { + assertWithdrawal(t, unit, metav1.ConditionUnknown, "DNSProbeFailed") + } else if result.RequeueAfter != time.Second || unit.Status.Withdrawals[0].VerifiedAt != nil { + t.Fatal("stale result committed") + } + }) + } +} + +func TestWithdrawalContinuesDuringUnitDeletion(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(req, "192.0.2.10", 60) + }} + r, c, _, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + var unit dnsv1alpha1.ZoneUnit + if err := c.Get(context.Background(), request.NamespacedName, &unit); err != nil { + t.Fatal(err) + } + unit.Finalizers = []string{"core-retention"} + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + if err := c.Delete(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, got := reconcileOnce(t, r, request) + assertWithdrawal(t, got, metav1.ConditionFalse, "DNSStillResolves") +} + +func TestWithdrawalWindowExtendsNeverShortens(t *testing.T) { + now := time.Date(2026, 10, 18, 0, 0, 0, 0, time.UTC) + status := dnsv1alpha1.ZoneUnitWithdrawalStatus{} + stored := withdrawalState{} + for _, ttl := range []uint32{60, 120, 30} { + decision := probe.WithdrawalDecision{Decision: probe.Decision{Status: metav1.ConditionTrue}, Cache: []probe.WithdrawalCacheEvidence{{Name: "www.example.test", View: "recursive/a", QType: "AAAA", NegativeTTL: &ttl}}} + got := withdrawalWindow(&status, &stored, decision, now) + if got.Reason != "NegativeTTLNotElapsed" { + t.Fatalf("unexpected decision %#v", got) + } + status.LastProbeTime = timePointer(now) + now = now.Add(30 * time.Second) + } + if !status.NotBefore.Equal(timePointer(status.AbsentSince.Add(120 * time.Second))) { + t.Fatal("changed answer shortened or slid the horizon") + } +} + +func TestWithdrawalSubsecondHorizonSurvivesStatusPrecision(t *testing.T) { + start := time.Date(2026, 10, 18, 0, 0, 0, 900000000, time.UTC) + ttl := uint32(60) + decision := probe.WithdrawalDecision{Decision: probe.Decision{Status: metav1.ConditionTrue}, Cache: []probe.WithdrawalCacheEvidence{{Name: "www.example.test", View: "recursive/a", QType: "A", NegativeTTL: &ttl}}} + status := dnsv1alpha1.ZoneUnitWithdrawalStatus{} + state := withdrawalState{} + withdrawalWindow(&status, &state, decision, start) + // metav1.Time JSON drops fractional seconds; the private anchor must not. + status.AbsentSince = timePointer(start.Truncate(time.Second)) + status.LastProbeTime = timePointer(start.Truncate(time.Second)) + if status.NotBefore.Nanosecond() != 0 || !status.NotBefore.Equal(timePointer(start.Truncate(time.Second).Add(61*time.Second))) { + t.Fatal("public horizon rounded down") + } + if got := withdrawalWindow(&status, &state, decision, start.Truncate(time.Second).Add(60*time.Second)); got.Status == metav1.ConditionTrue { + t.Fatal("subsecond expiry released early") + } + if got := withdrawalWindow(&status, &state, decision, start.Truncate(time.Second).Add(61*time.Second)); got.Status != metav1.ConditionTrue { + t.Fatal("post-horizon confirming probe did not verify") + } +} + +func TestWithdrawalDeniedHistoryAndIPv6(t *testing.T) { + const address = "2001:db8::1" + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { + result, err := withdrawalAnswer(req, "", 3600) + for i := range result.Views { + result.Views[i].AAAA = []string{address} + result.Views[i].Answers[1].Addresses = []string{address} + ttl := uint32(3600) + result.Views[i].Answers[1].PositiveTTL = &ttl + } + return result, err + }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", address)) + _, unit := reconcileOnce(t, r, request) + old := assertWithdrawal(t, unit, metav1.ConditionFalse, "DNSStillResolves") + unit.Spec.Withdrawals[0].Allowed = false + if err := c.Update(context.Background(), &unit); err != nil { + t.Fatal(err) + } + _, denied := reconcileOnce(t, r, request) + if len(resolver.requests) != 1 || !meta.IsStatusConditionFalse(denied.Status.Withdrawals[0].Conditions, "Accepted") { + t.Fatal("denied withdrawal was probed or accepted") + } + denied.Spec.Withdrawals[0].Allowed = true + if err := c.Update(context.Background(), &denied); err != nil { + t.Fatal(err) + } + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) } + *now = now.Add(30 * time.Second) + _, restored := reconcileOnce(t, r, request) + got := assertWithdrawal(t, restored, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if !got.NotBefore.Equal(timePointer(old.LastProbeTime.Add(time.Hour))) { + t.Fatal("denial discarded IPv6 positive horizon") + } +} + +func TestWithdrawalBindingRejectionNoStatusLoop(t *testing.T) { + resolver := &fakeResolver{answer: func(req probe.ProbeRequest) (probe.ProbeResult, error) { + return withdrawalAnswer(req, "192.0.2.10", 3600) + }} + r, c, now, request := newFixture(t, 0, resolver) + addWithdrawal(t, c, request, withdrawalItem("www", "192.0.2.10")) + _, unit := reconcileOnce(t, r, request) + old := assertWithdrawal(t, unit, metav1.ConditionFalse, "DNSStillResolves") + var class dnsv1alpha1.ZoneClass + if err := c.Get(context.Background(), client.ObjectKey{Namespace: "app", Name: "external"}, &class); err != nil { + t.Fatal(err) + } + class.Status.Conditions[0].Status = metav1.ConditionFalse + if err := c.Status().Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + _, rejected := reconcileOnce(t, r, request) + assertWithdrawal(t, rejected, metav1.ConditionUnknown, "DNSProbeFailed") + *now = now.Add(time.Second) + _, same := reconcileOnce(t, r, request) + if same.ResourceVersion != rejected.ResourceVersion { + t.Fatal("unchanged rejected binding patched status") + } + class.Status.Conditions[0].Status = metav1.ConditionTrue + if err := c.Status().Update(context.Background(), &class); err != nil { + t.Fatal(err) + } + resolver.answer = func(req probe.ProbeRequest) (probe.ProbeResult, error) { return withdrawalAnswer(req, "", 0) } + _, restored := reconcileOnce(t, r, request) + got := assertWithdrawal(t, restored, metav1.ConditionFalse, "NegativeTTLNotElapsed") + if !got.NotBefore.Equal(timePointer(old.LastProbeTime.Add(time.Hour))) { + t.Fatal("binding rejection discarded known positive horizon") + } +} diff --git a/internal/go/providers/external/probe/metrics.go b/internal/go/providers/external/probe/metrics.go index 937b3de..830d510 100644 --- a/internal/go/providers/external/probe/metrics.go +++ b/internal/go/providers/external/probe/metrics.go @@ -9,15 +9,17 @@ import ( ) var ( - probes = prometheus.NewCounterVec(prometheus.CounterOpts{Name: "dns_external_probes_total", Help: "Completed external DNS observations."}, []string{"outcome", "qtype"}) - duration = prometheus.NewHistogramVec(prometheus.HistogramOpts{Name: "dns_external_probe_duration_seconds", Help: "Admitted external DNS probe duration."}, []string{"outcome"}) - queueDelay = prometheus.NewHistogram(prometheus.HistogramOpts{Name: "dns_external_probe_queue_delay_seconds", Help: "Scheduling and limiter admission delay for external DNS probes."}) - pending = prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: "dns_external_pending_records", Help: "Active unverified external record claims."}, []string{"reason"}) - pendingMu sync.Mutex - pendingUnits = map[string]map[string]int{} + probes = prometheus.NewCounterVec(prometheus.CounterOpts{Name: "dns_external_probes_total", Help: "Completed external DNS observations."}, []string{"outcome", "qtype"}) + duration = prometheus.NewHistogramVec(prometheus.HistogramOpts{Name: "dns_external_probe_duration_seconds", Help: "Admitted external DNS probe duration."}, []string{"outcome"}) + queueDelay = prometheus.NewHistogram(prometheus.HistogramOpts{Name: "dns_external_probe_queue_delay_seconds", Help: "Scheduling and limiter admission delay for external DNS probes."}) + pending = prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: "dns_external_pending_records", Help: "Active unverified external record claims."}, []string{"reason"}) + pendingMu sync.Mutex + pendingUnits = map[string]map[string]int{} + withdrawalsPending = prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: "dns_external_withdrawals_pending", Help: "Retained external DNS withdrawals awaiting verified exclusion."}, []string{"reason"}) + withdrawalUnits = map[string]map[string]int{} ) -func init() { metrics.Registry.MustRegister(probes, duration, queueDelay, pending) } +func init() { metrics.Registry.MustRegister(probes, duration, queueDelay, pending, withdrawalsPending) } func ObserveQueueDelay(delay time.Duration) { queueDelay.Observe(max(0, delay.Seconds())) } @@ -45,3 +47,28 @@ func SetPending(unit string, counts map[string]int) { pending.WithLabelValues(reason).Add(float64(count)) } } + +// SetWithdrawalsPending uses only DEP's closed pending reasons as labels. +func SetWithdrawalsPending(unit string, counts map[string]int) { + pendingMu.Lock() + defer pendingMu.Unlock() + for reason, count := range withdrawalUnits[unit] { + withdrawalsPending.WithLabelValues(reason).Sub(float64(count)) + } + delete(withdrawalUnits, unit) + next := map[string]int{} + for reason, count := range counts { + switch reason { + case "DNSStillResolves", "NegativeTTLNotElapsed", "DNSProbeFailed", "PolicyChanged", "InvalidWithdrawal", "NotAllowed", "UnsupportedProvider": + default: + reason = "DNSProbeFailed" + } + next[reason] += count + } + if len(next) != 0 { + withdrawalUnits[unit] = next + } + for reason, count := range next { + withdrawalsPending.WithLabelValues(reason).Add(float64(count)) + } +} diff --git a/internal/go/providers/external/probe/types.go b/internal/go/providers/external/probe/types.go index 3d110b2..440a2b2 100644 --- a/internal/go/providers/external/probe/types.go +++ b/internal/go/providers/external/probe/types.go @@ -35,6 +35,8 @@ type ProbeRequest struct { ZoneOnly bool // Evidence is cycle-local, never a process-wide DNS answer cache. Evidence *ZoneEvidence + // WithdrawalAddress selects exclusion evaluation, never exact-set publication. + WithdrawalAddress string } type ProbeResult struct { @@ -54,6 +56,9 @@ type View struct { // Failure is a closed diagnostic code; raw transport errors never enter status. Failure string Mismatch bool + // Answers retains each chain query's complete cache evidence. Publication's + // minimum-TTL summaries are not sufficient to authorize address reuse. + Answers []AnswerEvidence } type AuthorityObservation struct { diff --git a/internal/go/providers/external/probe/wire.go b/internal/go/providers/external/probe/wire.go index ee4dd16..abc1288 100644 --- a/internal/go/providers/external/probe/wire.go +++ b/internal/go/providers/external/probe/wire.go @@ -120,7 +120,7 @@ func (r *WireResolver) Probe(ctx context.Context, request ProbeRequest) (ProbeRe for _, authority := range request.Evidence.Authorities { view := session.resolve(ctx, request.Name, request.Policy.Resolver.Servers[0], authority, request.Zone) result.Views = append(result.Views, view) - foreign := false + foreign := request.WithdrawalAddress != "" for _, target := range view.Chain { if target != request.Zone && !strings.HasSuffix(target, "."+request.Zone) { foreign = true @@ -138,6 +138,8 @@ func (r *WireResolver) Probe(ctx context.Context, request ProbeRequest) (ProbeRe decision := Evaluate(request, result) if request.ZoneOnly { decision = EvaluateZone(request, result.Zone) + } else if request.WithdrawalAddress != "" { + decision = EvaluateWithdrawal(request, result).Decision } outcome := decision.Outcome() for qtype, count := range session.qtypes { @@ -245,6 +247,7 @@ func (s *wireSession) resolve(ctx context.Context, name, recursive, authority, z view.Authoritative = true } rcodes[i] = reply.Rcode + view.Answers = append(view.Answers, answerEvidence(reply, current, qtype)) ownerAnswers := 0 for _, rr := range reply.Answer { if CanonicalName(rr.Header().Name) != current { diff --git a/internal/go/providers/external/probe/withdrawal.go b/internal/go/providers/external/probe/withdrawal.go new file mode 100644 index 0000000..8ddf88d --- /dev/null +++ b/internal/go/providers/external/probe/withdrawal.go @@ -0,0 +1,325 @@ +package probe + +import ( + "fmt" + "net/netip" + "slices" + "strings" + + "github.com/appthrust/dns-api/internal/go/providers/external/config" + "github.com/miekg/dns" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// AnswerEvidence describes one complete query, including CNAME links. TTLs are +// maxima of relevant positive records, or RFC 2308 negative TTLs, not poll caps. +type AnswerEvidence struct { + Name string `json:"name"` + QType string `json:"qtype"` + RCode int `json:"rcode"` + Addresses []string `json:"addresses,omitempty"` + CNAME string `json:"cname,omitempty"` + PositiveTTL *uint32 `json:"positiveTTL,omitempty"` + NegativeTTL *uint32 `json:"negativeTTL,omitempty"` + Failure string `json:"failure,omitempty"` +} + +// WithdrawalCacheEvidence is complete decision evidence, not a diagnostic preview. +// PositiveTTL is retained only where this observation saw the retired address. +type WithdrawalCacheEvidence struct { + Name string `json:"name"` + View string `json:"view"` + QType string `json:"qtype"` + PositiveTTL *uint32 `json:"positiveTTL,omitempty"` + NegativeTTL *uint32 `json:"negativeTTL,omitempty"` +} + +type WithdrawalDecision struct { + Decision + Cache []WithdrawalCacheEvidence +} + +func WithdrawalFailed(name, view, failure string) WithdrawalDecision { + switch failure { + case "timeout", "SERVFAIL", "REFUSED", "transport", "invalid-response", "resolver-config", "chain-limit", "probe-limit", "negative-soa-missing": + default: + failure = "transport" + } + return WithdrawalDecision{Decision: Decision{Status: metav1.ConditionUnknown, Reason: "DNSProbeFailed", Message: bounded(fmt.Sprintf("External DNS withdrawal cannot be verified for %s in %s: %s.", name, view, failure))}} +} + +// EvaluateWithdrawal proves only current exclusion. The reconciler additionally +// requires a continuous absence window, retained cache horizons and a fresh +// confirming attempt before it may publish Withdrawn=True. +func EvaluateWithdrawal(request ProbeRequest, result ProbeResult) WithdrawalDecision { + address, err := netip.ParseAddr(request.WithdrawalAddress) + if err != nil || address.Zone() != "" { + return WithdrawalFailed(request.Name, "required-views", "invalid-response") + } + address = address.Unmap() + fail := func(view, code string) WithdrawalDecision { + decision := WithdrawalFailed(request.Name, view, code) + decision.Cache = positiveWithdrawalEvidence(result, address) + return decision + } + if len(request.Policy.Resolver.Servers) == 0 || len(result.Views) == 0 { + return fail("required-views", "invalid-response") + } + for _, server := range request.Policy.Resolver.Servers { + if !slices.ContainsFunc(result.Views, func(v View) bool { return v.ID == "recursive/"+server }) { + return fail("recursive/"+server, "invalid-response") + } + } + if len(request.Policy.ExpectedNameservers) != 0 { + if request.Evidence == nil || request.Evidence.Zone != request.Zone || request.Evidence.PolicyFingerprint != Fingerprint(request.Policy) || EvaluateZone(request, request.Evidence).Status != metav1.ConditionTrue { + return fail("required-authorities", "invalid-response") + } + for _, authority := range request.Evidence.Authorities { + prefix := "authority/" + authority + "/via/" + for _, server := range request.Policy.Resolver.Servers { + if !slices.ContainsFunc(result.Views, func(v View) bool { return v.ID == prefix+server && v.Authoritative }) { + return fail(prefix+server, "invalid-response") + } + } + } + } + views := slices.Clone(result.Views) + slices.SortFunc(views, func(a, b View) int { return strings.Compare(a.ID, b.ID) }) + decision := WithdrawalDecision{Decision: Decision{Status: metav1.ConditionTrue, Reason: "WithdrawalVerified"}} + for i, view := range views { + if i > 0 && views[i-1].ID == view.ID { + return fail(view.ID, "invalid-response") + } + decision.Views = append(decision.Views, view.ID) + cache, qtype, failure := withdrawalViewEvidence(request.Name, address, view) + decision.Cache = append(decision.Cache, cache...) + if failure != "" { + return fail(view.ID, failure) + } + for _, item := range cache { + decision.NegativeTTL = maxTTL(decision.NegativeTTL, item.NegativeTTL) + } + if qtype != "" && decision.Status == metav1.ConditionTrue { + decision.Status, decision.Reason = metav1.ConditionFalse, "DNSStillResolves" + decision.Message = bounded(fmt.Sprintf("DNS %s still resolves to retired address %s in %s (%s).", request.Name, address, view.ID, qtype)) + } + } + return decision +} + +func withdrawalViewEvidence(name string, address netip.Addr, view View) ([]WithdrawalCacheEvidence, string, string) { + if view.Failure != "" { + return nil, "", view.Failure + } + if view.RCode != dns.RcodeSuccess && view.RCode != dns.RcodeNameError { + code := "invalid-response" + if view.RCode == dns.RcodeServerFailure { + code = "SERVFAIL" + } + if view.RCode == dns.RcodeRefused { + code = "REFUSED" + } + return nil, "", code + } + if view.Mismatch || len(view.Chain) > MaxCNAMEs { + return nil, "", "chain-limit" + } + owners := append([]string{CanonicalName(name)}, namesInOrder(view.Chain)...) + seen := map[string]bool{} + for _, owner := range owners { + if seen[owner] || !config.ValidName(owner) { + return nil, "", "invalid-response" + } + seen[owner] = true + } + if len(view.Answers) != 2*len(owners) { + return nil, "", "invalid-response" + } + answers := slices.Clone(view.Answers) + slices.SortFunc(answers, func(a, b AnswerEvidence) int { + if c := strings.Compare(a.Name, b.Name); c != 0 { + return c + } + return strings.Compare(a.QType, b.QType) + }) + qtype := "" + presentA, presentAAAA := false, false + for _, answer := range answers { + for _, value := range answer.Addresses { + ip, err := netip.ParseAddr(value) + if err != nil || ip.Zone() != "" || (answer.QType == "A" && !ip.Is4()) || (answer.QType == "AAAA" && !ip.Is6()) { + return nil, "", "invalid-response" + } + if ip.Unmap() == address { + qtype = answer.QType + presentA = presentA || answer.QType == "A" + presentAAAA = presentAAAA || answer.QType == "AAAA" + } + } + } + cache := make([]WithdrawalCacheEvidence, 0, len(answers)) + for i, owner := range owners { + for _, family := range []string{"A", "AAAA"} { + var found *AnswerEvidence + for j := range answers { + if CanonicalName(answers[j].Name) == owner && answers[j].QType == family { + if found != nil { + return cache, qtype, "invalid-response" + } + found = &answers[j] + } + } + if found == nil { + return cache, qtype, "invalid-response" + } + if found.Failure != "" { + return cache, qtype, found.Failure + } + if found.RCode != dns.RcodeSuccess && found.RCode != dns.RcodeNameError { + return cache, qtype, "invalid-response" + } + item := WithdrawalCacheEvidence{Name: owner, View: view.ID, QType: family} + if i < len(owners)-1 { + if found.RCode != dns.RcodeSuccess || CanonicalName(found.CNAME) != owners[i+1] || len(found.Addresses) != 0 || found.PositiveTTL == nil { + return cache, qtype, "invalid-response" + } + } else { + terminal := view.A + if family == "AAAA" { + terminal = view.AAAA + } + if found.RCode != view.RCode || !slices.Equal(Addresses(found.Addresses), Addresses(terminal)) { + return cache, qtype, "invalid-response" + } + if found.CNAME != "" || (found.RCode == dns.RcodeNameError && len(found.Addresses) != 0) { + return cache, qtype, "invalid-response" + } + if len(found.Addresses) == 0 { + if found.NegativeTTL == nil { + return cache, qtype, "negative-soa-missing" + } + item.NegativeTTL = found.NegativeTTL + } else if found.PositiveTTL == nil { + return cache, qtype, "invalid-response" + } + } + if qtype != "" && (found.CNAME != "" || (family == "A" && presentA) || (family == "AAAA" && presentAAAA)) { + item.PositiveTTL = found.PositiveTTL + } + cache = append(cache, item) + } + } + slices.SortFunc(cache, func(a, b WithdrawalCacheEvidence) int { + if c := strings.Compare(a.Name, b.Name); c != 0 { + return c + } + return strings.Compare(a.QType, b.QType) + }) + return cache, qtype, "" +} + +func namesInOrder(values []string) []string { + out := make([]string, len(values)) + for i, value := range values { + out[i] = CanonicalName(value) + } + return out +} + +// Failed/incomplete observations still retain any positively observed address +// and CNAME horizon. An earlier failed view must not hide a later positive one. +func positiveWithdrawalEvidence(result ProbeResult, address netip.Addr) []WithdrawalCacheEvidence { + var cache []WithdrawalCacheEvidence + for _, view := range result.Views { + presentA, presentAAAA := false, false + for _, answer := range view.Answers { + for _, value := range answer.Addresses { + if ip, err := netip.ParseAddr(value); err == nil && ip.Unmap() == address { + presentA = presentA || answer.QType == "A" + presentAAAA = presentAAAA || answer.QType == "AAAA" + } + } + } + if !presentA && !presentAAAA { + continue + } + for _, answer := range view.Answers { + if answer.PositiveTTL != nil && (answer.CNAME != "" || (answer.QType == "A" && presentA) || (answer.QType == "AAAA" && presentAAAA)) { + cache = append(cache, WithdrawalCacheEvidence{Name: answer.Name, View: view.ID, QType: answer.QType, PositiveTTL: answer.PositiveTTL}) + } + } + } + slices.SortFunc(cache, func(a, b WithdrawalCacheEvidence) int { + if c := strings.Compare(a.View, b.View); c != 0 { + return c + } + if c := strings.Compare(a.Name, b.Name); c != 0 { + return c + } + return strings.Compare(a.QType, b.QType) + }) + return cache +} + +func maxTTL(current, next *uint32) *uint32 { + if next != nil && (current == nil || *next > *current) { + value := *next + return &value + } + return current +} + +// answerEvidence validates query-local records without changing publication's +// existing evaluation semantics. Missing negative SOA is never invented as zero. +func answerEvidence(reply *dns.Msg, name string, qtype uint16) AnswerEvidence { + answer := AnswerEvidence{Name: name, QType: dns.TypeToString[qtype], RCode: reply.Rcode} + for _, rr := range reply.Answer { + if CanonicalName(rr.Header().Name) != name { + continue + } + ttl := rr.Header().Ttl + switch value := rr.(type) { + case *dns.CNAME: + target := CanonicalName(value.Target) + if answer.CNAME != "" && answer.CNAME != target { + answer.Failure = "invalid-response" + } + answer.CNAME = target + case *dns.A: + if qtype != dns.TypeA { + answer.Failure = "invalid-response" + } + answer.Addresses = append(answer.Addresses, value.A.String()) + case *dns.AAAA: + if qtype != dns.TypeAAAA { + answer.Failure = "invalid-response" + } + answer.Addresses = append(answer.Addresses, value.AAAA.String()) + default: + continue + } + if rr.Header().Class != dns.ClassINET { + answer.Failure = "invalid-response" + } + answer.PositiveTTL = maxTTL(answer.PositiveTTL, &ttl) + } + if answer.CNAME != "" && len(answer.Addresses) != 0 { + answer.Failure = "invalid-response" + } + if reply.Rcode == dns.RcodeNameError || (answer.CNAME == "" && len(answer.Addresses) == 0) { + for _, rr := range reply.Ns { + soa, ok := rr.(*dns.SOA) + if !ok { + continue + } + owner := CanonicalName(soa.Hdr.Name) + if (owner != "" && owner != name && !strings.HasSuffix(name, "."+owner)) || soa.Hdr.Class != dns.ClassINET || !config.ValidName(CanonicalName(soa.Ns)) || !config.ValidName(CanonicalName(soa.Mbox)) { + answer.Failure = "invalid-response" + continue + } + ttl := min(soa.Hdr.Ttl, soa.Minttl) + answer.NegativeTTL = maxTTL(answer.NegativeTTL, &ttl) + } + } + return answer +} diff --git a/internal/go/providers/external/probe/withdrawal_test.go b/internal/go/providers/external/probe/withdrawal_test.go new file mode 100644 index 0000000..ecbe8d0 --- /dev/null +++ b/internal/go/providers/external/probe/withdrawal_test.go @@ -0,0 +1,190 @@ +package probe + +import ( + "fmt" + "net" + "slices" + "testing" + + externalv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/external/v1alpha1" + "github.com/miekg/dns" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func withdrawalView(id string, a, aaaa []string, ttl uint32) View { + return View{ID: id, A: a, AAAA: aaaa, Answers: []AnswerEvidence{ + {Name: "www.example.test", QType: "A", Addresses: a, PositiveTTL: &ttl, NegativeTTL: &ttl}, + {Name: "www.example.test", QType: "AAAA", Addresses: aaaa, PositiveTTL: &ttl, NegativeTTL: &ttl}, + }} +} + +func TestEvaluateWithdrawal(t *testing.T) { + request := ProbeRequest{Name: "www.example.test", Zone: "example.test", WithdrawalAddress: "192.0.2.1", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{"192.0.2.53:53", "192.0.2.54:53"}}}} + cases := []struct { + name string + mutate func(*ProbeRequest, *[]View) + status metav1.ConditionStatus + reason string + }{ + {"replacement views differ", func(_ *ProbeRequest, v *[]View) { + (*v)[1] = withdrawalView((*v)[1].ID, []string{"192.0.2.3"}, nil, 600) + }, metav1.ConditionTrue, "WithdrawalVerified"}, + {"old address", func(_ *ProbeRequest, v *[]View) { + (*v)[1] = withdrawalView((*v)[1].ID, []string{"192.0.2.1"}, nil, 600) + }, metav1.ConditionFalse, "DNSStillResolves"}, + {"opposite family mapped", func(_ *ProbeRequest, v *[]View) { + (*v)[1] = withdrawalView((*v)[1].ID, nil, []string{"::ffff:192.0.2.1"}, 600) + }, metav1.ConditionFalse, "DNSStillResolves"}, + {"IPv6", func(r *ProbeRequest, v *[]View) { + r.WithdrawalAddress = "2001:db8::1" + (*v)[1] = withdrawalView((*v)[1].ID, nil, []string{"2001:0DB8:0:0::1"}, 600) + }, metav1.ConditionFalse, "DNSStillResolves"}, + {"missing view", func(_ *ProbeRequest, v *[]View) { *v = (*v)[:1] }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"missing family", func(_ *ProbeRequest, v *[]View) { (*v)[0].Answers = (*v)[0].Answers[:1] }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"missing SOA", func(_ *ProbeRequest, v *[]View) { (*v)[0].Answers[1].NegativeTTL = nil }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"failed view", func(_ *ProbeRequest, v *[]View) { (*v)[1].Failure = "timeout" }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"CNAME loop", func(_ *ProbeRequest, v *[]View) { (*v)[0].Chain = []string{"www.example.test"} }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"malformed address", func(_ *ProbeRequest, v *[]View) { (*v)[0].Answers[0].Addresses = []string{"not-an-address"} }, metav1.ConditionUnknown, "DNSProbeFailed"}, + {"NXDOMAIN", func(_ *ProbeRequest, v *[]View) { + for i := range *v { + (*v)[i] = withdrawalView((*v)[i].ID, nil, nil, 600) + (*v)[i].RCode = dns.RcodeNameError + for j := range (*v)[i].Answers { + (*v)[i].Answers[j].RCode = dns.RcodeNameError + } + } + }, metav1.ConditionTrue, "WithdrawalVerified"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + r := request + views := []View{withdrawalView("recursive/192.0.2.53:53", []string{"192.0.2.2"}, nil, 120), withdrawalView("recursive/192.0.2.54:53", []string{"192.0.2.2"}, nil, 600)} + tc.mutate(&r, &views) + decision := EvaluateWithdrawal(r, ProbeResult{Views: views}) + if decision.Status != tc.status || decision.Reason != tc.reason { + t.Fatalf("decision = %#v", decision) + } + if decision.Status == metav1.ConditionTrue && (decision.NegativeTTL == nil || *decision.NegativeTTL != 600) { + t.Fatalf("maximum negative horizon lost: %#v", decision) + } + if len(decision.Message) > 2048 { + t.Fatal("unbounded message") + } + }) + } +} + +func TestWithdrawalEvidenceOrderStable(t *testing.T) { + r := ProbeRequest{Name: "www.example.test", WithdrawalAddress: "192.0.2.1", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{"a", "b"}}}} + result := ProbeResult{Views: []View{withdrawalView("recursive/a", nil, nil, 0), withdrawalView("recursive/b", []string{"192.0.2.3", "192.0.2.2"}, nil, 60)}} + first := EvaluateWithdrawal(r, result) + slices.Reverse(result.Views) + for i := range result.Views { + slices.Reverse(result.Views[i].Answers) + } + second := EvaluateWithdrawal(r, result) + if Fingerprint(first) != Fingerprint(second) { + t.Fatal("resolver order changed withdrawal evidence") + } +} + +func TestWithdrawalAuthorityAndChainEvidence(t *testing.T) { + request := ProbeRequest{Name: "www.example.test", Zone: "example.test", WithdrawalAddress: "192.0.2.1", + Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{"recursive"}}, ExpectedNameservers: []string{"ns.example.test"}}} + request.Evidence = &ZoneEvidence{Zone: request.Zone, PolicyFingerprint: Fingerprint(request.Policy), Authorities: []string{"192.0.2.53:53"}, Views: []AuthorityObservation{ + {ID: "recursive/recursive", Nameservers: []string{"ns.example.test"}, SOAOwner: request.Zone}, + {ID: "authority/192.0.2.53:53", Nameservers: []string{"ns.example.test"}, SOAOwner: request.Zone, Authoritative: true}, + }} + for _, links := range []int{0, 8, 9} { + result := ProbeResult{Views: []View{withdrawalView("recursive/recursive", nil, nil, 60), withdrawalView("authority/192.0.2.53:53/via/recursive", nil, nil, 3600)}} + result.Views[1].Authoritative = true + for i := range result.Views { + view := &result.Views[i] + terminal := view.Answers + view.Answers = nil + owner := request.Name + for n := range links { + target := fmt.Sprintf("link-%d.foreign.test", n) + view.Chain = append(view.Chain, target) + ttl := uint32(7200) + for _, family := range []string{"A", "AAAA"} { + view.Answers = append(view.Answers, AnswerEvidence{Name: owner, QType: family, CNAME: target, PositiveTTL: &ttl}) + } + owner = target + } + for j := range terminal { + terminal[j].Name = owner + } + view.Answers = append(view.Answers, terminal...) + } + decision := EvaluateWithdrawal(request, result) + if links <= 8 && (decision.Status != metav1.ConditionTrue || decision.NegativeTTL == nil || *decision.NegativeTTL != 3600) { + t.Fatalf("links=%d decision=%#v", links, decision) + } + if links > 8 && decision.Status != metav1.ConditionUnknown { + t.Fatal("accepted overlong chain") + } + result.Views = result.Views[:1] + if EvaluateWithdrawal(request, result).Status != metav1.ConditionUnknown { + t.Fatal("missing authority authorized withdrawal") + } + } +} + +func TestWithdrawalQueryCacheEvidence(t *testing.T) { + for _, family := range []uint16{dns.TypeA, dns.TypeAAAA} { + soa := &dns.SOA{Hdr: dns.RR_Header{Name: "example.test.", Rrtype: dns.TypeSOA, Class: dns.ClassINET, Ttl: 3600}, Ns: "ns.example.test.", Mbox: "hostmaster.example.test.", Minttl: 7200} + reply := &dns.Msg{MsgHdr: dns.MsgHdr{Rcode: dns.RcodeNameError}, Ns: []dns.RR{soa}} + evidence := answerEvidence(reply, "www.example.test", family) + if evidence.NegativeTTL == nil || *evidence.NegativeTTL != 3600 || evidence.Failure != "" { + t.Fatalf("negative evidence=%#v", evidence) + } + soa.Minttl = 0 + if got := answerEvidence(reply, "www.example.test", family); got.NegativeTTL == nil || *got.NegativeTTL != 0 { + t.Fatal("zero negative TTL was not retained") + } + soa.Hdr.Name = "unrelated.test." + if got := answerEvidence(reply, "www.example.test", family); got.Failure != "invalid-response" { + t.Fatal("foreign SOA accepted") + } + reply.Ns = nil + if got := answerEvidence(reply, "www.example.test", family); got.NegativeTTL != nil { + t.Fatal("missing SOA invented zero TTL") + } + } + reply := &dns.Msg{Answer: []dns.RR{ + &dns.A{Hdr: dns.RR_Header{Name: "www.example.test.", Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 60}, A: net.ParseIP("192.0.2.1")}, + &dns.A{Hdr: dns.RR_Header{Name: "www.example.test.", Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 3600}, A: net.ParseIP("192.0.2.2")}, + }} + if got := answerEvidence(reply, "www.example.test", dns.TypeA); got.PositiveTTL == nil || *got.PositiveTTL != 3600 { + t.Fatal("positive cache horizon used minimum TTL") + } +} + +func TestWithdrawalFailureRetainsLaterPositiveHorizon(t *testing.T) { + request := ProbeRequest{Name: "www.example.test", WithdrawalAddress: "192.0.2.1", Policy: externalv1alpha1.ExternalZoneClassParameters{Resolver: externalv1alpha1.ResolverPolicy{Servers: []string{"a", "b"}}}} + result := ProbeResult{Views: []View{{ID: "recursive/a", Failure: "timeout"}, withdrawalView("recursive/b", []string{"192.0.2.1"}, nil, 3600)}} + decision := EvaluateWithdrawal(request, result) + if decision.Status != metav1.ConditionUnknown || len(decision.Cache) == 0 || decision.Cache[0].PositiveTTL == nil || *decision.Cache[0].PositiveTTL != 3600 { + t.Fatalf("lost positive evidence=%#v", decision) + } +} + +func TestWithdrawalPendingMetricClosedReasons(t *testing.T) { + const unit = "test-withdrawal-metric" + t.Cleanup(func() { SetWithdrawalsPending(unit, nil) }) + SetWithdrawalsPending(unit, map[string]int{"private/unbounded": 1, "DNSStillResolves": 2}) + pendingMu.Lock() + counts := withdrawalUnits[unit] + if len(counts) != 2 || counts["DNSProbeFailed"] != 1 || counts["DNSStillResolves"] != 2 { + t.Errorf("unbounded reason labels: %#v", counts) + } + pendingMu.Unlock() + SetWithdrawalsPending(unit, nil) + pendingMu.Lock() + _, exists := withdrawalUnits[unit] + pendingMu.Unlock() + if exists { + t.Fatal("deleted unit retained pending metrics") + } +} From eff45445d3db401744b87b4fd48402417bbd346f Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:19:54 +0900 Subject: [PATCH 14/16] feat(endpoint): capture retired DNS pairs before source deletion --- .../endpointrecordset/controller.go | 45 ++- .../endpointrecordset/withdrawal.go | 295 ++++++++++++++++++ .../endpointrecordset/withdrawal_test.go | 244 +++++++++++++++ .../go/apps/endpoint/retirement/retirement.go | 191 ++++++++++++ .../endpoint/retirement/retirement_test.go | 90 ++++++ .../controllers/gateway/controller.go | 23 +- .../controllers/gateway/receipt_sync_test.go | 11 +- .../controllers/gateway/retention_test.go | 25 +- .../controllers/gateway/withdrawal_test.go | 100 ++++++ .../controllers/service/controller.go | 27 +- .../controllers/service/controller_test.go | 4 +- .../controllers/service/withdrawal_test.go | 57 ++++ 12 files changed, 1084 insertions(+), 28 deletions(-) create mode 100644 internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal.go create mode 100644 internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal_test.go create mode 100644 internal/go/apps/endpoint/retirement/retirement.go create mode 100644 internal/go/apps/endpoint/retirement/retirement_test.go create mode 100644 internal/go/apps/gatewayendpoint/controllers/gateway/withdrawal_test.go create mode 100644 internal/go/apps/serviceendpoint/controllers/service/withdrawal_test.go diff --git a/internal/go/apps/endpoint/controllers/endpointrecordset/controller.go b/internal/go/apps/endpoint/controllers/endpointrecordset/controller.go index 6651bc9..f4f88a0 100644 --- a/internal/go/apps/endpoint/controllers/endpointrecordset/controller.go +++ b/internal/go/apps/endpoint/controllers/endpointrecordset/controller.go @@ -15,6 +15,7 @@ import ( endpointconversionv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/conversion/v1alpha1" endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" route53v1alpha1 "github.com/appthrust/dns-api/pkg/go/api/route53/v1alpha1" + apiequality "k8s.io/apimachinery/pkg/api/equality" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -40,7 +41,7 @@ const ( generatedLabelEndpointRecordSetName = "endpoint.dns.appthrust.io/endpointrecordset-name" route53RecordSetAdoptionAnnotation = "endpoint.dns.appthrust.io/route53-recordset-adoption" route53RecordTypeAnnotation = "endpoint.dns.appthrust.io/route53-record-type" - generatedRecordSetsFinalizer = "endpoint.dns.appthrust.io/generated-recordsets" + generatedRecordSetsFinalizer = endpointv1alpha1.GeneratedRecordSetsFinalizer ) type Reconciler struct { @@ -59,6 +60,8 @@ type Reconciler struct { // +kubebuilder:rbac:groups=dns.appthrust.io,resources=zones,verbs=get;list;watch // +kubebuilder:rbac:groups=dns.appthrust.io,resources=zoneunits,verbs=get;list;watch // +kubebuilder:rbac:groups=dns.appthrust.io,resources=recordsets,verbs=create;delete;get;list;watch;patch;update +// +kubebuilder:rbac:groups=dns.appthrust.io,resources=recordwithdrawals,verbs=create;get;list;watch +// +kubebuilder:rbac:groups=dns.appthrust.io,resources=providers,verbs=get;list;watch // +kubebuilder:rbac:groups="",resources=namespaces,verbs=get;list;watch func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { recordSetNamespace := r.RecordSetNamespace @@ -83,11 +86,21 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu } return ctrl.Result{}, nil } + if hasRetirements(&endpointRecordSet) { + if err := r.reconcileRetirements(ctx, &endpointRecordSet, recordSetNamespace); err != nil { + return ctrl.Result{}, err + } + return ctrl.Result{Requeue: !retireOnly(&endpointRecordSet)}, nil + } status, desired, err := r.buildStatusAndRecordSets(ctx, &endpointRecordSet, recordSetNamespace) if err != nil { return ctrl.Result{}, err } + r.annotateWithdrawalStatus(ctx, &endpointRecordSet, &status) + if protectionNeedsBarrier(&endpointRecordSet, status) { + return ctrl.Result{Requeue: true}, r.patchStatus(ctx, &endpointRecordSet, status) + } if err := r.applyRecordSets(ctx, &endpointRecordSet, recordSetNamespace, desired); err != nil { return ctrl.Result{}, err } @@ -95,6 +108,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu return ctrl.Result{}, err } setAggregateConditions(&status, endpointRecordSet.Generation) + r.annotateWithdrawalStatus(ctx, &endpointRecordSet, &status) if err := r.patchStatus(ctx, &endpointRecordSet, status); err != nil { return ctrl.Result{}, err } @@ -109,6 +123,7 @@ func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { Named("endpoint-recordset"). For(&endpointv1alpha1.EndpointRecordSet{}). Watches(&dnsv1alpha1.Zone{}, handler.EnqueueRequestsFromMapFunc(r.mapAllEndpointRecordSets)). + Watches(&dnsv1alpha1.Provider{}, handler.EnqueueRequestsFromMapFunc(r.mapAllEndpointRecordSets)). Watches(&endpointv1alpha1.EndpointProviderCapability{}, handler.EnqueueRequestsFromMapFunc(r.mapAllEndpointRecordSets)). Watches(&dnsv1alpha1.RecordSet{}, handler.EnqueueRequestsFromMapFunc(r.mapRecordSetToEndpointRecordSet)). Complete(r) @@ -537,12 +552,15 @@ func (r *Reconciler) reconcileDeletion(ctx context.Context, endpointRecordSet *e if !slices.Contains(endpointRecordSet.Finalizers, generatedRecordSetsFinalizer) { return nil } + if err := r.captureWithdrawals(ctx, endpointRecordSet, recordSetNamespace, true); err != nil { + return r.captureFailed(ctx, endpointRecordSet, err) + } remaining, err := r.deleteGeneratedRecordSets(ctx, recordSetNamespace, endpointRecordSet.Namespace, endpointRecordSet.Name) if err != nil { return err } if remaining > 0 { - status := endpointRecordSet.Status + status := *endpointRecordSet.Status.DeepCopy() meta.SetStatusCondition(&status.Conditions, metav1.Condition{ Type: "Ready", Status: metav1.ConditionFalse, @@ -556,7 +574,7 @@ func (r *Reconciler) reconcileDeletion(ctx context.Context, endpointRecordSet *e endpointRecordSet.Finalizers = slices.DeleteFunc(endpointRecordSet.Finalizers, func(finalizer string) bool { return finalizer == generatedRecordSetsFinalizer }) - return r.Patch(ctx, endpointRecordSet, client.MergeFrom(base)) + return r.Patch(ctx, endpointRecordSet, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) } func (r *Reconciler) deleteGeneratedRecordSets(ctx context.Context, recordSetNamespace, endpointRecordSetNamespace, endpointRecordSetName string) (int, error) { @@ -639,13 +657,22 @@ func setAggregateCondition(status *endpointv1alpha1.EndpointRecordSetStatus, con } func (r *Reconciler) patchStatus(ctx context.Context, endpointRecordSet *endpointv1alpha1.EndpointRecordSet, status endpointv1alpha1.EndpointRecordSetStatus) error { - var current endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, client.ObjectKeyFromObject(endpointRecordSet), ¤t); err != nil { - return err + base := endpointRecordSet.DeepCopy() + // Normalize against the observed object, rather than reading a potentially + // older cache snapshot after a successful write. Preserve transition times. + conditions := append([]metav1.Condition(nil), endpointRecordSet.Status.Conditions...) + for _, condition := range status.Conditions { + meta.SetStatusCondition(&conditions, condition) + } + conditions = slices.DeleteFunc(conditions, func(condition metav1.Condition) bool { + return !slices.ContainsFunc(status.Conditions, func(desired metav1.Condition) bool { return desired.Type == condition.Type }) + }) + status.Conditions = conditions + if apiequality.Semantic.DeepEqual(endpointRecordSet.Status, status) { + return nil } - base := current.DeepCopy() - current.Status = status - return client.IgnoreNotFound(r.Status().Patch(ctx, ¤t, client.MergeFrom(base))) + endpointRecordSet.Status = status + return client.IgnoreNotFound(r.Status().Patch(ctx, endpointRecordSet, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{}))) } func relativeRecordName(hostname, zoneDomainName string) string { diff --git a/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal.go b/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal.go new file mode 100644 index 0000000..e0eb2e8 --- /dev/null +++ b/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal.go @@ -0,0 +1,295 @@ +package endpointrecordset + +import ( + "context" + "crypto/sha256" + "fmt" + "net/netip" + "slices" + "strconv" + + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + apiequality "k8s.io/apimachinery/pkg/api/equality" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +const ( + withdrawalCreatorAnnotation = "withdrawal.dns.appthrust.io/source-kind" + withdrawalSourceNamespaceAnnotation = "withdrawal.dns.appthrust.io/source-namespace" + withdrawalSourceNameAnnotation = "withdrawal.dns.appthrust.io/source-name" + withdrawalSourceUIDAnnotation = "withdrawal.dns.appthrust.io/source-uid" + withdrawalSourceGenerationAnnotation = "withdrawal.dns.appthrust.io/source-generation" + withdrawalSourceHostnameAnnotation = "withdrawal.dns.appthrust.io/source-hostname" +) + +func hasRetirements(ers *endpointv1alpha1.EndpointRecordSet) bool { + return len(ers.Spec.RetiredTargets) != 0 || len(ers.Spec.RetiredHostnames) != 0 +} + +func retireOnly(ers *endpointv1alpha1.EndpointRecordSet) bool { + return len(ers.Spec.Hostnames) == 0 || len(ers.Spec.Targets) == 0 +} + +// captureWithdrawals performs create-before-prune. Successful Create returns +// the persisted object; AlreadyExists is followed by a shape-checked Get. A +// stale cache miss fails closed and retries without discarding source intent. +func (r *Reconciler) captureWithdrawals(ctx context.Context, ers *endpointv1alpha1.EndpointRecordSet, namespace string, includeLive bool) error { + if len(ers.Spec.RetiredTargets) > retirement.MaxRetiredTargets || len(ers.Spec.RetiredHostnames) > retirement.MaxRetiredHostnames { + return fmt.Errorf("%w: retirement collection limits exceeded", retirement.ErrCapturePending) + } + for _, entry := range ers.Spec.RetiredTargets { + if len(entry.Hostnames) > retirement.MaxRetiredHostnames { + return fmt.Errorf("%w: target hostname snapshot limit exceeded", retirement.ErrCapturePending) + } + } + for _, entry := range ers.Spec.RetiredHostnames { + if len(entry.Addresses) > retirement.MaxRetiredTargets { + return fmt.Errorf("%w: hostname address snapshot limit exceeded", retirement.ErrCapturePending) + } + } + type binding struct { + zone *dnsv1alpha1.Zone + protected bool + } + bindings := map[string]binding{} + captured := map[string]struct{}{} + capture := func(hostname, value string, generation int64) error { + address, err := netip.ParseAddr(value) + if err != nil || address.Zone() != "" { + return fmt.Errorf("%w: invalid retired address %q", retirement.ErrCapturePending, value) + } + address = address.Unmap() + hostname = retirement.CanonicalHostname(hostname) + key := hostname + "/" + address.String() + selected, found := bindings[key] + if !found { + zone, err := r.withdrawalZone(ctx, ers, namespace, hostname, address) + if err != nil { + return err + } + supported, err := r.withdrawalSupported(ctx, zone.Spec.Provider) + if err != nil { + return err + } + selected = binding{zone: zone, protected: supported} + bindings[key] = selected + } + if !selected.protected { + return nil + } + if ers.UID == "" || generation < 1 { + return fmt.Errorf("%w: EndpointRecordSet UID and retirement generation are required", retirement.ErrCapturePending) + } + claim := withdrawalForPair(ers, namespace, selected.zone, hostname, address, generation) + if _, found := captured[claim.Name]; found { + return nil + } + if err := r.ensureWithdrawal(ctx, claim); err != nil { + return err + } + captured[claim.Name] = struct{}{} + return nil + } + for _, entry := range ers.Spec.RetiredTargets { + for _, hostname := range entry.Hostnames { + if err := capture(hostname, entry.Address, entry.SourceGeneration); err != nil { + return err + } + } + } + for _, entry := range ers.Spec.RetiredHostnames { + for _, address := range entry.Addresses { + if err := capture(entry.Hostname, address, entry.SourceGeneration); err != nil { + return err + } + } + } + if includeLive && retirement.Protected(ers) { + for _, target := range ers.Spec.Targets { + if target.Type != endpointv1alpha1.EndpointTargetTypeIPAddress { + continue + } + for _, hostname := range ers.Spec.Hostnames { + if err := capture(hostname, target.Value, ers.Generation); err != nil { + return err + } + } + } + } + return nil +} + +// withdrawalZone prefers the retained publication binding. Without one, run the +// same publication resolver with a nonempty single-pair snapshot, including +// conversion rejection and Zone fallback. This never applies its RecordSets. +func (r *Reconciler) withdrawalZone(ctx context.Context, ers *endpointv1alpha1.EndpointRecordSet, namespace, hostname string, address netip.Addr) (*dnsv1alpha1.Zone, error) { + for _, published := range ers.Status.Hostnames { + if retirement.CanonicalHostname(published.Hostname) != hostname || published.Zone == nil { + continue + } + var zone dnsv1alpha1.Zone + if err := r.Get(ctx, client.ObjectKey{Namespace: published.Zone.Ref.Namespace, Name: published.Zone.Ref.Name}, &zone); err != nil { + return nil, err + } + if zone.Spec.DomainName != published.Zone.DomainName || zone.Spec.Provider != published.Zone.Provider { + return nil, fmt.Errorf("%w: published Zone binding changed for %s", retirement.ErrCapturePending, hostname) + } + return &zone, nil + } + snapshot := *ers + snapshot.Spec.Targets = []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: address.String()}} + resolved, _, err := r.recordSetsForHostname(ctx, &snapshot, hostname, namespace) + if err != nil { + return nil, err + } + if resolved.Zone == nil { + return nil, fmt.Errorf("%w: no publication Zone binding resolved for %s", retirement.ErrCapturePending, hostname) + } + return &dnsv1alpha1.Zone{ + ObjectMeta: metav1.ObjectMeta{Namespace: resolved.Zone.Ref.Namespace, Name: resolved.Zone.Ref.Name}, + Spec: dnsv1alpha1.ZoneSpec{DomainName: resolved.Zone.DomainName, Provider: resolved.Zone.Provider}, + }, nil +} + +func (r *Reconciler) withdrawalSupported(ctx context.Context, ref dnsv1alpha1.ProviderReference) (bool, error) { + var provider dnsv1alpha1.Provider + if err := r.Get(ctx, client.ObjectKey{Name: ref.Name}, &provider); err != nil { + return false, err + } + for _, version := range provider.Spec.Versions { + if version.Name == ref.Version { + return version.RecordWithdrawal != nil && version.RecordWithdrawal.Supported, nil + } + } + return false, fmt.Errorf("Provider %s does not declare version %s", ref.Name, ref.Version) +} + +func withdrawalForPair(ers *endpointv1alpha1.EndpointRecordSet, namespace string, zone *dnsv1alpha1.Zone, hostname string, address netip.Addr, generation int64) *dnsv1alpha1.RecordWithdrawal { + owner := relativeRecordName(hostname, zone.Spec.DomainName) + generationString := strconv.FormatInt(generation, 10) + sum := sha256.Sum256([]byte(string(ers.UID) + "\x00" + generationString + "\x00" + hostname + "\x00" + address.String())) + return &dnsv1alpha1.RecordWithdrawal{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: namespace, Name: fmt.Sprintf("ers-%x", sum[:])[:63], + Labels: map[string]string{dnsv1alpha1.AddressLabelKey: dnsv1alpha1.AddressLabelValue(address)}, + Annotations: map[string]string{ + withdrawalCreatorAnnotation: "EndpointRecordSet", + withdrawalSourceNamespaceAnnotation: ers.Namespace, + withdrawalSourceNameAnnotation: ers.Name, + withdrawalSourceUIDAnnotation: string(ers.UID), + withdrawalSourceGenerationAnnotation: generationString, + withdrawalSourceHostnameAnnotation: hostname, + }, + }, + Spec: dnsv1alpha1.RecordWithdrawalSpec{ZoneRef: dnsv1alpha1.ZoneReference{Namespace: &zone.Namespace, Name: zone.Name}, Name: owner, Address: address.String()}, + } +} + +func (r *Reconciler) ensureWithdrawal(ctx context.Context, desired *dnsv1alpha1.RecordWithdrawal) error { + if err := r.Create(ctx, desired, client.FieldOwner(fieldOwner)); err != nil { + if !apierrors.IsAlreadyExists(err) { + return err + } + var current dnsv1alpha1.RecordWithdrawal + if err := r.Get(ctx, client.ObjectKeyFromObject(desired), ¤t); err != nil { + return err + } + if !current.DeletionTimestamp.IsZero() || len(current.OwnerReferences) != 0 || len(current.Finalizers) != 0 || + !apiequality.Semantic.DeepEqual(current.Spec, desired.Spec) || current.Labels[dnsv1alpha1.AddressLabelKey] != desired.Labels[dnsv1alpha1.AddressLabelKey] { + return fmt.Errorf("RecordWithdrawal collision at %s/%s: immutable shape or lifecycle differs", desired.Namespace, desired.Name) + } + for key, value := range desired.Annotations { + if current.Annotations[key] != value { + return fmt.Errorf("RecordWithdrawal collision at %s/%s: source provenance differs", desired.Namespace, desired.Name) + } + } + } + return nil +} + +func (r *Reconciler) reconcileRetirements(ctx context.Context, ers *endpointv1alpha1.EndpointRecordSet, namespace string) error { + if err := r.captureWithdrawals(ctx, ers, namespace, false); err != nil { + return r.captureFailed(ctx, ers, err) + } + if !retireOnly(ers) { + base := ers.DeepCopy() + ers.Spec.RetiredTargets = nil + ers.Spec.RetiredHostnames = nil + return r.Patch(ctx, ers, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) + } + // Keep captured entries until DELETE: admission requires live OR retired + // intent. Acknowledgment is only capture, never verified DNS withdrawal. + if _, err := r.deleteGeneratedRecordSets(ctx, namespace, ers.Namespace, ers.Name); err != nil { + return err + } + status := *ers.Status.DeepCopy() + status.ObservedGeneration = ers.Generation + setAggregateCondition(&status, retirement.CapturedCondition, metav1.ConditionTrue, ers.Generation, "Captured", "Every retired hostname/address pair has been durably captured.") + for _, conditionType := range []string{"Resolved", "Accepted", "Programmed", "Ready"} { + setAggregateCondition(&status, conditionType, metav1.ConditionFalse, ers.Generation, "Retiring", "EndpointRecordSet is retiring; no active records are published.") + } + return r.patchStatus(ctx, ers, status) +} + +func (r *Reconciler) captureFailed(ctx context.Context, ers *endpointv1alpha1.EndpointRecordSet, cause error) error { + status := *ers.Status.DeepCopy() + message := cause.Error() + if len(message) > 2048 { + message = message[:2048] + } + setAggregateCondition(&status, retirement.CapturedCondition, metav1.ConditionFalse, ers.Generation, retirement.PendingCondition, message) + setAggregateCondition(&status, "Ready", metav1.ConditionFalse, ers.Generation, retirement.PendingCondition, message) + if err := r.patchStatus(ctx, ers, status); err != nil { + return err + } + return cause +} + +// annotateWithdrawalStatus publishes the marker consumed by provider-neutral +// sources. True is sticky for an ERS incarnation. Unknown never permits loss. +func (r *Reconciler) annotateWithdrawalStatus(ctx context.Context, ers *endpointv1alpha1.EndpointRecordSet, status *endpointv1alpha1.EndpointRecordSetStatus) { + protection := metav1.ConditionFalse + if meta.IsStatusConditionTrue(ers.Status.Conditions, retirement.ProtectedCondition) { + protection = metav1.ConditionTrue + } else { + if len(status.Hostnames) != len(ers.Spec.Hostnames) || len(status.Hostnames) == 0 { + protection = metav1.ConditionUnknown + } + for _, hostname := range status.Hostnames { + if hostname.Zone == nil { + protection = metav1.ConditionUnknown + continue + } + supported, err := r.withdrawalSupported(ctx, hostname.Zone.Provider) + if err != nil { + protection = metav1.ConditionUnknown + continue + } + if supported { + protection = metav1.ConditionTrue + break + } + } + } + setAggregateCondition(status, retirement.ProtectedCondition, protection, ers.Generation, "ProviderCapability", "Withdrawal protection follows the resolved Provider recordWithdrawal capability; unknown bindings retain intent.") + setAggregateCondition(status, retirement.CapturedCondition, metav1.ConditionTrue, ers.Generation, "Captured", "No uncaptured retirement entries remain.") + if pending := meta.FindStatusCondition(ers.Status.Conditions, retirement.PendingCondition); pending != nil { + meta.SetStatusCondition(&status.Conditions, *pending) + if pending.Status == metav1.ConditionTrue { + setAggregateCondition(status, "Ready", metav1.ConditionFalse, ers.Generation, retirement.PendingCondition, pending.Message) + } + } +} + +func protectionNeedsBarrier(ers *endpointv1alpha1.EndpointRecordSet, status endpointv1alpha1.EndpointRecordSetStatus) bool { + old := meta.FindStatusCondition(ers.Status.Conditions, retirement.ProtectedCondition) + return old != nil && old.Status == metav1.ConditionFalse && old.ObservedGeneration == ers.Generation && + !slices.ContainsFunc(status.Conditions, func(condition metav1.Condition) bool { + return condition.Type == retirement.ProtectedCondition && condition.Status == metav1.ConditionFalse + }) +} diff --git a/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal_test.go b/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal_test.go new file mode 100644 index 0000000..7a3fb41 --- /dev/null +++ b/internal/go/apps/endpoint/controllers/endpointrecordset/withdrawal_test.go @@ -0,0 +1,244 @@ +package endpointrecordset + +import ( + "context" + "errors" + "fmt" + "net/netip" + "strings" + "testing" + + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" + dnsv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/dns/v1alpha1" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + "sigs.k8s.io/controller-runtime/pkg/client/interceptor" +) + +func withdrawalFixture(t *testing.T, supported bool, intercept interceptor.Funcs) (*Reconciler, *endpointv1alpha1.EndpointRecordSet, *dnsv1alpha1.Zone) { + t.Helper() + scheme := runtime.NewScheme() + if err := endpointv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + if err := dnsv1alpha1.AddToScheme(scheme); err != nil { + t.Fatal(err) + } + ref := dnsv1alpha1.ProviderReference{Name: "test.dns.appthrust.io", Version: "v1alpha1"} + provider := &dnsv1alpha1.Provider{ObjectMeta: metav1.ObjectMeta{Name: ref.Name}, Spec: dnsv1alpha1.ProviderSpec{Versions: []dnsv1alpha1.ProviderVersion{{Name: ref.Version, RecordWithdrawal: &dnsv1alpha1.ProviderRecordWithdrawal{Supported: supported}}}}} + zone := &dnsv1alpha1.Zone{ObjectMeta: metav1.ObjectMeta{Namespace: "dns", Name: "example", UID: "zone-uid"}, Spec: dnsv1alpha1.ZoneSpec{DomainName: "example.com", Provider: ref}} + capability := &endpointv1alpha1.EndpointProviderCapability{ObjectMeta: metav1.ObjectMeta{Name: "test"}, Spec: endpointv1alpha1.EndpointProviderCapabilitySpec{Provider: ref}} + ers := &endpointv1alpha1.EndpointRecordSet{ + ObjectMeta: metav1.ObjectMeta{Namespace: "dns", Name: "ers", UID: "ers-uid", Generation: 8, Finalizers: []string{generatedRecordSetsFinalizer}}, + Spec: endpointv1alpha1.EndpointRecordSetSpec{ + Hostnames: []string{"h2.example.com"}, + Targets: []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.2"}}, + RetiredTargets: []endpointv1alpha1.RetiredTarget{{ID: "target-old", SourceGeneration: 7, Address: "192.0.2.1", Hostnames: []string{"h1.example.com", "h2.example.com"}}}, + RetiredHostnames: []endpointv1alpha1.RetiredHostname{{ID: "hostname-old", SourceGeneration: 7, Hostname: "h1.example.com", Addresses: []string{"192.0.2.1"}}}, + }, + } + c := fake.NewClientBuilder().WithScheme(scheme).WithStatusSubresource(&endpointv1alpha1.EndpointRecordSet{}, &dnsv1alpha1.RecordWithdrawal{}). + WithObjects(provider, zone, capability, ers).WithInterceptorFuncs(intercept).Build() + return &Reconciler{Client: c, Scheme: scheme, conversionFunc: func(_ context.Context, _ *endpointv1alpha1.EndpointProviderCapability, input endpointv1alpha1.EndpointRecordSetConversionInput) ([]endpointv1alpha1.RecordSetSpecFragment, string, error) { + if len(input.Targets) != 1 { + t.Fatal("retirement invoked conversion without a nonempty pair snapshot") + } + return []endpointv1alpha1.RecordSetSpecFragment{{Type: endpointv1alpha1.EndpointRecordSetTypeA, Name: input.Name, A: &dnsv1alpha1.ARecordSet{Addresses: []string{input.Targets[0].Value}}}}, "", nil + }}, ers, zone +} + +func getWithdrawalERS(t *testing.T, c client.Client, key client.ObjectKey) *endpointv1alpha1.EndpointRecordSet { + t.Helper() + var ers endpointv1alpha1.EndpointRecordSet + if err := c.Get(context.Background(), key, &ers); err != nil { + t.Fatal(err) + } + return &ers +} + +func listWithdrawalClaims(t *testing.T, c client.Client) []dnsv1alpha1.RecordWithdrawal { + t.Helper() + var claims dnsv1alpha1.RecordWithdrawalList + if err := c.List(context.Background(), &claims); err != nil { + t.Fatal(err) + } + return claims.Items +} + +func TestWithdrawalCreateBeforePruneSurvivesCrash(t *testing.T) { + ctx := context.Background() + crash := true + r, ers, _ := withdrawalFixture(t, true, interceptor.Funcs{Patch: func(ctx context.Context, c client.WithWatch, obj client.Object, patch client.Patch, opts ...client.PatchOption) error { + if _, ok := obj.(*endpointv1alpha1.EndpointRecordSet); ok && crash { + crash = false + return errors.New("simulated crash after claim persistence") + } + return c.Patch(ctx, obj, patch, opts...) + }}) + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err == nil || !strings.Contains(err.Error(), "simulated crash") { + t.Fatalf("first reconcile = %v", err) + } + current := getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + if !hasRetirements(current) { + t.Fatal("crash discarded source obligations") + } + claims := listWithdrawalClaims(t, r.Client) + if len(claims) != 2 { + t.Fatalf("claims = %d, want deduplicated (H1,V1), (H2,V1)", len(claims)) + } + for _, claim := range claims { + if claim.Spec.Address != "192.0.2.1" || claim.Labels[dnsv1alpha1.AddressLabelKey] != "192.0.2.1" || claim.Annotations[withdrawalSourceGenerationAnnotation] != "7" || len(claim.OwnerReferences) != 0 || len(claim.Finalizers) != 0 { + t.Fatalf("incorrect retained claim: %#v", claim) + } + } + restarted := &Reconciler{Client: r.Client, Scheme: r.Scheme, conversionFunc: r.conversionFunc} + if _, err := restarted.Reconcile(ctx, ctrlRequest(ers)); err != nil { + t.Fatal(err) + } + current = getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + if hasRetirements(current) || len(listWithdrawalClaims(t, r.Client)) != 2 || current.Spec.Targets[0].Value != "192.0.2.2" { + t.Fatal("restart failed idempotent create-then-prune") + } +} + +func TestWithdrawalPruneConflictPreservesConcurrentEntry(t *testing.T) { + ctx := context.Background() + interleave := true + r, ers, _ := withdrawalFixture(t, true, interceptor.Funcs{Patch: func(ctx context.Context, c client.WithWatch, obj client.Object, patch client.Patch, opts ...client.PatchOption) error { + if _, ok := obj.(*endpointv1alpha1.EndpointRecordSet); ok && interleave { + interleave = false + var current endpointv1alpha1.EndpointRecordSet + if err := c.Get(ctx, client.ObjectKeyFromObject(obj), ¤t); err != nil { return err } + current.Spec.RetiredTargets = append(current.Spec.RetiredTargets, endpointv1alpha1.RetiredTarget{ID: "concurrent", SourceGeneration: 8, Address: "192.0.2.3", Hostnames: []string{"h2.example.com"}}) + if err := c.Update(ctx, ¤t); err != nil { return err } + } + return c.Patch(ctx, obj, patch, opts...) + }}) + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); !apierrors.IsConflict(err) { + t.Fatalf("prune = %v, want resourceVersion conflict", err) + } + current := getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + if len(current.Spec.RetiredTargets) != 2 { + t.Fatal("concurrent entry was lost") + } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + if len(listWithdrawalClaims(t, r.Client)) != 3 || hasRetirements(getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers))) { + t.Fatal("retry did not capture concurrent entry") + } +} + +func TestWithdrawalDirectDeleteCapturesLiveAndRetired(t *testing.T) { + ctx := context.Background() + r, ers, _ := withdrawalFixture(t, true, interceptor.Funcs{}) + if err := r.Delete(ctx, ers); err != nil { t.Fatal(err) } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + var current endpointv1alpha1.EndpointRecordSet + if err := r.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); !apierrors.IsNotFound(err) { + t.Fatalf("ERS after capture = %v", err) + } + claims := listWithdrawalClaims(t, r.Client) + if len(claims) != 3 { t.Fatalf("claims = %d, want old pairs plus (H2,V2)", len(claims)) } +} + +func TestWithdrawalRetireOnlyAcknowledgesWithoutEmptyPublication(t *testing.T) { + ctx := context.Background() + r, ers, _ := withdrawalFixture(t, true, interceptor.Funcs{}) + ers.Spec.Hostnames = nil + ers.Spec.Targets = nil + if err := r.Update(ctx, ers); err != nil { t.Fatal(err) } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + current := getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + ready := meta.FindStatusCondition(current.Status.Conditions, "Ready") + if !retirement.Captured(current) || !hasRetirements(current) || ready == nil || ready.Status != metav1.ConditionFalse || ready.Reason != "Retiring" { + t.Fatalf("retire-only status = %#v", current.Status) + } + before := current.ResourceVersion + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + if getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)).ResourceVersion != before { + t.Fatal("stable capture acknowledgment caused status churn") + } + if err := retirement.RetireOrDelete(ctx, r.Client, current); err != nil { t.Fatal(err) } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + if len(listWithdrawalClaims(t, r.Client)) != 2 { t.Fatal("deletion changed retirement identity") } +} + +func TestWithdrawalCollisionAndCreateFailureRetainIntent(t *testing.T) { + for _, failure := range []string{"shape", "source", "create"} { + t.Run(failure, func(t *testing.T) { + ctx := context.Background() + r, ers, zone := withdrawalFixture(t, true, interceptor.Funcs{Create: func(ctx context.Context, c client.WithWatch, obj client.Object, opts ...client.CreateOption) error { + if _, ok := obj.(*dnsv1alpha1.RecordWithdrawal); ok && failure == "create" { return fmt.Errorf("create denied") } + return c.Create(ctx, obj, opts...) + }}) + if failure != "create" { + claim := withdrawalForPair(ers, "dns", zone, "h1.example.com", netip.MustParseAddr("192.0.2.1"), 7) + if failure == "shape" { claim.Spec.Address = "192.0.2.99" } else { claim.Annotations[withdrawalSourceUIDAnnotation] = "foreign" } + if err := r.Create(ctx, claim); err != nil { t.Fatal(err) } + } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err == nil { t.Fatal("unsafe capture succeeded") } + current := getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + ready := meta.FindStatusCondition(current.Status.Conditions, "Ready") + if !hasRetirements(current) || ready == nil || ready.Reason != retirement.PendingCondition { t.Fatal("failure discarded or hid capture obligation") } + }) + } +} + +func TestWithdrawalNonExternalPrunesWithoutClaims(t *testing.T) { + r, ers, _ := withdrawalFixture(t, false, interceptor.Funcs{}) + if _, err := r.Reconcile(context.Background(), ctrlRequest(ers)); err != nil { t.Fatal(err) } + if len(listWithdrawalClaims(t, r.Client)) != 0 || hasRetirements(getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers))) { + t.Fatal("non-external retirement gained an external wait") + } +} + +func TestWithdrawalOverflowReportsPending(t *testing.T) { + r, ers, _ := withdrawalFixture(t, true, interceptor.Funcs{}) + for len(ers.Spec.RetiredTargets) <= retirement.MaxRetiredTargets { + ers.Spec.RetiredTargets = append(ers.Spec.RetiredTargets, ers.Spec.RetiredTargets[0]) + } + if err := r.Update(context.Background(), ers); err != nil { t.Fatal(err) } + if _, err := r.Reconcile(context.Background(), ctrlRequest(ers)); !errors.Is(err, retirement.ErrCapturePending) { t.Fatalf("overflow = %v", err) } + current := getWithdrawalERS(t, r.Client, client.ObjectKeyFromObject(ers)) + if len(current.Spec.RetiredTargets) != 65 || len(listWithdrawalClaims(t, r.Client)) != 0 || meta.FindStatusCondition(current.Status.Conditions, "Ready").Reason != retirement.PendingCondition { + t.Fatal("overflow was truncated or acknowledged") + } +} + +func TestWithdrawalResolutionUsesPublicationFallback(t *testing.T) { + ctx := context.Background() + r, ers, zone := withdrawalFixture(t, true, interceptor.Funcs{}) + narrow := zone.DeepCopy() + narrow.Name = "narrow" + narrow.UID = "narrow-zone" + narrow.ResourceVersion = "" + narrow.Spec.DomainName = "h1.example.com" + if err := r.Create(ctx, narrow); err != nil { t.Fatal(err) } + baseConversion := r.conversionFunc + r.conversionFunc = func(ctx context.Context, capability *endpointv1alpha1.EndpointProviderCapability, input endpointv1alpha1.EndpointRecordSetConversionInput) ([]endpointv1alpha1.RecordSetSpecFragment, string, error) { + if input.Zone.DomainName == narrow.Spec.DomainName { return nil, "conversion rejected narrow zone", nil } + return baseConversion(ctx, capability, input) + } + if _, err := r.Reconcile(ctx, ctrlRequest(ers)); err != nil { t.Fatal(err) } + for _, claim := range listWithdrawalClaims(t, r.Client) { + if claim.Spec.ZoneRef.Name != zone.Name { t.Fatalf("retirement bypassed publication fallback: %#v", claim.Spec) } + } +} + +func TestWithdrawalProtectionMarkerFollowsCapability(t *testing.T) { + ctx := context.Background() + r, ers, zone := withdrawalFixture(t, true, interceptor.Funcs{}) + ers.Spec.RetiredTargets = nil + ers.Spec.RetiredHostnames = nil + ers.Status.Conditions = []metav1.Condition{{Type:retirement.ProtectedCondition, Status:metav1.ConditionFalse, ObservedGeneration:ers.Generation}} + status := endpointv1alpha1.EndpointRecordSetStatus{Hostnames:[]endpointv1alpha1.EndpointRecordSetHostnameStatus{{Hostname:"h2.example.com", Zone:&endpointv1alpha1.EndpointRecordSetZoneStatus{Ref:dnsv1alpha1.ObjectReference{Namespace:zone.Namespace, Name:zone.Name}, DomainName:zone.Spec.DomainName, Provider:zone.Spec.Provider}}}} + r.annotateWithdrawalStatus(ctx, ers, &status) + if !meta.IsStatusConditionTrue(status.Conditions, retirement.ProtectedCondition) || !protectionNeedsBarrier(ers, status) { t.Fatal("new protected publication lacks a source-retention barrier") } + ers.Status = status + r.annotateWithdrawalStatus(ctx, ers, &status) + if protectionNeedsBarrier(ers, status) { t.Fatal("stable protected marker does not converge") } +} diff --git a/internal/go/apps/endpoint/retirement/retirement.go b/internal/go/apps/endpoint/retirement/retirement.go new file mode 100644 index 0000000..70b33b7 --- /dev/null +++ b/internal/go/apps/endpoint/retirement/retirement.go @@ -0,0 +1,191 @@ +// Package retirement implements the provider-neutral source side of DEP-0003. +package retirement + +import ( + "context" + "crypto/sha256" + "errors" + "fmt" + "net/netip" + "slices" + "strconv" + "strings" + + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +const ( + ProtectedCondition = "WithdrawalProtected" + CapturedCondition = "WithdrawalCaptured" + PendingCondition = "WithdrawalCapturePending" + MaxRetiredTargets = 64 + MaxRetiredHostnames = 256 +) + +var ErrCapturePending = errors.New("WithdrawalCapturePending") + +// Protected defaults closed until the endpoint controller has resolved every +// hostname to a Provider without withdrawal support at this exact generation. +func Protected(ers *endpointv1alpha1.EndpointRecordSet) bool { + condition := meta.FindStatusCondition(ers.Status.Conditions, ProtectedCondition) + return condition == nil || condition.Status != metav1.ConditionFalse || condition.ObservedGeneration != ers.Generation +} + +func Captured(ers *endpointv1alpha1.EndpointRecordSet) bool { + condition := meta.FindStatusCondition(ers.Status.Conditions, CapturedCondition) + return condition != nil && condition.Status == metav1.ConditionTrue && condition.ObservedGeneration == ers.Generation +} + +// Append keeps every existing retirement entry and snapshots old associations. +// It does not mutate current or desired, including on overflow. Generation is +// the pre-update ERS generation, so repeated writes and duplicate collections +// identify the same retirement while a later re-add/remove identifies a new one. +func Append(current *endpointv1alpha1.EndpointRecordSet, desired endpointv1alpha1.EndpointRecordSetSpec) (endpointv1alpha1.EndpointRecordSetSpec, error) { + out := desired + out.RetiredTargets = slices.Clone(current.Spec.RetiredTargets) + out.RetiredHostnames = slices.Clone(current.Spec.RetiredHostnames) + if !Protected(current) { + return out, nil + } + oldAddresses, err := addresses(current.Spec.Targets) + if err != nil { + return out, err + } + newAddresses, err := addresses(desired.Targets) + if err != nil { + return out, err + } + if len(oldAddresses) == 0 || len(current.Spec.Hostnames) == 0 { + return out, nil + } + removedAddresses := make([]string, 0) + for _, address := range oldAddresses { + if !slices.Contains(newAddresses, address) || len(desired.Hostnames) == 0 { + removedAddresses = append(removedAddresses, address) + } + } + removedHostnames := make([]string, 0) + for _, hostname := range current.Spec.Hostnames { + if !slices.Contains(desired.Hostnames, hostname) || len(newAddresses) == 0 { + removedHostnames = append(removedHostnames, hostname) + } + } + if len(removedAddresses) == 0 && len(removedHostnames) == 0 { + return out, nil + } + if current.UID == "" || current.Generation < 1 { + return out, fmt.Errorf("%w: EndpointRecordSet UID and generation are required", ErrCapturePending) + } + if len(current.Spec.Hostnames) > MaxRetiredHostnames || len(oldAddresses) > MaxRetiredTargets || + len(out.RetiredTargets)+len(removedAddresses) > MaxRetiredTargets || + len(out.RetiredHostnames)+len(removedHostnames) > MaxRetiredHostnames { + return out, fmt.Errorf("%w: retirement snapshot limits exceeded; old intent retained", ErrCapturePending) + } + for _, address := range removedAddresses { + out.RetiredTargets = append(out.RetiredTargets, endpointv1alpha1.RetiredTarget{ + ID: entryID(current, "target", address), SourceGeneration: current.Generation, + Address: address, Hostnames: slices.Clone(current.Spec.Hostnames), + }) + } + for _, hostname := range removedHostnames { + out.RetiredHostnames = append(out.RetiredHostnames, endpointv1alpha1.RetiredHostname{ + ID: entryID(current, "hostname", hostname), SourceGeneration: current.Generation, + Hostname: hostname, Addresses: slices.Clone(oldAddresses), + }) + } + return out, nil +} + +func entryID(ers *endpointv1alpha1.EndpointRecordSet, kind, value string) string { + sum := sha256.Sum256([]byte(string(ers.UID) + "\x00" + strconv.FormatInt(ers.Generation, 10) + "\x00" + kind + "\x00" + value)) + return fmt.Sprintf("%x", sum[:]) +} + +func addresses(targets []endpointv1alpha1.EndpointTarget) ([]string, error) { + var out []string + for _, target := range targets { + if target.Type != endpointv1alpha1.EndpointTargetTypeIPAddress { + continue + } + address, err := netip.ParseAddr(target.Value) + if err != nil || address.Zone() != "" { + return nil, fmt.Errorf("%w: invalid IP target %q", ErrCapturePending, target.Value) + } + value := address.Unmap().String() + if !slices.Contains(out, value) { + out = append(out, value) + } + } + return out, nil +} + +// ReportPending leaves the old spec untouched and makes overflow visible. The +// source owns only PendingCondition; Ready is also projected by endpoint. +func ReportPending(ctx context.Context, c client.Client, ers *endpointv1alpha1.EndpointRecordSet, cause error) error { + base := ers.DeepCopy() + message := cause.Error() + if len(message) > 2048 { + message = message[:2048] + } + for _, conditionType := range []string{PendingCondition, "Ready"} { + status := metav1.ConditionTrue + if conditionType == "Ready" { + status = metav1.ConditionFalse + } + meta.SetStatusCondition(&ers.Status.Conditions, metav1.Condition{Type: conditionType, Status: status, ObservedGeneration: ers.Generation, Reason: PendingCondition, Message: message}) + } + if err := c.Status().Patch(ctx, ers, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})); err != nil { + return err + } + return cause +} + +// ClearPending is called after a source can represent its complete desired +// transition. It never acknowledges capture or prunes retirement intent. +func ClearPending(ctx context.Context, c client.Client, ers *endpointv1alpha1.EndpointRecordSet) error { + if !meta.IsStatusConditionTrue(ers.Status.Conditions, PendingCondition) { + return nil + } + base := ers.DeepCopy() + meta.RemoveStatusCondition(&ers.Status.Conditions, PendingCondition) + return c.Status().Patch(ctx, ers, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})) +} + +// RetireOrDelete is shared by every source cleanup path. A protected ERS is +// deleted only after a current-generation endpoint capture acknowledgment. +func RetireOrDelete(ctx context.Context, c client.Client, ers *endpointv1alpha1.EndpointRecordSet) error { + if !ers.DeletionTimestamp.IsZero() { + return nil + } + if !Protected(ers) || (len(ers.Spec.Hostnames) == 0 || len(ers.Spec.Targets) == 0) && Captured(ers) { + return deleteExact(ctx, c, ers) + } + spec, err := Append(ers, endpointv1alpha1.EndpointRecordSetSpec{}) + if err != nil { + return ReportPending(ctx, c, ers, err) + } + if len(spec.RetiredTargets) == 0 && len(spec.RetiredHostnames) == 0 { + // Hostname targets do not carry address withdrawal obligations. + return deleteExact(ctx, c, ers) + } + if len(ers.Spec.Hostnames) != 0 || len(ers.Spec.Targets) != 0 { + base := ers.DeepCopy() + ers.Spec = spec + if err := c.Patch(ctx, ers, client.MergeFromWithOptions(base, client.MergeFromWithOptimisticLock{})); err != nil { + return err + } + } + return fmt.Errorf("%w: waiting for EndpointRecordSet %s/%s capture", ErrCapturePending, ers.Namespace, ers.Name) +} + +func deleteExact(ctx context.Context, c client.Client, ers *endpointv1alpha1.EndpointRecordSet) error { + return client.IgnoreNotFound(c.Delete(ctx, ers, client.Preconditions{UID: &ers.UID, ResourceVersion: &ers.ResourceVersion})) +} + +// CanonicalHostname is shared by claim identity and source snapshot consumers. +func CanonicalHostname(hostname string) string { + return strings.ToLower(strings.TrimSuffix(hostname, ".")) +} diff --git a/internal/go/apps/endpoint/retirement/retirement_test.go b/internal/go/apps/endpoint/retirement/retirement_test.go new file mode 100644 index 0000000..68b2a55 --- /dev/null +++ b/internal/go/apps/endpoint/retirement/retirement_test.go @@ -0,0 +1,90 @@ +package retirement + +import ( + "errors" + "fmt" + "reflect" + "testing" + + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func retirementFixture() *endpointv1alpha1.EndpointRecordSet { + return &endpointv1alpha1.EndpointRecordSet{ + ObjectMeta: metav1.ObjectMeta{Name: "ers", Namespace: "dns", UID: "ers-uid", Generation: 5}, + Spec: endpointv1alpha1.EndpointRecordSetSpec{ + Hostnames: []string{"h1.example.com", "h2.example.com"}, + Targets: []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.1"}}, + }, + } +} + +func TestWithdrawalSnapshotPreservesCorrelatedPairs(t *testing.T) { + current := retirementFixture() + desired := endpointv1alpha1.EndpointRecordSetSpec{Hostnames: []string{"h2.example.com"}, Targets: []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.2"}}} + got, err := Append(current, desired) + if err != nil { + t.Fatal(err) + } + if len(got.RetiredTargets) != 1 || len(got.RetiredHostnames) != 1 { + t.Fatalf("snapshot = %#v", got) + } + if got.RetiredTargets[0].Address != "192.0.2.1" || !reflect.DeepEqual(got.RetiredTargets[0].Hostnames, current.Spec.Hostnames) || + got.RetiredHostnames[0].Hostname != "h1.example.com" || !reflect.DeepEqual(got.RetiredHostnames[0].Addresses, []string{"192.0.2.1"}) { + t.Fatalf("lost old pair or invented (H1,V2): %#v", got) + } + if got.RetiredTargets[0].SourceGeneration != 5 || got.RetiredHostnames[0].SourceGeneration != 5 { + t.Fatal("snapshot generation was not the pre-update ERS generation") + } + got.RetiredTargets[0].Hostnames[0] = "changed" + if current.Spec.Hostnames[0] != "h1.example.com" { + t.Fatal("snapshot aliases live source") + } + current.Generation++ + later, err := Append(current, desired) + if err != nil || later.RetiredTargets[0].ID == got.RetiredTargets[0].ID { + t.Fatalf("re-add/remove reused retirement identity: %v", err) + } +} + +func TestWithdrawalSnapshotOverflowRetainsOldIntent(t *testing.T) { + for _, kind := range []string{"targets", "hostnames", "hostname snapshot", "address snapshot"} { + t.Run(kind, func(t *testing.T) { + current := retirementFixture() + switch kind { + case "targets": + current.Spec.RetiredTargets = make([]endpointv1alpha1.RetiredTarget, MaxRetiredTargets) + case "hostnames": + current.Spec.RetiredHostnames = make([]endpointv1alpha1.RetiredHostname, MaxRetiredHostnames) + case "hostname snapshot": + for i := len(current.Spec.Hostnames); i <= MaxRetiredHostnames; i++ { + current.Spec.Hostnames = append(current.Spec.Hostnames, fmt.Sprintf("h%d.example.com", i)) + } + case "address snapshot": + for i := 2; i <= MaxRetiredTargets+1; i++ { + current.Spec.Targets = append(current.Spec.Targets, endpointv1alpha1.EndpointTarget{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: fmt.Sprintf("192.0.2.%d", i)}) + } + } + before := current.DeepCopy() + _, err := Append(current, endpointv1alpha1.EndpointRecordSetSpec{}) + if !errors.Is(err, ErrCapturePending) || !reflect.DeepEqual(current, before) { + t.Fatalf("overflow mutated intent or did not hold: %v", err) + } + }) + } +} + +func TestWithdrawalSnapshotNonExternalAndStaleMarker(t *testing.T) { + current := retirementFixture() + current.Status.Conditions = []metav1.Condition{{Type: ProtectedCondition, Status: metav1.ConditionFalse, ObservedGeneration: current.Generation}} + got, err := Append(current, endpointv1alpha1.EndpointRecordSetSpec{}) + if err != nil || len(got.RetiredTargets)+len(got.RetiredHostnames) != 0 { + t.Fatalf("non-external retirement changed: %#v, %v", got, err) + } + current.Generation++ + got, err = Append(current, endpointv1alpha1.EndpointRecordSetSpec{}) + if err != nil || len(got.RetiredTargets) != 1 { + t.Fatalf("stale marker dropped an obligation: %#v, %v", got, err) + } +} diff --git a/internal/go/apps/gatewayendpoint/controllers/gateway/controller.go b/internal/go/apps/gatewayendpoint/controllers/gateway/controller.go index 9d099a6..a31a7fb 100644 --- a/internal/go/apps/gatewayendpoint/controllers/gateway/controller.go +++ b/internal/go/apps/gatewayendpoint/controllers/gateway/controller.go @@ -8,17 +8,16 @@ import ( "sort" "strings" + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" apiequality "k8s.io/apimachinery/pkg/api/equality" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/builder" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/handler" - "sigs.k8s.io/controller-runtime/pkg/predicate" "sigs.k8s.io/controller-runtime/pkg/reconcile" gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" ) @@ -46,6 +45,7 @@ type Reconciler struct { // +kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=httproutes,verbs=get;list;watch;update // +kubebuilder:rbac:groups=gateway.networking.k8s.io,resources=httproutes/finalizers,verbs=update // +kubebuilder:rbac:groups=endpoint.dns.appthrust.io,resources=endpointrecordsets,verbs=create;delete;get;list;watch;patch;update +// +kubebuilder:rbac:groups=endpoint.dns.appthrust.io,resources=endpointrecordsets/status,verbs=get;patch;update func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { namespace := r.EndpointRecordSetNamespace if namespace == "" { @@ -105,8 +105,7 @@ func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { Watches(&gatewayv1.Gateway{}, handler.EnqueueRequestsFromMapFunc(r.mapGatewayToHTTPRoutes)). // Cross-namespace projections cannot use owner references. The final // delete event must wake routes even when their own state has not changed. - Watches(&endpointv1alpha1.EndpointRecordSet{}, handler.EnqueueRequestsFromMapFunc(r.mapEndpointRecordSetToHTTPRoutes), - builder.WithPredicates(predicate.Or(predicate.GenerationChangedPredicate{}, predicate.LabelChangedPredicate{}))). + Watches(&endpointv1alpha1.EndpointRecordSet{}, handler.EnqueueRequestsFromMapFunc(r.mapEndpointRecordSetToHTTPRoutes)). Complete(r) } @@ -133,6 +132,9 @@ func (r *Reconciler) mapEndpointRecordSetToHTTPRoutes(ctx context.Context, obj c if labels[managedByLabel] != managedByValue { return nil } + if labels[generatedLabelRouteNamespace] != "" && labels[generatedLabelRouteName] != "" { + return []reconcile.Request{{NamespacedName: client.ObjectKey{Namespace: labels[generatedLabelRouteNamespace], Name: labels[generatedLabelRouteName]}}} + } key := client.ObjectKey{Namespace: labels[generatedLabelGatewayNamespace], Name: labels[generatedLabelGatewayName]} if key.Namespace == "" || key.Name == "" { return nil @@ -350,6 +352,7 @@ func (r *Reconciler) applyGatewayEndpointRecordSets(ctx context.Context, namespa var existing endpointv1alpha1.EndpointRecordSet err := r.Get(ctx, client.ObjectKey{Namespace: desired.Namespace, Name: desired.Name}, &existing) if apierrors.IsNotFound(err) { + controllerutil.AddFinalizer(&desired, endpointv1alpha1.GeneratedRecordSetsFinalizer) if err := r.Create(ctx, &desired, client.FieldOwner(fieldOwner)); err != nil { return err } @@ -363,6 +366,14 @@ func (r *Reconciler) applyGatewayEndpointRecordSets(ctx context.Context, namespa // provider cleanup; the EndpointRecordSet delete watch retries creation. continue } + spec, err := retirement.Append(&existing, desired.Spec) + if err != nil { + return retirement.ReportPending(ctx, r.Client, &existing, err) + } + desired.Spec = spec + if err := retirement.ClearPending(ctx, r.Client, &existing); err != nil { + return err + } desiredReceipt := desired.Annotations[admissionReceiptAnnotation] if apiequality.Semantic.DeepEqual(existing.Labels, desired.Labels) && apiequality.Semantic.DeepEqual(existing.Spec, desired.Spec) && @@ -396,7 +407,7 @@ func (r *Reconciler) applyGatewayEndpointRecordSets(ctx context.Context, namespa continue } item := item - if err := r.Delete(ctx, &item); err != nil && !apierrors.IsNotFound(err) { + if err := retirement.RetireOrDelete(ctx, r.Client, &item); err != nil && !apierrors.IsNotFound(err) { return err } } @@ -414,7 +425,7 @@ func (r *Reconciler) cleanupForRoute(ctx context.Context, endpointRecordSetNames } for _, item := range existing.Items { item := item - if err := r.Delete(ctx, &item); err != nil && !apierrors.IsNotFound(err) { + if err := retirement.RetireOrDelete(ctx, r.Client, &item); err != nil && !apierrors.IsNotFound(err) { return err } } diff --git a/internal/go/apps/gatewayendpoint/controllers/gateway/receipt_sync_test.go b/internal/go/apps/gatewayendpoint/controllers/gateway/receipt_sync_test.go index 357ebf4..283d402 100644 --- a/internal/go/apps/gatewayendpoint/controllers/gateway/receipt_sync_test.go +++ b/internal/go/apps/gatewayendpoint/controllers/gateway/receipt_sync_test.go @@ -5,7 +5,6 @@ import ( "testing" endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" - apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" @@ -21,7 +20,7 @@ func TestGatewayEndpointRecordSetReconcilesReceiptGatewaysAfterParentChange(t *t reconcileRetentionRoute(t, r, routeKey) var removed endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, gatewayARecordKey, &removed); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, gatewayARecordKey, &removed); err != nil || removed.DeletionTimestamp.IsZero() { t.Fatalf("detached Gateway retained DNS intent: %v", err) } assertRetentionRecord(t, r, gatewayBRecordKey, []string{"shared.example.com"}, "192.0.2.20") @@ -34,7 +33,7 @@ func TestGatewayEndpointRecordSetReconcilesReceiptGatewaysAfterParentChange(t *t for _, key := range []client.ObjectKey{gatewayARecordKey, gatewayBRecordKey} { var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, key, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, key, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("deleted route left prior Gateway DNS intent %s: %v", key, err) } } @@ -56,7 +55,7 @@ func TestGatewayEndpointRecordSetDoesNotMintReceiptWithoutCurrentAddresses(t *te deleteRoute(t, r, haKey) var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, recordKey, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, recordKey, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("fresh route inherited a receipt while addresses were absent: %v", err) } } @@ -95,13 +94,15 @@ func receiptSyncFixture(t *testing.T) (*Reconciler, client.ObjectKey, client.Obj gatewayA := receiptSyncGateway("gateway-a", "gateway-a-uid", "192.0.2.10", section) gatewayB := receiptSyncGateway("gateway-b", "gateway-b-uid", "192.0.2.20", section) k8sClient := fake.NewClientBuilder().WithScheme(testScheme(t)). - WithStatusSubresource(route, gatewayA, gatewayB). + WithStatusSubresource(route, gatewayA, gatewayB, &endpointv1alpha1.EndpointRecordSet{}). WithObjects(route, gatewayA, gatewayB).Build() r := &Reconciler{Client: k8sClient, EndpointRecordSetNamespace: "dns-system"} routeKey := client.ObjectKeyFromObject(route) reconcileRetentionRoute(t, r, routeKey) gatewayARecordKey := client.ObjectKey{Namespace: "dns-system", Name: generatedGatewayEndpointRecordSetName(gatewayA.Namespace, gatewayA.Name)} gatewayBRecordKey := client.ObjectKey{Namespace: "dns-system", Name: generatedGatewayEndpointRecordSetName(gatewayB.Namespace, gatewayB.Name)} + markUnprotected(t, r, gatewayARecordKey) + markUnprotected(t, r, gatewayBRecordKey) assertRetentionRecord(t, r, gatewayARecordKey, []string{"shared.example.com"}, "192.0.2.10") assertRetentionRecord(t, r, gatewayBRecordKey, []string{"shared.example.com"}, "192.0.2.20") return r, routeKey, gatewayARecordKey, gatewayBRecordKey diff --git a/internal/go/apps/gatewayendpoint/controllers/gateway/retention_test.go b/internal/go/apps/gatewayendpoint/controllers/gateway/retention_test.go index 5556554..58a671c 100644 --- a/internal/go/apps/gatewayendpoint/controllers/gateway/retention_test.go +++ b/internal/go/apps/gatewayendpoint/controllers/gateway/retention_test.go @@ -138,7 +138,7 @@ func TestGatewayEndpointRecordSetRevokesExplicitFalseAndForeignGateway(t *testin reconcileRetentionRoute(t, r, haKey) var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, recordKey, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, recordKey, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("explicitly rejected route retained DNS intent: %v", err) } }) @@ -150,7 +150,7 @@ func TestGatewayEndpointRecordSetRevokesExplicitFalseAndForeignGateway(t *testin reconcileRetentionRoute(t, r, haKey) var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, recordKey, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, recordKey, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("explicitly rejected Gateway retained DNS intent: %v", err) } }) @@ -198,7 +198,7 @@ func TestGatewayEndpointRecordSetRevokesExplicitFalseAndForeignGateway(t *testin reconcileRetentionRoute(t, r, haKey) var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, recordKey, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, recordKey, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("changed listener ownership retained DNS intent: %v", err) } }) @@ -297,7 +297,7 @@ func TestGatewayEndpointRecordSetDoesNotRetainWithoutReceiptOrSelectorProof(t *t reconcileRetentionRoute(t, r, haKey) var record endpointv1alpha1.EndpointRecordSet - if err := r.Get(ctx, recordKey, &record); !apierrors.IsNotFound(err) { + if err := r.Get(ctx, recordKey, &record); err != nil || record.DeletionTimestamp.IsZero() { t.Fatalf("unproven retained DNS intent: %v", err) } }) @@ -333,15 +333,30 @@ func retentionFixtureWithNamespacePolicy(t *testing.T, includePayment bool, name objects = append(objects, payment) paymentKey = client.ObjectKeyFromObject(payment) } - k8sClient := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(ha, gateway).WithObjects(objects...).Build() + k8sClient := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(ha, gateway, &endpointv1alpha1.EndpointRecordSet{}).WithObjects(objects...).Build() r := &Reconciler{Client: k8sClient, EndpointRecordSetNamespace: "dns-system"} if includePayment { reconcileRetentionRoute(t, r, paymentKey) } reconcileRetentionRoute(t, r, client.ObjectKeyFromObject(ha)) + markUnprotected(t, r, client.ObjectKey{Namespace: "dns-system", Name: generatedGatewayEndpointRecordSetName("platform", "public")}) return r, paymentKey, client.ObjectKeyFromObject(ha), client.ObjectKeyFromObject(gateway), client.ObjectKey{Namespace: "dns-system", Name: generatedGatewayEndpointRecordSetName("platform", "public")} } +// These admission-receipt tests exercise ordinary, non-external publication. +// External protection and capture are covered by withdrawal_test.go. +func markUnprotected(t *testing.T, r *Reconciler, key client.ObjectKey) { + t.Helper() + var record endpointv1alpha1.EndpointRecordSet + if err := r.Get(context.Background(), key, &record); err != nil { + t.Fatal(err) + } + record.Status.Conditions = []metav1.Condition{{Type: "WithdrawalProtected", Status: metav1.ConditionFalse, ObservedGeneration: record.Generation, Reason: "ProviderCapability"}} + if err := r.Status().Update(context.Background(), &record); err != nil { + t.Fatal(err) + } +} + func retentionRoute(namespace, name, uid, hostname string, status metav1.ConditionStatus) *gatewayv1.HTTPRoute { section := gatewayv1.SectionName("https") gatewayNamespace := gatewayv1.Namespace("platform") diff --git a/internal/go/apps/gatewayendpoint/controllers/gateway/withdrawal_test.go b/internal/go/apps/gatewayendpoint/controllers/gateway/withdrawal_test.go new file mode 100644 index 0000000..64ed3ca --- /dev/null +++ b/internal/go/apps/gatewayendpoint/controllers/gateway/withdrawal_test.go @@ -0,0 +1,100 @@ +package gateway + +import ( + "context" + "errors" + "reflect" + "testing" + + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + gatewayv1 "sigs.k8s.io/gateway-api/apis/v1" +) + +func gatewayWithdrawalERS() *endpointv1alpha1.EndpointRecordSet { + return &endpointv1alpha1.EndpointRecordSet{ + ObjectMeta: metav1.ObjectMeta{Namespace: "dns", Name: generatedGatewayEndpointRecordSetName("app", "public"), UID: "ers-uid", Generation: 3, + Finalizers: []string{endpointv1alpha1.GeneratedRecordSetsFinalizer}, Labels: map[string]string{ + managedByLabel: managedByValue, generatedLabelGatewayNamespace: "app", generatedLabelGatewayName: "public", + generatedLabelRouteNamespace: "app", generatedLabelRouteName: "route", + }}, + Spec: endpointv1alpha1.EndpointRecordSetSpec{Hostnames: []string{"h1.example.com", "h2.example.com"}, Targets: []endpointv1alpha1.EndpointTarget{{Type: endpointv1alpha1.EndpointTargetTypeIPAddress, Value: "192.0.2.1"}}}, + } +} + +func TestWithdrawalGatewayCleanupPathsRequireCapture(t *testing.T) { + for _, path := range []string{"missing gateway", "deleted gateway", "rejected admission", "cleanupForRoute"} { + t.Run(path, func(t *testing.T) { + ctx := context.Background() + ers := gatewayWithdrawalERS() + objects := []client.Object{ers} + if path == "deleted gateway" || path == "rejected admission" { + gateway := &gatewayv1.Gateway{ObjectMeta: metav1.ObjectMeta{Namespace: "app", Name: "public"}, Status: gatewayv1.GatewayStatus{Conditions: []metav1.Condition{{Type: "Accepted", Status: metav1.ConditionFalse}}}} + if path == "deleted gateway" { + now := metav1.Now() + gateway.DeletionTimestamp = &now + gateway.Finalizers = []string{"test.example/hold"} + } + objects = append(objects, gateway) + } + c := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(&endpointv1alpha1.EndpointRecordSet{}).WithObjects(objects...).Build() + r := &Reconciler{Client: c} + cleanup := func() error { + if path == "cleanupForRoute" { return r.cleanupForRoute(ctx, "dns", "app", "route") } + return r.syncGatewayEndpointRecordSet(ctx, "dns", client.ObjectKey{Namespace: "app", Name: "public"}) + } + if err := cleanup(); !errors.Is(err, retirement.ErrCapturePending) { t.Fatalf("retire = %v", err) } + var current endpointv1alpha1.EndpointRecordSet + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil { t.Fatal(err) } + if !current.DeletionTimestamp.IsZero() || len(current.Spec.Hostnames) != 0 || len(current.Spec.Targets) != 0 || len(current.Spec.RetiredTargets) != 1 || len(current.Spec.RetiredHostnames) != 2 { + t.Fatalf("cleanup lost intent: %#v", current) + } + current.Status.Conditions = []metav1.Condition{{Type: retirement.CapturedCondition, Status: metav1.ConditionTrue, ObservedGeneration: current.Generation-1, Reason: "Captured"}} + if err := c.Status().Update(ctx, ¤t); err != nil { t.Fatal(err) } + if err := cleanup(); !errors.Is(err, retirement.ErrCapturePending) { t.Fatalf("stale acknowledgment permitted DELETE: %v", err) } + current.Status.Conditions[0].ObservedGeneration = current.Generation + if err := c.Status().Update(ctx, ¤t); err != nil { t.Fatal(err) } + if err := cleanup(); err != nil { t.Fatal(err) } + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil || current.DeletionTimestamp.IsZero() { t.Fatalf("capture did not permit deletion: %v", err) } + }) + } +} + +func TestWithdrawalGatewayUpdateSnapshotsAndOverflow(t *testing.T) { + for _, overflow := range []bool{false, true} { + t.Run(map[bool]string{false:"snapshot", true:"overflow"}[overflow], func(t *testing.T) { + ctx := context.Background() + ers := gatewayWithdrawalERS() + if overflow { ers.Spec.RetiredTargets = make([]endpointv1alpha1.RetiredTarget, retirement.MaxRetiredTargets) } + c := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(&endpointv1alpha1.EndpointRecordSet{}).WithObjects(ers).Build() + r := &Reconciler{Client: c} + desired := ers.DeepCopy() + desired.Spec.Hostnames = []string{"h2.example.com"} + desired.Spec.Targets[0].Value = "192.0.2.2" + err := r.applyGatewayEndpointRecordSets(ctx, "dns", client.ObjectKey{Namespace:"app", Name:"public"}, []endpointv1alpha1.EndpointRecordSet{*desired}) + var current endpointv1alpha1.EndpointRecordSet + if getErr := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); getErr != nil { t.Fatal(getErr) } + if overflow { + if !errors.Is(err, retirement.ErrCapturePending) || !reflect.DeepEqual(current.Spec, ers.Spec) || meta.FindStatusCondition(current.Status.Conditions, "Ready").Reason != retirement.PendingCondition { t.Fatalf("overflow = %#v, %v", current, err) } + return + } + if err != nil || len(current.Spec.RetiredTargets) != 1 || len(current.Spec.RetiredHostnames) != 1 || current.Spec.RetiredHostnames[0].Addresses[0] != "192.0.2.1" { t.Fatalf("snapshot = %#v, %v", current.Spec, err) } + }) + } +} + +func TestWithdrawalGatewayInstallsFinalizerAtCreate(t *testing.T) { + ctx := context.Background() + ers := gatewayWithdrawalERS() + ers.Finalizers = nil + c := fake.NewClientBuilder().WithScheme(testScheme(t)).Build() + r := &Reconciler{Client:c} + if err := r.applyGatewayEndpointRecordSets(ctx, "dns", client.ObjectKey{Namespace:"app", Name:"public"}, []endpointv1alpha1.EndpointRecordSet{*ers}); err != nil { t.Fatal(err) } + var current endpointv1alpha1.EndpointRecordSet + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil { t.Fatal(err) } + if !reflect.DeepEqual(current.Finalizers, []string{endpointv1alpha1.GeneratedRecordSetsFinalizer}) { t.Fatal("ERS create omitted endpoint finalizer") } +} diff --git a/internal/go/apps/serviceendpoint/controllers/service/controller.go b/internal/go/apps/serviceendpoint/controllers/service/controller.go index dda2e96..5628cdc 100644 --- a/internal/go/apps/serviceendpoint/controllers/service/controller.go +++ b/internal/go/apps/serviceendpoint/controllers/service/controller.go @@ -9,6 +9,7 @@ import ( "sort" "strings" + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -17,6 +18,8 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" ) const ( @@ -39,6 +42,7 @@ type Reconciler struct { // +kubebuilder:rbac:groups="",resources=services,verbs=get;list;watch;update // +kubebuilder:rbac:groups="",resources=services/finalizers,verbs=update // +kubebuilder:rbac:groups=endpoint.dns.appthrust.io,resources=endpointrecordsets,verbs=create;delete;get;list;watch;patch;update +// +kubebuilder:rbac:groups=endpoint.dns.appthrust.io,resources=endpointrecordsets/status,verbs=get;patch;update func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { namespace := r.EndpointRecordSetNamespace if namespace == "" { @@ -100,9 +104,18 @@ func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). Named("service-endpoint"). For(&corev1.Service{}). + Watches(&endpointv1alpha1.EndpointRecordSet{}, handler.EnqueueRequestsFromMapFunc(r.mapEndpointRecordSetToService)). Complete(r) } +func (r *Reconciler) mapEndpointRecordSetToService(_ context.Context, obj client.Object) []reconcile.Request { + labels := obj.GetLabels() + if labels[managedByLabel] != managedByValue || labels[generatedLabelServiceNamespace] == "" || labels[generatedLabelServiceName] == "" { + return nil + } + return []reconcile.Request{{NamespacedName: client.ObjectKey{Namespace: labels[generatedLabelServiceNamespace], Name: labels[generatedLabelServiceName]}}} +} + func (r *Reconciler) endpointRecordSetForService(service *corev1.Service, namespace string) *endpointv1alpha1.EndpointRecordSet { hostnames := hostnamesForService(service) targets := endpointTargets(service.Status.LoadBalancer.Ingress) @@ -130,11 +143,23 @@ func (r *Reconciler) applyEndpointRecordSet(ctx context.Context, desired *endpoi var existing endpointv1alpha1.EndpointRecordSet err := r.Get(ctx, client.ObjectKey{Namespace: desired.Namespace, Name: desired.Name}, &existing) if apierrors.IsNotFound(err) { + controllerutil.AddFinalizer(desired, endpointv1alpha1.GeneratedRecordSetsFinalizer) return r.Create(ctx, desired, client.FieldOwner(fieldOwner)) } if err != nil { return err } + if !existing.DeletionTimestamp.IsZero() { + return nil + } + spec, err := retirement.Append(&existing, desired.Spec) + if err != nil { + return retirement.ReportPending(ctx, r.Client, &existing, err) + } + desired.Spec = spec + if err := retirement.ClearPending(ctx, r.Client, &existing); err != nil { + return err + } if !endpointRecordSetNeedsUpdate(&existing, desired) { return nil } @@ -152,7 +177,7 @@ func (r *Reconciler) cleanupForService(ctx context.Context, endpointRecordSetNam return err } for i := range existing.Items { - if err := r.Delete(ctx, &existing.Items[i]); err != nil && !apierrors.IsNotFound(err) { + if err := retirement.RetireOrDelete(ctx, r.Client, &existing.Items[i]); err != nil && !apierrors.IsNotFound(err) { return err } } diff --git a/internal/go/apps/serviceendpoint/controllers/service/controller_test.go b/internal/go/apps/serviceendpoint/controllers/service/controller_test.go index d5ed13f..779c5b9 100644 --- a/internal/go/apps/serviceendpoint/controllers/service/controller_test.go +++ b/internal/go/apps/serviceendpoint/controllers/service/controller_test.go @@ -150,8 +150,8 @@ func TestReconcilerCleansEndpointRecordSetWhenServiceDNSIntentIsRemoved(t *testi if err := k8sClient.List(ctx, &list, client.InNamespace("app")); err != nil { t.Fatalf("list EndpointRecordSets: %v", err) } - if len(list.Items) != 0 { - t.Fatalf("EndpointRecordSets = %d, want 0", len(list.Items)) + if len(list.Items) != 1 || list.Items[0].DeletionTimestamp.IsZero() || !containsString(list.Items[0].Finalizers, endpointv1alpha1.GeneratedRecordSetsFinalizer) { + t.Fatalf("EndpointRecordSets = %#v, want deletion pending endpoint cleanup", list.Items) } if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(service), &updated); err != nil { t.Fatalf("get cleaned Service: %v", err) diff --git a/internal/go/apps/serviceendpoint/controllers/service/withdrawal_test.go b/internal/go/apps/serviceendpoint/controllers/service/withdrawal_test.go new file mode 100644 index 0000000..218e4f4 --- /dev/null +++ b/internal/go/apps/serviceendpoint/controllers/service/withdrawal_test.go @@ -0,0 +1,57 @@ +package service + +import ( + "context" + "errors" + "testing" + + "github.com/appthrust/dns-api/internal/go/apps/endpoint/retirement" + endpointv1alpha1 "github.com/appthrust/dns-api/pkg/go/api/endpoint/v1alpha1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" +) + +func TestWithdrawalServiceUsesCreateRetireCaptureContract(t *testing.T) { + ctx := context.Background() + c := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(&endpointv1alpha1.EndpointRecordSet{}).Build() + r := &Reconciler{Client:c} + ers := &endpointv1alpha1.EndpointRecordSet{ + ObjectMeta: metav1.ObjectMeta{Namespace:"dns", Name:"service", UID:"service-ers", Generation:4, Labels:map[string]string{managedByLabel:managedByValue, generatedLabelServiceNamespace:"app", generatedLabelServiceName:"web"}}, + Spec:endpointv1alpha1.EndpointRecordSetSpec{Hostnames:[]string{"h1.example.com", "h2.example.com"}, Targets:[]endpointv1alpha1.EndpointTarget{{Type:endpointv1alpha1.EndpointTargetTypeIPAddress, Value:"192.0.2.1"}}}, + } + if err := r.applyEndpointRecordSet(ctx, ers); err != nil { t.Fatal(err) } + if !containsString(ers.Finalizers, endpointv1alpha1.GeneratedRecordSetsFinalizer) { t.Fatal("create omitted finalizer") } + desired := ers.DeepCopy() + desired.Spec.Hostnames = []string{"h2.example.com"} + desired.Spec.Targets[0].Value = "192.0.2.2" + if err := r.applyEndpointRecordSet(ctx, desired); err != nil { t.Fatal(err) } + var current endpointv1alpha1.EndpointRecordSet + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil { t.Fatal(err) } + if len(current.Spec.RetiredTargets) != 1 || len(current.Spec.RetiredHostnames) != 1 || current.Spec.RetiredHostnames[0].Addresses[0] != "192.0.2.1" { t.Fatal("service lost old pair") } + // Real API updates advance generation; fake client does not. + current.Generation++ + if err := c.Update(ctx, ¤t); err != nil { t.Fatal(err) } + if err := r.cleanupForService(ctx, "dns", "app", "web"); !errors.Is(err, retirement.ErrCapturePending) { t.Fatalf("cleanup = %v", err) } + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil { t.Fatal(err) } + if !current.DeletionTimestamp.IsZero() || len(current.Spec.Hostnames) != 0 || len(current.Spec.RetiredTargets) != 2 { t.Fatal("cleanup discarded uncaptured state") } + current.Status.Conditions = []metav1.Condition{{Type:retirement.CapturedCondition, Status:metav1.ConditionTrue, ObservedGeneration:current.Generation, Reason:"Captured"}} + if err := c.Status().Update(ctx, ¤t); err != nil { t.Fatal(err) } + if err := r.cleanupForService(ctx, "dns", "app", "web"); err != nil { t.Fatal(err) } + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ¤t); err != nil || current.DeletionTimestamp.IsZero() { t.Fatalf("capture did not permit cleanup: %v", err) } +} + +func TestWithdrawalServiceNonExternalUpdateUnchanged(t *testing.T) { + ctx := context.Background() + ers := &endpointv1alpha1.EndpointRecordSet{ObjectMeta:metav1.ObjectMeta{Namespace:"dns", Name:"ers", Generation:1}, + Spec:endpointv1alpha1.EndpointRecordSetSpec{Hostnames:[]string{"h1.example.com"}, Targets:[]endpointv1alpha1.EndpointTarget{{Type:endpointv1alpha1.EndpointTargetTypeIPAddress, Value:"192.0.2.1"}}}, + Status:endpointv1alpha1.EndpointRecordSetStatus{Conditions:[]metav1.Condition{{Type:retirement.ProtectedCondition, Status:metav1.ConditionFalse, ObservedGeneration:1}}}, + } + c := fake.NewClientBuilder().WithScheme(testScheme(t)).WithStatusSubresource(ers).WithObjects(ers).Build() + r := &Reconciler{Client:c} + desired := ers.DeepCopy() + desired.Spec.Targets[0].Value = "192.0.2.2" + if err := r.applyEndpointRecordSet(ctx, desired); err != nil { t.Fatal(err) } + if err := c.Get(ctx, client.ObjectKeyFromObject(ers), ers); err != nil { t.Fatal(err) } + if len(ers.Spec.RetiredTargets)+len(ers.Spec.RetiredHostnames) != 0 || ers.Spec.Targets[0].Value != "192.0.2.2" { t.Fatal("non-external source behavior changed") } +} From bc6922625a841ec3e9a77eb3c06b09f6f118d53e Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 15:30:49 +0900 Subject: [PATCH 15/16] fix(core): preserve admission parity and existing Zone deletion messages --- .../validating-webhook-configuration.yaml | 32 +++++++++---------- deps/dep-0003-record-withdrawal.md | 2 ++ .../zoneunit/zoneunit_controller.go | 6 +++- 3 files changed, 23 insertions(+), 17 deletions(-) diff --git a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml index 105abbc..2e99c09 100644 --- a/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml +++ b/deploy/charts/dns-api/templates/validating-webhook-configuration.yaml @@ -139,22 +139,6 @@ webhooks: resources: - recordsets - recordsets/status - {{- range $kind := list "recordwithdrawal" "recordwithdrawalwaiver" }} - - name: {{ $kind }}s.dns.appthrust.io - admissionReviewVersions: [v1] - sideEffects: None - failurePolicy: {{ $.Values.webhook.failurePolicy }} - clientConfig: - service: - namespace: {{ $.Release.Namespace }} - name: {{ include "dns-api.webhookServiceName" $ }} - path: /validate-dns-appthrust-io-v1alpha1-{{ $kind }} - rules: - - apiGroups: [dns.appthrust.io] - apiVersions: [v1alpha1] - operations: [CREATE, UPDATE] - resources: [{{ $kind }}s, {{ $kind }}s/status] - {{- end }} - name: zoneunits.dns.appthrust.io admissionReviewVersions: - v1 @@ -177,5 +161,21 @@ webhooks: resources: - zoneunits - zoneunits/status + {{- range $kind := list "recordwithdrawal" "recordwithdrawalwaiver" }} + - name: {{ $kind }}s.dns.appthrust.io + admissionReviewVersions: [v1] + sideEffects: None + failurePolicy: {{ $.Values.webhook.failurePolicy }} + clientConfig: + service: + namespace: {{ $.Release.Namespace }} + name: {{ include "dns-api.webhookServiceName" $ }} + path: /validate-dns-appthrust-io-v1alpha1-{{ $kind }} + rules: + - apiGroups: [dns.appthrust.io] + apiVersions: [v1alpha1] + operations: [CREATE, UPDATE] + resources: [{{ $kind }}s, {{ $kind }}s/status] + {{- end }} {{- end }} diff --git a/deps/dep-0003-record-withdrawal.md b/deps/dep-0003-record-withdrawal.md index e96fa08..d3c15c0 100644 --- a/deps/dep-0003-record-withdrawal.md +++ b/deps/dep-0003-record-withdrawal.md @@ -209,6 +209,8 @@ Two claim creators are intentional: platform CapacityReservation creates the man The dns-api Core follow-up release, reviewed platform import, and CRD delivery precede platform Phase 4b support. Missing CRD/watch/verifier, denied/truncated reads, missing/stale results, or a mode switch cannot erase held external obligations. Pre-4b retirements that already lost history are not reconstructed by epochs or synthetic success. +`ZoneUnit.spec.zone` binds the ZoneClass UID and observed generation; withdrawal projection requires current Unit status generation and current Class binding, and consumers compare the live Class UID/generation before consuming results. + ## Documentation On acceptance, reflect this contract in present tense into: diff --git a/internal/go/core/controllers/zoneunit/zoneunit_controller.go b/internal/go/core/controllers/zoneunit/zoneunit_controller.go index c91bfba..cc42333 100644 --- a/internal/go/core/controllers/zoneunit/zoneunit_controller.go +++ b/internal/go/core/controllers/zoneunit/zoneunit_controller.go @@ -229,6 +229,10 @@ func (r *ZoneUnitCompositionReconciler) reconcileZone(ctx context.Context, req c if unitExists { providerStatus = existing.Status.Zone } + dependencyMessage := fmt.Sprintf("%d RecordSet resources still reference this Zone; delete or move them before Zone cleanup can continue", len(recordSets)) + if len(withdrawals) > 0 { + dependencyMessage = fmt.Sprintf("%d RecordSet and %d RecordWithdrawal resources still reference this Zone; archive dependent claims before Zone cleanup can continue", len(recordSets), len(withdrawals)) + } return ctrl.Result{}, r.setZoneClaimStatusWithMessages( ctx, &zone, @@ -237,7 +241,7 @@ func (r *ZoneUnitCompositionReconciler) reconcileZone(ctx context.Context, req c "Zone is accepted", metav1.ConditionFalse, "ProviderChangePending", - fmt.Sprintf("%d RecordSet and %d RecordWithdrawal resources still reference this Zone; archive dependent claims before Zone cleanup can continue", len(recordSets), len(withdrawals)), + dependencyMessage, providerStatus, ) } From 0feaa2dff127fc84c4a2add858768d63d8ddf454 Mon Sep 17 00:00:00 2001 From: reoring Date: Sun, 18 Oct 2026 16:04:42 +0900 Subject: [PATCH 16/16] docs: accept DEP-0003 and reflect verified record withdrawal into docs/design --- deps/README.md | 2 +- deps/dep-0003-record-withdrawal.md | 7 +++-- docs/design/controllers-webhooks.md | 16 +++++++++- docs/design/core-api.md | 35 +++++++++++++++++---- docs/design/endpoint-api.md | 22 +++++++++----- docs/design/external-provider.md | 42 ++++++++++++++++++++++++-- docs/design/gateway-api-integration.md | 8 +++++ docs/design/provider-extension.md | 8 +++++ 8 files changed, 121 insertions(+), 19 deletions(-) diff --git a/deps/README.md b/deps/README.md index 44b55fe..d34278b 100644 --- a/deps/README.md +++ b/deps/README.md @@ -6,4 +6,4 @@ Use [the template](./dep-template.md) for a new proposal. Single Markdown files | DEP | Title | Status | | --- | --- | --- | -| [0003](./dep-0003-record-withdrawal.md) | Verified record withdrawal before address reuse | Draft | +| [0003](./dep-0003-record-withdrawal.md) | Verified record withdrawal before address reuse | Accepted | diff --git a/deps/dep-0003-record-withdrawal.md b/deps/dep-0003-record-withdrawal.md index d3c15c0..f90479e 100644 --- a/deps/dep-0003-record-withdrawal.md +++ b/deps/dep-0003-record-withdrawal.md @@ -1,14 +1,17 @@ # DEP-0003: Verified record withdrawal before address reuse -- Status: Draft +- Status: Accepted - Owner: dns-api maintainers (Core, Endpoint App, and external provider) - Created: 2026-10-18 +- Accepted: 2026-10-18 + +The accepted contract is now carried by `docs/design/`; this DEP retains rationale, alternatives, and compatibility history. ## Summary Introduce independent Core `RecordWithdrawal` and `RecordWithdrawalWaiver` claims, Core composition/projection through `ZoneUnit`, and external-provider exclusion verification. Endpoint sources preserve retired hostname/address pairs until the Endpoint App durably creates claims. Platform consumers use verified results before member retirement and address reuse; exceptional operator risk acceptance remains distinct from DNS verification. -This is the dns-api part of the reviewed **Phase 4b: verified external-DNS withdrawal before VIP reuse** design, especially §§3.2–3.4, 4.1–4.3, 6–7, 8.2, and 9. It is a follow-up after PR #16, not additional stage-1 implementation scope. It is proposed behavior, not an implementation or runtime-verification report. Under the [DEP process](../docs/design/common.md#dep-process), committed `docs/design/` remains current specification until acceptance and reflection there. +This is the dns-api part of the reviewed **Phase 4b: verified external-DNS withdrawal before VIP reuse** design, especially §§3.2–3.4, 4.1–4.3, 6–7, 8.2, and 9. It follows PR #16 rather than expanding stage-1 scope. Acceptance is not a runtime-verification report. Under the [DEP process](../docs/design/common.md#dep-process), `docs/design/` carries the current contract. ## Motivation diff --git a/docs/design/controllers-webhooks.md b/docs/design/controllers-webhooks.md index 2406b9a..e4505c9 100644 --- a/docs/design/controllers-webhooks.md +++ b/docs/design/controllers-webhooks.md @@ -91,6 +91,20 @@ If composition acceptance is `False`, the claim `Accepted` is `False` and the it `Accepted=False` means the controller has enough information to reject the claim. `Accepted=Unknown` means the validation material is incomplete or temporarily unavailable, such as missing `Zone`, missing `ZoneClass`, missing provider identity state, missing owner state during GitOps recovery, unavailable Provider storage version, unavailable Provider conversion webhook, or a transient core error. +## Withdrawal Composition and Projection + +Core composes independent `RecordWithdrawal` claims into `ZoneUnit.spec.withdrawals[]`; providers do not read those claims, waivers, EndpointRecordSets, or platform kinds. Core alone writes public withdrawal/waiver status. The Endpoint App only creates retained pair claims; source Apps only preserve endpoint retirement intent. + +Composition resolves the live Zone, Class UID/generation, served input and storage Provider versions, and exact current nondeleting Unit binding. Both input and storage versions must support `recordWithdrawal`. Core applies `allowedZones` and the corresponding A/AAAA `allowedRecordSets` grants based on address family. Cross-namespace access and wildcard access require explicit grants. Withdrawals reserve no DNS record ownership and do not conflict with replacement publication. + +Acceptance is `True/Accepted` or `False/InvalidWithdrawal`, `NotAllowed`, or `UnsupportedProvider`; unlike ordinary unresolved claim acceptance, an unavailable withdrawal binding is `False/InvalidWithdrawal`. Valid-shaped rejected claims remain in the Unit with `allowed=false`. If the Zone inventory exceeds 256 claims, Core rejects the inventory as a whole, retains previous Unit entries disallowed with waivers removed, and never truncates obligations. + +Core validates waiver claim UID, immutable-spec digest, canonical address, and namespace scope directly before attachment; deterministic namespace/name ordering selects the first matching waiver. An independent waiver reconciler maintains `Accepted` even if the claim or Zone disappears. No SubjectAccessReview, client-supplied audit identity, or receiving platform-resource lookup is involved. + +Projection requires current Unit `status.observedGeneration`, Class UID/generation, exact current desired item including authorization/waiver, matching claim keys/UID/generation, and current `Withdrawn`/`Ready` condition generations. It preserves provider messages and public evidence, never private provider state. Override projection additionally requires the exact composed waiver name/UID/digest/reservation reference and current `OverrideAccepted=True/OperatorAuthorizedRelease`. Missing or stale results yield `Withdrawn=Unknown`, `Ready=False`, reason `DNSProbeFailed`, message `No current identity-bound withdrawal observation is available`. Same-name recreation cannot inherit success. + +Core retains every withdrawal as a Zone deletion dependency, including verified claims, and continues composition/verification while the Zone waits for explicit evidence archival. Pending withdrawal status does not change unrelated Zone/RecordSet `Programmed`. + ## Admission Webhook Admission webhook is the API input boundary. It validates core schema, immutable fields, object-local references, object-local policy shape, and provider schema / CEL validation. It does not call DNS provider APIs, evaluate cross-object namespace / record access policy, or update status. @@ -103,7 +117,7 @@ The webhook server runs as part of the controller-runtime manager and listens on Webhook TLS certificates are issued by cert-manager. Tilt installs cert-manager as a Helm chart. dns-api manifests apply `Certificate`, `Issuer`, and `ValidatingWebhookConfiguration`. cert-manager CA injection sets the webhook `caBundle`. -Core validating webhook covers at least spec create/update for `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`, and status update for `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit`. Provider-specific validating webhooks cover enabled provider-specific resources such as `Route53Identity` and `CloudflareIdentity`; external has no identity admission handler. The core webhook must not import provider API packages such as `api/route53` or `api/cloudflare`. +Core validating webhook covers spec create/update and status update for `RecordWithdrawal` and `RecordWithdrawalWaiver` in addition to `Provider`, `ZoneClass`, `Zone`, `RecordSet`, and `ZoneUnit` as applicable. Withdrawal admission validates canonical relative name/address, required address-label consistency, no ownerReferences/finalizers, immutable spec, reference/digest shapes, and bounded conditions/evidence. It performs no DNS queries or cross-object grant checks. Unit withdrawal map keys must match nested claim identity; same-incarnation immutable shape cannot change. Status validates generation, the closed withdrawal reason table, 2048-byte messages, and public provider data against `recordWithdrawal.schemas.statusProviderData`; claim `provider.state` is forbidden. Provider-specific validating webhooks cover enabled resources such as `Route53Identity` and `CloudflareIdentity`; external has no identity handler. Core does not import provider API packages. Provider controllers do not fully trust the webhook. The webhook is the primary user feedback boundary, but controllers keep minimal safety checks. Controller-side validation failure is returned as `Accepted=False` or `Programmed=False`. diff --git a/docs/design/core-api.md b/docs/design/core-api.md index 8c90e00..e294057 100644 --- a/docs/design/core-api.md +++ b/docs/design/core-api.md @@ -678,9 +678,27 @@ spec: - ns-222.example-dns.net ``` +## RecordWithdrawal + +`RecordWithdrawal` is a namespace-scoped retained request to exclude one address from customer-managed DNS. Its immutable `spec` contains `zoneRef` (omitted namespace means the claim namespace), canonical zone-relative `name` (`@`, exact name, or leftmost wildcard), and one canonical unscoped, unmapped IPv4 or IPv6 `address`. It has no provider selection, resolver policy, ownerReferences, or finalizers. Admission rejects spec changes with `spec is immutable`. + +The required `endpoint.dns.appthrust.io/address` label equals the canonical IPv4 string or lowercase unpadded base32 of the normalized IPv6 bytes. It is a discovery key, not authorization or proof. Source annotations carry provenance, not authority. Endpoint-created names bind ERS UID, pre-update retirement generation, hostname, and address; create-or-get checks immutable shape, lifecycle, label, and source annotations. + +Core owns public status: `observedGeneration`, resolved `zone` and `unit` namespace/name/UID references, `providerRef`, `policyFingerprint`, `checkedNames`, `requiredViews`, `sampledWildcard`, `absentSince`, `notBefore`, `lastProbeTime`, `nextProbeTime`, `verifiedAt`, `resultDigest`, public `provider.data`, and optional `waiverRef`. Checked names and views each have a 256-item bound. At most four generation-aware conditions describe `Accepted`, `Withdrawn`, `Ready`, and optional `OverrideAccepted`. Provider-private state is not projected. The [external verification contract](./external-provider.md#withdrawal-verification) defines the condition/reason/message table and cache proof. + +Claims remain after source deletion and after verification; there is no age-based garbage collection. Every claim referencing a Zone is a deletion dependency, regardless of readiness. Operators explicitly archive/remove evidence when it is no longer consumable. Claim or binding disappearance never proves withdrawal. + +## RecordWithdrawalWaiver + +`RecordWithdrawalWaiver` is a separate namespace-scoped immutable declaration of stale-DNS risk acceptance. Its spec contains `withdrawalRef` namespace/name/UID, `claimSpecDigest` (`sha256:` plus 64 lowercase hex digits over the immutable claim wire spec), canonical `address`, receiving `reservationRef` namespace/name/UID, nonempty `reason` (at most 2048 bytes), `ticketRef` (at most 1024 bytes), and `acceptStaleDNSRisk: true`. It has no ownerReferences or finalizers. + +Waiver creation is separately RBAC-authorized; Kubernetes audit supplies actor/time/provenance. Client-supplied identity fields and platform lookups are not authority. Core validates the exact claim UID/spec digest/address and same-namespace scope, then composes the waiver name/UID/spec digest/reservation reference into the Unit. Public waiver status contains `observedGeneration` and `Accepted=True/Accepted` or `Accepted=False/InvalidWithdrawal`, maintained independently of Zone existence. + +The provider reports `OverrideAccepted=True/OperatorAuthorizedRelease` for the exact Core-validated scope. A waiver does not set `Withdrawn=True` or make DNS ready, and another claim or reservation incarnation cannot consume it. Claims and waivers are retained independently of their sources. + ## ZoneUnit -`ZoneUnit` is the zone-scoped desired-state and ownership ledger built from accepted `Zone` and `RecordSet` claims. It is namespace-scoped and uses the same namespace and name as the owning `Zone`. Users normally create `Zone` and `RecordSet` claims; the Core ZoneUnit Controller creates and updates `ZoneUnit`. +`ZoneUnit` is the zone-scoped desired-state and ownership ledger built from `Zone`, `RecordSet`, and retained `RecordWithdrawal` claims. It is namespace-scoped and uses the same namespace and name as the owning `Zone`. Users normally create claims; the Core ZoneUnit Controller creates and updates `ZoneUnit`. `ZoneUnit.metadata.name` must match `spec.zone.ref.name`; the CRD validates this with CEL. `ZoneUnit.metadata.namespace` must match `spec.zone.ref.namespace`; Kubernetes CRD CEL does not expose `metadata.namespace`, so the core validating webhook rejects namespace mismatches on create and update. Provider controllers do not define an `InvalidZoneUnitName` condition reason and do not handle mismatched names defensively, because invalid `ZoneUnit` names are rejected before provider reconciliation. @@ -695,6 +713,7 @@ spec: - `zone.observedGeneration`: `Zone.metadata.generation` used to build this item. - `zone.domainName`: accepted zone domain name. - `zone.zoneClassRef`: resolved `ZoneClass` reference. +- `zone.zoneClassUID`, `zone.zoneClassObservedGeneration`: resolved Class incarnation and generation used for composition. These optional wire fields fence withdrawal policy changes; missing legacy bindings cannot establish withdrawal. - `zone.adoption`: provider-specific adoption payload after storage-version conversion. - `recordSets`: accepted desired state for owner `RecordSet` claims. - `recordSets[].recordSetNamespace`: owner `RecordSet` namespace. This is part of the list-map key. @@ -709,6 +728,10 @@ spec: `recordSets` is a Kubernetes `listType=map` keyed by scalar fields `recordSetNamespace` and `recordSetName`. `recordSetRef` is not used inside the list item because nested-object list-map keys are not a CRD-safe shape across Kubernetes versions. `recordSetUID` is an incarnation fence, not an additional list key: only one owner incarnation occupies a claim key at a time. Provider controllers do not read `RecordSet` claims; core supplies the UID from the actual claim. A pending predecessor remains its own cleanup item until its UID-bound cleanup completes. A recreated same-name claim cannot inherit the predecessor's provider payload or completion. +`spec.withdrawals` and `status.withdrawals` are map-lists of at most 256 items, keyed by scalar `claimNamespace` and `claimName`. Each item also carries `claim{namespace,name,uid}`; CEL requires its namespace/name to equal the scalar keys. UID is nonempty and is an incarnation fence, not a list key. Spec items contain claim `observedGeneration`, canonical relative `name`, `address`, `allowed`, and optional Core-validated `waiver{name,uid,digest,reservationRef}`. For the same nested claim identity, admission forbids changes to name, address, or observedGeneration; authorization and waiver attachment may change. + +Status items mirror claim identity/generation and carry the withdrawal policy, checked-name/view, cache-window, probe-time, digest, condition, public provider-data, and waiver-reference fields described above. Core projects only the current exact desired item with matching claim UID/generation, condition generations, current Unit status generation, and nonempty Class UID/generation. Withdrawals do not participate in active `Programmed` aggregation or claim ownership conflicts and never create empty A/AAAA RecordSets. + `ZoneUnit.spec.zone` does not contain `Zone.metadata.uid`. Kubernetes assigns `metadata.uid`, so a `ZoneUnit` pre-applied for GitOps recovery cannot know the UID of a `Zone` claim that will be restored later. Provider controllers must not require the `Zone` UID as desired input. DNS record identity is still `zone`, `type`, and zone-relative `name`. Same `type` and zone-relative `name` always conflicts. Core also applies composition validation `forbid-cname-coexistence` by default: a `type=CNAME` item owns the whole record name and cannot coexist with any other record type at the same `Zone` and name, and a non-CNAME item cannot coexist with a CNAME item at the same `Zone` and name. A Provider storage version may disable this validation for specific `self` / `other` record-set item pairs through `zoneUnit.disableValidations[]` when the provider intentionally supports that owner-name combination. Route 53 keeps the default CNAME exclusivity. Cloudflare disables `forbid-cname-coexistence` for `CNAME` with `TXT`, `MX`, and `CAA`, and keeps conflicts for `CNAME` with `A` or `AAAA`. @@ -770,17 +793,17 @@ Deletion uses both core and provider finalizers on `ZoneUnit`: - Core keeps a finalizer on `Zone` and `ZoneUnit` while the external cleanup result must be observed. - Provider controllers keep their own finalizer on `ZoneUnit` while provider cleanup is in progress. -- `RecordSet` is a restrict-style child of `Zone`. While any `RecordSet` still references a deleting `Zone`, core keeps the matching `ZoneUnit` active and does not send the provider cleanup signal. The deleting `Zone` reports `Programmed=False`, reason `ProviderChangePending`, with a message that the remaining `RecordSet` resources must be deleted or moved first. +- `RecordSet` and `RecordWithdrawal` are restrict-style dependent claims of `Zone`. While either kind still references a deleting Zone, core keeps the matching Unit active and does not send provider cleanup. The Zone reports `Programmed=False/ProviderChangePending`; when withdrawals exist, its message includes both dependency counts and asks operators to archive dependent claims. Verified withdrawals still count. - If a `RecordSet` that references a deleting `Zone` is itself deleting, core keeps or writes its `ZoneUnit.spec.recordSets[]` item with `deletionRequested=true` until provider record-set cleanup completes. -- After no `RecordSet` references the deleting `Zone`, core deletes the matching `ZoneUnit` before removing the `Zone` finalizer. -- If the deleting `Zone` has no remaining `RecordSet` references and the matching `ZoneUnit` does not exist, core removes the `Zone` finalizer because there is no provider-owned cleanup state to observe. +- After no `RecordSet` or `RecordWithdrawal` references the deleting Zone, core deletes the matching Unit before removing the Zone finalizer. +- If neither kind references the deleting Zone and the matching Unit does not exist, core removes the Zone finalizer because no provider-owned cleanup state remains. - `ZoneUnit.metadata.deletionTimestamp` is the provider cleanup signal. Provider controllers delete or retain external resources according to provider policy, update `ZoneUnit.status`, and remove their provider finalizer after cleanup completes. - Core removes its `ZoneUnit` finalizer after provider cleanup is complete, then removes the `Zone` finalizer. - Deleting a `RecordSet` sets `recordSets[].deletionRequested=true`. Provider controllers delete the external record target, then report completion. Core removes the `RecordSet` finalizer and later removes the item from `ZoneUnit.spec.recordSets[]`. -The validating webhook rejects manual `ZoneUnit` deletion while the owning `Zone` exists and is not deleting. The owning `Zone` is resolved from `ZoneUnit.spec.zone.ref`. This prevents a user or GitOps controller from sending the provider cleanup signal while the desired `Zone` claim is still active. +The validating webhook rejects manual `ZoneUnit` deletion while any `RecordWithdrawal` references its Zone, with `ZoneUnit cannot be deleted while RecordWithdrawal resources reference its Zone`. It also rejects deletion while the owning Zone exists and is not deleting. The owning Zone is resolved from `ZoneUnit.spec.zone.ref`. -`ZoneUnit` deletion is allowed when the owning `Zone` exists and has `metadata.deletionTimestamp`; this is the normal path started by the Core ZoneUnit Controller during `Zone` deletion. `ZoneUnit` deletion is also allowed when the owning `Zone` is missing, so operators can remove a pre-applied or orphaned GitOps recovery ledger. Core does not delete a `ZoneUnit` merely because the owning `Zone` is temporarily missing during GitOps restore. +With no withdrawal dependency, `ZoneUnit` deletion is allowed when the owning Zone is deleting or missing. Core does not delete a Unit merely because the Zone is temporarily missing during GitOps restore. Provider controllers perform provider cleanup only for `ZoneUnit` resources that are in their scope and have their provider finalizer or otherwise have provider-owned cleanup to complete. They do not add claim finalizers and do not remove core finalizers. If a deleting `ZoneUnit` has no provider finalizer and no provider-owned cleanup state, provider controllers may ignore it. diff --git a/docs/design/endpoint-api.md b/docs/design/endpoint-api.md index 65897b1..aa868d4 100644 --- a/docs/design/endpoint-api.md +++ b/docs/design/endpoint-api.md @@ -36,6 +36,12 @@ manifest has been rewritten. - `hostnames`: fully qualified DNS hostnames to publish. - `targets`: endpoint addresses. The initial target types are `Hostname` and `IPAddress`. +- `retiredTargets`: map-list keyed by `id`, at most 64 entries, each containing `sourceGeneration`, removed canonical IP `address`, and a nonempty snapshot of at most 256 old `hostnames`. +- `retiredHostnames`: map-list keyed by `id`, at most 256 entries, each containing `sourceGeneration`, removed `hostname`, and a nonempty snapshot of at most 64 old IP `addresses`. + +Retirement IDs bind the ERS UID, pre-update generation, entry kind, and removed value. Snapshots preserve old associations: simultaneous removal of H1 and replacement V1→V2 retains `(H1,V1)`, not `(H1,V2)`. Sources append and preserve existing entries with resourceVersion concurrency; only the endpoint controller prunes captured intent. + +CEL requires either nonempty live hostnames **and** targets, or a nonempty retirement collection (`nonempty hostname and target intent or retirement entries are required`). A retire-only object may omit either live side. There is no ERS admission webhook; trusted source writes enforce the retirement protocol. It does not contain: @@ -89,13 +95,15 @@ prior-generation child status is not ready. Each generated child status exposes both its current metadata `generation` and mirrored `observedGeneration` so a consumer can independently verify the aggregate. -The endpoint controller holds the -`endpoint.dns.appthrust.io/generated-recordsets` finalizer while generated -`RecordSet` resources exist. On deletion it requests deletion of every generated -child and removes the finalizer only after a label-selected read proves that no -child remains in the Kubernetes API. Provider cleanup failures therefore retain -both the child provider finalizer and the parent endpoint finalizer; parent -disappearance alone is never used as inferred provider cleanup evidence. +Sources install `endpoint.dns.appthrust.io/generated-recordsets` at ERS creation, before first endpoint reconciliation; the endpoint controller also ensures it on existing active objects. This closes the source-create/direct-delete capture gap. + +Before pruning retirement entries or deleting generated children, the endpoint controller resolves each retired hostname/IP pair to its retained publication Zone binding, or the ordinary single-pair publication resolution/conversion/fallback path. For a Provider version supporting withdrawal it durably creates a deterministic independent `RecordWithdrawal`; successful Create is persisted evidence, and AlreadyExists requires Get plus exact immutable shape, lifecycle, address label, and provenance checks. Duplicate pairs at the same retirement generation share a claim. Missing resolution, collisions, write failures, and collection overflow hold capture rather than discard intent. A known non-supporting Provider requires no withdrawal claim. + +Active ERSs clear captured retirement collections with an optimistic-lock patch; a concurrent append conflicts instead of being lost. Retire-only ERSs retain entries until DELETE and report current-generation `WithdrawalCaptured=True/Captured`, message `Every retired hostname/address pair has been durably captured.` Their `Resolved`, `Accepted`, `Programmed`, and `Ready` conditions are `False/Retiring`, message `EndpointRecordSet is retiring; no active records are published.` They issue no empty conversion request or RecordSet. Capture failure reports `WithdrawalCapturePending` and `Ready=False`; source overflow leaves the old spec unchanged and is never truncated. + +`WithdrawalProtected` defaults closed: only current-generation `False` after all hostname providers resolve without withdrawal support permits the old unprotected cleanup path. Once True, protection is sticky for that ERS incarnation. `WithdrawalCaptured=True` is accepted only at the current ERS generation. Capture acknowledgment proves independent obligation persistence, not DNS withdrawal. + +Direct ERS deletion captures every live IP/hostname pair and all retirement entries before normal child deletion. The endpoint finalizer is removed only after a label-selected read proves no generated RecordSet remains. Capture/provider cleanup failures retain finalizers. Independent withdrawal claims survive parent deletion and do not delay ordinary deletion while the customer changes DNS. ## EndpointProviderCapability diff --git a/docs/design/external-provider.md b/docs/design/external-provider.md index c5b4ade..8fefb44 100644 --- a/docs/design/external-provider.md +++ b/docs/design/external-provider.md @@ -6,7 +6,7 @@ The external provider verifies customer-managed DNS. The customer owns authorita This document defines the stage-1 design, not evidence that the implementation or a deployment has passed verification. Stage 1 preserves the normal EndpointRecordSet → generated RecordSet → ZoneUnit → Accepted/Programmed path. The platform's Phase 4 acceptance is a remote-member HTTPPublish becoming Ready through real resolution and HTTPS, not merely creating a Provider or reporting a configured Zone. Platform wiring and that acceptance proof are outside this provider's implementation boundary. -Verified-withdrawal evidence before retired VIP reuse is not part of this provider. Until a `Withdrawn` receipt contract exists, member replacement and deletion under the platform's `external` mode are unsupported. Kubernetes claim deletion described below is supported, but does not make member retirement or address reuse safe. +The provider verifies retired-address exclusion through independent Core `RecordWithdrawal` claims and `ZoneUnit.spec/status.withdrawals[]`. This evidence is separate from ordinary Kubernetes deletion bookkeeping. Platform retirement and address reuse require their own current-result consumption and allocation/drain gates; this document does not establish deployment acceptance or global cache eradication. ## Provider Declaration @@ -265,7 +265,45 @@ Process at most 32 due names per ZoneUnit reconcile. Resume fairly through priva External owns no DNS object. For a matching-UID ledger item with `DeletionRequested=true`, stop probes and mark `DeletionCompleted=true` after Kubernetes bookkeeping. Complete ZoneUnit finalization without a DNS delete call and without waiting for the customer to remove a wildcard. Existing core claim/endpoint finalizers still wait for the appropriate Kubernetes receipts and generated-child disappearance. Old-incarnation deletion completion must not finalize a replacement claim. -**Deletion completion is not verified DNS withdrawal.** Neither claim disappearance, observedGeneration, a completed verifier finalizer, nor a timer establishes that retired targets have disappeared from DNS or caches. A separate `Withdrawn` condition/receipt contract is required before retired VIP reuse; its schema, persistence, consumers, and cache-safety protocol are outside this provider's scope. The provider does not emit or accept `Withdrawn`. Platform member replacement/deletion under external remains unsupported until that contract exists; allocations must not be released based on this bookkeeping-only completion. +**Deletion completion is not verified DNS withdrawal.** Claim disappearance, observedGeneration, a completed verifier finalizer, and timer expiry do not prove address exclusion. Independent retained `RecordWithdrawal` claims continue verification after generated RecordSets and EndpointRecordSets disappear. Pending withdrawals do not affect unrelated active Zone/RecordSet `Programmed` aggregation. Every retained withdrawal, including a verified one, blocks Zone/ZoneUnit deletion until explicitly archived or removed. + +### Withdrawal Verification + +Withdrawal uses a separate exclusion evaluator, not the active exact-set evaluator: + +```text +excluded(address) := address ∉ normalize(terminal A ∪ terminal AAAA) +``` + +Every checked name is probed in both families through every effective Cluster/Recursive resolver and, with `expectedNameservers`, every discovered authority endpoint via every configured resolver. Required NS/SOA/delegation and authoritative-answer checks still apply. Replacement addresses may remain and differ between views. Complete NOERROR replacement answers prove current exclusion; NODATA/NXDOMAIN require valid negative SOA evidence. Missing views/families, failures, malformed or unresolved chains, loops, and CNAME/probe limits are Unknown, never partial success. A foreign CNAME boundary is not sufficient exclusion evidence: complete terminal A/AAAA evidence is required. + +A wildcard uses a stable concrete `dns-retire-` witness derived from the claim UID, Zone reference, and Unit UID, plus any retained publication witness for that address. Recreated claims use a different witness. `sampledWildcard=true` describes sampling, not exhaustive proof for descendants or explicit owners. + +The first complete all-view exclusion starts `absentSince=t0`. `notBefore` is the maximum of t0, retained positive-address/CNAME expiry bounds, and `t0 + negativeTTL` for every required negative observation; negative TTL is `min(SOA TTL, SOA.MINIMUM)`. Complete per-view/per-family evidence drives this calculation, not bounded diagnostic previews or minimum-TTL summaries. Repeated identical negatives do not slide t0; larger bounds extend but never shorten the window. Public horizons round up to a whole second. A separate complete probe at or after `notBefore`, later than the preceding attempt, is required even for zero TTL. Polling caps never shorten a cache horizon. + +Reappearance, failed/incomplete observations, changed policy/views, clock reversal, or a schedule stale beyond `nextProbeTime + 300s` invalidates success and the absence window while retaining known positive horizons. Restart preserves only matching persisted identity/policy/window evidence. Required resolver, expected-nameserver, and discovered-view history remains a floor: weakening or missing historical evidence reports `PolicyChanged` rather than proving equivalence. Retained claims are reverified; consumers also reject stale/future/missing timestamps and mismatched bindings, and require `verifiedAt >= notBefore`. + +`Accepted=True/Accepted` means valid authorized intent; rejection uses `InvalidWithdrawal` or `NotAllowed`. Core also reports `UnsupportedProvider`. Provider `Withdrawn` and `Ready` use: + +| Withdrawn | Ready | Reason | +| --- | --- | --- | +| False | False | `DNSStillResolves` | +| False | False | `NegativeTTLNotElapsed` | +| Unknown | False | `DNSProbeFailed` | +| Unknown | False | `PolicyChanged` | +| True | True | `WithdrawalVerified` | + +Messages retain these templates, with bounded 2048-byte diagnostics: + +- `DNS still resolves to retired address
in ().` +- `Retired address
is absent in every required view; waiting until for cache safety and a confirming probe.` +- `Retired address
was verified absent from in every required view at .` +- `External DNS withdrawal cannot be verified for in : .` +- `External DNS withdrawal policy changed for ; the prior required view policy remains a floor.` + +Core supplies a validated waiver's exact name, UID, immutable-spec digest, and receiving reservation namespace/name/UID. The provider echoes that scope and emits `OverrideAccepted=True/OperatorAuthorizedRelease` with message `Core-validated withdrawal waiver is bound to its exact UID and immutable scope.` This never changes `Withdrawn` or `Ready` into DNS success and performs no platform-object lookup. + +Withdrawal work shares the existing limiter, concurrency, transport, deadlines, jitter, and retry intervals. A separate persisted fair cursor schedules claims; each concrete checked name consumes one name-budget slot. When active and withdrawal work are both due, withdrawal selection receives half the pass budget and active work uses the remaining budget. Requeue uses the earlier active/withdrawal deadline (at least one second); a deleting Unit still verifies retained withdrawals. There is no timeout-based release. `dns_external_withdrawals_pending{reason}` counts nonverified claims with the closed labels `DNSStillResolves`, `NegativeTTLNotElapsed`, `DNSProbeFailed`, `PolicyChanged`, `InvalidWithdrawal`, `NotAllowed`, and `UnsupportedProvider`; other reasons fold into `DNSProbeFailed`. Existing probe, duration, and queue-delay metrics remain shared. ## Admission and Security diff --git a/docs/design/gateway-api-integration.md b/docs/design/gateway-api-integration.md index 5c49f54..80db7d9 100644 --- a/docs/design/gateway-api-integration.md +++ b/docs/design/gateway-api-integration.md @@ -57,6 +57,10 @@ Provider controllers still make final provider-side acceptance and programming d The Gateway controller creates one `EndpointRecordSet` per Gateway. Multiple routes and listeners on the same Gateway are deduplicated into a single endpoint intent so HTTP and HTTPS routes for the same hostname do not create competing Core `RecordSet` owners. +The source installs the endpoint-owned `endpoint.dns.appthrust.io/generated-recordsets` finalizer in the Create request. Updates are monotone for retirement intent: read the current ERS, preserve all existing retirement entries, append removed-address snapshots of old hostnames and removed-hostname snapshots of old IPs, then write desired live intent with resourceVersion concurrency. IDs use ERS UID and the pre-update generation; retries cannot overwrite concurrent obligations. The source never selects Zones/providers, creates Core withdrawal claims, prunes retirement entries, or treats capture as verified DNS absence. + +Protection defaults closed until a current-generation `WithdrawalProtected=False` establishes no withdrawal-capable binding. Collection/snapshot overflow reports `WithdrawalCapturePending`, preserves the old spec, and holds `Ready=False` rather than truncating. Successful source writes clear only their pending marker; the endpoint controller owns capture acknowledgment. + For a route: ```yaml @@ -190,6 +194,10 @@ authority or relax UID, generation, Gateway, listener, or hostname matching. ## Deletion and Recovery +Every source cleanup path uses the same retire-or-delete protocol: missing or deleting Gateway, current rejected Gateway admission, no remaining admitted hostname/usable target intent, removed generated intents, route disappearance/deletion or binding changes, and legacy route-labelled `cleanupForRoute` cleanup. Protected IP/hostname associations first move to retire-only ERS snapshots. DELETE waits for `WithdrawalCaptured=True` at the current ERS generation, then uses exact UID/resourceVersion preconditions. While capture is pending, cleanup returns `WithdrawalCapturePending`; it does not drop the ERS or strip finalizers. Hostname-only targets carry no address-withdrawal obligation; a current known-unprotected ERS follows ordinary deletion. + +The endpoint controller captures independent claims and performs generated-child cleanup. Gateway and Service source controllers share this protocol. Manual finalizer stripping or overwriting retained evidence is outside the supported contract. + An `EndpointRecordSet` may still be terminating when its route becomes accepted again. The Gateway controller must not update that terminating intent or remove its finalizers: provider cleanup belongs to the endpoint controller. diff --git a/docs/design/provider-extension.md b/docs/design/provider-extension.md index 187fe75..58bfc29 100644 --- a/docs/design/provider-extension.md +++ b/docs/design/provider-extension.md @@ -190,6 +190,14 @@ Only explicitly published core validations can be disabled. Initial disableable These validations cannot be disabled: `require-provider`, `require-zone-ref`, `require-type`, `require-name`, `require-cname-target`, `require-mx-records`, `require-caa-records`, `require-ns-name-servers`, record name grammar, lowercase canonical form, label length, wildcard placement, type-specific underscore allowance, delegated NS apex rejection, delegated NS wildcard rejection, CNAME target validation, type/body exclusivity, `supportedTypes` consistency, namespace / record access policy, and `ZoneUnit` ownership validation. +### Record Withdrawal Capability + +`spec.versions[].recordWithdrawal` is an optional `ProviderRecordWithdrawal` capability with `supported`, `schemas.statusProviderData.openAPIV3Schema`, and `additionalConditionReasons`. Omission or `supported=false` does not support independent withdrawal. Core checks both the served input and storage contracts before composing verification work. External advertises support; Route 53 and Cloudflare do not advertise or reconcile withdrawals. + +The status schema validates public `RecordWithdrawal.status.provider.data` and `ZoneUnit.status.withdrawals[].provider.data`, not provider-private scheduler state. External exposes only a bounded `views` preview there; typed claim/Unit evidence, never truncated previews, drives decisions. + +Core owns `Accepted=True/Accepted` and `Accepted=False` with `InvalidWithdrawal`, `NotAllowed`, or `UnsupportedProvider`. Withdrawal additional reasons use the closed contract: `Withdrawn=False` and `Ready=False` with `DNSStillResolves` or `NegativeTTLNotElapsed`; `Withdrawn=Unknown` and `Ready=False` with `DNSProbeFailed` or `PolicyChanged`; `Withdrawn=True` and `Ready=True` with `WithdrawalVerified`; and `OverrideAccepted=True/OperatorAuthorizedRelease`. External declares these eleven combinations. Admission validates schema fragments and rejects duplicate, core-acceptance, or out-of-contract reason combinations. + ### Controller Responsibility `Provider` is not reconciled by a controller. It is installed as a cluster-scoped resource by provider packages and used by the core validating webhook for provider-specific validation of `ZoneClass`, `Zone`, and `RecordSet`. It also publishes identity resource metadata used by UI and generic reference display.