diff --git a/content/master/composition/composition-revisions.md b/content/master/composition/composition-revisions.md index b9695f6f5..a38cb12a4 100644 --- a/content/master/composition/composition-revisions.md +++ b/content/master/composition/composition-revisions.md @@ -123,6 +123,158 @@ spec: name: example-18pdg ``` +## Roll out function changes + +{{}} +Referencing functions by OCI reference is an alpha feature. Enable it by +setting the `--enable-pipeline-oci-references` feature flag. +{{}} + +A `CompositionRevision` only captures what's in the `Composition`. A pipeline +step that uses `functionRef` calls whatever version of that Function is active +in the cluster. When you upgrade a Function every `CompositionRevision` that +references it starts calling the new version, including revisions you've pinned +XRs to with the `Manual` update policy. + +To make the function version part of the revision, reference the function +package directly with the {{}}function{{}} +field instead of using `functionRef`: + +```yaml {label="ociref",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: example +spec: + compositeTypeRef: + apiVersion: example.org/v1alpha1 + kind: PlatformDB + mode: Pipeline + pipeline: + - step: render-templates + function: xpkg.crossplane.io/crossplane-contrib/function-go-templating@sha256:2edec9149ed78cfc109fdda53fef847055e4789f1d5984ffc2f9ce2b54241abc + input: + apiVersion: gotemplating.fn.crossplane.io/v1beta1 + kind: GoTemplate + source: Inline + # Removed for brevity +``` + +The `function` field must be a fully qualified OCI reference that includes the +registry, the repository and a digest. A digest provides an immutable reference +to a specific version of a package, so a `CompositionRevision` that references a +function this way always runs the same function code. + +Each pipeline step uses either `function` or `functionRef`, not both. A pipeline +can contain a mix of steps using the two fields. + +You don't need to install a function before you reference it with `function`. +When Crossplane creates a `CompositionRevision`, it creates a `FunctionRevision` +for each function package the pipeline references. `CompositionRevisions` that +reference the same digest share a `FunctionRevision`. Different digests of the +same function run side by side, each with its own function pod. + +The `CompositionRevisions` that reference a `FunctionRevision` own it. The +`FunctionRevision` is garbage collected by Kubernetes when you delete the last +`CompositionRevision` that references it. + +Read the [Functions]({{}}) +documentation for more details on how Crossplane manages these functions. + +### Canary a function upgrade + +Combine `function` with a `compositionRevisionSelector` to roll a new function +version out to some XRs before others. + +Start with a `Composition` that calls version `v0.13.1` of +function-go-templating and has a +{{}}channel: stable{{}} label: + +```yaml {label="canary1",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + labels: + channel: stable + name: example +spec: + compositeTypeRef: + apiVersion: example.org/v1alpha1 + kind: PlatformDB + mode: Pipeline + pipeline: + - step: render-templates + function: xpkg.crossplane.io/crossplane-contrib/function-go-templating:v0.13.1@sha256:2edec9149ed78cfc109fdda53fef847055e4789f1d5984ffc2f9ce2b54241abc + input: + # Removed for brevity +``` + +Most of your XRs select the `stable` channel. XRs that should use the new +version first select `canary`: + +```yaml {copy-lines="none"} +apiVersion: example.org/v1alpha1 +kind: PlatformDB +metadata: + namespace: default + name: canary-db +spec: + storageGB: 20 + crossplane: + compositionRevisionSelector: + matchLabels: + channel: canary +``` + +To try function-go-templating `v0.13.2`, update the `function` reference +and change the label to +{{}}channel: canary{{}} in the same +edit: + +```yaml {label="canary2",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + labels: + channel: canary + name: example +spec: + compositeTypeRef: + apiVersion: example.org/v1alpha1 + kind: PlatformDB + mode: Pipeline + pipeline: + - step: render-templates + function: xpkg.crossplane.io/crossplane-contrib/function-go-templating:v0.13.2@sha256:20198cb6ea5b4b8a48042343410ec9031e5ec43431d874364ecd3eac8e861290 + input: + # Removed for brevity +``` + +Crossplane creates a new `CompositionRevision` and a new `FunctionRevision` +for `v0.13.2`. Both versions of the function are now running: + +```shell {copy-lines="1"} +kubectl get functionrevisions -o="custom-columns=NAME:.metadata.name,PACKAGE:.spec.image,STATE:.spec.desiredState" +NAME PACKAGE STATE +crossplane-contrib-function-go-templating-91622b9260e8 xpkg.crossplane.io/crossplane-contrib/function-go-templating@sha256:20198cb6ea5b4b8a48042343410ec9031e5ec43431d874364ecd3eac8e861290 Active +crossplane-contrib-function-go-templating-d66b8d13aa14 xpkg.crossplane.io/crossplane-contrib/function-go-templating@sha256:2edec9149ed78cfc109fdda53fef847055e4789f1d5984ffc2f9ce2b54241abc Active +``` + +XRs that select `canary` use the new `CompositionRevision` and call `v0.13.2` of +the function, while XRs that select `stable` stay on the previous +`CompositionRevision` that uses `v0.13.1`. + +When you're happy with `v0.13.2`, change the `Composition` label back to +`channel: stable`. Crossplane creates another `CompositionRevision` with the +same `v0.13.2` reference and the `stable` label, and the rest of your XRs use +it. + +{{}} +Use the `Manual` update policy for tighter control. Each XR stays on the +`CompositionRevision` that last composed it, and the pinned function version, +until you change its `compositionRevisionRef`. +{{< /hint >}} + ## Complete example This tutorial discusses how CompositionRevisions work and how they manage Composite Resource diff --git a/content/master/composition/compositions.md b/content/master/composition/compositions.md index 5cab1afbd..cda06ccfd 100644 --- a/content/master/composition/compositions.md +++ b/content/master/composition/compositions.md @@ -202,6 +202,59 @@ spec: region: "us-east-2" ``` + +### Reference a function by OCI reference + + +{{}} +Referencing functions by OCI reference is an alpha feature. Enable it by +setting the `--enable-pipeline-oci-references` feature flag. +{{}} + +Instead of a `functionRef`, a step can use the +{{}}function{{}} field to name a function +package directly. The reference must include the registry, the repository and +a digest. + +```yaml {label="ociref",copy-lines="none"} +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +# Removed for Brevity +spec: + # Removed for Brevity + mode: Pipeline + pipeline: + - step: create-bucket + function: xpkg.crossplane.io/crossplane-contrib/function-go-templating@sha256:2edec9149ed78cfc109fdda53fef847055e4789f1d5984ffc2f9ce2b54241abc + input: + apiVersion: gotemplating.fn.crossplane.io/v1beta1 + kind: GoTemplate + source: Inline + inline: + template: | + apiVersion: s3.aws.m.upbound.io/v1beta1 + kind: Bucket + metadata: + annotations: + gotemplating.fn.crossplane.io/composition-resource-name: storage-bucket + spec: + forProvider: + region: "us-east-2" +``` + +You don't need to install the function first. Crossplane installs it when it +creates the Composition's +[revision]({{}}). + +Because the digest is part of the Composition, each Composition revision always +calls the same version of the function. This lets you roll out a new function +version to some composite resources before others. Read +[Roll out function changes]({{}}) +for an example. + +Each pipeline step uses either `function` or `functionRef`, not both. A pipeline +can contain a mix of steps using the two fields. + ### Use a pipeline of functions in a composition Crossplane can ask more than one Function what to do when a composite resource diff --git a/content/master/get-started/install.md b/content/master/get-started/install.md index ddea92d70..2d7afadd6 100644 --- a/content/master/get-started/install.md +++ b/content/master/get-started/install.md @@ -115,6 +115,7 @@ at the table below. | Alpha | `--enable-function-response-cache` | Enable caching of composition function responses to improve performance. | | Alpha | `--enable-operations` | Enable support for Operations. | | Alpha | `--enable-pipeline-inspector` | Enable support for emitting data about function pipeline runs to a sidecar. | +| Alpha | `--enable-pipeline-oci-references` | Enable support for referencing functions by OCI reference in Composition and Operation pipelines. | | Alpha | `--enable-provider-deletion-protection` | Enable automatic protection of Providers from deletion when they have active managed resources. Requires `--enable-usages`. | | Alpha | `--enable-signature-verification` | Enable support for package signature verification via ImageConfig API. | {{< /table >}} diff --git a/content/master/operations/operation.md b/content/master/operations/operation.md index 75daba17e..efcb44661 100644 --- a/content/master/operations/operation.md +++ b/content/master/operations/operation.md @@ -259,6 +259,61 @@ spec: healthEndpoint: /health ``` + +### Reference a function by OCI reference + + +{{}} +Referencing functions by OCI reference is an alpha feature. Enable it by +setting the `--enable-pipeline-oci-references` feature flag. +{{}} + +Instead of a `functionRef`, a step can use the +{{}}function{{}} field to name a function +package directly. The reference must include the registry, the repository and +a digest. + +```yaml {label="op-ociref"} +apiVersion: ops.crossplane.io/v1alpha1 +kind: Operation +metadata: + name: create-configmap +spec: + mode: Pipeline + pipeline: + - step: create-a-configmap + function: xpkg.crossplane.io/crossplane-contrib/function-dummy@sha256:b8a8590f61fb002115313d8f5f756675aa6b55ef6c31568ebcbcf65d1f17a727 + input: + apiVersion: dummy.fn.crossplane.io/v1beta1 + kind: Response + response: + desired: + resources: + configmap: + resource: + apiVersion: v1 + kind: ConfigMap + metadata: + namespace: default + name: example + data: + key: value +``` + +You don't need to install the function first. Crossplane installs it before it +runs the Operation, and waits for it to be healthy. + +{{}} +Because it's normal for functions to take some time to start, Crossplane retries +an `Operation` until all functions specified by OCI reference become +ready. These retries aren't counted toward the Operation's failure limit. This +means an incorrect reference can cause the `Operation` to retry forever, so be +careful that all references are valid and correct. +{{}} + +Each pipeline step uses either `function` or `functionRef`, not both. A pipeline +can contain a mix of steps using the two fields. + ### RBAC permissions If your Operation needs to access resources that Crossplane doesn't have diff --git a/content/master/packages/functions.md b/content/master/packages/functions.md index a1cc03138..d056ff2c2 100644 --- a/content/master/packages/functions.md +++ b/content/master/packages/functions.md @@ -69,3 +69,82 @@ spec: # Removed for brevity ``` {{}} + +## Functions managed by Crossplane + +{{}} +Referencing functions by OCI reference is an alpha feature. Enable it by +setting the `--enable-pipeline-oci-references` feature flag. +{{}} + +A Composition or Operation pipeline step can reference a function package by +digest with the `function` field instead of naming an installed Function with +`functionRef`. Read +[Roll out function changes]({{}}) +to learn why you might want to use this feature. + +You don't install these functions yourself. For each digest a pipeline +references, Crossplane creates a `FunctionRevision`. Crossplane groups the +revisions under a Function named after the package repository. For example, +`xpkg.crossplane.io/crossplane-contrib/function-go-templating` becomes +`crossplane-contrib-function-go-templating`, the same name Crossplane uses when +it installs the function as a +[dependency]({{}}). + +{{}} +When Crossplane creates a `Function` and `FunctionRevision` for pipeline steps, +it sets an owner reference to the relevant pipeline resource +(`CompositionRevision` or `Operation`). If you later want to manually install +the function by setting `spec.package`, you should remove the owner references; +otherwise, Kubernetes garbage collection can delete the Function when you delete +the pipeline resource. +{{}} + +The `CompositionRevisions` and `Operations` that reference a function by OCI +reference own its `FunctionRevision`. The `FunctionRevision` is garbage +collected by Kubernetes when you delete the last `CompositionRevision` or +`Operation` that references it. + +### External revisions + +For Functions installed by users or as dependencies, Crossplane's package +manager sets one active revision. The revisions created for `function` +references in the `CompositionRevisions` and `Operations` are +different. Controllers outside the package manager manage these "external" +revisions. The package manager pulls the package, runs a function pod for each +one, and records their names in the parent `Function`'s +`status.externalRevisionRefs` field. Multiple external revisions of the same +Function can be active at once and the package manager doesn't determine which +should be active. + +A Function that only has external revisions has no `spec.package`. Its +{{}}status.externalRevisionRefs{{}} lists +its revisions: + +```yaml {label="ext",copy-lines="none"} +apiVersion: pkg.crossplane.io/v1 +kind: Function +metadata: + name: crossplane-contrib-function-go-templating +spec: + # No package + # Removed for brevity +status: + conditions: + # Removed for brevity + externalRevisionRefs: + - name: crossplane-contrib-function-go-templating-91622b9260e8 + - name: crossplane-contrib-function-go-templating-d66b8d13aa14 +``` + +This Function reports `HEALTHY` as `True` only when all its external revisions +are healthy. It has no current revision. + +A Function can have both a `spec.package` and external revisions. This happens +when you install a function and also reference it by digest in a pipeline. The +package manager manages the revisions for `spec.package` as usual and lists the +others in `status.externalRevisionRefs`. Pipeline steps that use `functionRef` +call only the revision the package manager manages. + +If another package depends on a function that has only external revisions, the +package manager takes over the Function and sets its `spec.package`.