Add enable-identity-aware-routing ops-file (RFC0055)

operations/README.md

@@ -47,6 +47,7 @@ This is the README for Ops-files. To learn more about `cf-deployment`, go to the
 | [`enable-cc-worker-metrics.yml`](enable-cc-worker-metrics.yml) | Enable metrics for cc-workers. | This will setup the metrics endpoint, configure the prom_scraper job and uses mTLS. If you want to use another scraper than prom_scraper you can additionally set `cc.prom_scraper.disabled` to true. | **NO** |
 | [`enable-scheduler-metrics.yml`](enable-scheduler-metrics.yml) | Enable metrics for schedulers. | This will setup the metrics endpoint, configure the prom_scraper job and uses mTLS. If you want to use another scraper than prom_scraper you can additionally set `cc.prom_scraper.disabled` to true. | **NO** |
 | [`enable-cpu-throttling.yml`](enable-cpu-throttling.yml) | Configure Garden containers with CPU entitlement. | This ops file requires `set-cpu-weight.yml`. | **YES** |
+| [`enable-identity-aware-routing.yml`](enable-identity-aware-routing.yml) | Enables identity-aware routing with mTLS domains. | Creates a new self-signed CA (`apps_identity_ca`) for identity routing on `*.apps.identity` domain. Generates router TLS certificate and configures gorouter and Diego to trust the CA in both cflinuxfs4 and cflinuxfs5 app containers. | **NO** |
 | [`enable-nfs-ldap.yml`](enable-nfs-ldap.yml) | Enables LDAP authentication for NFS volume services | Requires `enable-nfs-volume-service.yml`. Introduces [new variables](example-vars-files/vars-enable-nfs-ldap.yml) | **NO** |
 | [`enable-nfs-volume-service.yml`](enable-nfs-volume-service.yml) | Enables volume support and deploys an NFS broker and volume driver | As of cf-deployment v2, you must use the `nfsbrokerpush` errand to cf push the nfs broker after `bosh deploy` completes. | **YES** |
 | [`enable-privileged-container-support.yml`](enable-privileged-container-support.yml) | Enables Diego privileged container support. | | **NO** |
@@ -75,6 +76,7 @@ This is the README for Ops-files. To learn more about `cf-deployment`, go to the
 | [`use-latest-stemcell.yml`](use-latest-stemcell.yml) | Use the latest stemcell available on your BOSH director instead of the one in `cf-deployment.yml`. **Caution**: This ops-file should not be used in conjunction with `use-compiled-releases.yml`, since the latter relies on a specific stemcell version being used. |  | **NO** |
 | [`use-latest-windows2019-stemcell.yml`](use-latest-windows2019-stemcell.yml) | Use the latest `windows2019` stemcell available on your BOSH director instead of the one in `windows2019-cell.yml` | Requires `windows2019-cell.yml` | **NO** |
 | [`use-metric-store.yml`](use-metric-store.yml) | Adds a single-node metric store. | | **NO** |
+| [`use-operator-provided-identity-routing-domain.yml`](use-operator-provided-identity-routing-domain.yml) | Allows operators to provide a custom domain for identity-aware routing. | Requires `enable-identity-aware-routing.yml`. Introduces [new variables](example-vars-files/vars-use-operator-provided-identity-routing-domain.yml) for the custom identity routing domain. | **NO** |
 | [`use-operator-provided-router-tls-certificates.yml`](use-operator-provided-router-tls-certificates.yml) | Allows an operator to provide their own certificates for the gorouter by providing variables [`router_ssl_pem`](example-vars-files/vars-use-operator-provided-router-tls-certificates.yml) | This is required if using AWS Network Load Balancers. | **YES** |
 | [`use-postgres.yml`](use-postgres.yml) | Replaces the MySQL instance group with a postgres instance group. **Warning**: this will lead to total data loss if applied to an existing deployment with MySQL or removed from an existing deployment with postgres. | If combined with [`use-compiled-releases.yml`](use-compiled-releases.yml), you must apply this ops file **after** `use-compiled-releases.yml`. | **YES** |
 | [`use-trusted-ca-cert-for-apps.yml`](use-trusted-ca-cert-for-apps.yml) | Injects the CA specified with `trusted_cert_for_apps` into the Diego `rep` job's trust store and cf-deployment's default root filesystem | Applications that *explicitly* look in the canonical location (`/etc/cf-system-certificates`) will trust certificates signed by the given CA, regardless of filesystem. Applications that use the default root filesystem will trust certificates signed by the given CA *implicitly*. <br/> Please see the [documentation](https://docs.cloudfoundry.org/running/trusted-system-certificates.html) for information about configuring additional trusted CA certificates. | **NO** |

operations/enable-identity-aware-routing.yml

@@ -0,0 +1,75 @@
+---
+# BOSH DNS alias: *.apps.identity -> router instances
+# (BOSH DNS uses underscore prefix for wildcards; _.apps.identity matches *.apps.identity)
+- type: replace
+  path: /addons/name=bosh-dns-aliases/jobs/name=bosh-dns-aliases/properties/aliases/-
+  value:
+    domain: _.apps.identity
+    targets:
+    - deployment: cf
+      domain: bosh
+      instance_group: router
+      network: default
+      query: '*'
+
+# Configure mTLS domain on gorouter.
+# Gorouter verifies client certs against diego_instance_identity_ca (existing CF PKI).
+# sanitize_set: strips any incoming XFCC header (prevents spoofing) and sets a fresh one from the verified client cert.
+# xfcc_format: envoy format is required by Diego/Envoy sidecars consuming the header.
+- type: replace
+  path: /instance_groups/name=router/jobs/name=gorouter/properties/router/domains?
+  value:
+    - name: "*.apps.identity"
+      ca_certs: ((diego_instance_identity_ca.certificate))
+      forwarded_client_cert: sanitize_set
+      xfcc_format: envoy
+
+# Add *.apps.identity TLS cert to gorouter (served via SNI)
+- type: replace
+  path: /instance_groups/name=router/jobs/name=gorouter/properties/router/tls_pem/-
+  value:
+    cert_chain: ((apps_identity_router_tls.certificate))
+    private_key: ((apps_identity_router_tls.private_key))
+
+# Trust apps_identity_ca in cflinuxfs4 app containers so outbound TLS to
+# *.apps.identity succeeds.
+- type: replace
+  path: /instance_groups/name=diego-cell/jobs/name=cflinuxfs4-rootfs-setup/properties/cflinuxfs4-rootfs/trusted_certs/-
+  value: ((apps_identity_ca.certificate))
+
+# Trust apps_identity_ca in cflinuxfs5 app containers so outbound TLS to
+# *.apps.identity succeeds. cflinuxfs5-rootfs-setup is part of cf-deployment.yml
+# unconditionally (see #1354), so no companion ops-file is needed here.
+- type: replace
+  path: /instance_groups/name=diego-cell/jobs/name=cflinuxfs5-rootfs-setup/properties/cflinuxfs5-rootfs/trusted_certs/-
+  value: ((apps_identity_ca.certificate))
+
+# Trust apps_identity_ca in the Diego rep container trust store (Envoy sidecars).
+- type: replace
+  path: /instance_groups/name=diego-cell/jobs/name=rep/properties/containers/trusted_ca_certificates/-
+  value: ((apps_identity_ca.certificate))
+
+# Generate apps_identity_ca: self-signed CA that issues the router SNI cert and is
+# trusted by app containers.
+- type: replace
+  path: /variables/-
+  value:
+    name: apps_identity_ca
+    type: certificate
+    options:
+      common_name: apps-identity-ca
+      is_ca: true
+
+# Generate apps_identity_router_tls: server cert for *.apps.identity served via SNI.
+- type: replace
+  path: /variables/-
+  value:
+    name: apps_identity_router_tls
+    type: certificate
+    options:
+      ca: apps_identity_ca
+      common_name: "*.apps.identity"
+      alternative_names:
+        - "*.apps.identity"
+      extended_key_usage:
+        - server_auth

operations/example-vars-files/vars-use-operator-provided-identity-routing-domain.yml

@@ -0,0 +1 @@
+identity_routing_domain: apps.example.com

operations/use-operator-provided-identity-routing-domain.yml

@@ -0,0 +1,28 @@
+---
+# Override the hardcoded apps.identity domain set by enable-identity-aware-routing.yml.
+# Apply on top of that ops-file:
+#
+#   bosh deploy cf-deployment.yml \
+#     -o operations/enable-identity-aware-routing.yml \
+#     -o operations/use-operator-provided-identity-routing-domain.yml \
+#     -v identity_routing_domain=apps.example.com
+
+# Override BOSH DNS alias domain.
+- type: replace
+  path: /addons/name=bosh-dns-aliases/jobs/name=bosh-dns-aliases/properties/aliases/domain=_.apps.identity/domain
+  value: "_.((identity_routing_domain))"
+
+# Override gorouter mTLS domain name.
+- type: replace
+  path: /instance_groups/name=router/jobs/name=gorouter/properties/router/domains/name=*.apps.identity/name
+  value: "*.((identity_routing_domain))"
+
+# Override TLS cert common_name.
+- type: replace
+  path: /variables/name=apps_identity_router_tls/options/common_name
+  value: "*.((identity_routing_domain))"
+
+# Override TLS cert SAN (first and only entry).
+- type: replace
+  path: /variables/name=apps_identity_router_tls/options/alternative_names/0
+  value: "*.((identity_routing_domain))"

units/tests/semantic_test/semantic_test.go

@@ -244,6 +244,9 @@ func TestSemantic(t *testing.T) {
 				if ca == "((diego_instance_identity_ca.certificate))" {
 					continue
 				}
+				if ca == "((apps_identity_ca.certificate))" {
+					continue
+				}
 
 				t.Errorf("CAs should be referenced from their certificate variables: %s in %s", ca, strings.Replace(path, operationsSubDirectory, "operations", 1))
 			}

units/tests/standard_test/operations.yml

@@ -23,6 +23,7 @@ enable-cc-v2-rate-limiting.yml:
 enable-cc-worker-metrics.yml: {}
 enable-scheduler-metrics.yml: {}
 enable-cpu-throttling.yml: {}
+enable-identity-aware-routing.yml: {}
 enable-nfs-ldap.yml:
   ops:
   - enable-nfs-volume-service.yml
@@ -136,6 +137,12 @@ use-online-windows2019fs.yml:
   ops:
   - windows2019-cell.yml
   - use-online-windows2019fs.yml
+use-operator-provided-identity-routing-domain.yml:
+  ops:
+  - enable-identity-aware-routing.yml
+  - use-operator-provided-identity-routing-domain.yml
+  varsfiles:
+  - example-vars-files/vars-use-operator-provided-identity-routing-domain.yml
 use-operator-provided-router-tls-certificates.yml:
   varsfiles:
   - example-vars-files/vars-use-operator-provided-router-tls-certificates.yml