diff --git a/TEST.md b/TEST.md index 7b4b535bc..3bb518248 100644 --- a/TEST.md +++ b/TEST.md @@ -1,6 +1,7 @@ # Sourcegraph Helm Chart Test Guide -This is a temporary test guide before we have more thorough automated integration tests. +This is a temporary test guide before we have more thorough automated +integration tests. ## Lint @@ -12,7 +13,8 @@ helm lint charts/sourcegraph/. ## Unit testing -We utilize [helm-unittest](https://github.com/helm-unittest/helm-unittest/), a BDD styled unit test framework, to validate our helm chart. +We utilize [helm-unittest](https://github.com/helm-unittest/helm-unittest/), a +BDD styled unit test framework, to validate our helm chart. helm-unittest can be installed with: @@ -20,15 +22,33 @@ helm-unittest can be installed with: helm plugin install https://github.com/helm-unittest/helm-unittest ``` -Once the plugin is installed, you can run the unit tests using the following: +Once the plugin is installed, you can run the unit tests for a single chart: ```bash helm unittest ./charts/sourcegraph ``` -We currently do not have testing best practices or require unit tests for new changes, so add test cases at your best judgement if possible. +Or run every chart the same way CI does: -You may check out our [existing test cases](https://github.com/sourcegraph/deploy-sourcegraph-helm/tree/main/charts/sourcegraph/tests) and helm-unittest [docs](https://github.com/helm-unittest/helm-unittest/blob/master/DOCUMENT.md). +```bash +helm unittest \ + ./charts/sourcegraph \ + ./charts/sourcegraph-migrator \ + ./charts/sourcegraph-executor/k8s \ + ./charts/sourcegraph-executor/dind +``` + +Each chart keeps its tests in a `tests/` directory next to its `templates/`. CI +runs all of them via `scripts/ci/helm-unittest.sh`, so a new chart only needs a +`tests/` directory and an entry in that script. + +We currently do not have testing best practices or require unit tests for new +changes, so add test cases at your best judgement if possible. + +You may check out our [existing test +cases](https://github.com/sourcegraph/deploy-sourcegraph-helm/tree/main/charts/sourcegraph/tests) +and helm-unittest +[docs](https://github.com/helm-unittest/helm-unittest/blob/master/DOCUMENT.md). ## Manual testing @@ -52,11 +72,16 @@ sourcegraph: # More values to be added in order to test your change ``` -Make sure you test both enabled and disabled toggles. For example, if you added a new values to conditional render some templates, turn it on and off in the `override.yaml` to make sure they both work. You can also include your `override.yaml` in the `Test plan` during PR review to help others understand your testing strategy. +Make sure you test both enabled and disabled toggles. For example, if you added +a new values to conditional render some templates, turn it on and off in the +`override.yaml` to make sure they both work. You can also include your +`override.yaml` in the `Test plan` during PR review to help others understand +your testing strategy. ### Update the image tag -You have two options to target specific Sourcegraph version. Add the below to your `override.yaml`: +You have two options to target specific Sourcegraph version. Add the below to +your `override.yaml`: ```yaml sourcegraph: @@ -65,11 +90,14 @@ sourcegraph: useGlobalTagAsDefault: true ``` -Alternatively, you can use `sg` to update the image tags directly in the default `values.yaml`, **DO NOT COMMIT** the changes: +Alternatively, you can use `sg` to update the image tags directly in the default +`values.yaml`, **DO NOT COMMIT** the changes: ```sh cd charts/sourcegraph -sg ops update-images -kind helm -pin-tag 6.10.0 -cr-username=$DOCKER_HUB_USERNAME -cr-password=$DOCKER_HUB_PERSONAL_ACCESS_TOKEN . +sg ops update-images -kind helm -pin-tag 6.10.0 \ + -cr-username=$DOCKER_HUB_USERNAME \ + -cr-password=$DOCKER_HUB_PERSONAL_ACCESS_TOKEN . ``` ### Inspect the entire rendered template @@ -82,20 +110,25 @@ helm template -f ./override.yaml sourcegraph charts/sourcegraph/. ### Inspect the diff -Perform a diff of the rendered helm manifests before and after your change. There're many ways to produce the diff: +Perform a diff of the rendered helm manifests before and after your change. +There're many ways to produce the diff: -- Run `helm template` before and after the change, then run `diff bundle.old.yaml bundle.new.yaml`. -- Run `helm install` before the change, then run `helm diff` to inspect the diff. +- Run `helm template` before and after the change, then run `diff + bundle.old.yaml bundle.new.yaml`. +- Run `helm install` before the change, then run `helm diff` to inspect the + diff. ### Deploy the chart You should make sure your change can be deployed ```sh -helm upgrade --install --create-namespace -n sourcegraph -f ./override.yaml sourcegraph charts/sourcegraph/. +helm upgrade --install --create-namespace -n sourcegraph \ + -f ./override.yaml sourcegraph charts/sourcegraph/. ``` -Follow the [confirm instance health] guide but use `kubectl port-forward` instead to validate instance health. +Follow the [confirm instance health] guide but use `kubectl port-forward` +instead to validate instance health. ```sh kubectl -n sourcegraph port-forward svc/sourcegraph-frontend 30080 diff --git a/charts/sourcegraph-executor/dind/tests/executor_test.yaml b/charts/sourcegraph-executor/dind/tests/executor_test.yaml index 8c972552c..ebc890027 100644 --- a/charts/sourcegraph-executor/dind/tests/executor_test.yaml +++ b/charts/sourcegraph-executor/dind/tests/executor_test.yaml @@ -2,32 +2,46 @@ suite: executor templates: - executor/docker-daemon.ConfigMap.yaml - executor/executor.Deployment.yaml - - executor/executor.Service.yaml tests: - - it: should render the Deployment and Service if executor is enabled + - it: should render the ConfigMap and a single Deployment named after the queue set: executor: - enabled: true + queueName: batches asserts: + - containsDocument: + kind: ConfigMap + apiVersion: v1 + name: docker-config + template: executor/docker-daemon.ConfigMap.yaml - containsDocument: kind: Deployment apiVersion: apps/v1 - name: executor + name: executor-batches + template: executor/executor.Deployment.yaml + - contains: + path: spec.template.spec.containers[0].env + content: + name: EXECUTOR_QUEUE_NAME + value: batches template: executor/executor.Deployment.yaml - - containsDocument: - kind: Service - apiVersion: v1 - name: executor - template: executor/executor.Service.yaml - - it: should not render any resources if executor is disabled + - it: should render one Deployment per entry in queues + template: executor/executor.Deployment.yaml set: - executor: - enabled: false + queues: + - name: codeintel + - name: batches + queueName: batches asserts: - hasDocuments: - count: 0 - templates: - - executor/docker-daemon.ConfigMap.yaml - - executor/executor.Deployment.yaml - - executor/executor.Service.yaml + count: 2 + - containsDocument: + kind: Deployment + apiVersion: apps/v1 + name: executor-codeintel + documentIndex: 0 + - containsDocument: + kind: Deployment + apiVersion: apps/v1 + name: executor-batches + documentIndex: 1 diff --git a/charts/sourcegraph-executor/dind/tests/privateDockerRegistry_test.yaml b/charts/sourcegraph-executor/dind/tests/privateDockerRegistry_test.yaml index ac3f71388..cbff736c7 100644 --- a/charts/sourcegraph-executor/dind/tests/privateDockerRegistry_test.yaml +++ b/charts/sourcegraph-executor/dind/tests/privateDockerRegistry_test.yaml @@ -1,29 +1,23 @@ suite: privateDockerRegistry templates: - - private-docker-registry.Deployment.yaml - - private-docker-registry.PersistentVolumeClaim.yaml - - private-docker-registry.Service.yaml + - private-docker-registry/private-docker-registry.Statefulset.yaml + - private-docker-registry/private-docker-registry.Service.yaml tests: - - it: should render the Deployment, Service and PVC if registry is enabled + - it: should render the StatefulSet and Service if registry is enabled set: privateDockerRegistry: enabled: true asserts: - containsDocument: - kind: Deployment + kind: StatefulSet apiVersion: apps/v1 name: private-docker-registry - template: private-docker-registry.Deployment.yaml + template: private-docker-registry/private-docker-registry.Statefulset.yaml - containsDocument: kind: Service apiVersion: v1 name: private-docker-registry - template: private-docker-registry.Service.yaml - - containsDocument: - kind: PersistentVolumeClaim - apiVersion: v1 - name: private-docker-registry - template: private-docker-registry.PersistentVolumeClaim.yaml + template: private-docker-registry/private-docker-registry.Service.yaml - it: should not render any resources if registry is disabled set: @@ -32,7 +26,3 @@ tests: asserts: - hasDocuments: count: 0 - templates: - - private-docker-registry.Deployment.yaml - - private-docker-registry.PersistentVolumeClaim.yaml - - private-docker-registry.Service.yaml diff --git a/charts/sourcegraph-migrator/tests/migrator_test.yaml b/charts/sourcegraph-migrator/tests/migrator_test.yaml new file mode 100644 index 000000000..126d7c2a1 --- /dev/null +++ b/charts/sourcegraph-migrator/tests/migrator_test.yaml @@ -0,0 +1,129 @@ +suite: migrator +templates: + - migrator/sourcegraph-migrator.Job.yaml +tests: + - it: should render a Job with a random name suffix and default args + asserts: + - isKind: + of: Job + - matchRegex: + path: metadata.name + pattern: "^migrator-[a-z0-9]{5}$" + - matchRegex: + path: spec.template.spec.containers[0].image + pattern: "^index.docker.io/sourcegraph/migrator:" + - equal: + path: spec.template.spec.containers[0].args + value: + - up + - -db=all + - equal: + path: spec.template.spec.restartPolicy + value: Never + + - it: should use migrator.name and migrator.args when set + set: + migrator: + name: db-migrate + args: + - upto + - -db=frontend + - -target=1648051770 + asserts: + - matchRegex: + path: metadata.name + pattern: "^db-migrate-[a-z0-9]{5}$" + - equal: + path: spec.template.spec.containers[0].args + value: + - upto + - -db=frontend + - -target=1648051770 + + - it: should read all three database credentials from the default secrets + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: PGHOST + valueFrom: + secretKeyRef: + key: host + name: pgsql-auth + - contains: + path: spec.template.spec.containers[0].env + content: + name: CODEINTEL_PGHOST + valueFrom: + secretKeyRef: + key: host + name: codeintel-db-auth + - contains: + path: spec.template.spec.containers[0].env + content: + name: CODEINSIGHTS_PGHOST + valueFrom: + secretKeyRef: + key: host + name: codeinsights-db-auth + + - it: should only swap the secret for the database that sets existingSecret + set: + pgsql: + auth: + existingSecret: "my-secret" + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: PGHOST + valueFrom: + secretKeyRef: + key: host + name: "my-secret" + - contains: + path: spec.template.spec.containers[0].env + content: + name: CODEINTEL_PGHOST + valueFrom: + secretKeyRef: + key: host + name: codeintel-db-auth + + - it: should append migrator.env after the database credentials + set: + migrator: + env: + SRC_LOG_LEVEL: + value: dbug + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: SRC_LOG_LEVEL + value: dbug + + - it: should prefer migrator.priorityClassName over the global one + set: + sourcegraph: + priorityClassName: global-class + migrator: + priorityClassName: migrator-class + asserts: + - equal: + path: spec.template.spec.priorityClassName + value: migrator-class + + - it: should fall back to the global priorityClassName + set: + sourcegraph: + priorityClassName: global-class + asserts: + - equal: + path: spec.template.spec.priorityClassName + value: global-class + + - it: should not set priorityClassName by default + asserts: + - isNull: + path: spec.template.spec.priorityClassName diff --git a/scripts/ci/helm-unittest.sh b/scripts/ci/helm-unittest.sh index 3ec626947..5de3ebe5d 100755 --- a/scripts/ci/helm-unittest.sh +++ b/scripts/ci/helm-unittest.sh @@ -10,4 +10,10 @@ HELM_UNITTEST_VERSION="v1.0.2" helm plugin install https://github.com/helm-unittest/helm-unittest --version "$HELM_UNITTEST_VERSION" --verify=false ### Run the helm tests -helm unittest -q charts/sourcegraph +# helm unittest runs every chart even after a failure, prints a per-chart +# summary, and exits non-zero if any chart failed. +helm unittest \ + charts/sourcegraph \ + charts/sourcegraph-migrator \ + charts/sourcegraph-executor/k8s \ + charts/sourcegraph-executor/dind