diff --git a/cluster-api/README.md b/cluster-api/README.md index 01850bb2..568a0ad3 100644 --- a/cluster-api/README.md +++ b/cluster-api/README.md @@ -2,29 +2,55 @@ This project contains the current versions of Kubernetes that the Cloud Group will support and builds images for. + +## Contents: + +- [Setup](#setup) + + + + + ## Setup -1. Install Ansible and any other pip requirements: +1. Clone this repository. + ```shell + git clone https://github.com/stfc/cloud-image-builders.git + cd cloud-image-builders/cluster-api + ``` + +2. Install Python requirements. ```shell sudo apt-get install python3-venv unzip -y # or sudo dnf install python3-venv unzip -y + # Then python3 -m venv venv source venv/bin/activate - pip install -r os_builders/requirements.txt + pip install -r ../os_builders/requirements.txt ``` 2. Install Packer and dependencies: ```shell - cd os_builders + cd ../os_builders # If sudo is passwordless: ansible-playbook prep_builder.yml # If password is required for sudo: ansible-playbook prep_builder.yml --ask-become-pass + cd ../cluster-api + ``` +3. Set up OpenStack authentication. Get a clouds.yaml for the **packer** project. See [here](https://stfc.atlassian.net/wiki/spaces/CLOUDKB/pages/211484774/Application+Credentials) + ```shell + # Copy clouds.yaml to the config directory + mkdir -p ~/.config/openstack + cp ~/.config/openstack/clouds.yaml + + # Export the cloud name in clouds.yaml, default is openstack + export OS_CLOUD=openstack ``` -## Rate Limiting +### Rate Limiting (Optional) You may run into GitHub rate limiting issues when building images. To avoid this, you can set the following environment variable: @@ -32,16 +58,57 @@ You may run into GitHub rate limiting issues when building images. To avoid this The token can be generated from your GitHub settings, under developer access, and only needs the `public_repo` scope (i.e. the default). -## OpenStack authentication +## Update the CAPI image versions + +1. Clone the repository with write access, usually by SSH + ```shell + git clone git@github.com:stfc/cloud-image-builders.git + cd cloud-image-builders/cluster-api + ``` -You need to set up credentials for OpenStack authentication as we are using a remote builder. Create a clouds.yaml application credential and place it into `~/.config/openstack/clouds.yaml`. See [here](https://stfc.atlassian.net/wiki/spaces/CLOUDKB/pages/211484774/Application+Credentials) for help. +2. Create a new branch called update_k8s_versions + ```shell + git switch -c update_k8s_versions + ``` -Addtionally, you will need to specify the cloud name in your session by exporting it. By default this will be `openstack` as seen in the standard `clouds.yaml` file: +3. Update the k8s submodule ```shell - export OS_CLOUD= + git submodule update --init --recursive --remote ``` -## Build a specific version +4. Perform the following checks against the releases page [here](https://kubernetes.io/releases/). + 1. If there is a new version, add a new file in [versions](./versions). Use the other versions as a template. We don't release the latest version. + ```shell + cd versions + cp v1_35.json v1_36.json + + # Update to the version number + vim v1_36.json + ``` + 2. If [versions](./versions) contains more than 3 version files, delete the oldest so we have only 3. + +5. Run update_capi_versions.sh and update each version file with the latest version. + ```shell + ./update_capi_versions.sh + ... + (venv) venv ❯ ./update_capi_versions.sh + kubernetes_series kubernetes_semver kubernetes_deb_version + ------------------- ------------------- ------------------------ + v1.32 v1.32.13 1.32.13-1.1 + v1.33 v1.33.13 1.33.13-1.1 + v1.34 v1.34.11 1.34.11-1.1 + ... + +6. Commit these changes to the branch and make a pull request + ```shell + git add versions/ ../k8s-image-builder + git commit + ... + Commit message + ... + git push --set-upstream origin update_k8s_versions + +## Build a CAPI image 1. Grab the latest version of the K8s Image Builder submodule: ```shell @@ -69,41 +136,6 @@ Addtionally, you will need to specify the cloud name in your session by exportin ``` 4. Follow steps for release [here](#update-an-image-for-release) -## Adding a new version -1. Update the image builder: - ```shell - cd .. # back to repo root - git submodule update --init --recursive --remote - ``` -2. Add new version file: - 1. Navigate to https://kubernetes.io/releases/ - 2. Find the version you want to add or update - 3. Create/Update the semver in the relevant JSON file. There should be a 1:1 mapping of JSON files to major versions of Kubernetes. E.g. a file for 1.24, 1.25, etc. -3. Follow steps to build a specific version [here](#build-a-specific-version) -4. Follow steps for release [here](#update-an-image-for-release) - -## Rebuild all images -1. Update the image builder: - ```shell - cd .. # back to repo root - git submodule update --init --recursive --remote - ``` -2. Update patch versions in version files - ``` - # Contents of ./versions/v1_33.json - { - "kubernetes_series": "v1.33", - "kubernetes_semver": "v1.33.3", -> "v1.33.4" - "kubernetes_deb_version": "1.33.3-1.1" -> "v1.33.4-1.1" - } - ``` -3. Run build-all.sh - ```shell - cd scripts - ./build-all.sh # dev or prod - ``` -4. Follow steps for release for each image [Update and image for release](#update-an-image-for-release) - ## Update an image for release 1. Check the image you are working with before you make any changes ```shell diff --git a/cluster-api/update_capi_versions.sh b/cluster-api/update_capi_versions.sh new file mode 100755 index 00000000..944bf557 --- /dev/null +++ b/cluster-api/update_capi_versions.sh @@ -0,0 +1,22 @@ +#!/bin/bash + +set -euo pipefail + +printf "%-20s %-20s %-25s\n" "kubernetes_series" "kubernetes_semver" "kubernetes_deb_version" +printf "%-20s %-20s %-25s\n" "-------------------" "-------------------" "------------------------" + +for v in $(find versions -iname "v*.json" -printf "%f\n" | sed -e "s/.json//g" -e "s/_/./g"); do + deb_version=$( + curl -fsSL "https://pkgs.k8s.io/core:/stable:/$v/deb/Packages" | + awk ' + /^Package: kubeadm$/ {found=1} + found && /^Version:/ {print $2; found=0} + ' | + sort -V | + tail -1 + ) + + semver=$(echo "$deb_version" | sed -E 's/^([0-9]+\.[0-9]+\.[0-9]+).*/\1/') + + printf "%-20s %-20s %-25s\n" "$v" "v$semver" "$deb_version" +done \ No newline at end of file diff --git a/cluster-api/versions/v1_31.json b/cluster-api/versions/v1_31.json deleted file mode 100644 index df64c4a4..00000000 --- a/cluster-api/versions/v1_31.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "kubernetes_series": "v1.31", - "kubernetes_semver": "v1.31.14", - "kubernetes_deb_version": "1.31.14-1.1" -} diff --git a/cluster-api/versions/v1_32.json b/cluster-api/versions/v1_32.json deleted file mode 100644 index 7066680c..00000000 --- a/cluster-api/versions/v1_32.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "kubernetes_series": "v1.32", - "kubernetes_semver": "v1.32.13", - "kubernetes_deb_version": "1.32.13-1.1" -} diff --git a/cluster-api/versions/v1_33.json b/cluster-api/versions/v1_33.json index 5e13a949..c6bc9b45 100644 --- a/cluster-api/versions/v1_33.json +++ b/cluster-api/versions/v1_33.json @@ -1,5 +1,5 @@ { "kubernetes_series": "v1.33", - "kubernetes_semver": "v1.33.12", - "kubernetes_deb_version": "1.33.12-1.1" + "kubernetes_semver": "v1.33.13", + "kubernetes_deb_version": "1.33.13-1.1" } diff --git a/cluster-api/versions/v1_34.json b/cluster-api/versions/v1_34.json index 644dcf3a..220ee05d 100644 --- a/cluster-api/versions/v1_34.json +++ b/cluster-api/versions/v1_34.json @@ -1,5 +1,5 @@ { "kubernetes_series": "v1.34", - "kubernetes_semver": "v1.34.8", - "kubernetes_deb_version": "1.34.8-1.1" + "kubernetes_semver": "v1.34.11", + "kubernetes_deb_version": "1.34.11-1.1" } diff --git a/cluster-api/versions/v1_35.json b/cluster-api/versions/v1_35.json new file mode 100644 index 00000000..dbd2ca46 --- /dev/null +++ b/cluster-api/versions/v1_35.json @@ -0,0 +1,5 @@ +{ + "kubernetes_series": "v1.35", + "kubernetes_semver": "v1.35.8", + "kubernetes_deb_version": "1.35.8-1.1" +} diff --git a/cluster-api/versions/v1_36.json b/cluster-api/versions/v1_36.json new file mode 100644 index 00000000..f0cf560c --- /dev/null +++ b/cluster-api/versions/v1_36.json @@ -0,0 +1,5 @@ +{ + "kubernetes_series": "v1.36", + "kubernetes_semver": "v1.36.4", + "kubernetes_deb_version": "1.36.4-1.1" +} diff --git a/k8s-image-builder b/k8s-image-builder index fd68cd55..4825f241 160000 --- a/k8s-image-builder +++ b/k8s-image-builder @@ -1 +1 @@ -Subproject commit fd68cd55491928f70643efb671a5076c874b5c06 +Subproject commit 4825f24107f9ab2f5788c96abab93b97b3cfa933