Update k8s versions

cluster-api/README.md

@@ -2,46 +2,113 @@
 
 This project contains the current versions of Kubernetes that the Cloud Group will support and builds images for.
 
+
+## Contents:
+
+- [Setup](#setup)
+
+
+
+
+
 ## Setup
 
-1. Install Ansible and any other pip requirements:
+1. Clone this repository.
+   ```shell
+   git clone https://github.com/stfc/cloud-image-builders.git
+   cd cloud-image-builders/cluster-api
+   ```
+
+2. Install Python requirements.
    ```shell
    sudo apt-get install python3-venv unzip -y
    # or
    sudo dnf install python3-venv unzip -y
 
+   # Then
    python3 -m venv venv
    source venv/bin/activate
-   pip install -r os_builders/requirements.txt
+   pip install -r ../os_builders/requirements.txt
    ```
 
 2. Install Packer and dependencies:
    ```shell
-   cd os_builders
+   cd ../os_builders
    # If sudo is passwordless:
    ansible-playbook prep_builder.yml
    # If password is required for sudo:
    ansible-playbook prep_builder.yml --ask-become-pass
+   cd ../cluster-api
+   ```
+3. Set up OpenStack authentication. Get a clouds.yaml for the **packer** project. See [here](https://stfc.atlassian.net/wiki/spaces/CLOUDKB/pages/211484774/Application+Credentials)
+   ```shell
+   # Copy clouds.yaml to the config directory
+   mkdir -p ~/.config/openstack
+   cp <path-to-clouds.yaml> ~/.config/openstack/clouds.yaml
+
+   # Export the cloud name in clouds.yaml, default is openstack
+   export OS_CLOUD=openstack
    ```
 
-## Rate Limiting
+### Rate Limiting (Optional)
 
 You may run into GitHub rate limiting issues when building images. To avoid this, you can set the following environment variable:
 
 `PACKER_GITHUB_API_TOKEN=<token>`
 
 The token can be generated from your GitHub settings, under developer access, and only needs the `public_repo` scope (i.e. the default).
 
-## OpenStack authentication
+## Update the CAPI image versions
+
+1. Clone the repository with write access, usually by SSH
+   ```shell
+   git clone git@github.com:stfc/cloud-image-builders.git
+   cd cloud-image-builders/cluster-api
+   ```
 
-You need to set up credentials for OpenStack authentication as we are using a remote builder. Create a clouds.yaml application credential and place it into `~/.config/openstack/clouds.yaml`. See [here](https://stfc.atlassian.net/wiki/spaces/CLOUDKB/pages/211484774/Application+Credentials) for help.
+2. Create a new branch called update_k8s_versions
+   ```shell
+   git switch -c update_k8s_versions
+   ```
 
-Addtionally, you will need to specify the cloud name in your session by exporting it. By default this will be `openstack` as seen in the standard `clouds.yaml` file:
+3. Update the k8s submodule
    ```shell
-   export OS_CLOUD=<cloud-name>
+   git submodule update --init --recursive --remote
    ```
 
-## Build a specific version
+4. Perform the following checks against the releases page [here](https://kubernetes.io/releases/).
+   1. If there is a new version, add a new file in [versions](./versions). Use the other versions as a template. We don't release the latest version.
+      ```shell
+      cd versions
+      cp v1_35.json v1_36.json
+
+      # Update to the version number
+      vim v1_36.json
+      ```
+   2. If [versions](./versions) contains more than 3 version files, delete the oldest so we have only 3.
+
+5. Run update_capi_versions.sh and update each version file with the latest version.
+   ```shell
+   ./update_capi_versions.sh
+   ...
+   (venv) venv ❯ ./update_capi_versions.sh
+   kubernetes_series    kubernetes_semver    kubernetes_deb_version   
+   -------------------  -------------------  ------------------------ 
+   v1.32                v1.32.13             1.32.13-1.1              
+   v1.33                v1.33.13             1.33.13-1.1              
+   v1.34                v1.34.11             1.34.11-1.1      
+   ...
+
+6. Commit these changes to the branch and make a pull request
+   ```shell
+   git add versions/ ../k8s-image-builder
+   git commit 
+   ...
+   Commit message
+   ...
+   git push --set-upstream origin update_k8s_versions
+
+## Build a CAPI image
 
 1. Grab the latest version of the K8s Image Builder submodule:
    ```shell
@@ -69,41 +136,6 @@ Addtionally, you will need to specify the cloud name in your session by exportin
    ```
 4. Follow steps for release [here](#update-an-image-for-release)
 
-## Adding a new version
-1. Update the image builder:
-   ```shell
-   cd ..  # back to repo root
-   git submodule update --init --recursive --remote
-   ```
-2. Add new version file:
-    1. Navigate to https://kubernetes.io/releases/
-    2. Find the version you want to add or update
-    3. Create/Update the semver in the relevant JSON file. There should be a 1:1 mapping of JSON files to major versions of Kubernetes. E.g. a file for 1.24, 1.25, etc.
-3. Follow steps to build a specific version [here](#build-a-specific-version)
-4. Follow steps for release [here](#update-an-image-for-release)
-
-## Rebuild all images
-1. Update the image builder:
-   ```shell
-   cd ..  # back to repo root
-   git submodule update --init --recursive --remote
-   ```
-2. Update patch versions in version files
-   ```
-   # Contents of ./versions/v1_33.json
-   {
-       "kubernetes_series": "v1.33",
-       "kubernetes_semver": "v1.33.3", -> "v1.33.4"
-       "kubernetes_deb_version": "1.33.3-1.1" -> "v1.33.4-1.1"
-   }
-   ```
-3. Run build-all.sh
-   ```shell
-   cd scripts
-   ./build-all.sh <env>  # dev or prod
-   ```
-4. Follow steps for release for each image [Update and image for release](#update-an-image-for-release)
-
 ## Update an image for release
 1. Check the image you are working with before you make any changes
    ```shell

cluster-api/update_capi_versions.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -euo pipefail
+
+printf "%-20s %-20s %-25s\n" "kubernetes_series" "kubernetes_semver" "kubernetes_deb_version"
+printf "%-20s %-20s %-25s\n" "-------------------" "-------------------" "------------------------"
+
+for v in $(find versions -iname "v*.json" -printf "%f\n" | sed -e "s/.json//g" -e "s/_/./g"); do
+ deb_version=$(
+    curl -fsSL "https://pkgs.k8s.io/core:/stable:/$v/deb/Packages" |
+    awk '
+      /^Package: kubeadm$/ {found=1}
+      found && /^Version:/ {print $2; found=0}
+    ' |
+    sort -V |
+    tail -1
+  )
+
+ semver=$(echo "$deb_version" | sed -E 's/^([0-9]+\.[0-9]+\.[0-9]+).*/\1/')
+
+ printf "%-20s %-20s %-25s\n" "$v" "v$semver" "$deb_version"
+done

cluster-api/versions/v1_33.json

@@ -1,5 +1,5 @@
 {
     "kubernetes_series": "v1.33",
-    "kubernetes_semver": "v1.33.12",
-    "kubernetes_deb_version": "1.33.12-1.1"
+    "kubernetes_semver": "v1.33.13",
+    "kubernetes_deb_version": "1.33.13-1.1"
 }

cluster-api/versions/v1_34.json

@@ -1,5 +1,5 @@
 {
     "kubernetes_series": "v1.34",
-    "kubernetes_semver": "v1.34.8",
-    "kubernetes_deb_version": "1.34.8-1.1"
+    "kubernetes_semver": "v1.34.11",
+    "kubernetes_deb_version": "1.34.11-1.1"
 }

cluster-api/versions/v1_35.json

@@ -0,0 +1,5 @@
+{
+    "kubernetes_series": "v1.35",
+    "kubernetes_semver": "v1.35.8",
+    "kubernetes_deb_version": "1.35.8-1.1"
+}

cluster-api/versions/v1_36.json

@@ -0,0 +1,5 @@
+{
+    "kubernetes_series": "v1.36",
+    "kubernetes_semver": "v1.36.4",
+    "kubernetes_deb_version": "1.36.4-1.1"
+}