Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/publisher-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,53 @@ jobs:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
# The version check compares against the newest release tag.
fetch-depth: 0
- uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
with:
version: v3.17.3
- run: k8s/dolos-publisher/test.sh
- run: k8s/registry/test.sh
- name: Package the charts
run: k8s/package-charts.sh "$RUNNER_TEMP/charts"
- name: Test the chart version check
run: k8s/check-chart-versions.test.sh
- name: Test the chart push against a local registry
run: k8s/push-charts.test.sh
- name: Check changed charts bumped their version
if: github.event_name == 'pull_request'
env:
BASE_REF: ${{ github.base_ref }}
run: k8s/check-chart-versions.sh "$(git describe --tags --abbrev=0 --match 'v*' "origin/$BASE_REF")"

charts:
name: Publish charts
if: startsWith(github.ref, 'refs/tags/v')
needs: [chart]
runs-on: ubuntu-22.04
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
# The version check compares against the previous release tag.
fetch-depth: 0
- uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
with:
version: v3.17.3
- name: Check charts changed since the previous release bumped their version
run: k8s/check-chart-versions.sh "$(k8s/previous-release-tag.sh "$GITHUB_REF_NAME")"
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Package the charts
run: k8s/package-charts.sh "$RUNNER_TEMP/charts"
- name: Push chart versions not yet published
run: k8s/push-charts.sh "$RUNNER_TEMP/charts"

publish:
name: Publish multi-architecture image
Expand Down
15 changes: 15 additions & 0 deletions docs/publisher-packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,21 @@ distinct, explicit decisions.
`gh attestation verify oci://ghcr.io/txpipe/stelae-publisher@sha256:<digest> --repo txpipe/stelae`.
Operations selects the tested image for a separate deployment.

5. The same tag publishes the charts under `oci://ghcr.io/txpipe/charts/`:
`dolos-publisher` and `stelae-registry`. Each is published at its own
`Chart.yaml` `version:`, independent of the workspace version, and only
once: a version the registry already holds is skipped, never replaced. A
pull request that changes a chart since the newest release tag must bump
that chart's version, or the chart check fails. The release repeats that
check against the previous release tag before it pushes, so a change that
reached the branch without a bump fails the release instead of going
unpublished. The previous release is the highest `v*` tag below the one
being released, even when it was tagged on a commit the branch does not
reach, as a squash-merged release PR leaves it.
6. The first push of a chart creates a private package. The organization
owner makes it public by hand in the package settings; later versions
inherit that visibility.

Pull requests and branch-based manual runs only build and test. A manual run
on a version tag follows the same publication path as a tag push. The workflow
does not create GitHub Releases, tags, or deployments.
Expand Down
30 changes: 30 additions & 0 deletions k8s/check-chart-versions.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/sh
# Fail for each chart whose files differ from <base-ref> while its Chart.yaml
# version does not. Run inside the repository; a chart absent at <base-ref>
# passes.
set -eu

if [ $# -ne 1 ]; then
echo "usage: $0 <base-ref>" >&2
exit 2
fi
BASE=$1
cd "$(git rev-parse --show-toplevel)"

failed=0
for chart in k8s/dolos-publisher/chart k8s/registry/chart; do
if git diff --quiet "$BASE" -- "$chart"; then
continue
fi
if ! base_yaml=$(git show "$BASE:$chart/Chart.yaml" 2>/dev/null); then
continue
fi
base_version=$(printf '%s\n' "$base_yaml" | sed -n 's/^version: *//p')
version=$(sed -n 's/^version: *//p' "$chart/Chart.yaml")
name=$(sed -n 's/^name: *//p' "$chart/Chart.yaml")
if [ "$version" = "$base_version" ]; then
echo "$name: $chart changed since $BASE but its version is still $version" >&2
failed=1
fi
done
exit "$failed"
98 changes: 98 additions & 0 deletions k8s/check-chart-versions.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
#!/bin/sh
# check-chart-versions.sh in a throwaway repository: a changed chart with an
# unchanged version fails, a bumped version passes, an untouched chart passes,
# and at release time previous-release-tag.sh finds the tag to check against,
# even when that release was tagged off the branch.
set -eu

HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
CHECK=$HERE/check-chart-versions.sh
PREVIOUS=$HERE/previous-release-tag.sh
REPO=$(mktemp -d)
trap 'rm -rf "$REPO"' EXIT
cd "$REPO"

git init -q
git config user.email test@example.invalid
git config user.name test
for chart in dolos-publisher registry; do
mkdir -p "k8s/$chart/chart/templates"
printf 'apiVersion: v2\nname: %s\nversion: 0.1.0\n' "$chart" >"k8s/$chart/chart/Chart.yaml"
echo 'kind: ConfigMap' >"k8s/$chart/chart/templates/configmap.yaml"
done
git add k8s
git commit -q -m base
git tag v0.1.0

# Untouched charts pass.
"$CHECK" v0.1.0

# A changed template with the same version fails and names its chart.
echo 'data: {}' >>k8s/dolos-publisher/chart/templates/configmap.yaml
git commit -q -am "change a template"
if "$CHECK" v0.1.0 2>"$REPO/err"; then
echo "a changed chart with an unchanged version passed" >&2
exit 1
fi
grep -F 'dolos-publisher:' "$REPO/err"
if grep -F 'stelae-registry' "$REPO/err" || grep -F 'registry:' "$REPO/err"; then
echo "the untouched chart was reported" >&2
exit 1
fi

# A bumped version passes; the other chart stays untouched.
sed -i.bak 's/^version: 0.1.0$/version: 0.1.1/' k8s/dolos-publisher/chart/Chart.yaml
rm k8s/dolos-publisher/chart/Chart.yaml.bak
git commit -q -am "bump the chart"
"$CHECK" v0.1.0

# At release time, a change that landed without a bump after the previous
# release fails against the tag the release job resolves.
git tag v0.1.1
echo 'kind: Secret' >k8s/registry/chart/templates/secret.yaml
git add k8s/registry/chart/templates/secret.yaml
git commit -q -m "add a template without a bump"
git tag v0.1.2
previous=$("$PREVIOUS" v0.1.2)
test "$previous" = v0.1.1
if "$PREVIOUS" v0.1.0 2>/dev/null; then
echo "the first release resolved a previous release" >&2
exit 1
fi
if "$CHECK" "$previous" 2>"$REPO/err"; then
echo "an unbumped change since the previous release passed" >&2
exit 1
fi
grep -F 'registry:' "$REPO/err"

# A release tagged on a side commit, then squashed onto the branch as an
# identical commit, is still the previous release of the next tag.
main=$(git symbolic-ref --short HEAD)
git checkout -q -b release
sed -i.bak 's/^version: 0.1.1$/version: 0.1.2/' k8s/dolos-publisher/chart/Chart.yaml
rm k8s/dolos-publisher/chart/Chart.yaml.bak
echo 'data: {}' >>k8s/dolos-publisher/chart/templates/configmap.yaml
git commit -q -am "release: v0.1.3"
git tag v0.1.3
git checkout -q "$main"
git checkout v0.1.3 -- k8s
git commit -q -m "release: v0.1.3 (squash)"
test -z "$(git diff v0.1.3 HEAD)"
if git merge-base --is-ancestor v0.1.3 HEAD; then
echo "the side release is reachable from the branch" >&2
exit 1
fi
echo 'data: {}' >>k8s/dolos-publisher/chart/templates/configmap.yaml
git commit -q -am "change the chart without a bump"
git tag v0.1.4
# Ancestry alone would resolve v0.1.2, where the version differs, and pass.
test "$(git describe --tags --abbrev=0 --match 'v*' HEAD^)" = v0.1.2
previous=$("$PREVIOUS" v0.1.4)
test "$previous" = v0.1.3
if "$CHECK" "$previous" 2>"$REPO/err"; then
echo "an unbumped change since a release tagged off the branch passed" >&2
exit 1
fi
grep -F 'dolos-publisher:' "$REPO/err"

echo "check-chart-versions: ok"
9 changes: 9 additions & 0 deletions k8s/dolos-publisher/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ The chart runs a publisher built on the Dolos v2 configuration: `dolosToml`
is a v2 file keeping full history. There is no host selector. The directory and chart name stay
`dolos-publisher` to preserve release and object naming.

## Getting the chart

A release publishes the chart as an OCI package at its own `version:`. Pull
or install a pinned version without a checkout:

```sh
helm pull oci://ghcr.io/txpipe/charts/dolos-publisher --version <v>
```

## Rendered resources

- ConfigMap `<release>-config`, mounted at `/etc/publisher`, carries
Expand Down
21 changes: 21 additions & 0 deletions k8s/package-charts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/sh
# Package every Stelae chart into <out-dir> at its own Chart.yaml version and
# print "<name> <version>" for each.
set -eu

ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)

if [ $# -ne 1 ]; then
echo "usage: $0 <out-dir>" >&2
exit 2
fi
OUT=$1
mkdir -p "$OUT"

for chart in "$ROOT"/k8s/dolos-publisher/chart "$ROOT"/k8s/registry/chart; do
helm package "$chart" --destination "$OUT" >/dev/null
name=$(sed -n 's/^name: *//p' "$chart/Chart.yaml")
version=$(sed -n 's/^version: *//p' "$chart/Chart.yaml")
test -f "$OUT/$name-$version.tgz"
echo "$name $version"
done
22 changes: 22 additions & 0 deletions k8s/previous-release-tag.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/sh
# Print the highest v* tag ordered below <tag> by version, whether or not the
# branch reaches its commit: a release tagged off the branch still counts.
# Fail when <tag> is not a tag or no earlier release exists.
set -eu

if [ $# -ne 1 ]; then
echo "usage: $0 <tag>" >&2
exit 2
fi
TAG=$1

if ! git rev-parse -q --verify "refs/tags/$TAG" >/dev/null; then
echo "$TAG is not a tag" >&2
exit 1
fi
previous=$(git tag --list 'v*' --sort=-v:refname | awk -v tag="$TAG" 'seen { print; exit } $0 == tag { seen = 1 }')
if [ -z "$previous" ]; then
echo "no release tag precedes $TAG" >&2
exit 1
fi
echo "$previous"
47 changes: 47 additions & 0 deletions k8s/push-charts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/bin/sh
# Push every chart package in <out-dir> to the chart registry, skipping any
# version the registry already holds: a published version is never replaced.
# STELAE_CHART_REGISTRY overrides the registry root and STELAE_CHART_PLAIN_HTTP=1
# talks to it over plain HTTP; both exist for tests.
set -eu

REGISTRY=${STELAE_CHART_REGISTRY:-oci://ghcr.io/txpipe/charts}

if [ $# -ne 1 ]; then
echo "usage: $0 <out-dir>" >&2
exit 2
fi
OUT=$1

set --
if [ "${STELAE_CHART_PLAIN_HTTP:-}" = 1 ]; then
set -- --plain-http
fi

found=0
for package in "$OUT"/*.tgz; do
test -f "$package" || continue
found=1
name=$(helm show chart "$package" | sed -n 's/^name: *//p')
version=$(helm show chart "$package" | sed -n 's/^version: *//p')
if error=$(helm show chart "$@" "$REGISTRY/$name" --version "$version" 2>&1 >/dev/null); then
echo "$name $version: already published, skipped"
continue
fi
# Only a missing version is pushed; any other lookup failure stops here,
# since pushing then could replace a version the lookup failed to see.
case $error in
*"not found"* | *"NOT_FOUND"* | *"MANIFEST_UNKNOWN"* | *"NAME_UNKNOWN"*) ;;
*)
echo "$name $version: cannot tell whether it is published: $error" >&2
exit 1
;;
esac
helm push "$@" "$package" "$REGISTRY"
echo "$name $version: published"
done

if [ "$found" = 0 ]; then
echo "no chart packages in $OUT" >&2
exit 1
fi
56 changes: 56 additions & 0 deletions k8s/push-charts.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/bin/sh
# push-charts.sh against a local OCI registry: the first run publishes every
# chart, the second skips each one and exits 0.
set -eu

K8S=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
# zot from GHCR: ECR Public throttles anonymous pulls from shared runner IPs.
IMAGE=${STELAE_TEST_REGISTRY_IMAGE:-ghcr.io/project-zot/zot:v2.1.20}
OUT=$(mktemp -d)
container=
cleanup() {
if [ -n "$container" ]; then docker rm -f "$container" >/dev/null; fi
rm -rf "$OUT"
}
trap cleanup EXIT

# A failed pull is infrastructure, not a finding: back off and try again.
for attempt in 1 2 3 4 5; do
docker pull --quiet "$IMAGE" >/dev/null && break
if [ "$attempt" = 5 ]; then
echo "cannot pull $IMAGE" >&2
exit 1
fi
sleep $((attempt * 15))
done
container=$(docker run -d -p 127.0.0.1::5000 "$IMAGE")
port=$(docker port "$container" 5000/tcp | sed -n '1s/.*://p')
for attempt in $(seq 30); do
curl -fsS "http://127.0.0.1:$port/v2/" >/dev/null 2>&1 && break
if [ "$attempt" = 30 ]; then
echo "the local registry did not start" >&2
exit 1
fi
sleep 1
done

export STELAE_CHART_REGISTRY="oci://127.0.0.1:$port/charts"
export STELAE_CHART_PLAIN_HTTP=1

"$K8S/package-charts.sh" "$OUT/charts" >"$OUT/packaged"
count=$(grep -c '' "$OUT/packaged")

"$K8S/push-charts.sh" "$OUT/charts" >"$OUT/first"
cat "$OUT/first"
test "$(grep -c ': published$' "$OUT/first")" = "$count"

"$K8S/push-charts.sh" "$OUT/charts" >"$OUT/second"
cat "$OUT/second"
test "$(grep -c ': already published, skipped$' "$OUT/second")" = "$count"

while read -r name version; do
helm show chart --plain-http "$STELAE_CHART_REGISTRY/$name" --version "$version" |
grep -Fx "version: $version"
done <"$OUT/packaged"

echo "push-charts: ok"
9 changes: 9 additions & 0 deletions k8s/registry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,15 @@ Render and install are the operator's responsibility. `templates/NOTES.txt`
prints the smoke test: a `401` on `/v2/`, a sub-second blob `HEAD`, and a
`307` on a blob `GET`.

## Getting the chart

A release publishes the chart as an OCI package at its own `version:`. Pull
or install a pinned version without a checkout:

```sh
helm pull oci://ghcr.io/txpipe/charts/stelae-registry --version <v>
```

## The interim image

zot v2.1.17–v2.1.20 resolve a blob `HEAD`'s Content-Type by walking the
Expand Down
Loading