Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,9 @@ jobs:
# it — the break lands on main and surfaces in someone else's PR.
tb-cli:
- 'docker/tb-cli/**'
tinybird-slim:
- 'docker/tinybird-local-slim/**'
- 'compose.dev.analytics.yaml'

- name: Define Node test matrix
# Node lines the unit / legacy / acceptance suites must keep passing on.
Expand Down Expand Up @@ -310,6 +313,7 @@ jobs:
changed_docs: ${{ steps.changed.outputs.docs }}
changed_package_standards: ${{ steps.changed.outputs.package-standards }}
changed_tb_cli: ${{ steps.changed.outputs.tb-cli }}
changed_tinybird_slim: ${{ steps.changed.outputs.tinybird-slim }}
# Single gate for the build + browser-E2E lane. True for tags, or when a
# changed file could affect a running Ghost instance (see the `e2e` path
# filter above). A test-only / docs-only change keeps this false.
Expand Down Expand Up @@ -2013,10 +2017,12 @@ jobs:
#
# Its GHCR package is internal (the upstream licence only permits
# distribution within our own organization), so it is unreadable from a
# cross-repo PR's scoped token — those runs stay on the upstream image.
# cross-repo PR's scoped token — those runs stay on the upstream image
# unless they change the slim image and build it locally below.
# infra-up.sh falls back on a failed pull regardless, so an unexpected
# permission gap degrades rather than breaks.
GHOST_E2E_TINYBIRD_SLIM: ${{ github.repository_owner == 'TryGhost' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) }}
GHOST_E2E_TINYBIRD_SLIM: ${{ needs.job_setup.outputs.changed_tinybird_slim == 'true' || (github.repository_owner == 'TryGhost' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)) }}
GHOST_E2E_TINYBIRD_SLIM_IMAGE: ${{ needs.job_setup.outputs.changed_tinybird_slim == 'true' && 'tinybird-local-slim:e2e' || 'ghcr.io/tryghost/tinybird-local-slim:latest' }}
strategy:
fail-fast: true
matrix:
Expand Down Expand Up @@ -2090,7 +2096,7 @@ jobs:

- name: Log in to GitHub Container Registry
# Needed to read the internal tinybird-local-slim package.
if: matrix.analytics == 'true' && env.GHOST_E2E_TINYBIRD_SLIM == 'true'
if: matrix.analytics == 'true' && env.GHOST_E2E_TINYBIRD_SLIM == 'true' && needs.job_setup.outputs.changed_tinybird_slim != 'true'
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
continue-on-error: true
with:
Expand Down Expand Up @@ -2126,6 +2132,20 @@ jobs:
docker build --tag "$COMPOSE_IMAGE" --file docker/tb-cli/Dockerfile .
docker builder prune --all --force

- name: Build changed Tinybird slim image
if: matrix.analytics == 'true' && needs.job_setup.outputs.changed_tinybird_slim == 'true'
# Build before loading the Ghost image or installing dependencies: the
# upstream layers need several GB that can be reclaimed after flattening.
# This image stays on the runner; only the main publish workflow pushes it.
run: |
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /usr/local/.ghcup
upstream_ref=$(grep -oE 'tinybirdco/tinybird-local:[^ ]+' compose.dev.analytics.yaml)
docker build --platform linux/amd64 \
--build-arg "TINYBIRD_LOCAL_REF=$upstream_ref" \
--file docker/tinybird-local-slim/Dockerfile \
--tag "$GHOST_E2E_TINYBIRD_SLIM_IMAGE" .
docker builder prune --all --force

- name: Load Image
uses: ./.github/actions/load-docker-image
id: load
Expand Down
2 changes: 1 addition & 1 deletion docker/stripe/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ fi

# Start stripe listen in the background
echo "Starting Stripe webhook listener forwarding to ${GHOST_URL}/members/webhooks/stripe/"
stripe listen --forward-to ${GHOST_URL}/members/webhooks/stripe/ --api-key "${STRIPE_SECRET_KEY}" &
stripe listen --all-snapshot --forward-to "${GHOST_URL}/members/webhooks/stripe/" --api-key "${STRIPE_SECRET_KEY}" &
child=$!

# Wait for the child process
Expand Down
4 changes: 2 additions & 2 deletions docker/tinybird-local-slim/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
CLICKHOUSE_CLUSTER_INTERNAL_HTTP_PORT=8123 \
CLICKHOUSE_INTERNAL_PORT=8123 \
USE_GATHERER=False \
MINIO_ROOT_USER=admin \
MINIO_ROOT_PASSWORD=password
OBJECT_STORAGE_ROOT_USER=admin \
OBJECT_STORAGE_ROOT_PASSWORD=password
WORKDIR /app
EXPOSE 7181 7182
# Copied verbatim from upstream, notably the licence pointers — the flatten drops
Expand Down
10 changes: 8 additions & 2 deletions docker/tinybird-local-slim/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ A distilled build of `tinybirdco/tinybird-local` for CI. Published to
[`publish-tinybird-local-slim.yml`](../../.github/workflows/publish-tinybird-local-slim.yml)
and used by the analytics E2E jobs via `GHOST_E2E_TINYBIRD_SLIM=true`.

When `docker/tinybird-local-slim/**` or `compose.dev.analytics.yaml` changes,
analytics E2E jobs build the slim image from the checked-out commit and test
that image instead of the published `latest`. This also applies to fork PRs;
the image stays on the runner and is not published. Other fork PRs use upstream.

## Why

The upstream image is ~2.1GB to pull and ~6.9GB once unpacked, which does not
Expand Down Expand Up @@ -64,8 +69,9 @@ GHOST_E2E_TINYBIRD_SLIM=true GHOST_E2E_TINYBIRD_SLIM_IMAGE=tinybird-local-slim:l
need it access through package settings rather than making it public.

An internal package is unreadable from a PR opened from a public fork, whose
token is scoped to the fork. CI leaves `GHOST_E2E_TINYBIRD_SLIM` off for those
runs so they use upstream directly, and `e2e/scripts/infra-up.sh` falls back to
token is scoped to the fork. Unless the run builds the image locally as described
above, CI leaves `GHOST_E2E_TINYBIRD_SLIM` off for those runs so they use upstream
directly. `e2e/scripts/infra-up.sh` also falls back to
upstream on any failed pull regardless — so a missing access grant, or the
window before the package is first published, degrades to a slower, fatter run
rather than a broken one.
Expand Down
Loading