diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 7afaaebb5c..d1e31cd255 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -32,7 +32,9 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Runners and toolchain -- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. +- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image, except Release Build. actionlint learns the label from `.github/actionlint.yaml`. +- Release Build runs on `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, Xcode 27.1 at `/Applications/Xcode.app`), run by launchd as the maintainer's user. +- No other workflow names `tablepro-release`. Release Build never takes a PR, and fork PR workflows need approval for every outside contributor. - The Xcode version and Apple's released build of it are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. - A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). - A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs the pinned build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. @@ -63,8 +65,10 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Release Build (`release-build.yml`) -- Push to main or `release/**`, and dispatch; never a PR. Never cancelled. Not a required check. -- `Changes (Release)` outputs `run` (detector `release`). `Release Build` runs on `xcode-27` with a 150-minute limit, with VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. +- Nightly at 19:00 UTC on main, push to `release/**`, and dispatch; never a PR. Never cancelled. Not a required check. +- `Changes (Release)` outputs `run`. A dispatch always builds. A nightly runs the `release` detector against the head of the last nightly that ended green, read with `gh api`, so a quiet day builds nothing. A `release/**` push diffs `github.event.before`, and builds when the push created the branch. +- `Release Build` runs on the self-hosted Mac; a dispatch with `runner: github` runs it on `xcode-27`. It has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. +- While the Mac is offline a run waits for it, and GitHub cancels a job still queued after 24 hours. The tag gate refuses a commit with any red run, so re-run that run once the Mac is back; a `runner: github` dispatch on the same commit does not replace it. - Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both` and `build-hana.sh both`. - `scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. - `scripts/release/package.sh app --unsigned` writes the release assets: the DMG, the `.tar.xz` (extracted and compared with the app) and the dSYMs zip; `package.sh plugins` writes one zip per plugin and the plugin dSYMs zip. @@ -129,7 +133,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## What runs where - A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title, plus Release Build; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or changelog-only push skips the suites through the detector. +- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or changelog-only push skips the suites through the detector. A `release/**` push also runs Release Build; main gets it nightly. - A `v*` tag runs only `release.yml`, and a `linux-v*` tag only `release-linux.yml`, which calls `linux-packages.yml`. The test workflows never run on a tag. - `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. - `build-linux.yml` builds and tests the Linux app (`linux/docs/testing.md`, CI). Its `Changes (Linux)` job skips the rest when nothing under `linux/` or the scripts and workflows it reads changed, and its `Linux` job gates on all of them. @@ -140,7 +144,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other Apple cache: no libs or DerivedData cache, and none in the iOS or plugin workflows. The Linux jobs cache Cargo builds through `Swatinem/rust-cache` and the Flatpak build. -- CI fits the GitHub Free and Pro plans: no paid larger runner, no self-hosted Mac. At most 5 macOS jobs run at once, and main runs queue behind PR runs for those slots. +- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. Release Build takes none of them unless a dispatch picks `runner: github`. ## Writing workflow and lint code diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index 889c6b5f5a..100cead085 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -28,7 +28,8 @@ Before any decision, gather: - open regressions: `gh issue list --label regression --state open`; - the newest beta's age, `gh release view v.0-beta. --json publishedAt -q .publishedAt`, and the code on the line after it, `git log --oneline v.0-beta...origin/release/ -- . ':!changelog.d' ':!CHANGELOG.md'`; - open Dependabot alerts: `gh api --paginate 'repos/TableProApp/TablePro/dependabot/alerts?state=open&per_page=100' -q '.[].security_advisory.summary'`; -- main's push runs: `gh run list --branch main --event push --limit 10`, with macOS Tests, iOS and Package Tests and Release Build green on the commit you would cut; +- main's push runs: `gh run list --branch main --event push --limit 10`, with macOS Tests and iOS and Package Tests green on the commit you would cut; +- main's Release Builds: `gh run list --workflow release-build.yml --branch main --status success --limit 5`. The commit you would cut needs one, nightly or dispatched (`gh workflow run release-build.yml --ref main`); - whether `main` has the last stable's `## [X.Y.Z]` heading. If not, the forward-port is missing and the next tag will refuse to build: do that first (section 9). ## 3. Decide what ships diff --git a/.github/workflows/release-build.yml b/.github/workflows/release-build.yml index 5305f51ea0..c9b5cf0450 100644 --- a/.github/workflows/release-build.yml +++ b/.github/workflows/release-build.yml @@ -1,12 +1,21 @@ name: Release Build # The Release configuration of the app and every registry plugin, universal and unsigned, packaged -# the way a tag packages it, so a packaging failure shows on the commit instead of on the tag. +# the way a tag packages it, so a packaging failure shows before the tag: nightly on main, on every +# release branch push, and on demand. on: push: - branches: [main, 'release/**'] + branches: ['release/**'] + schedule: + - cron: '0 19 * * *' workflow_dispatch: + inputs: + runner: + description: Where the build runs. github is the fallback while the self-hosted Mac is offline. + type: choice + options: [self-hosted, github] + default: self-hosted permissions: contents: read @@ -16,6 +25,9 @@ jobs: name: Changes (Release) runs-on: ubuntu-latest timeout-minutes: 5 + permissions: + actions: read + contents: read outputs: run: ${{ steps.decide.outputs.run }} steps: @@ -23,13 +35,35 @@ jobs: with: fetch-depth: 0 + # A nightly diffs against the last nightly that ended green, built or skipped: main up to + # that commit is covered. A push that creates a release branch builds the cut commit, which + # the merge-base rule would skip. - name: Decide whether the Release build runs id: decide env: - BASE: ${{ github.event.before }} + EVENT: ${{ github.event_name }} + CREATED: ${{ github.event.created }} + BEFORE: ${{ github.event.before }} + GH_TOKEN: ${{ github.token }} run: | set -euo pipefail - run=$(scripts/ci/detect-changed-paths.sh --base "$BASE" release) + case "$EVENT" in + workflow_dispatch) + run=true + ;; + schedule) + base=$(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/release-build.yml/runs?event=schedule&status=success&per_page=1" \ + --jq '.workflow_runs[0].head_sha // ""') + run=$(scripts/ci/detect-changed-paths.sh --base "$base" release) + ;; + *) + if [ "$CREATED" = true ]; then + run=true + else + run=$(scripts/ci/detect-changed-paths.sh --base "$BEFORE" release) + fi + ;; + esac echo "run=$run" >> "$GITHUB_OUTPUT" # Reads "not false" so a failed changes job builds instead of passing. @@ -37,7 +71,7 @@ jobs: name: Release Build needs: changes if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} - runs-on: xcode-27 + runs-on: ${{ inputs.runner == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} timeout-minutes: 150 # The tag's version settings run here first; 9999999 is the largest build at MAJOR 0 (RELEASING.md, Versions). env: diff --git a/RELEASING.md b/RELEASING.md index 9bf88d57ff..aef9ca877b 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -48,7 +48,7 @@ Tags go out between 13:00 and 15:00 ICT (UTC+7). ## Cutting a beta -beta.1 cuts the line at the newest `main` commit whose push runs are all green; beta.N tags the line's head once the picks are on it. +beta.1 cuts the line at the newest `main` commit a green nightly or dispatched Release Build ran on, once its push runs are green too. The push that creates the branch builds it again, and the tag's `verify` waits for that build. beta.N tags the line's head once the picks are on it. ```bash git fetch origin @@ -96,7 +96,7 @@ python3 scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro-docs/ A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. -- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable whose tree differs from its line's newest published beta outside `CHANGELOG.md` and `changelog.d/`, until a stable tag on the line has matched that beta; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a stable whose tag still holds fragments in `changelog.d/`. Then it needs macOS Tests, iOS and Package Tests and Release Build green on the tag commit or the nearest first-parent commit that ran them, with no change to the files they read after it, waiting on a pending run. Last, it writes the notes. +- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable whose tree differs from its line's newest published beta outside `CHANGELOG.md` and `changelog.d/`, until a stable tag on the line has matched that beta; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a stable whose tag still holds fragments in `changelog.d/`. Then it needs macOS Tests, iOS and Package Tests and Release Build green on the tag commit or the nearest first-parent commit that ran them, with no change to the files they read after it, waiting on a pending run. A push, nightly or dispatched run counts; a PR run builds a merge commit, so it does not. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. - `plugins` (`signing`): the Dameng and HANA bridges, `build.sh plugins` (each target `registry.py targets` prints, universal, signed with Developer ID, the hardened runtime and a timestamp), one notary submission for all, `package.sh plugins` (the zips, described as the manifest will read them). - `github-release`: `actions/attest` attests `dist/*`; `gh release create` drafts, uploads and publishes; `gh release verify` and `verify-asset` check the published release. diff --git a/scripts/release/test_version.py b/scripts/release/test_version.py index ef0e8e8248..11f4a2c7fb 100644 --- a/scripts/release/test_version.py +++ b/scripts/release/test_version.py @@ -81,7 +81,7 @@ def __init__(self, releases=(), runs=None, jobs=None): def releases(self): return self._releases - def push_runs(self, workflow, sha): + def commit_runs(self, workflow, sha): self.run_queries.append((workflow, sha)) snapshots = self._runs.get((workflow, sha), [[]]) return snapshots.pop(0) if len(snapshots) > 1 else snapshots[0] @@ -478,7 +478,8 @@ def run(command, **_): path = command[command.index("--jq") - 1] outputs = { "/releases": "v0.82.0\tfalse\nplugin-oracle-v1.0.0\ttrue\n", - "/runs?": "11\tcompleted\tsuccess\n12\tin_progress\t\n", + "/runs?": ("11\tcompleted\tsuccess\tpush\n12\tin_progress\t\tschedule\n" + "13\tcompleted\tsuccess\tworkflow_dispatch\n14\tcompleted\tfailure\tpull_request\n"), "/jobs?": "success\n\n", } output = next(text for key, text in outputs.items() if key in path) @@ -486,11 +487,11 @@ def run(command, **_): github = version.GitHub("Owner/Repo", run=run) self.assertEqual(github.releases(), [("v0.82.0", False), ("plugin-oracle-v1.0.0", True)]) - self.assertEqual(github.push_runs("macos-tests.yml", "abc"), - [("11", "completed", "success"), ("12", "in_progress", "")]) + self.assertEqual(github.commit_runs("macos-tests.yml", "abc"), + [("11", "completed", "success"), ("12", "in_progress", ""), ("13", "completed", "success")]) self.assertEqual(github.job_conclusions("11", "macOS Tests"), ["success", ""]) self.assertIn("--paginate", calls[0]) - self.assertIn("repos/Owner/Repo/actions/workflows/macos-tests.yml/runs?head_sha=abc&event=push&per_page=100", + self.assertIn("repos/Owner/Repo/actions/workflows/macos-tests.yml/runs?head_sha=abc&per_page=100", calls[1]) def test_a_gh_failure_is_raised(self): diff --git a/scripts/release/version.py b/scripts/release/version.py index f36e467966..fabb3ba95e 100755 --- a/scripts/release/version.py +++ b/scripts/release/version.py @@ -27,6 +27,8 @@ DETECTOR = "scripts/ci/detect-changed-paths.sh" FRAGMENTS = "changelog.d/" POLL_SECONDS = 60 +# A pull_request run builds a merge commit, not its head_sha; these events build the commit itself. +COMMIT_EVENTS = ("push", "schedule", "workflow_dispatch") PENDING_STATUSES = {"queued", "in_progress", "waiting", "requested", "pending"} @@ -155,12 +157,13 @@ def releases(self) -> List[Tuple[str, bool]]: ".[] | [.tag_name, .draft] | @tsv", paginate=True) return [(tag, draft == "true") for tag, draft in (line.split("\t") for line in lines if line)] - def push_runs(self, workflow: str, sha: str) -> List[Tuple[str, str, str]]: + def commit_runs(self, workflow: str, sha: str) -> List[Tuple[str, str, str]]: lines = self._api(f"repos/{self.repository}/actions/workflows/{workflow}/runs" - f"?head_sha={sha}&event=push&per_page=100", - ".workflow_runs[] | [.id, .status, .conclusion // \"\"] | @tsv") + f"?head_sha={sha}&per_page=100", + ".workflow_runs[] | [.id, .status, .conclusion // \"\", .event] | @tsv") return [(run_id, status, conclusion) - for run_id, status, conclusion in (line.split("\t") for line in lines if line)] + for run_id, status, conclusion, event in (line.split("\t") for line in lines if line) + if event in COMMIT_EVENTS] def job_conclusions(self, run_id: str, job: str) -> List[str]: return self._api(f"repos/{self.repository}/actions/runs/{run_id}/jobs?filter=latest&per_page=100", @@ -284,7 +287,7 @@ def fragment_errors(paths: List[str], version: Version) -> List[str]: def is_green(github: GitHub, workflow: str, job: str, sha: str, sleep: Callable[[float], None], log: Callable[[str], None]) -> bool: while True: - runs = github.push_runs(workflow, sha) + runs = github.commit_runs(workflow, sha) pending = [run_id for run_id, status, _ in runs if status in PENDING_STATUSES] if not pending: break