From f82e49e4c5aad4f32f5b8028a00ba45e3e3f9801 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Mon, 19 Oct 2026 20:01:42 +0700 Subject: [PATCH 1/5] ci(release): build, sign and publish on the self-hosted Mac --- .claude/rules/ci.md | 23 ++++---- .github/actions/apple-signing/action.yml | 51 ++++++++++++------ .../actions/apple-signing/cleanup/action.yml | 30 +++++++++++ .github/actions/prepare-build/action.yml | 17 ++++++ .github/workflows/release-build.yml | 21 +++++--- .github/workflows/release.yml | 26 +++++++-- apple/RELEASING.md | 7 +-- apple/scripts/release/build.sh | 3 +- apple/scripts/release/notarize.sh | 16 +++--- apple/scripts/release/package.sh | 10 +++- apple/scripts/release/test_notarize.py | 53 ++++++++++++++++--- 11 files changed, 201 insertions(+), 56 deletions(-) create mode 100644 .github/actions/apple-signing/cleanup/action.yml diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 7c1ac2d94f..b78bf8cdc0 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -33,9 +33,12 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Runners and toolchain -- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image, except Release Build. actionlint learns the label from `.github/actionlint.yaml`. -- Release Build runs on `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. -- No other workflow names `tablepro-release`. Release Build never takes a PR, and fork PR workflows need approval for every outside contributor. +- Every Xcode job in the test workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. +- The repository variable `MAC_RUNNER` picks the machine for Release Build and for `app`, `plugins` and `feed` in `release.yml`. Unset is `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. `github` sends them to hosted runners: `xcode-27`, and `macos-latest` for `feed`. Switching: `apple/RELEASING.md`, When a run fails. +- Only trusted events reach `tablepro-release`: `release.yml` runs on `v*` tags, Release Build on its schedules and dispatch. No `pull_request` or `pull_request_target` job names the label or reads `MAC_RUNNER`, and fork PR workflows need approval for every outside contributor. +- One runner takes one job at a time, so the Mac jobs need no concurrency group; `feed` keeps its `gh-pages` group (`apple/RELEASING.md`, What a tag runs). A second runner on the label would race on the user keychain search list. +- The Mac keeps its home folder and keychain search list between jobs. `.github/actions/apple-signing` saves the user search list, puts a keychain of its own in front, and writes the App Store Connect .p8 under `RUNNER_TEMP`, which `notarize.sh` passes to every `notarytool` call, so no notary profile goes into a keychain. `app` and `plugins` end with an `if: always()` step, `apple-signing/cleanup`, which restores the list and deletes the keychain, the .p8 and the provisioning profile; `app` also deletes `Configs/Secrets.xcconfig`. +- The Mac runs as the maintainer: the login keychain holds a Developer ID Application identity, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone, and the global git config signs commits, so `feed` pushes `gh-pages` with `GIT_CONFIG_GLOBAL=/dev/null`. - The Xcode version, the build every job uses and Apple's released build are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version and build, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. The version alone is no pin: the 27.1 beta, 27A9269, is also 27.1. - A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). - A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs Apple's released build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. @@ -66,10 +69,10 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Release Build (`release-build.yml`) -- Nightly at 19:00 UTC on main and dispatch; never a PR or a push. Never cancelled. Not a required check, and no tag waits for it. -- `Changes (Release)` outputs `run`. A dispatch always builds. A nightly runs the `release` detector against the head of the last nightly that ended green, read with `gh api`, so a quiet day builds nothing. -- `Release Build` runs on the self-hosted Mac, or on `xcode-27` when a dispatch picks `runner: github` or, for a nightly, the repository variable `RELEASE_BUILD_RUNNER` is `github`. It has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. -- While the Mac is offline a run waits for it, and GitHub cancels a job still queued after 24 hours. `gh variable set RELEASE_BUILD_RUNNER --body github` sends nightlies to `xcode-27` until `gh variable delete RELEASE_BUILD_RUNNER`. +- Nightly at 19:00 UTC on main, weekly at 07:00 UTC on Sunday, and dispatch; never a PR or a push. Never cancelled. Not a required check, and no tag waits for it. +- `Changes (Release)` outputs `run`. A dispatch and the weekly run always build. A nightly runs the `release` detector against the head of the last scheduled run that ended green, read with `gh api`, so a quiet day builds nothing. +- `Release Build` runs where `MAC_RUNNER` says (Runners and toolchain). A dispatch's `runner` input, `self-hosted` or `github`, overrides it, and the weekly run always takes `xcode-27`, so the hosted fallback stays tested. It has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. +- While the Mac is offline a run waits for it, and GitHub cancels a job still queued after 24 hours. - Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both` and `build-hana.sh both`. - `apple/scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `apple/scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `apple/build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. - `apple/scripts/release/package.sh app --unsigned` writes the release assets: the DMG, the `.tar.xz` (extracted and compared with the app) and the dSYMs zip; `package.sh plugins` writes one zip per plugin and the plugin dSYMs zip. @@ -133,7 +136,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## What runs where - A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full. On main a docs-only or changelog-only push skips the suites through the detector; a `release/**` push runs every suite (Change detector). Release Build runs nightly on main and by dispatch only. +- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full. On main a docs-only or changelog-only push skips the suites through the detector; a `release/**` push runs every suite (Change detector). Release Build runs nightly on main, weekly on `xcode-27` and by dispatch only. - A `v*` tag runs only `release.yml`, and a `linux-v*` tag only `release-linux.yml`, which calls `linux-packages.yml`. The test workflows never run on a tag. - `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. - `build-linux.yml` builds and tests the Linux app, and `linux-packages.yml` builds its packages (`linux/docs/testing.md`, CI). What starts each is in its `on:` and job `if:` lines. Flatpak Devel and Minimum Rust version never run on a PR or push, and `linux-packages.yml` never on a push. @@ -142,10 +145,10 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Caches and runner budget - Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. +- `release.yml` neither restores nor saves it. `app` and `plugins` pass `restore-cache: false` to `prepare-build`, which points `build.sh` at an empty folder under `RUNNER_TEMP` through `SPM_PACKAGES_DIR`, so a signed build reads no cache another run wrote and not the `~/.spm-cache` the Mac keeps. - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other Apple cache: no libs or DerivedData cache, and none in the iOS or plugin workflows. The Linux jobs cache Cargo builds through `Swatinem/rust-cache` and the Flatpak build. -- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. Release Build takes none of them unless a dispatch or `RELEASE_BUILD_RUNNER` picks `github`. +- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. The Mac jobs take none of them unless `MAC_RUNNER`, a dispatch or the weekly Release Build picks `github`. ## Writing workflow and lint code diff --git a/.github/actions/apple-signing/action.yml b/.github/actions/apple-signing/action.yml index 9a230d9909..92d9cf2189 100644 --- a/.github/actions/apple-signing/action.yml +++ b/.github/actions/apple-signing/action.yml @@ -1,5 +1,5 @@ name: Apple signing -description: Imports the Developer ID certificate, stores the notary credentials as the TablePro profile, and installs the provisioning profile. +description: Imports the Developer ID certificate into a keychain of its own, writes the notary key notarize.sh reads, and installs the provisioning profile. apple-signing/cleanup removes all of it. inputs: certificate-p12: @@ -22,6 +22,8 @@ inputs: description: Contents of that key's .p8 file. required: true +# Everything but the provisioning profile goes under RUNNER_TEMP/apple-signing. The self-hosted Mac keeps +# its home folder and keychain search list between jobs, so apple-signing/cleanup removes it all. runs: using: composite steps: @@ -31,21 +33,36 @@ runs: CERTIFICATES_P12: ${{ inputs.certificate-p12 }} CERTIFICATES_PASSWORD: ${{ inputs.certificate-password }} # A keychain of its own, unlocked with an empty password and given a six hour lock timeout, - # so a long notarization wait cannot relock it mid-run. The runner is thrown away afterwards. + # so a long notarization wait cannot relock it mid-run. run: | set -euo pipefail - KEYCHAIN_PATH="$RUNNER_TEMP/app-signing.keychain-db" + dir="$RUNNER_TEMP/apple-signing" + KEYCHAIN_PATH="$dir/signing.keychain-db" + mkdir -p "$dir" + chmod 700 "$dir" + + # A runner killed mid-job leaves its deleted keychain on the list, so only existing files are kept. + security list-keychains -d user | sed -e 's/^[[:space:]]*"//' -e 's/"$//' > "$dir/listed" + saved=() + while IFS= read -r keychain; do + if [ -e "$keychain" ]; then + saved+=("$keychain") + fi + done < "$dir/listed" + printf '%s\n' ${saved[@]+"${saved[@]}"} > "$dir/search-list" + security create-keychain -p "" "$KEYCHAIN_PATH" security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH" security unlock-keychain -p "" "$KEYCHAIN_PATH" - echo "$CERTIFICATES_P12" | base64 --decode > "$RUNNER_TEMP/certificate.p12" - security import "$RUNNER_TEMP/certificate.p12" -P "$CERTIFICATES_PASSWORD" \ + echo "$CERTIFICATES_P12" | base64 --decode > "$dir/certificate.p12" + security import "$dir/certificate.p12" -P "$CERTIFICATES_PASSWORD" \ -A -t cert -f pkcs12 -k "$KEYCHAIN_PATH" - rm -f "$RUNNER_TEMP/certificate.p12" + rm -f "$dir/certificate.p12" security set-key-partition-list -S apple-tool:,apple: -k "" "$KEYCHAIN_PATH" - security list-keychain -d user -s "$KEYCHAIN_PATH" login.keychain + security list-keychains -d user -s "$KEYCHAIN_PATH" ${saved[@]+"${saved[@]}"} + echo "SIGNING_KEYCHAIN=$KEYCHAIN_PATH" >> "$GITHUB_ENV" - certificate="$RUNNER_TEMP/certificate.pem" + certificate="$dir/certificate.pem" security find-certificate -c "Developer ID Application" -p "$KEYCHAIN_PATH" > "$certificate" expiry="$(openssl x509 -noout -enddate -in "$certificate" | sed 's/^notAfter=//')" if ! openssl x509 -noout -checkend $((60 * 24 * 3600)) -in "$certificate" > /dev/null; then @@ -54,22 +71,24 @@ runs: rm -f "$certificate" echo "Developer ID Application certificate valid until $expiry" - - name: Store the notary credentials + - name: Write the notary key shell: bash env: ASC_KEY_ID: ${{ inputs.asc-key-id }} ASC_ISSUER_ID: ${{ inputs.asc-issuer-id }} ASC_KEY_P8: ${{ inputs.asc-key-p8 }} - # notarytool copies the key into the keychain item, so the file can go at once. + # notarize.sh passes the key to every notarytool call, so no notary profile goes into a keychain. + # history checks the key now rather than after the build. run: | set -euo pipefail - key="$RUNNER_TEMP/AuthKey_$ASC_KEY_ID.p8" + key="$RUNNER_TEMP/apple-signing/AuthKey_$ASC_KEY_ID.p8" (umask 077 && printf '%s\n' "$ASC_KEY_P8" > "$key") - status=0 - xcrun notarytool store-credentials "TablePro" --key "$key" --key-id "$ASC_KEY_ID" \ - --issuer "$ASC_ISSUER_ID" || status=$? - rm -f "$key" - exit "$status" + xcrun notarytool history --key "$key" --key-id "$ASC_KEY_ID" --issuer "$ASC_ISSUER_ID" > /dev/null + { + echo "NOTARY_KEY=$key" + echo "NOTARY_KEY_ID=$ASC_KEY_ID" + echo "NOTARY_ISSUER=$ASC_ISSUER_ID" + } >> "$GITHUB_ENV" - name: Install provisioning profile if: inputs.provisioning-profile != '' diff --git a/.github/actions/apple-signing/cleanup/action.yml b/.github/actions/apple-signing/cleanup/action.yml new file mode 100644 index 0000000000..3faf1dd26b --- /dev/null +++ b/.github/actions/apple-signing/cleanup/action.yml @@ -0,0 +1,30 @@ +name: Apple signing cleanup +description: Restores the user keychain search list apple-signing saved and deletes its keychain, notary key and provisioning profile. Run it in an always() step. + +runs: + using: composite + steps: + # A leftover is a warning, not a failed release: the runner empties RUNNER_TEMP after the job anyway. + - shell: bash + run: | + set -uo pipefail + dir="$RUNNER_TEMP/apple-signing" + if [ -f "$dir/search-list" ]; then + saved=() + while IFS= read -r keychain; do + if [ -n "$keychain" ]; then + saved+=("$keychain") + fi + done < "$dir/search-list" + security list-keychains -d user -s ${saved[@]+"${saved[@]}"} \ + || echo "::warning::Could not restore the user keychain search list" + fi + if [ -e "$dir/signing.keychain-db" ]; then + security delete-keychain "$dir/signing.keychain-db" \ + || echo "::warning::Could not delete the signing keychain" + fi + rm -rf "$dir" || echo "::warning::Could not delete $dir" + rm -f "$HOME/Library/MobileDevice/Provisioning Profiles/tablepro.provisionprofile" \ + || echo "::warning::Could not delete the provisioning profile" + echo "User keychain search list:" + security list-keychains -d user || true diff --git a/.github/actions/prepare-build/action.yml b/.github/actions/prepare-build/action.yml index 007d7f6f68..53c3459209 100644 --- a/.github/actions/prepare-build/action.yml +++ b/.github/actions/prepare-build/action.yml @@ -1,6 +1,12 @@ name: Prepare build description: Installs XcodeGen, downloads the static libraries, generates apple/TablePro.xcodeproj and restores the Swift package checkouts. +inputs: + restore-cache: + description: false skips the restore and points build.sh at an empty folder under RUNNER_TEMP, so a release job builds from no cache another run wrote, nor from the ~/.spm-cache the self-hosted Mac keeps. + required: false + default: "true" + outputs: cache-hit: description: The restore step's cache-hit output. @@ -27,8 +33,19 @@ runs: # macos-tests.yml saves this cache. - name: Restore Swift package checkouts id: spm + if: inputs.restore-cache != 'false' uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.spm-cache key: spm-${{ runner.os }}-${{ hashFiles('apple/TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} restore-keys: spm-${{ runner.os }}- + + - name: Use an empty Swift package folder + if: inputs.restore-cache == 'false' + shell: bash + run: | + set -euo pipefail + packages="$RUNNER_TEMP/spm-packages" + rm -rf "$packages" + mkdir -p "$packages" + echo "SPM_PACKAGES_DIR=$packages" >> "$GITHUB_ENV" diff --git a/.github/workflows/release-build.yml b/.github/workflows/release-build.yml index 7995797fa6..c2f52d8b2c 100644 --- a/.github/workflows/release-build.yml +++ b/.github/workflows/release-build.yml @@ -1,18 +1,22 @@ name: Release Build # The Release configuration of the app and every registry plugin, universal and unsigned, packaged -# the way a tag packages it, so a packaging failure shows before the tag: nightly on main and on demand. +# the way a tag packages it, so a packaging failure shows before the tag: nightly on main, weekly on +# the hosted runner, and on demand. on: schedule: - cron: '0 19 * * *' + # Weekly on the hosted runner, so the MAC_RUNNER fallback stays tested. The changes job and + # runs-on below match this string. + - cron: '0 7 * * 0' workflow_dispatch: inputs: runner: - description: Where the build runs. github is the fallback while the self-hosted Mac is offline. + description: Where the build runs. default follows the MAC_RUNNER repository variable. type: choice - options: [self-hosted, github] - default: self-hosted + options: [default, self-hosted, github] + default: default permissions: contents: read @@ -32,16 +36,17 @@ jobs: with: fetch-depth: 0 - # A nightly diffs against the last nightly that ended green, built or skipped: main up to + # A nightly diffs against the last scheduled run that ended green, built or skipped: main up to # that commit is covered. - name: Decide whether the Release build runs id: decide env: EVENT: ${{ github.event_name }} + SCHEDULE: ${{ github.event.schedule }} GH_TOKEN: ${{ github.token }} run: | set -euo pipefail - if [ "$EVENT" = workflow_dispatch ]; then + if [ "$EVENT" = workflow_dispatch ] || [ "$SCHEDULE" = '0 7 * * 0' ]; then run=true else base=$(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/release-build.yml/runs?event=schedule&status=success&per_page=1" \ @@ -55,8 +60,8 @@ jobs: name: Release Build needs: changes if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} - # A nightly has no input, so it reads the RELEASE_BUILD_RUNNER repository variable; unset is the Mac. - runs-on: ${{ (inputs.runner || vars.RELEASE_BUILD_RUNNER) == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} + # The dispatch input wins, then the weekly schedule, then MAC_RUNNER; unset is the Mac. + runs-on: ${{ (inputs.runner == 'github' || (inputs.runner != 'self-hosted' && (github.event.schedule == '0 7 * * 0' || vars.MAC_RUNNER == 'github'))) && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} timeout-minutes: 150 # The tag's version settings run here first; 9999999 is the largest build at MAJOR 0 (apple/RELEASING.md, Versions). env: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 6c2024c48f..709865a25b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -2,6 +2,7 @@ name: Release # A v* tag builds, signs and notarizes the app and every registry plugin, publishes the GitHub # release, then adds it to the gh-pages branch and deploys that to updates.tablepro.app. +# App, Plugins and Feed run on the self-hosted Mac, or on hosted runners when MAC_RUNNER is github. on: push: @@ -85,7 +86,7 @@ jobs: app: name: App needs: verify - runs-on: xcode-27 + runs-on: ${{ vars.MAC_RUNNER == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} environment: signing # The unsigned archive alone has taken up to 123 minutes in Release Build; signing and two notarizations come on top. timeout-minutes: 180 @@ -115,6 +116,8 @@ jobs: run: printf 'ANALYTICS_HMAC_SECRET = %s\n' "$ANALYTICS_HMAC_SECRET" > Configs/Secrets.xcconfig - uses: ./.github/actions/prepare-build + with: + restore-cache: "false" - uses: ./.github/actions/apple-signing with: @@ -144,10 +147,18 @@ jobs: if-no-files-found: error retention-days: 30 + - name: Remove Secrets.xcconfig + if: always() + run: rm -f Configs/Secrets.xcconfig + + - name: Remove the signing material + if: always() + uses: ./.github/actions/apple-signing/cleanup + plugins: name: Plugins needs: verify - runs-on: xcode-27 + runs-on: ${{ vars.MAC_RUNNER == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} environment: signing timeout-minutes: 90 permissions: @@ -166,6 +177,8 @@ jobs: require-release: ${{ needs.verify.outputs.channel == 'stable' }} - uses: ./.github/actions/prepare-build + with: + restore-cache: "false" - name: Set up Go for the SAP HANA helper uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 @@ -203,6 +216,10 @@ jobs: if-no-files-found: error retention-days: 30 + - name: Remove the signing material + if: always() + uses: ./.github/actions/apple-signing/cleanup + github-release: name: GitHub release needs: [verify, app, plugins] @@ -276,7 +293,7 @@ jobs: feed: name: Feed needs: [verify, github-release] - runs-on: macos-latest + runs-on: ${{ vars.MAC_RUNNER == 'github' && 'macos-latest' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} environment: release timeout-minutes: 20 concurrency: @@ -316,8 +333,11 @@ jobs: python3 apple/scripts/release/feed.py add "$TAG" --site build/gh-pages --notes notes/feed.md \ ${flags[@]+"${flags[@]}"} + # The Mac runs as the maintainer, whose global git config signs commits with their key. - name: Push gh-pages working-directory: build/gh-pages + env: + GIT_CONFIG_GLOBAL: /dev/null run: | set -euo pipefail git config user.name "github-actions[bot]" diff --git a/apple/RELEASING.md b/apple/RELEASING.md index be48db3bfe..44e5cd158f 100644 --- a/apple/RELEASING.md +++ b/apple/RELEASING.md @@ -98,7 +98,7 @@ python3 apple/scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro ## What a tag runs -A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. +A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a run, and every job reads the version, build and channel from `verify`. `app`, `plugins` and `feed` run on the self-hosted Mac unless `MAC_RUNNER` says `github` (`.claude/rules/ci.md`, Runners and toolchain). - `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's push runs on `release/` once: macOS Tests and iOS and Package Tests must each have completed with success, their test job run and not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. @@ -111,6 +111,7 @@ A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and ev ## When a run fails - Tags never move. Before `github-release` publishes, an infrastructure failure is a re-run of the failed jobs; a code failure is fixed on `main`, picked to the line, and ships as the next number. +- The Mac is offline, or `app`, `plugins` or `feed` fails on its infrastructure: `gh variable set MAC_RUNNER --body github`, then `gh run rerun --failed`; those jobs then take the hosted runners. A job still queued for the offline Mac is cancelled first (`gh run cancel `). `gh variable delete MAC_RUNNER` switches back. - After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, a feed item with the same version and build, or a manifest release with the same build and sha256. - A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). - No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. @@ -134,7 +135,7 @@ gh workflow run withdraw.yml --ref v -f version=X.Y.Z | `github-pages` | tags `v*` | none | - No repo-level secret or deploy key is left, and no environment above has required reviewers: admin-only, immutable tags are the gate. `linux-release` also deploys from `main`, so it takes a reviewer (`linux/docs/releasing.md`). Rulesets: `.claude/rules/ci.md`, Rulesets. -- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` stores as the `TablePro` notarytool profile. The action warns when the certificate expires within 60 days. +- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` writes as a .p8 under `RUNNER_TEMP` and `notarize.sh` passes to every `notarytool` call. The action warns when the certificate expires within 60 days. - An offline backup of the Sparkle EdDSA key exists and the maintainer holds it. The key never rotates without a signed transition, since every app from 0.82 verifies the feed with it. - `gh-pages` takes pushes from `github-actions[bot]` with `GITHUB_TOKEN` and has no required checks. `feed.py add` and `withdraw` refuse an `appcast.xml` whose signature does not verify, so a hand edit is re-signed with `sign_update` before it is pushed. - `updates.tablepro.app` enforces HTTPS (Settings > Pages). @@ -169,4 +170,4 @@ apple/scripts/build-dameng.sh both && apple/scripts/build-hana.sh both apple/scripts/release/build.sh plugins --unsigned && apple/scripts/release/package.sh plugins --unsigned ``` -- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. CI builds the same in Release Build (`.claude/rules/ci.md`, Release Build). Signed: drop `--unsigned`, export the tag's `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs the profile `xcrun notarytool store-credentials TablePro --key --key-id --issuer ` creates. +- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. CI builds the same in Release Build (`.claude/rules/ci.md`, Release Build). Signed: drop `--unsigned`, export the tag's `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs `NOTARY_KEY` (the .p8 path), `NOTARY_KEY_ID` and `NOTARY_ISSUER`. diff --git a/apple/scripts/release/build.sh b/apple/scripts/release/build.sh index 8523cd0c17..357cb38744 100755 --- a/apple/scripts/release/build.sh +++ b/apple/scripts/release/build.sh @@ -3,6 +3,7 @@ set -euo pipefail # Usage: build.sh app|plugins [--unsigned] # VERSION and BUILD, when set, become MARKETING_VERSION and CURRENT_PROJECT_VERSION. +# SPM_PACKAGES_DIR, when set, holds the Swift package checkouts instead of ~/.spm-cache. # Writes build/release/TablePro.app and its archive, or build/release/plugins and plugins-dSYMs. usage() { @@ -67,7 +68,7 @@ xcodebuild_project() { shift run_xcodebuild "$name" -project TablePro.xcodeproj -configuration Release \ -destination 'generic/platform=macOS' -derivedDataPath "$DERIVED" \ - -clonedSourcePackagesDirPath "$HOME/.spm-cache" -skipPackagePluginValidation \ + -clonedSourcePackagesDirPath "${SPM_PACKAGES_DIR:-$HOME/.spm-cache}" -skipPackagePluginValidation \ "$@" "${SETTINGS[@]}" } diff --git a/apple/scripts/release/notarize.sh b/apple/scripts/release/notarize.sh index f7146efc34..fff3c0c609 100755 --- a/apple/scripts/release/notarize.sh +++ b/apple/scripts/release/notarize.sh @@ -3,10 +3,9 @@ set -euo pipefail # Usage: notarize.sh ... # One notary submission for every path, then on each: a stapled ticket and spctl. -# A path is a bundle, or a single disk image on its own. Credentials come from the TablePro keychain -# profile that .github/actions/apple-signing stores. - -PROFILE="TablePro" +# A path is a bundle, or a single disk image on its own. Every notarytool call authenticates with the +# App Store Connect API key in NOTARY_KEY (the .p8 path), NOTARY_KEY_ID and NOTARY_ISSUER, which +# .github/actions/apple-signing exports. usage() { echo "Usage: notarize.sh ..." >&2 @@ -39,6 +38,11 @@ staple() { } [ $# -gt 0 ] || usage +for name in NOTARY_KEY NOTARY_KEY_ID NOTARY_ISSUER; do + [ -n "${!name:-}" ] || die "$name is not set" +done +[ -f "$NOTARY_KEY" ] || die "no notary key at $NOTARY_KEY" +CREDENTIALS=(--key "$NOTARY_KEY" --key-id "$NOTARY_KEY_ID" --issuer "$NOTARY_ISSUER") WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT @@ -68,7 +72,7 @@ fi echo "Submitting $(basename "$submission") (${#PATHS[@]} item(s)) for notarization" result="$WORK/result.json" status=0 -xcrun notarytool submit "$submission" --keychain-profile "$PROFILE" --wait --timeout 30m \ +xcrun notarytool submit "$submission" "${CREDENTIALS[@]}" --wait --timeout 30m \ --output-format json > "$result" || status=$? cat "$result" echo @@ -76,7 +80,7 @@ id="$(plutil -extract id raw -o - "$result" 2> /dev/null || true)" state="$(plutil -extract status raw -o - "$result" 2> /dev/null || true)" if [ -n "$id" ]; then echo "Notary log for $id:" - xcrun notarytool log "$id" --keychain-profile "$PROFILE" || true + xcrun notarytool log "$id" "${CREDENTIALS[@]}" || true fi # notarytool has exited 0 on a rejected submission, so the status is read as well. [ "$status" -eq 0 ] && [ "$state" = "Accepted" ] || die "the submission ended '${state:-without a status}' (notarytool exit $status)" diff --git a/apple/scripts/release/package.sh b/apple/scripts/release/package.sh index f5d2a08682..d226c23c6c 100755 --- a/apple/scripts/release/package.sh +++ b/apple/scripts/release/package.sh @@ -3,6 +3,7 @@ set -euo pipefail # Usage: package.sh app|plugins [--unsigned] # Turns what scripts/release/build.sh wrote to build/release into the release assets in dist/, named by VERSION. +# SIGNING_KEYCHAIN, when set, is the only keychain the disk image's signing identity comes from. usage() { echo "Usage: package.sh app|plugins [--unsigned]" >&2 @@ -37,7 +38,7 @@ trap 'rm -rf "$WORK"' EXIT package_app() { local app="$OUT/TablePro.app" dsyms="$OUT/TablePro.xcarchive/dSYMs" - local dmg="$DIST/TablePro-$VERSION.dmg" update="$DIST/TablePro-$VERSION.tar.xz" + local dmg="$DIST/TablePro-$VERSION.dmg" update="$DIST/TablePro-$VERSION.tar.xz" keychain local symbols="$DIST/TablePro-$VERSION-dSYMs.zip" extracted="$WORK/extracted" [ -d "$app" ] || die "no $app; run scripts/release/build.sh app first" [ -d "$dsyms" ] || die "no $dsyms; run scripts/release/build.sh app first" @@ -48,7 +49,12 @@ package_app() { ditto "$app" "$WORK/dmg/TablePro.app" ln -s /Applications "$WORK/dmg/Applications" diskutil image create from "$WORK/dmg" "$dmg" --volumeName "TablePro $VERSION" - [ "$SIGNED" = false ] || codesign --sign "Developer ID Application" --timestamp "$dmg" + if [ "$SIGNED" = true ]; then + # The self-hosted Mac's login keychain holds a Developer ID Application identity too. + keychain=() + [ -z "${SIGNING_KEYCHAIN:-}" ] || keychain=(--keychain "$SIGNING_KEYCHAIN") + codesign --sign "Developer ID Application" ${keychain[@]+"${keychain[@]}"} --timestamp "$dmg" + fi # tar keeps the framework symlinks codesign seals; a signed bundle needs no extended attributes. /usr/bin/tar --no-mac-metadata --no-xattrs --uid 0 --gid 0 \ diff --git a/apple/scripts/release/test_notarize.py b/apple/scripts/release/test_notarize.py index e66137c3d1..db2b68b8d2 100644 --- a/apple/scripts/release/test_notarize.py +++ b/apple/scripts/release/test_notarize.py @@ -12,8 +12,14 @@ "xcrun": r"""#!/usr/bin/env bash set -euo pipefail case "$1 $2" in - "notarytool submit") echo '{"status":"Accepted","id":"stub-id","message":"Processing complete"}' ;; - "notarytool log") echo '{}' ;; + "notarytool submit") + echo "$*" >> "$STUB_STATE/notary" + echo '{"status":"Accepted","id":"stub-id","message":"Processing complete"}' + ;; + "notarytool log") + echo "$*" >> "$STUB_STATE/notary" + echo '{}' + ;; "stapler staple") name="$(basename "$3")" echo "staple $name" >> "$STUB_STATE/calls" @@ -49,7 +55,7 @@ } -class StapleTests(unittest.TestCase): +class NotarizeCase(unittest.TestCase): def setUp(self): directory = tempfile.TemporaryDirectory() self.addCleanup(directory.cleanup) @@ -58,6 +64,9 @@ def setUp(self): self.state = self.root / "state" self.bin.mkdir() self.state.mkdir() + self.key = self.root / "AuthKey_KEYID.p8" + self.key.write_text("key\n", encoding="utf-8") + self.credentials = {"NOTARY_KEY": str(self.key), "NOTARY_KEY_ID": "KEYID", "NOTARY_ISSUER": "ISSUER"} for name, body in STUBS.items(): stub = self.bin / name stub.write_text(body, encoding="utf-8") @@ -69,14 +78,18 @@ def bundle(self, name, staple_exits=()): (self.state / f"staple-{name}").write_text("".join(f"{code}\n" for code in staple_exits), encoding="utf-8") return path - def notarize(self, *paths): - env = {**os.environ, "PATH": f"{self.bin}{os.pathsep}{os.environ['PATH']}", "STUB_STATE": str(self.state)} + def notarize(self, *paths, credentials=None): + env = {key: value for key, value in os.environ.items() if not key.startswith("NOTARY_")} + env.update(self.credentials if credentials is None else credentials) + env.update({"PATH": f"{self.bin}{os.pathsep}{os.environ['PATH']}", "STUB_STATE": str(self.state)}) return subprocess.run(["bash", str(SCRIPT), *map(str, paths)], env=env, capture_output=True, text=True) - def calls(self): - calls = self.state / "calls" + def calls(self, name="calls"): + calls = self.state / name return calls.read_text(encoding="utf-8").splitlines() if calls.exists() else [] + +class StapleTests(NotarizeCase): def test_a_late_ticket_is_stapled_on_a_retry(self): late = self.bundle("Late.tableplugin", [65, 68]) ready = self.bundle("Ready.tableplugin") @@ -110,5 +123,31 @@ def test_any_other_stapler_failure_is_final(self): self.assertIn("stapling Unsigned.tableplugin failed (stapler exit 66)", result.stderr) +class CredentialTests(NotarizeCase): + def test_every_notarytool_call_passes_the_api_key(self): + bundle = self.bundle("Ready.tableplugin") + result = self.notarize(bundle) + self.assertEqual(result.returncode, 0, result.stderr) + calls = self.calls("notary") + self.assertEqual([call.split()[1] for call in calls], ["submit", "log"]) + for call in calls: + self.assertIn(f"--key {self.key} --key-id KEYID --issuer ISSUER", call) + self.assertNotIn("--keychain-profile", call) + + def test_missing_credentials_fail_before_submitting(self): + bundle = self.bundle("Ready.tableplugin") + result = self.notarize(bundle, credentials={"NOTARY_KEY": str(self.key), "NOTARY_ISSUER": "ISSUER"}) + self.assertEqual(result.returncode, 1) + self.assertIn("NOTARY_KEY_ID is not set", result.stderr) + self.assertEqual(self.calls("notary"), []) + + def test_a_missing_key_file_fails_before_submitting(self): + bundle = self.bundle("Ready.tableplugin") + result = self.notarize(bundle, credentials={**self.credentials, "NOTARY_KEY": str(self.root / "gone.p8")}) + self.assertEqual(result.returncode, 1) + self.assertIn("no notary key at", result.stderr) + self.assertEqual(self.calls("notary"), []) + + if __name__ == "__main__": unittest.main() From 27a64ca98ac9e7ba73a9cf2f126c1d0c6a955078 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Mon, 19 Oct 2026 20:32:59 +0700 Subject: [PATCH 2/5] ci(release): keep feed on macos-latest so the Sparkle key stays off the Mac --- .claude/rules/ci.md | 7 ++++--- .github/workflows/release.yml | 9 ++++----- apple/RELEASING.md | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index b78bf8cdc0..f4336aa855 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -34,11 +34,12 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Runners and toolchain - Every Xcode job in the test workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. -- The repository variable `MAC_RUNNER` picks the machine for Release Build and for `app`, `plugins` and `feed` in `release.yml`. Unset is `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. `github` sends them to hosted runners: `xcode-27`, and `macos-latest` for `feed`. Switching: `apple/RELEASING.md`, When a run fails. +- The repository variable `MAC_RUNNER` picks the machine for Release Build and for `app` and `plugins` in `release.yml`. Unset is `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. `github` sends them to `xcode-27`. Switching: `apple/RELEASING.md`, When a run fails. +- `feed` stays on `macos-latest` whatever `MAC_RUNNER` says. The Sparkle key cannot rotate (`apple/RELEASING.md`, Keys and environments), and on the Mac any process of the maintainer's can read a step's environment with `ps -E`. - Only trusted events reach `tablepro-release`: `release.yml` runs on `v*` tags, Release Build on its schedules and dispatch. No `pull_request` or `pull_request_target` job names the label or reads `MAC_RUNNER`, and fork PR workflows need approval for every outside contributor. -- One runner takes one job at a time, so the Mac jobs need no concurrency group; `feed` keeps its `gh-pages` group (`apple/RELEASING.md`, What a tag runs). A second runner on the label would race on the user keychain search list. +- One runner takes one job at a time, so the Mac jobs need no concurrency group. A second runner on the label would race on the user keychain search list. - The Mac keeps its home folder and keychain search list between jobs. `.github/actions/apple-signing` saves the user search list, puts a keychain of its own in front, and writes the App Store Connect .p8 under `RUNNER_TEMP`, which `notarize.sh` passes to every `notarytool` call, so no notary profile goes into a keychain. `app` and `plugins` end with an `if: always()` step, `apple-signing/cleanup`, which restores the list and deletes the keychain, the .p8 and the provisioning profile; `app` also deletes `Configs/Secrets.xcconfig`. -- The Mac runs as the maintainer: the login keychain holds a Developer ID Application identity, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone, and the global git config signs commits, so `feed` pushes `gh-pages` with `GIT_CONFIG_GLOBAL=/dev/null`. +- The Mac runs as the maintainer: the login keychain holds a Developer ID Application identity, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone. - The Xcode version, the build every job uses and Apple's released build are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version and build, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. The version alone is no pin: the 27.1 beta, 27A9269, is also 27.1. - A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). - A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs Apple's released build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 709865a25b..87f9821f26 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -2,7 +2,7 @@ name: Release # A v* tag builds, signs and notarizes the app and every registry plugin, publishes the GitHub # release, then adds it to the gh-pages branch and deploys that to updates.tablepro.app. -# App, Plugins and Feed run on the self-hosted Mac, or on hosted runners when MAC_RUNNER is github. +# App and Plugins run on the self-hosted Mac, or on xcode-27 when MAC_RUNNER is github. on: push: @@ -293,7 +293,9 @@ jobs: feed: name: Feed needs: [verify, github-release] - runs-on: ${{ vars.MAC_RUNNER == 'github' && 'macos-latest' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} + # Hosted whatever MAC_RUNNER says: the Sparkle key cannot rotate, and a process on the Mac can + # read a job's environment. Feed needs neither Xcode nor the signing keychain. + runs-on: macos-latest environment: release timeout-minutes: 20 concurrency: @@ -333,11 +335,8 @@ jobs: python3 apple/scripts/release/feed.py add "$TAG" --site build/gh-pages --notes notes/feed.md \ ${flags[@]+"${flags[@]}"} - # The Mac runs as the maintainer, whose global git config signs commits with their key. - name: Push gh-pages working-directory: build/gh-pages - env: - GIT_CONFIG_GLOBAL: /dev/null run: | set -euo pipefail git config user.name "github-actions[bot]" diff --git a/apple/RELEASING.md b/apple/RELEASING.md index 44e5cd158f..f2832b0a7c 100644 --- a/apple/RELEASING.md +++ b/apple/RELEASING.md @@ -98,7 +98,7 @@ python3 apple/scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro ## What a tag runs -A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a run, and every job reads the version, build and channel from `verify`. `app`, `plugins` and `feed` run on the self-hosted Mac unless `MAC_RUNNER` says `github` (`.claude/rules/ci.md`, Runners and toolchain). +A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a run, and every job reads the version, build and channel from `verify`. `app` and `plugins` run on the self-hosted Mac unless `MAC_RUNNER` says `github` (`.claude/rules/ci.md`, Runners and toolchain). - `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's push runs on `release/` once: macOS Tests and iOS and Package Tests must each have completed with success, their test job run and not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. @@ -111,7 +111,7 @@ A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a r ## When a run fails - Tags never move. Before `github-release` publishes, an infrastructure failure is a re-run of the failed jobs; a code failure is fixed on `main`, picked to the line, and ships as the next number. -- The Mac is offline, or `app`, `plugins` or `feed` fails on its infrastructure: `gh variable set MAC_RUNNER --body github`, then `gh run rerun --failed`; those jobs then take the hosted runners. A job still queued for the offline Mac is cancelled first (`gh run cancel `). `gh variable delete MAC_RUNNER` switches back. +- The Mac is offline, or `app` or `plugins` fails on its infrastructure: `gh variable set MAC_RUNNER --body github`, then `gh run rerun --failed`; those jobs then take `xcode-27`. A job still queued for the offline Mac is cancelled first (`gh run cancel `). `gh variable delete MAC_RUNNER` switches back. - After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, a feed item with the same version and build, or a manifest release with the same build and sha256. - A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). - No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. From 1ee830fffe12ca3f63995ad5cdbfa2cbad580e18 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Mon, 19 Oct 2026 20:33:10 +0700 Subject: [PATCH 3/5] docs(ci): name what reaches the Mac runner and what it reuses --- .claude/rules/ci.md | 7 ++++--- .github/actions/prepare-build/action.yml | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index f4336aa855..c7223ebf46 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -36,10 +36,11 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Every Xcode job in the test workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. - The repository variable `MAC_RUNNER` picks the machine for Release Build and for `app` and `plugins` in `release.yml`. Unset is `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. `github` sends them to `xcode-27`. Switching: `apple/RELEASING.md`, When a run fails. - `feed` stays on `macos-latest` whatever `MAC_RUNNER` says. The Sparkle key cannot rotate (`apple/RELEASING.md`, Keys and environments), and on the Mac any process of the maintainer's can read a step's environment with `ps -E`. -- Only trusted events reach `tablepro-release`: `release.yml` runs on `v*` tags, Release Build on its schedules and dispatch. No `pull_request` or `pull_request_target` job names the label or reads `MAC_RUNNER`, and fork PR workflows need approval for every outside contributor. +- No `pull_request` or `pull_request_target` job on main names `tablepro-release` or reads `MAC_RUNNER`. A run takes its workflow files from its own ref, though, so the label also gets any job a pushed branch's workflow sends it, a Release Build dispatched on a branch, and a fork PR's own workflows once its run is approved. The repo asks approval for every outside contributor, so that approval is the gate: a fork PR that changes `.github/workflows/` can send any job there, under any spelling of the label. +- Every job on the label runs as the maintainer, with their `gh` and git credentials, their login keychain, the runner's own files in `~/actions-runner` and a user-owned `/Applications/Xcode.app`. It can read what a later tag's `app` and `plugins` get and change what they build with, so the label takes only code the maintainer would run locally. - One runner takes one job at a time, so the Mac jobs need no concurrency group. A second runner on the label would race on the user keychain search list. - The Mac keeps its home folder and keychain search list between jobs. `.github/actions/apple-signing` saves the user search list, puts a keychain of its own in front, and writes the App Store Connect .p8 under `RUNNER_TEMP`, which `notarize.sh` passes to every `notarytool` call, so no notary profile goes into a keychain. `app` and `plugins` end with an `if: always()` step, `apple-signing/cleanup`, which restores the list and deletes the keychain, the .p8 and the provisioning profile; `app` also deletes `Configs/Secrets.xcconfig`. -- The Mac runs as the maintainer: the login keychain holds a Developer ID Application identity, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone. +- The maintainer's login keychain holds a Developer ID Application identity too, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone. - The Xcode version, the build every job uses and Apple's released build are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version and build, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. The version alone is no pin: the 27.1 beta, 27A9269, is also 27.1. - A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). - A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs Apple's released build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. @@ -146,7 +147,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Caches and runner budget - Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `release.yml` neither restores nor saves it. `app` and `plugins` pass `restore-cache: false` to `prepare-build`, which points `build.sh` at an empty folder under `RUNNER_TEMP` through `SPM_PACKAGES_DIR`, so a signed build reads no cache another run wrote and not the `~/.spm-cache` the Mac keeps. +- `release.yml` neither restores nor saves it. `app` and `plugins` pass `restore-cache: false` to `prepare-build`, which points `build.sh` at an empty folder under `RUNNER_TEMP` through `SPM_PACKAGES_DIR`, so a signed build restores no Actions cache and skips the `~/.spm-cache` the Mac keeps. On the Mac it still builds with what the maintainer's account keeps between jobs: Xcode, the tool cache (XcodeGen once its `--version` matches, Go), rustup and Cargo, Go's module and build caches, and SwiftPM's repository cache (Runners and toolchain). - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other Apple cache: no libs or DerivedData cache, and none in the iOS or plugin workflows. The Linux jobs cache Cargo builds through `Swatinem/rust-cache` and the Flatpak build. - CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. The Mac jobs take none of them unless `MAC_RUNNER`, a dispatch or the weekly Release Build picks `github`. diff --git a/.github/actions/prepare-build/action.yml b/.github/actions/prepare-build/action.yml index 53c3459209..33aa7beba5 100644 --- a/.github/actions/prepare-build/action.yml +++ b/.github/actions/prepare-build/action.yml @@ -3,7 +3,7 @@ description: Installs XcodeGen, downloads the static libraries, generates apple/ inputs: restore-cache: - description: false skips the restore and points build.sh at an empty folder under RUNNER_TEMP, so a release job builds from no cache another run wrote, nor from the ~/.spm-cache the self-hosted Mac keeps. + description: false skips the restore and points build.sh at an empty folder under RUNNER_TEMP, so a release job restores no Actions cache and skips the ~/.spm-cache the self-hosted Mac keeps. required: false default: "true" From 940ea6bf347b6c24d14ffc8584952999dd8c532e Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Mon, 19 Oct 2026 20:46:38 +0700 Subject: [PATCH 4/5] revert(release): keep signing, notarization and feed on hosted runners --- .github/actions/apple-signing/action.yml | 51 ++++++------------ .../actions/apple-signing/cleanup/action.yml | 30 ----------- .github/actions/prepare-build/action.yml | 17 ------ .github/workflows/release.yml | 23 +------- apple/scripts/release/build.sh | 3 +- apple/scripts/release/notarize.sh | 16 +++--- apple/scripts/release/package.sh | 10 +--- apple/scripts/release/test_notarize.py | 53 +++---------------- 8 files changed, 34 insertions(+), 169 deletions(-) delete mode 100644 .github/actions/apple-signing/cleanup/action.yml diff --git a/.github/actions/apple-signing/action.yml b/.github/actions/apple-signing/action.yml index 92d9cf2189..9a230d9909 100644 --- a/.github/actions/apple-signing/action.yml +++ b/.github/actions/apple-signing/action.yml @@ -1,5 +1,5 @@ name: Apple signing -description: Imports the Developer ID certificate into a keychain of its own, writes the notary key notarize.sh reads, and installs the provisioning profile. apple-signing/cleanup removes all of it. +description: Imports the Developer ID certificate, stores the notary credentials as the TablePro profile, and installs the provisioning profile. inputs: certificate-p12: @@ -22,8 +22,6 @@ inputs: description: Contents of that key's .p8 file. required: true -# Everything but the provisioning profile goes under RUNNER_TEMP/apple-signing. The self-hosted Mac keeps -# its home folder and keychain search list between jobs, so apple-signing/cleanup removes it all. runs: using: composite steps: @@ -33,36 +31,21 @@ runs: CERTIFICATES_P12: ${{ inputs.certificate-p12 }} CERTIFICATES_PASSWORD: ${{ inputs.certificate-password }} # A keychain of its own, unlocked with an empty password and given a six hour lock timeout, - # so a long notarization wait cannot relock it mid-run. + # so a long notarization wait cannot relock it mid-run. The runner is thrown away afterwards. run: | set -euo pipefail - dir="$RUNNER_TEMP/apple-signing" - KEYCHAIN_PATH="$dir/signing.keychain-db" - mkdir -p "$dir" - chmod 700 "$dir" - - # A runner killed mid-job leaves its deleted keychain on the list, so only existing files are kept. - security list-keychains -d user | sed -e 's/^[[:space:]]*"//' -e 's/"$//' > "$dir/listed" - saved=() - while IFS= read -r keychain; do - if [ -e "$keychain" ]; then - saved+=("$keychain") - fi - done < "$dir/listed" - printf '%s\n' ${saved[@]+"${saved[@]}"} > "$dir/search-list" - + KEYCHAIN_PATH="$RUNNER_TEMP/app-signing.keychain-db" security create-keychain -p "" "$KEYCHAIN_PATH" security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH" security unlock-keychain -p "" "$KEYCHAIN_PATH" - echo "$CERTIFICATES_P12" | base64 --decode > "$dir/certificate.p12" - security import "$dir/certificate.p12" -P "$CERTIFICATES_PASSWORD" \ + echo "$CERTIFICATES_P12" | base64 --decode > "$RUNNER_TEMP/certificate.p12" + security import "$RUNNER_TEMP/certificate.p12" -P "$CERTIFICATES_PASSWORD" \ -A -t cert -f pkcs12 -k "$KEYCHAIN_PATH" - rm -f "$dir/certificate.p12" + rm -f "$RUNNER_TEMP/certificate.p12" security set-key-partition-list -S apple-tool:,apple: -k "" "$KEYCHAIN_PATH" - security list-keychains -d user -s "$KEYCHAIN_PATH" ${saved[@]+"${saved[@]}"} - echo "SIGNING_KEYCHAIN=$KEYCHAIN_PATH" >> "$GITHUB_ENV" + security list-keychain -d user -s "$KEYCHAIN_PATH" login.keychain - certificate="$dir/certificate.pem" + certificate="$RUNNER_TEMP/certificate.pem" security find-certificate -c "Developer ID Application" -p "$KEYCHAIN_PATH" > "$certificate" expiry="$(openssl x509 -noout -enddate -in "$certificate" | sed 's/^notAfter=//')" if ! openssl x509 -noout -checkend $((60 * 24 * 3600)) -in "$certificate" > /dev/null; then @@ -71,24 +54,22 @@ runs: rm -f "$certificate" echo "Developer ID Application certificate valid until $expiry" - - name: Write the notary key + - name: Store the notary credentials shell: bash env: ASC_KEY_ID: ${{ inputs.asc-key-id }} ASC_ISSUER_ID: ${{ inputs.asc-issuer-id }} ASC_KEY_P8: ${{ inputs.asc-key-p8 }} - # notarize.sh passes the key to every notarytool call, so no notary profile goes into a keychain. - # history checks the key now rather than after the build. + # notarytool copies the key into the keychain item, so the file can go at once. run: | set -euo pipefail - key="$RUNNER_TEMP/apple-signing/AuthKey_$ASC_KEY_ID.p8" + key="$RUNNER_TEMP/AuthKey_$ASC_KEY_ID.p8" (umask 077 && printf '%s\n' "$ASC_KEY_P8" > "$key") - xcrun notarytool history --key "$key" --key-id "$ASC_KEY_ID" --issuer "$ASC_ISSUER_ID" > /dev/null - { - echo "NOTARY_KEY=$key" - echo "NOTARY_KEY_ID=$ASC_KEY_ID" - echo "NOTARY_ISSUER=$ASC_ISSUER_ID" - } >> "$GITHUB_ENV" + status=0 + xcrun notarytool store-credentials "TablePro" --key "$key" --key-id "$ASC_KEY_ID" \ + --issuer "$ASC_ISSUER_ID" || status=$? + rm -f "$key" + exit "$status" - name: Install provisioning profile if: inputs.provisioning-profile != '' diff --git a/.github/actions/apple-signing/cleanup/action.yml b/.github/actions/apple-signing/cleanup/action.yml deleted file mode 100644 index 3faf1dd26b..0000000000 --- a/.github/actions/apple-signing/cleanup/action.yml +++ /dev/null @@ -1,30 +0,0 @@ -name: Apple signing cleanup -description: Restores the user keychain search list apple-signing saved and deletes its keychain, notary key and provisioning profile. Run it in an always() step. - -runs: - using: composite - steps: - # A leftover is a warning, not a failed release: the runner empties RUNNER_TEMP after the job anyway. - - shell: bash - run: | - set -uo pipefail - dir="$RUNNER_TEMP/apple-signing" - if [ -f "$dir/search-list" ]; then - saved=() - while IFS= read -r keychain; do - if [ -n "$keychain" ]; then - saved+=("$keychain") - fi - done < "$dir/search-list" - security list-keychains -d user -s ${saved[@]+"${saved[@]}"} \ - || echo "::warning::Could not restore the user keychain search list" - fi - if [ -e "$dir/signing.keychain-db" ]; then - security delete-keychain "$dir/signing.keychain-db" \ - || echo "::warning::Could not delete the signing keychain" - fi - rm -rf "$dir" || echo "::warning::Could not delete $dir" - rm -f "$HOME/Library/MobileDevice/Provisioning Profiles/tablepro.provisionprofile" \ - || echo "::warning::Could not delete the provisioning profile" - echo "User keychain search list:" - security list-keychains -d user || true diff --git a/.github/actions/prepare-build/action.yml b/.github/actions/prepare-build/action.yml index 33aa7beba5..007d7f6f68 100644 --- a/.github/actions/prepare-build/action.yml +++ b/.github/actions/prepare-build/action.yml @@ -1,12 +1,6 @@ name: Prepare build description: Installs XcodeGen, downloads the static libraries, generates apple/TablePro.xcodeproj and restores the Swift package checkouts. -inputs: - restore-cache: - description: false skips the restore and points build.sh at an empty folder under RUNNER_TEMP, so a release job restores no Actions cache and skips the ~/.spm-cache the self-hosted Mac keeps. - required: false - default: "true" - outputs: cache-hit: description: The restore step's cache-hit output. @@ -33,19 +27,8 @@ runs: # macos-tests.yml saves this cache. - name: Restore Swift package checkouts id: spm - if: inputs.restore-cache != 'false' uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.spm-cache key: spm-${{ runner.os }}-${{ hashFiles('apple/TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} restore-keys: spm-${{ runner.os }}- - - - name: Use an empty Swift package folder - if: inputs.restore-cache == 'false' - shell: bash - run: | - set -euo pipefail - packages="$RUNNER_TEMP/spm-packages" - rm -rf "$packages" - mkdir -p "$packages" - echo "SPM_PACKAGES_DIR=$packages" >> "$GITHUB_ENV" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 87f9821f26..6c2024c48f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -2,7 +2,6 @@ name: Release # A v* tag builds, signs and notarizes the app and every registry plugin, publishes the GitHub # release, then adds it to the gh-pages branch and deploys that to updates.tablepro.app. -# App and Plugins run on the self-hosted Mac, or on xcode-27 when MAC_RUNNER is github. on: push: @@ -86,7 +85,7 @@ jobs: app: name: App needs: verify - runs-on: ${{ vars.MAC_RUNNER == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} + runs-on: xcode-27 environment: signing # The unsigned archive alone has taken up to 123 minutes in Release Build; signing and two notarizations come on top. timeout-minutes: 180 @@ -116,8 +115,6 @@ jobs: run: printf 'ANALYTICS_HMAC_SECRET = %s\n' "$ANALYTICS_HMAC_SECRET" > Configs/Secrets.xcconfig - uses: ./.github/actions/prepare-build - with: - restore-cache: "false" - uses: ./.github/actions/apple-signing with: @@ -147,18 +144,10 @@ jobs: if-no-files-found: error retention-days: 30 - - name: Remove Secrets.xcconfig - if: always() - run: rm -f Configs/Secrets.xcconfig - - - name: Remove the signing material - if: always() - uses: ./.github/actions/apple-signing/cleanup - plugins: name: Plugins needs: verify - runs-on: ${{ vars.MAC_RUNNER == 'github' && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} + runs-on: xcode-27 environment: signing timeout-minutes: 90 permissions: @@ -177,8 +166,6 @@ jobs: require-release: ${{ needs.verify.outputs.channel == 'stable' }} - uses: ./.github/actions/prepare-build - with: - restore-cache: "false" - name: Set up Go for the SAP HANA helper uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 @@ -216,10 +203,6 @@ jobs: if-no-files-found: error retention-days: 30 - - name: Remove the signing material - if: always() - uses: ./.github/actions/apple-signing/cleanup - github-release: name: GitHub release needs: [verify, app, plugins] @@ -293,8 +276,6 @@ jobs: feed: name: Feed needs: [verify, github-release] - # Hosted whatever MAC_RUNNER says: the Sparkle key cannot rotate, and a process on the Mac can - # read a job's environment. Feed needs neither Xcode nor the signing keychain. runs-on: macos-latest environment: release timeout-minutes: 20 diff --git a/apple/scripts/release/build.sh b/apple/scripts/release/build.sh index 357cb38744..8523cd0c17 100755 --- a/apple/scripts/release/build.sh +++ b/apple/scripts/release/build.sh @@ -3,7 +3,6 @@ set -euo pipefail # Usage: build.sh app|plugins [--unsigned] # VERSION and BUILD, when set, become MARKETING_VERSION and CURRENT_PROJECT_VERSION. -# SPM_PACKAGES_DIR, when set, holds the Swift package checkouts instead of ~/.spm-cache. # Writes build/release/TablePro.app and its archive, or build/release/plugins and plugins-dSYMs. usage() { @@ -68,7 +67,7 @@ xcodebuild_project() { shift run_xcodebuild "$name" -project TablePro.xcodeproj -configuration Release \ -destination 'generic/platform=macOS' -derivedDataPath "$DERIVED" \ - -clonedSourcePackagesDirPath "${SPM_PACKAGES_DIR:-$HOME/.spm-cache}" -skipPackagePluginValidation \ + -clonedSourcePackagesDirPath "$HOME/.spm-cache" -skipPackagePluginValidation \ "$@" "${SETTINGS[@]}" } diff --git a/apple/scripts/release/notarize.sh b/apple/scripts/release/notarize.sh index fff3c0c609..f7146efc34 100755 --- a/apple/scripts/release/notarize.sh +++ b/apple/scripts/release/notarize.sh @@ -3,9 +3,10 @@ set -euo pipefail # Usage: notarize.sh ... # One notary submission for every path, then on each: a stapled ticket and spctl. -# A path is a bundle, or a single disk image on its own. Every notarytool call authenticates with the -# App Store Connect API key in NOTARY_KEY (the .p8 path), NOTARY_KEY_ID and NOTARY_ISSUER, which -# .github/actions/apple-signing exports. +# A path is a bundle, or a single disk image on its own. Credentials come from the TablePro keychain +# profile that .github/actions/apple-signing stores. + +PROFILE="TablePro" usage() { echo "Usage: notarize.sh ..." >&2 @@ -38,11 +39,6 @@ staple() { } [ $# -gt 0 ] || usage -for name in NOTARY_KEY NOTARY_KEY_ID NOTARY_ISSUER; do - [ -n "${!name:-}" ] || die "$name is not set" -done -[ -f "$NOTARY_KEY" ] || die "no notary key at $NOTARY_KEY" -CREDENTIALS=(--key "$NOTARY_KEY" --key-id "$NOTARY_KEY_ID" --issuer "$NOTARY_ISSUER") WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT @@ -72,7 +68,7 @@ fi echo "Submitting $(basename "$submission") (${#PATHS[@]} item(s)) for notarization" result="$WORK/result.json" status=0 -xcrun notarytool submit "$submission" "${CREDENTIALS[@]}" --wait --timeout 30m \ +xcrun notarytool submit "$submission" --keychain-profile "$PROFILE" --wait --timeout 30m \ --output-format json > "$result" || status=$? cat "$result" echo @@ -80,7 +76,7 @@ id="$(plutil -extract id raw -o - "$result" 2> /dev/null || true)" state="$(plutil -extract status raw -o - "$result" 2> /dev/null || true)" if [ -n "$id" ]; then echo "Notary log for $id:" - xcrun notarytool log "$id" "${CREDENTIALS[@]}" || true + xcrun notarytool log "$id" --keychain-profile "$PROFILE" || true fi # notarytool has exited 0 on a rejected submission, so the status is read as well. [ "$status" -eq 0 ] && [ "$state" = "Accepted" ] || die "the submission ended '${state:-without a status}' (notarytool exit $status)" diff --git a/apple/scripts/release/package.sh b/apple/scripts/release/package.sh index d226c23c6c..f5d2a08682 100755 --- a/apple/scripts/release/package.sh +++ b/apple/scripts/release/package.sh @@ -3,7 +3,6 @@ set -euo pipefail # Usage: package.sh app|plugins [--unsigned] # Turns what scripts/release/build.sh wrote to build/release into the release assets in dist/, named by VERSION. -# SIGNING_KEYCHAIN, when set, is the only keychain the disk image's signing identity comes from. usage() { echo "Usage: package.sh app|plugins [--unsigned]" >&2 @@ -38,7 +37,7 @@ trap 'rm -rf "$WORK"' EXIT package_app() { local app="$OUT/TablePro.app" dsyms="$OUT/TablePro.xcarchive/dSYMs" - local dmg="$DIST/TablePro-$VERSION.dmg" update="$DIST/TablePro-$VERSION.tar.xz" keychain + local dmg="$DIST/TablePro-$VERSION.dmg" update="$DIST/TablePro-$VERSION.tar.xz" local symbols="$DIST/TablePro-$VERSION-dSYMs.zip" extracted="$WORK/extracted" [ -d "$app" ] || die "no $app; run scripts/release/build.sh app first" [ -d "$dsyms" ] || die "no $dsyms; run scripts/release/build.sh app first" @@ -49,12 +48,7 @@ package_app() { ditto "$app" "$WORK/dmg/TablePro.app" ln -s /Applications "$WORK/dmg/Applications" diskutil image create from "$WORK/dmg" "$dmg" --volumeName "TablePro $VERSION" - if [ "$SIGNED" = true ]; then - # The self-hosted Mac's login keychain holds a Developer ID Application identity too. - keychain=() - [ -z "${SIGNING_KEYCHAIN:-}" ] || keychain=(--keychain "$SIGNING_KEYCHAIN") - codesign --sign "Developer ID Application" ${keychain[@]+"${keychain[@]}"} --timestamp "$dmg" - fi + [ "$SIGNED" = false ] || codesign --sign "Developer ID Application" --timestamp "$dmg" # tar keeps the framework symlinks codesign seals; a signed bundle needs no extended attributes. /usr/bin/tar --no-mac-metadata --no-xattrs --uid 0 --gid 0 \ diff --git a/apple/scripts/release/test_notarize.py b/apple/scripts/release/test_notarize.py index db2b68b8d2..e66137c3d1 100644 --- a/apple/scripts/release/test_notarize.py +++ b/apple/scripts/release/test_notarize.py @@ -12,14 +12,8 @@ "xcrun": r"""#!/usr/bin/env bash set -euo pipefail case "$1 $2" in - "notarytool submit") - echo "$*" >> "$STUB_STATE/notary" - echo '{"status":"Accepted","id":"stub-id","message":"Processing complete"}' - ;; - "notarytool log") - echo "$*" >> "$STUB_STATE/notary" - echo '{}' - ;; + "notarytool submit") echo '{"status":"Accepted","id":"stub-id","message":"Processing complete"}' ;; + "notarytool log") echo '{}' ;; "stapler staple") name="$(basename "$3")" echo "staple $name" >> "$STUB_STATE/calls" @@ -55,7 +49,7 @@ } -class NotarizeCase(unittest.TestCase): +class StapleTests(unittest.TestCase): def setUp(self): directory = tempfile.TemporaryDirectory() self.addCleanup(directory.cleanup) @@ -64,9 +58,6 @@ def setUp(self): self.state = self.root / "state" self.bin.mkdir() self.state.mkdir() - self.key = self.root / "AuthKey_KEYID.p8" - self.key.write_text("key\n", encoding="utf-8") - self.credentials = {"NOTARY_KEY": str(self.key), "NOTARY_KEY_ID": "KEYID", "NOTARY_ISSUER": "ISSUER"} for name, body in STUBS.items(): stub = self.bin / name stub.write_text(body, encoding="utf-8") @@ -78,18 +69,14 @@ def bundle(self, name, staple_exits=()): (self.state / f"staple-{name}").write_text("".join(f"{code}\n" for code in staple_exits), encoding="utf-8") return path - def notarize(self, *paths, credentials=None): - env = {key: value for key, value in os.environ.items() if not key.startswith("NOTARY_")} - env.update(self.credentials if credentials is None else credentials) - env.update({"PATH": f"{self.bin}{os.pathsep}{os.environ['PATH']}", "STUB_STATE": str(self.state)}) + def notarize(self, *paths): + env = {**os.environ, "PATH": f"{self.bin}{os.pathsep}{os.environ['PATH']}", "STUB_STATE": str(self.state)} return subprocess.run(["bash", str(SCRIPT), *map(str, paths)], env=env, capture_output=True, text=True) - def calls(self, name="calls"): - calls = self.state / name + def calls(self): + calls = self.state / "calls" return calls.read_text(encoding="utf-8").splitlines() if calls.exists() else [] - -class StapleTests(NotarizeCase): def test_a_late_ticket_is_stapled_on_a_retry(self): late = self.bundle("Late.tableplugin", [65, 68]) ready = self.bundle("Ready.tableplugin") @@ -123,31 +110,5 @@ def test_any_other_stapler_failure_is_final(self): self.assertIn("stapling Unsigned.tableplugin failed (stapler exit 66)", result.stderr) -class CredentialTests(NotarizeCase): - def test_every_notarytool_call_passes_the_api_key(self): - bundle = self.bundle("Ready.tableplugin") - result = self.notarize(bundle) - self.assertEqual(result.returncode, 0, result.stderr) - calls = self.calls("notary") - self.assertEqual([call.split()[1] for call in calls], ["submit", "log"]) - for call in calls: - self.assertIn(f"--key {self.key} --key-id KEYID --issuer ISSUER", call) - self.assertNotIn("--keychain-profile", call) - - def test_missing_credentials_fail_before_submitting(self): - bundle = self.bundle("Ready.tableplugin") - result = self.notarize(bundle, credentials={"NOTARY_KEY": str(self.key), "NOTARY_ISSUER": "ISSUER"}) - self.assertEqual(result.returncode, 1) - self.assertIn("NOTARY_KEY_ID is not set", result.stderr) - self.assertEqual(self.calls("notary"), []) - - def test_a_missing_key_file_fails_before_submitting(self): - bundle = self.bundle("Ready.tableplugin") - result = self.notarize(bundle, credentials={**self.credentials, "NOTARY_KEY": str(self.root / "gone.p8")}) - self.assertEqual(result.returncode, 1) - self.assertIn("no notary key at", result.stderr) - self.assertEqual(self.calls("notary"), []) - - if __name__ == "__main__": unittest.main() From cebd60b7e62d6a7bf171699e3f2337e5639f9d8b Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Mon, 19 Oct 2026 20:46:38 +0700 Subject: [PATCH 5/5] ci(release-build): one MAC_RUNNER switch and a weekly hosted build --- .claude/rules/ci.md | 29 +++++++++++---------- .github/workflows/release-build.yml | 39 +---------------------------- apple/RELEASING.md | 8 +++--- 3 files changed, 19 insertions(+), 57 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 4a5e330b76..89c49f647b 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -27,20 +27,20 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - `gh-pages` takes no required checks (`apple/RELEASING.md`, Keys and environments). - The Apple workflows have no gate job. A test job runs on `!cancelled() && needs.changes.outputs.run != 'false'`. A job its `if:` skips reports success, and a failed changes job runs the suite rather than skipping it. -- `Changes (macOS)`, `Changes (iOS)`, `Changes (Linux)` and `Changes (Release)` are not required checks. +- `Changes (macOS)`, `Changes (iOS)` and `Changes (Linux)` are not required checks. - A PR need not be up to date with main to merge (`strict_required_status_checks_policy: false`). - A required check's name is its job's `name:`. Renaming a job changes the ruleset in the same change, and every open PR then merges main in before its checks report the new name. ## Runners and toolchain -- Every Xcode job in the test workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. -- The repository variable `MAC_RUNNER` picks the machine for Release Build and for `app` and `plugins` in `release.yml`. Unset is `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. `github` sends them to `xcode-27`. Switching: `apple/RELEASING.md`, When a run fails. -- `feed` stays on `macos-latest` whatever `MAC_RUNNER` says. The Sparkle key cannot rotate (`apple/RELEASING.md`, Keys and environments), and on the Mac any process of the maintainer's can read a step's environment with `ps -E`. -- No `pull_request` or `pull_request_target` job on main names `tablepro-release` or reads `MAC_RUNNER`. A run takes its workflow files from its own ref, though, so the label also gets any job a pushed branch's workflow sends it, a Release Build dispatched on a branch, and a fork PR's own workflows once its run is approved. The repo asks approval for every outside contributor, so that approval is the gate: a fork PR that changes `.github/workflows/` can send any job there, under any spelling of the label. -- Every job on the label runs as the maintainer, with their `gh` and git credentials, their login keychain, the runner's own files in `~/actions-runner` and a user-owned `/Applications/Xcode.app`. It can read what a later tag's `app` and `plugins` get and change what they build with, so the label takes only code the maintainer would run locally. -- One runner takes one job at a time, so the Mac jobs need no concurrency group. A second runner on the label would race on the user keychain search list. -- The Mac keeps its home folder and keychain search list between jobs. `.github/actions/apple-signing` saves the user search list, puts a keychain of its own in front, and writes the App Store Connect .p8 under `RUNNER_TEMP`, which `notarize.sh` passes to every `notarytool` call, so no notary profile goes into a keychain. `app` and `plugins` end with an `if: always()` step, `apple-signing/cleanup`, which restores the list and deletes the keychain, the .p8 and the provisioning profile; `app` also deletes `Configs/Secrets.xcconfig`. -- The maintainer's login keychain holds a Developer ID Application identity too, so `package.sh` signs the disk image from `SIGNING_KEYCHAIN` alone. +- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image, except Release Build. actionlint learns the label from `.github/actionlint.yaml`. +- Only Release Build uses `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. +- The repository variable `MAC_RUNNER` picks Release Build's machine: unset is the Mac, `github` is `xcode-27`. A dispatch's `runner` input overrides it. Switching: `apple/RELEASING.md`, Reproducing a Release build. +- The weekly Release Build always takes `xcode-27`, so it proves the hosted path while the Mac takes the nightlies. +- Signing, notarization and the feed never run on the Mac: `app`, `plugins` and `feed` in `release.yml` stay on hosted runners, and `MAC_RUNNER` does not reach them. +- On main only Release Build names `tablepro-release`. A run takes its workflow files from its own ref, though, so any workflow on a pushed branch can send a job there, and so can any fork PR run the maintainer approves. The repo asks approval for every outside contributor's run. +- Every job on the label runs as the maintainer's user, with their `gh` and git credentials. +- Never approve a fork PR run that changes `.github/` without reading that change first. - The Xcode version, the build every job uses and Apple's released build are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version and build, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. The version alone is no pin: the 27.1 beta, 27A9269, is also 27.1. - A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). - A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs Apple's released build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. @@ -71,9 +71,8 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Release Build (`release-build.yml`) -- Nightly at 19:00 UTC on main, weekly at 07:00 UTC on Sunday, and dispatch; never a PR or a push. Never cancelled. Not a required check, and no tag waits for it. -- `Changes (Release)` outputs `run`. A dispatch and the weekly run always build. A nightly runs the `release` detector against the head of the last scheduled run that ended green, read with `gh api`, so a quiet day builds nothing. -- `Release Build` runs where `MAC_RUNNER` says (Runners and toolchain). A dispatch's `runner` input, `self-hosted` or `github`, overrides it, and the weekly run always takes `xcode-27`, so the hosted fallback stays tested. It has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. +- Nightly at 19:00 UTC on main, weekly at 07:00 UTC on Sunday, and dispatch; never a PR or a push. Every run builds. Never cancelled. Not a required check, and no tag waits for it. +- The machine: Runners and toolchain. `Release Build` has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. - While the Mac is offline a run waits for it, and GitHub cancels a job still queued after 24 hours. - Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both` and `build-hana.sh both`. - `apple/scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `apple/scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `apple/build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. @@ -138,7 +137,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## What runs where - A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full. On main a docs-only or changelog-only push skips the suites through the detector; a `release/**` push runs every suite (Change detector). Release Build runs nightly on main, weekly on `xcode-27` and by dispatch only. +- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full. On main a docs-only or changelog-only push skips the suites through the detector; a `release/**` push runs every suite (Change detector). Release Build runs nightly and weekly on main and by dispatch only. - A `v*` tag runs only `release.yml`, and a `linux-v*` tag only `release-linux.yml`, which calls `linux-packages.yml`. The test workflows never run on a tag. - `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. - `build-linux.yml` builds and tests the Linux app, and `linux-packages.yml` builds its packages (`linux/docs/testing.md`, CI). What starts each is in its `on:` and job `if:` lines. Flatpak Devel and Minimum Rust version never run on a PR or push, and `linux-packages.yml` never on a push. @@ -147,10 +146,10 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## Caches and runner budget - Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `release.yml` neither restores nor saves it. `app` and `plugins` pass `restore-cache: false` to `prepare-build`, which points `build.sh` at an empty folder under `RUNNER_TEMP` through `SPM_PACKAGES_DIR`, so a signed build restores no Actions cache and skips the `~/.spm-cache` the Mac keeps. On the Mac it still builds with what the maintainer's account keeps between jobs: Xcode, the tool cache (XcodeGen once its `--version` matches, Go), rustup and Cargo, Go's module and build caches, and SwiftPM's repository cache (Runners and toolchain). +- `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other Apple cache: no libs or DerivedData cache, and none in the iOS or plugin workflows. The Linux jobs cache Cargo builds through `Swatinem/rust-cache` and the Flatpak build. -- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. The Mac jobs take none of them unless `MAC_RUNNER`, a dispatch or the weekly Release Build picks `github`. +- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. Release Build takes one only in its weekly run, or when `MAC_RUNNER` or a dispatch picks `github`. ## Writing workflow and lint code diff --git a/.github/workflows/release-build.yml b/.github/workflows/release-build.yml index c2f52d8b2c..b92a7f00da 100644 --- a/.github/workflows/release-build.yml +++ b/.github/workflows/release-build.yml @@ -7,8 +7,7 @@ name: Release Build on: schedule: - cron: '0 19 * * *' - # Weekly on the hosted runner, so the MAC_RUNNER fallback stays tested. The changes job and - # runs-on below match this string. + # Always hosted, so the MAC_RUNNER fallback stays tested. runs-on matches this string. - cron: '0 7 * * 0' workflow_dispatch: inputs: @@ -22,44 +21,8 @@ permissions: contents: read jobs: - changes: - name: Changes (Release) - runs-on: ubuntu-latest - timeout-minutes: 5 - permissions: - actions: read - contents: read - outputs: - run: ${{ steps.decide.outputs.run }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - # A nightly diffs against the last scheduled run that ended green, built or skipped: main up to - # that commit is covered. - - name: Decide whether the Release build runs - id: decide - env: - EVENT: ${{ github.event_name }} - SCHEDULE: ${{ github.event.schedule }} - GH_TOKEN: ${{ github.token }} - run: | - set -euo pipefail - if [ "$EVENT" = workflow_dispatch ] || [ "$SCHEDULE" = '0 7 * * 0' ]; then - run=true - else - base=$(gh api "repos/$GITHUB_REPOSITORY/actions/workflows/release-build.yml/runs?event=schedule&status=success&per_page=1" \ - --jq '.workflow_runs[0].head_sha // ""') - run=$(scripts/ci/detect-changed-paths.sh --base "$base" release) - fi - echo "run=$run" >> "$GITHUB_OUTPUT" - - # Reads "not false" so a failed changes job builds instead of passing. build: name: Release Build - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} # The dispatch input wins, then the weekly schedule, then MAC_RUNNER; unset is the Mac. runs-on: ${{ (inputs.runner == 'github' || (inputs.runner != 'self-hosted' && (github.event.schedule == '0 7 * * 0' || vars.MAC_RUNNER == 'github'))) && 'xcode-27' || fromJSON('["self-hosted", "macOS", "ARM64", "tablepro-release"]') }} timeout-minutes: 150 diff --git a/apple/RELEASING.md b/apple/RELEASING.md index 0c1b959ce3..d3a453a351 100644 --- a/apple/RELEASING.md +++ b/apple/RELEASING.md @@ -98,7 +98,7 @@ python3 apple/scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro ## What a tag runs -A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a run, and every job reads the version, build and channel from `verify`. `app` and `plugins` run on the self-hosted Mac unless `MAC_RUNNER` says `github` (`.claude/rules/ci.md`, Runners and toolchain). +A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. `app`, `plugins` and `feed` run on GitHub-hosted runners. - `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a lead on a patch, of more than 6 lines, or with a line starting with `#` or `- `; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's push runs on `release/` once: macOS Tests and iOS and Package Tests must each have completed with success, their test job run and not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. @@ -111,7 +111,6 @@ A `v*` tag runs `release.yml` and nothing else. No concurrency group cancels a r ## When a run fails - Tags never move. Before `github-release` publishes, an infrastructure failure is a re-run of the failed jobs; a code failure is fixed on `main`, picked to the line, and ships as the next number. -- The Mac is offline, or `app` or `plugins` fails on its infrastructure: `gh variable set MAC_RUNNER --body github`, then `gh run rerun --failed`; those jobs then take `xcode-27`. A job still queued for the offline Mac is cancelled first (`gh run cancel `). `gh variable delete MAC_RUNNER` switches back. - After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, a feed item with the same version and build, or a manifest release with the same build and sha256. - A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). - No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. @@ -135,7 +134,7 @@ gh workflow run withdraw.yml --ref v -f version=X.Y.Z | `github-pages` | tags `v*` | none | - No repo-level secret or deploy key is left, and no environment above has required reviewers: admin-only, immutable tags are the gate. `linux-release` also deploys from `main`, so it takes a reviewer (`linux/docs/releasing.md`). Rulesets: `.claude/rules/ci.md`, Rulesets. -- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` writes as a .p8 under `RUNNER_TEMP` and `notarize.sh` passes to every `notarytool` call. The action warns when the certificate expires within 60 days. +- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` stores as the `TablePro` notarytool profile. The action warns when the certificate expires within 60 days. - An offline backup of the Sparkle EdDSA key exists and the maintainer holds it. The key never rotates without a signed transition, since every app from 0.82 verifies the feed with it. - `gh-pages` takes pushes from `github-actions[bot]` with `GITHUB_TOKEN` and has no required checks. `feed.py add` and `withdraw` refuse an `appcast.xml` whose signature does not verify, so a hand edit is re-signed with `sign_update` before it is pushed. - `updates.tablepro.app` enforces HTTPS (Settings > Pages). @@ -170,4 +169,5 @@ apple/scripts/build-dameng.sh both && apple/scripts/build-hana.sh both apple/scripts/release/build.sh plugins --unsigned && apple/scripts/release/package.sh plugins --unsigned ``` -- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. CI builds the same in Release Build (`.claude/rules/ci.md`, Release Build). Signed: drop `--unsigned`, export the tag's `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs `NOTARY_KEY` (the .p8 path), `NOTARY_KEY_ID` and `NOTARY_ISSUER`. +- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. CI builds the same in Release Build (`.claude/rules/ci.md`, Release Build). Signed: drop `--unsigned`, export the tag's `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs the profile `xcrun notarytool store-credentials TablePro --key --key-id --issuer ` creates. +- Nightly Release Builds run on the self-hosted Mac. While it is offline, `gh variable set MAC_RUNNER --body github` sends them to `xcode-27`, and `gh variable delete MAC_RUNNER` switches back.