From 19adc31b0faad7454fd6c0799bb90a39c365ee7a Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 04:36:35 +0700 Subject: [PATCH 01/10] refactor(plugins): stamp the PluginKit version from one build setting --- .claude/rules/plugin-system.md | 5 +- .github/workflows/build-plugin.yml | 15 +++--- .github/workflows/build.yml | 8 +-- Configs/Base.xcconfig | 9 ++-- Configs/PluginKit.xcconfig | 3 ++ Plugins/BeancountDriverPlugin/Info.plist | 2 +- Plugins/BigQueryDriverPlugin/Info.plist | 2 +- Plugins/BigtableDriverPlugin/Info.plist | 2 +- Plugins/CSVExportPlugin/Info.plist | 2 +- Plugins/CSVImportPlugin/Info.plist | 2 +- Plugins/CassandraDriverPlugin/Info.plist | 2 +- Plugins/ClickHouseDriverPlugin/Info.plist | 2 +- Plugins/CloudflareD1DriverPlugin/Info.plist | 2 +- .../CloudflareR2SQLDriverPlugin/Info.plist | 2 +- Plugins/DamengDriverPlugin/Info.plist | 2 +- Plugins/DuckDBDriverPlugin/Info.plist | 2 +- Plugins/DynamoDBDriverPlugin/Info.plist | 2 +- Plugins/ElasticsearchDriverPlugin/Info.plist | 2 +- Plugins/EtcdDriverPlugin/Info.plist | 2 +- Plugins/FirebaseDriverPlugin/Info.plist | 2 +- Plugins/HanaDriverPlugin/Info.plist | 2 +- Plugins/JSONExportPlugin/Info.plist | 2 +- Plugins/JSONImportPlugin/Info.plist | 2 +- Plugins/KafkaDriverPlugin/Info.plist | 2 +- Plugins/LibSQLDriverPlugin/Info.plist | 2 +- Plugins/MQLExportPlugin/Info.plist | 2 +- Plugins/MSSQLDriverPlugin/Info.plist | 2 +- Plugins/MarkdownExportPlugin/Info.plist | 2 +- Plugins/MongoDBDriverPlugin/Info.plist | 2 +- Plugins/MySQLDriverPlugin/Info.plist | 2 +- Plugins/OracleDriverPlugin/Info.plist | 2 +- Plugins/ParquetExportPlugin/Info.plist | 2 +- Plugins/PostgreSQLDriverPlugin/Info.plist | 2 +- Plugins/RedisDriverPlugin/Info.plist | 2 +- Plugins/SQLExportPlugin/Info.plist | 2 +- Plugins/SQLImportPlugin/Info.plist | 2 +- Plugins/SQLiteDriverPlugin/Info.plist | 2 +- Plugins/SnowflakeDriverPlugin/Info.plist | 2 +- Plugins/SpannerDriverPlugin/Info.plist | 2 +- Plugins/SurrealDBDriverPlugin/Info.plist | 2 +- Plugins/TeradataDriverPlugin/Info.plist | 2 +- Plugins/TrinoDriverPlugin/Info.plist | 2 +- Plugins/TypesenseDriverPlugin/Info.plist | 2 +- Plugins/WeaviateDriverPlugin/Info.plist | 2 +- Plugins/XLSXExportPlugin/Info.plist | 2 +- Plugins/XLSXImportPlugin/Info.plist | 2 +- TablePro/Core/Plugins/PluginManager.swift | 14 ++++- TablePro/Info.plist | 4 ++ .../Plugins/BundledPluginStampTests.swift | 41 +++++++++++++++ project.yml | 12 +++++ scripts/ci/check-pluginkit-bump-cadence.py | 41 +++++++++------ .../ci/test_check_pluginkit_bump_cadence.py | 52 +++++++++++++++---- scripts/release-plugin-for-shipped-app.sh | 12 +++-- 53 files changed, 208 insertions(+), 90 deletions(-) create mode 100644 Configs/PluginKit.xcconfig create mode 100644 TableProTests/Core/Plugins/BundledPluginStampTests.swift diff --git a/.claude/rules/plugin-system.md b/.claude/rules/plugin-system.md index de573eb205..0d52c96e8a 100644 --- a/.claude/rules/plugin-system.md +++ b/.claude/rules/plugin-system.md @@ -24,9 +24,10 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke - Add a requirement only with a default implementation, then bridge it in `PluginDriverAdapter`. - Additive: a new requirement with a default, a new field on a non-`@frozen` struct, a new case on a non-`@frozen` enum, a reordering. - Breaking: a changed, removed or renamed requirement or signature, a requirement without a default, a new case on a `@frozen` enum, a change to a frozen type's layout. -- Every ABI change bumps `currentPluginKitVersion` (in `PluginManager.swift`) and `TableProPluginKitVersion` in every plugin `Info.plist`, at most once per release cycle: later changes in the cycle reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. +- Every ABI change bumps `TABLEPRO_PLUGINKIT_VERSION` in `Configs/PluginKit.xcconfig`, at most once per release cycle: later changes in the cycle reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. +- The build stamps that value into `TableProPluginKitVersion` of the app and of every plugin. A plugin's `Info.plist` holds `TABLEPRO_PLUGINKIT_VERSION` and is preprocessed, because every shipped app reads the key with `as? Int`; the source file therefore fails `plutil -lint`. - An additive change re-publishes nothing. Built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. -- A breaking change also raises `minimumCompatiblePluginKitVersion`, and every registry plugin is re-released with `scripts/release-all-plugins.sh ` before the app tag, because the release's `Registry Readiness` job fails while a registry driver has no binary in the app's kit range (`RELEASING.md`, The release pipeline). +- A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`, and every registry plugin is re-released with `scripts/release-all-plugins.sh ` before the app tag, because the release's `Registry Readiness` job fails while a registry driver has no binary in the app's kit range (`RELEASING.md`, The release pipeline). - Check with `verify.sh abi ` (`scripts/check-pluginkit-abi.sh`). It builds PluginKit at HEAD and at the base with the same toolchain and diffs the public `.swiftinterface`. A base with no interface passes; a diff prints the additive and breaking rules and exits 1. - It refuses uncommitted changes, so run it after the commit and before the push. No workflow runs it. - The diff is textual, so a `-` line can be a declaration that moved. Only a removed line with no matching added line, or a requirement added without a default, is breaking. This lists the removed lines that do not come back: diff --git a/.github/workflows/build-plugin.yml b/.github/workflows/build-plugin.yml index 6dfcfff2ea..e36897cfaa 100644 --- a/.github/workflows/build-plugin.yml +++ b/.github/workflows/build-plugin.yml @@ -8,7 +8,7 @@ on: tags: description: > Comma-separated plugin tag:pluginKitVersion pairs. - pluginKitVersion defaults to currentPluginKitVersion from PluginManager.swift. + pluginKitVersion defaults to TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig. Examples: plugin-mongodb-v1.0.25, plugin-mongodb-v1.0.25:13 required: true type: string @@ -21,7 +21,7 @@ on: required: false type: string -# Declared per job rather than here. resolve-tags reads two integers out of a Swift file and needs +# Declared per job rather than here. resolve-tags reads an integer out of an xcconfig and needs # nothing, and a workflow-level grant hands it a write token it has no use for. permissions: contents: read @@ -66,18 +66,17 @@ jobs: - name: Check the plugin manifest against the plugin classes run: python3 scripts/ci/check-plugin-manifest.py - - name: Read currentPluginKitVersion + - name: Read the PluginKit version id: pkv run: | - VERSION=$(grep -E 'static let currentPluginKitVersion\s*=\s*[0-9]+' \ - TablePro/Core/Plugins/PluginManager.swift \ - | grep -oE '[0-9]+$' | head -1) + VERSION=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' \ + Configs/PluginKit.xcconfig) if [ -z "$VERSION" ]; then - echo "::error::Could not parse currentPluginKitVersion from PluginManager.swift" + echo "::error::Could not parse TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig" exit 1 fi echo "version=$VERSION" >> "$GITHUB_OUTPUT" - echo "currentPluginKitVersion=$VERSION" + echo "TABLEPRO_PLUGINKIT_VERSION=$VERSION" - name: Build matrix id: tags diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index f77343fafc..14e249bfa2 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -185,7 +185,7 @@ jobs: build/Release/TablePro-*.dmg build/Release/TablePro-*.tar.xz - # Reads two integers out of a Swift file and fetches a JSON manifest, so it wants nothing + # Reads two integers out of an xcconfig and fetches a JSON manifest, so it wants nothing # from macOS and does not need to spend a macOS runner minute (billed at ten times Linux). registry-readiness: name: Registry Readiness @@ -199,9 +199,9 @@ jobs: - name: Verify registry has compatible plugin binaries run: | - MANAGER="TablePro/Core/Plugins/PluginManager.swift" - CURRENT=$(grep -E 'static let currentPluginKitVersion = ' "$MANAGER" | grep -oE '[0-9]+' | head -1) - FLOOR=$(grep -E 'static let minimumCompatiblePluginKitVersion = ' "$MANAGER" | grep -oE '[0-9]+' | head -1) + CONFIG="Configs/PluginKit.xcconfig" + CURRENT=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") + FLOOR=$(sed -nE 's/^TABLEPRO_PLUGINKIT_MINIMUM_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") echo "PluginKit floor=$FLOOR current=$CURRENT" python3 scripts/check-registry-readiness.py --floor "$FLOOR" --current "$CURRENT" diff --git a/Configs/Base.xcconfig b/Configs/Base.xcconfig index 3f39b27db6..323f91bffe 100644 --- a/Configs/Base.xcconfig +++ b/Configs/Base.xcconfig @@ -2,6 +2,8 @@ // Structure rule: project-wide and per-configuration settings live in Configs/*.xcconfig. // Per-target settings live in project.yml, where the target templates are. +#include "PluginKit.xcconfig" + ALWAYS_SEARCH_USER_PATHS = NO ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS = YES CLANG_ANALYZER_NONNULL = YES @@ -60,10 +62,9 @@ STRINGS_FILE_OUTPUT_ENCODING = binary SWIFT_VERSION = 6.0 CODE_SIGN_STYLE = Automatic -// Signing knobs. Only the two app targets read them, so plugin bundles and the helper -// tool sign ad-hoc and a contributor without the official team can still build everything. -// A contributor overrides these in Configs/Secrets.xcconfig without touching a tracked file, -// and the release scripts pass their own values on the xcodebuild command line. +// Signing knobs. The app targets and the plugin bundles sign with this team; the helper tool +// signs ad hoc. A contributor overrides these in Configs/Secrets.xcconfig without touching a +// tracked file, and the release scripts pass their own values on the xcodebuild command line. TABLEPRO_DEVELOPMENT_TEAM = D7HJ5TFYCU TABLEPRO_APP_BUNDLE_IDENTIFIER = com.TablePro diff --git a/Configs/PluginKit.xcconfig b/Configs/PluginKit.xcconfig new file mode 100644 index 0000000000..9f5ac3b3d0 --- /dev/null +++ b/Configs/PluginKit.xcconfig @@ -0,0 +1,3 @@ +// When to raise each, and how the build stamps them: .claude/rules/plugin-system.md, ABI. +TABLEPRO_PLUGINKIT_VERSION = 39 +TABLEPRO_PLUGINKIT_MINIMUM_VERSION = 19 diff --git a/Plugins/BeancountDriverPlugin/Info.plist b/Plugins/BeancountDriverPlugin/Info.plist index 5eb0eeab09..8bb97777cc 100644 --- a/Plugins/BeancountDriverPlugin/Info.plist +++ b/Plugins/BeancountDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Beancount diff --git a/Plugins/BigQueryDriverPlugin/Info.plist b/Plugins/BigQueryDriverPlugin/Info.plist index 1f46e12387..2191e071b0 100644 --- a/Plugins/BigQueryDriverPlugin/Info.plist +++ b/Plugins/BigQueryDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/BigtableDriverPlugin/Info.plist b/Plugins/BigtableDriverPlugin/Info.plist index df5147fef9..cab40835a2 100644 --- a/Plugins/BigtableDriverPlugin/Info.plist +++ b/Plugins/BigtableDriverPlugin/Info.plist @@ -5,7 +5,7 @@ TableProMinAppVersion 0.81.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Bigtable diff --git a/Plugins/CSVExportPlugin/Info.plist b/Plugins/CSVExportPlugin/Info.plist index bc844bd4ea..81a04239c9 100644 --- a/Plugins/CSVExportPlugin/Info.plist +++ b/Plugins/CSVExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds csv diff --git a/Plugins/CSVImportPlugin/Info.plist b/Plugins/CSVImportPlugin/Info.plist index c92042694f..a09c94dbca 100644 --- a/Plugins/CSVImportPlugin/Info.plist +++ b/Plugins/CSVImportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesImportFormatIds csv diff --git a/Plugins/CassandraDriverPlugin/Info.plist b/Plugins/CassandraDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/CassandraDriverPlugin/Info.plist +++ b/Plugins/CassandraDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/ClickHouseDriverPlugin/Info.plist b/Plugins/ClickHouseDriverPlugin/Info.plist index 5189083c91..10fa22e650 100644 --- a/Plugins/ClickHouseDriverPlugin/Info.plist +++ b/Plugins/ClickHouseDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds ClickHouse diff --git a/Plugins/CloudflareD1DriverPlugin/Info.plist b/Plugins/CloudflareD1DriverPlugin/Info.plist index 1f46e12387..2191e071b0 100644 --- a/Plugins/CloudflareD1DriverPlugin/Info.plist +++ b/Plugins/CloudflareD1DriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/CloudflareR2SQLDriverPlugin/Info.plist b/Plugins/CloudflareR2SQLDriverPlugin/Info.plist index edd28fb2d7..9625b89836 100644 --- a/Plugins/CloudflareR2SQLDriverPlugin/Info.plist +++ b/Plugins/CloudflareR2SQLDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Cloudflare R2 SQL diff --git a/Plugins/DamengDriverPlugin/Info.plist b/Plugins/DamengDriverPlugin/Info.plist index 767f6b3e09..024e929450 100644 --- a/Plugins/DamengDriverPlugin/Info.plist +++ b/Plugins/DamengDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Dameng diff --git a/Plugins/DuckDBDriverPlugin/Info.plist b/Plugins/DuckDBDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/DuckDBDriverPlugin/Info.plist +++ b/Plugins/DuckDBDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/DynamoDBDriverPlugin/Info.plist b/Plugins/DynamoDBDriverPlugin/Info.plist index 1f46e12387..2191e071b0 100644 --- a/Plugins/DynamoDBDriverPlugin/Info.plist +++ b/Plugins/DynamoDBDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/ElasticsearchDriverPlugin/Info.plist b/Plugins/ElasticsearchDriverPlugin/Info.plist index 302d83a253..87e95599df 100644 --- a/Plugins/ElasticsearchDriverPlugin/Info.plist +++ b/Plugins/ElasticsearchDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.53.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/EtcdDriverPlugin/Info.plist b/Plugins/EtcdDriverPlugin/Info.plist index 1f46e12387..2191e071b0 100644 --- a/Plugins/EtcdDriverPlugin/Info.plist +++ b/Plugins/EtcdDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/FirebaseDriverPlugin/Info.plist b/Plugins/FirebaseDriverPlugin/Info.plist index 7862226b67..8ad5854eb9 100644 --- a/Plugins/FirebaseDriverPlugin/Info.plist +++ b/Plugins/FirebaseDriverPlugin/Info.plist @@ -5,7 +5,7 @@ TableProMinAppVersion 0.80.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Firestore diff --git a/Plugins/HanaDriverPlugin/Info.plist b/Plugins/HanaDriverPlugin/Info.plist index 2494774fd1..37b21f2e8d 100644 --- a/Plugins/HanaDriverPlugin/Info.plist +++ b/Plugins/HanaDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds SAP HANA diff --git a/Plugins/JSONExportPlugin/Info.plist b/Plugins/JSONExportPlugin/Info.plist index ceea251c9b..4077dfde3a 100644 --- a/Plugins/JSONExportPlugin/Info.plist +++ b/Plugins/JSONExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds json diff --git a/Plugins/JSONImportPlugin/Info.plist b/Plugins/JSONImportPlugin/Info.plist index f8fb2ead03..d0beb83840 100644 --- a/Plugins/JSONImportPlugin/Info.plist +++ b/Plugins/JSONImportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesImportFormatIds json diff --git a/Plugins/KafkaDriverPlugin/Info.plist b/Plugins/KafkaDriverPlugin/Info.plist index 06a6c6dba0..2f105003d2 100644 --- a/Plugins/KafkaDriverPlugin/Info.plist +++ b/Plugins/KafkaDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Kafka diff --git a/Plugins/LibSQLDriverPlugin/Info.plist b/Plugins/LibSQLDriverPlugin/Info.plist index 1f46e12387..2191e071b0 100644 --- a/Plugins/LibSQLDriverPlugin/Info.plist +++ b/Plugins/LibSQLDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/MQLExportPlugin/Info.plist b/Plugins/MQLExportPlugin/Info.plist index 29ea7006fe..7cc82fecc2 100644 --- a/Plugins/MQLExportPlugin/Info.plist +++ b/Plugins/MQLExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds mql diff --git a/Plugins/MSSQLDriverPlugin/Info.plist b/Plugins/MSSQLDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/MSSQLDriverPlugin/Info.plist +++ b/Plugins/MSSQLDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/MarkdownExportPlugin/Info.plist b/Plugins/MarkdownExportPlugin/Info.plist index cb1c7d34de..9d9b33d127 100644 --- a/Plugins/MarkdownExportPlugin/Info.plist +++ b/Plugins/MarkdownExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds md diff --git a/Plugins/MongoDBDriverPlugin/Info.plist b/Plugins/MongoDBDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/MongoDBDriverPlugin/Info.plist +++ b/Plugins/MongoDBDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/MySQLDriverPlugin/Info.plist b/Plugins/MySQLDriverPlugin/Info.plist index 2cca4834c1..e736a2c92e 100644 --- a/Plugins/MySQLDriverPlugin/Info.plist +++ b/Plugins/MySQLDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds MySQL diff --git a/Plugins/OracleDriverPlugin/Info.plist b/Plugins/OracleDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/OracleDriverPlugin/Info.plist +++ b/Plugins/OracleDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/ParquetExportPlugin/Info.plist b/Plugins/ParquetExportPlugin/Info.plist index 0d5a128b54..758650d5c3 100644 --- a/Plugins/ParquetExportPlugin/Info.plist +++ b/Plugins/ParquetExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds parquet diff --git a/Plugins/PostgreSQLDriverPlugin/Info.plist b/Plugins/PostgreSQLDriverPlugin/Info.plist index b118efa2ba..27c87e63de 100644 --- a/Plugins/PostgreSQLDriverPlugin/Info.plist +++ b/Plugins/PostgreSQLDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds PostgreSQL diff --git a/Plugins/RedisDriverPlugin/Info.plist b/Plugins/RedisDriverPlugin/Info.plist index e2085f42ec..3bb3e5f4b7 100644 --- a/Plugins/RedisDriverPlugin/Info.plist +++ b/Plugins/RedisDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Redis diff --git a/Plugins/SQLExportPlugin/Info.plist b/Plugins/SQLExportPlugin/Info.plist index 279c69fe5a..fd6dcc5ec1 100644 --- a/Plugins/SQLExportPlugin/Info.plist +++ b/Plugins/SQLExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds sql diff --git a/Plugins/SQLImportPlugin/Info.plist b/Plugins/SQLImportPlugin/Info.plist index 369a47df4b..262e0c28d1 100644 --- a/Plugins/SQLImportPlugin/Info.plist +++ b/Plugins/SQLImportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesImportFormatIds sql diff --git a/Plugins/SQLiteDriverPlugin/Info.plist b/Plugins/SQLiteDriverPlugin/Info.plist index 56a956e9dd..5354fb9599 100644 --- a/Plugins/SQLiteDriverPlugin/Info.plist +++ b/Plugins/SQLiteDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds SQLite diff --git a/Plugins/SnowflakeDriverPlugin/Info.plist b/Plugins/SnowflakeDriverPlugin/Info.plist index ae2952b33f..3d429e5f55 100644 --- a/Plugins/SnowflakeDriverPlugin/Info.plist +++ b/Plugins/SnowflakeDriverPlugin/Info.plist @@ -5,6 +5,6 @@ TableProMinAppVersion 0.48.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/SpannerDriverPlugin/Info.plist b/Plugins/SpannerDriverPlugin/Info.plist index 07dfe685ea..bf282d67ff 100644 --- a/Plugins/SpannerDriverPlugin/Info.plist +++ b/Plugins/SpannerDriverPlugin/Info.plist @@ -5,7 +5,7 @@ TableProMinAppVersion 0.42.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Spanner diff --git a/Plugins/SurrealDBDriverPlugin/Info.plist b/Plugins/SurrealDBDriverPlugin/Info.plist index acb61d7be8..34c7f92701 100644 --- a/Plugins/SurrealDBDriverPlugin/Info.plist +++ b/Plugins/SurrealDBDriverPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds SurrealDB diff --git a/Plugins/TeradataDriverPlugin/Info.plist b/Plugins/TeradataDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/TeradataDriverPlugin/Info.plist +++ b/Plugins/TeradataDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/TrinoDriverPlugin/Info.plist b/Plugins/TrinoDriverPlugin/Info.plist index 88fdb3660c..ad014bef07 100644 --- a/Plugins/TrinoDriverPlugin/Info.plist +++ b/Plugins/TrinoDriverPlugin/Info.plist @@ -3,6 +3,6 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION diff --git a/Plugins/TypesenseDriverPlugin/Info.plist b/Plugins/TypesenseDriverPlugin/Info.plist index dca18efec4..67c295cfaa 100644 --- a/Plugins/TypesenseDriverPlugin/Info.plist +++ b/Plugins/TypesenseDriverPlugin/Info.plist @@ -5,7 +5,7 @@ TableProMinAppVersion 0.73.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Typesense diff --git a/Plugins/WeaviateDriverPlugin/Info.plist b/Plugins/WeaviateDriverPlugin/Info.plist index 28579a91f5..2f8c78f6e0 100644 --- a/Plugins/WeaviateDriverPlugin/Info.plist +++ b/Plugins/WeaviateDriverPlugin/Info.plist @@ -5,7 +5,7 @@ TableProMinAppVersion 0.73.0 TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesDatabaseTypeIds Weaviate diff --git a/Plugins/XLSXExportPlugin/Info.plist b/Plugins/XLSXExportPlugin/Info.plist index 1f676a78e2..661aa2ce9e 100644 --- a/Plugins/XLSXExportPlugin/Info.plist +++ b/Plugins/XLSXExportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesExportFormatIds xlsx diff --git a/Plugins/XLSXImportPlugin/Info.plist b/Plugins/XLSXImportPlugin/Info.plist index 5407e53283..a87e8e8ac3 100644 --- a/Plugins/XLSXImportPlugin/Info.plist +++ b/Plugins/XLSXImportPlugin/Info.plist @@ -3,7 +3,7 @@ TableProPluginKitVersion - 39 + TABLEPRO_PLUGINKIT_VERSION TableProProvidesImportFormatIds xlsx diff --git a/TablePro/Core/Plugins/PluginManager.swift b/TablePro/Core/Plugins/PluginManager.swift index 3341bb3898..7c83409601 100644 --- a/TablePro/Core/Plugins/PluginManager.swift +++ b/TablePro/Core/Plugins/PluginManager.swift @@ -9,9 +9,19 @@ import TableProPluginKit @MainActor final class PluginManager: ObservableObject { static let shared = PluginManager(userDefaults: AppStorageEnvironment.shared.defaults) - nonisolated static let currentPluginKitVersion = 39 + nonisolated static let currentPluginKitVersion = pluginKitVersion(infoKey: "TableProPluginKitVersion") - nonisolated static let minimumCompatiblePluginKitVersion = 19 + nonisolated static let minimumCompatiblePluginKitVersion = pluginKitVersion( + infoKey: "TableProPluginKitMinimumVersion" + ) + + nonisolated private static func pluginKitVersion(infoKey: String) -> Int { + guard let value = Bundle.main.object(forInfoDictionaryKey: infoKey) as? String, + let version = Int(value) else { + preconditionFailure("Info.plist has no integer \(infoKey); Configs/PluginKit.xcconfig sets it") + } + return version + } private let builtInPluginsURL: URL? internal let userPluginsDir: URL diff --git a/TablePro/Info.plist b/TablePro/Info.plist index 376238ace1..d34f4b71b4 100644 --- a/TablePro/Info.plist +++ b/TablePro/Info.plist @@ -20,6 +20,10 @@ 86400 SUAutomaticallyUpdate + TableProPluginKitVersion + $(TABLEPRO_PLUGINKIT_VERSION) + TableProPluginKitMinimumVersion + $(TABLEPRO_PLUGINKIT_MINIMUM_VERSION) CFBundleDocumentTypes diff --git a/TableProTests/Core/Plugins/BundledPluginStampTests.swift b/TableProTests/Core/Plugins/BundledPluginStampTests.swift new file mode 100644 index 0000000000..77dbc8b7cd --- /dev/null +++ b/TableProTests/Core/Plugins/BundledPluginStampTests.swift @@ -0,0 +1,41 @@ +import Foundation +@testable import TablePro +import Testing + +struct BundledPluginStampTests { + private struct StampedPlugin { + let name: String + let info: [String: Any] + } + + private static func bundledPlugins() throws -> [StampedPlugin] { + let plugIns = try #require(Bundle.main.builtInPlugInsURL) + let bundles = try FileManager.default.contentsOfDirectory(at: plugIns, includingPropertiesForKeys: nil) + .filter { $0.pathExtension == "tableplugin" } + return try bundles.map { bundle in + let data = try Data(contentsOf: bundle.appendingPathComponent("Contents/Info.plist")) + let plist = try PropertyListSerialization.propertyList(from: data, format: nil) + let info = try #require(plist as? [String: Any], "\(bundle.lastPathComponent) has no Info.plist dictionary") + return StampedPlugin(name: bundle.lastPathComponent, info: info) + } + } + + @Test("Every bundled plugin declares the app's PluginKit version as an integer") + func bundledPluginsCarryTheAppKit() throws { + let plugins = try Self.bundledPlugins() + #expect(!plugins.isEmpty) + for plugin in plugins { + let declared = plugin.info["TableProPluginKitVersion"] + #expect( + declared as? Int == PluginManager.currentPluginKitVersion, + "\(plugin.name) declares \(String(describing: declared)), the app is \(PluginManager.currentPluginKitVersion)" + ) + } + } + + @Test("The app's PluginKit range is stamped from the build setting") + func appKitRangeIsOrdered() { + #expect(PluginManager.minimumCompatiblePluginKitVersion > 0) + #expect(PluginManager.minimumCompatiblePluginKitVersion <= PluginManager.currentPluginKitVersion) + } +} diff --git a/project.yml b/project.yml index fd3efe4dfc..bc75aa5a70 100644 --- a/project.yml +++ b/project.yml @@ -96,9 +96,14 @@ targetTemplates: base: COMBINE_HIDPI_IMAGES: YES CURRENT_PROJECT_VERSION: 1 + DEVELOPMENT_TEAM: $(TABLEPRO_DEVELOPMENT_TEAM) + ENABLE_HARDENED_RUNTIME: YES GENERATE_INFOPLIST_FILE: YES INFOPLIST_FILE: Plugins/${folder}/Info.plist INFOPLIST_KEY_NSPrincipalClass: $(PRODUCT_MODULE_NAME).${principalClass} + # Keeps TableProPluginKitVersion an : .claude/rules/plugin-system.md, ABI. + INFOPLIST_PREPROCESS: YES + INFOPLIST_PREPROCESSOR_DEFINITIONS: TABLEPRO_PLUGINKIT_VERSION=$(TABLEPRO_PLUGINKIT_VERSION) LD_RUNPATH_SEARCH_PATHS: - $(inherited) - "@executable_path/../Frameworks" @@ -115,6 +120,9 @@ targetTemplates: SWIFT_APPROACHABLE_CONCURRENCY: YES SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY: YES WRAPPER_EXTENSION: tableplugin + configs: + Release: + DEPLOYMENT_POSTPROCESSING: YES # A bundled plugin's folder compiled again with its tests and no host app. It links the same # TableProPluginKit framework the plugin does: a package product that depended on the package's @@ -134,6 +142,8 @@ targetTemplates: settings: base: GENERATE_INFOPLIST_FILE: YES + # Set, or XcodeGen picks the plugin folder's Info.plist, which only the plugin preprocesses. + INFOPLIST_FILE: "" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.${target_name} STRING_CATALOG_GENERATE_SYMBOLS: NO SWIFT_APPROACHABLE_CONCURRENCY: YES @@ -841,6 +851,8 @@ targets: settings: base: GENERATE_INFOPLIST_FILE: YES + # Set for the reason the PluginTests template gives. + INFOPLIST_FILE: "" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.FormatPluginTests STRING_CATALOG_GENERATE_SYMBOLS: NO SWIFT_APPROACHABLE_CONCURRENCY: YES diff --git a/scripts/ci/check-pluginkit-bump-cadence.py b/scripts/ci/check-pluginkit-bump-cadence.py index 335a7d141b..c05a33d55c 100644 --- a/scripts/ci/check-pluginkit-bump-cadence.py +++ b/scripts/ci/check-pluginkit-bump-cadence.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Fail when currentPluginKitVersion has moved more than one past the newest release tag. +"""Fail when TABLEPRO_PLUGINKIT_VERSION has moved more than one past the newest release tag. Only the value at release time ever ships, so every PluginKit change in one release cycle reuses the number the first one took. The registry keeps binaries for the three newest kit versions, and @@ -10,7 +10,8 @@ The newest release is the highest vX.Y.Z tag by version number. With --remote the tag list comes from that remote and the one tag is fetched when it is missing, so a shallow CI checkout with no tags works. The fetch is at depth 1 only in a checkout that is already shallow: in a full clone it would -cut the history every worktree shares down to that one commit. +cut the history every worktree shares down to that one commit. A tag cut before +Configs/PluginKit.xcconfig existed declares the kit in PluginManager.swift. test_check_pluginkit_bump_cadence.py pins what it must and must not report. """ @@ -23,8 +24,10 @@ from pathlib import Path ROOT = Path(__file__).resolve().parents[2] -PLUGIN_MANAGER = "TablePro/Core/Plugins/PluginManager.swift" -KIT_DECLARATION = re.compile(r"\bstatic\s+let\s+currentPluginKitVersion\s*=\s*(\d+)\b") +KIT_CONFIG = "Configs/PluginKit.xcconfig" +KIT_SETTING = re.compile(r"^TABLEPRO_PLUGINKIT_VERSION\s*=\s*(\d+)\s*$", re.MULTILINE) +LEGACY_PLUGIN_MANAGER = "TablePro/Core/Plugins/PluginManager.swift" +LEGACY_DECLARATION = re.compile(r"\bstatic\s+let\s+currentPluginKitVersion\s*=\s*(\d+)\b") RELEASE_TAG = re.compile(r"^v(\d+)\.(\d+)\.(\d+)$") @@ -35,13 +38,20 @@ def git(repo: Path, *args: str) -> str: return result.stdout -def kit_version(source: str, origin: str) -> int: - match = KIT_DECLARATION.search(source) +def kit_version(source: str, origin: str, pattern: re.Pattern[str] = KIT_SETTING) -> int: + match = pattern.search(source) if match is None: - raise SystemExit(f"ERROR: no currentPluginKitVersion declaration in {origin}") + raise SystemExit(f"ERROR: no PluginKit version in {origin}") return int(match.group(1)) +def released_kit_version(repo: Path, tag: str) -> int: + for path, pattern in ((KIT_CONFIG, KIT_SETTING), (LEGACY_PLUGIN_MANAGER, LEGACY_DECLARATION)): + if has_object(repo, f"{tag}:{path}"): + return kit_version(git(repo, "show", f"{tag}:{path}"), f"{tag}:{path}", pattern) + raise SystemExit(f"ERROR: {tag} has neither {KIT_CONFIG} nor {LEGACY_PLUGIN_MANAGER}") + + def newest_release_tag(names: list[str]) -> str | None: releases = [ (tuple(int(part) for part in match.groups()), name) @@ -60,16 +70,16 @@ def remote_tag_names(repo: Path, remote: str) -> list[str]: return [line.split("refs/tags/", 1)[1] for line in output.splitlines() if "refs/tags/" in line] -def has_commit(repo: Path, ref: str) -> bool: +def has_object(repo: Path, ref: str) -> bool: result = subprocess.run( - ["git", "-C", str(repo), "rev-parse", "--quiet", "--verify", f"{ref}^{{commit}}"], + ["git", "-C", str(repo), "rev-parse", "--quiet", "--verify", ref], capture_output=True, ) return result.returncode == 0 def fetch_tag(repo: Path, remote: str, tag: str) -> None: - if has_commit(repo, f"refs/tags/{tag}"): + if has_object(repo, f"refs/tags/{tag}^{{commit}}"): return shallow = git(repo, "rev-parse", "--is-shallow-repository").strip() == "true" depth = ["--depth=1"] if shallow else [] @@ -80,7 +90,7 @@ def cadence_violation(tag: str, released: int, current: int) -> str | None: if current <= released + 1: return None return ( - f"currentPluginKitVersion is {current}, but {tag} shipped {released}. It moves at most once " + f"TABLEPRO_PLUGINKIT_VERSION is {current}, but {tag} shipped {released}. It moves at most once " f"per release cycle, so every PluginKit change until the next release reuses {released + 1}." ) @@ -94,19 +104,18 @@ def main(argv: list[str] | None = None) -> int: names = remote_tag_names(args.repo, args.remote) if args.remote else local_tag_names(args.repo) tag = newest_release_tag(names) if tag is None: - raise SystemExit("ERROR: no vX.Y.Z release tag to compare currentPluginKitVersion against") + raise SystemExit("ERROR: no vX.Y.Z release tag to compare TABLEPRO_PLUGINKIT_VERSION against") if args.remote: fetch_tag(args.repo, args.remote, tag) - released = kit_version(git(args.repo, "show", f"{tag}:{PLUGIN_MANAGER}"), f"{tag}:{PLUGIN_MANAGER}") - current_path = args.repo / PLUGIN_MANAGER - current = kit_version(current_path.read_text(encoding="utf-8"), PLUGIN_MANAGER) + released = released_kit_version(args.repo, tag) + current = kit_version((args.repo / KIT_CONFIG).read_text(encoding="utf-8"), KIT_CONFIG) violation = cadence_violation(tag, released, current) if violation: print(violation, file=sys.stderr) return 1 - print(f"currentPluginKitVersion {current}; {tag} shipped {released}.") + print(f"TABLEPRO_PLUGINKIT_VERSION {current}; {tag} shipped {released}.") return 0 diff --git a/scripts/ci/test_check_pluginkit_bump_cadence.py b/scripts/ci/test_check_pluginkit_bump_cadence.py index dc64c7f80b..add50f8b36 100644 --- a/scripts/ci/test_check_pluginkit_bump_cadence.py +++ b/scripts/ci/test_check_pluginkit_bump_cadence.py @@ -11,7 +11,8 @@ from pathlib import Path SCRIPT = Path(__file__).with_name("check-pluginkit-bump-cadence.py") -PLUGIN_MANAGER = Path("TablePro/Core/Plugins/PluginManager.swift") +KIT_CONFIG = Path("Configs/PluginKit.xcconfig") +LEGACY_PLUGIN_MANAGER = Path("TablePro/Core/Plugins/PluginManager.swift") GIT_ENV = { **os.environ, @@ -24,7 +25,11 @@ } -def plugin_manager_source(kit: int) -> str: +def kit_config_source(kit: int) -> str: + return f"TABLEPRO_PLUGINKIT_VERSION = {kit}\nTABLEPRO_PLUGINKIT_MINIMUM_VERSION = 19\n" + + +def legacy_plugin_manager_source(kit: int) -> str: return ( "final class PluginManager {\n" f" nonisolated static let currentPluginKitVersion = {kit}\n" @@ -46,14 +51,21 @@ def git(self, *args: str) -> str: text=True, ).stdout - def write_kit(self, kit: int) -> None: - target = self.path / PLUGIN_MANAGER + def write(self, relative: Path, text: str) -> None: + target = self.path / relative target.parent.mkdir(parents=True, exist_ok=True) - target.write_text(plugin_manager_source(kit), encoding="utf-8") + target.write_text(text, encoding="utf-8") - def commit_kit(self, kit: int, tag: str | None = None) -> None: - self.write_kit(kit) - self.git("add", str(PLUGIN_MANAGER)) + def write_kit(self, kit: int) -> None: + self.write(KIT_CONFIG, kit_config_source(kit)) + + def commit_kit(self, kit: int, tag: str | None = None, legacy: bool = False) -> None: + if legacy: + self.write(LEGACY_PLUGIN_MANAGER, legacy_plugin_manager_source(kit)) + self.git("add", str(LEGACY_PLUGIN_MANAGER)) + else: + self.write_kit(kit) + self.git("add", str(KIT_CONFIG)) self.git("commit", "--quiet", "--allow-empty", "-m", f"kit {kit}") if tag: self.git("tag", tag) @@ -126,8 +138,28 @@ def test_a_repository_without_a_release_tag_fails(self) -> None: def test_a_missing_declaration_fails(self) -> None: self.repo.commit_kit(30, tag="v0.74.0") - (self.repo.path / PLUGIN_MANAGER).write_text("final class PluginManager {}\n", encoding="utf-8") - self.assertFails(self.repo.run_check(), "currentPluginKitVersion") + self.repo.write(KIT_CONFIG, "TABLEPRO_PLUGINKIT_MINIMUM_VERSION = 19\n") + self.assertFails(self.repo.run_check(), str(KIT_CONFIG)) + + def test_a_release_before_the_xcconfig_is_read_from_swift(self) -> None: + self.repo.commit_kit(39, tag="v0.81.0", legacy=True) + self.repo.commit_kit(40) + self.assertPasses(self.repo.run_check()) + + self.repo.write_kit(41) + self.assertFails(self.repo.run_check(), "41", "v0.81.0", "39") + + def test_the_xcconfig_wins_over_a_leftover_swift_literal(self) -> None: + self.repo.commit_kit(30, legacy=True) + self.repo.commit_kit(39, tag="v0.82.0") + self.repo.write_kit(40) + self.assertPasses(self.repo.run_check()) + + def test_a_release_with_no_kit_fails(self) -> None: + self.repo.git("commit", "--quiet", "--allow-empty", "-m", "empty") + self.repo.git("tag", "v0.74.0") + self.repo.write_kit(30) + self.assertFails(self.repo.run_check(), "v0.74.0", str(KIT_CONFIG)) def test_a_shallow_clone_reads_the_release_from_its_remote(self) -> None: self.repo.commit_kit(25, tag="v0.73.0") diff --git a/scripts/release-plugin-for-shipped-app.sh b/scripts/release-plugin-for-shipped-app.sh index 7290e0d258..930b97c849 100755 --- a/scripts/release-plugin-for-shipped-app.sh +++ b/scripts/release-plugin-for-shipped-app.sh @@ -66,10 +66,16 @@ if ! git -C "$REPO_ROOT" rev-parse --verify "$APP_TAG^{commit}" >/dev/null 2>&1; exit 1 fi -KIT=$(git -C "$REPO_ROOT" show "$APP_TAG:TablePro/Core/Plugins/PluginManager.swift" \ - | sed -n 's/.*currentPluginKitVersion = \([0-9]*\).*/\1/p' | head -1) +if git -C "$REPO_ROOT" cat-file -e "$APP_TAG:Configs/PluginKit.xcconfig" 2>/dev/null; then + KIT=$(git -C "$REPO_ROOT" show "$APP_TAG:Configs/PluginKit.xcconfig" \ + | sed -n 's/^TABLEPRO_PLUGINKIT_VERSION *= *\([0-9][0-9]*\) *$/\1/p') +else + # A tag cut before Configs/PluginKit.xcconfig existed declares the kit in Swift. + KIT=$(git -C "$REPO_ROOT" show "$APP_TAG:TablePro/Core/Plugins/PluginManager.swift" \ + | sed -n 's/.*currentPluginKitVersion = \([0-9]*\).*/\1/p' | head -1) +fi if [ -z "$KIT" ]; then - echo "error: could not read currentPluginKitVersion out of $APP_TAG" >&2 + echo "error: could not read the PluginKit version out of $APP_TAG" >&2 exit 1 fi From ae54f22907d0ab305a076882ec4523e13679e857 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 04:31:24 +0700 Subject: [PATCH 02/10] build(release): add the feed, registry and GitHub release tooling --- .claude/rules/ci.md | 3 +- .github/actions/setup-sparkle/action.yml | 51 +++ .github/workflows/repo-hygiene.yml | 3 - project.yml | 3 +- scripts/ci/check-sparkle-version.py | 139 ------ scripts/release/feed.py | 521 ++++++++++++++++++++++ scripts/release/publish.py | 224 ++++++++++ scripts/release/registry.py | 337 ++++++++++++++ scripts/release/test_feed.py | 531 +++++++++++++++++++++++ scripts/release/test_publish.py | 299 +++++++++++++ scripts/release/test_registry.py | 327 ++++++++++++++ 11 files changed, 2294 insertions(+), 144 deletions(-) create mode 100644 .github/actions/setup-sparkle/action.yml delete mode 100755 scripts/ci/check-sparkle-version.py create mode 100755 scripts/release/feed.py create mode 100755 scripts/release/publish.py create mode 100755 scripts/release/registry.py create mode 100644 scripts/release/test_feed.py create mode 100644 scripts/release/test_publish.py create mode 100644 scripts/release/test_registry.py diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 0e588c7bdc..25290306ae 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -27,6 +27,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. `build-plugin.yml` picks its runner from the ref it builds (`RELEASING.md`). - The Xcode version is written once, in `.github/actions/setup-xcode/action.yml`, which runs `xcode-select --switch` and prints `xcodebuild -version`. A workflow never selects Xcode another way. - XcodeGen comes from `.github/actions/setup-xcodegen`, pinned by version and SHA-256. +- Sparkle's `generate_appcast` and `sign_update` come from `.github/actions/setup-sparkle`. It reads the version and revision from the app's `Package.resolved` (`project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. - The compilation cache is off on CI (`COMPILATION_CACHE_ENABLE_CACHING=NO`); `Configs/Base.xcconfig` turns it on locally. ## macOS Tests (`macos-tests.yml`) @@ -88,7 +89,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - `check-lib-sourcing.py`, `check-plugin-manifest.py`, `check-deployment-target.sh`, `check-ios-shared-isolation.py`; - `localization.py plugins` and `plugins --target ios`; - `check-string-catalogs.py`, `check-license-inventory.py`, `check-entitlements.py`, `check-test-plans.py`; - - `check-pluginkit-bump-cadence.py`, `check-sparkle-version.py`; + - `check-pluginkit-bump-cadence.py`; - `scripts/audit-refactor-health.sh --check`, `check-hub-ratchet.py`; - the Go checks on the HANA helper, when the `hana` detector is true. - `audit-refactor-health.sh --check` fails a file defining a production `DatabaseType` outside `Packages/TableProCore/Sources/TableProCoreTypes/DatabaseType.swift`, unless `.github/duplicate-contract-baseline.txt` lists its path. Moving a listed file moves its baseline path in the same commit. Never add an entry to let a new copy pass. diff --git a/.github/actions/setup-sparkle/action.yml b/.github/actions/setup-sparkle/action.yml new file mode 100644 index 0000000000..41c936027e --- /dev/null +++ b/.github/actions/setup-sparkle/action.yml @@ -0,0 +1,51 @@ +name: Setup Sparkle +description: Installs Sparkle's generate_appcast and sign_update at the version the app resolves, and exports SPARKLE_BIN. + +runs: + using: composite + steps: + - name: Install the Sparkle tools + shell: bash + # These tools hold the EdDSA key that signs every update, so they come from the archive SwiftPM + # verifies for the framework: Sparkle's Package.swift at the pinned revision names its checksum. + env: + RESOLVED: TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved + run: | + set -euo pipefail + pin="$RUNNER_TEMP/sparkle-pin.txt" + python3 - "$RESOLVED" > "$pin" << 'PYTHON' + import json + import sys + + pins = [pin for pin in json.load(open(sys.argv[1]))["pins"] if pin["identity"] == "sparkle"] + if len(pins) != 1: + sys.exit(f"{sys.argv[1]} pins Sparkle {len(pins)} times") + print(pins[0]["state"]["version"], pins[0]["state"]["revision"]) + PYTHON + read -r version revision < "$pin" + case "$version" in *[!0-9.]* | '') echo "::error::Sparkle pin '$version' is not a version"; exit 1 ;; esac + case "$revision" in *[!0-9a-f]* | '') echo "::error::Sparkle pin '$revision' is not a commit"; exit 1 ;; esac + + manifest="$(curl -fsSL --retry 3 "https://raw.githubusercontent.com/sparkle-project/Sparkle/$revision/Package.swift")" + checksum="$(sed -n 's/^let checksum = "\([0-9a-f]\{64\}\)"$/\1/p' <<< "$manifest")" + tag="$(sed -n 's/^let tag = "\(.*\)"$/\1/p' <<< "$manifest")" + if [ -z "$checksum" ] || [ "$tag" != "$version" ]; then + echo "::error::Sparkle's Package.swift at $revision names tag '$tag' and checksum '$checksum', not $version" + exit 1 + fi + + archive="$RUNNER_TEMP/sparkle-$version.zip" + dir="$RUNNER_TEMP/sparkle-$version" + curl -fsSL --retry 3 -o "$archive" \ + "https://github.com/sparkle-project/Sparkle/releases/download/$version/Sparkle-for-Swift-Package-Manager.zip" + echo "$checksum $archive" | shasum -a 256 -c - + rm -rf "$dir" + unzip -q "$archive" -d "$dir" + for tool in generate_appcast sign_update; do + if [ ! -x "$dir/bin/$tool" ]; then + echo "::error::the Sparkle $version archive has no bin/$tool" + exit 1 + fi + done + echo "SPARKLE_BIN=$dir/bin" >> "$GITHUB_ENV" + echo "Sparkle $version tools in $dir/bin" diff --git a/.github/workflows/repo-hygiene.yml b/.github/workflows/repo-hygiene.yml index 940ff29759..9cab3d2a8f 100644 --- a/.github/workflows/repo-hygiene.yml +++ b/.github/workflows/repo-hygiene.yml @@ -115,9 +115,6 @@ jobs: - name: Check currentPluginKitVersion moved at most once since the last release run: python3 scripts/ci/check-pluginkit-bump-cadence.py --remote origin - - name: Check the Sparkle version agrees everywhere - run: python3 scripts/ci/check-sparkle-version.py - - name: Check the shared-contract drift gates run: scripts/audit-refactor-health.sh --check diff --git a/project.yml b/project.yml index fd3efe4dfc..e4bf756589 100644 --- a/project.yml +++ b/project.yml @@ -48,9 +48,10 @@ packages: path: Packages/TableProMacCore TableProOracle: path: Packages/TableProOracle + # Exact, because .github/actions/setup-sparkle installs the feed signing tools from this pin. Sparkle: url: https://github.com/sparkle-project/Sparkle - majorVersion: 2.10.0 + exactVersion: 2.10.0 Yams: url: https://github.com/jpsim/Yams majorVersion: 5.1.0 diff --git a/scripts/ci/check-sparkle-version.py b/scripts/ci/check-sparkle-version.py deleted file mode 100755 index bfada43686..0000000000 --- a/scripts/ci/check-sparkle-version.py +++ /dev/null @@ -1,139 +0,0 @@ -#!/usr/bin/env python3 -"""Fails on the Sparkle configuration invariants nothing else checks. - -The framework the app links and the generate_appcast that signs its feed are the same project, and -a release that mixes them is a release nobody can measure. Nothing checked this before: project.yml -declares `majorVersion`, which XcodeGen writes as upToNextMajorVersion, so a resolve can move the -framework on its own while the signing tool stays where it was. - -The second invariant is the deployment target. TablePro publishes two items per release and tells -them apart by `sparkle:hardwareRequirements`, which `generate_appcast` emits only while -`LSMinimumSystemVersion` is below 27.0 (ArchiveItem.swift:255-262, unchanged in 2.10.0): at or above -it, `mayNeedHardwareRequirement` is forced false and the element is never written. merge-appcast.py -then refuses the release, about forty minutes in, with both architectures already notarized and the -tag already pushed. - -Runs on Repo Hygiene, which has no Xcode. Pure text. -""" - -import pathlib -import re -import sys - -ROOT = pathlib.Path(__file__).resolve().parents[2] - -PROJECT_YML = ROOT / "project.yml" -PACKAGE_RESOLVED = ROOT / "TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved" -SIGN_SCRIPT = ROOT / "scripts/ci/sign-and-appcast.sh" -LICENSES = ROOT / "TablePro/Resources/ThirdPartyLicenses/licenses.yml" - - -class VersionError(Exception): - """A disagreement. Every one of these fails the job that raised it.""" - - -def read(path): - try: - return path.read_text(encoding="utf-8") - except OSError as error: - raise VersionError(f"{path.relative_to(ROOT)}: {error}") from error - - -def sole_match(pattern, text, path, what): - matches = re.findall(pattern, text, re.MULTILINE) - if len(matches) != 1: - raise VersionError( - f"{path.relative_to(ROOT)}: expected exactly one {what}, found {len(matches)}" - ) - return matches[0] - - -HARDWARE_REQUIREMENTS_CEILING = (27, 0) - - -def deployment_target(): - project = read(PROJECT_YML) - block = re.search(r"^ deploymentTarget:\n(?:^ {4}.*\n)+", project, re.MULTILINE) - if block is None: - raise VersionError("project.yml: no deploymentTarget block") - return sole_match(r'^ {4}macOS:\s*"?([0-9.]+)"?\s*$', block.group(0), PROJECT_YML, "macOS deployment target") - - -def check_deployment_target(): - raw = deployment_target() - parts = tuple(int(part) for part in raw.split(".")[:2]) - padded = parts + (0,) * (2 - len(parts)) - if padded >= HARDWARE_REQUIREMENTS_CEILING: - raise VersionError( - f"the macOS deployment target is {raw}, at or above " - f"{'.'.join(str(n) for n in HARDWARE_REQUIREMENTS_CEILING)}. generate_appcast stops " - "emitting sparkle:hardwareRequirements there, so the two per-architecture items become " - "indistinguishable and merge-appcast.py refuses the release after both notarizations. " - "Move the release to one Universal archive before raising the target" - ) - return raw - - -def declared_versions(): - """Every place a Sparkle version is written down, as {label: version}.""" - project = read(PROJECT_YML) - sparkle_block = re.search( - r"^ Sparkle:\n(?:^ {4}.*\n)+", project, re.MULTILINE - ) - if sparkle_block is None: - raise VersionError("project.yml: no Sparkle package block") - - versions = { - "project.yml majorVersion": sole_match( - r"^ {4}majorVersion:\s*(\S+)\s*$", sparkle_block.group(0), PROJECT_YML, "majorVersion" - ), - "Package.resolved": sole_match( - r'"identity"\s*:\s*"sparkle".*?"version"\s*:\s*"([^"]+)"', - read(PACKAGE_RESOLVED).replace("\n", " "), - PACKAGE_RESOLVED, - "sparkle pin", - ), - "sign-and-appcast.sh SPARKLE_VERSION": sole_match( - r'^SPARKLE_VERSION="([^"]+)"\s*$', read(SIGN_SCRIPT), SIGN_SCRIPT, "SPARKLE_VERSION" - ), - } - - licenses = read(LICENSES) - entry = re.search(r"^- id: sparkle\n(?:^(?![-]).*\n)+", licenses, re.MULTILINE) - if entry is None: - raise VersionError("licenses.yml: no sparkle entry") - versions["licenses.yml version"] = sole_match( - r"^ version:\s*(\S+)\s*$", entry.group(0), LICENSES, "version" - ) - versions["licenses.yml licenseTextURL"] = sole_match( - r"licenseTextURL:\s*https://github\.com/sparkle-project/Sparkle/blob/([^/]+)/LICENSE", - entry.group(0), - LICENSES, - "licenseTextURL", - ) - return versions - - -def main(): - try: - target = check_deployment_target() - versions = declared_versions() - except VersionError as error: - print(f"::error::check-sparkle-version: {error}", file=sys.stderr) - return 1 - - distinct = set(versions.values()) - if len(distinct) != 1: - print("::error::check-sparkle-version: the Sparkle version is not the same everywhere", file=sys.stderr) - for label, version in versions.items(): - print(f" {version:<10} {label}", file=sys.stderr) - return 1 - - version = distinct.pop() - print(f"Sparkle {version} agrees across {len(versions)} declarations") - print(f"macOS deployment target {target} still emits sparkle:hardwareRequirements") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/release/feed.py b/scripts/release/feed.py new file mode 100755 index 0000000000..701c3a1dc0 --- /dev/null +++ b/scripts/release/feed.py @@ -0,0 +1,521 @@ +#!/usr/bin/env python3 + +import argparse +import hashlib +import json +import lzma +import os +import plistlib +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import xml.etree.ElementTree as ET +from contextlib import contextmanager +from dataclasses import dataclass +from pathlib import Path +from typing import Callable, Dict, Iterator, List, Optional, Tuple + +import registry +import version + +ROOT = Path(__file__).resolve().parent.parent.parent +SPARKLE = "{http://www.andymatuschak.org/xml-namespaces/sparkle}" +BRANCH = "feeds" +REMOTE_BRANCH = f"refs/remotes/origin/{BRANCH}" +APPCAST = "appcast.xml" +MANIFEST = "plugins/v3.json" +METADATA = ".github/plugin-registry.json" +PROJECT = "project.yml" +PLUGINKIT_CONFIG = "Configs/PluginKit.xcconfig" +PHASED_ROLLOUT_SECONDS = 21600 +PUSH_ATTEMPTS = 3 +BOT_NAME = "github-actions[bot]" +BOT_EMAIL = "41898282+github-actions[bot]@users.noreply.github.com" +ITEM_TEXT = re.compile(r"[ \t]*.*?[ \t]*\n?", re.DOTALL) +INTEGER = re.compile(r"[0-9]+") + + +class FeedError(Exception): + pass + + +@dataclass(frozen=True) +class Item: + build: Optional[str] + short_version: Optional[str] + channel: Optional[str] + is_critical: bool + critical_version: Optional[str] + url: Optional[str] + signature: Optional[str] + length: Optional[str] + + @property + def label(self) -> str: + return self.short_version or self.build or "an item" + + +@dataclass(frozen=True) +class Expected: + release: version.Version + url: str + + +def _sparkle_value(element: ET.Element, name: str) -> Optional[str]: + child = element.find(SPARKLE + name) + if child is not None: + return (child.text or "").strip() + enclosure = element.find("enclosure") + return enclosure.get(SPARKLE + name) if enclosure is not None else None + + +def parse_items(data: bytes) -> List[Item]: + try: + root = ET.fromstring(data) + except ET.ParseError as error: + raise FeedError(f"{APPCAST} is not well-formed XML: {error}") from error + channel = root.find("channel") + if root.tag != "rss" or channel is None: + raise FeedError(f"{APPCAST} has no rss channel") + items = [] + for element in channel.findall("item"): + enclosure = element.find("enclosure") + critical = element.find(f".//{SPARKLE}criticalUpdate") + name = element.find(SPARKLE + "channel") + items.append(Item( + build=_sparkle_value(element, "version"), + short_version=_sparkle_value(element, "shortVersionString"), + channel=(name.text or "").strip() if name is not None else None, + is_critical=critical is not None, + critical_version=critical.get(SPARKLE + "version") if critical is not None else None, + url=enclosure.get("url") if enclosure is not None else None, + signature=enclosure.get(SPARKLE + "edSignature") if enclosure is not None else None, + length=enclosure.get("length") if enclosure is not None else None, + )) + return items + + +def carried_critical(items: List[Item]) -> Optional[int]: + builds = [] + for item in items: + if item.channel is not None or not item.is_critical: + continue + # A critical update with no version is critical below its own build. + text = item.critical_version or item.build + if text is None or not INTEGER.fullmatch(text): + raise FeedError(f"{item.label}: the critical update version {text!r} is not an integer") + builds.append(int(text)) + return max(builds) if builds else None + + +def feed_problems(data: bytes, previous: Optional[bytes] = None, expected: Optional[Expected] = None) -> List[str]: + items = parse_items(data) + problems = [] + if previous is not None and any(item.channel is None for item in parse_items(previous)) \ + and not any(item.channel is None for item in items): + problems.append("the feed no longer has a default-channel item") + + versions: Dict[int, set] = {} + for item in items: + if item.url is None: + problems.append(f"{item.label} has no enclosure") + if not item.signature: + problems.append(f"{item.label}'s enclosure has no sparkle:edSignature") + if item.length is None or not INTEGER.fullmatch(item.length): + problems.append(f"{item.label}'s enclosure has no length") + if item.channel is not None and item.channel != "beta": + problems.append(f"{item.label} is on channel {item.channel!r}; the only channel is beta") + if item.channel == "beta" and item.is_critical: + problems.append(f"{item.label} is a beta marked critical") + if item.build is None or not INTEGER.fullmatch(item.build): + problems.append(f"{item.label}: sparkle:version {item.build!r} is not an integer") + continue + try: + parsed = version.parse_version(item.short_version or "") + except ValueError: + problems.append(f"build {item.build}: shortVersionString {item.short_version!r} is not a release version") + continue + versions.setdefault(int(item.build), set()).add(parsed) + + ordered = [] + for build, found in sorted(versions.items()): + if len(found) > 1: + problems.append(f"build {build} carries {', '.join(sorted(str(each) for each in found))}") + else: + ordered.append((build, found.pop())) + for (low_build, low), (high_build, high) in zip(ordered, ordered[1:]): + if not low.build < high.build: + problems.append(f"{high} has the higher build ({high_build} over {low_build}) but not the higher version") + + if expected is not None: + release = expected.release + channel = "beta" if release.is_beta else None + if not any(item.build == str(release.build) and item.short_version == str(release) + and item.channel == channel and item.url == expected.url for item in items): + problems.append(f"no {release.channel} item for {release} ({release.build}) at {expected.url}") + return problems + + +def remove_items(text: str, short_version: str) -> Tuple[str, int]: + items = parse_items(text.encode("utf-8")) + spans = list(ITEM_TEXT.finditer(text)) + if len(spans) != len(items): + raise FeedError(f"{APPCAST} has {len(items)} items but {len(spans)} blocks in its text; edit it by hand") + doomed = [span for span, item in zip(spans, items) if item.short_version == short_version] + for span in reversed(doomed): + text = text[:span.start()] + text[span.end():] + left = [item for item in items if item.short_version != short_version] + if parse_items(text.encode("utf-8")) != left: + raise FeedError(f"removing {short_version} from the text of {APPCAST} changed other items; edit it by hand") + return text, len(doomed) + + +def appcast_has(items: List[Item], release: version.Version) -> bool: + matches = [item for item in items if item.short_version == str(release) or item.build == str(release.build)] + for item in matches: + if item.short_version != str(release) or item.build != str(release.build): + raise FeedError(f"{APPCAST} has {item.short_version} at build {item.build}, " + f"which clashes with {release} at {release.build}") + return bool(matches) + + +def appcast_arguments(stage: Path, tag: str, release: version.Version, critical_build: Optional[int], + repository: str) -> List[str]: + command = [ + "--ed-key-file", "-", "--embed-release-notes", "--maximum-deltas", "0", + "--versions", str(release.build), + "--download-url-prefix", f"https://github.com/{repository}/releases/download/{tag}/", + "--full-release-notes-url", f"https://github.com/{repository}/releases/tag/{tag}", + ] + if release.is_beta: + command += ["--channel", "beta"] + else: + command += ["--phased-rollout-interval", str(PHASED_ROLLOUT_SECONDS)] + if critical_build is not None: + command += ["--critical-update-version", str(critical_build)] + return command + [str(stage)] + + +def pluginkit_version(root: Path) -> int: + path = root / PLUGINKIT_CONFIG + try: + text = path.read_text(encoding="utf-8") + except OSError as error: + raise FeedError(f"{PLUGINKIT_CONFIG}: {error}") from error + found = re.search(r"^TABLEPRO_PLUGINKIT_VERSION\s*=\s*([0-9]+)\s*$", text, re.MULTILINE) + if found is None: + raise FeedError(f"{PLUGINKIT_CONFIG} sets no TABLEPRO_PLUGINKIT_VERSION") + return int(found.group(1)) + + +def check_archive(path: Path, release: version.Version) -> None: + name = "TablePro.app/Contents/Info.plist" + try: + with tarfile.open(path, "r:xz") as archive: + member = archive.extractfile(name) + info = plistlib.loads(member.read()) if member is not None else None + except KeyError: + info = None + except (tarfile.TarError, lzma.LZMAError, plistlib.InvalidFileException, ValueError, OSError) as error: + raise FeedError(f"{path.name}: {error}") from error + if info is None: + raise FeedError(f"{path.name} has no {name}") + found = (info.get("CFBundleShortVersionString"), info.get("CFBundleVersion")) + if found != (str(release), str(release.build)): + raise FeedError(f"{path.name} holds {found[0]} ({found[1]}), not {release} ({release.build})") + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +class Releases: + def __init__(self, repository: str, run: Callable = subprocess.run): + self.repository = repository + self._run = run + + def _gh(self, *args: str) -> str: + result = self._run(["gh", *args, "--repo", self.repository], capture_output=True, text=True) + if result.returncode != 0: + raise FeedError(f"gh {' '.join(args[:3])}: {result.stderr.strip()}") + return result.stdout + + def digests(self, tag: str) -> Dict[str, str]: + release = json.loads(self._gh("release", "view", tag, "--json", "isDraft,assets")) + if release.get("isDraft"): + raise FeedError(f"{tag} is a draft; the feed only points at a published release") + return {asset["name"]: asset.get("digest") or "" for asset in release.get("assets", [])} + + def download(self, tag: str, names: List[str], directory: Path) -> None: + patterns = [argument for name in names for argument in ("--pattern", name)] + self._gh("release", "download", tag, "--dir", str(directory), "--clobber", *patterns) + + +def fetch_assets(releases: Releases, tag: str, names: List[str], directory: Path) -> None: + wanted = [name for name in names if not (directory / name).is_file()] + if not wanted: + return + digests = releases.digests(tag) + missing = [name for name in wanted if name not in digests] + if missing: + raise FeedError(f"{tag} has no {', '.join(missing)}") + directory.mkdir(parents=True, exist_ok=True) + releases.download(tag, wanted, directory) + for name in wanted: + path = directory / name + if not path.is_file(): + raise FeedError(f"gh did not download {name} from {tag}") + found = f"sha256:{_sha256(path)}" + if digests[name] != found: + path.unlink() + raise FeedError(f"{name} downloaded as {found}, but {tag} lists {digests[name] or 'no digest'}") + + +def _git(root: Path, *args: str, run: Callable = subprocess.run, check: bool = True, + env: Optional[Dict[str, str]] = None) -> subprocess.CompletedProcess: + result = run(["git", "-C", str(root), *args], capture_output=True, text=True, + env={**os.environ, **env} if env else None) + if check and result.returncode != 0: + raise FeedError(f"git {' '.join(args)}: {result.stderr.strip()}") + return result + + +@contextmanager +def feeds_worktree(root: Path, run: Callable = subprocess.run) -> Iterator[Path]: + _git(root, "fetch", "--quiet", "--no-tags", "origin", f"+refs/heads/{BRANCH}:{REMOTE_BRANCH}", run=run) + parent = Path(tempfile.mkdtemp(prefix="feeds-")) + tree = parent / BRANCH + _git(root, "worktree", "add", "--quiet", "--detach", str(tree), REMOTE_BRANCH, run=run) + try: + yield tree + finally: + _git(root, "worktree", "remove", "--force", str(tree), run=run, check=False) + shutil.rmtree(parent, ignore_errors=True) + + +def commit(tree: Path, message: str, run: Callable = subprocess.run) -> bool: + paths = [path for path in (APPCAST, MANIFEST) if (tree / path).exists()] + _git(tree, "add", "--", *paths, run=run) + if _git(tree, "diff", "--cached", "--quiet", run=run, check=False).returncode == 0: + return False + identity = {f"GIT_{role}_{field}": value for role in ("AUTHOR", "COMMITTER") + for field, value in (("NAME", BOT_NAME), ("EMAIL", BOT_EMAIL))} + _git(tree, "commit", "--quiet", "-m", message, run=run, env=identity) + return True + + +def push(tree: Path, run: Callable = subprocess.run) -> bool: + result = _git(tree, "push", "--quiet", "origin", f"HEAD:refs/heads/{BRANCH}", run=run, check=False) + if result.returncode == 0: + return True + # A rejected fast-forward or a lost ref lock means another run pushed first. + if "[rejected]" in result.stderr or ("[remote rejected]" in result.stderr and "cannot lock ref" in result.stderr): + return False + raise FeedError(f"git push origin {BRANCH}: {result.stderr.strip()}") + + +def _tool(tools: Path, name: str, key: str, *args: str, run: Callable = subprocess.run) -> str: + result = run([str(tools / name), *args], input=key, capture_output=True, text=True) + output = (result.stdout + result.stderr).strip() + if result.returncode != 0: + raise FeedError(f"{name} failed: {output}") + return output + + +def sign(path: Path, key: str, tools: Path, run: Callable = subprocess.run) -> str: + return _tool(tools, "sign_update", key, "--ed-key-file", "-", str(path), run=run) + + +def verify_signature(path: Path, key: str, tools: Path, run: Callable = subprocess.run) -> List[str]: + try: + _tool(tools, "sign_update", key, "--verify", "--ed-key-file", "-", str(path), run=run) + except FeedError as error: + return [str(error)] + return [] + + +def _refuse(problems: List[str]) -> None: + if problems: + raise FeedError("the feed fails its checks, so nothing was pushed:\n " + "\n ".join(problems)) + + +def _publish(tree: Path, message: str, run: Callable, log: Callable[[str], None]) -> Optional[bool]: + if not commit(tree, message, run): + log(f"origin/{BRANCH} already has everything from '{message}'") + return False + if push(tree, run): + log(f"pushed '{message}' to origin/{BRANCH}") + return True + return None + + +def add(root: Path, tag: str, build: int, channel: str, notes: Path, critical: bool, tools: Path, key: str, + releases: Releases, repository: str, run: Callable = subprocess.run, + log: Callable[[str], None] = print) -> bool: + release = version.parse_tag(tag) + if build != release.build or channel != release.channel: + raise FeedError(f"{tag} is build {release.build} on {release.channel}, not build {build} on {channel}") + if critical and release.is_beta: + raise FeedError(f"{tag} is a beta, and a beta is never critical") + if not notes.is_file(): + raise FeedError(f"{notes} does not exist") + metadata = registry.load_metadata(root / METADATA) + bundled = registry.bundled_targets((root / PROJECT).read_text(encoding="utf-8")) + plugins = [f"{target}-{release}.zip" for target in registry.publish_set(metadata, bundled)] + kit = pluginkit_version(root) + archive = f"TablePro-{release}.tar.xz" + url_base = f"https://github.com/{repository}/releases/download/{tag}/" + expected = Expected(release, url_base + archive) + + with tempfile.TemporaryDirectory(prefix="feed-") as scratch: + downloads = Path(scratch) / "downloads" + entries = None + for attempt in range(1, PUSH_ATTEMPTS + 1): + wanted: Optional[Expected] = expected + with feeds_worktree(root, run) as tree: + appcast = tree / APPCAST + if not appcast.is_file(): + raise FeedError(f"origin/{BRANCH} has no {APPCAST}; seed the branch first (RELEASING.md)") + previous = appcast.read_bytes() + items = parse_items(previous) + if appcast_has(items, release): + log(f"{APPCAST} already has {release} ({release.build})") + else: + fetch_assets(releases, tag, [archive], downloads) + check_archive(downloads / archive, release) + stage = Path(scratch) / f"stage-{attempt}" + stage.mkdir() + shutil.copy2(downloads / archive, stage / archive) + shutil.copy2(notes, stage / f"TablePro-{release}.md") + shutil.copy2(appcast, stage / APPCAST) + critical_build = release.build if critical else None + if critical_build is None and not release.is_beta: + critical_build = carried_critical(items) + arguments = appcast_arguments(stage, tag, release, critical_build, repository) + log(_tool(tools, "generate_appcast", key, *arguments, run=run)) + if appcast_has(parse_items((stage / APPCAST).read_bytes()), release): + shutil.copy2(stage / APPCAST, appcast) + # generate_appcast signs the feed only once an archive sets SURequireSignedFeed. + log(sign(appcast, key, tools, run)) + else: + # The archive matched the tag, so only Sparkle's per-branch limit left it out. + log(f"generate_appcast keeps newer items on {release}'s branch, so {APPCAST} stays as it is") + wanted = None + if entries is None: + fetch_assets(releases, tag, plugins, downloads) + entries = registry.describe(downloads, str(release), release.build, kit, metadata, bundled) + registry.add(tree / MANIFEST, entries, release.channel, url_base, metadata["retainFromPluginKit"], log) + _refuse(feed_problems(appcast.read_bytes(), previous, wanted) + + verify_signature(appcast, key, tools, run)) + pushed = _publish(tree, f"release: {tag}", run, log) + if pushed is not None: + return pushed + log(f"origin/{BRANCH} moved (attempt {attempt} of {PUSH_ATTEMPTS}); starting again from a fresh fetch") + raise FeedError(f"origin/{BRANCH} moved during each of {PUSH_ATTEMPTS} attempts") + + +def withdraw(root: Path, text: str, tools: Path, key: str, run: Callable = subprocess.run, + log: Callable[[str], None] = print) -> bool: + release = version.parse_version(text) + for attempt in range(1, PUSH_ATTEMPTS + 1): + with feeds_worktree(root, run) as tree: + appcast = tree / APPCAST + previous = appcast.read_bytes() + edited, removed = remove_items(previous.decode("utf-8"), str(release)) + if removed: + if any(item.channel is None for item in parse_items(previous)) \ + and not any(item.channel is None for item in parse_items(edited.encode("utf-8"))): + raise FeedError(f"withdrawing {release} would leave the feed with no default-channel item") + appcast.write_text(edited, encoding="utf-8") + log(sign(appcast, key, tools, run)) + log(f"removed {removed} {release} item(s) from {APPCAST}") + else: + log(f"{APPCAST} has no {release} item") + if (tree / MANIFEST).exists(): + registry.withdraw(tree / MANIFEST, str(release), log) + _refuse(feed_problems(appcast.read_bytes(), previous) + verify_signature(appcast, key, tools, run)) + pushed = _publish(tree, f"withdraw: v{release}", run, log) + if pushed is not None: + return pushed + log(f"origin/{BRANCH} moved (attempt {attempt} of {PUSH_ATTEMPTS}); starting again from a fresh fetch") + raise FeedError(f"origin/{BRANCH} moved during each of {PUSH_ATTEMPTS} attempts") + + +def check(root: Path, tools: Path, key: str, run: Callable = subprocess.run) -> List[str]: + with feeds_worktree(root, run) as tree: + appcast = tree / APPCAST + if not appcast.is_file(): + return [f"origin/{BRANCH} has no {APPCAST}"] + problems = feed_problems(appcast.read_bytes()) + verify_signature(appcast, key, tools, run) + try: + if registry.load_manifest(tree / MANIFEST) is None: + problems.append(f"origin/{BRANCH} has no {MANIFEST}") + except registry.RegistryError as error: + problems.append(str(error)) + return problems + + +def published_items(root: Path, run: Callable = subprocess.run) -> List[Item]: + _git(root, "fetch", "--quiet", "--no-tags", "origin", f"+refs/heads/{BRANCH}:{REMOTE_BRANCH}", run=run) + result = run(["git", "-C", str(root), "show", f"{REMOTE_BRANCH}:{APPCAST}"], capture_output=True) + if result.returncode != 0: + raise FeedError(f"origin/{BRANCH} has no {APPCAST}: {result.stderr.decode(errors='replace').strip()}") + return parse_items(result.stdout) + + +def _environment() -> Tuple[Path, str]: + tools = os.environ.get("SPARKLE_BIN") + key = os.environ.get("SPARKLE_PRIVATE_KEY") + if not tools: + raise FeedError("SPARKLE_BIN is not set; run the setup-sparkle action first") + if not key: + raise FeedError("SPARKLE_PRIVATE_KEY is not set") + for name in ("generate_appcast", "sign_update"): + if not os.access(Path(tools) / name, os.X_OK): + raise FeedError(f"{tools} has no {name}") + return Path(tools), key + + +def main(argv: Optional[List[str]] = None) -> int: + parser = argparse.ArgumentParser(description="Writes the appcast and the plugin manifest on the feeds branch.") + commands = parser.add_subparsers(dest="command", required=True) + command = commands.add_parser("add", help="add a published release to the feed and the manifest") + command.add_argument("tag") + command.add_argument("--build", type=int, required=True) + command.add_argument("--channel", choices=registry.CHANNELS, required=True) + command.add_argument("--notes", type=Path, required=True) + command.add_argument("--critical", action="store_true") + commands.add_parser("withdraw", help="remove a version from the feed and the manifest").add_argument("version") + commands.add_parser("check", help="check the published feed") + args = parser.parse_args(argv) + + repository = os.environ.get("GITHUB_REPOSITORY", version.REPOSITORY) + try: + tools, key = _environment() + if args.command == "add": + add(ROOT, args.tag, args.build, args.channel, args.notes, args.critical, tools, key, + Releases(repository), repository) + elif args.command == "withdraw": + withdraw(ROOT, args.version, tools, key) + else: + problems = check(ROOT, tools, key) + for problem in problems: + print(f"error: {problem}", file=sys.stderr) + if problems: + return 1 + print(f"origin/{BRANCH} passes its checks") + except (FeedError, registry.RegistryError, ValueError, OSError) as error: + print(f"error: {error}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/release/publish.py b/scripts/release/publish.py new file mode 100755 index 0000000000..43902cc940 --- /dev/null +++ b/scripts/release/publish.py @@ -0,0 +1,224 @@ +#!/usr/bin/env python3 + +import argparse +import hashlib +import json +import os +import subprocess +import sys +import time +from pathlib import Path +from typing import Callable, Dict, List, Optional + +import feed +import version + +ROOT = Path(__file__).resolve().parent.parent.parent +FIND_ATTEMPTS = 5 +FIND_DELAY_SECONDS = 2 + + +class PublishError(Exception): + pass + + +def draft_action(is_published: bool, has_draft: bool) -> str: + if is_published: + return "published" + return "reuse" if has_draft else "create" + + +def should_be_latest(release: version.Version, latest_tag: Optional[str]) -> bool: + if release.is_beta: + return False + latest = version.try_parse_tag(latest_tag) if latest_tag else None + return latest is None or release.build > latest.build + + +def release_flags(release: version.Version, latest_tag: Optional[str]) -> List[str]: + flags = ["--draft=false"] + if release.is_beta: + return flags + ["--prerelease", "--latest=false"] + return flags + [f"--latest={'true' if should_be_latest(release, latest_tag) else 'false'}"] + + +def highest_stable(items: List[feed.Item]) -> Optional[version.Version]: + stables = [] + for item in items: + if item.channel is not None: + continue + parsed = version.try_parse_tag(f"v{item.short_version}") if item.short_version else None + if parsed is not None and not parsed.is_beta: + stables.append(parsed) + return max(stables, key=lambda candidate: candidate.build) if stables else None + + +def asset_problems(files: Dict[str, str], assets: List[dict]) -> List[str]: + found = {asset.get("name"): asset for asset in assets} + problems = [] + for name, digest in sorted(files.items()): + asset = found.get(name) + if asset is None: + problems.append(f"the release has no {name}") + elif asset.get("state") != "uploaded": + problems.append(f"{name} is {asset.get('state')}, not uploaded") + elif asset.get("digest") != f"sha256:{digest}": + found_digest = asset.get("digest") or "missing its digest" + problems.append(f"{name} is {found_digest} on GitHub but sha256:{digest} here") + return problems + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +class GitHub: + def __init__(self, repository: str, run: Callable = subprocess.run): + self.repository = repository + self._run = run + + def _gh(self, *args: str, check: bool = True) -> subprocess.CompletedProcess: + result = self._run(["gh", *args], capture_output=True, text=True) + if check and result.returncode != 0: + raise PublishError(f"gh {' '.join(args[:3])}: {result.stderr.strip()}") + return result + + def _release(self, *args: str) -> List[str]: + return ["release", *args, "--repo", self.repository] + + def is_published(self, tag: str) -> bool: + result = self._gh("api", f"repos/{self.repository}/releases/tags/{tag}", "--jq", ".draft", check=False) + if result.returncode == 0: + return result.stdout.strip() == "false" + if "HTTP 404" in result.stderr: + return False + raise PublishError(f"gh api releases/tags/{tag}: {result.stderr.strip()}") + + # gh races its published and draft lookups, so this runs only once no published release holds the tag. + def draft(self, tag: str) -> Optional[dict]: + result = self._gh(*self._release("view", tag, "--json", "isDraft,assets"), check=False) + if result.returncode != 0: + if "release not found" in result.stderr: + return None + raise PublishError(f"gh release view {tag}: {result.stderr.strip()}") + release = json.loads(result.stdout) + if not release.get("isDraft"): + raise PublishError(f"{tag} was published while this ran") + return release + + def create_draft(self, tag: str, notes: Path, files: List[Path], prerelease: bool) -> None: + self._gh(*self._release("create", tag, "--draft", "--verify-tag", "--title", tag, "--notes-file", str(notes), + f"--prerelease={'true' if prerelease else 'false'}", *[str(path) for path in files])) + + def update_draft(self, tag: str, notes: Path, files: List[Path], prerelease: bool) -> None: + self._gh(*self._release("edit", tag, "--title", tag, "--notes-file", str(notes), + f"--prerelease={'true' if prerelease else 'false'}")) + self._gh(*self._release("upload", tag, "--clobber", *[str(path) for path in files])) + + def edit(self, tag: str, flags: List[str]) -> None: + self._gh(*self._release("edit", tag, *flags)) + + def latest_tag(self) -> Optional[str]: + result = self._gh("api", f"repos/{self.repository}/releases/latest", "--jq", ".tag_name", check=False) + if result.returncode == 0: + return result.stdout.strip() or None + if "HTTP 404" in result.stderr: + return None + raise PublishError(f"gh api releases/latest: {result.stderr.strip()}") + + +def draft(github: GitHub, tag: str, notes: Path, files: List[Path], sleep: Callable[[float], None] = time.sleep, + log: Callable[[str], None] = print) -> str: + parsed = version.parse_tag(tag) + missing = [str(path) for path in [notes, *files] if not path.is_file()] + if missing: + raise PublishError(f"{', '.join(missing)} does not exist") + names = [path.name for path in files] + if len(set(names)) != len(names): + raise PublishError("two assets share a file name") + + published = github.is_published(tag) + existing = None if published else github.draft(tag) + action = draft_action(published, existing is not None) + if action == "published": + log(f"{tag} is already published") + return "published" + if action == "create": + github.create_draft(tag, notes, files, parsed.is_beta) + log(f"created the draft for {tag}") + else: + github.update_draft(tag, notes, files, parsed.is_beta) + log(f"reused the draft an earlier attempt left for {tag}") + + current = None + for _ in range(FIND_ATTEMPTS): + current = github.draft(tag) + if current is not None: + break + sleep(FIND_DELAY_SECONDS) + if current is None: + raise PublishError(f"the draft for {tag} was created but cannot be found") + problems = asset_problems({path.name: _sha256(path) for path in files}, current.get("assets", [])) + if problems: + raise PublishError(f"the draft for {tag} does not hold this build, so it stays unpublished:\n " + + "\n ".join(problems)) + log(f"the draft for {tag} holds all {len(files)} assets") + return "draft" + + +def release(github: GitHub, tag: str, log: Callable[[str], None] = print) -> None: + parsed = version.parse_tag(tag) + if github.is_published(tag): + log(f"{tag} is already published") + return + if github.draft(tag) is None: + raise PublishError(f"{tag} has no draft; run publish.py draft first") + flags = release_flags(parsed, github.latest_tag()) + github.edit(tag, flags) + log(f"published {tag} ({' '.join(flags[1:])})") + + +def latest(github: GitHub, items: List[feed.Item], log: Callable[[str], None] = print) -> str: + highest = highest_stable(items) + if highest is None: + raise PublishError("the feed has no stable item") + github.edit(highest.tag, ["--latest=true"]) + log(f"{highest.tag} is Latest") + return highest.tag + + +def _stderr(message: str) -> None: + print(message, file=sys.stderr) + + +def main(argv: Optional[List[str]] = None) -> int: + parser = argparse.ArgumentParser(description="Drafts, publishes and marks GitHub releases.") + commands = parser.add_subparsers(dest="command", required=True) + command = commands.add_parser("draft", help="put every asset on a draft and check its digests") + command.add_argument("tag") + command.add_argument("--notes", type=Path, required=True) + command.add_argument("files", type=Path, nargs="+") + commands.add_parser("release", help="publish the draft").add_argument("tag") + commands.add_parser("latest", help="mark the highest stable left in the feed as Latest") + args = parser.parse_args(argv) + + github = GitHub(os.environ.get("GITHUB_REPOSITORY", version.REPOSITORY)) + try: + if args.command == "draft": + print(f"state={draft(github, args.tag, args.notes, args.files, log=_stderr)}") + elif args.command == "release": + release(github, args.tag, _stderr) + else: + latest(github, feed.published_items(ROOT), _stderr) + except (PublishError, feed.FeedError, ValueError, OSError) as error: + print(f"error: {error}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/release/registry.py b/scripts/release/registry.py new file mode 100755 index 0000000000..2bd77268ed --- /dev/null +++ b/scripts/release/registry.py @@ -0,0 +1,337 @@ +#!/usr/bin/env python3 + +import argparse +import hashlib +import json +import plistlib +import re +import sys +import zipfile +from pathlib import Path +from typing import Callable, Dict, List, Optional, Set + +import version + +ROOT = Path(__file__).resolve().parent.parent.parent +METADATA = ROOT / ".github/plugin-registry.json" +PROJECT = ROOT / "project.yml" +SCHEMA_VERSION = 3 +CHANNELS = ("stable", "beta") +RELEASES_PER_KEY = 2 +PROVIDES = ("TableProProvidesDatabaseTypeIds", "TableProProvidesExportFormatIds", "TableProProvidesImportFormatIds") +SYSTEM_VERSION = re.compile(r"[0-9]+(\.[0-9]+){0,2}") +SHA256 = re.compile(r"[0-9a-f]{64}") + + +class RegistryError(Exception): + pass + + +def load_metadata(path: Path) -> dict: + try: + metadata = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as error: + raise RegistryError(f"{path}: {error}") from error + shape = "{retainFromPluginKit, plugins: {: {name, iconName}}}" + plugins = metadata.get("plugins") if isinstance(metadata, dict) else None + if not isinstance(plugins, dict) or not _is_int(metadata.get("retainFromPluginKit")): + raise RegistryError(f"{path} is not in the {shape} shape") + for target, entry in plugins.items(): + if not isinstance(entry, dict) or not all(isinstance(entry.get(key), str) for key in ("name", "iconName")): + raise RegistryError(f"{path}: {target} needs a name and an iconName, as in {shape}") + return metadata + + +def bundled_targets(project: str) -> Set[str]: + block = re.search(r"^aggregateTargets:\n(?:^(?: .*)?\n)*?^ BundledPlugins:\n^ targets:\n((?:^ - .*\n)+)", + project, re.MULTILINE) + if block is None: + raise RegistryError("project.yml has no BundledPlugins aggregate with a targets list") + return {line.strip()[2:].strip() for line in block.group(1).splitlines()} + + +def publish_set(metadata: dict, bundled: Set[str]) -> List[str]: + targets = sorted(metadata["plugins"]) + inside = [target for target in targets if target in bundled] + if inside: + raise RegistryError(f"the app bundles {', '.join(inside)}, so the registry must not publish them") + if not targets: + raise RegistryError("the registry metadata publishes no plugin") + return targets + + +def _is_int(value) -> bool: + return isinstance(value, int) and not isinstance(value, bool) + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def read_info(archive: Path, target: str) -> dict: + name = f"{target}.tableplugin/Contents/Info.plist" + try: + with zipfile.ZipFile(archive) as bundle: + return plistlib.loads(bundle.read(name)) + except KeyError as error: + raise RegistryError(f"{archive.name} has no {name}") from error + except (OSError, zipfile.BadZipFile, plistlib.InvalidFileException, ValueError) as error: + raise RegistryError(f"{archive.name}: {error}") from error + + +def describe_one(archive: Path, target: str, entry: dict, release: str, build: int, kit: int) -> dict: + info = read_info(archive, target) + problems = [] + if info.get("CFBundleShortVersionString") != release: + problems.append(f"CFBundleShortVersionString is {info.get('CFBundleShortVersionString')!r}, not {release!r}") + if info.get("CFBundleVersion") != str(build): + problems.append(f"CFBundleVersion is {info.get('CFBundleVersion')!r}, not '{build}'") + found_kit = info.get("TableProPluginKitVersion") + if not _is_int(found_kit) or found_kit != kit: + problems.append(f"TableProPluginKitVersion is {found_kit!r}, not the integer {kit}") + identifier = info.get("CFBundleIdentifier") + if not isinstance(identifier, str) or not identifier: + problems.append("CFBundleIdentifier is missing") + system = info.get("LSMinimumSystemVersion") + if not isinstance(system, str) or not SYSTEM_VERSION.fullmatch(system): + problems.append(f"LSMinimumSystemVersion is {system!r}") + type_ids = info.get(PROVIDES[0], []) + if not isinstance(type_ids, list) or not all(isinstance(type_id, str) for type_id in type_ids): + problems.append(f"{PROVIDES[0]} is not a list of strings") + # The app finds the plugin that serves a database type or a file format through these keys. + if not any(info.get(key) for key in PROVIDES): + problems.append(f"it declares none of {', '.join(PROVIDES)}") + if problems: + raise RegistryError(f"{archive.name}: " + "; ".join(problems)) + return { + "id": identifier, + "name": entry["name"], + "iconName": entry["iconName"], + "databaseTypeIds": type_ids, + "version": release, + "build": build, + "pluginKit": kit, + "minimumSystemVersion": system, + "file": archive.name, + "sha256": _sha256(archive), + } + + +def describe(zips: Path, release: str, build: int, kit: int, metadata: dict, bundled: Set[str]) -> List[dict]: + entries = [] + for target in publish_set(metadata, bundled): + archive = zips / f"{target}-{release}.zip" + if not archive.is_file(): + raise RegistryError(f"{archive} does not exist") + entries.append(describe_one(archive, target, metadata["plugins"][target], release, build, kit)) + seen: Dict[str, str] = {} + for entry in entries: + if entry["id"] in seen: + raise RegistryError(f"{entry['file']} and {seen[entry['id']]} share the bundle id {entry['id']}") + seen[entry["id"]] = entry["file"] + return entries + + +def empty_manifest(retain_from: int) -> dict: + return {"schemaVersion": SCHEMA_VERSION, "retainFromPluginKit": retain_from, "plugins": []} + + +def load_manifest(path: Path) -> Optional[dict]: + if not path.exists(): + return None + try: + manifest = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as error: + raise RegistryError(f"{path}: {error}") from error + if not isinstance(manifest, dict) or manifest.get("schemaVersion") != SCHEMA_VERSION: + found = manifest.get("schemaVersion") if isinstance(manifest, dict) else None + raise RegistryError(f"{path} has schemaVersion {found!r}; this tool writes only {SCHEMA_VERSION}") + if not _is_int(manifest.get("retainFromPluginKit")) or not isinstance(manifest.get("plugins"), list): + raise RegistryError(f"{path} needs an integer retainFromPluginKit and a plugins list") + for plugin in manifest["plugins"]: + if not isinstance(plugin, dict) or not isinstance(plugin.get("id"), str) \ + or not isinstance(plugin.get("releases"), list): + raise RegistryError(f"{path}: every plugin needs an id and a releases list") + for release in plugin["releases"]: + if not isinstance(release, dict) or not all(_is_int(release.get(key)) for key in ("build", "pluginKit")) \ + or release.get("channel") not in CHANNELS \ + or not all(isinstance(release.get(key), str) for key in ("version", "minimumSystemVersion")): + raise RegistryError(f"{path}: a release of {plugin['id']} lacks an integer build or pluginKit, " + "a channel, a version or a minimumSystemVersion") + return manifest + + +def write_manifest(path: Path, manifest: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(manifest, indent=2, ensure_ascii=False) + "\n", encoding="utf-8") + + +def retain(releases: List[dict], floor: int) -> List[dict]: + kept = [release for release in releases if release["pluginKit"] >= floor] + kept = [ + release for release in kept + if release["channel"] != "beta" or not any( + other["channel"] == "stable" and other["pluginKit"] == release["pluginKit"] + and other["minimumSystemVersion"] == release["minimumSystemVersion"] and other["build"] > release["build"] + for other in kept) + ] + groups: Dict[tuple, List[dict]] = {} + for release in sorted(kept, key=lambda candidate: candidate["build"], reverse=True): + key = (release["channel"], release["pluginKit"], release["minimumSystemVersion"]) + groups.setdefault(key, []).append(release) + survivors = [release for group in groups.values() for release in group[:RELEASES_PER_KEY]] + return sorted(survivors, key=lambda candidate: candidate["build"], reverse=True) + + +def _release(entry: dict, channel: str, url_base: str) -> dict: + return { + "version": entry["version"], + "build": entry["build"], + "channel": channel, + "pluginKit": entry["pluginKit"], + "minimumSystemVersion": entry["minimumSystemVersion"], + "url": url_base.rstrip("/") + "/" + entry["file"], + "sha256": entry["sha256"], + } + + +def _check_entry(entry: dict, channel: str) -> None: + missing = [key for key in ("id", "name", "iconName", "databaseTypeIds", "version", "build", "pluginKit", + "minimumSystemVersion", "file", "sha256") if key not in entry] + if missing: + raise RegistryError(f"an entry has no {', '.join(missing)}") + if not SHA256.fullmatch(str(entry["sha256"])): + raise RegistryError(f"{entry['id']}: sha256 {entry['sha256']!r} is not 64 lowercase hex digits") + parsed = version.parse_version(entry["version"]) + if parsed.build != entry["build"]: + raise RegistryError(f"{entry['id']}: build {entry['build']} is not {parsed.build}, the build of {parsed}") + if parsed.channel != channel: + raise RegistryError(f"{entry['id']}: {parsed} belongs to the {parsed.channel} channel, not {channel}") + + +def add(path: Path, entries: List[dict], channel: str, url_base: str, retain_from: int, + log: Callable[[str], None] = print) -> bool: + if channel not in CHANNELS: + raise RegistryError(f"channel {channel!r} is not one of {', '.join(CHANNELS)}") + if not url_base.startswith("https://"): + raise RegistryError(f"the URL base {url_base!r} is not https") + for entry in entries: + _check_entry(entry, channel) + loaded = load_manifest(path) + before = json.dumps(loaded, sort_keys=True) if loaded is not None else None + manifest = loaded if loaded is not None else empty_manifest(retain_from) + floor = max(manifest["retainFromPluginKit"], retain_from) + plugins = {plugin["id"]: plugin for plugin in manifest["plugins"]} + + arrived = [] + for entry in entries: + release = _release(entry, channel, url_base) + plugin = plugins.setdefault(entry["id"], {"id": entry["id"], "releases": []}) + same = [existing for existing in plugin["releases"] if existing["build"] == release["build"]] + if same and same[0].get("sha256") != release["sha256"]: + raise RegistryError(f"{entry['id']} already has build {release['build']} with sha256 " + f"{same[0].get('sha256')}, not {release['sha256']}") + if same: + log(f"{entry['id']} {release['version']}: already in the manifest") + continue + plugin["releases"].append(release) + arrived.append((entry, release)) + + for plugin in plugins.values(): + plugin["releases"] = retain(plugin["releases"], floor) + for entry, release in arrived: + plugin = plugins[entry["id"]] + if not any(kept is release for kept in plugin["releases"]): + log(f"{entry['id']} {release['version']}: pruned on arrival by retention, not added") + elif plugin["releases"][0] is release: + plugin.update({key: entry[key] for key in ("name", "iconName", "databaseTypeIds")}) + + manifest = { + "schemaVersion": SCHEMA_VERSION, + "retainFromPluginKit": floor, + "plugins": [_ordered(plugins[key]) for key in sorted(plugins) if plugins[key]["releases"]], + } + if json.dumps(manifest, sort_keys=True) == before: + return False + write_manifest(path, manifest) + return True + + +def _ordered(plugin: dict) -> dict: + keys = ("id", "name", "iconName", "databaseTypeIds") + return {**{key: plugin[key] for key in keys if key in plugin}, "releases": plugin["releases"]} + + +def withdraw(path: Path, release: str, log: Callable[[str], None] = print) -> bool: + version.parse_version(release) + manifest = load_manifest(path) + if manifest is None: + raise RegistryError(f"{path} does not exist") + removed = 0 + for plugin in manifest["plugins"]: + kept = [candidate for candidate in plugin["releases"] if candidate.get("version") != release] + removed += len(plugin["releases"]) - len(kept) + plugin["releases"] = kept + if not removed: + log(f"{path.name} has no {release} release") + return False + manifest["plugins"] = [plugin for plugin in manifest["plugins"] if plugin["releases"]] + write_manifest(path, manifest) + log(f"withdrew {removed} {release} release(s) from {path.name}") + return True + + +def main(argv: Optional[List[str]] = None) -> int: + parser = argparse.ArgumentParser(description="The registry plugins and the v3 plugin manifest.") + commands = parser.add_subparsers(dest="command", required=True) + command = commands.add_parser("targets", help="print the targets the registry publishes") + command.add_argument("--metadata", type=Path, default=METADATA) + command.add_argument("--project", type=Path, default=PROJECT) + command = commands.add_parser("describe", help="read the built plugin zips into manifest entries") + command.add_argument("--zips", type=Path, required=True) + command.add_argument("--version", required=True) + command.add_argument("--build", type=int, required=True) + command.add_argument("--kit", type=int, required=True) + command.add_argument("--metadata", type=Path, required=True) + command.add_argument("--project", type=Path, default=PROJECT) + command.add_argument("--out", type=Path, required=True) + command = commands.add_parser("add", help="add described releases to the manifest") + command.add_argument("--manifest", type=Path, required=True) + command.add_argument("--entries", type=Path, required=True) + command.add_argument("--channel", choices=CHANNELS, required=True) + command.add_argument("--url-base", required=True) + command.add_argument("--retain-from", type=int, required=True) + command = commands.add_parser("withdraw", help="remove one version's releases from the manifest") + command.add_argument("--manifest", type=Path, required=True) + command.add_argument("--version", required=True) + args = parser.parse_args(argv) + + try: + if args.command == "targets": + bundled = bundled_targets(args.project.read_text(encoding="utf-8")) + print("\n".join(publish_set(load_metadata(args.metadata), bundled))) + elif args.command == "describe": + bundled = bundled_targets(args.project.read_text(encoding="utf-8")) + entries = describe(args.zips, args.version, args.build, args.kit, load_metadata(args.metadata), bundled) + args.out.write_text(json.dumps(entries, indent=2) + "\n", encoding="utf-8") + print(f"described {len(entries)} plugins in {args.out}") + elif args.command == "add": + entries = json.loads(args.entries.read_text(encoding="utf-8")) + if not isinstance(entries, list): + raise RegistryError(f"{args.entries} is not a list of entries") + changed = add(args.manifest, entries, args.channel, args.url_base, args.retain_from) + print(f"{args.manifest} {'updated' if changed else 'unchanged'}") + elif args.command == "withdraw": + withdraw(args.manifest, args.version) + except (RegistryError, ValueError, OSError) as error: + print(f"error: {error}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/release/test_feed.py b/scripts/release/test_feed.py new file mode 100644 index 0000000000..2b5abc4584 --- /dev/null +++ b/scripts/release/test_feed.py @@ -0,0 +1,531 @@ +#!/usr/bin/env python3 + +import io +import json +import os +import plistlib +import subprocess +import sys +import tarfile +import tempfile +import unittest +import zipfile +from contextlib import redirect_stderr +from pathlib import Path +from unittest import mock + +import feed +import registry +import version + +KEY = "cHJvYmUta2V5LW5vdC1hLXJlYWwtb25l" +REPOSITORY = "TableProApp/TablePro" +GIT_ENV = { + "GIT_CONFIG_GLOBAL": os.devnull, + "GIT_CONFIG_NOSYSTEM": "1", + "GIT_AUTHOR_NAME": "Fixture", + "GIT_AUTHOR_EMAIL": "fixture@example.com", + "GIT_COMMITTER_NAME": "Fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.com", +} + +SIGNATURE = r''' +import hashlib, re +TRAILER = re.compile(r"\s*$", re.S) +def content(text): + return TRAILER.sub("", text) +def signed(text, key): + body = content(text) + signature = hashlib.sha256((key.strip() + body).encode()).hexdigest() + return body + "\n" +''' + +FAKE_GH = SIGNATURE + r''' +import json, os, shutil, sys +from pathlib import Path +args = sys.argv[1:] +with open(os.environ["FAKE_LOG"], "a") as log: + log.write(json.dumps(["gh", *args]) + "\n") +releases = Path(os.environ["FAKE_RELEASES"]) +if args[:2] == ["release", "view"]: + folder = releases / args[2] + if not folder.is_dir(): + sys.exit("release not found") + assets = [] + for path in sorted(folder.iterdir()): + digest = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest() + if path.name == os.environ.get("FAKE_BAD_DIGEST"): + digest = "sha256:" + "0" * 64 + assets.append({"name": path.name, "digest": digest, "state": "uploaded"}) + print(json.dumps({"isDraft": os.environ.get("FAKE_DRAFT") == "1", "assets": assets})) +elif args[:2] == ["release", "download"]: + out = Path(args[args.index("--dir") + 1]) + for index, arg in enumerate(args): + if arg == "--pattern": + shutil.copy(releases / args[2] / args[index + 1], out / args[index + 1]) +else: + sys.exit("fake gh cannot answer " + " ".join(args)) +''' + +FAKE_GENERATE_APPCAST = SIGNATURE + r''' +import json, os, subprocess, sys +from pathlib import Path +args = sys.argv[1:] +key = sys.stdin.read() +with open(os.environ["FAKE_LOG"], "a") as log: + log.write(json.dumps(["generate_appcast", *args, "stdin=" + key.strip()]) + "\n") +race = os.environ.get("FAKE_RACE_REPO") +if race and not os.path.exists(race + ".done"): + open(race + ".done", "w").close() + subprocess.run(["git", "-C", race, "push", "--quiet", "origin", "HEAD:refs/heads/feeds"], check=True) +def option(name): + return args[args.index(name) + 1] if name in args else None +stage = Path(args[-1]) +appcast = stage / "appcast.xml" +if os.environ.get("FAKE_PRUNE"): + appcast.write_text(content(appcast.read_text())) + print("Wrote 0 new updates") + sys.exit(0) +archive = next(stage.glob("*.tar.xz")) +short = archive.name[len("TablePro-"):-len(".tar.xz")] +lines = ["", "" + short + ""] +if option("--channel"): + lines.append("" + option("--channel") + "") +if option("--phased-rollout-interval"): + interval = option("--phased-rollout-interval") + lines.append("" + interval + "") +if option("--critical-update-version") is not None: + lines.append('') +lines.append("" + option("--versions") + "") +lines.append("" + short + "") +notes = (stage / ("TablePro-" + short + ".md")).read_text() +lines.append('') +signature = "" if os.environ.get("FAKE_UNSIGNED_ENCLOSURE") else ' sparkle:edSignature="c2ln"' +lines.append('") +lines.append("") +text = content(appcast.read_text()) +index = text.index("") if "" in text else text.index("") +appcast.write_text(text[:index] + "\n".join(lines) + "\n" + text[index:]) +print("Wrote 1 new update") +''' + +FAKE_SIGN_UPDATE = SIGNATURE + r''' +import json, os, sys +from pathlib import Path +args = sys.argv[1:] +key = sys.stdin.read() +with open(os.environ["FAKE_LOG"], "a") as log: + log.write(json.dumps(["sign_update", *args, "stdin=" + key.strip()]) + "\n") +path = Path(args[-1]) +text = path.read_text() +if "--verify" in args: + if os.environ.get("FAKE_VERIFY_FAIL") or signed(text, key) != text: + print("Error: failed to pass signing verification.") + sys.exit(1) + sys.exit(0) +path.write_text(signed(text, key)) +print("") +''' + +SEED = """ + + + TablePro + + 0.81.0 + + 140 + 0.81.0 + + + + + 0.80.0 + 21600 + 139 + 0.80.0 + + + + +""" +METADATA = { + "retainFromPluginKit": 39, + "plugins": { + "OracleDriver": {"name": "Oracle Driver", "iconName": "oracle-icon"}, + "ParquetExport": {"name": "Parquet Export", "iconName": "parquet-icon"}, + }, +} +PROJECT = """aggregateTargets: + BundledPlugins: + targets: + - MySQLDriver +""" + + +def item(build, short, channel=None, critical=None, signature="c2ln", length="10"): + lines = [""] + if channel is not None: + lines.append(f"{channel}") + if critical is not None: + lines.append(f'' if critical else + "") + lines += [f"{build}", + f"{short}"] + signed = f' sparkle:edSignature="{signature}"' if signature else "" + lines += [f'', ""] + return "\n".join(lines) + + +def appcast(*items): + return (f'' + f'TablePro{"".join(items)}').encode() + + +class FeedRulesTests(unittest.TestCase): + def test_a_clean_feed_passes(self): + self.assertEqual(feed.feed_problems(appcast(item(820001, "0.82.0-beta.1", "beta"), item(140, "0.81.0"))), []) + + def test_each_rule(self): + cases = [ + (appcast(item(140, "0.81.0", signature="")), "has no sparkle:edSignature"), + (appcast(item(140, "0.81.0", length="")), "has no length"), + (appcast(item("1.4", "0.81.0")), "sparkle:version '1.4' is not an integer"), + (appcast(item(140, "0.81")), "shortVersionString '0.81' is not a release version"), + (appcast(item(820001, "0.82.0-beta.1", "alpha")), "the only channel is beta"), + (appcast(item(820001, "0.82.0-beta.1", "beta", critical="")), "a beta marked critical"), + (appcast(item(141, "0.81.0"), item(140, "0.82.0")), "0.81.0 has the higher build (141 over 140)"), + (appcast(item(140, "0.81.0"), item(140, "0.80.0")), "build 140 carries 0.80.0, 0.81.0"), + ] + for data, message in cases: + with self.subTest(message=message): + self.assertTrue(any(message in problem for problem in feed.feed_problems(data)), + feed.feed_problems(data)) + + def test_the_last_default_channel_item_cannot_go(self): + before = appcast(item(140, "0.81.0")) + after = appcast(item(820001, "0.82.0-beta.1", "beta")) + self.assertIn("the feed no longer has a default-channel item", feed.feed_problems(after, before)) + + def test_the_new_item_must_be_present(self): + release = version.parse_tag("v0.82.0-beta.1") + url = "https://example.com/TablePro-0.82.0-beta.1.tar.xz" + self.assertEqual(feed.feed_problems(appcast(item(820001, "0.82.0-beta.1", "beta")), None, + feed.Expected(release, url)), []) + problems = feed.feed_problems(appcast(item(820001, "0.82.0-beta.1")), None, feed.Expected(release, url)) + self.assertEqual(problems, [f"no beta item for 0.82.0-beta.1 (820001) at {url}"]) + + def test_the_critical_build_carries_forward(self): + items = feed.parse_items(appcast(item(820099, "0.82.0", critical="140"), item(140, "0.81.0", critical=""), + item(830001, "0.83.0-beta.1", "beta"))) + self.assertEqual(feed.carried_critical(items), 140) + self.assertEqual(feed.carried_critical(feed.parse_items(SEED.encode())), 140) + self.assertIsNone(feed.carried_critical(feed.parse_items(appcast(item(139, "0.80.0"))))) + + def test_the_generate_appcast_arguments(self): + stage = Path("/stage") + prefix = ["--ed-key-file", "-", "--embed-release-notes", "--maximum-deltas", "0"] + beta = feed.appcast_arguments(stage, "v0.82.0-beta.1", version.parse_tag("v0.82.0-beta.1"), None, REPOSITORY) + self.assertEqual(beta, prefix + [ + "--versions", "820001", + "--download-url-prefix", "https://github.com/TableProApp/TablePro/releases/download/v0.82.0-beta.1/", + "--full-release-notes-url", "https://github.com/TableProApp/TablePro/releases/tag/v0.82.0-beta.1", + "--channel", "beta", "/stage"]) + stable = feed.appcast_arguments(stage, "v0.82.0", version.parse_tag("v0.82.0"), 140, REPOSITORY) + self.assertEqual(stable[stable.index("--phased-rollout-interval"):], + ["--phased-rollout-interval", "21600", "--critical-update-version", "140", "/stage"]) + plain = feed.appcast_arguments(stage, "v0.82.1", version.parse_tag("v0.82.1"), None, REPOSITORY) + self.assertNotIn("--critical-update-version", plain) + self.assertNotIn("--channel", plain) + + def test_removes_items_by_version_and_keeps_the_rest_of_the_text(self): + text = SEED + edited, removed = feed.remove_items(text, "0.80.0") + self.assertEqual(removed, 1) + self.assertEqual([each.short_version for each in feed.parse_items(edited.encode())], ["0.81.0"]) + self.assertEqual(edited, text[:text.index(" \n 0.80.0")] + + text[text.index(" </channel>"):]) + + def test_refuses_to_edit_text_it_cannot_match(self): + text = SEED.replace("# 0.81.0", "# 0.81.0 </item> in the notes") + with self.assertRaises(feed.FeedError): + feed.remove_items(text, "0.81.0") + + +def git(path, *args): + return subprocess.run(["git", "-C", str(path), *args], env={**os.environ, **GIT_ENV}, check=True, + capture_output=True, text=True).stdout.strip() + + +def fake_signed(text, key=KEY): + namespace = {} + exec(SIGNATURE, namespace) + return namespace["signed"](text, key) + + +def app_archive(path, text, build): + info = plistlib.dumps({"CFBundleShortVersionString": text, "CFBundleVersion": str(build)}) + with tarfile.open(path, "w:xz") as archive: + member = tarfile.TarInfo("TablePro.app/Contents/Info.plist") + member.size = len(info) + archive.addfile(member, io.BytesIO(info)) + + +def plugin_zip(path, target, text, build, kit=39, **extra): + info = {"CFBundleIdentifier": f"com.TablePro.{target}", "CFBundleShortVersionString": text, + "CFBundleVersion": str(build), "TableProPluginKitVersion": kit, "LSMinimumSystemVersion": "13.0", + **extra} + with zipfile.ZipFile(path, "w") as archive: + archive.writestr(f"{target}.tableplugin/Contents/Info.plist", plistlib.dumps(info)) + + +class FeedBranchTests(unittest.TestCase): + def setUp(self): + directory = tempfile.TemporaryDirectory() + self.addCleanup(directory.cleanup) + self.base = Path(directory.name) + self.tools = self.base / "bin" + self.tools.mkdir() + for name, source in (("gh", FAKE_GH), ("generate_appcast", FAKE_GENERATE_APPCAST), + ("sign_update", FAKE_SIGN_UPDATE)): + path = self.tools / name + path.write_text(f"#!{sys.executable}\n{source}") + path.chmod(0o755) + self.log_path = self.base / "calls.log" + self.releases = self.base / "releases" + environment = mock.patch.dict(os.environ, { + **GIT_ENV, + "PATH": f"{self.tools}{os.pathsep}{os.environ['PATH']}", + "FAKE_LOG": str(self.log_path), + "FAKE_RELEASES": str(self.releases), + }) + environment.start() + self.addCleanup(environment.stop) + + self.origin = self.base / "origin.git" + git(self.base, "init", "--quiet", "--bare", "--initial-branch=main", str(self.origin)) + seed = self.base / "seed" + git(self.base, "init", "--quiet", "--initial-branch=feeds", str(seed)) + (seed / "appcast.xml").write_text(SEED) + (seed / "plugins").mkdir() + (seed / "plugins/v3.json").write_text(json.dumps(registry.empty_manifest(39))) + git(seed, "add", ".") + git(seed, "commit", "--quiet", "-m", "seed") + git(seed, "push", "--quiet", str(self.origin), "feeds") + + self.root = self.base / "tag" + git(self.base, "init", "--quiet", "--initial-branch=main", str(self.root)) + (self.root / ".github").mkdir() + (self.root / ".github/plugin-registry.json").write_text(json.dumps(METADATA)) + (self.root / "Configs").mkdir() + (self.root / "Configs/PluginKit.xcconfig").write_text( + "TABLEPRO_PLUGINKIT_VERSION = 39\nTABLEPRO_PLUGINKIT_MINIMUM_VERSION = 19\n") + (self.root / "project.yml").write_text(PROJECT) + git(self.root, "add", ".") + git(self.root, "commit", "--quiet", "-m", "tag") + git(self.root, "remote", "add", "origin", str(self.origin)) + self.notes = self.base / "feed.md" + self.notes.write_text("- A fix. (#1)\n") + + def publish(self, tag, kit=39, archive=None): + release = version.parse_tag(tag) + folder = self.releases / tag + folder.mkdir(parents=True) + app_archive(folder / f"TablePro-{release}.tar.xz", *(archive or (str(release), release.build))) + plugin_zip(folder / f"OracleDriver-{release}.zip", "OracleDriver", str(release), release.build, kit, + TableProProvidesDatabaseTypeIds=["Oracle"]) + plugin_zip(folder / f"ParquetExport-{release}.zip", "ParquetExport", str(release), release.build, kit, + TableProProvidesExportFormatIds=["parquet"]) + + def add(self, tag, critical=False): + release = version.parse_tag(tag) + messages = [] + changed = feed.add(self.root, tag, release.build, release.channel, self.notes, critical, self.tools, KEY, + feed.Releases(REPOSITORY), REPOSITORY, log=messages.append) + return changed, messages + + def replace_appcast(self, text): + clone = self.base / "replace" + git(self.base, "clone", "--quiet", "--branch", "feeds", str(self.origin), str(clone)) + (clone / "appcast.xml").write_text(text) + git(clone, "commit", "--quiet", "-am", "replace the appcast") + git(clone, "push", "--quiet", "origin", "feeds") + + def calls(self, tool): + if not self.log_path.exists(): + return [] + return [call[1:] for call in map(json.loads, self.log_path.read_text().splitlines()) if call[0] == tool] + + def published(self, path): + return subprocess.run(["git", "-C", str(self.origin), "show", f"feeds:{path}"], check=True, + capture_output=True, text=True).stdout + + def history(self): + return git(self.origin, "log", "--format=%an <%ae> %s", "feeds").splitlines() + + def test_adds_a_beta_and_its_plugins(self): + self.publish("v0.82.0-beta.1") + changed, _ = self.add("v0.82.0-beta.1") + self.assertTrue(changed) + self.assertEqual(self.history(), [ + "github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> release: v0.82.0-beta.1", + "Fixture <fixture@example.com> seed", + ]) + (generate,) = self.calls("generate_appcast") + self.assertEqual(generate[-1], f"stdin={KEY}") + signing = [call for call in self.calls("sign_update") if "--verify" not in call] + self.assertEqual([call[:2] + call[-1:] for call in signing], [["--ed-key-file", "-", f"stdin={KEY}"]]) + self.assertIn("--channel", generate) + self.assertNotIn("--critical-update-version", generate) + self.assertNotIn("--phased-rollout-interval", generate) + items = feed.parse_items(self.published("appcast.xml").encode()) + self.assertEqual([(each.short_version, each.channel) for each in items], + [("0.82.0-beta.1", "beta"), ("0.81.0", None), ("0.80.0", None)]) + manifest = json.loads(self.published("plugins/v3.json")) + self.assertEqual([plugin["id"] for plugin in manifest["plugins"]], + ["com.TablePro.OracleDriver", "com.TablePro.ParquetExport"]) + oracle = manifest["plugins"][0] + self.assertEqual(oracle["databaseTypeIds"], ["Oracle"]) + self.assertEqual(oracle["releases"][0]["url"], + "https://github.com/TableProApp/TablePro/releases/download/v0.82.0-beta.1/" + "OracleDriver-0.82.0-beta.1.zip") + self.assertEqual(oracle["releases"][0]["channel"], "beta") + + def test_a_stable_carries_the_critical_build_and_phases(self): + self.publish("v0.82.0") + self.add("v0.82.0") + (generate,) = self.calls("generate_appcast") + self.assertEqual(generate[generate.index("--phased-rollout-interval") + 1], "21600") + self.assertEqual(generate[generate.index("--critical-update-version") + 1], "140") + + def test_a_critical_stable_is_critical_below_its_own_build(self): + self.publish("v0.82.0") + self.add("v0.82.0", critical=True) + (generate,) = self.calls("generate_appcast") + self.assertEqual(generate[generate.index("--critical-update-version") + 1], "820099") + + def test_a_critical_beta_is_refused(self): + with self.assertRaisesRegex(feed.FeedError, "a beta is never critical"): + self.add("v0.82.0-beta.1", critical=True) + + def test_a_build_or_channel_that_disagrees_with_the_tag_is_refused(self): + with self.assertRaisesRegex(feed.FeedError, "is build 820001 on beta"): + feed.add(self.root, "v0.82.0-beta.1", 820099, "stable", self.notes, False, self.tools, KEY, + feed.Releases(REPOSITORY), REPOSITORY, log=lambda message: None) + + def test_a_rerun_changes_nothing(self): + self.publish("v0.82.0-beta.1") + self.add("v0.82.0-beta.1") + head = git(self.origin, "rev-parse", "feeds") + changed, messages = self.add("v0.82.0-beta.1") + self.assertFalse(changed) + self.assertEqual(git(self.origin, "rev-parse", "feeds"), head) + self.assertEqual(len(self.calls("generate_appcast")), 1) + self.assertIn("appcast.xml already has 0.82.0-beta.1 (820001)", messages) + + def test_starts_again_when_another_run_pushed_first(self): + self.publish("v0.82.0-beta.1") + race = self.base / "race" + git(self.base, "clone", "--quiet", "--branch", "feeds", str(self.origin), str(race)) + (race / "other.txt").write_text("another run\n") + git(race, "add", "other.txt") + git(race, "commit", "--quiet", "-m", "release: v0.81.1") + with mock.patch.dict(os.environ, {"FAKE_RACE_REPO": str(race)}): + changed, messages = self.add("v0.82.0-beta.1") + self.assertTrue(changed) + self.assertEqual([line.split("> ")[1] for line in self.history()], + ["release: v0.82.0-beta.1", "release: v0.81.1", "seed"]) + self.assertEqual(len(self.calls("generate_appcast")), 2) + self.assertTrue(any("attempt 1 of 3" in message for message in messages)) + self.assertEqual(self.published("other.txt"), "another run\n") + + def test_an_archive_of_another_build_is_refused(self): + self.publish("v0.82.0-beta.1", archive=("0.82.0-beta.1", 140)) + with self.assertRaisesRegex(feed.FeedError, r"holds 0.82.0-beta.1 \(140\), not 0.82.0-beta.1 \(820001\)"): + self.add("v0.82.0-beta.1") + self.assertEqual(self.calls("generate_appcast"), []) + + def test_an_item_sparkle_prunes_on_arrival_leaves_the_appcast_alone(self): + self.replace_appcast(fake_signed(SEED)) + self.publish("v0.79.1") + with mock.patch.dict(os.environ, {"FAKE_PRUNE": "1"}): + changed, messages = self.add("v0.79.1") + self.assertTrue(changed) + self.assertEqual(self.published("appcast.xml"), fake_signed(SEED)) + self.assertIn("generate_appcast keeps newer items on 0.79.1's branch, so appcast.xml stays as it is", messages) + self.assertEqual(len(json.loads(self.published("plugins/v3.json"))["plugins"]), 2) + + def test_a_failed_check_pushes_nothing(self): + self.publish("v0.82.0-beta.1") + with mock.patch.dict(os.environ, {"FAKE_UNSIGNED_ENCLOSURE": "1"}): + with self.assertRaisesRegex(feed.FeedError, "0.82.0-beta.1's enclosure has no sparkle:edSignature"): + self.add("v0.82.0-beta.1") + self.assertEqual(len(self.history()), 1) + + def test_a_rejected_signature_pushes_nothing(self): + self.publish("v0.82.0-beta.1") + with mock.patch.dict(os.environ, {"FAKE_VERIFY_FAIL": "1"}): + with self.assertRaisesRegex(feed.FeedError, "sign_update failed"): + self.add("v0.82.0-beta.1") + self.assertEqual(len(self.history()), 1) + + def test_a_wrong_digest_is_refused(self): + self.publish("v0.82.0-beta.1") + with mock.patch.dict(os.environ, {"FAKE_BAD_DIGEST": "TablePro-0.82.0-beta.1.tar.xz"}): + with self.assertRaisesRegex(feed.FeedError, "but v0.82.0-beta.1 lists sha256:0000"): + self.add("v0.82.0-beta.1") + self.assertEqual(self.calls("generate_appcast"), []) + + def test_a_draft_is_refused(self): + self.publish("v0.82.0-beta.1") + with mock.patch.dict(os.environ, {"FAKE_DRAFT": "1"}): + with self.assertRaisesRegex(feed.FeedError, "is a draft"): + self.add("v0.82.0-beta.1") + + def test_a_plugin_built_against_another_kit_is_refused(self): + self.publish("v0.82.0-beta.1", kit=38) + with self.assertRaisesRegex(registry.RegistryError, "TableProPluginKitVersion is 38"): + self.add("v0.82.0-beta.1") + self.assertEqual(len(self.history()), 1) + + def test_withdraw_removes_the_version_and_re_signs(self): + self.publish("v0.82.0-beta.1") + self.publish("v0.82.0") + self.add("v0.82.0-beta.1") + self.add("v0.82.0") + signed_before = sum("--verify" not in call for call in self.calls("sign_update")) + self.assertTrue(feed.withdraw(self.root, "0.82.0", self.tools, KEY, log=lambda message: None)) + self.assertEqual(self.history()[0].split("> ")[1], "withdraw: v0.82.0") + published = self.published("appcast.xml") + self.assertEqual([each.short_version for each in feed.parse_items(published.encode())], + ["0.82.0-beta.1", "0.81.0", "0.80.0"]) + self.assertEqual(fake_signed(published), published) + self.assertEqual(sum("--verify" not in call for call in self.calls("sign_update")), signed_before + 1) + self.assertEqual(json.loads(self.published("plugins/v3.json"))["plugins"], []) + self.assertFalse(feed.withdraw(self.root, "0.82.0", self.tools, KEY, log=lambda message: None)) + + def test_withdraw_keeps_a_default_channel_item(self): + self.replace_appcast(fake_signed(feed.remove_items(SEED, "0.80.0")[0])) + with self.assertRaisesRegex(feed.FeedError, "no default-channel item"): + feed.withdraw(self.root, "0.81.0", self.tools, KEY, log=lambda message: None) + + def test_check_reads_the_published_branch(self): + self.assertTrue(any("sign_update failed" in problem for problem in feed.check(self.root, self.tools, KEY))) + self.publish("v0.82.0-beta.1") + self.add("v0.82.0-beta.1") + self.assertEqual(feed.check(self.root, self.tools, KEY), []) + + def test_main_needs_the_sparkle_tools_and_key(self): + with mock.patch.dict(os.environ, {"SPARKLE_BIN": "", "SPARKLE_PRIVATE_KEY": KEY}), \ + redirect_stderr(io.StringIO()) as errors: + self.assertEqual(feed.main(["check"]), 1) + self.assertIn("SPARKLE_BIN is not set", errors.getvalue()) + with mock.patch.dict(os.environ, {"SPARKLE_BIN": str(self.tools), "SPARKLE_PRIVATE_KEY": ""}), \ + redirect_stderr(io.StringIO()) as errors: + self.assertEqual(feed.main(["check"]), 1) + self.assertIn("SPARKLE_PRIVATE_KEY is not set", errors.getvalue()) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/release/test_publish.py b/scripts/release/test_publish.py new file mode 100644 index 0000000000..cdcb6cb161 --- /dev/null +++ b/scripts/release/test_publish.py @@ -0,0 +1,299 @@ +#!/usr/bin/env python3 + +import hashlib +import io +import json +import os +import sys +import tempfile +import unittest +from contextlib import redirect_stderr, redirect_stdout +from pathlib import Path +from unittest import mock + +import feed +import publish +import version + +REPOSITORY = "TableProApp/TablePro" + +FAKE_GH = r''' +import hashlib, json, os, sys +from pathlib import Path +args = sys.argv[1:] +state_path = Path(os.environ["FAKE_STATE"]) +state = json.loads(state_path.read_text()) +state["calls"].append(args) +releases = state["releases"] + +def save(): + state_path.write_text(json.dumps(state)) + +def fail(message): + save() + sys.exit(message) + +def option(name): + return args[args.index(name) + 1] if name in args else None + +def files(): + return [arg for arg in args[3:] if os.path.isfile(arg) and arg != option("--notes-file")] + +def upload(release, paths): + for path in paths: + digest = "sha256:" + hashlib.sha256(Path(path).read_bytes()).hexdigest() + if Path(path).name == os.environ.get("FAKE_CORRUPT"): + digest = "sha256:" + "0" * 64 + release["assets"][Path(path).name] = digest + +if args[0] == "api": + path = args[1] + if path.endswith("/releases/latest"): + if state["latest"] is None: + fail("gh: Not Found (HTTP 404)") + print(state["latest"]) + else: + tag = path.rsplit("/", 1)[1] + if tag not in releases or releases[tag]["draft"]: + fail("gh: Not Found (HTTP 404)") + print("false") +elif args[:2] == ["release", "view"]: + release = releases.get(args[2]) + if release is None: + fail("release not found") + if state.get("hide_drafts", 0) > 0: + state["hide_drafts"] -= 1 + fail("release not found") + assets = [{"name": name, "digest": digest, "state": "uploaded"} for name, digest in release["assets"].items()] + print(json.dumps({"isDraft": release["draft"], "assets": assets})) +elif args[:2] == ["release", "create"]: + if args[2] not in state["tags"]: + fail("tag " + args[2] + " doesn't exist in the repo") + release = {"draft": "--draft" in args, "prerelease": "--prerelease=true" in args, "latest": None, "assets": {}} + releases[args[2]] = release + upload(release, files()) +elif args[:2] == ["release", "upload"]: + upload(releases[args[2]], files()) +elif args[:2] == ["release", "edit"]: + release = releases[args[2]] + for arg in args[3:]: + if arg == "--draft=false": + release["draft"] = False + elif arg in ("--prerelease", "--prerelease=true"): + release["prerelease"] = True + elif arg == "--prerelease=false": + release["prerelease"] = False + elif arg == "--latest=true": + state["latest"] = args[2] + elif arg == "--latest=false": + release["latest"] = False +else: + fail("fake gh cannot answer " + " ".join(args)) +save() +''' + + +def items(*pairs): + return [feed.Item(build=str(build), short_version=short, channel=channel, is_critical=False, + critical_version=None, url="https://example.com", signature="c2ln", length="1") + for build, short, channel in pairs] + + +class DecisionTests(unittest.TestCase): + def test_the_draft_action(self): + self.assertEqual(publish.draft_action(True, False), "published") + self.assertEqual(publish.draft_action(True, True), "published") + self.assertEqual(publish.draft_action(False, True), "reuse") + self.assertEqual(publish.draft_action(False, False), "create") + + def test_latest_goes_only_to_the_highest_stable(self): + cases = [ + ("v0.82.0", "v0.81.0", True), + ("v0.82.0", None, True), + ("v0.82.0", "plugin-oracle-v1.0.0", True), + ("v0.81.1", "v0.82.0", False), + ("v0.82.0", "v0.82.0", False), + ("v0.83.0-beta.1", "v0.82.0", False), + ] + for tag, latest, expected in cases: + with self.subTest(tag=tag, latest=latest): + self.assertEqual(publish.should_be_latest(version.parse_tag(tag), latest), expected) + + def test_the_release_flags(self): + self.assertEqual(publish.release_flags(version.parse_tag("v0.82.0-beta.1"), "v0.81.0"), + ["--draft=false", "--prerelease", "--latest=false"]) + self.assertEqual(publish.release_flags(version.parse_tag("v0.82.0"), "v0.81.0"), + ["--draft=false", "--latest=true"]) + self.assertEqual(publish.release_flags(version.parse_tag("v0.81.1"), "v0.82.0"), + ["--draft=false", "--latest=false"]) + + def test_the_highest_stable_in_the_feed(self): + feed_items = items((830001, "0.83.0-beta.1", "beta"), (820199, "0.82.1", None), (820099, "0.82.0", None), + (140, "0.81.0", None)) + self.assertEqual(str(publish.highest_stable(feed_items)), "0.82.1") + self.assertEqual(str(publish.highest_stable(items((140, "0.81.0", None), (139, "0.80.0", None)))), "0.81.0") + self.assertIsNone(publish.highest_stable(items((830001, "0.83.0-beta.1", "beta")))) + + def test_asset_problems(self): + assets = [{"name": "a.zip", "digest": "sha256:" + "1" * 64, "state": "uploaded"}, + {"name": "b.zip", "digest": "sha256:" + "2" * 64, "state": "starter"}] + self.assertEqual(publish.asset_problems({"a.zip": "1" * 64}, assets), []) + self.assertEqual(publish.asset_problems({"a.zip": "3" * 64, "b.zip": "2" * 64, "c.zip": "4" * 64}, assets), [ + f"a.zip is sha256:{'1' * 64} on GitHub but sha256:{'3' * 64} here", + "b.zip is starter, not uploaded", + "the release has no c.zip", + ]) + + +class GitHubTests(unittest.TestCase): + def setUp(self): + directory = tempfile.TemporaryDirectory() + self.addCleanup(directory.cleanup) + self.base = Path(directory.name) + tools = self.base / "bin" + tools.mkdir() + (tools / "gh").write_text(f"#!{sys.executable}\n{FAKE_GH}") + (tools / "gh").chmod(0o755) + self.state_path = self.base / "state.json" + self.state({"releases": {"v0.81.0": {"draft": False, "prerelease": False, "assets": {}}}, + "latest": "v0.81.0", "tags": ["v0.81.0", "v0.82.0-beta.1", "v0.82.0", "v0.81.1"], "calls": []}) + environment = mock.patch.dict(os.environ, {"PATH": f"{tools}{os.pathsep}{os.environ['PATH']}", + "FAKE_STATE": str(self.state_path), + "GITHUB_REPOSITORY": REPOSITORY}) + environment.start() + self.addCleanup(environment.stop) + self.github = publish.GitHub(REPOSITORY) + self.notes = self.base / "body.md" + self.notes.write_text("Notes\n") + self.files = [] + for name in ("TablePro-0.82.0.dmg", "TablePro-0.82.0.tar.xz", "OracleDriver-0.82.0.zip"): + path = self.base / name + path.write_bytes(name.encode()) + self.files.append(path) + self.log = [] + + def state(self, value=None): + if value is not None: + self.state_path.write_text(json.dumps(value)) + return json.loads(self.state_path.read_text()) + + def verbs(self): + return [call[:2] if call[0] == "release" else call[:1] for call in self.state()["calls"]] + + def draft(self, tag="v0.82.0"): + return publish.draft(self.github, tag, self.notes, self.files, sleep=lambda seconds: None, + log=self.log.append) + + def digests(self): + return {path.name: "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest() for path in self.files} + + def test_creates_a_draft_and_checks_its_digests(self): + self.assertEqual(self.draft(), "draft") + release = self.state()["releases"]["v0.82.0"] + self.assertEqual((release["draft"], release["prerelease"], release["assets"]), (True, False, self.digests())) + create = next(call for call in self.state()["calls"] if call[:2] == ["release", "create"]) + self.assertEqual(create[:9], ["release", "create", "v0.82.0", "--draft", "--verify-tag", "--title", "v0.82.0", + "--notes-file", str(self.notes)]) + + def test_a_beta_draft_is_a_prerelease(self): + self.files = [self.base / "TablePro-0.82.0.dmg"] + self.draft("v0.82.0-beta.1") + self.assertTrue(self.state()["releases"]["v0.82.0-beta.1"]["prerelease"]) + + def test_a_published_release_is_left_alone(self): + state = self.state() + state["releases"]["v0.82.0"] = {"draft": False, "prerelease": False, "assets": {}} + self.state(state) + self.assertEqual(self.draft(), "published") + self.assertEqual(self.verbs(), [["api"]]) + + def test_an_earlier_draft_is_reused(self): + self.draft() + state = self.state() + state["calls"] = [] + self.state(state) + self.files[0].write_bytes(b"rebuilt") + self.assertEqual(self.draft(), "draft") + self.assertNotIn(["release", "create"], self.verbs()) + self.assertIn(["release", "upload"], self.verbs()) + self.assertEqual(self.state()["releases"]["v0.82.0"]["assets"], self.digests()) + + def test_waits_for_a_new_draft_to_appear(self): + state = self.state() + state["hide_drafts"] = 2 + self.state(state) + self.assertEqual(self.draft(), "draft") + self.assertEqual(self.verbs().count(["release", "view"]), 4) + + def test_a_draft_that_never_appears_is_an_error(self): + state = self.state() + state["hide_drafts"] = publish.FIND_ATTEMPTS + self.state(state) + with self.assertRaisesRegex(publish.PublishError, "cannot be found"): + self.draft() + + def test_a_wrong_digest_keeps_the_draft_unpublished(self): + with mock.patch.dict(os.environ, {"FAKE_CORRUPT": "OracleDriver-0.82.0.zip"}): + with self.assertRaisesRegex(publish.PublishError, "OracleDriver-0.82.0.zip is sha256:0000"): + self.draft() + self.assertTrue(self.state()["releases"]["v0.82.0"]["draft"]) + + def test_a_missing_file_is_refused_before_any_call(self): + self.files.append(self.base / "missing.zip") + with self.assertRaisesRegex(publish.PublishError, "missing.zip does not exist"): + self.draft() + self.assertEqual(self.state()["calls"], []) + + def test_publishes_a_beta_as_a_prerelease_that_is_never_latest(self): + self.files = [self.base / "TablePro-0.82.0.dmg"] + self.draft("v0.82.0-beta.1") + publish.release(self.github, "v0.82.0-beta.1", self.log.append) + state = self.state() + self.assertEqual(state["releases"]["v0.82.0-beta.1"]["draft"], False) + self.assertEqual(state["latest"], "v0.81.0") + self.assertEqual(state["calls"][-1], ["release", "edit", "v0.82.0-beta.1", "--draft=false", "--prerelease", + "--latest=false", "--repo", REPOSITORY]) + + def test_a_stable_above_latest_becomes_latest(self): + self.draft() + publish.release(self.github, "v0.82.0", self.log.append) + self.assertEqual(self.state()["latest"], "v0.82.0") + + def test_a_patch_on_an_older_line_is_not_latest(self): + state = self.state() + state["latest"] = "v0.82.0" + self.state(state) + self.draft("v0.81.1") + publish.release(self.github, "v0.81.1", self.log.append) + self.assertEqual(self.state()["latest"], "v0.82.0") + self.assertIn("--latest=false", self.state()["calls"][-1]) + + def test_releasing_twice_edits_once(self): + self.draft() + publish.release(self.github, "v0.82.0", self.log.append) + publish.release(self.github, "v0.82.0", self.log.append) + self.assertEqual(sum(call[:2] == ["release", "edit"] for call in self.state()["calls"]), 1) + + def test_releasing_without_a_draft_is_refused(self): + with self.assertRaisesRegex(publish.PublishError, "has no draft"): + publish.release(self.github, "v0.82.0", self.log.append) + + def test_latest_marks_the_highest_stable_left_in_the_feed(self): + feed_items = items((830001, "0.83.0-beta.1", "beta"), (140, "0.81.0", None)) + self.assertEqual(publish.latest(self.github, feed_items, self.log.append), "v0.81.0") + self.assertEqual(self.state()["calls"][-1], ["release", "edit", "v0.81.0", "--latest=true", + "--repo", REPOSITORY]) + + def test_draft_prints_only_the_state_on_stdout(self): + arguments = ["draft", "v0.82.0", "--notes", str(self.notes), *[str(path) for path in self.files]] + with redirect_stdout(io.StringIO()) as output, redirect_stderr(io.StringIO()): + self.assertEqual(publish.main(arguments), 0) + self.assertEqual(output.getvalue(), "state=draft\n") + with redirect_stdout(io.StringIO()) as output, redirect_stderr(io.StringIO()): + self.assertEqual(publish.main(["release", "v0.82.0"]), 0) + self.assertEqual(publish.main(arguments), 0) + self.assertEqual(output.getvalue(), "state=published\n") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/release/test_registry.py b/scripts/release/test_registry.py new file mode 100644 index 0000000000..53eec9d538 --- /dev/null +++ b/scripts/release/test_registry.py @@ -0,0 +1,327 @@ +#!/usr/bin/env python3 + +import io +import json +import plistlib +import tempfile +import unittest +import zipfile +from contextlib import redirect_stderr, redirect_stdout +from pathlib import Path + +import registry +import version + +URL_BASE = "https://github.com/TableProApp/TablePro/releases/download/v0.82.0/" +PROJECT = """\ +targets: + OracleDriver: + templates: [DriverPlugin] + +aggregateTargets: + BundledPlugins: + targets: + - MySQLDriver + - SQLiteDriver + + RegistryPlugins: + targets: + - OracleDriver + - ParquetExport + scheme: {} +""" +METADATA = { + "retainFromPluginKit": 39, + "plugins": { + "OracleDriver": {"name": "Oracle Driver", "iconName": "oracle-icon"}, + "ParquetExport": {"name": "Parquet Export", "iconName": "parquet-icon"}, + }, +} + + +def entry(text="0.82.0", kit=39, system="13.0", sha="a", plugin="com.TablePro.OracleDriver", name="Oracle Driver"): + return { + "id": plugin, "name": name, "iconName": "oracle-icon", "databaseTypeIds": ["Oracle"], + "version": text, "build": version.parse_version(text).build, "pluginKit": kit, + "minimumSystemVersion": system, "file": f"OracleDriver-{text}.zip", "sha256": sha * 64, + } + + +def select(manifest, plugin, channels, kit, system): + def system_key(text): + return tuple(int(part) for part in text.split(".")) + + releases = next(entry["releases"] for entry in manifest["plugins"] if entry["id"] == plugin) + eligible = [release for release in releases if release["channel"] in channels and release["pluginKit"] == kit + and system_key(release["minimumSystemVersion"]) <= system_key(system)] + return max(eligible, key=lambda release: release["build"])["version"] if eligible else None + + +class ManifestCase(unittest.TestCase): + def setUp(self): + directory = tempfile.TemporaryDirectory() + self.addCleanup(directory.cleanup) + self.path = Path(directory.name) / "plugins" / "v3.json" + self.log = [] + + def add(self, entries, channel="stable", retain_from=39): + return registry.add(self.path, entries, channel, URL_BASE, retain_from, self.log.append) + + def manifest(self): + return json.loads(self.path.read_text(encoding="utf-8")) + + def builds(self, plugin="com.TablePro.OracleDriver"): + return [release["build"] for entry in self.manifest()["plugins"] if entry["id"] == plugin + for release in entry["releases"]] + + +class AddTests(ManifestCase): + def test_creates_the_manifest_and_an_entry(self): + self.assertTrue(self.add([entry()])) + self.assertEqual(self.manifest(), { + "schemaVersion": 3, + "retainFromPluginKit": 39, + "plugins": [{ + "id": "com.TablePro.OracleDriver", "name": "Oracle Driver", "iconName": "oracle-icon", + "databaseTypeIds": ["Oracle"], + "releases": [{ + "version": "0.82.0", "build": 820099, "channel": "stable", "pluginKit": 39, + "minimumSystemVersion": "13.0", "url": URL_BASE + "OracleDriver-0.82.0.zip", "sha256": "a" * 64, + }], + }], + }) + + def test_a_rerun_is_a_no_op(self): + self.add([entry()]) + written = self.path.read_bytes() + self.assertFalse(self.add([entry()])) + self.assertEqual(self.path.read_bytes(), written) + + def test_a_changed_sha256_for_the_same_build_is_an_error(self): + self.add([entry()]) + with self.assertRaisesRegex(registry.RegistryError, "already has build 820099"): + self.add([entry(sha="b")]) + + def test_keeps_two_builds_per_channel_kit_and_system(self): + for text in ("0.82.0", "0.82.1", "0.82.2"): + self.add([entry(text)]) + self.assertEqual(self.builds(), [820299, 820199]) + + def test_a_beta_goes_once_a_higher_stable_shares_its_kit_and_system(self): + self.add([entry("0.82.0-beta.1")], channel="beta") + self.add([entry("0.82.0-beta.2")], channel="beta") + self.assertEqual(self.builds(), [820002, 820001]) + self.add([entry("0.82.0")]) + self.assertEqual(self.builds(), [820099]) + + def test_a_beta_on_another_kit_survives_a_stable(self): + self.add([entry("0.83.0-beta.1", kit=40)], channel="beta") + self.add([entry("0.82.1")]) + self.assertEqual(self.builds(), [830001, 820199]) + + def test_a_higher_minimum_system_keeps_the_older_build(self): + self.add([entry("0.82.0")]) + self.add([entry("0.82.1", system="14.0")]) + self.add([entry("0.82.2", system="14.0")]) + self.assertEqual(self.builds(), [820299, 820199, 820099]) + manifest = self.manifest() + self.assertEqual(select(manifest, "com.TablePro.OracleDriver", {"stable"}, 39, "13.5"), "0.82.0") + self.assertEqual(select(manifest, "com.TablePro.OracleDriver", {"stable"}, 39, "14.1"), "0.82.2") + + def test_releases_below_the_floor_are_pruned(self): + self.add([entry("0.82.0", kit=38)], retain_from=38) + self.add([entry("0.83.0", kit=39)], retain_from=39) + self.assertEqual(self.builds(), [830099]) + self.assertEqual(self.manifest()["retainFromPluginKit"], 39) + + def test_an_older_line_never_lowers_the_floor(self): + self.add([entry("0.83.0", kit=40)], retain_from=40) + self.add([entry("0.82.1", kit=40)], retain_from=39) + self.assertEqual(self.manifest()["retainFromPluginKit"], 40) + + def test_a_release_pruned_on_arrival_is_a_logged_no_op(self): + self.add([entry("0.83.0", kit=40)], retain_from=40) + written = self.path.read_bytes() + self.assertFalse(self.add([entry("0.82.1", kit=39)], retain_from=39)) + self.assertEqual(self.path.read_bytes(), written) + self.assertIn("com.TablePro.OracleDriver 0.82.1: pruned on arrival by retention, not added", self.log) + + def test_metadata_comes_only_from_the_highest_build(self): + newer = entry("0.83.0-beta.1", kit=40) + newer["name"] = "Oracle" + self.add([newer], channel="beta") + older = entry("0.82.1") + older["name"] = "Oracle Driver (old)" + self.add([older]) + self.assertEqual(self.manifest()["plugins"][0]["name"], "Oracle") + newest = entry("0.83.0", kit=40) + newest["name"] = "Oracle Database" + self.add([newest]) + self.assertEqual(self.manifest()["plugins"][0]["name"], "Oracle Database") + + def test_plugins_sort_by_id_and_releases_by_build(self): + self.add([entry("0.82.0", plugin="com.TablePro.ZDriver"), entry("0.82.0", plugin="com.TablePro.ADriver")]) + self.add([entry("0.82.1", plugin="com.TablePro.ADriver")]) + manifest = self.manifest() + self.assertEqual([plugin["id"] for plugin in manifest["plugins"]], + ["com.TablePro.ADriver", "com.TablePro.ZDriver"]) + self.assertEqual([release["build"] for release in manifest["plugins"][0]["releases"]], [820199, 820099]) + + def test_another_schema_is_refused(self): + self.path.parent.mkdir(parents=True) + self.path.write_text(json.dumps({"schemaVersion": 4, "retainFromPluginKit": 39, "plugins": []})) + with self.assertRaisesRegex(registry.RegistryError, "schemaVersion 4"): + self.add([entry()]) + + def test_a_build_that_is_not_the_versions_build_is_refused(self): + wrong = entry() + wrong["build"] = 140 + with self.assertRaisesRegex(registry.RegistryError, "build 140 is not 820099"): + self.add([wrong]) + + def test_a_version_on_the_wrong_channel_is_refused(self): + with self.assertRaisesRegex(registry.RegistryError, "belongs to the beta channel"): + self.add([entry("0.82.0-beta.1")], channel="stable") + + +class WithdrawTests(ManifestCase): + def test_a_withdraw_brings_the_previous_build_back_into_selection(self): + self.add([entry("0.82.0")]) + self.add([entry("0.82.1")]) + self.assertEqual(select(self.manifest(), "com.TablePro.OracleDriver", {"stable"}, 39, "26.0"), "0.82.1") + self.assertTrue(registry.withdraw(self.path, "0.82.1", self.log.append)) + self.assertEqual(select(self.manifest(), "com.TablePro.OracleDriver", {"stable"}, 39, "26.0"), "0.82.0") + + def test_withdrawing_a_missing_version_changes_nothing(self): + self.add([entry("0.82.0")]) + written = self.path.read_bytes() + self.assertFalse(registry.withdraw(self.path, "0.82.5", self.log.append)) + self.assertEqual(self.path.read_bytes(), written) + + def test_a_plugin_left_with_no_release_leaves_the_manifest(self): + self.add([entry("0.82.0")]) + registry.withdraw(self.path, "0.82.0", self.log.append) + self.assertEqual(self.manifest()["plugins"], []) + + +class TargetsTests(unittest.TestCase): + def test_reads_the_bundled_aggregate(self): + self.assertEqual(registry.bundled_targets(PROJECT), {"MySQLDriver", "SQLiteDriver"}) + + def test_reads_the_bundled_aggregate_of_this_repository(self): + bundled = registry.bundled_targets(registry.PROJECT.read_text(encoding="utf-8")) + self.assertIn("MySQLDriver", bundled) + self.assertNotIn("OracleDriver", bundled) + + def test_prints_the_publish_set(self): + with tempfile.TemporaryDirectory() as directory: + metadata, project = Path(directory) / "registry.json", Path(directory) / "project.yml" + metadata.write_text(json.dumps(METADATA)) + project.write_text(PROJECT) + with redirect_stdout(io.StringIO()) as output: + self.assertEqual(registry.main(["targets", "--metadata", str(metadata), "--project", str(project)]), 0) + self.assertEqual(output.getvalue(), "OracleDriver\nParquetExport\n") + + def test_refuses_a_bundled_target(self): + metadata = {"retainFromPluginKit": 39, "plugins": {**METADATA["plugins"], + "MySQLDriver": {"name": "MySQL", "iconName": "mysql"}}} + with self.assertRaisesRegex(registry.RegistryError, "the app bundles MySQLDriver"): + registry.publish_set(metadata, registry.bundled_targets(PROJECT)) + + def test_refuses_metadata_in_another_shape(self): + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "registry.json" + path.write_text(json.dumps({"plugins": {"oracle": {"target": "OracleDriver"}}})) + with self.assertRaisesRegex(registry.RegistryError, "shape"): + registry.load_metadata(path) + + +class DescribeTests(unittest.TestCase): + def setUp(self): + directory = tempfile.TemporaryDirectory() + self.addCleanup(directory.cleanup) + self.zips = Path(directory.name) + + def plugin(self, target, **overrides): + info = { + "CFBundleIdentifier": f"com.TablePro.{target}", "CFBundleShortVersionString": "0.82.0", + "CFBundleVersion": "820099", "TableProPluginKitVersion": 39, "LSMinimumSystemVersion": "13.0", + "TableProProvidesDatabaseTypeIds": ["Oracle"], + } + info.update(overrides) + info = {key: value for key, value in info.items() if value is not None} + path = self.zips / f"{target}-0.82.0.zip" + with zipfile.ZipFile(path, "w") as archive: + archive.writestr(f"{target}.tableplugin/Contents/Info.plist", plistlib.dumps(info)) + archive.writestr(f"{target}.tableplugin/Contents/MacOS/{target}", b"binary") + return path + + def describe(self, metadata=None): + return registry.describe(self.zips, "0.82.0", 820099, 39, metadata or METADATA, + registry.bundled_targets(PROJECT)) + + def test_reads_each_zip(self): + self.plugin("OracleDriver") + self.plugin("ParquetExport", CFBundleIdentifier="com.TablePro.ParquetExportPlugin", + TableProProvidesDatabaseTypeIds=None, TableProProvidesExportFormatIds=["parquet"]) + oracle, parquet = self.describe() + self.assertEqual(oracle["id"], "com.TablePro.OracleDriver") + self.assertEqual(oracle["databaseTypeIds"], ["Oracle"]) + self.assertEqual((oracle["version"], oracle["build"], oracle["pluginKit"], oracle["minimumSystemVersion"]), + ("0.82.0", 820099, 39, "13.0")) + self.assertEqual(oracle["file"], "OracleDriver-0.82.0.zip") + self.assertRegex(oracle["sha256"], "^[0-9a-f]{64}$") + self.assertEqual((parquet["id"], parquet["databaseTypeIds"], parquet["name"]), + ("com.TablePro.ParquetExportPlugin", [], "Parquet Export")) + + def test_rejects_a_wrong_version_build_or_kit(self): + cases = [ + ({"CFBundleShortVersionString": "0.81.0"}, "CFBundleShortVersionString is '0.81.0'"), + ({"CFBundleVersion": "140"}, "CFBundleVersion is '140'"), + ({"TableProPluginKitVersion": 38}, "TableProPluginKitVersion is 38"), + ({"TableProPluginKitVersion": "39"}, "TableProPluginKitVersion is '39'"), + ] + self.plugin("ParquetExport", TableProProvidesExportFormatIds=["parquet"]) + for overrides, message in cases: + with self.subTest(overrides=overrides): + self.plugin("OracleDriver", **overrides) + with self.assertRaisesRegex(registry.RegistryError, message): + self.describe() + + def test_rejects_a_plugin_that_provides_nothing(self): + self.plugin("OracleDriver", TableProProvidesDatabaseTypeIds=None) + self.plugin("ParquetExport", TableProProvidesExportFormatIds=["parquet"]) + with self.assertRaisesRegex(registry.RegistryError, "OracleDriver-0.82.0.zip: it declares none of"): + self.describe() + + def test_rejects_a_missing_zip(self): + self.plugin("OracleDriver") + with self.assertRaisesRegex(registry.RegistryError, "ParquetExport-0.82.0.zip does not exist"): + self.describe() + + def test_rejects_two_plugins_with_one_bundle_id(self): + self.plugin("OracleDriver") + self.plugin("ParquetExport", CFBundleIdentifier="com.TablePro.OracleDriver") + with self.assertRaisesRegex(registry.RegistryError, "share the bundle id"): + self.describe() + + def test_the_cli_writes_entries_that_add_reads(self): + self.plugin("OracleDriver") + self.plugin("ParquetExport", TableProProvidesExportFormatIds=["parquet"]) + metadata, project = self.zips / "registry.json", self.zips / "project.yml" + metadata.write_text(json.dumps(METADATA)) + project.write_text(PROJECT) + entries, manifest = self.zips / "entries.json", self.zips / "v3.json" + with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()): + self.assertEqual(registry.main([ + "describe", "--zips", str(self.zips), "--version", "0.82.0", "--build", "820099", "--kit", "39", + "--metadata", str(metadata), "--project", str(project), "--out", str(entries)]), 0) + self.assertEqual(registry.main([ + "add", "--manifest", str(manifest), "--entries", str(entries), "--channel", "stable", + "--url-base", URL_BASE, "--retain-from", "39"]), 0) + self.assertEqual(registry.main(["withdraw", "--manifest", str(manifest), "--version", "0.82.0"]), 0) + self.assertEqual(json.loads(manifest.read_text())["plugins"], []) + + +if __name__ == "__main__": + unittest.main() From 79d553cb3391407c4a285a2de101c1d98c8d7049 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat <datlechin@gmail.com> Date: Sun, 18 Oct 2026 06:14:58 +0700 Subject: [PATCH 03/10] build(release): add the feed, registry and GitHub release tooling --- .claude/rules/ci.md | 3 ++- scripts/release/test_feed.py | 21 ++++++++++++++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 25290306ae..0d3d405eb5 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -27,7 +27,8 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. `build-plugin.yml` picks its runner from the ref it builds (`RELEASING.md`). - The Xcode version is written once, in `.github/actions/setup-xcode/action.yml`, which runs `xcode-select --switch` and prints `xcodebuild -version`. A workflow never selects Xcode another way. - XcodeGen comes from `.github/actions/setup-xcodegen`, pinned by version and SHA-256. -- Sparkle's `generate_appcast` and `sign_update` come from `.github/actions/setup-sparkle`. It reads the version and revision from the app's `Package.resolved` (`project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. +- `scripts/release/feed.py` runs Sparkle's `generate_appcast` and `sign_update` from `SPARKLE_BIN`, which `.github/actions/setup-sparkle` sets. The action reads the version and revision from the app's `Package.resolved` (`project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. +- The tag release (`build.yml`) signs the feed with the copy `scripts/ci/sign-and-appcast.sh` downloads, pinned by its own `SPARKLE_VERSION` and `SPARKLE_SHA256`. No check compares them with `project.yml`: a Sparkle bump changes all three. - The compilation cache is off on CI (`COMPILATION_CACHE_ENABLE_CACHING=NO`); `Configs/Base.xcconfig` turns it on locally. ## macOS Tests (`macos-tests.yml`) diff --git a/scripts/release/test_feed.py b/scripts/release/test_feed.py index 2b5abc4584..d4eb227211 100644 --- a/scripts/release/test_feed.py +++ b/scripts/release/test_feed.py @@ -15,6 +15,7 @@ from unittest import mock import feed +import publish import registry import version @@ -63,7 +64,7 @@ def signed(text, key): for index, arg in enumerate(args): if arg == "--pattern": shutil.copy(releases / args[2] / args[index + 1], out / args[index + 1]) -else: +elif args[:2] != ["release", "edit"]: sys.exit("fake gh cannot answer " + " ".join(args)) ''' @@ -510,6 +511,24 @@ def test_withdraw_keeps_a_default_channel_item(self): with self.assertRaisesRegex(feed.FeedError, "no default-channel item"): feed.withdraw(self.root, "0.81.0", self.tools, KEY, log=lambda message: None) + def test_latest_after_a_withdraw_reads_the_pushed_branch(self): + self.publish("v0.82.0-beta.1") + self.publish("v0.82.0") + self.add("v0.82.0-beta.1") + self.add("v0.82.0") + before = git(self.origin, "rev-parse", "feeds") + feed.withdraw(self.root, "0.82.0", self.tools, KEY, log=lambda message: None) + git(self.root, "branch", "feeds", before) + git(self.root, "update-ref", feed.REMOTE_BRANCH, before) + self.assertEqual([each.short_version for each in feed.published_items(self.root)], + ["0.82.0-beta.1", "0.81.0", "0.80.0"]) + + git(self.root, "update-ref", feed.REMOTE_BRANCH, before) + with mock.patch.object(publish, "ROOT", self.root), \ + mock.patch.dict(os.environ, {"GITHUB_REPOSITORY": REPOSITORY}), redirect_stderr(io.StringIO()): + self.assertEqual(publish.main(["latest"]), 0) + self.assertEqual(self.calls("gh")[-1], ["release", "edit", "v0.81.0", "--latest=true", "--repo", REPOSITORY]) + def test_check_reads_the_published_branch(self): self.assertTrue(any("sign_update failed" in problem for problem in feed.check(self.root, self.tools, KEY))) self.publish("v0.82.0-beta.1") From 3768be32d06d73c108d5a59cb035457604f3f975 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat <datlechin@gmail.com> Date: Sun, 18 Oct 2026 08:52:29 +0700 Subject: [PATCH 04/10] ci(release): build, notarize and publish from v* tags in release.yml --- .../skills/release/scripts/changed-plugins.py | 109 ---- .github/actions/apple-signing/action.yml | 63 +- .github/actions/setup-xcode/action.yml | 31 +- .github/plugin-registry.json | 538 +++--------------- .github/release-flags.json | 4 - .github/scripts/test_update_registry.py | 320 ----------- .github/scripts/update-registry.py | 214 ------- .github/workflows/build-plugin.yml | 490 ---------------- .github/workflows/build.yml | 512 ----------------- .github/workflows/pages.yml | 40 ++ .github/workflows/release.yml | 367 ++++++++++++ .github/workflows/repo-hygiene.yml | 7 +- .github/workflows/withdraw.yml | 49 ++ scripts/build-plugin.sh | 278 --------- scripts/build-release.sh | 505 ---------------- scripts/check-registry-readiness.py | 81 --- scripts/ci/appcast_feed.py | 198 ------- scripts/ci/changelog_credits.py | 214 ------- scripts/ci/check-plugin-manifest.py | 65 +-- scripts/ci/detect-changed-paths.sh | 21 +- scripts/ci/draft-release.sh | 174 ------ scripts/ci/extract-release-notes.py | 129 ----- scripts/ci/extract-release-notes.sh | 10 - scripts/ci/merge-appcast.py | 229 -------- scripts/ci/notify-telegram.sh | 71 --- scripts/ci/package-artifacts.sh | 70 --- scripts/ci/plugin-info.py | 77 --- scripts/ci/prepare-libs.sh | 27 - scripts/ci/pull-release.py | 158 ----- scripts/ci/sign-and-appcast.sh | 161 ------ scripts/ci/test_appcast_feed.py | 160 ------ scripts/ci/test_changelog_credits.py | 196 ------- scripts/ci/test_check_plugin_manifest.py | 46 +- scripts/ci/test_create_dmg_args.sh | 93 --- scripts/ci/test_detect_changed_paths.py | 15 + scripts/ci/test_draft_release.sh | 258 --------- scripts/ci/test_merge_appcast.py | 233 -------- scripts/ci/test_pull_release.py | 224 -------- scripts/ci/test_release_notes.py | 239 -------- scripts/ci/verify-build.sh | 191 ------- scripts/ci/verify-update-archive.sh | 40 -- scripts/create-dmg.sh | 285 ---------- scripts/lib/macos.sh | 40 -- scripts/lib/notarize.sh | 82 --- scripts/release-all-plugins.sh | 83 --- scripts/release-plugin-for-shipped-app.sh | 87 --- scripts/release/ExportOptions.plist | 19 + scripts/release/build.sh | 12 +- scripts/release/notarize.sh | 78 +++ scripts/release/package.sh | 12 +- 50 files changed, 800 insertions(+), 6805 deletions(-) delete mode 100644 .claude/skills/release/scripts/changed-plugins.py delete mode 100644 .github/release-flags.json delete mode 100644 .github/scripts/test_update_registry.py delete mode 100755 .github/scripts/update-registry.py delete mode 100644 .github/workflows/build-plugin.yml delete mode 100644 .github/workflows/build.yml create mode 100644 .github/workflows/pages.yml create mode 100644 .github/workflows/release.yml create mode 100644 .github/workflows/withdraw.yml delete mode 100755 scripts/build-plugin.sh delete mode 100755 scripts/build-release.sh delete mode 100644 scripts/check-registry-readiness.py delete mode 100644 scripts/ci/appcast_feed.py delete mode 100644 scripts/ci/changelog_credits.py delete mode 100755 scripts/ci/draft-release.sh delete mode 100644 scripts/ci/extract-release-notes.py delete mode 100755 scripts/ci/extract-release-notes.sh delete mode 100755 scripts/ci/merge-appcast.py delete mode 100755 scripts/ci/notify-telegram.sh delete mode 100755 scripts/ci/package-artifacts.sh delete mode 100755 scripts/ci/plugin-info.py delete mode 100755 scripts/ci/prepare-libs.sh delete mode 100755 scripts/ci/pull-release.py delete mode 100755 scripts/ci/sign-and-appcast.sh delete mode 100644 scripts/ci/test_appcast_feed.py delete mode 100644 scripts/ci/test_changelog_credits.py delete mode 100755 scripts/ci/test_create_dmg_args.sh delete mode 100755 scripts/ci/test_draft_release.sh delete mode 100755 scripts/ci/test_merge_appcast.py delete mode 100755 scripts/ci/test_pull_release.py delete mode 100644 scripts/ci/test_release_notes.py delete mode 100755 scripts/ci/verify-build.sh delete mode 100755 scripts/ci/verify-update-archive.sh delete mode 100755 scripts/create-dmg.sh delete mode 100644 scripts/lib/notarize.sh delete mode 100755 scripts/release-all-plugins.sh delete mode 100755 scripts/release-plugin-for-shipped-app.sh create mode 100644 scripts/release/ExportOptions.plist create mode 100755 scripts/release/notarize.sh diff --git a/.claude/skills/release/scripts/changed-plugins.py b/.claude/skills/release/scripts/changed-plugins.py deleted file mode 100644 index 4db1c613c9..0000000000 --- a/.claude/skills/release/scripts/changed-plugins.py +++ /dev/null @@ -1,109 +0,0 @@ -#!/usr/bin/env python3 -"""Lists every registry plugin with its last tag and the commits since it. - -The slug, the tag name and the target come from .github/plugin-registry.json. The source paths -come from the target in project.yml: its `folder:` under Plugins/, every local package it depends -on except TableProCore, which every plugin shares, and every Native/ bridge it or its templates -name. -Nothing here restates the plugin list. - -Usage: - python3 .claude/skills/release/scripts/changed-plugins.py [--all] - -Prints only plugins with commits of their own or PluginKit commits since their last tag, unless ---all is given. Run it from anywhere inside the TablePro repo. -""" - -import argparse -import json -import re -import subprocess -import sys -from pathlib import Path - -KIT = "Plugins/TableProPluginKit/" - - -def git(root, *args): - return subprocess.run( - ["git", "-C", str(root), *args], capture_output=True, text=True, check=True - ).stdout.strip() - - -def repo_root(): - try: - return Path(git(Path(__file__).resolve().parent, "rev-parse", "--show-toplevel")) - except subprocess.CalledProcessError: - sys.exit("changed-plugins.py: not inside a git checkout") - - -def blocks(project_yml): - sections = {"targetTemplates": {}, "targets": {}} - section = name = None - for line in project_yml.read_text(encoding="utf-8").splitlines(): - top = re.match(r"^(\w+):", line) - if top: - section = sections.get(top.group(1)) - name = None - continue - header = re.match(r"^ ([A-Za-z0-9]+):\s*$", line) - if section is not None and header: - name = header.group(1) - section[name] = [] - elif section is not None and name: - section[name].append(line) - return sections["targetTemplates"], sections["targets"] - - -def source_paths(root, target, templates, targets): - lines = targets.get(target) - if lines is None: - sys.exit(f"changed-plugins.py: no target {target} in project.yml") - text = "\n".join(lines) - for used in re.findall(r"templates:\s*\[([^\]]*)\]", text): - for template in (t.strip() for t in used.split(",")): - text += "\n" + "\n".join(templates.get(template, [])) - folder = re.search(r"^\s+folder:\s*(\S+)", text, re.M) - if folder is None: - sys.exit(f"changed-plugins.py: target {target} has no folder in project.yml") - paths = [f"Plugins/{folder.group(1)}/"] - for package in re.findall(r"-\s*package:\s*(\S+)", text): - if package != "TableProCore" and (root / "Packages" / package).is_dir(): - paths.append(f"Packages/{package}/") - paths += [f"Native/{bridge}/" for bridge in re.findall(r"Native/(\w+)", text)] - return sorted(set(paths)) - - -def last_tag(root, slug): - tags = git(root, "tag", "-l", f"plugin-{slug}-v*", "--sort=-version:refname") - return tags.splitlines()[0] if tags else None - - -def commit_count(root, since, paths): - revision = f"{since}..HEAD" if since else "HEAD" - return int(git(root, "rev-list", "--count", revision, "--", *paths)) - - -def main(): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--all", action="store_true", help="list unchanged plugins too") - args = parser.parse_args() - - root = repo_root() - plugins = json.loads((root / ".github/plugin-registry.json").read_text())["plugins"] - templates, targets = blocks(root / "project.yml") - - print(f"{'slug':<20} {'bundled':<8} {'last tag':<34} own kit paths") - for slug, entry in sorted(plugins.items()): - paths = source_paths(root, entry["target"], templates, targets) - tag = last_tag(root, slug) - own = commit_count(root, tag, paths) - kit = commit_count(root, tag, [KIT]) - if not args.all and own == 0 and kit == 0: - continue - bundled = "yes" if entry["bundled"] else "no" - print(f"{slug:<20} {bundled:<8} {tag or 'never tagged':<34} {own:>3} {kit:>3} {' '.join(paths)}") - - -if __name__ == "__main__": - main() diff --git a/.github/actions/apple-signing/action.yml b/.github/actions/apple-signing/action.yml index b8abb77a9b..ba9f90e1bb 100644 --- a/.github/actions/apple-signing/action.yml +++ b/.github/actions/apple-signing/action.yml @@ -1,5 +1,5 @@ name: Apple signing -description: Imports the Developer ID certificate, stores notary credentials, and installs the provisioning profile. +description: Imports the Developer ID certificate, stores the notary credentials as the TablePro profile, and installs the provisioning profile. inputs: certificate-p12: @@ -8,19 +8,19 @@ inputs: certificate-password: description: Password for that .p12. required: true - apple-id: - description: Apple ID the notary submissions are made under. - required: true - team-id: - description: Apple Developer team identifier. - required: true - notary-password: - description: App-specific password for notarytool. - required: true provisioning-profile: description: Base64 of the provisioning profile. Omit to skip installing one. required: false default: "" + asc-key-id: + description: Key ID of the App Store Connect API key notarytool authenticates with. + required: true + asc-issuer-id: + description: Issuer ID of that key. + required: true + asc-key-p8: + description: Contents of that key's .p8 file. + required: true runs: using: composite @@ -45,18 +45,31 @@ runs: security set-key-partition-list -S apple-tool:,apple: -k "" "$KEYCHAIN_PATH" security list-keychain -d user -s "$KEYCHAIN_PATH" login.keychain - - name: Configure notarization + certificate="$RUNNER_TEMP/certificate.pem" + security find-certificate -c "Developer ID Application" -p "$KEYCHAIN_PATH" > "$certificate" + expiry="$(openssl x509 -noout -enddate -in "$certificate" | sed 's/^notAfter=//')" + if ! openssl x509 -noout -checkend $((60 * 24 * 3600)) -in "$certificate" > /dev/null; then + echo "::warning::The Developer ID Application certificate expires on $expiry" + fi + rm -f "$certificate" + echo "Developer ID Application certificate valid until $expiry" + + - name: Store the notary credentials shell: bash env: - APPLE_ID: ${{ inputs.apple-id }} - APPLE_TEAM_ID: ${{ inputs.team-id }} - NOTARY_PASSWORD: ${{ inputs.notary-password }} + ASC_KEY_ID: ${{ inputs.asc-key-id }} + ASC_ISSUER_ID: ${{ inputs.asc-issuer-id }} + ASC_KEY_P8: ${{ inputs.asc-key-p8 }} + # notarytool copies the key into the keychain item, so the file can go at once. run: | set -euo pipefail - xcrun notarytool store-credentials "TablePro" \ - --apple-id "$APPLE_ID" \ - --team-id "$APPLE_TEAM_ID" \ - --password "$NOTARY_PASSWORD" + key="$RUNNER_TEMP/AuthKey_$ASC_KEY_ID.p8" + (umask 077 && printf '%s\n' "$ASC_KEY_P8" > "$key") + status=0 + xcrun notarytool store-credentials "TablePro" --key "$key" --key-id "$ASC_KEY_ID" \ + --issuer "$ASC_ISSUER_ID" || status=$? + rm -f "$key" + exit "$status" - name: Install provisioning profile if: inputs.provisioning-profile != '' @@ -66,5 +79,17 @@ runs: run: | set -euo pipefail profiles="$HOME/Library/MobileDevice/Provisioning Profiles" + decoded="$RUNNER_TEMP/profile.plist" mkdir -p "$profiles" - echo "$PROVISIONING_PROFILE" | base64 --decode > "$profiles/tablepro.provisionprofile" + echo "$PROVISIONING_PROFILE" | base64 --decode > "$RUNNER_TEMP/tablepro.provisionprofile" + security cms -D -i "$RUNNER_TEMP/tablepro.provisionprofile" > "$decoded" + name="$(plutil -extract Name raw -o - "$decoded")" + uuid="$(plutil -extract UUID raw -o - "$decoded")" + expiry="$(plutil -extract ExpirationDate raw -o - "$decoded")" + mv "$RUNNER_TEMP/tablepro.provisionprofile" "$profiles/$uuid.provisionprofile" + rm -f "$decoded" + # Both are ISO 8601 in UTC, so they compare as strings. + if [[ "$expiry" < "$(date -u -v+60d +%Y-%m-%dT%H:%M:%SZ)" ]]; then + echo "::warning::The provisioning profile '$name' expires on $expiry" + fi + echo "Installed the provisioning profile '$name' ($uuid), valid until $expiry" diff --git a/.github/actions/setup-xcode/action.yml b/.github/actions/setup-xcode/action.yml index 24e0226134..803891e043 100644 --- a/.github/actions/setup-xcode/action.yml +++ b/.github/actions/setup-xcode/action.yml @@ -1,12 +1,41 @@ name: Select Xcode description: Selects the one Xcode every TablePro build and test job uses. +inputs: + require-release: + description: true fails the job unless the selected Xcode is a GA release, as a stable build requires. + required: false + default: "false" + # The only place the version is written. runs-on names the image (xcode-27); this picks the build # inside it, the one developers install locally. The image has no Xcode 26. runs: using: composite steps: - shell: bash + env: + XCODE: /Applications/Xcode_27.1.app + REQUIRE_RELEASE: ${{ inputs.require-release }} run: | - sudo xcode-select --switch /Applications/Xcode_27.1.app + set -euo pipefail + sudo xcode-select --switch "$XCODE" xcodebuild -version + [ "$REQUIRE_RELEASE" = true ] || exit 0 + + # The image links Xcode_<version>.app to whichever build it carries, a beta or RC included. + app="$(cd "$XCODE" && pwd -P)" + case "$(basename "$app")" in + *_beta* | *_Release_Candidate*) + echo "::error::$XCODE is $app, not a GA Xcode; a stable release builds only on a GA Xcode" + exit 1 + ;; + esac + license="$app/Contents/Resources/LicenseInfo.plist" + type="$(plutil -extract licenseType raw -o - "$license" 2> /dev/null || true)" + case "$(tr '[:upper:]' '[:lower:]' <<< "$type")" in + *beta*) + echo "::error::$app has the license type '$type', not a GA Xcode; a stable release builds only on a GA Xcode" + exit 1 + ;; + esac + echo "$app is a GA Xcode" diff --git a/.github/plugin-registry.json b/.github/plugin-registry.json index 78d112ef4e..5455fb8da1 100644 --- a/.github/plugin-registry.json +++ b/.github/plugin-registry.json @@ -1,489 +1,109 @@ { - "$comment": [ - "The release metadata for every plugin published to the registry, keyed by the slug in its", - "git tag (plugin-<slug>-v<version>). This is data the repo owns nowhere else: the summary,", - "the category and the homepage exist only here, and the slug-to-target mapping has no", - "derivation rule (mssql -> MSSQLDriver, cloudflare-d1 -> CloudflareD1DriverPlugin).", - "", - "'icon' and 'databaseTypeIds' do exist elsewhere, in the plugin's own DriverPlugin class,", - "and scripts/ci/check-plugin-manifest.py fails if the two disagree. Seven icons here were", - "already wrong when that check was first run.", - "", - "The 7 plugins absent from this file are the ones bundled inside the app and never published", - "to the registry: MySQL, PostgreSQL, CSV/JSON/SQL export, CSV/JSON import.", - "", - "'bundled' marks a plugin that also ships inside the app. Those ten can still be published", - "individually, but the app never applies a registry update to a bundled plugin, so a published", - "build reaches only users who install its zip from a file. A bulk ABI re-release skips them,", - "because their binaries ride with the next app release." - ], + "retainFromPluginKit": 39, "plugins": { - "beancount": { - "target": "BeancountDriver", - "bundleName": "BeancountDriver", - "bundleId": "com.TablePro.BeancountDriver", - "bundled": false, - "displayName": "Beancount Driver", - "summary": "Read-only Beancount ledger driver using user-provided rledger or Python Beancount", - "databaseTypeIds": [ - "Beancount" - ], - "icon": "beancount-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/beancount" + "BeancountDriver": { + "name": "Beancount Driver", + "iconName": "beancount-icon" }, - "bigquery": { - "target": "BigQueryDriverPlugin", - "bundleName": "BigQueryDriverPlugin", - "bundleId": "com.TablePro.BigQueryDriverPlugin", - "bundled": false, - "displayName": "BigQuery Driver", - "summary": "Google BigQuery analytics database driver via REST API", - "databaseTypeIds": [ - "BigQuery" - ], - "icon": "bigquery-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/bigquery" + "BigQueryDriverPlugin": { + "name": "BigQuery Driver", + "iconName": "bigquery-icon" }, - "bigtable": { - "target": "BigtableDriverPlugin", - "bundleName": "BigtableDriverPlugin", - "bundleId": "com.TablePro.BigtableDriverPlugin", - "bundled": false, - "displayName": "Bigtable Driver", - "summary": "Google Cloud Bigtable driver over the REST API", - "databaseTypeIds": [ - "Bigtable" - ], - "icon": "bigtable-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/bigtable" + "BigtableDriverPlugin": { + "name": "Bigtable Driver", + "iconName": "bigtable-icon" }, - "cassandra": { - "target": "CassandraDriver", - "bundleName": "CassandraDriver", - "bundleId": "com.TablePro.CassandraDriver", - "bundled": false, - "displayName": "Cassandra Driver", - "summary": "Apache Cassandra and ScyllaDB driver via DataStax C driver", - "databaseTypeIds": [ - "Cassandra", - "ScyllaDB" - ], - "icon": "cassandra-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cassandra" + "CassandraDriver": { + "name": "Cassandra Driver", + "iconName": "cassandra-icon" }, - "clickhouse": { - "target": "ClickHouseDriver", - "bundleName": "ClickHouseDriver", - "bundleId": "com.TablePro.ClickHouseDriver", - "bundled": true, - "displayName": "ClickHouse Driver", - "summary": "ClickHouse OLAP database driver via HTTP interface", - "databaseTypeIds": [ - "ClickHouse" - ], - "icon": "clickhouse-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/clickhouse" + "CloudflareD1DriverPlugin": { + "name": "Cloudflare D1 Driver", + "iconName": "cloudflare-d1-icon" }, - "cloudflare-d1": { - "target": "CloudflareD1DriverPlugin", - "bundleName": "CloudflareD1DriverPlugin", - "bundleId": "com.TablePro.CloudflareD1DriverPlugin", - "bundled": false, - "displayName": "Cloudflare D1 Driver", - "summary": "Cloudflare D1 serverless SQLite-compatible database driver via REST API", - "databaseTypeIds": [ - "Cloudflare D1" - ], - "icon": "cloudflare-d1-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cloudflare-d1" + "CloudflareR2SQLDriverPlugin": { + "name": "Cloudflare R2 SQL Driver", + "iconName": "cloudflare-r2-sql-icon" }, - "cloudflare-r2-sql": { - "target": "CloudflareR2SQLDriverPlugin", - "bundleName": "CloudflareR2SQLDriverPlugin", - "bundleId": "com.TablePro.CloudflareR2SQLDriverPlugin", - "bundled": false, - "displayName": "Cloudflare R2 SQL Driver", - "summary": "Read-only Cloudflare R2 SQL driver for Iceberg tables in R2 Data Catalog via REST API", - "databaseTypeIds": [ - "Cloudflare R2 SQL" - ], - "icon": "cloudflare-r2-sql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/cloudflare-r2-sql" + "DamengDriver": { + "name": "Dameng Driver", + "iconName": "cylinder" }, - "dameng": { - "target": "DamengDriver", - "bundleName": "DamengDriver", - "bundleId": "com.TablePro.DamengDriver", - "bundled": false, - "displayName": "Dameng Driver", - "summary": "Dameng DM8 database driver via a native wire client", - "databaseTypeIds": [ - "Dameng" - ], - "icon": "cylinder", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/dameng" + "DuckDBDriver": { + "name": "DuckDB Driver", + "iconName": "duckdb-icon" }, - "duckdb": { - "target": "DuckDBDriver", - "bundleName": "DuckDBDriver", - "bundleId": "com.TablePro.DuckDBDriver", - "bundled": false, - "displayName": "DuckDB Driver", - "summary": "DuckDB analytical database driver", - "databaseTypeIds": [ - "DuckDB" - ], - "icon": "duckdb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/duckdb" + "DynamoDBDriverPlugin": { + "name": "DynamoDB Driver", + "iconName": "dynamodb-icon" }, - "dynamodb": { - "target": "DynamoDBDriverPlugin", - "bundleName": "DynamoDBDriverPlugin", - "bundleId": "com.TablePro.DynamoDBDriverPlugin", - "bundled": false, - "displayName": "DynamoDB Driver", - "summary": "Amazon DynamoDB: index-aware browsing, typed editing, PartiQL, the DynamoDB API and table management", - "databaseTypeIds": [ - "DynamoDB" - ], - "icon": "dynamodb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/dynamodb" + "ElasticsearchDriverPlugin": { + "name": "Elasticsearch Driver", + "iconName": "elasticsearch-icon" }, - "elasticsearch": { - "target": "ElasticsearchDriverPlugin", - "bundleName": "ElasticsearchDriverPlugin", - "bundleId": "com.TablePro.ElasticsearchDriverPlugin", - "bundled": false, - "displayName": "Elasticsearch Driver", - "summary": "Elasticsearch driver via the REST API with a Query DSL console", - "databaseTypeIds": [ - "Elasticsearch" - ], - "icon": "elasticsearch-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/elasticsearch" + "EtcdDriverPlugin": { + "name": "etcd Driver", + "iconName": "etcd-icon" }, - "etcd": { - "target": "EtcdDriverPlugin", - "bundleName": "EtcdDriverPlugin", - "bundleId": "com.TablePro.EtcdDriverPlugin", - "bundled": false, - "displayName": "etcd Driver", - "summary": "etcd v3 key-value store driver with prefix-tree browsing and lease management", - "databaseTypeIds": [ - "etcd" - ], - "icon": "etcd-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/etcd" + "FirebaseDriverPlugin": { + "name": "Firebase Driver", + "iconName": "firebase-icon" }, - "firebase": { - "target": "FirebaseDriverPlugin", - "bundleName": "FirebaseDriverPlugin", - "bundleId": "com.TablePro.FirebaseDriverPlugin", - "bundled": false, - "displayName": "Firebase Driver", - "summary": "Cloud Firestore and Firebase Realtime Database driver over their REST APIs", - "databaseTypeIds": [ - "Firestore", - "Firebase Realtime Database" - ], - "icon": "firebase-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/firebase" + "HanaDriver": { + "name": "SAP HANA Driver", + "iconName": "cylinder" }, - "hana": { - "target": "HanaDriver", - "bundleName": "HanaDriver", - "bundleId": "com.TablePro.HanaDriver", - "bundled": false, - "displayName": "SAP HANA Driver", - "summary": "SAP HANA SQL driver via SAP/go-hdb", - "databaseTypeIds": [ - "SAP HANA" - ], - "icon": "cylinder", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/sap-hana" + "KafkaDriverPlugin": { + "name": "Kafka Driver", + "iconName": "kafka-icon" }, - "kafka": { - "target": "KafkaDriverPlugin", - "bundleName": "KafkaDriverPlugin", - "bundleId": "com.TablePro.KafkaDriverPlugin", - "bundled": false, - "displayName": "Kafka Driver", - "summary": "Apache Kafka topic and message browser over the native wire protocol", - "databaseTypeIds": [ - "Kafka" - ], - "icon": "kafka-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/kafka" + "LibSQLDriverPlugin": { + "name": "libSQL / Turso Driver", + "iconName": "libsql-icon" }, - "libsql": { - "target": "LibSQLDriverPlugin", - "bundleName": "LibSQLDriverPlugin", - "bundleId": "com.TablePro.LibSQLDriverPlugin", - "bundled": false, - "displayName": "libSQL / Turso Driver", - "summary": "libSQL and Turso database support via Hrana HTTP protocol", - "databaseTypeIds": [ - "libSQL", - "Turso" - ], - "icon": "libsql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/libsql" + "MSSQLDriver": { + "name": "MSSQL Driver", + "iconName": "mssql-icon" }, - "markdown": { - "target": "MarkdownExport", - "bundleName": "MarkdownExport", - "bundleId": "com.TablePro.MarkdownExportPlugin", - "bundled": true, - "displayName": "Markdown Export", - "summary": "Export data as Markdown tables", - "databaseTypeIds": null, - "icon": "text.alignleft", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "MongoDBDriver": { + "name": "MongoDB Driver", + "iconName": "mongodb-icon" }, - "mongodb": { - "target": "MongoDBDriver", - "bundleName": "MongoDBDriver", - "bundleId": "com.TablePro.MongoDBDriver", - "bundled": false, - "displayName": "MongoDB Driver", - "summary": "MongoDB document database driver via libmongoc", - "databaseTypeIds": [ - "MongoDB" - ], - "icon": "mongodb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/mongodb" + "OracleDriver": { + "name": "Oracle Driver", + "iconName": "oracle-icon" }, - "mql": { - "target": "MQLExport", - "bundleName": "MQLExport", - "bundleId": "com.TablePro.MQLExportPlugin", - "bundled": true, - "displayName": "MQL Export", - "summary": "Export MongoDB data as MQL statements", - "databaseTypeIds": null, - "icon": "leaf", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "ParquetExport": { + "name": "Parquet Export", + "iconName": "square.grid.3x3" }, - "mssql": { - "target": "MSSQLDriver", - "bundleName": "MSSQLDriver", - "bundleId": "com.TablePro.MSSQLDriver", - "bundled": false, - "displayName": "MSSQL Driver", - "summary": "Microsoft SQL Server driver via FreeTDS", - "databaseTypeIds": [ - "SQL Server" - ], - "icon": "mssql-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/mssql" + "SnowflakeDriverPlugin": { + "name": "Snowflake Driver", + "iconName": "snowflake-icon" }, - "oracle": { - "target": "OracleDriver", - "bundleName": "OracleDriver", - "bundleId": "com.TablePro.OracleDriver", - "bundled": false, - "displayName": "Oracle Driver", - "summary": "Oracle Database 12c+ driver via OracleNIO", - "databaseTypeIds": [ - "Oracle" - ], - "icon": "oracle-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/oracle" + "SpannerDriverPlugin": { + "name": "Spanner Driver", + "iconName": "spanner-icon" }, - "parquet": { - "target": "ParquetExport", - "bundleName": "ParquetExport", - "bundleId": "com.TablePro.ParquetExportPlugin", - "bundled": false, - "displayName": "Parquet Export", - "summary": "Export data to Apache Parquet", - "databaseTypeIds": null, - "icon": "square.grid.3x3", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "SurrealDBDriverPlugin": { + "name": "SurrealDB Driver", + "iconName": "surrealdb-icon" }, - "redis": { - "target": "RedisDriver", - "bundleName": "RedisDriver", - "bundleId": "com.TablePro.RedisDriver", - "bundled": true, - "displayName": "Redis Driver", - "summary": "Redis in-memory data store driver via hiredis", - "databaseTypeIds": [ - "Redis" - ], - "icon": "redis-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/redis" + "TeradataDriver": { + "name": "Teradata Driver", + "iconName": "teradata-icon" }, - "snowflake": { - "target": "SnowflakeDriverPlugin", - "bundleName": "SnowflakeDriverPlugin", - "bundleId": "com.TablePro.SnowflakeDriverPlugin", - "bundled": false, - "displayName": "Snowflake Driver", - "summary": "Snowflake cloud data warehouse driver via the connector REST protocol", - "databaseTypeIds": [ - "Snowflake" - ], - "icon": "snowflake-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/snowflake" + "TrinoDriverPlugin": { + "name": "Trino Driver", + "iconName": "trino-icon" }, - "spanner": { - "target": "SpannerDriverPlugin", - "bundleName": "SpannerDriverPlugin", - "bundleId": "com.TablePro.SpannerDriverPlugin", - "bundled": false, - "displayName": "Spanner Driver", - "summary": "Google Cloud Spanner driver via REST API", - "databaseTypeIds": [ - "Spanner" - ], - "icon": "spanner-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/spanner" + "TypesenseDriverPlugin": { + "name": "Typesense Driver", + "iconName": "typesense-icon" }, - "sqlimport": { - "target": "SQLImport", - "bundleName": "SQLImport", - "bundleId": "com.TablePro.SQLImportPlugin", - "bundled": true, - "displayName": "SQL Import", - "summary": "Import data from SQL dump files", - "databaseTypeIds": null, - "icon": "doc.text", - "category": "import-format", - "homepage": "https://docs.tablepro.app/features/import-export" - }, - "sqlite": { - "target": "SQLiteDriver", - "bundleName": "SQLiteDriver", - "bundleId": "com.TablePro.SQLiteDriver", - "bundled": true, - "displayName": "SQLite Driver", - "summary": "SQLite embedded database driver", - "databaseTypeIds": [ - "SQLite" - ], - "icon": "sqlite-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/sqlite" - }, - "surrealdb": { - "target": "SurrealDBDriverPlugin", - "bundleName": "SurrealDBDriverPlugin", - "bundleId": "com.TablePro.SurrealDBDriverPlugin", - "bundled": false, - "displayName": "SurrealDB Driver", - "summary": "SurrealDB driver over the HTTP RPC protocol with SurrealQL", - "databaseTypeIds": [ - "SurrealDB" - ], - "icon": "surrealdb-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/surrealdb" - }, - "teradata": { - "target": "TeradataDriver", - "bundleName": "TeradataDriver", - "bundleId": "com.TablePro.TeradataDriver", - "bundled": false, - "displayName": "Teradata Driver", - "summary": "Teradata Vantage driver via a native Swift TD2 client", - "databaseTypeIds": [ - "Teradata" - ], - "icon": "teradata-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/teradata" - }, - "trino": { - "target": "TrinoDriverPlugin", - "bundleName": "TrinoDriverPlugin", - "bundleId": "com.TablePro.TrinoDriverPlugin", - "bundled": false, - "displayName": "Trino Driver", - "summary": "Trino distributed SQL engine driver via the REST client protocol", - "databaseTypeIds": [ - "Trino" - ], - "icon": "trino-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/trino" - }, - "typesense": { - "target": "TypesenseDriverPlugin", - "bundleName": "TypesenseDriverPlugin", - "bundleId": "com.TablePro.TypesenseDriverPlugin", - "bundled": false, - "displayName": "Typesense Driver", - "summary": "Typesense driver over the REST API with a request console", - "databaseTypeIds": [ - "Typesense" - ], - "icon": "typesense-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/typesense" - }, - "weaviate": { - "target": "WeaviateDriverPlugin", - "bundleName": "WeaviateDriverPlugin", - "bundleId": "com.TablePro.WeaviateDriverPlugin", - "bundled": false, - "displayName": "Weaviate Driver", - "summary": "Weaviate driver over the REST API with a GraphQL console", - "databaseTypeIds": [ - "Weaviate" - ], - "icon": "weaviate-icon", - "category": "database-driver", - "homepage": "https://docs.tablepro.app/databases/weaviate" - }, - "xlsx": { - "target": "XLSXExport", - "bundleName": "XLSXExport", - "bundleId": "com.TablePro.XLSXExportPlugin", - "bundled": true, - "displayName": "XLSX Export", - "summary": "Export data to Microsoft Excel XLSX format", - "databaseTypeIds": null, - "icon": "tablecells", - "category": "export-format", - "homepage": "https://docs.tablepro.app/features/import-export" - }, - "xlsximport": { - "target": "XLSXImport", - "bundleName": "XLSXImport", - "bundleId": "com.TablePro.XLSXImportPlugin", - "bundled": true, - "displayName": "XLSX Import", - "summary": "Import data from Excel workbooks", - "databaseTypeIds": null, - "icon": "tablecells", - "category": "import-format", - "homepage": "https://docs.tablepro.app/features/import-export" + "WeaviateDriverPlugin": { + "name": "Weaviate Driver", + "iconName": "weaviate-icon" } } } diff --git a/.github/release-flags.json b/.github/release-flags.json deleted file mode 100644 index cbe1b4e077..0000000000 --- a/.github/release-flags.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "$comment": "Per-release flags read by .github/workflows/build.yml. criticalUpdate: when to set it is the Critical update flag section of .claude/skills/release/SKILL.md; what it does is the Staged rollout and critical updates section of RELEASING.md.", - "criticalUpdate": true -} diff --git a/.github/scripts/test_update_registry.py b/.github/scripts/test_update_registry.py deleted file mode 100644 index 4d90015a4d..0000000000 --- a/.github/scripts/test_update_registry.py +++ /dev/null @@ -1,320 +0,0 @@ -#!/usr/bin/env python3 -"""Tests for update-registry.py PluginKit version pruning (#1322). - -Run: python3 .github/scripts/test_update_registry.py -""" -import contextlib -import importlib.util -import io -import os -import sys -from unittest import mock - -_spec = importlib.util.spec_from_file_location( - "update_registry", - os.path.join(os.path.dirname(__file__), "update-registry.py"), -) -update_registry = importlib.util.module_from_spec(_spec) -_spec.loader.exec_module(update_registry) - - -def test_kit_version_rejects_non_int(): - assert update_registry.kit_version({"pluginKitVersion": 14}) == 14 - assert update_registry.kit_version({"pluginKitVersion": None}) is None - assert update_registry.kit_version({}) is None - - -def test_prune_drops_null_kit_binary(): - binaries = [ - {"architecture": "arm64", "pluginKitVersion": None, "downloadURL": "legacy"}, - {"architecture": "arm64", "pluginKitVersion": 14, "downloadURL": "v14"}, - {"architecture": "arm64", "pluginKitVersion": 13, "downloadURL": "v13"}, - ] - kept = update_registry.prune_old_kit_versions(binaries, keep_count=2) - versions = sorted(update_registry.kit_version(b) for b in kept) - assert versions == [13, 14], versions - assert all(update_registry.kit_version(b) is not None for b in kept) - - -def test_prune_keeps_only_two_newest(): - binaries = [ - {"architecture": "arm64", "pluginKitVersion": v, "downloadURL": str(v)} - for v in (12, 13, 14) - ] - kept = update_registry.prune_old_kit_versions(binaries, keep_count=2) - versions = sorted(update_registry.kit_version(b) for b in kept) - assert versions == [13, 14], versions - - -def test_update_entry_drops_legacy_null_binary(): - manifest = { - "schemaVersion": 2, - "plugins": [ - { - "id": "com.TablePro.DynamoDBDriverPlugin", - "name": "DynamoDB", - "version": "1.0.15", - "summary": "old", - "category": "database-driver", - "binaries": [ - {"architecture": "arm64", "pluginKitVersion": None, "downloadURL": "legacy"}, - ], - } - ], - } - - class Args: - id = "com.TablePro.DynamoDBDriverPlugin" - name = "DynamoDB" - version = "1.0.16" - summary = "new" - db_type_ids = '["dynamodb"]' - arm64_url = "https://x/arm64" - arm64_sha = "a" - x86_64_url = "https://x/x86_64" - x86_64_sha = "b" - min_app_version = "0.43.0" - icon = "icon" - homepage = "https://tablepro.app" - category = "database-driver" - plugin_kit_version = 14 - keep_kit_versions = 2 - - result = update_registry.update_plugin_entry(manifest, Args()) - entry = next(p for p in result["plugins"] if p["id"] == Args.id) - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14], kits - assert all(update_registry.kit_version(b) is not None for b in entry["binaries"]) - - -def _args(version, pkv, keep=2, min_app_version="0.43.0"): - class Args: - id = "com.TablePro.DynamoDBDriverPlugin" - name = "DynamoDB" - summary = "new" - db_type_ids = '["dynamodb"]' - arm64_url = "https://x/arm64" - arm64_sha = "a" - x86_64_url = "https://x/x86_64" - x86_64_sha = "b" - icon = "icon" - homepage = "https://tablepro.app" - category = "database-driver" - - Args.version = version - Args.plugin_kit_version = pkv - Args.keep_kit_versions = keep - Args.min_app_version = min_app_version - return Args - - -def _manifest(*kit_versions): - return { - "schemaVersion": 2, - "plugins": [ - { - "id": "com.TablePro.DynamoDBDriverPlugin", - "name": "DynamoDB", - "version": "1.0.15", - "summary": "old", - "category": "database-driver", - "binaries": [ - { - "architecture": arch, - "pluginKitVersion": pkv, - "downloadURL": f"old-{pkv}-{arch}", - "sha256": "x", - } - for pkv in kit_versions - for arch in ("arm64", "x86_64") - ], - } - ], - } - - -def test_publishing_a_new_kit_version_keeps_the_previous_one(): - """The retention policy's whole point: a user on the previous app can still install. - - Nothing covered this. The one existing merge test starts from a null-version binary, which - is dropped rather than retained, so the surviving-binary path was never exercised. - """ - result = update_registry.update_plugin_entry(_manifest(14), _args("1.0.16", 15)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14, 15, 15], kits - urls = {b["downloadURL"] for b in entry["binaries"]} - assert "old-14-arm64" in urls, urls - - -def test_a_third_kit_version_evicts_the_oldest(): - result = update_registry.update_plugin_entry(_manifest(13, 14), _args("1.0.17", 15)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14, 15, 15], kits - - -def test_republishing_the_same_kit_version_replaces_its_binaries(): - result = update_registry.update_plugin_entry(_manifest(14), _args("1.0.16", 14)) - entry = next(p for p in result["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - kits = sorted(update_registry.kit_version(b) for b in entry["binaries"]) - assert kits == [14, 14], kits - urls = {b["downloadURL"] for b in entry["binaries"]} - assert urls == {"https://x/arm64", "https://x/x86_64"}, urls - - -def test_publishing_below_the_retained_window_refuses(): - """Otherwise the entry advertises a version whose binary the prune just discarded.""" - try: - update_registry.update_plugin_entry(_manifest(15, 16), _args("1.0.18", 14)) - except SystemExit as error: - assert "older than the" in str(error), error - else: - raise AssertionError("expected SystemExit when the new binary would be pruned away") - - -def _entry(manifest): - return next(p for p in manifest["plugins"] if p["id"] == "com.TablePro.DynamoDBDriverPlugin") - - -def test_min_app_version_admits_the_oldest_retained_binary(): - manifest = {"schemaVersion": 2, "plugins": []} - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 25, keep=3, min_app_version="0.73.0") - ) - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.17", 33, keep=3, min_app_version="0.76.1") - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.73.0", entry["minAppVersion"] - assert { - (update_registry.kit_version(b), b["minAppVersion"]) for b in entry["binaries"] - } == {(25, "0.73.0"), (33, "0.76.1")}, entry["binaries"] - - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.18", 33, keep=3, min_app_version="0.76.1") - ) - assert _entry(manifest)["minAppVersion"] == "0.73.0", _entry(manifest)["minAppVersion"] - - -def test_min_app_version_compares_numerically(): - manifest = {"schemaVersion": 2, "plugins": []} - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 25, keep=3, min_app_version="0.9.0") - ) - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.17", 33, keep=3, min_app_version="0.10.0") - ) - assert _entry(manifest)["minAppVersion"] == "0.9.0", _entry(manifest)["minAppVersion"] - - -def test_min_app_version_rises_once_the_oldest_binary_is_evicted(): - manifest = {"schemaVersion": 2, "plugins": []} - for pkv, app in ((25, "0.73.0"), (30, "0.74.0"), (32, "0.75.0"), (33, "0.76.1")): - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", pkv, keep=3, min_app_version=app) - ) - entry = _entry(manifest) - assert sorted({update_registry.kit_version(b) for b in entry["binaries"]}) == [30, 32, 33] - assert entry["minAppVersion"] == "0.74.0", entry["minAppVersion"] - - -def test_republishing_a_kit_never_raises_its_minimum_app_version(): - manifest = {"schemaVersion": 2, "plugins": []} - for pkv, app in ((33, "0.76.0"), (33, "0.76.1"), (34, "0.77.0"), (35, "0.78.0")): - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", pkv, keep=3, min_app_version=app) - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.76.0", entry["minAppVersion"] - assert sorted( - b["minAppVersion"] for b in entry["binaries"] if update_registry.kit_version(b) == 33 - ) == ["0.76.0", "0.76.0"], entry["binaries"] - - -def test_republishing_a_kit_never_raises_it_over_a_binary_without_its_own_value(): - manifest = _manifest(33) - _entry(manifest)["minAppVersion"] = "0.76.0" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.1") - ) - entry = _entry(manifest) - assert entry["minAppVersion"] == "0.76.0", entry["minAppVersion"] - assert {b["minAppVersion"] for b in entry["binaries"]} == {"0.76.0"}, entry["binaries"] - - -def test_binaries_without_their_own_min_app_version_keep_the_entry_value(): - manifest = _manifest(30) - _entry(manifest)["minAppVersion"] = "0.74.0" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.1") - ) - assert _entry(manifest)["minAppVersion"] == "0.74.0", _entry(manifest)["minAppVersion"] - - manifest = _manifest(30) - _entry(manifest)["minAppVersion"] = "0.76.1" - manifest = update_registry.update_plugin_entry( - manifest, _args("1.0.16", 33, keep=3, min_app_version="0.76.0") - ) - assert _entry(manifest)["minAppVersion"] == "0.76.0", _entry(manifest)["minAppVersion"] - - -_REQUIRED_FLAGS = [ - "--manifest", "plugins.json", - "--id", "com.TablePro.DynamoDBDriverPlugin", - "--name", "DynamoDB", - "--version", "1.0.16", - "--summary", "new", - "--db-type-ids", '["dynamodb"]', - "--arm64-url", "https://x/arm64", - "--arm64-sha", "a", - "--x86_64-url", "https://x/x86_64", - "--x86_64-sha", "b", - "--min-app-version", "0.76.1", - "--icon", "icon", - "--homepage", "https://tablepro.app", - "--plugin-kit-version", "33", -] - - -def _parse(flags): - with mock.patch.object(sys, "argv", ["update-registry.py", *flags]): - return update_registry.parse_args() - - -def test_omitting_the_retention_count_refuses_to_run(): - stderr = io.StringIO() - try: - with contextlib.redirect_stderr(stderr): - args = _parse(_REQUIRED_FLAGS) - except SystemExit as error: - assert error.code != 0, error.code - assert "--keep-kit-versions" in stderr.getvalue(), stderr.getvalue() - else: - raise AssertionError( - f"expected SystemExit without --keep-kit-versions, got {args.keep_kit_versions}" - ) - - -def test_the_retention_count_is_the_one_passed(): - assert _parse([*_REQUIRED_FLAGS, "--keep-kit-versions", "3"]).keep_kit_versions == 3 - - -if __name__ == "__main__": - test_kit_version_rejects_non_int() - test_prune_drops_null_kit_binary() - test_prune_keeps_only_two_newest() - test_update_entry_drops_legacy_null_binary() - test_publishing_a_new_kit_version_keeps_the_previous_one() - test_a_third_kit_version_evicts_the_oldest() - test_republishing_the_same_kit_version_replaces_its_binaries() - test_publishing_below_the_retained_window_refuses() - test_min_app_version_admits_the_oldest_retained_binary() - test_min_app_version_compares_numerically() - test_min_app_version_rises_once_the_oldest_binary_is_evicted() - test_republishing_a_kit_never_raises_its_minimum_app_version() - test_republishing_a_kit_never_raises_it_over_a_binary_without_its_own_value() - test_binaries_without_their_own_min_app_version_keep_the_entry_value() - test_omitting_the_retention_count_refuses_to_run() - test_the_retention_count_is_the_one_passed() - print("All update-registry tests passed.") diff --git a/.github/scripts/update-registry.py b/.github/scripts/update-registry.py deleted file mode 100755 index 9cf3ab0d98..0000000000 --- a/.github/scripts/update-registry.py +++ /dev/null @@ -1,214 +0,0 @@ -#!/usr/bin/env python3 -""" -Update a TablePro plugin registry manifest (plugins.json) with a new binary set. - -For a given plugin ID and PluginKit version, this script: - - Adds or replaces the arm64 and x86_64 binaries for that PluginKit version - - Preserves binaries for other PluginKit versions (up to --keep-kit-versions) - - Drops binaries for PluginKit versions older than (newest - keep + 1) - - Updates plugin-level metadata (name, summary, etc.) from the newest binary set - - Records minAppVersion on each new binary and sets the entry's minAppVersion to the lowest - across the retained binaries. Binaries on one PluginKit version share one ABI, so republishing - a kit keeps the lowest minAppVersion that kit already carried rather than raising it - - Sets schemaVersion to 2 - - Writes atomically via a temp file + rename -""" - -import argparse -import json -import os -import re -import sys -import tempfile - - -def parse_args(): - parser = argparse.ArgumentParser(description="Update plugins.json registry entry") - parser.add_argument("--manifest", required=True) - parser.add_argument("--id", required=True) - parser.add_argument("--name", required=True) - parser.add_argument("--version", required=True) - parser.add_argument("--summary", required=True) - parser.add_argument("--db-type-ids", required=True) - parser.add_argument("--arm64-url", required=True) - parser.add_argument("--arm64-sha", required=True) - parser.add_argument("--x86_64-url", required=True) - parser.add_argument("--x86_64-sha", required=True) - parser.add_argument("--min-app-version", required=True) - parser.add_argument("--icon", required=True) - parser.add_argument("--homepage", required=True) - parser.add_argument("--category", default="database-driver") - parser.add_argument("--plugin-kit-version", required=True, type=int) - parser.add_argument( - "--keep-kit-versions", - required=True, - type=int, - help="Number of distinct PluginKit versions to retain per plugin. Oldest dropped first.", - ) - return parser.parse_args() - - -def load_manifest(path): - with open(path, "r", encoding="utf-8") as file: - return json.load(file) - - -def write_manifest_atomic(path, manifest): - dir_path = os.path.dirname(os.path.abspath(path)) - fd, tmp_path = tempfile.mkstemp(dir=dir_path, suffix=".tmp") - try: - with os.fdopen(fd, "w", encoding="utf-8") as file: - json.dump(manifest, file, indent=2) - file.write("\n") - os.replace(tmp_path, path) - except Exception: - try: - os.unlink(tmp_path) - except OSError: - pass - raise - - -def kit_version(binary): - value = binary.get("pluginKitVersion") - return value if isinstance(value, int) else None - - -def prune_old_kit_versions(binaries, keep_count): - # Drop binaries without a concrete integer PluginKit version. The app matches - # binaries on an exact integer, so a null/missing version can never resolve and - # only shadows valid binaries (the cause of the DynamoDB noCompatibleBinary in #1322). - typed = [b for b in binaries if kit_version(b) is not None] - - versions_seen = [] - for binary in typed: - pkv = kit_version(binary) - if pkv not in versions_seen: - versions_seen.append(pkv) - - versions_seen.sort(reverse=True) - versions_to_keep = set(versions_seen[:keep_count]) - - return [b for b in typed if kit_version(b) in versions_to_keep] - - -def app_version_key(version): - return tuple(int(part) for part in re.findall(r"\d+", version)) - - -def kit_min_app_version(existing_entry, pkv, requested): - if existing_entry is None: - return requested - previous_entry_value = existing_entry.get("minAppVersion") - carried = [ - b.get("minAppVersion") or previous_entry_value - for b in existing_entry.get("binaries", []) - if kit_version(b) == pkv - ] - return min([requested, *filter(None, carried)], key=app_version_key) - - -def entry_min_app_version(binaries, previous_entry_value): - candidates = [b["minAppVersion"] for b in binaries if b.get("minAppVersion")] - if previous_entry_value and any(not b.get("minAppVersion") for b in binaries): - candidates.append(previous_entry_value) - return min(candidates, key=app_version_key) - - -def update_plugin_entry(manifest, args): - bundle_id = args.id - db_type_ids = json.loads(args.db_type_ids) - pkv = args.plugin_kit_version - - existing_plugins = manifest.get("plugins", []) - existing_entry = next((p for p in existing_plugins if p["id"] == bundle_id), None) - min_app_version = kit_min_app_version(existing_entry, pkv, args.min_app_version) - - new_binaries = [ - { - "architecture": "arm64", - "pluginKitVersion": pkv, - "downloadURL": args.arm64_url, - "sha256": args.arm64_sha, - "minAppVersion": min_app_version, - }, - { - "architecture": "x86_64", - "pluginKitVersion": pkv, - "downloadURL": args.x86_64_url, - "sha256": args.x86_64_sha, - "minAppVersion": min_app_version, - }, - ] - - if existing_entry is not None: - surviving = [ - b for b in existing_entry.get("binaries", []) - if kit_version(b) is not None and kit_version(b) != pkv - ] - merged_binaries = surviving + new_binaries - else: - merged_binaries = new_binaries - - merged_binaries = prune_old_kit_versions(merged_binaries, args.keep_kit_versions) - - # The prune keeps the newest N PluginKit versions, so publishing for an older one drops the - # binary that was just built while still advertising its version number. The entry would then - # point every app at a download that does not exist for it. - if not any(kit_version(b) == pkv for b in merged_binaries): - raise SystemExit( - f"PluginKit {pkv} is older than the {args.keep_kit_versions} retained versions; " - f"publishing would advertise {args.version} with no binary for it" - ) - - previous_min_app_version = existing_entry.get("minAppVersion") if existing_entry else None - - updated_entry = { - "id": bundle_id, - "name": args.name, - "version": args.version, - "summary": args.summary, - "author": {"name": "TablePro", "url": "https://tablepro.app"}, - "homepage": args.homepage, - "category": args.category, - "databaseTypeIds": db_type_ids, - "iconName": args.icon, - "isVerified": True, - "minAppVersion": entry_min_app_version(merged_binaries, previous_min_app_version), - "binaries": merged_binaries, - } - - if existing_entry is not None and existing_entry.get("metadata"): - updated_entry["metadata"] = existing_entry["metadata"] - - manifest["plugins"] = [p for p in existing_plugins if p["id"] != bundle_id] - manifest["plugins"].append(updated_entry) - manifest["schemaVersion"] = 2 - - return manifest - - -def main(): - args = parse_args() - - if not os.path.exists(args.manifest): - print(f"ERROR: manifest not found: {args.manifest}", file=sys.stderr) - sys.exit(1) - - manifest = load_manifest(args.manifest) - manifest = update_plugin_entry(manifest, args) - write_manifest_atomic(args.manifest, manifest) - - entry = next(p for p in manifest["plugins"] if p["id"] == args.id) - kept_versions = sorted( - {b.get("pluginKitVersion", 0) for b in entry["binaries"]}, - reverse=True, - ) - print( - f"Updated {args.id} v{args.version} (PluginKit {args.plugin_kit_version}). " - f"Retained PluginKit versions: {kept_versions}" - ) - - -if __name__ == "__main__": - main() diff --git a/.github/workflows/build-plugin.yml b/.github/workflows/build-plugin.yml deleted file mode 100644 index e36897cfaa..0000000000 --- a/.github/workflows/build-plugin.yml +++ /dev/null @@ -1,490 +0,0 @@ -name: Build Plugin - -on: - push: - tags: ["plugin-*-v*"] - workflow_dispatch: - inputs: - tags: - description: > - Comma-separated plugin tag:pluginKitVersion pairs. - pluginKitVersion defaults to TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig. - Examples: plugin-mongodb-v1.0.25, plugin-mongodb-v1.0.25:13 - required: true - type: string - baseRef: - description: > - Build the plugin from this ref instead of the default branch, so its binary links against - that release's PluginKit and declares that release's TableProPluginKitVersion. Use a - shipped app tag (v0.73.0) to publish a driver fix users on that release can install. - Leave empty to build from the branch the workflow runs on. - required: false - type: string - -# Declared per job rather than here. resolve-tags reads an integer out of an xcconfig and needs -# nothing, and a workflow-level grant hands it a write token it has no use for. -permissions: - contents: read - -# Without this, two runs for the same tag race each other through a delete-and-recreate of the same -# GitHub Release. Never cancel one in flight: it holds a notarization submission open with Apple and -# publishes release assets. -concurrency: - group: build-plugin-${{ github.ref }} - cancel-in-progress: false - -jobs: - resolve-tags: - name: Resolve Plugin Tags - runs-on: ubuntu-latest - outputs: - matrix: ${{ steps.tags.outputs.matrix }} - currentPluginKitVersion: ${{ steps.pkv.outputs.version }} - runner: ${{ steps.runner.outputs.runner }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - # The build job runs the setup-xcode action of the ref it builds. A ref from before Xcode 27 - # pins 26.x there, and the xcode-27 image has no Xcode 26. - - name: Pick the runner for the ref being built - id: runner - env: - REF: ${{ inputs.baseRef || github.ref }} - run: | - set -euo pipefail - git fetch --no-tags --depth=1 origin "$REF" - ACTION="$(git show FETCH_HEAD:.github/actions/setup-xcode/action.yml)" - if grep -qE '(default: "|Xcode_)26\.' <<< "$ACTION"; then - echo "runner=macos-26" >> "$GITHUB_OUTPUT" - else - echo "runner=xcode-27" >> "$GITHUB_OUTPUT" - fi - - - name: Validate registry update script - run: python3 .github/scripts/test_update_registry.py - - - name: Check the plugin manifest against the plugin classes - run: python3 scripts/ci/check-plugin-manifest.py - - - name: Read the PluginKit version - id: pkv - run: | - VERSION=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' \ - Configs/PluginKit.xcconfig) - if [ -z "$VERSION" ]; then - echo "::error::Could not parse TABLEPRO_PLUGINKIT_VERSION from Configs/PluginKit.xcconfig" - exit 1 - fi - echo "version=$VERSION" >> "$GITHUB_OUTPUT" - echo "TABLEPRO_PLUGINKIT_VERSION=$VERSION" - - - name: Build matrix - id: tags - env: - DEFAULT_PKV: ${{ steps.pkv.outputs.version }} - INPUT_TAGS: ${{ inputs.tags }} - REF_NAME: ${{ github.ref_name }} - run: | - if [ -n "$INPUT_TAGS" ]; then - IFS=',' read -ra RAW_TAGS <<< "$INPUT_TAGS" - else - RAW_TAGS=("$REF_NAME") - fi - - JSON='{"include":[' - FIRST=true - for ITEM in "${RAW_TAGS[@]}"; do - ITEM="${ITEM#"${ITEM%%[![:space:]]*}"}" - ITEM="${ITEM%"${ITEM##*[![:space:]]}"}" - TAG=$(echo "$ITEM" | cut -d: -f1) - PKV=$(echo "$ITEM" | cut -d: -f2 -s) - if [ -z "$PKV" ]; then - PKV="$DEFAULT_PKV" - fi - - # Both values are checked here, on ubuntu, before any macOS job exists to import a - # signing certificate. TAG reaches a shell and PKV is spliced unquoted into the JSON - # below, and neither had ever been validated: nothing downstream checks the version - # format either, and update-registry.py takes --version as an arbitrary string, so a - # malformed tag publishes a garbage version into the plugins.json every app reads. - # Checked against all 366 existing plugin-* tags: every one matches. - if [[ ! "$TAG" =~ ^plugin-[a-z0-9-]+-v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then - echo "::error::Malformed plugin tag: $TAG" - exit 1 - fi - if [[ ! "$PKV" =~ ^[0-9]+$ ]]; then - echo "::error::Non-numeric pluginKitVersion for $TAG: $PKV" - exit 1 - fi - - if [ "$FIRST" = true ]; then FIRST=false; else JSON+=','; fi - JSON+="{\"tag\":\"$TAG\",\"pluginKitVersion\":$PKV}" - done - JSON+=']}' - echo "matrix=$JSON" >> "$GITHUB_OUTPUT" - echo "Matrix: $JSON" - - build-plugin: - name: "Build ${{ matrix.tag }} (PluginKit ${{ matrix.pluginKitVersion }})" - needs: resolve-tags - # A dispatch runs under main's workflow group, so a tag push for the same plugin tag would - # otherwise stage its release at the same time. - concurrency: - group: build-plugin-release-${{ matrix.tag }} - cancel-in-progress: false - permissions: - contents: write - runs-on: ${{ needs.resolve-tags.outputs.runner }} - timeout-minutes: 30 - strategy: - matrix: ${{ fromJson(needs.resolve-tags.outputs.matrix) }} - fail-fast: false - - steps: - # A plugin's declared TableProPluginKitVersion and the symbols it links against both come - # from the tree it is built in, so they can only agree by building at the ref whose kit the - # binary should carry. Relabelling a binary built from main to an older kit would make an - # older app accept it and then fail Bundle.loadAndReturnError, which is #1917 and 0.49.0. - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - lfs: true - ref: ${{ inputs.baseRef || github.ref }} - - - name: Pull LFS files - run: git lfs pull - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - - name: Setup XcodeGen - uses: ./.github/actions/setup-xcodegen - - - name: Download static libraries - env: - GH_TOKEN: ${{ github.token }} - run: scripts/download-libs.sh - - # No provisioning profile: a .tableplugin is signed with Developer ID and notarized, and - # never carries an embedded profile the way the app does. - - name: Set up Apple signing - uses: ./.github/actions/apple-signing - with: - certificate-p12: ${{ secrets.CERTIFICATES_P12 }} - certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} - apple-id: ${{ secrets.APPLE_ID }} - team-id: ${{ secrets.APPLE_TEAM_ID }} - notary-password: ${{ secrets.NOTARY_PASSWORD }} - - - name: Resolve plugin info - id: plugin - env: - MATRIX_TAG: ${{ matrix.tag }} - # The metadata lives in .github/plugin-registry.json. It used to be a 120-line `case` block - # here, which is shell holding data, and seven of its icon values had drifted away from the - # plugin classes they describe with nothing to notice. - run: python3 scripts/ci/plugin-info.py "$MATRIX_TAG" >> "$GITHUB_OUTPUT" - - - name: Build Dameng native bridge - if: ${{ contains(matrix.tag, 'plugin-dameng-') }} - run: ./scripts/build-dameng.sh both - - - name: Set up Go for the SAP HANA helper - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0 - with: - go-version-file: Native/HanaBridge/go.mod - cache-dependency-path: Native/HanaBridge/go.sum - - # Universal, because both architecture builds below embed the same helper. build-plugin.sh - # signs it with Developer ID, the hardened runtime and a timestamp, and checks all three. - - name: Build the SAP HANA helper - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - run: ./scripts/build-hana.sh both - - # The helper ships inside the plugin, and only a release changes what users run. - - name: Check the SAP HANA helper for known vulnerabilities - if: ${{ contains(matrix.tag, 'plugin-hana-') }} - working-directory: Native/HanaBridge - run: go run golang.org/x/vuln/cmd/govulncheck@v1.8.0 ./... - - # NOTARIZE is not optional. Gatekeeper refuses to load an unnotarized plugin bundle - # into TablePro, so a build that skips it publishes a driver no user can open. - - name: Build plugin binaries - env: - TEAM_ID: ${{ secrets.APPLE_TEAM_ID }} - NOTARIZE: "true" - TARGET: ${{ steps.plugin.outputs.target }} - VERSION: ${{ steps.plugin.outputs.version }} - # One invocation, not one per architecture. The script has always taken "both", and - # each invocation regenerates both Xcode projects and wipes build/DerivedData, so calling - # it twice threw away everything the first architecture had compiled. - run: ./scripts/build-plugin.sh "$TARGET" both "$VERSION" - - # Assert the result rather than trusting the flag. The previous notarization step was - # gated on an environment variable nothing defined, so it never ran and nothing noticed - # for every plugin release the registry has ever published. - # Before anything is published. The registry serves these to users directly, so a bundle - # that will not load is a plugin nobody can install. - - name: Verify the built bundles load - env: - BUNDLE_NAME: ${{ steps.plugin.outputs.bundleName }} - run: | - set -euo pipefail - for ARCH in arm64 x86_64; do - WORK="$(mktemp -d)" - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - # The distribution zip carries the plugin alone, so dyld has nothing to resolve - # @rpath/TableProPluginKit.framework against and every plugin fails for the wrong - # reason. build-plugin.sh builds both into one flat CONFIGURATION_BUILD_DIR, so the - # framework is put beside the unzipped bundle rather than added as a second search - # directory, which would make the script scan the build dir as another bundle. - cp -R "build/Plugins/${ARCH}/TableProPluginKit.framework" "$WORK/" - # Only the native slice can be dlopened on this runner; the other is checked by - # `lipo -info` having produced it and by Gatekeeper below. - if [ "$ARCH" = "arm64" ]; then - scripts/ci/verify-plugin-loads.sh "$WORK" - fi - rm -rf "$WORK" - done - - - name: Verify the built bundles are notarized - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - for ARCH in arm64 x86_64; do - WORK=$(mktemp -d) - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - PLUGIN="$WORK/${BUNDLE_NAME}.tableplugin" - if ! xcrun stapler validate "$PLUGIN"; then - echo "::error::${BUNDLE_NAME}-${ARCH} carries no stapled notarization ticket. Gatekeeper refuses to load such a bundle into TablePro." - exit 1 - fi - if ! spctl -a -vvv -t open --context context:primary-signature "$PLUGIN" 2>&1 | grep -q "source=Notarized Developer ID"; then - echo "::error::Gatekeeper rejects ${BUNDLE_NAME}-${ARCH}:" - spctl -a -vvv -t open --context context:primary-signature "$PLUGIN" || true - exit 1 - fi - echo "Verified ${BUNDLE_NAME}-${ARCH}: notarized, stapled and accepted by Gatekeeper." - done - - - name: Verify built PluginKit version matches the release label - env: - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - EXPECTED="$MATRIX_PKV" - for ARCH in arm64 x86_64; do - WORK=$(mktemp -d) - unzip -oq "build/Plugins/${BUNDLE_NAME}-${ARCH}.zip" -d "$WORK" - PLIST=$(find "$WORK" -path '*.tableplugin/Contents/Info.plist' | head -1) - if [ -z "$PLIST" ]; then - echo "::error::Could not find Info.plist in the built ${BUNDLE_NAME}-${ARCH} bundle." - exit 1 - fi - ACTUAL=$(plutil -extract TableProPluginKitVersion raw "$PLIST") - if [ "$ACTUAL" != "$EXPECTED" ]; then - echo "::error::${BUNDLE_NAME}-${ARCH} was built for PluginKit $ACTUAL but this release is labeled PluginKit $EXPECTED. Refusing to publish a mislabeled binary. Re-release from a commit whose plugin Info.plist matches the target PluginKit version." - exit 1 - fi - echo "Verified ${BUNDLE_NAME}-${ARCH}: built PluginKit $ACTUAL matches the release label." - done - - - name: Read checksums - id: sha - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - ARM64_SHA=$(cat "build/Plugins/${BUNDLE_NAME}-arm64.zip.sha256") - X86_SHA=$(cat "build/Plugins/${BUNDLE_NAME}-x86_64.zip.sha256") - { - echo "arm64=$ARM64_SHA" - echo "x86_64=$X86_SHA" - } >> "$GITHUB_OUTPUT" - - # The checkout above is `baseRef` when one is given, and a shipped app tag can predate - # draft-release.sh. - - name: Check out the release script - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.sha }} - path: .release-tooling - sparse-checkout: scripts/ci - persist-credentials: false - - # A published release is immutable, so both zips go on the draft first. - - name: Create the draft release - id: draft - env: - GH_TOKEN: ${{ github.token }} - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - PLUGIN_VERSION: ${{ steps.plugin.outputs.version }} - run: | - TAG="$MATRIX_TAG" - DISPLAY_NAME="${{ steps.plugin.outputs.displayName }}" - VERSION="$PLUGIN_VERSION" - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - ARM64_SHA="${{ steps.sha.outputs.arm64 }}" - X86_SHA="${{ steps.sha.outputs.x86_64 }}" - PKV="$MATRIX_PKV" - - RELEASE_BODY="## $DISPLAY_NAME v$VERSION - - Plugin release for TablePro (PluginKit $PKV). - - ### Installation - TablePro offers to install this plugin where it is needed. To install it by hand, add the zip below in Settings > Plugins. - - ### SHA-256 - - ARM64: \`$ARM64_SHA\` - - x86_64: \`$X86_SHA\`" - - EXISTING_PKV=$(gh api "repos/${GITHUB_REPOSITORY}/releases/tags/${TAG}" --jq .body 2>/dev/null \ - | grep -oiE 'PluginKit [0-9]+' | grep -oE '[0-9]+' | head -1 || true) - if [ -n "$EXISTING_PKV" ] && [ "$EXISTING_PKV" != "$PKV" ]; then - echo "::error::Release $TAG already exists for PluginKit $EXISTING_PKV. Refusing to overwrite it with PluginKit $PKV; publish the new ABI under a new plugin version." - exit 1 - fi - - # Tagged at the commit this job built, which is `baseRef` when one was given. Without - # --target GitHub tags the default branch's head instead, so a plugin built against a - # shipped release recorded whatever main had moved on to: the v0.75.0 plugins point at a - # commit carrying PluginKit 33, while their binaries are 32. - NOTES="$(mktemp)" - printf '%s' "$RELEASE_BODY" > "$NOTES" - .release-tooling/scripts/ci/draft-release.sh --target "$(git rev-parse HEAD)" \ - "$TAG" "$DISPLAY_NAME v$VERSION" "$NOTES" \ - "build/Plugins/${BUNDLE_NAME}-arm64.zip" "build/Plugins/${BUNDLE_NAME}-x86_64.zip" - - # Without make_latest=false, publishing marks the plugin release Latest instead of the app's. - - name: Publish the release - if: steps.draft.outputs.id != '' - env: - GH_TOKEN: ${{ github.token }} - RELEASE_ID: ${{ steps.draft.outputs.id }} - run: | - gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \ - -F draft=false -f make_latest=false --jq '"Published \(.tag_name): \(.html_url)"' - - - name: Verify published assets match the PluginKit label - env: - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - run: | - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - TAG="$MATRIX_TAG" - PKV="$MATRIX_PKV" - REPO="${{ github.repository }}" - for ARCH in arm64 x86_64; do - URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-${ARCH}.zip" - WORK=$(mktemp -d) - DOWNLOADED="" - for attempt in 1 2 3 4 5; do - if curl -fsSL "$URL" -o "$WORK/asset.zip"; then - DOWNLOADED="yes" - break - fi - echo "Published asset not downloadable yet (attempt $attempt/5), retrying in 5s..." - sleep 5 - done - if [ -z "$DOWNLOADED" ]; then - echo "::error::Could not download the published asset at $URL to verify its PluginKit version." - exit 1 - fi - unzip -oq "$WORK/asset.zip" -d "$WORK" - PLIST=$(find "$WORK" -path '*.tableplugin/Contents/Info.plist' | head -1) - if [ -z "$PLIST" ]; then - echo "::error::No .tableplugin Info.plist in the published ${BUNDLE_NAME}-${ARCH} asset." - exit 1 - fi - ACTUAL=$(plutil -extract TableProPluginKitVersion raw "$PLIST") - if [ "$ACTUAL" != "$PKV" ]; then - echo "::error::Published ${BUNDLE_NAME}-${ARCH} at $URL is PluginKit $ACTUAL but the registry will record $PKV. Refusing to record a mismatched binary." - exit 1 - fi - echo "Verified published ${BUNDLE_NAME}-${ARCH}: served PluginKit $ACTUAL matches the registry label." - done - - - name: Update plugin registry - env: - REGISTRY_DEPLOY_KEY: ${{ secrets.REGISTRY_DEPLOY_KEY }} - GH_TOKEN: ${{ github.token }} - MATRIX_TAG: ${{ matrix.tag }} - MATRIX_PKV: ${{ matrix.pluginKitVersion }} - PLUGIN_VERSION: ${{ steps.plugin.outputs.version }} - run: | - TAG="$MATRIX_TAG" - BUNDLE_NAME="${{ steps.plugin.outputs.bundleName }}" - BUNDLE_ID="${{ steps.plugin.outputs.bundleId }}" - DISPLAY_NAME="${{ steps.plugin.outputs.displayName }}" - VERSION="$PLUGIN_VERSION" - SUMMARY="${{ steps.plugin.outputs.summary }}" - DB_TYPE_IDS='${{ steps.plugin.outputs.dbTypeIds }}' - MIN_APP_VERSION="${{ steps.plugin.outputs.minAppVersion }}" - ICON="${{ steps.plugin.outputs.icon }}" - HOMEPAGE="${{ steps.plugin.outputs.homepage }}" - CATEGORY="${{ steps.plugin.outputs.category }}" - ARM64_SHA="${{ steps.sha.outputs.arm64 }}" - X86_SHA="${{ steps.sha.outputs.x86_64 }}" - PKV="$MATRIX_PKV" - REPO="${{ github.repository }}" - ARM64_URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-arm64.zip" - X86_64_URL="https://github.com/${REPO}/releases/download/${TAG}/${BUNDLE_NAME}-x86_64.zip" - - SCRIPT_PATH="$(pwd)/.github/scripts/update-registry.py" - - WORK=$(mktemp -d) - eval "$(ssh-agent -s)" - trap 'ssh-add -D 2>/dev/null || true; eval "$(ssh-agent -k)" 2>/dev/null || true' EXIT - echo "$REGISTRY_DEPLOY_KEY" | ssh-add - - - git clone git@github.com:TableProApp/plugins.git "$WORK/registry" - cd "$WORK/registry" - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - - MAX_RETRIES=10 - for attempt in $(seq 1 $MAX_RETRIES); do - echo "Registry update attempt $attempt/$MAX_RETRIES" - git reset --hard origin/main - git pull --rebase origin main - - python3 "$SCRIPT_PATH" \ - --manifest plugins.json \ - --id "$BUNDLE_ID" \ - --name "$DISPLAY_NAME" \ - --version "$VERSION" \ - --summary "$SUMMARY" \ - --db-type-ids "$DB_TYPE_IDS" \ - --arm64-url "$ARM64_URL" \ - --arm64-sha "$ARM64_SHA" \ - --x86_64-url "$X86_64_URL" \ - --x86_64-sha "$X86_SHA" \ - --min-app-version "$MIN_APP_VERSION" \ - --icon "$ICON" \ - --homepage "$HOMEPAGE" \ - --category "$CATEGORY" \ - --plugin-kit-version "$PKV" \ - --keep-kit-versions 3 - - git add plugins.json - git commit -m "Update $DISPLAY_NAME to v$VERSION (PluginKit $PKV)" - - if git push; then - echo "Registry updated on attempt $attempt" - curl -sf "https://purge.jsdelivr.net/gh/TableProApp/plugins@main/plugins.json" >/dev/null \ - && echo "Purged jsDelivr cache for plugins.json" \ - || echo "::warning::jsDelivr purge request failed; the cache will refresh on its own shortly" - break - fi - - if [ "$attempt" -eq "$MAX_RETRIES" ]; then - echo "::error::Failed to push registry update after $MAX_RETRIES attempts" - exit 1 - fi - - DELAY=$((2 + RANDOM % 4)) - echo "Push rejected (concurrent update), retrying in ${DELAY}s..." - sleep "$DELAY" - done - - echo "$DISPLAY_NAME v$VERSION released (PluginKit $PKV)" diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml deleted file mode 100644 index 14e249bfa2..0000000000 --- a/.github/workflows/build.yml +++ /dev/null @@ -1,512 +0,0 @@ -name: Build TablePro - -on: - workflow_dispatch: - # No paths filter here. GitHub does not evaluate one for a tag push, and a tag push is the - # only thing that triggers this workflow, so the paths-ignore list that used to sit here - # never excluded anything. - push: - tags: ["v*"] - -# A release must never be cancelled part-way through: the build jobs hold notarization -# submissions open with Apple, and the release job pushes a commit and publishes artifacts. -# Re-running the same tag queues behind the run already in flight rather than racing it. -concurrency: - group: build-tablepro-${{ github.ref }} - cancel-in-progress: false - -# The release job raises this to write for itself. Everything else here builds and notarizes and -# needs nothing beyond the checkout. -permissions: - contents: read - -jobs: - # Runs no suite. A release commit changes only release files, so its own push skips both suites. - # For each suite the verdict is the newest first-parent commit on main where its job ran, waited - # for when still running, and it holds only if nothing the suite reads changed since. - verdict: - name: Tested on main - if: startsWith(github.ref, 'refs/tags/v') - runs-on: ubuntu-latest - timeout-minutes: 240 - permissions: - contents: read - actions: read - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Find the passing runs that cover this commit - env: - GH_TOKEN: ${{ github.token }} - GH_REPO: ${{ github.repository }} - run: | - set -euo pipefail - # The walk below follows the tag's own parents, so a tag off main would borrow main's verdict. - if ! git merge-base --is-ancestor HEAD origin/main; then - echo "::error::$GITHUB_SHA is not on main" - exit 1 - fi - covered() { - local workflow="$1" job="$2" suite="$3" sha run conclusion changed - for sha in $(git rev-list --first-parent --max-count=50 HEAD); do - run=$(gh run list --workflow "$workflow" --commit "$sha" --event push --limit 1 \ - --json databaseId --jq '.[0].databaseId // empty') || return 1 - [ -n "$run" ] || continue - gh run watch "$run" --interval 60 > /dev/null || return 1 - conclusion=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$run/jobs" \ - --jq ".jobs[] | select(.name == \"$job\") | .conclusion") || return 1 - case "$conclusion" in - skipped) continue ;; - success) ;; - *) echo "::error::$job ended '$conclusion' on $sha (run $run)"; return 1 ;; - esac - changed=$(scripts/ci/detect-changed-paths.sh --base "$sha" "$suite") || return 1 - if [ "$changed" != false ]; then - echo "::error::$job passed on $sha, but files it reads changed before $GITHUB_SHA" - return 1 - fi - echo "$job: run $run on $sha" >> "$GITHUB_STEP_SUMMARY" - return 0 - done - echo "::error::no $job run among the last 50 commits on main" - return 1 - } - status=0 - covered macos-tests.yml "macOS Tests" macos || status=1 - covered ios-tests.yml "iOS and Package Tests" ios || status=1 - exit "$status" - - build: - name: Build ${{ matrix.arch }} - runs-on: xcode-27 - timeout-minutes: ${{ matrix.timeout }} - strategy: - # No fail-fast. A failing arch would otherwise cancel its sibling mid-notarization, which is - # the same thing the concurrency block above exists to prevent. - fail-fast: false - matrix: - include: - # One budget for both arms. The arms run on separate runners at the same time, so a - # tighter budget on one buys nothing and only decides which arm dies first. v0.73.0 is - # what that cost: ARM64 reached "Build complete" at 30:01 and was cancelled 13 seconds - # into notarization, while x86_64 finished the whole job in 31:15. The two arms are not - # reliably ordered (over v0.62 to v0.67 x86_64 was the slower one, once by 22 minutes), - # and the notary queue is not something a build budget can predict, so the budget has to - # clear the slowest arm plus a notary wait rather than the arm's own past average. - - arch: arm64 - timeout: 45 - - arch: x86_64 - timeout: 45 - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - - name: Download static libraries - env: - GH_TOKEN: ${{ github.token }} - run: scripts/download-libs.sh --force - - # build-release.sh clones packages into ~/.spm-cache via -clonedSourcePackagesDirPath, - # so this restores the checkouts instead of re-cloning 30 repositories on every release. - # Package.resolved pins every revision and is tracked, which is why it can key the cache. - # macos-tests.yml saves it from main; a save here would land in this tag's scope, which no - # other tag can read. - - name: Restore Swift package checkouts - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - with: - path: ~/.spm-cache - key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} - restore-keys: spm-${{ runner.os }}- - - # create-dmg is the only Homebrew formula either arm needs. The MySQL plugin links - # Libs/libmariadb.a, which download-libs.sh vendors and prepare-libs.sh selects per - # architecture, and its headers live in Plugins/MySQLDriverPlugin/CMariaDB/include. - # LIBRARY_SEARCH_PATHS names $(SRCROOT)/Libs and no Homebrew prefix, so nothing in the - # build ever read mariadb-connector-c. macos-tests.yml proves it: its build job links all - # 31 plugin bundles with no Homebrew mariadb installed at all. - # - # The x86_64 arm is a cross-compile on an arm64 runner: every tool that runs during the - # build is native and the x86_64 slices of the static libraries are vendored in Libs. - # Bootstrapping a second Homebrew prefix cost five minutes a release to install a - # mariadb-connector-c the build never opened. create-dmg is a shell script, so the native - # prefix serves both arms. - - name: Install create-dmg - run: brew list create-dmg &>/dev/null || brew install create-dmg - - - name: Prepare libraries - run: scripts/ci/prepare-libs.sh ${{ matrix.arch }} - - - name: Create Secrets.xcconfig - env: - ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} - run: echo "ANALYTICS_HMAC_SECRET = ${ANALYTICS_HMAC_SECRET}" > Configs/Secrets.xcconfig - - - name: Setup XcodeGen - uses: ./.github/actions/setup-xcodegen - - - name: Set up Apple signing - uses: ./.github/actions/apple-signing - with: - certificate-p12: ${{ secrets.CERTIFICATES_P12 }} - certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} - apple-id: ${{ secrets.APPLE_ID }} - team-id: ${{ secrets.APPLE_TEAM_ID }} - notary-password: ${{ secrets.NOTARY_PASSWORD }} - provisioning-profile: ${{ secrets.PROVISIONING_PROFILE }} - - - name: Build ${{ matrix.arch }} - env: - ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} - NOTARIZE: "true" - run: scripts/build-release.sh ${{ matrix.arch }} - - - name: Verify build - run: scripts/ci/verify-build.sh ${{ matrix.arch }} - - - name: Package artifacts - env: - NOTARIZE: "true" - run: scripts/ci/package-artifacts.sh ${{ matrix.arch }} - - - name: Verify the update archive - run: scripts/ci/verify-update-archive.sh "build/Release/TablePro-${{ matrix.arch }}.tar.xz" - - - name: Upload artifacts - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 - with: - name: artifacts-${{ matrix.arch }} - path: | - build/Release/TablePro-*.dmg - build/Release/TablePro-*.tar.xz - - # Reads two integers out of an xcconfig and fetches a JSON manifest, so it wants nothing - # from macOS and does not need to spend a macOS runner minute (billed at ten times Linux). - registry-readiness: - name: Registry Readiness - runs-on: ubuntu-latest - if: startsWith(github.ref, 'refs/tags/v') - timeout-minutes: 5 - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - name: Verify registry has compatible plugin binaries - run: | - CONFIG="Configs/PluginKit.xcconfig" - CURRENT=$(sed -nE 's/^TABLEPRO_PLUGINKIT_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") - FLOOR=$(sed -nE 's/^TABLEPRO_PLUGINKIT_MINIMUM_VERSION[[:space:]]*=[[:space:]]*([0-9]+)[[:space:]]*$/\1/p' "$CONFIG") - echo "PluginKit floor=$FLOOR current=$CURRENT" - python3 scripts/check-registry-readiness.py --floor "$FLOOR" --current "$CURRENT" - - release: - name: Create GitHub Release - runs-on: xcode-27 - needs: [verdict, build, registry-readiness] - if: startsWith(github.ref, 'refs/tags/v') - # Holds APPCAST_DEPLOY_KEY, released only to v* tags. - environment: release - timeout-minutes: 10 - permissions: - contents: write - - steps: - - name: Checkout code - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Select Xcode - uses: ./.github/actions/setup-xcode - - # Only the two build artifacts. Unfiltered, this also pulled the four test-result bundles and - # test-products, which is 874MB fetched to use 106MB of it, and every megabyte is another - # chance for the blob store to drop the transfer: v0.69.0 failed here twice in a row with - # "Artifact download failed after 5 retries", both times inside two minutes. Flatten takes any - # *.dmg or *.tar.xz it finds, so one inside a test bundle could also reach the release. - - name: Download build artifacts - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 - with: - pattern: artifacts-* - path: artifacts-raw/ - merge-multiple: true - - - name: Flatten artifacts - run: | - mkdir -p artifacts/ - find artifacts-raw/ -type f \( -name "*.dmg" -o -name "*.tar.xz" \) -exec mv {} artifacts/ \; - rm -rf artifacts-raw/ - echo "Artifacts:" - ls -lh artifacts/ - - - name: Verify the tag matches the declared version - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - TAG_VERSION="${GITHUB_REF#refs/tags/v}" - XCCONFIG_VERSION=$(sed -n 's/^MARKETING_VERSION[[:space:]]*=[[:space:]]*//p' Configs/Version.xcconfig | tr -d ' ') - if [ "$TAG_VERSION" != "$XCCONFIG_VERSION" ]; then - echo "::error::tag v$TAG_VERSION does not match MARKETING_VERSION $XCCONFIG_VERSION" - exit 1 - fi - echo "tag and MARKETING_VERSION agree on $TAG_VERSION" - - # CURRENT_PROJECT_VERSION is what Sparkle compares, as <sparkle:version>. A build number - # that did not increase means no installed app is offered the release, and until now only - # a sentence in the release runbook stopped that. Compared against the newest PUBLISHED - # release rather than the newest tag, because re-cutting a release that failed leaves a - # tag at the same build and that is a legitimate state to be in. - BUILD=$(sed -n 's/^CURRENT_PROJECT_VERSION[[:space:]]*=[[:space:]]*//p' Configs/Version.xcconfig | tr -d ' ') - case "$BUILD" in - ''|*[!0-9]*) - echo "::error::CURRENT_PROJECT_VERSION '$BUILD' is not a plain integer" - exit 1 - ;; - esac - - # No `|| true`. A gh outage used to turn this gate into "no earlier published release" - # and exit 0, which is the same output a genuine first release produces. - PREVIOUS_TAG=$(gh release list --limit 100 --json tagName,isPrerelease,publishedAt \ - -q '[.[] | select(.tagName|startswith("v")) | select(.isPrerelease==false)] - | sort_by(.publishedAt) | last | .tagName') - if [ "$PREVIOUS_TAG" = "null" ]; then - PREVIOUS_TAG="" - fi - if [ -z "$PREVIOUS_TAG" ] || [ "$PREVIOUS_TAG" = "v$TAG_VERSION" ]; then - echo "no earlier published release to compare the build number against" - exit 0 - fi - - PREVIOUS_BUILD=$(git show "${PREVIOUS_TAG}:Configs/Version.xcconfig" \ - | sed -n 's/^CURRENT_PROJECT_VERSION[[:space:]]*=[[:space:]]*//p' | tr -d ' ') - if [ -n "$PREVIOUS_BUILD" ] && [ "$BUILD" -le "$PREVIOUS_BUILD" ]; then - echo "::error::CURRENT_PROJECT_VERSION $BUILD is not above $PREVIOUS_BUILD from $PREVIOUS_TAG; no install would be offered this update" - exit 1 - fi - echo "build $BUILD is above $PREVIOUS_BUILD from $PREVIOUS_TAG" - - - name: Verify and organize artifacts for release - run: | - VERSION=${GITHUB_REF#refs/tags/v} - - if [ -z "$VERSION" ]; then - echo "❌ ERROR: Failed to extract version from ref: $GITHUB_REF" - exit 1 - fi - - echo "Preparing artifacts for version: $VERSION" - echo "Contents of artifacts directory:" - ls -la artifacts/ - - # The DMGs are already versioned by package-artifacts.sh; the update archives are not. - # Both renames used to be guarded by `if [ -f ... ]` with no else, so a missing or - # differently named archive printed "organized successfully" and the release published - # whatever the upload glob happened to match. - for ARCH in arm64 x86_64; do - if [ ! -f "artifacts/TablePro-${ARCH}.tar.xz" ]; then - echo "::error::artifacts/TablePro-${ARCH}.tar.xz is missing; the ${ARCH} build did not produce an update archive" - ls -la artifacts/ >&2 - exit 1 - fi - mv "artifacts/TablePro-${ARCH}.tar.xz" "artifacts/TablePro-${VERSION}-${ARCH}.tar.xz" - - if [ ! -f "artifacts/TablePro-${VERSION}-${ARCH}.dmg" ]; then - echo "::error::artifacts/TablePro-${VERSION}-${ARCH}.dmg is missing" - ls -la artifacts/ >&2 - exit 1 - fi - done - - echo "✅ Artifacts organized successfully" - echo "Final artifacts:" - ls -lh artifacts/ - - # No `if: env.SPARKLE_PRIVATE_KEY != ''` on these steps. This job only runs for a v* tag, - # so the key is always meant to be present, and gating on it meant a missing or rotated - # secret published a release that no existing install could ever see. A signing key that - # is not there is now a red job, not a silent skip. - # The tag's own appcast.xml is the feed as of that commit, which is missing every version - # published since. The merge base has to be the feed that is actually live. - - name: Fetch the published feed to merge into - run: | - git fetch origin main - mkdir -p appcast - git show origin/main:appcast.xml > appcast/base.xml - - # A flag left true from the previous release would mark an ordinary build critical and - # interrupt every user, so it has to have been touched in this release's own diff. - - name: Read the release flags - id: flags - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - CRITICAL=$(python3 -c " - import json - print('1' if json.load(open('.github/release-flags.json')).get('criticalUpdate') is True else '0') - ") - echo "critical=$CRITICAL" >> "$GITHUB_OUTPUT" - if [ "$CRITICAL" != "1" ]; then - echo "Ordinary release, staged over 36 hours" - exit 0 - fi - - RELEASES=$(gh release list --limit 100 --json tagName,isPrerelease,publishedAt \ - -q '[.[] | select(.tagName|startswith("v")) | select(.isPrerelease==false)] - | sort_by(.publishedAt) | .[].tagName') - # A re-run after this tag's release went out finds it among the published ones. - PREVIOUS_TAG=$(printf '%s\n' "$RELEASES" | grep -vxF "${GITHUB_REF#refs/tags/}" | tail -1) - if [ -n "$PREVIOUS_TAG" ] && [ "$PREVIOUS_TAG" != "null" ] \ - && git diff --quiet "$PREVIOUS_TAG" HEAD -- .github/release-flags.json; then - echo "::error::criticalUpdate is true but .github/release-flags.json has not changed since $PREVIOUS_TAG, which is what a flag left over from the previous release looks like" - exit 1 - fi - echo "::warning::This release is marked critical and will interrupt every user" - - - name: Sign update archives with Sparkle - env: - SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} - BASE_APPCAST: appcast/base.xml - CRITICAL_UPDATE: ${{ steps.flags.outputs.critical }} - run: scripts/ci/sign-and-appcast.sh "${GITHUB_REF#refs/tags/v}" - - - name: Upload appcast artifact - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 - with: - name: appcast-${{ github.sha }} - path: appcast/appcast.xml - retention-days: 90 - - # Fails before anything is published when the secret does not match a deploy key. - - name: Check the release deploy key - env: - APPCAST_DEPLOY_KEY: ${{ secrets.APPCAST_DEPLOY_KEY }} - run: | - eval "$(ssh-agent -s)" > /dev/null - trap 'ssh-agent -k > /dev/null' EXIT - ssh-add - <<< "$APPCAST_DEPLOY_KEY" - git ls-remote "git@github.com:${GITHUB_REPOSITORY}.git" HEAD > /dev/null - - # A published release is immutable, so every asset goes on the draft first. - - name: Create the draft release - id: draft - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - PRERELEASE: ${{ contains(github.ref, '-beta') || contains(github.ref, '-alpha') || contains(github.ref, '-rc') }} - run: | - TAG="${GITHUB_REF#refs/tags/}" - scripts/ci/draft-release.sh --prerelease "$PRERELEASE" "$TAG" "$TAG" release_notes.md \ - artifacts/*.dmg artifacts/*.tar.xz - - - name: Publish the release - if: steps.draft.outputs.id != '' - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - RELEASE_ID: ${{ steps.draft.outputs.id }} - run: | - gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" -F draft=false \ - --jq '"Published \(.tag_name): \(.html_url)"' - - # Publishing the feed comes after publishing the artifacts it points at, so there is no - # window where Sparkle advertises a download URL that still 404s. A draft's URLs 404 too. - # - # This step used to be continue-on-error, which made a failed push indistinguishable from - # a release that went out clean: every user's updater would keep reporting "up to date" - # for a version that shipped. A rejected push now rebases and retries, and a push that - # still will not land fails the job. - # Pushed with the release deploy key, the ruleset's only bypass actor: a fresh commit on main - # has no checks of its own yet. - - name: Commit appcast.xml to repo - env: - APPCAST_DEPLOY_KEY: ${{ secrets.APPCAST_DEPLOY_KEY }} - run: | - VERSION="${GITHUB_REF#refs/tags/v}" - - if [ ! -f appcast/items/arm64.xml ] || [ ! -f appcast/items/x86_64.xml ]; then - echo "❌ ERROR: the signed per-architecture items are missing, the Sparkle step produced no feed" >&2 - exit 1 - fi - - # The merge below runs after the tree has been reset to main, so the scripts are taken - # from the tag first. Otherwise a release would publish its feed with whatever - # scripts/ci main happened to hold, rather than the code the tag was tested with. - TOOLS=$(mktemp -d) - cp -R scripts/ci "$TOOLS/" - - eval "$(ssh-agent -s)" > /dev/null - trap 'ssh-agent -k > /dev/null' EXIT - ssh-add - <<< "$APPCAST_DEPLOY_KEY" - - git config user.name "github-actions[bot]" - git config user.email "github-actions[bot]@users.noreply.github.com" - git checkout -B main - - # The whole attempt is inside the retry, because the merge is what has to be redone when - # main moves. It used to run once against a snapshot taken before the build and then be - # copied over main, which reverted anything pushed in between: a pull-release.py - # withdrawal landing inside that window was undone and the withdrawn build went back on - # offer. The rewind guard could not see it, because a restoration adds versions rather - # than dropping them. Rebasing was no answer either, since the merge rewrites the file - # wholesale and every retry conflicted. - for attempt in 1 2 3; do - git fetch origin main - git reset --hard origin/main - - python3 "$TOOLS/ci/merge-appcast.py" \ - --base appcast.xml \ - --version "$VERSION" \ - --arm64 appcast/items/arm64.xml \ - --x86-64 appcast/items/x86_64.xml \ - --download-prefix "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/download/v${VERSION}/" \ - --out appcast.xml - - # The feed is pruned to the newest releases, so an old version leaving it is the policy - # working. What must never leave is the newest item, the only one any install can be - # offered, or the release being published. - # An empty newest_before is a feed with no items yet, which is a legitimate first - # release rather than something to refuse. - newest_before=$(git show origin/main:appcast.xml \ - | grep -o '<sparkle:shortVersionString>[^<]*' | sed 's/.*>//' | head -1) - for required in "$newest_before" "$VERSION"; do - if [ -z "$required" ]; then - continue - fi - if ! grep -q "<sparkle:shortVersionString>${required}</sparkle:shortVersionString>" appcast.xml; then - echo "❌ ERROR: the generated feed no longer advertises $required" >&2 - exit 1 - fi - done - first=$(grep -o '<sparkle:shortVersionString>[^<]*' appcast.xml | sed 's/.*>//' | head -1) - if [ "$first" != "$VERSION" ]; then - echo "❌ ERROR: the feed leads with $first, not the release being published ($VERSION)" >&2 - exit 1 - fi - - git add appcast.xml - if git diff --cached --quiet; then - echo "appcast.xml already advertises v${VERSION}" - exit 0 - fi - - git commit -m "Update appcast.xml for v${VERSION}" - if git push "git@github.com:${GITHUB_REPOSITORY}.git" HEAD:main; then - exit 0 - fi - echo "Push rejected, re-merging onto the current origin/main (attempt ${attempt})" - done - - echo "❌ ERROR: could not push appcast.xml after 3 attempts" >&2 - exit 1 - - - name: Notify Telegram - if: success() && env.TELEGRAM_BOT_TOKEN != '' - env: - TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} - TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }} - TELEGRAM_TOPIC_ID: ${{ secrets.TELEGRAM_TOPIC_ID }} - run: scripts/ci/notify-telegram.sh "${GITHUB_REF#refs/tags/v}" diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml new file mode 100644 index 0000000000..f247191350 --- /dev/null +++ b/.github/workflows/pages.yml @@ -0,0 +1,40 @@ +name: Pages + +# Deploys the feeds branch to updates.tablepro.app. A branch-source Pages site would not do: a push +# made with GITHUB_TOKEN starts no Pages build. Dispatch it at a v* tag, which github-pages requires. + +on: + workflow_call: + workflow_dispatch: + +permissions: + contents: read + +jobs: + deploy: + name: Deploy + runs-on: ubuntu-latest + timeout-minutes: 10 + environment: + name: github-pages + url: ${{ steps.deploy.outputs.page_url }} + # Never cancelled. A newer pending deploy replaces an older pending one, and both deploy the feeds head. + concurrency: + group: pages + cancel-in-progress: false + permissions: + contents: read + pages: write + id-token: write + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: feeds + persist-credentials: false + + - uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 + with: + path: . + + - id: deploy + uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000000..5a07bdddf8 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,367 @@ +name: Release + +# A v* tag builds, signs and notarizes the app and every registry plugin, publishes the GitHub +# release, then adds it to the feeds branch and deploys that to updates.tablepro.app. + +on: + push: + tags: ['v*'] + +# Never cancelled: the build jobs hold notary submissions, and every step after them is idempotent, +# so a re-run of the same tag finishes what the first run left. +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +permissions: + contents: read + +jobs: + verify: + name: Verify + runs-on: ubuntu-latest + timeout-minutes: 240 + permissions: + contents: read + actions: read + pull-requests: read + outputs: + tag: ${{ steps.parse.outputs.tag }} + version: ${{ steps.parse.outputs.version }} + build: ${{ steps.parse.outputs.build }} + channel: ${{ steps.parse.outputs.channel }} + critical: ${{ steps.notes.outputs.critical }} + env: + TAG: ${{ github.ref_name }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + filter: blob:none + + - name: Set up Python 3.9 + id: python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.9" + update-environment: false + + - name: Install the release tooling + env: + SETUP_PYTHON: ${{ steps.python.outputs.python-path }} + run: | + set -euo pipefail + "$SETUP_PYTHON" -m venv "$RUNNER_TEMP/release-venv" + "$RUNNER_TEMP/release-venv/bin/pip" install --quiet --disable-pip-version-check --require-hashes \ + -r scripts/release/requirements.txt + + - name: Check the tag can build + run: '"$RUNNER_TEMP/release-venv/bin/python" scripts/release/version.py check "$TAG"' + + - name: Read the version + id: parse + run: | + set -euo pipefail + parsed="$("$RUNNER_TEMP/release-venv/bin/python" scripts/release/version.py parse "$TAG")" + printf 'tag=%s\n%s\n' "$TAG" "$parsed" >> "$GITHUB_OUTPUT" + + - name: Write the release notes + id: notes + env: + VERSION: ${{ steps.parse.outputs.version }} + run: | + set -euo pipefail + python="$RUNNER_TEMP/release-venv/bin/python" + mkdir -p build/notes + for kind in body feed lead; do + "$python" scripts/release/notes.py "$kind" "$VERSION" > "build/notes/$kind.md" + done + critical="$("$python" scripts/release/notes.py critical "$VERSION")" + echo "critical=$critical" >> "$GITHUB_OUTPUT" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: notes + path: build/notes + if-no-files-found: error + retention-days: 30 + + app: + name: App + needs: verify + runs-on: xcode-27 + environment: signing + timeout-minutes: 120 + permissions: + contents: read + env: + VERSION: ${{ needs.verify.outputs.version }} + BUILD: ${{ needs.verify.outputs.build }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-xcode + with: + require-release: ${{ needs.verify.outputs.channel == 'stable' }} + + - name: Check the Production CloudKit schema + env: + CLOUDKIT_MANAGEMENT_TOKEN: ${{ secrets.CLOUDKIT_MANAGEMENT_TOKEN }} + run: | + set -euo pipefail + xcrun cktool save-token --type management --method file --force "$CLOUDKIT_MANAGEMENT_TOKEN" + scripts/export-cloudkit-schema.sh --check + + - uses: ./.github/actions/setup-xcodegen + + - name: Download static libraries + env: + GH_TOKEN: ${{ github.token }} + run: scripts/download-libs.sh + + - name: Write Secrets.xcconfig + env: + ANALYTICS_HMAC_SECRET: ${{ secrets.ANALYTICS_HMAC_SECRET }} + run: printf 'ANALYTICS_HMAC_SECRET = %s\n' "$ANALYTICS_HMAC_SECRET" > Configs/Secrets.xcconfig + + - name: Generate the Xcode project + run: scripts/generate-project.sh macos + + # macos-tests.yml saves this cache; a save in tag scope would be unreadable from other refs. + - name: Restore Swift package checkouts + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.spm-cache + key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} + restore-keys: spm-${{ runner.os }}- + + - uses: ./.github/actions/apple-signing + with: + certificate-p12: ${{ secrets.CERTIFICATES_P12 }} + certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} + provisioning-profile: ${{ secrets.PROVISIONING_PROFILE }} + asc-key-id: ${{ secrets.ASC_KEY_ID }} + asc-issuer-id: ${{ secrets.ASC_ISSUER_ID }} + asc-key-p8: ${{ secrets.ASC_KEY_P8 }} + + - name: Install create-dmg + run: brew list create-dmg > /dev/null 2>&1 || brew install create-dmg + + - name: Build the app + run: scripts/release/build.sh app + + - name: Notarize the app + run: scripts/release/notarize.sh build/release/TablePro.app + + - name: Package the app + run: scripts/release/package.sh app "$VERSION" + + - name: Notarize the disk image + run: scripts/release/notarize.sh "dist/TablePro-$VERSION.dmg" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: app + path: dist/ + if-no-files-found: error + retention-days: 30 + + plugins: + name: Plugins + needs: verify + runs-on: xcode-27 + environment: signing + timeout-minutes: 90 + permissions: + contents: read + env: + VERSION: ${{ needs.verify.outputs.version }} + BUILD: ${{ needs.verify.outputs.build }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-xcode + with: + require-release: ${{ needs.verify.outputs.channel == 'stable' }} + + - uses: ./.github/actions/setup-xcodegen + + - name: Download static libraries + env: + GH_TOKEN: ${{ github.token }} + run: scripts/download-libs.sh + + - name: Generate the Xcode project + run: scripts/generate-project.sh macos + + - name: Restore Swift package checkouts + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.spm-cache + key: spm-${{ runner.os }}-${{ hashFiles('TablePro.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved') }} + restore-keys: spm-${{ runner.os }}- + + - name: Set up Go for the SAP HANA helper + uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0 + with: + go-version-file: Native/HanaBridge/go.mod + cache: false + + # The Dameng and HANA targets copy these artifacts in and never build them. + - name: Build the native bridges + run: | + scripts/build-dameng.sh both + scripts/build-hana.sh both + + - uses: ./.github/actions/apple-signing + with: + certificate-p12: ${{ secrets.CERTIFICATES_P12 }} + certificate-password: ${{ secrets.CERTIFICATES_PASSWORD }} + asc-key-id: ${{ secrets.ASC_KEY_ID }} + asc-issuer-id: ${{ secrets.ASC_ISSUER_ID }} + asc-key-p8: ${{ secrets.ASC_KEY_P8 }} + + - name: Build the registry plugins + run: scripts/release/build.sh plugins + + - name: Notarize the plugins + run: scripts/release/notarize.sh build/release/plugins/*.tableplugin + + - name: Package the plugins + run: scripts/release/package.sh plugins "$VERSION" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: plugins + path: dist/ + if-no-files-found: error + retention-days: 30 + + github-release: + name: GitHub release + needs: [verify, app, plugins] + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: write + id-token: write + attestations: write + env: + TAG: ${{ needs.verify.outputs.tag }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: app + path: dist + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: plugins + path: dist + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + # A published release is immutable, so every asset goes on the draft and is checked there first. + - name: Draft the release + id: draft + run: python3 scripts/release/publish.py draft "$TAG" --notes notes/body.md dist/* >> "$GITHUB_OUTPUT" + + - name: Attest the assets + if: steps.draft.outputs.state == 'draft' + uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2 + with: + subject-path: dist/* + + - name: Publish the release + run: python3 scripts/release/publish.py release "$TAG" + + feed: + name: Feed + needs: [verify, github-release] + runs-on: macos-latest + environment: release + timeout-minutes: 20 + permissions: + contents: write + env: + TAG: ${{ needs.verify.outputs.tag }} + BUILD: ${{ needs.verify.outputs.build }} + CHANNEL: ${{ needs.verify.outputs.channel }} + CRITICAL: ${{ needs.verify.outputs.critical }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-sparkle + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + - name: Add the release to the feed and the plugin manifest + env: + GH_TOKEN: ${{ github.token }} + SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} + run: | + set -euo pipefail + flags=() + [ "$CRITICAL" != true ] || flags+=(--critical) + python3 scripts/release/feed.py add "$TAG" --build "$BUILD" --channel "$CHANNEL" \ + --notes notes/feed.md ${flags[@]+"${flags[@]}"} + + pages: + name: Pages + needs: feed + permissions: + contents: read + pages: write + id-token: write + uses: ./.github/workflows/pages.yml + + notify: + name: Notify + needs: [verify, pages] + if: needs.verify.outputs.channel == 'stable' + runs-on: ubuntu-latest + environment: release + continue-on-error: true + timeout-minutes: 5 + permissions: + contents: read + env: + TAG: ${{ needs.verify.outputs.tag }} + VERSION: ${{ needs.verify.outputs.version }} + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: notes + path: notes + + # Plain text: the lead goes out as written, with no parse_mode to escape for. + - name: Post to Telegram + env: + TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} + TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }} + TELEGRAM_TOPIC_ID: ${{ secrets.TELEGRAM_TOPIC_ID }} + run: | + set -euo pipefail + { + printf 'TablePro %s\n' "$VERSION" + if [ -s notes/lead.md ]; then + printf '\n' + cat notes/lead.md + fi + printf '\n%s/%s/releases/tag/%s\n' "$GITHUB_SERVER_URL" "$GITHUB_REPOSITORY" "$TAG" + } > msg.txt + topic=() + [ -z "$TELEGRAM_TOPIC_ID" ] || topic=(--data-urlencode "message_thread_id=$TELEGRAM_TOPIC_ID") + curl -fsS --retry 3 -o /dev/null --data-urlencode "chat_id=$TELEGRAM_CHAT_ID" \ + --data-urlencode text@msg.txt ${topic[@]+"${topic[@]}"} \ + "https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage" diff --git a/.github/workflows/repo-hygiene.yml b/.github/workflows/repo-hygiene.yml index 9cab3d2a8f..4b52196fed 100644 --- a/.github/workflows/repo-hygiene.yml +++ b/.github/workflows/repo-hygiene.yml @@ -42,16 +42,12 @@ jobs: mapfile -t SCRIPTS < <(find scripts .claude/skills -name '*.sh' | sort) shellcheck --severity=warning "${SCRIPTS[@]}" - # update-registry's tests are plain functions, which unittest discovery would skip. - name: Test the CI and release tooling run: | set -euo pipefail python3 -m unittest discover -s scripts/ci -p 'test_*.py' python3 -m unittest discover -s scripts/lib -p 'test_*.py' python3 -m unittest discover -s scripts -p 'test_localization.py' - python3 .github/scripts/test_update_registry.py - bash scripts/ci/test_create_dmg_args.sh - bash scripts/ci/test_draft_release.sh bash scripts/ci/test_libs_release.sh # 3.9 is the macOS /usr/bin/python3 that runs notes.py in the build. The other steps keep the @@ -137,8 +133,7 @@ jobs: go-version-file: Native/HanaBridge/go.mod cache: false - # The helper is pure Go, so Linux analyses it as well as the Mac that builds it. govulncheck - # runs in build-plugin.yml, before a HANA plugin ships. + # The helper is pure Go, so Linux analyses it as well as the Mac that builds it. - name: Check the SAP HANA helper if: steps.native.outputs.hana == 'true' working-directory: Native/HanaBridge diff --git a/.github/workflows/withdraw.yml b/.github/workflows/withdraw.yml new file mode 100644 index 0000000000..a728e5100b --- /dev/null +++ b/.github/workflows/withdraw.yml @@ -0,0 +1,49 @@ +name: Withdraw + +# Removes a version from the feed and the plugin manifest, marks the highest stable left as Latest, +# and redeploys. Dispatch it at the newest v* tag, which the release environment requires. +# The GitHub release stays. + +on: + workflow_dispatch: + inputs: + version: + description: The version to withdraw, such as 0.82.1 or 0.83.0-beta.2. + required: true + type: string + +permissions: + contents: read + +jobs: + withdraw: + name: Withdraw + runs-on: macos-latest + environment: release + timeout-minutes: 20 + permissions: + contents: write + env: + VERSION: ${{ inputs.version }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - uses: ./.github/actions/setup-sparkle + + - name: Remove the version from the feed and the plugin manifest + env: + SPARKLE_PRIVATE_KEY: ${{ secrets.SPARKLE_PRIVATE_KEY }} + run: python3 scripts/release/feed.py withdraw "$VERSION" + + - name: Mark the highest stable as Latest + run: python3 scripts/release/publish.py latest + + pages: + name: Pages + needs: withdraw + permissions: + contents: read + pages: write + id-token: write + uses: ./.github/workflows/pages.yml diff --git a/scripts/build-plugin.sh b/scripts/build-plugin.sh deleted file mode 100755 index 9ff0d2d90f..0000000000 --- a/scripts/build-plugin.sh +++ /dev/null @@ -1,278 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/notarize.sh" - -# Build script for creating standalone plugin bundles -# Usage: ./scripts/build-plugin.sh <PluginTarget> [arm64|x86_64|both] [version] -# Example: ./scripts/build-plugin.sh OracleDriverPlugin arm64 1.0.0 -# -# Version (3rd arg or PLUGIN_VERSION env) is injected as MARKETING_VERSION so -# CFBundleShortVersionString in the built bundle matches the registry version. -# Required for bundled drivers that also ship via registry. Without it, the -# user copy ties with built-in v1.0 and PluginManager prunes it on load. - -PLUGIN_TARGET="${1:?Usage: $0 <PluginTarget> [arm64|x86_64|both] [version]}" -ARCH="${2:-both}" -PLUGIN_VERSION="${3:-${PLUGIN_VERSION:-}}" -PROJECT="TablePro.xcodeproj" -CONFIG="Release" -BUILD_DIR="build/Plugins" -SIGN_IDENTITY="${SIGN_IDENTITY:-}" -TEAM_ID="${TEAM_ID:-}" -NOTARIZE="${NOTARIZE:-false}" - -if [ -z "$TEAM_ID" ]; then - echo "ERROR: TEAM_ID is not set. Pass via env or set in your shell profile." >&2 - echo " Example: TEAM_ID=ABCDEFGHIJ ./scripts/build-plugin.sh $PLUGIN_TARGET" >&2 - exit 1 -fi - -if [ -z "$SIGN_IDENTITY" ]; then - # Try the canonical "Developer ID Application: <Name> (<TEAMID>)" pattern. - # If your keychain stores the identity differently, set SIGN_IDENTITY explicitly. - SIGN_IDENTITY=$(security find-identity -v -p codesigning 2>/dev/null \ - | awk -F'"' -v team="$TEAM_ID" '$2 ~ /Developer ID Application/ && $2 ~ team {print $2; exit}') - if [ -z "$SIGN_IDENTITY" ]; then - echo "ERROR: No Developer ID Application identity found in keychain for team $TEAM_ID." >&2 - echo " Either install the cert or set SIGN_IDENTITY explicitly." >&2 - exit 1 - fi -fi - -if [ -n "$PLUGIN_VERSION" ]; then - echo "Building plugin: $PLUGIN_TARGET v$PLUGIN_VERSION for $ARCH" -else - echo "Building plugin: $PLUGIN_TARGET for $ARCH (no version override)" -fi - -is_helper_executable() { - local candidate="$1" - local main_binary="$2" - [ -f "$candidate" ] || return 1 - [ ! -L "$candidate" ] || return 1 - [ "$candidate" != "$main_binary" ] || return 1 - case "$(file -b "$candidate")" in - *Mach-O*) return 0 ;; - *) return 1 ;; - esac -} - -sign_helper_executables() { - local bundle="$1" - local main_binary="$2" - local helper - for helper in "$bundle/Contents/MacOS"/* "$bundle/Contents/Helpers"/*; do - is_helper_executable "$helper" "$main_binary" || continue - echo " Signing helper: $(basename "$helper")" >&2 - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$helper" - done -} - -# codesign --verify --deep --strict also accepts an ad-hoc signed helper, which notarization -# then rejects, so each helper is checked for what notarization needs. -verify_helper_signatures() { - local bundle="$1" - local main_binary="$2" - local helper - local details - for helper in "$bundle/Contents/MacOS"/* "$bundle/Contents/Helpers"/*; do - is_helper_executable "$helper" "$main_binary" || continue - if ! details="$(codesign -dvvv "$helper" 2>&1)"; then - echo "FATAL: $helper is not signed: $details" >&2 - exit 1 - fi - if ! grep -qE '^CodeDirectory .*flags=0x[0-9a-f]+\([^)]*runtime' <<< "$details"; then - echo "FATAL: $helper is not signed with the hardened runtime" >&2 - exit 1 - fi - if ! grep -q '^Authority=Developer ID Application: ' <<< "$details"; then - echo "FATAL: $helper is not signed with a Developer ID Application identity" >&2 - exit 1 - fi - if ! grep -q '^Timestamp=' <<< "$details"; then - echo "FATAL: $helper has no secure timestamp" >&2 - exit 1 - fi - echo " Verified helper: $(basename "$helper") (hardened runtime, Developer ID, timestamp)" >&2 - done -} - -build_plugin() { - local arch=$1 - local build_dir="$BUILD_DIR/$arch" - - echo "Building $PLUGIN_TARGET ($arch)..." >&2 - - # Use -scheme (not -target) with -derivedDataPath to ensure proper - # transitive SPM dependency resolution in explicit module builds. - # project.yml declares one shared scheme per plugin target, named after the target. - DERIVED_DATA_DIR="build/DerivedData" - - local marketing_version_arg="" - if [ -n "$PLUGIN_VERSION" ]; then - marketing_version_arg="MARKETING_VERSION=$PLUGIN_VERSION" - fi - - if ! xcodebuild \ - -project "$PROJECT" \ - -scheme "$PLUGIN_TARGET" \ - -configuration "$CONFIG" \ - -arch "$arch" \ - ONLY_ACTIVE_ARCH=YES \ - CONFIGURATION_BUILD_DIR="$(pwd)/$build_dir" \ - CODE_SIGN_IDENTITY="$SIGN_IDENTITY" \ - CODE_SIGN_STYLE=Manual \ - DEVELOPMENT_TEAM="$TEAM_ID" \ - ${marketing_version_arg:+"$marketing_version_arg"} \ - -skipPackagePluginValidation \ - -derivedDataPath "$DERIVED_DATA_DIR" \ - build > "build-plugin-${arch}.log" 2>&1; then - echo "FATAL: xcodebuild failed for $PLUGIN_TARGET ($arch)" >&2 - echo "=== Swift errors (grep error:) ===" >&2 - grep -nE "error:|cannot|undefined symbol" "build-plugin-${arch}.log" | head -40 >&2 || true - echo "=== Last 80 lines of build log ===" >&2 - tail -80 "build-plugin-${arch}.log" >&2 - exit 1 - fi - - # Find the built plugin bundle by target name - local plugin_bundle="$build_dir/${PLUGIN_TARGET}.tableplugin" - - if [ ! -d "$plugin_bundle" ]; then - echo "FATAL: Plugin bundle not found at $plugin_bundle" >&2 - exit 1 - fi - - echo "Built: $plugin_bundle" >&2 - - if [ -n "$PLUGIN_VERSION" ]; then - actual_version=$(plutil -extract CFBundleShortVersionString raw -o - "$plugin_bundle/Contents/Info.plist" 2>/dev/null || echo "") - if [ "$actual_version" != "$PLUGIN_VERSION" ]; then - echo "FATAL: Built bundle CFBundleShortVersionString='$actual_version' but expected '$PLUGIN_VERSION'" >&2 - echo " MARKETING_VERSION injection failed. Users would see 'Update to v$PLUGIN_VERSION' loops." >&2 - exit 1 - fi - echo "Bundle version verified: CFBundleShortVersionString=$actual_version" >&2 - fi - - # Strip the plugin binary to reduce size - local plugin_name - plugin_name=$(basename "$plugin_bundle" .tableplugin) - local plugin_binary="$plugin_bundle/Contents/MacOS/$plugin_name" - if [ -f "$plugin_binary" ]; then - local before after - before=$(ls -lh "$plugin_binary" | awk '{print $5}') - strip -x "$plugin_binary" - after=$(ls -lh "$plugin_binary" | awk '{print $5}') - echo "Stripped binary: $before -> $after" >&2 - fi - - # Code sign inside-out: nested frameworks/dylibs and helper executables first, then binary, then bundle - echo "Code signing with: $SIGN_IDENTITY" >&2 - - # Sign nested frameworks - if [ -d "$plugin_bundle/Contents/Frameworks" ]; then - find "$plugin_bundle/Contents/Frameworks" -name "*.framework" -o -name "*.dylib" | sort | while read -r nested; do - echo " Signing nested: $(basename "$nested")" >&2 - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$nested" - done - fi - - sign_helper_executables "$plugin_bundle" "$plugin_binary" - - # Sign the main binary - if [ -f "$plugin_binary" ]; then - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_binary" - fi - - # Sign the outer bundle - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_bundle" - - if ! codesign --verify --deep --strict "$plugin_bundle" 2>&1; then - echo "FATAL: Code signature verification failed" >&2 - exit 1 - fi - verify_helper_signatures "$plugin_bundle" "$plugin_binary" - echo "Code signature verified" >&2 - - # Only the path goes to stdout (return value) - echo "$plugin_bundle" -} - -create_zip() { - local plugin_path=$1 - local arch=$2 - local plugin_name - plugin_name=$(basename "$plugin_path" .tableplugin) - local zip_name="${plugin_name}-${arch}.zip" - local zip_path="$BUILD_DIR/$zip_name" - - echo "Creating ZIP: $zip_name" - ditto -c -k --keepParent "$plugin_path" "$zip_path" - - # Print SHA-256 for registry manifest - local sha256 - sha256=$(shasum -a 256 "$zip_path" | awk '{print $1}') - # Write SHA-256 to sidecar file for CI automation - echo "$sha256" > "${zip_path}.sha256" - - echo "ZIP created: $zip_path" - echo " SHA-256: $sha256" - echo " Size: $(ls -lh "$zip_path" | awk '{print $5}')" -} - -# Notarization has to happen BEFORE create_zip, and it has to staple. -# -# Gatekeeper refuses to load an unnotarized bundle into TablePro, and -# com.apple.security.cs.disable-library-validation does not exempt it: a quarantined -# unnotarized plugin fails with "library load disallowed by system policy", and the user -# gets a "could not verify it is free of malware" panel instead of a driver. Every plugin -# published before this ran was unnotarized, because the workflow gated the step on an -# environment variable nothing ever set. -# -# Stapling rewrites the bundle, so the distribution zip and its SHA-256 must both be produced -# after it: the registry manifest pins that checksum and PluginInstaller rejects a mismatch. -notarize_plugin() { - if [ "$NOTARIZE" != "true" ]; then - echo "Skipping notarization (set NOTARIZE=true to enable)" - return - fi - # "open", not "exec": a plugin bundle is opened by the app, not launched. - notarize_and_staple "$1" open -} - -# TablePro.xcodeproj is generated and not in git, so a fresh checkout has none. -# Generation also declares the per-plugin scheme this script builds with. -scripts/generate-project.sh - -# Clean DerivedData for fresh builds; preserve BUILD_DIR across arch invocations -rm -rf build/DerivedData -mkdir -p "$BUILD_DIR" - -case "$ARCH" in - arm64|x86_64) - plugin_path=$(build_plugin "$ARCH") - notarize_plugin "$plugin_path" - create_zip "$plugin_path" "$ARCH" - ;; - both) - arm64_path=$(build_plugin "arm64") - x86_path=$(build_plugin "x86_64") - - notarize_plugin "$arm64_path" - notarize_plugin "$x86_path" - - create_zip "$arm64_path" "arm64" - create_zip "$x86_path" "x86_64" - ;; - *) - echo "Invalid architecture: $ARCH (use arm64, x86_64, or both)" - exit 1 - ;; -esac - -echo "" -echo "Plugin build complete!" -echo "Output: $BUILD_DIR/" -ls -lh "$BUILD_DIR/"*.zip 2>/dev/null || echo "No ZIP files found" diff --git a/scripts/build-release.sh b/scripts/build-release.sh deleted file mode 100755 index 61c2b32294..0000000000 --- a/scripts/build-release.sh +++ /dev/null @@ -1,505 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# shellcheck source=lib/macos.sh -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/macos.sh" -source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/notarize.sh" - -# Build script for creating architecture-specific releases -# Usage: ./build-release.sh [arm64|x86_64|both] - -ARCH="${1:-both}" -PROJECT="TablePro.xcodeproj" -SCHEME="TablePro" -CONFIG="Release" -BUILD_DIR="build/Release" -SIGN_IDENTITY="${SIGN_IDENTITY:-Developer ID Application: Dat Ngo Quoc (D7HJ5TFYCU)}" -TEAM_ID="D7HJ5TFYCU" -NOTARIZE="${NOTARIZE:-false}" - -echo "🏗️ Building TablePro for: $ARCH" - -# Bundle non-system dynamic libraries into the app bundle -# so the app runs without Homebrew on end-user machines. -bundle_dylibs() { - local app_path=$1 - local binary="$app_path/Contents/MacOS/TablePro" - local frameworks_dir="$app_path/Contents/Frameworks" - - echo "📦 Bundling dynamic libraries into app bundle..." - mkdir -p "$frameworks_dir" - - # Iteratively discover and copy all non-system dylibs. - # Each pass scans the main binary + already-copied dylibs; - # repeat until no new dylibs are found (handles transitive deps). - local changed=1 - while [ "$changed" -eq 1 ]; do - changed=0 - for target in "$binary" "$frameworks_dir"/*.dylib; do - [ -f "$target" ] || continue - - while IFS= read -r dep; do - # Keep only non-system, non-rewritten absolute paths - case "$dep" in - /usr/lib/*|/System/*|@*|"") continue ;; - esac - - local name - name=$(basename "$dep") - - # Already bundled - [ -f "$frameworks_dir/$name" ] && continue - - if [ -f "$dep" ]; then - echo " Copying $name" - cp "$dep" "$frameworks_dir/$name" - chmod 644 "$frameworks_dir/$name" - changed=1 - else - echo " ⚠️ WARNING: $dep not found on disk, skipping" - fi - done < <(otool -L "$target" 2>/dev/null | awk 'NR>1 {print $1}') - done - done - - # Count bundled dylibs - local count - count=$(find "$frameworks_dir" -name '*.dylib' 2>/dev/null | wc -l | tr -d ' ') - - if [ "$count" -eq 0 ]; then - echo " No non-system dylibs to bundle" - return 0 - fi - - # Rewrite each dylib's own install name - for fw in "$frameworks_dir"/*.dylib; do - [ -f "$fw" ] || continue - local name - name=$(basename "$fw") - install_name_tool -id "@executable_path/../Frameworks/$name" "$fw" - done - - # Rewrite all references in the main binary and every bundled dylib - for target in "$binary" "$frameworks_dir"/*.dylib; do - [ -f "$target" ] || continue - - while IFS= read -r dep; do - case "$dep" in - /usr/lib/*|/System/*|@*|"") continue ;; - esac - - local name - name=$(basename "$dep") - - if [ -f "$frameworks_dir/$name" ]; then - install_name_tool -change "$dep" "@executable_path/../Frameworks/$name" "$target" - fi - done < <(otool -L "$target" 2>/dev/null | awk 'NR>1 {print $1}') - done - - # Verify bundled dylibs are compatible with the deployment target. - # Homebrew builds libraries targeting the *host* macOS version, so if - # the build machine runs macOS 26.0, libpq will require 26.0 symbols - # (e.g. strchrnul) that don't exist on earlier OS versions → launch crash. - echo " Verifying deployment target compatibility..." - # Read the deployment target off the built app rather than the project file: the - # bundle is the only source that cannot disagree with what actually shipped. - local deploy_target - deploy_target=$(plutil -extract LSMinimumSystemVersion raw -o - "$app_path/Contents/Info.plist" 2>/dev/null || true) - if [ -n "$deploy_target" ]; then - local deploy_major - deploy_major=$(echo "$deploy_target" | cut -d. -f1) - local failed=0 - for fw in "$frameworks_dir"/*.dylib; do - [ -f "$fw" ] || continue - local name min_ver min_major - name=$(basename "$fw") - # otool -l prints LC_BUILD_VERSION with minos field, or LC_VERSION_MIN_MACOSX with version field - min_ver=$(otool -l "$fw" 2>/dev/null | awk '/LC_BUILD_VERSION/{found=1} found && /minos/{print $2; exit}') - if [ -z "$min_ver" ]; then - min_ver=$(otool -l "$fw" 2>/dev/null | awk '/LC_VERSION_MIN_MACOSX/{found=1} found && /version/{print $2; exit}') - fi - if [ -n "$min_ver" ]; then - min_major=$(echo "$min_ver" | cut -d. -f1) - if [ "$min_major" -gt "$deploy_major" ]; then - echo " ❌ FATAL: $name requires macOS $min_ver but deployment target is $deploy_target" - echo " This library was built on a newer macOS. Rebuild libpq with:" - echo " MACOSX_DEPLOYMENT_TARGET=$deploy_target brew reinstall libpq --build-from-source" - echo " Or use a CI runner on macOS $deploy_target+" - failed=1 - fi - fi - done - if [ "$failed" -eq 1 ]; then - echo "" - echo " Bundled dylibs target a newer macOS than the app's deployment target." - echo " The app will crash at launch on macOS $deploy_target with 'Symbol not found'." - exit 1 - fi - echo " ✅ All dylibs compatible with macOS $deploy_target" - else - echo " ⚠️ WARNING: Could not determine deployment target, skipping dylib version check" - fi - - echo "✅ Bundled $count dynamic libraries into Frameworks/" - ls -lh "$frameworks_dir"/*.dylib 2>/dev/null -} - -build_for_arch() { - local arch=$1 - echo "" - echo "🔨 Building for $arch..." - - # Prepare architecture-specific libraries - echo "📦 Preparing static libraries for $arch..." - prepare_arch_libs "$arch" libmariadb libpq libpgcommon libpgport libssl libcrypto libmongoc libbson libhiredis libhiredis_ssl libsqlite3_vendored - - # Create OpenSSL shared dylibs for this architecture - echo "📦 Creating OpenSSL shared dylibs for $arch..." - scripts/create-openssl-dylibs.sh "$arch" - - # TablePro.xcodeproj is generated and not in git, so a fresh checkout has none. - scripts/generate-project.sh - - # Persistent SPM package cache (speeds up CI on self-hosted runners) - SPM_CACHE_DIR="${HOME}/.spm-cache" - mkdir -p "$SPM_CACHE_DIR" - - # Resolve the provisioning profile UUID. Only the app target reads - # TABLEPRO_PROVISIONING_PROFILE (see Configs/Base.xcconfig), so passing it on the - # xcodebuild command line reaches the app and nothing else. A bare - # PROVISIONING_PROFILE_SPECIFIER override would hit every plugin and package target. - PROFILE_ARGS=() - PROFILE_PATH=$(find ~/Library/MobileDevice/Provisioning\ Profiles -name "*.provisionprofile" -print -quit 2>/dev/null) - if [ -n "${PROFILE_PATH:-}" ]; then - PROFILE_UUID=$(/usr/libexec/PlistBuddy -c "Print UUID" /dev/stdin <<< "$(security cms -D -i "$PROFILE_PATH" 2>/dev/null)" || true) - if [ -n "${PROFILE_UUID:-}" ]; then - echo "📋 Using provisioning profile: $PROFILE_UUID" - PROFILE_ARGS=("TABLEPRO_PROVISIONING_PROFILE=$PROFILE_UUID") - fi - fi - - # Build with xcodebuild - echo "Running xcodebuild..." - if ! xcodebuild \ - -project "$PROJECT" \ - -scheme "$SCHEME" \ - -configuration "$CONFIG" \ - -arch "$arch" \ - ONLY_ACTIVE_ARCH=YES \ - CODE_SIGN_IDENTITY="$SIGN_IDENTITY" \ - CODE_SIGN_STYLE=Manual \ - DEVELOPMENT_TEAM="$TEAM_ID" \ - CLANG_COVERAGE_MAPPING=NO \ - ENABLE_CODE_COVERAGE=NO \ - ${PROFILE_ARGS[@]+"${PROFILE_ARGS[@]}"} \ - -skipPackagePluginValidation \ - -clonedSourcePackagesDirPath "$SPM_CACHE_DIR" \ - -derivedDataPath build/DerivedData \ - build 2>&1 | tee "build-${arch}.log"; then - echo "❌ FATAL: xcodebuild failed for $arch" - echo "Check build-${arch}.log for details" - exit 1 - fi - echo "✅ Build succeeded for $arch" - - # Deterministic path via -derivedDataPath (no -showBuildSettings needed) - DERIVED_DATA="build/DerivedData/Build/Products" - - APP_PATH="${DERIVED_DATA}/${CONFIG}/TablePro.app" - echo "📂 Expected app path: $APP_PATH" - - # Verify app bundle exists - if [ ! -d "$APP_PATH" ]; then - echo "❌ ERROR: Built app not found at expected path: $APP_PATH" - echo "Build may have failed silently" - exit 1 - fi - - # Create release directory - mkdir -p "$BUILD_DIR" || { - echo "❌ FATAL: Failed to create release directory: $BUILD_DIR" - exit 1 - } - - # Copy and rename app. Removed first: cp -R onto an existing bundle merges into it, so a - # rebuild carried the previous build's files into the one being signed and shipped. - OUTPUT_NAME="TablePro-${arch}.app" - rm -rf "${BUILD_DIR:?}/$OUTPUT_NAME" - echo "Copying app bundle to release directory..." - if ! cp -R "$APP_PATH" "$BUILD_DIR/$OUTPUT_NAME"; then - echo "❌ FATAL: Failed to copy app bundle" - echo "Source: $APP_PATH" - echo "Destination: $BUILD_DIR/$OUTPUT_NAME" - exit 1 - fi - - # Verify the copy succeeded - if [ ! -d "$BUILD_DIR/$OUTPUT_NAME" ]; then - echo "❌ FATAL: App bundle was not copied successfully" - exit 1 - fi - - # Strip plugin binaries — removes debug symbols, code coverage (__LLVM_COV), - # and dead LINKEDIT metadata that bloat the bundle (e.g., OracleDriver 43MB → ~15MB) - echo "🔪 Stripping plugin binaries..." - PLUGINS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/PlugIns" - if [ -d "$PLUGINS_DIR" ]; then - for plugin in "$PLUGINS_DIR"/*.tableplugin; do - [ -d "$plugin" ] || continue - local plugin_name - plugin_name=$(basename "$plugin" .tableplugin) - local plugin_binary="$plugin/Contents/MacOS/$plugin_name" - if [ -f "$plugin_binary" ]; then - local before - before=$(ls -lh "$plugin_binary" | awk '{print $5}') - strip -x "$plugin_binary" - local after - after=$(ls -lh "$plugin_binary" | awk '{print $5}') - echo " $plugin_name: $before → $after" - fi - done - echo "✅ Plugin binaries stripped" - fi - - # Strip main binary - local main_binary="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/TablePro" - if [ -f "$main_binary" ]; then - local before - before=$(ls -lh "$main_binary" | awk '{print $5}') - strip -x "$main_binary" - local after - after=$(ls -lh "$main_binary" | awk '{print $5}') - echo "🔪 Main binary: $before → $after" - fi - - # Strip helper executables in Contents/MacOS - for helper in "$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS"/*; do - [ -f "$helper" ] || continue - [ "$(basename "$helper")" = "TablePro" ] && continue - local hname - hname=$(basename "$helper") - local before - before=$(ls -lh "$helper" | awk '{print $5}') - strip -x "$helper" - local after - after=$(ls -lh "$helper" | awk '{print $5}') - echo " $hname: $before → $after" - done - - # Strip PluginKit framework - local pluginkit_binary="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks/TableProPluginKit.framework/Versions/A/TableProPluginKit" - if [ -f "$pluginkit_binary" ]; then - strip -x "$pluginkit_binary" - echo " TableProPluginKit framework stripped" - fi - - # Remove development rpaths (absolute source paths) from all binaries - echo "🔧 Stripping development rpaths..." - for binary in "$main_binary" \ - "$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS"/* \ - "$PLUGINS_DIR"/*.tableplugin/Contents/MacOS/*; do - [ -f "$binary" ] || continue - otool -l "$binary" 2>/dev/null | grep "Libs/dylibs" | awk '{print $2}' | while read -r rpath; do - install_name_tool -delete_rpath "$rpath" "$binary" 2>/dev/null || true - done || true - done - - # Strip Sparkle helper binaries - local sparkle_dir="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks/Sparkle.framework/Versions/B" - for sparkle_bin in \ - "$sparkle_dir/Autoupdate" \ - "$sparkle_dir/Updater.app/Contents/MacOS/Updater"; do - if [ -f "$sparkle_bin" ]; then - strip -x "$sparkle_bin" - echo " $(basename "$sparkle_bin") (Sparkle) stripped" - fi - done - - # Remove Sparkle XPC services (not needed for non-sandboxed apps) - if [ -d "$sparkle_dir/XPCServices" ]; then - rm -rf "$sparkle_dir/XPCServices" - echo " Removed Sparkle XPC services (non-sandboxed app)" - fi - - # Copy shared OpenSSL dylibs into Frameworks - echo "📦 Copying OpenSSL shared dylibs to Frameworks/..." - FRAMEWORKS_EMBED_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks" - mkdir -p "$FRAMEWORKS_EMBED_DIR" - for lib in libcrypto.3.dylib libssl.3.dylib; do - if [ -f "Libs/dylibs/$lib" ]; then - cp -f "Libs/dylibs/$lib" "$FRAMEWORKS_EMBED_DIR/$lib" - chmod 644 "$FRAMEWORKS_EMBED_DIR/$lib" - echo " Copied $lib" - else - echo " WARNING: Libs/dylibs/$lib not found" - fi - done - - # Bundle non-system dynamic libraries (libpq, etc.) - bundle_dylibs "$BUILD_DIR/$OUTPUT_NAME" - - # Sign the entire app bundle with Developer ID. - # Sign from inside out: nested binaries → frameworks → dylibs → app. - echo "🔏 Signing app bundle with: $SIGN_IDENTITY" - FRAMEWORKS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/Frameworks" - - # Sign all nested XPC services, helper apps, and executables inside frameworks - while IFS= read -r -d '' binary; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$binary" - done < <(find "$FRAMEWORKS_DIR" -type f \( -name "*.xpc" -o -perm +111 \) -not -name "*.dylib" -not -name "*.plist" -not -name "*.h" -not -name "*.strings" -not -name "*.nib" -not -name "*.png" -not -name "*.icns" -not -name "*.car" -not -name "CodeResources" -not -name "Info.plist" -print0 2>/dev/null) - - # Sign XPC service bundles - while IFS= read -r -d '' xpc; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$xpc" - done < <(find "$FRAMEWORKS_DIR" -name "*.xpc" -type d -print0 2>/dev/null) - - # Sign nested .app bundles (e.g., Sparkle's Updater.app) - while IFS= read -r -d '' app; do - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$app" - done < <(find "$FRAMEWORKS_DIR" -name "*.app" -type d -print0 2>/dev/null) - - # Sign top-level frameworks - for fw in "$FRAMEWORKS_DIR"/*.framework; do - [ -d "$fw" ] || continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$fw" - done - - # Sign top-level dylibs - for dylib in "$FRAMEWORKS_DIR"/*.dylib; do - [ -f "$dylib" ] || continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$dylib" - done - - # Sign plugin bundles (stripped binaries need re-signing) - # Sign binary first, then bundle — inside-out order required for valid signatures - if [ -d "$PLUGINS_DIR" ]; then - for plugin in "$PLUGINS_DIR"/*.tableplugin; do - [ -d "$plugin" ] || continue - local plugin_name - plugin_name=$(basename "$plugin" .tableplugin) - local plugin_binary="$plugin/Contents/MacOS/$plugin_name" - # Sign the binary inside the bundle first - if [ -f "$plugin_binary" ]; then - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin_binary" - fi - # Then sign the bundle - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$plugin" - done - fi - - # Sign helper executables in Contents/MacOS (e.g., mcp-server) - MACOS_DIR="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS" - for helper in "$MACOS_DIR"/*; do - [ -f "$helper" ] || continue - [ "$(basename "$helper")" = "TablePro" ] && continue - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp "$helper" - done - - # Embed provisioning profile (required for iCloud entitlements) - PROFILE=$(find ~/Library/MobileDevice/Provisioning\ Profiles -name "*.provisionprofile" -print -quit 2>/dev/null) - if [ -n "$PROFILE" ]; then - echo "📋 Embedding provisioning profile: $(basename "$PROFILE")" - cp "$PROFILE" "$BUILD_DIR/$OUTPUT_NAME/Contents/embedded.provisionprofile" - fi - - # Sign the app bundle last - codesign -fs "$SIGN_IDENTITY" --force --options runtime --timestamp --entitlements "TablePro/TablePro.entitlements" "$BUILD_DIR/$OUTPUT_NAME" - echo "✅ Code signing complete" - - # Verify signature - if ! codesign --verify --deep --strict "$BUILD_DIR/$OUTPUT_NAME" 2>&1; then - echo "❌ FATAL: Code signature verification failed" - exit 1 - fi - echo "✅ Signature verified" - - # Verify binary exists inside the copied bundle - BINARY_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/TablePro" - if [ ! -f "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary not found in copied app bundle: $BINARY_PATH" - exit 1 - fi - - # Verify binary is not empty - if [ ! -s "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary file is empty" - exit 1 - fi - - # Verify binary is executable - if [ ! -x "$BINARY_PATH" ]; then - echo "❌ FATAL: Binary is not executable" - exit 1 - fi - - # Verify embedded MCP stdio bridge made it into the bundle - MCP_CLI_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/MacOS/tablepro-mcp" - if [ ! -x "$MCP_CLI_PATH" ]; then - echo "❌ FATAL: tablepro-mcp helper missing from $MCP_CLI_PATH" - echo "Check the mcp-server target's Copy Files build phase on the TablePro target." - exit 1 - fi - - SHIM_PATH="$BUILD_DIR/$OUTPUT_NAME/Contents/SharedSupport/bin/tablepro" - if [ ! -x "$SHIM_PATH" ]; then - echo "❌ FATAL: tablepro shim missing from $SHIM_PATH" - echo "Check the copy-files phase for TablePro/CLI/tablepro on the TablePro target." - exit 1 - fi - - # Get size - SIZE=$(ls -lh "$BINARY_PATH" 2>/dev/null | awk '{print $5}') - if [ -z "$SIZE" ]; then - echo "⚠️ WARNING: Could not determine binary size" - SIZE="unknown" - fi - - echo "✅ Built: $OUTPUT_NAME ($SIZE)" - - # Verify and display architecture - if ! lipo -info "$BINARY_PATH"; then - echo "⚠️ WARNING: Could not verify binary architecture" - fi -} - -# Main -case "$ARCH" in - arm64) - build_for_arch arm64 - ;; - x86_64) - build_for_arch x86_64 - ;; - both) - build_for_arch arm64 - echo "" - echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" - echo "" - build_for_arch x86_64 - ;; - *) - echo "Usage: $0 [arm64|x86_64|both]" - exit 1 - ;; -esac - -echo "" -echo "🎉 Build complete!" -echo "📁 Output: $BUILD_DIR/" - -if ! ls -lh "$BUILD_DIR" 2>/dev/null; then - echo "⚠️ WARNING: Could not list build directory contents" - echo "Directory may be empty or inaccessible" -fi - -# Notarization (opt-in via NOTARIZE=true) -if [ "$NOTARIZE" = "true" ]; then - echo "" - echo "📮 Notarizing..." - for app in "$BUILD_DIR"/TablePro-*.app; do - [ -d "$app" ] || continue - notarize_and_staple "$app" exec - done - echo "✅ Notarization complete" -fi diff --git a/scripts/check-registry-readiness.py b/scripts/check-registry-readiness.py deleted file mode 100644 index 934f5dbb5e..0000000000 --- a/scripts/check-registry-readiness.py +++ /dev/null @@ -1,81 +0,0 @@ -#!/usr/bin/env python3 -"""Fail if any registry database driver lacks a binary compatible with the app's -PluginKit range [floor, current]. - -Run this before tagging an app release so the app can never ship ahead of its -plugin binaries. With range-aware binary selection an additive PluginKit bump -needs no re-publish (an older resilient binary still serves), so this only fails -after a breaking bump that raised the floor and left the registry behind. - -The gate reads the raw GitHub origin, not the jsDelivr CDN that clients use, so -it sees the manifest the moment the registry push lands rather than waiting out -(or racing) the CDN edge cache. -""" - -import argparse -import json -import sys -import urllib.request - -DEFAULT_MANIFEST_URL = "https://raw.githubusercontent.com/TableProApp/plugins/main/plugins.json" - - -def fetch_manifest(url, retries=4): - last_error = None - for attempt in range(retries): - try: - request = urllib.request.Request(url, headers={"Cache-Control": "no-cache"}) - with urllib.request.urlopen(request, timeout=30) as response: - return json.load(response) - except Exception as error: # noqa: BLE001 - surface any fetch/parse failure - last_error = error - raise SystemExit(f"ERROR: could not fetch registry manifest from {url}: {last_error}") - - -def compatible_kits(plugin): - return sorted({ - binary.get("pluginKitVersion") - for binary in plugin.get("binaries", []) - if binary.get("pluginKitVersion") is not None - }) - - -def has_compatible_binary(plugin, floor, current): - return any(floor <= kit <= current for kit in compatible_kits(plugin)) - - -def main(): - parser = argparse.ArgumentParser(description="Check plugin registry readiness for an app release") - parser.add_argument("--manifest-url", default=DEFAULT_MANIFEST_URL) - parser.add_argument("--floor", required=True, type=int, help="minimumCompatiblePluginKitVersion") - parser.add_argument("--current", required=True, type=int, help="currentPluginKitVersion") - args = parser.parse_args() - - manifest = fetch_manifest(args.manifest_url) - plugins = manifest.get("plugins", manifest if isinstance(manifest, list) else []) - drivers = [plugin for plugin in plugins if plugin.get("category") == "database-driver"] - - if not drivers: - raise SystemExit("ERROR: no database-driver plugins found in the registry manifest") - - not_ready = [] - for plugin in drivers: - if not has_compatible_binary(plugin, args.floor, args.current): - name = plugin.get("name") or plugin.get("id") or "?" - not_ready.append(f"{name}: no binary in [{args.floor},{args.current}] (has {compatible_kits(plugin)})") - - if not_ready: - print("Registry is NOT ready for this app release:", file=sys.stderr) - for entry in not_ready: - print(f" - {entry}", file=sys.stderr) - print( - "Run scripts/release-all-plugins.sh for the new PluginKit version before tagging the app.", - file=sys.stderr, - ) - sys.exit(1) - - print(f"Registry ready: all {len(drivers)} database drivers have a binary in [{args.floor},{args.current}].") - - -if __name__ == "__main__": - main() diff --git a/scripts/ci/appcast_feed.py b/scripts/ci/appcast_feed.py deleted file mode 100644 index 0aba8980e4..0000000000 --- a/scripts/ci/appcast_feed.py +++ /dev/null @@ -1,198 +0,0 @@ -"""Text-level primitives for editing the published Sparkle feed. - -Two scripts change appcast.xml: merge-appcast.py adds a release's items, pull-release.py removes -them again when a build has to be withdrawn. They have to agree on where an item starts and ends, -so that lives here rather than in both. - -Everything works on the file's text, because a full ElementTree round trip destroys every CDATA -section. Measured on the published 695 KB feed: the round trip takes 7 ms, keeps the sparkle prefix -exactly and is semantically identical, but entity-escapes all 155 release-note blocks and grows the -file 9%, which every install then downloads and every future diff then buries. (The older claim here -that a round trip rewrites namespace prefixes was wrong; `register_namespace` preserves them.) So -items that are not being touched are moved byte for byte and only parsed to be understood. -""" - -import pathlib -import xml.etree.ElementTree as ElementTree - -SPARKLE_NS = "http://www.andymatuschak.org/xml-namespaces/sparkle" -SHORT_VERSION = f"{{{SPARKLE_NS}}}shortVersionString" -BUNDLE_VERSION = f"{{{SPARKLE_NS}}}version" -HARDWARE = f"{{{SPARKLE_NS}}}hardwareRequirements" -ED_SIGNATURE = f"{{{SPARKLE_NS}}}edSignature" - - -class FeedError(Exception): - """A violated invariant. Every one of these fails the job that raised it.""" - - -def parse_text(text): - try: - return ElementTree.fromstring(text) - except ElementTree.ParseError as error: - raise FeedError(f"not well-formed XML: {error}") from error - - -def parse(path): - try: - return ElementTree.parse(path).getroot() - except (OSError, ElementTree.ParseError) as error: - raise FeedError(f"{path}: {error}") from error - - -def read(path): - try: - return pathlib.Path(path).read_text(encoding="utf-8") - except OSError as error: - raise FeedError(f"{path}: {error}") from error - - -def channel_items(root, path): - channel = root.find("channel") - if channel is None: - raise FeedError(f"{path}: no <channel> element") - return channel.findall("item") - - -def item_short_version(item): - element = item.find(SHORT_VERSION) - return None if element is None else (element.text or "").strip() - - -def item_bundle_version(item): - element = item.find(BUNDLE_VERSION) - return None if element is None else (element.text or "").strip() - - -def item_enclosures(item): - return list(item.findall("enclosure")) - - -SKIPPABLE = (("<![CDATA[", "]]>", "CDATA section"), ("<!--", "-->", "comment")) -ITEM_OPEN = "<item" -ITEM_CLOSE = "</item>" -TAG_DELIMITERS = frozenset(">/ \t\r\n") - - -def skip_region(text, index): - """The offset just past a CDATA section or comment starting at `index`, else None.""" - for opener, closer, label in SKIPPABLE: - if text.startswith(opener, index): - end = text.find(closer, index + len(opener)) - if end == -1: - raise FeedError(f"unterminated {label} in the feed") - return end + len(closer) - return None - - -def opens_item(text, index): - """True only for a real `<item>` tag. `<itemCount>` starts with the same five characters.""" - if not text.startswith(ITEM_OPEN, index): - return False - after = index + len(ITEM_OPEN) - return after >= len(text) or text[after] in TAG_DELIMITERS - - -def item_spans(text): - """Whole-line (start, end) offsets of every <item> in document order. - - Release-note CDATA can hold either literal tag, and so can an XML comment, so both are skipped - whole rather than matching tags pairwise across the document. An item never nests inside another - item, so the first `</item>` outside a skipped region closes it. Every marker begins with `<`, - so the scan hops between those rather than walking characters. - """ - spans = [] - index = 0 - length = len(text) - while index < length: - index = text.find("<", index) - if index == -1: - break - skip = skip_region(text, index) - if skip is not None: - index = skip - continue - if not opens_item(text, index): - index += 1 - continue - - open_at = index - cursor = open_at + len(ITEM_OPEN) - close_at = -1 - while cursor < length: - cursor = text.find("<", cursor) - if cursor == -1: - break - skip = skip_region(text, cursor) - if skip is not None: - cursor = skip - continue - if text.startswith(ITEM_CLOSE, cursor): - close_at = cursor - break - cursor += 1 - if close_at == -1: - raise FeedError("an <item> is never closed in the feed") - - line_start = text.rfind("\n", 0, open_at) + 1 - line_end = text.find("\n", close_at) - line_end = length if line_end == -1 else line_end + 1 - spans.append((line_start, line_end)) - index = close_at + len(ITEM_CLOSE) - return spans - - -def sole_item_text(path, version): - """The one item's own bytes, whole lines, taken from a generated single-item feed.""" - text = read(path) - spans = item_spans(text) - if len(spans) != 1: - raise FeedError(f"{path}: expected exactly one <item>, found {len(spans)}") - start, end = spans[0] - block = text[start:end] - return block if block.endswith("\n") else block + "\n" - - -def splice_point(text, path): - """Where new items go: before the newest existing item, else before </channel>.""" - spans = item_spans(text) - if spans: - return spans[0][0] - anchor = text.find("</channel>") - if anchor == -1: - raise FeedError(f"{path}: no <item> and no </channel> to splice against") - return text.rfind("\n", 0, anchor) + 1 - - -def insert_items(base_text, path, items_text): - """Inserts whole lines at a line boundary. - - The old check here compared the merged text minus the inserted slice against the base, which - concatenation makes true by construction: it could never fail. The invariant worth asserting is - the one `splice_point` promises and `sole_item_text` relies on, which is that both the cut and - the inserted block land on line boundaries. - """ - at = splice_point(base_text, path) - if at != 0 and base_text[at - 1] != "\n": - raise FeedError(f"{path}: the splice point is mid-line, so the inserted item would be merged into another") - if items_text and not items_text.endswith("\n"): - raise FeedError(f"{path}: the inserted items do not end on a line boundary") - return base_text[:at] + items_text + base_text[at:] - - -def remove_version(base_text, path, version): - """Every item advertising `version`, removed whole. Returns (text, count).""" - root = parse_text(base_text) - spans = item_spans(base_text) - items = channel_items(root, path) - if len(spans) != len(items): - raise FeedError( - f"{path}: found {len(spans)} item spans in the text but {len(items)} parsed items" - ) - doomed = [span for span, item in zip(spans, items) if item_short_version(item) == version] - if not doomed: - return base_text, 0 - result = base_text - for start, end in reversed(doomed): - result = result[:start] + result[end:] - return result, len(doomed) diff --git a/scripts/ci/changelog_credits.py b/scripts/ci/changelog_credits.py deleted file mode 100644 index 0923880f4e..0000000000 --- a/scripts/ci/changelog_credits.py +++ /dev/null @@ -1,214 +0,0 @@ -#!/usr/bin/env python3 -"""Credit every CHANGELOG entry of one version to the pull request that added it. - -An outside contributor's entry ends `(#123 by @author)`, a maintainer's ends `(#123)`. An issue -reference stays first: `(#1748, #2741 by @author)`. A maintainer is a pull request author whose role -on the repository is admin or maintain. - -The credit comes from the history: `git blame` names the commit that wrote each entry line, a squash -merge ends its subject with `(#123)`, and `gh pr view` names that pull request's author. Blame -reports the last commit to touch a line, so reword a contributor's entry in their own pull request. -An entry whose commit has no pull request number, or that is not committed yet, is left alone and -listed. When an author's role cannot be read, the entry keeps the handle and the login is listed. - -Usage: - python3 scripts/ci/changelog_credits.py [--section Unreleased] [--dry-run] - -Run it from the repository root. It rewrites CHANGELOG.md in place and is idempotent. -""" - -import argparse -import json -import re -import subprocess -import sys -from dataclasses import dataclass -from pathlib import Path - -REFERENCES = re.compile(r"\s*\((#\d+(?:,\s*#\d+)*)\)\s*$") -CREDITED = re.compile(r"\(#\d+(?:,\s*#\d+)* by @[A-Za-z0-9][A-Za-z0-9-]*\)\s*$") -PULL_REQUEST_SUBJECT = re.compile(r"\(#(\d+)\)\s*$") -UNCOMMITTED = "0" * 40 -MAINTAINER_ROLES = {"admin", "maintain"} - - -def is_entry(line): - return line.startswith("- ") - - -def is_credited(line): - return CREDITED.search(line) is not None - - -def normalized_login(login): - """`gh` reports a GitHub App as `app/name`, which GitHub itself renders as `@name`.""" - return login.split("/", 1)[1] if login.startswith("app/") else login - - -def credit_entry(line, pull_request, author=None): - """`line` with the pull request folded into its reference parens, and `by @author` unless None.""" - if is_credited(line): - return line - body = line.rstrip() - references = [] - existing = REFERENCES.search(body) - if existing: - references = [reference.strip() for reference in existing.group(1).split(",")] - body = body[:existing.start()] - if f"#{pull_request}" not in references: - references.append(f"#{pull_request}") - credit = "" if author is None else f" by @{normalized_login(author)}" - return f"{body} ({', '.join(references)}{credit})" - - -def section_bounds(lines, section): - """Zero-based `[start, end)` of the entries under `## [section]`, heading excluded.""" - heading = f"## [{section}]" - start = next((index for index, line in enumerate(lines) if line.startswith(heading)), None) - if start is None: - raise ValueError(f"CHANGELOG.md has no {heading} section") - end = next( - (index for index in range(start + 1, len(lines)) if lines[index].startswith("## [")), - len(lines), - ) - return start + 1, end - - -@dataclass -class BlamedLine: - sha: str - summary: str - - -def parse_blame(porcelain): - """One `BlamedLine` per line of `git blame --line-porcelain` output, in order.""" - blamed = [] - sha = None - summary = "" - for line in porcelain.split("\n"): - header = re.match(r"^([0-9a-f]{40}) \d+ \d+", line) - if header: - sha = header.group(1) - summary = "" - elif line.startswith("summary "): - summary = line[len("summary "):] - elif line.startswith("\t") and sha is not None: - blamed.append(BlamedLine(sha=sha, summary=summary)) - return blamed - - -def pull_request_of(blamed): - if blamed.sha == UNCOMMITTED: - return None - match = PULL_REQUEST_SUBJECT.search(blamed.summary) - return match.group(1) if match else None - - -def blame_section(start, end): - result = subprocess.run( - ["git", "blame", "--line-porcelain", "-L", f"{start + 1},{end}", "--", "CHANGELOG.md"], - capture_output=True, - text=True, - check=True, - ) - return parse_blame(result.stdout) - - -def author_of(pull_request, cache): - if pull_request not in cache: - result = subprocess.run( - ["gh", "pr", "view", pull_request, "--json", "author"], - capture_output=True, - text=True, - check=True, - ) - cache[pull_request] = json.loads(result.stdout)["author"]["login"] - return cache[pull_request] - - -class Maintainers: - """Whether a login holds the admin or maintain role on the repository, asked once per login.""" - - def __init__(self, run=subprocess.run): - self._run = run - self._roles = {} - self.unknown = set() - - def includes(self, login): - if login.startswith("app/"): - return False - if login not in self._roles: - self._roles[login] = self._role_of(login) - return self._roles[login] in MAINTAINER_ROLES - - def _role_of(self, login): - result = self._run( - ["gh", "api", f"repos/{{owner}}/{{repo}}/collaborators/{login}/permission", "--jq", ".role_name"], - capture_output=True, - text=True, - ) - if result.returncode != 0: - self.unknown.add(login) - return None - return result.stdout.strip() - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--section", default="Unreleased", help="the version heading to credit") - parser.add_argument("--dry-run", action="store_true", help="print the result instead of writing it") - args = parser.parse_args(argv) - - path = Path("CHANGELOG.md") - lines = path.read_text(encoding="utf-8").split("\n") - try: - start, end = section_bounds(lines, args.section) - except ValueError as error: - sys.exit(f"ERROR: {error}") - if start == end: - print(f"[{args.section}] has no entries") - return 0 - - blamed = blame_section(start, end) - if len(blamed) != end - start: - sys.exit("ERROR: git blame did not return one record per line") - - authors = {} - maintainers = Maintainers() - credited = 0 - skipped = [] - for offset, record in enumerate(blamed): - index = start + offset - line = lines[index] - if not is_entry(line) or is_credited(line): - continue - pull_request = pull_request_of(record) - if pull_request is None: - reason = "not committed yet" if record.sha == UNCOMMITTED else f"{record.sha[:9]} has no pull request number" - skipped.append(f"line {index + 1}: {reason}: {line[:90]}") - continue - author = author_of(pull_request, authors) - handle = None if maintainers.includes(author) else author - lines[index] = credit_entry(line, pull_request, handle) - credited += 1 - - if args.dry_run: - print("\n".join(lines[start:end])) - else: - path.write_text("\n".join(lines), encoding="utf-8") - - logins = set(authors.values()) - inside = sorted(normalized_login(login) for login in logins if maintainers.includes(login)) - outside = sorted(normalized_login(login) for login in logins if not maintainers.includes(login)) - print(f"Credited {credited} entries in [{args.section}] across {len(authors)} pull requests.") - print(f"Maintainers, credited by pull request only: {', '.join('@' + login for login in inside) or 'none'}") - print(f"Contributors: {', '.join('@' + login for login in outside) or 'none'}") - for login in sorted(maintainers.unknown): - print(f"Role unknown, credited by handle: @{login}") - for line in skipped: - print(f"Left uncredited, {line}") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/check-plugin-manifest.py b/scripts/ci/check-plugin-manifest.py index dca111f733..971802dce9 100755 --- a/scripts/ci/check-plugin-manifest.py +++ b/scripts/ci/check-plugin-manifest.py @@ -1,19 +1,20 @@ #!/usr/bin/env python3 """Fails when .github/plugin-registry.json, the plugin classes and the project.yml plugin aggregates disagree.""" -import json import pathlib import re import sys REPO_ROOT = pathlib.Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO_ROOT / "scripts" / "release")) + +import registry # noqa: E402 + MANIFEST = REPO_ROOT / ".github" / "plugin-registry.json" PROJECT = REPO_ROOT / "project.yml" PLUGINS_DIR = REPO_ROOT / "Plugins" ICON = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) iconName(?:\s*:\s*String)?\s*=\s*\"([^\"]+)\"", re.M) -PRIMARY_ID = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) databaseTypeId(?:\s*:\s*String)?\s*=\s*\"([^\"]+)\"", re.M) -EXTRA_IDS = re.compile(r"^\s*(?:public |internal |private )?static (?:let|var) additionalDatabaseTypeIds\s*(?::\s*\[String\]\s*)?=\s*\[([^\]]*)\]", re.M) TOP_LEVEL_KEY = re.compile(r"^([A-Za-z_][\w-]*):") ENTRY = re.compile(r"^ ([A-Za-z_][\w-]*):\s*$") @@ -72,7 +73,7 @@ def embedded_plugins(app): def publish_set(plugins): - return {entry["target"] for entry in plugins.values() if not entry.get("bundled")} + return set(plugins) def aggregate_problems(project, plugins): @@ -122,57 +123,35 @@ def normalise(name): return None -def declared(directory): - icon = ids = None - extra = [] +def declared_icon(directory): for source in sorted(directory.rglob("*.swift")): - text = source.read_text(encoding="utf-8") - if icon is None: - found = ICON.search(text) - icon = found.group(1) if found else None - if ids is None: - found = PRIMARY_ID.search(text) - ids = found.group(1) if found else None - found = EXTRA_IDS.search(text) - if found and not extra: - extra = [part.strip().strip('"') for part in found.group(1).split(",") if part.strip()] - return icon, ids, extra + found = ICON.search(source.read_text(encoding="utf-8")) + if found: + return found.group(1) + return None def declaration_problems(plugins): problems = [] - for slug, entry in sorted(plugins.items()): - directory = plugin_directory(entry["target"]) + for target, entry in sorted(plugins.items()): + directory = plugin_directory(target) if directory is None: - problems.append(f"{slug}: no plugin directory matches target {entry['target']}") + problems.append(f"{target}: no plugin directory matches it") continue - - icon, primary, extra = declared(directory) + icon = declared_icon(directory) if icon is None: - problems.append(f"{slug}: could not read iconName from {directory.name}") - elif icon != entry["icon"]: - problems.append(f"{slug}: manifest icon {entry['icon']!r} but the plugin declares {icon!r}") - - # An export or import plugin drives no database type and declares none. - if entry["databaseTypeIds"] is None: - if primary is not None: - problems.append(f"{slug}: manifest says no database types but the plugin declares {primary!r}") - continue - - if primary is None: - problems.append(f"{slug}: could not read databaseTypeId from {directory.name}") - continue - if [primary, *extra] != entry["databaseTypeIds"]: - problems.append( - f"{slug}: manifest databaseTypeIds {entry['databaseTypeIds']} " - f"but the plugin declares {[primary, *extra]}" - ) + problems.append(f"{target}: could not read iconName from {directory.name}") + elif icon != entry["iconName"]: + problems.append(f"{target}: {MANIFEST.name} says iconName {entry['iconName']!r} " + f"but the plugin declares {icon!r}") return problems def main(): - with open(MANIFEST, encoding="utf-8") as handle: - plugins = json.load(handle)["plugins"] + try: + plugins = registry.load_metadata(MANIFEST)["plugins"] + except registry.RegistryError as error: + sys.exit(f"::error::{error}") project = parse_project(PROJECT.read_text(encoding="utf-8")) problems = declaration_problems(plugins) + aggregate_problems(project, plugins) diff --git a/scripts/ci/detect-changed-paths.sh b/scripts/ci/detect-changed-paths.sh index 0062d4273b..6eab0add36 100755 --- a/scripts/ci/detect-changed-paths.sh +++ b/scripts/ci/detect-changed-paths.sh @@ -24,7 +24,7 @@ case "$SUITE" in 'TablePro\.xcodeproj/project\.xcworkspace/xcshareddata/swiftpm/Package\.resolved$' 'scripts/(generate-project|download-libs|create-openssl-dylibs|build-dameng|build-hana)\.sh$' 'scripts/lib/' 'scripts/ci/(detect-changed-paths|summarize-xcresult|verify-plugin-loads)\.sh$' - '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/macos-tests\.yml$') ;; + 'scripts/release/notes\.py$' '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/macos-tests\.yml$') ;; plugins) # Registry plugins link the TableProCore and TableProOracle products. PATTERNS=('Plugins/' 'Packages/TableProCore/' 'Packages/TableProOracle/' 'Native/' 'Libs/' @@ -75,26 +75,17 @@ if [ -z "$BASE" ]; then exit 0 fi -# A release commit changes only these, so it reuses the verdict of the code it ships. -RELEASE_ONLY='CHANGELOG\.md|appcast\.xml|TablePro/Resources/WhatsNew\.md|\.github/release-flags\.json' - -# Version.xcconfig is the TablePro target's config file, so it counts as release-only only while -# its diff touches nothing but the two version lines. -VERSION_DIFF="$(git diff -U0 "$BASE" HEAD -- Configs/Version.xcconfig)" -OTHER_EDITS="$(awk '/^@@/ { hunk = 1; next } - hunk && /^[+-]/ && !/^[+-](MARKETING_VERSION|CURRENT_PROJECT_VERSION) = [0-9.]+$/' <<< "$VERSION_DIFF")" -[ -n "$OTHER_EDITS" ] || RELEASE_ONLY+='|Configs/Version\.xcconfig' - -IGNORED="^($RELEASE_ONLY)\$" -[ -z "$SKIP" ] || IGNORED+="|^($SKIP)" - # Raw NUL-separated paths: core.quotePath would quote a non-ASCII path and defeat the ^ anchor. # Files rather than $(...), which drops NUL bytes and would hide grep's exit status. CHANGED="$(mktemp)" RELEVANT="$(mktemp)" trap 'rm -f "$CHANGED" "$RELEVANT"' EXIT git -c core.quotePath=false diff --no-renames --name-only -z "$BASE" HEAD > "$CHANGED" -LC_ALL=C grep -zvE "$IGNORED" "$CHANGED" > "$RELEVANT" || [ $? -eq 1 ] +if [ -n "$SKIP" ]; then + LC_ALL=C grep -zvE "^($SKIP)" "$CHANGED" > "$RELEVANT" || [ $? -eq 1 ] +else + cp "$CHANGED" "$RELEVANT" +fi JOINED="$(IFS='|'; echo "${PATTERNS[*]}")" status=0 diff --git a/scripts/ci/draft-release.sh b/scripts/ci/draft-release.sh deleted file mode 100755 index 87eb77ba9c..0000000000 --- a/scripts/ci/draft-release.sh +++ /dev/null @@ -1,174 +0,0 @@ -#!/usr/bin/env bash -# -# Puts every asset of a release on a draft, for the caller to publish in one last step. Once -# release immutability is on, a published release can never gain, replace or lose an asset. -# -# Usage: draft-release.sh [--target <commit>] [--prerelease true|false] <tag> <title> <notes-file> <asset>... -# -# Writes id=<draft id> to $GITHUB_OUTPUT, or id= when <tag> is already published with these exact -# assets. Fails when <tag> is published with anything else. Without --target the tag must exist. - -set -euo pipefail - -usage() { - sed -n 's/^# Usage: //p' "$0" >&2 - exit 2 -} - -TARGET="" -PRERELEASE="false" -while [ $# -gt 0 ]; do - case "$1" in - --target) - [ $# -ge 2 ] || usage - TARGET="$2" - shift 2 - ;; - --prerelease) - [ $# -ge 2 ] || usage - PRERELEASE="$2" - shift 2 - ;; - -*) usage ;; - *) break ;; - esac -done -[ $# -ge 4 ] || usage -case "$PRERELEASE" in - true | false) ;; - *) usage ;; -esac - -TAG="$1" -TITLE="$2" -NOTES="$3" -shift 3 - -for file in "$NOTES" "$@"; do - if [ ! -f "$file" ]; then - echo "::error::$file does not exist" - exit 1 - fi -done - -export GH_REPO="${GH_REPO:-${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is not set}}" -OUTPUT="${GITHUB_OUTPUT:-/dev/stdout}" -WORK="$(mktemp -d)" -trap 'rm -rf "$WORK"' EXIT - -sha256() { - shasum -a 256 "$1" | cut -d' ' -f1 -} - -# Only asked once no published release holds the tag: gh races its published and draft lookups. -draft_id() { - if gh release view "$TAG" --json databaseId,isDraft --jq 'select(.isDraft) | .databaseId' 2> "$WORK/view.err"; then - return 0 - fi - if grep -q "release not found" "$WORK/view.err"; then - return 0 - fi - cat "$WORK/view.err" >&2 - return 1 -} - -# 0 when release $1 holds every later argument under its own name, byte for byte; 1 when it does -# not; 2 when the release could not be read. -holds_assets() { - local id="$1" file name row state digest - shift - gh api "repos/$GH_REPO/releases/$id" \ - --jq '.assets[] | [.name, .state, (.digest // "")] | @tsv' > "$WORK/assets.tsv" || return 2 - for file in "$@"; do - name="$(basename "$file")" - row="$(awk -F'\t' -v name="$name" '$1 == name { print; exit }' "$WORK/assets.tsv")" - if [ -z "$row" ]; then - echo "$TAG has no $name" - return 1 - fi - IFS=$'\t' read -r _ state digest <<< "$row" - if [ "$state" != "uploaded" ]; then - echo "$name on $TAG is $state" - return 1 - fi - # Assets uploaded before GitHub recorded digests carry none. - if [ -z "$digest" ]; then - rm -rf "$WORK/download" - gh release download "$TAG" --pattern "$name" --dir "$WORK/download" || return 2 - digest="sha256:$(sha256 "$WORK/download/$name")" - fi - if [ "$digest" != "sha256:$(sha256 "$file")" ]; then - echo "$name on $TAG is not the file built here" - return 1 - fi - done -} - -if ! PUBLISHED="$(gh api "repos/$GH_REPO/releases/tags/$TAG" --jq .id 2> "$WORK/tag.err")"; then - if ! grep -q "HTTP 404" "$WORK/tag.err"; then - cat "$WORK/tag.err" >&2 - exit 1 - fi - PUBLISHED="" -fi - -if [ -n "$PUBLISHED" ]; then - STATUS=0 - holds_assets "$PUBLISHED" "$@" || STATUS=$? - case "$STATUS" in - 0) - echo "$TAG is already published with these assets" - echo "id=" >> "$OUTPUT" - exit 0 - ;; - 1) - echo "::error::$TAG is already published with other assets, and a published release cannot change them. Ship this build under a new tag." - exit 1 - ;; - *) exit 1 ;; - esac -fi - -DRAFT="$(draft_id)" -if [ -n "$DRAFT" ]; then - # Reused, never deleted: if someone publishes it meanwhile, GitHub refuses the asset upload, - # while a delete would take the published release with it. - echo "Reusing draft $DRAFT, left for $TAG by an earlier attempt" - EDIT=(-f "name=$TITLE" -F "body=@$NOTES" -F "prerelease=$PRERELEASE") - if [ -n "$TARGET" ]; then - EDIT+=(-f "target_commitish=$TARGET") - fi - gh api --method PATCH "repos/$GH_REPO/releases/$DRAFT" "${EDIT[@]}" > /dev/null - gh release upload "$TAG" "$@" --clobber -else - FLAGS=(--draft --title "$TITLE" --notes-file "$NOTES") - if [ -n "$TARGET" ]; then - FLAGS+=(--target "$TARGET") - else - FLAGS+=(--verify-tag) - fi - if [ "$PRERELEASE" = "true" ]; then - FLAGS+=(--prerelease) - fi - gh release create "$TAG" "${FLAGS[@]}" "$@" - - for _ in 1 2 3 4 5; do - DRAFT="$(draft_id)" - [ -z "$DRAFT" ] || break - sleep 2 - done - if [ -z "$DRAFT" ]; then - echo "::error::the draft for $TAG was created but cannot be found" - exit 1 - fi -fi - -STATUS=0 -holds_assets "$DRAFT" "$@" || STATUS=$? -if [ "$STATUS" -ne 0 ]; then - echo "::error::draft $DRAFT for $TAG does not hold this build; it stays unpublished" - exit 1 -fi - -echo "Draft $DRAFT holds every asset for $TAG" -echo "id=$DRAFT" >> "$OUTPUT" diff --git a/scripts/ci/extract-release-notes.py b/scripts/ci/extract-release-notes.py deleted file mode 100644 index 4b0036576f..0000000000 --- a/scripts/ci/extract-release-notes.py +++ /dev/null @@ -1,129 +0,0 @@ -#!/usr/bin/env python3 -"""Keep a release's complete Markdown, with its sections in Keep a Changelog order. - -With --highlights-only, emit just the lead block: the lines a version section carries before its -first `### ` heading. That is what the update window and the What's New surface read, because the -full section is not something anyone reads in a dialog. 0.73.0's appcast item ran 22,443 bytes and -231 list items, and the feed grows by about that much per release for every install that polls it. - -The lead block is optional. When a version has none, this falls back to the full notes rather than -failing: the release job runs under `set -euo pipefail` about forty minutes in, after both -notarized builds, and a forgotten heading is not worth losing that to. -""" - -import argparse -import re -import sys -from pathlib import Path - -SECTION_ORDER = { - "added": 1, - "features": 1, - "new features": 1, - "changed": 2, - "performance": 2, - "deprecated": 3, - "removed": 4, - "fixed": 5, - "security": 6, -} -UNKNOWN_SECTION_ORDER = 7 - -MAX_HIGHLIGHT_LINES = 6 - - -def extract_sections(changelog, version): - """Every section of one version, as (priority, lines). Index 0 is the lead block.""" - sections = [(0, [])] - found = False - fence = None - for line in changelog.splitlines(keepends=True): - if fence is not None: - if re.fullmatch(r" {0,3}" + re.escape(fence[0]) + "{" + str(len(fence)) + r",}\s*", line): - fence = None - if found: - sections[-1][1].append(line) - continue - - opening_fence = re.match(r"^ {0,3}(`{3,}|~{3,})", line) - if opening_fence: - fence = opening_fence[1] - if found: - sections[-1][1].append(line) - continue - - release = re.match(r"^## \[([^]]+)\]", line) - if release: - if found: - break - found = release[1] == version - continue - - if found: - heading = re.match(r"^### (.+?)\s*$", line) - if heading: - sections.append((SECTION_ORDER.get(heading[1].casefold(), UNKNOWN_SECTION_ORDER), [])) - sections[-1][1].append(line) - return sections - - -def extract_notes(changelog, version): - # Keep each section intact, including nested lists, code, and unknown headings. - # The sort is stable, so sections at the same priority keep their original order. - sections = extract_sections(changelog, version) - blocks = ["".join(lines).strip("\n") for _, lines in sorted(sections, key=lambda section: section[0])] - notes = "\n\n".join(block for block in blocks if block.strip()) - if not notes.strip(): - raise ValueError(f"No release notes found for version {version} in CHANGELOG.md") - return notes + "\n" - - -def extract_highlights(changelog, version, require_lead=False): - """The lead block, capped at MAX_HIGHLIGHT_LINES, as one Markdown paragraph per line. - - Falls back to the full notes when a version has none, unless `require_lead`. The feed wants - the fallback, because failing a release forty minutes in over a missing heading is worse than - a long dialog. A file shipped inside the app bundle wants the opposite: 270 entries is not - something to compile into the product. - """ - sections = extract_sections(changelog, version) - lead = [line.rstrip("\r\n") for line in sections[0][1] if line.strip()] - if not lead: - if require_lead: - raise ValueError(f"Version {version} has no lead block in CHANGELOG.md") - return extract_notes(changelog, version) - return "\n\n".join(lead[:MAX_HIGHLIGHT_LINES]) + "\n" - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("version") - parser.add_argument( - "--highlights-only", - action="store_true", - help="emit the lead block instead of every section, falling back to the full notes", - ) - parser.add_argument( - "--require-lead-block", - action="store_true", - help="with --highlights-only, fail instead of falling back to the full notes", - ) - parser.add_argument("--out", default="release_notes.md", help="where to write (default: release_notes.md)") - args = parser.parse_args(argv) - - try: - changelog = Path("CHANGELOG.md").read_text(encoding="utf-8") - notes = extract_highlights(changelog, args.version, require_lead=args.require_lead_block) \ - if args.highlights_only else extract_notes(changelog, args.version) - Path(args.out).write_text(notes, encoding="utf-8") - except (OSError, ValueError) as error: - sys.exit(f"ERROR: {error}") - - label = "Release highlights" if args.highlights_only else "Release notes" - print(f"{label} extracted for {args.version} into {args.out}:") - print(notes, end="") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/extract-release-notes.sh b/scripts/ci/extract-release-notes.sh deleted file mode 100755 index 77a908ec62..0000000000 --- a/scripts/ci/extract-release-notes.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Thin wrapper so callers do not repeat the python3 invocation. Every argument after the version -# is passed straight through, which is how sign-and-appcast.sh asks for --highlights-only. - -VERSION="${1:?Usage: extract-release-notes.sh <version> [--highlights-only] [--out PATH]}" -shift - -python3 "$(dirname "$0")/extract-release-notes.py" "$VERSION" "$@" diff --git a/scripts/ci/merge-appcast.py b/scripts/ci/merge-appcast.py deleted file mode 100755 index f1f6c30741..0000000000 --- a/scripts/ci/merge-appcast.py +++ /dev/null @@ -1,229 +0,0 @@ -#!/usr/bin/env python3 -"""Splices one release's two architecture items into the published Sparkle feed. - -`generate_appcast` keeps only what it is handed, so the release used to hand it the whole -published feed as a seed. That is safe only while the staging directory holds nothing but the new -archive, and it stops being true as soon as anything else is staged. Verified in the pinned -Sparkle 2.9.5 source: `FeedXML.swift:333-339` matches an existing item by version, and two writes then run outside the resulting `if createNewItem` guard. Line 603 -assigns the enclosure's attributes unconditionally, so a seeded item matched by a staged archive -has its download URL rewritten into the current release's directory, which is a permanent 404. -Lines 523-525 remove that item's `<description>` when the staged archive has no sibling notes -file. Neither is caught by the rewind guard in build.yml, which compares only version strings. - -So generate_appcast now runs against a directory holding one release's archive and nothing else, -and this script merges its output into the feed. Items already published are never handed to a -generator at all; see appcast_feed for how they are moved. - -This replaces an awk state machine that scanned for `<item>` and `</item>` line by line, on three -untested assumptions, one of which was that no release note ever contains the literal `</item>`. - -Nothing pruned the feed, so it grew by two items per release forever: 155 items and 695 KB by -0.74.0, of which 84% was embedded release-note CDATA and 92% was items no install can ever be -offered. Every install downloads the whole file on every scheduled check, daily by default. -`--keep-releases` bounds it. Dropping an old item is safe because Sparkle picks the highest version -present (`SUAppcastDriver.bestItemFromAppcastItems`) and never looks for the host's own version in -the feed; what it cannot survive is losing the newest item, which is what the caller's rewind guard -still checks. - -Run: python3 scripts/ci/merge-appcast.py --base appcast.xml --version 0.75.0 \ - --arm64 <dir>/appcast.xml --x86-64 <dir>/appcast.xml \ - --download-prefix https://.../download/v0.75.0/ --out appcast/appcast.xml -""" - -import argparse -import pathlib -import sys - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -from appcast_feed import ( # noqa: E402 - ED_SIGNATURE, - HARDWARE, - FeedError, - channel_items, - insert_items, - item_bundle_version, - item_enclosures, - item_short_version, - parse, - parse_text, - read, - remove_version, - sole_item_text, -) - -DEFAULT_KEEP_RELEASES = 5 - -MergeError = FeedError - - -def sole_item(path, version): - items = channel_items(parse(path), path) - matching = [item for item in items if item_short_version(item) == version] - if len(items) != 1 or len(matching) != 1: - raise FeedError( - f"{path}: expected exactly one item for {version}, " - f"found {len(items)} item(s) of which {len(matching)} match" - ) - return matching[0] - - -def check_new_items(arm64_item, x86_64_item, version, download_prefix): - if arm64_item.find(HARDWARE) is None: - raise FeedError( - f"the arm64 item for {version} carries no <sparkle:hardwareRequirements>; " - "every install would be offered the Apple Silicon build" - ) - hardware = (arm64_item.find(HARDWARE).text or "").strip() - if hardware != "arm64": - raise FeedError(f"the arm64 item declares hardwareRequirements '{hardware}', expected 'arm64'") - if x86_64_item.find(HARDWARE) is not None: - raise FeedError(f"the x86_64 item for {version} carries <sparkle:hardwareRequirements>") - - arm64_bundle = item_bundle_version(arm64_item) - x86_64_bundle = item_bundle_version(x86_64_item) - if not arm64_bundle or arm64_bundle != x86_64_bundle: - raise FeedError( - f"the two architectures disagree on <sparkle:version>: " - f"arm64 {arm64_bundle!r}, x86_64 {x86_64_bundle!r}" - ) - - for label, item in (("arm64", arm64_item), ("x86_64", x86_64_item)): - enclosures = item_enclosures(item) - if not enclosures: - raise FeedError(f"the {label} item for {version} advertises no enclosure") - for enclosure in enclosures: - url = enclosure.get("url", "") - if not url.startswith(download_prefix): - raise FeedError( - f"the {label} item advertises {url!r}, which is not under {download_prefix!r}" - ) - # generate_appcast signs an archive only when the app's own Info.plist declares - # SUPublicEDKey (Appcast.swift:199 in 2.9.5). Drop that key and it writes an unsigned - # enclosure, reports "Wrote 1 new update" and exits 0, and every installed updater - # then refuses the download with nothing said anywhere. Measured against the pinned - # 2.9.5 generate_appcast with a bundle that omits the key. - if not enclosure.get(ED_SIGNATURE): - raise FeedError( - f"the {label} enclosure {url!r} carries no sparkle:edSignature; " - "check that TablePro/Info.plist still declares SUPublicEDKey" - ) - return arm64_bundle - - -def check_base(base_root, base_path, version, bundle_version): - for item in channel_items(base_root, base_path): - if item_short_version(item) == version: - raise FeedError(f"{base_path} already advertises {version}") - existing = item_bundle_version(item) - if existing and int(existing) >= int(bundle_version): - raise FeedError( - f"{base_path} advertises build {existing}, which is not below the new build " - f"{bundle_version}; Sparkle compares builds, so the release would not be offered" - ) - - -def releases_in_order(text, path): - """Every distinct shortVersionString, newest first, in document order.""" - ordered = [] - for item in channel_items(parse_text(text), path): - version = item_short_version(item) - if version and version not in ordered: - ordered.append(version) - return ordered - - -def prune(text, path, keep_releases): - """Drops every item belonging to a release outside the newest `keep_releases`.""" - if keep_releases <= 0: - return text, [] - ordered = releases_in_order(text, path) - doomed = ordered[keep_releases:] - result = text - for version in doomed: - result, removed = remove_version(result, path, version) - if removed == 0: - raise FeedError(f"{path}: {version} was listed for pruning but no item carries it") - return result, doomed - - -def merge(base_path, arm64_path, x86_64_path, version, download_prefix, out_path, keep_releases): - base_text = read(base_path) - base_root = parse(base_path) - - arm64_item = sole_item(arm64_path, version) - x86_64_item = sole_item(x86_64_path, version) - bundle_version = check_new_items(arm64_item, x86_64_item, version, download_prefix) - try: - int(bundle_version) - except ValueError as error: - raise FeedError(f"<sparkle:version> {bundle_version!r} is not an integer build number") from error - check_base(base_root, base_path, version, bundle_version) - - # arm64 first: Sparkle offers the first item the host can run, so an x86_64 item ahead of it - # would hand every Apple Silicon Mac the Intel build. - insert = sole_item_text(arm64_path, version) + sole_item_text(x86_64_path, version) - merged = insert_items(base_text, base_path, insert) - - merged_items = channel_items(parse_text(merged), "merged feed") - if len(merged_items) != len(channel_items(base_root, base_path)) + 2: - raise FeedError("the merged feed does not hold exactly two more items than the base") - if [item_short_version(item) for item in merged_items[:2]] != [version, version]: - raise FeedError("the two new items are not the first two in the merged feed") - if merged_items[0].find(HARDWARE) is None: - raise FeedError("the arm64 item is not first; Sparkle offers the first item the host can run") - - merged, dropped = prune(merged, "merged feed", keep_releases) - if dropped: - pruned_items = channel_items(parse_text(merged), "pruned feed") - if not pruned_items: - raise FeedError("pruning emptied the feed") - if item_short_version(pruned_items[0]) != version: - raise FeedError(f"pruning left {item_short_version(pruned_items[0])} newest instead of {version}") - if pruned_items[0].find(HARDWARE) is None: - raise FeedError("pruning left a non-arm64 item first") - kept = releases_in_order(merged, "pruned feed") - if len(kept) != keep_releases: - raise FeedError(f"pruning left {len(kept)} releases, expected {keep_releases}") - print(f"pruned {len(dropped)} release(s) below the newest {keep_releases}: {', '.join(dropped)}") - - pathlib.Path(out_path).parent.mkdir(parents=True, exist_ok=True) - pathlib.Path(out_path).write_text(merged, encoding="utf-8") - return merged - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--base", required=True, help="the published feed to splice into") - parser.add_argument("--version", required=True, help="marketing version being released") - parser.add_argument("--arm64", required=True, help="generate_appcast output for the arm64 archive") - parser.add_argument("--x86-64", required=True, dest="x86_64", help="generate_appcast output for x86_64") - parser.add_argument("--download-prefix", required=True, help="every enclosure URL must start with this") - parser.add_argument("--out", required=True, help="where to write the merged feed") - parser.add_argument( - "--keep-releases", - type=int, - default=DEFAULT_KEEP_RELEASES, - help=f"how many releases the published feed keeps, newest first (default: {DEFAULT_KEEP_RELEASES}; 0 keeps all)", - ) - args = parser.parse_args(argv) - - try: - merge( - args.base, - args.arm64, - args.x86_64, - args.version, - args.download_prefix, - args.out, - args.keep_releases, - ) - except FeedError as error: - print(f"::error::merge-appcast: {error}", file=sys.stderr) - return 1 - print(f"merged {args.version} into {args.base} -> {args.out}") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/notify-telegram.sh b/scripts/ci/notify-telegram.sh deleted file mode 100755 index e7de51b66c..0000000000 --- a/scripts/ci/notify-telegram.sh +++ /dev/null @@ -1,71 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -VERSION="${1:?Usage: notify-telegram.sh <version>}" - -if [ -z "${TELEGRAM_BOT_TOKEN:-}" ]; then - echo "❌ ERROR: TELEGRAM_BOT_TOKEN environment variable is not set" - exit 1 -fi - -if [ -z "${TELEGRAM_CHAT_ID:-}" ]; then - echo "❌ ERROR: TELEGRAM_CHAT_ID environment variable is not set" - exit 1 -fi - -RELEASE_URL="https://github.com/TableProApp/TablePro/releases/tag/v${VERSION}" -NOTES=$(cat release_notes.md 2>/dev/null || echo "Bug fixes and improvements") - -ESCAPED=$(echo "$NOTES" | sed -e 's/&/\&/g' -e 's/</\</g' -e 's/>/\>/g') - -FORMATTED=$(echo "$ESCAPED" | sed -E \ - -e 's/^### (.+)$/<b>\1<\/b>/' \ - -e 's/^- /• /' \ - -e 's/`([^`]+)`/<code>\1<\/code>/g' \ - -e '/^[[:space:]]*$/d') - -MESSAGE_LIMIT=4096 -TRUNCATION_NOTICE="• Read the rest in the full release notes." - -build_message() { - printf '<b>TablePro v%s Released</b>\n\n%s\n\n<a href="%s">View Release</a>' "$VERSION" "$1" "$RELEASE_URL" -} - -message_length() { - printf '%s' "$1" | wc -m | tr -d ' ' -} - -TEXT=$(build_message "$FORMATTED") - -# Telegram rejects a message over MESSAGE_LIMIT outright, so keep whole entries -# until the budget runs out and point at the release for the remainder. -if [ "$(message_length "$TEXT")" -gt "$MESSAGE_LIMIT" ]; then - OVERHEAD=$(( $(message_length "$TEXT") - $(message_length "$FORMATTED") )) - BUDGET=$(( MESSAGE_LIMIT - OVERHEAD - ${#TRUNCATION_NOTICE} - 1 )) - KEPT=$(printf '%s\n' "$FORMATTED" | awk -v budget="$BUDGET" ' - { if (used + length($0) + 1 > budget) exit - used += length($0) + 1 - print }') - FORMATTED=$(printf '%s\n%s' "$KEPT" "$TRUNCATION_NOTICE") - TEXT=$(build_message "$FORMATTED") - echo "Release notes exceeded ${MESSAGE_LIMIT} characters, trimmed to $(message_length "$TEXT")" -fi - -PAYLOAD=$(jq -n \ - --arg chat_id "$TELEGRAM_CHAT_ID" \ - --arg text "$TEXT" \ - --arg topic_id "${TELEGRAM_TOPIC_ID:-}" \ - '{chat_id: $chat_id, text: $text, parse_mode: "HTML", disable_web_page_preview: true} - + (if $topic_id != "" then {message_thread_id: ($topic_id | tonumber)} else {} end)') - -RESPONSE=$(curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \ - -H "Content-Type: application/json" \ - -d "$PAYLOAD") - -if echo "$RESPONSE" | jq -e '.ok == true' > /dev/null; then - echo "Telegram notification sent for v${VERSION}" -else - echo "Telegram API rejected the message:" - echo "$RESPONSE" | jq . - exit 1 -fi diff --git a/scripts/ci/package-artifacts.sh b/scripts/ci/package-artifacts.sh deleted file mode 100755 index 2021c5ab6b..0000000000 --- a/scripts/ci/package-artifacts.sh +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -ARCH="${1:?Usage: package-artifacts.sh <arch> [staging_dir]}" -STAGING="${2:-}" - -if [[ "$ARCH" != "arm64" && "$ARCH" != "x86_64" ]]; then - echo "❌ ERROR: Invalid architecture: $ARCH (expected arm64 or x86_64)" - exit 1 -fi - -# Read from the bundle being packaged rather than inferred from git. `git describe --tags` picks -# up whatever tag is nearest, which on a release commit that also carries a plugin tag is the -# wrong one, and its "dev" fallback shipped that name to a public release rather than failing. -# Taken from the artifact, the DMG name cannot disagree with the app inside it. -APP_BUNDLE="build/Release/TablePro-${ARCH}.app" -VERSION=$(plutil -extract CFBundleShortVersionString raw -o - "$APP_BUNDLE/Contents/Info.plist") - -# --- Create DMG --- -echo "Creating DMG installer..." - -# create-dmg is pre-installed in CI dependencies step -if ! command -v create-dmg &>/dev/null; then - echo "📦 Installing create-dmg tool..." - brew install create-dmg -fi - -chmod +x scripts/create-dmg.sh - -echo "📌 Using version: $VERSION" -NOTARIZE="${NOTARIZE:-false}" scripts/create-dmg.sh "$VERSION" "$ARCH" "$APP_BUNDLE" - -# Verify DMG was created. A DMG under any other name is not the one the release will upload, so -# accepting it here only moved the failure to somewhere it was harder to read. -DMG_FILE="build/Release/TablePro-${VERSION}-${ARCH}.dmg" -if [ ! -f "$DMG_FILE" ]; then - echo "❌ ERROR: expected DMG not found at: $DMG_FILE" >&2 - ls -la build/Release/*.dmg 2>/dev/null || true - exit 1 -fi -echo "✅ DMG installer created successfully: $DMG_FILE" - -ls -lh build/Release/*.dmg - -# --- Create the Sparkle update archive --- -echo "Creating the update archive..." - -cd build/Release - -# tar keeps the framework symlinks, which codesign seals. A signed bundle needs no extended -# attributes, and the build machine's owner means nothing on a user's Mac. -if ! /usr/bin/tar --no-mac-metadata --no-xattrs --no-acls --uid 0 --gid 0 \ - --options xz:compression-level=9 -cJf "TablePro-${ARCH}.tar.xz" "TablePro-${ARCH}.app"; then - echo "❌ ERROR: Failed to create the update archive" - exit 1 -fi - -echo "✅ Update archive created" -ls -lh "TablePro-${ARCH}.tar.xz" - -cd - > /dev/null - -# --- Stage artifacts (optional, for local/self-hosted use) --- -if [ -n "$STAGING" ]; then - mkdir -p "$STAGING" - cp build/Release/*.dmg "$STAGING/" 2>/dev/null || true - cp "build/Release/TablePro-${ARCH}.tar.xz" "$STAGING/" 2>/dev/null || true - echo "Artifacts staged to $STAGING" - ls -lh "$STAGING" -fi diff --git a/scripts/ci/plugin-info.py b/scripts/ci/plugin-info.py deleted file mode 100755 index 91217a123a..0000000000 --- a/scripts/ci/plugin-info.py +++ /dev/null @@ -1,77 +0,0 @@ -#!/usr/bin/env python3 -"""Resolves a plugin tag to the release metadata the workflow needs. - -Replaces a 120-line `case` block in build-plugin.yml. That block was shell holding data, which is -how seven of its icon values drifted away from the plugin classes they describe without anything -noticing. - -Usage: plugin-info.py <tag> prints KEY=VALUE lines for $GITHUB_OUTPUT - plugin-info.py --list-slugs prints every publishable slug, one per line -""" - -import json -import pathlib -import re -import sys - -REPO_ROOT = pathlib.Path(__file__).resolve().parents[2] -MANIFEST = REPO_ROOT / ".github" / "plugin-registry.json" -VERSION_XCCONFIG = REPO_ROOT / "Configs" / "Version.xcconfig" -TAG_PATTERN = re.compile(r"^plugin-([a-z0-9-]+)-v([0-9]+\.[0-9]+\.[0-9]+)$") - - -def load_plugins(): - with open(MANIFEST, encoding="utf-8") as handle: - return json.load(handle)["plugins"] - - -def marketing_version(): - """Configs/Version.xcconfig is the single declaration of the app version.""" - for line in VERSION_XCCONFIG.read_text(encoding="utf-8").splitlines(): - if line.startswith("MARKETING_VERSION"): - return line.split("=", 1)[1].strip() - sys.exit(f"MARKETING_VERSION missing from {VERSION_XCCONFIG}") - - -def main(): - if len(sys.argv) != 2: - sys.exit(__doc__) - - plugins = load_plugins() - - if sys.argv[1] == "--list-slugs": - print("\n".join(sorted(plugins))) - return - - match = TAG_PATTERN.match(sys.argv[1]) - if not match: - sys.exit(f"::error::Malformed plugin tag: {sys.argv[1]}") - slug, version = match.groups() - - plugin = plugins.get(slug) - if plugin is None: - known = ", ".join(sorted(plugins)) - sys.exit(f"::error::Unknown plugin '{slug}'. {MANIFEST.name} knows: {known}") - - ids = plugin["databaseTypeIds"] - fields = { - "target": plugin["target"], - "bundleId": plugin["bundleId"], - "displayName": plugin["displayName"], - "summary": plugin["summary"], - "dbTypeIds": "null" if ids is None else json.dumps(ids, separators=(",", ":")), - "icon": plugin["icon"], - "bundleName": plugin["bundleName"], - "category": plugin["category"], - "homepage": plugin["homepage"], - "version": version, - "minAppVersion": marketing_version(), - } - for key, value in fields.items(): - if "\n" in value: - sys.exit(f"::error::{slug}.{key} contains a newline, which $GITHUB_OUTPUT cannot carry") - print(f"{key}={value}") - - -if __name__ == "__main__": - main() diff --git a/scripts/ci/prepare-libs.sh b/scripts/ci/prepare-libs.sh deleted file mode 100755 index df0b4c083e..0000000000 --- a/scripts/ci/prepare-libs.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Selects the per-architecture slice of every vendored static library, so an Xcode build links the -# right one. Called by the release workflow before scripts/build-release.sh. -# -# The selection lives in scripts/lib/macos.sh and is shared with build-release.sh, which used to -# carry four near-identical copies of it. This script did a bare `cp` of the per-architecture file -# with no fallback, so the two took different sources for the same library. - -# shellcheck source=../lib/macos.sh -source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/lib/macos.sh" - -ARCH="${1:-}" -if [[ "$ARCH" != "arm64" && "$ARCH" != "x86_64" ]]; then - echo "Usage: $0 <arm64|x86_64>" >&2 - exit 1 -fi - -echo "📦 Preparing static libraries for $ARCH..." -prepare_arch_libs "$ARCH" \ - libmariadb \ - libpq libpgcommon libpgport libssl libcrypto \ - libmongoc libbson \ - libhiredis libhiredis_ssl \ - libsqlite3_vendored -echo "✅ Static libraries ready for $ARCH" diff --git a/scripts/ci/pull-release.py b/scripts/ci/pull-release.py deleted file mode 100755 index 4127a1154e..0000000000 --- a/scripts/ci/pull-release.py +++ /dev/null @@ -1,158 +0,0 @@ -#!/usr/bin/env python3 -"""Withdraws a released version from the Sparkle feed. - -Removing an item from appcast.xml is how a Sparkle release is withdrawn. There is no API for it -and no other mechanism, so the only question is whether the operator has a rehearsed tool or edits -a 640 KB XML file by hand under time pressure. - -This became necessary the moment updates started installing without being accepted first. Before -that a bad build was contained by the user, who saw a dialog and could decline it. Phased rollout -bounds the first wave, but it does not stop the cohorts behind it. - -The GitHub Release is deliberately left in place: people who downloaded the DMG directly still -need their link to resolve, and the corrective release is what supersedes the build. Only the feed -entry goes, so no updater offers it again. - -The items are removed with the same primitives merge-appcast.py uses to insert them, so the two -cannot disagree about where an item begins and ends. - -The push target is checked rather than assumed. SUFeedURL names main, so a withdrawal pushed to -any other branch leaves every install downloading the bad build while this script reports success. -`git push origin HEAD` from one of this repo's worktrees did exactly that. - -Run: python3 scripts/ci/pull-release.py 0.75.0 [--dry-run] [--feed appcast.xml] [--no-push] -""" - -import argparse -import pathlib -import subprocess -import sys - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -from appcast_feed import ( # noqa: E402 - FeedError, - channel_items, - item_short_version, - parse_text, - read, - remove_version, -) - - -PUBLISHED_BRANCH = "main" - - -def git(*arguments, check=True): - return subprocess.run(["git", *arguments], check=check, capture_output=True, text=True) - - -def checked_out_branch(): - """The current branch name, or None on a detached HEAD.""" - result = git("symbolic-ref", "--quiet", "--short", "HEAD", check=False) - return result.stdout.strip() or None - - -def require_publishable_checkout(): - """The feed every install polls is `main`, so nothing else may be pushed as a withdrawal.""" - branch = checked_out_branch() - if branch is None: - raise FeedError( - "HEAD is detached, so there is no branch to push. Check out " - f"{PUBLISHED_BRANCH} and run this again" - ) - if branch != PUBLISHED_BRANCH: - raise FeedError( - f"on branch {branch!r}, but SUFeedURL serves {PUBLISHED_BRANCH}. Pushing here would " - f"report the build withdrawn while every install kept downloading it. Check out " - f"{PUBLISHED_BRANCH} and run this again" - ) - return branch - - -def require_in_step_with_origin(): - """The release key skips every check, so the push may carry the withdrawal and nothing else.""" - fetch = git("fetch", "origin", PUBLISHED_BRANCH, check=False) - if fetch.returncode != 0: - raise FeedError(f"could not fetch origin/{PUBLISHED_BRANCH}: {fetch.stderr.strip()}") - head = git("rev-parse", "HEAD").stdout.strip() - published = git("rev-parse", "FETCH_HEAD").stdout.strip() - if head != published: - raise FeedError( - f"local {PUBLISHED_BRANCH} is not origin/{PUBLISHED_BRANCH}, so the push would carry other " - f"commits past every check or be refused. Make {PUBLISHED_BRANCH} match " - f"origin/{PUBLISHED_BRANCH} and run this again" - ) - - -def withdraw(feed_path, version): - """Returns (new_text, removed_count). Raises FeedError when the result would be wrong.""" - text = read(feed_path) - before = channel_items(parse_text(text), feed_path) - if not any(item_short_version(item) == version for item in before): - raise FeedError(f"{feed_path} does not advertise {version}, so there is nothing to withdraw") - - result, removed = remove_version(text, feed_path, version) - after = channel_items(parse_text(result), feed_path) - if len(after) != len(before) - removed: - raise FeedError("removing the items changed the number of other items in the feed") - if any(item_short_version(item) == version for item in after): - raise FeedError(f"{version} is still advertised after the removal") - if not after: - raise FeedError( - f"withdrawing {version} would empty the feed, and an empty feed tells every install " - "it is up to date forever" - ) - return result, removed - - -def main(argv=None): - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("version", help="the marketing version to withdraw, without the leading v") - parser.add_argument("--feed", default="appcast.xml", help="path to the feed (default: appcast.xml)") - parser.add_argument("--dry-run", action="store_true", help="report what would change, write nothing") - parser.add_argument("--no-push", action="store_true", help="commit but do not push") - args = parser.parse_args(argv) - - try: - if not args.dry_run and not args.no_push: - require_publishable_checkout() - require_in_step_with_origin() - result, removed = withdraw(args.feed, args.version) - except FeedError as error: - print(f"error: {error}", file=sys.stderr) - return 1 - - remaining = channel_items(parse_text(result), args.feed) - newest = item_short_version(remaining[0]) - print(f"{args.version}: {removed} item(s) to remove, {len(remaining)} left, newest becomes {newest}") - - if args.dry_run: - print("dry run, nothing written") - return 0 - - pathlib.Path(args.feed).write_text(result, encoding="utf-8") - - status = git("status", "--porcelain", "--", args.feed) - if not status.stdout.strip(): - print(f"{args.feed} is unchanged on disk, nothing to commit") - return 0 - - git("add", "--", args.feed) - git("commit", "-m", f"release: withdraw v{args.version} from the update feed") - if args.no_push: - print("committed, not pushed") - return 0 - - push = git("push", "origin", f"HEAD:{PUBLISHED_BRANCH}", check=False) - if push.returncode != 0: - print(push.stdout, file=sys.stderr) - print(push.stderr, file=sys.stderr) - print("error: the commit is local; push it before telling anyone the build is pulled", file=sys.stderr) - return 1 - print(f"withdrawn: v{args.version} is no longer offered to any install") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/scripts/ci/sign-and-appcast.sh b/scripts/ci/sign-and-appcast.sh deleted file mode 100755 index 68cc314b89..0000000000 --- a/scripts/ci/sign-and-appcast.sh +++ /dev/null @@ -1,161 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Signs release archives and publishes them into appcast.xml using Sparkle's generate_appcast, -# the official tool for building Sparkle update feeds. -# -# generate_appcast only ever sees the release being published. It is never handed the existing -# feed, and it never rewrites an item it did not create. Everything already published is merged -# in afterwards by scripts/ci/merge-appcast.py, which splices text rather than re-serializing. -# The reason is in that script's docstring: two writes in Sparkle's FeedXML.swift run outside its -# `if createNewItem` guard, so a previously published item that a staged archive happens to match -# has its download URL rewritten and its release notes dropped. -# -# generate_appcast rejects two archives sharing a bundle version in one directory -# (Unarchive.swift:96, SUSparkleErrorDomain 1002, reproduced against 2.9.5 and 2.10.0), so each -# architecture gets its own staging directory and its own generate_appcast run. -# -# Usage: sign-and-appcast.sh <version> -# Requires: SPARKLE_PRIVATE_KEY env var, artifacts/ directory with both architectures' tar.xz -# archives. -# Optional: CRITICAL_UPDATE=1 to mark the release critical. - -BASE_APPCAST="${BASE_APPCAST:-appcast.xml}" -CRITICAL_UPDATE="${CRITICAL_UPDATE:-0}" -VERSION="${1:?Usage: sign-and-appcast.sh <version>}" - -if [ -z "${SPARKLE_PRIVATE_KEY:-}" ]; then - echo "::error::SPARKLE_PRIVATE_KEY environment variable is not set" - exit 1 -fi - -if [ ! -f "$BASE_APPCAST" ]; then - echo "::error::base appcast $BASE_APPCAST does not exist" - exit 1 -fi - -# --------------------------------------------------------------------------- -# 1. Extract the notes -# --------------------------------------------------------------------------- -# Two files, for two audiences. release_notes.md is the whole section and becomes the GitHub -# release body, where a reader has scrolled to it on purpose. release_highlights.md is the lead -# block and is what goes in the feed, because an appcast item is downloaded by every install on -# every check and read inside a dialog. 0.73.0's ran 22,443 bytes and 231 list items. -# -# A version with no lead block falls back to the full notes, so this changes nothing until the -# convention is used. -bash "$(dirname "$0")/extract-release-notes.sh" "$VERSION" -bash "$(dirname "$0")/extract-release-notes.sh" "$VERSION" --highlights-only --out release_highlights.md - -# --------------------------------------------------------------------------- -# 2. Locate Sparkle tools -# --------------------------------------------------------------------------- -# Pinned and checksum-verified rather than installed from a cask that tracks latest. This step -# holds the EdDSA private key that signs every update every user receives, so it should not run a -# binary whose contents can change between releases. The version matches the Sparkle framework -# pinned in Package.resolved, and check-sparkle-version.py fails Repo Hygiene when they drift. -SPARKLE_VERSION="2.10.0" -SPARKLE_SHA256="c2bf58aa8387266ac179357b1415d6f2635f044da8be41042af32425dae6da0c" -SPARKLE_DIR="$(mktemp -d)" -curl -sSLo "$SPARKLE_DIR/sparkle.tar.xz" \ - "https://github.com/sparkle-project/Sparkle/releases/download/$SPARKLE_VERSION/Sparkle-$SPARKLE_VERSION.tar.xz" -echo "$SPARKLE_SHA256 $SPARKLE_DIR/sparkle.tar.xz" | shasum -a 256 -c - -tar xf "$SPARKLE_DIR/sparkle.tar.xz" -C "$SPARKLE_DIR" -SPARKLE_BIN="$SPARKLE_DIR/bin" - -DOWNLOAD_PREFIX="${GITHUB_SERVER_URL:-https://github.com}/${GITHUB_REPOSITORY:-TableProApp/TablePro}/releases/download/v${VERSION}/" - -KEY_FILE=$(mktemp) -STAGING_ROOT=$(mktemp -d) -trap 'rm -rf "$KEY_FILE" "$STAGING_ROOT"' EXIT - -echo "$SPARKLE_PRIVATE_KEY" > "$KEY_FILE" - -# --------------------------------------------------------------------------- -# 3. Generate one item per architecture, each in its own directory -# --------------------------------------------------------------------------- -generate_item() { - local arch="$1" - local archive="artifacts/TablePro-${VERSION}-${arch}.tar.xz" - local staging="$STAGING_ROOT/$arch" - - if [ ! -f "$archive" ]; then - echo "::error::$archive is missing, so the $arch build produced no update archive" - return 1 - fi - - mkdir -p "$staging" - cp "$archive" "$staging/" - - # Sparkle 2.9 renders Markdown natively, including code and links. Feeding hand-built HTML - # left Markdown visible and interpreted literal SQL/XML angle brackets as HTML tags. - # - # The name matches the archive's without .tar.xz, which is how generate_appcast pairs notes to - # an archive. Without the pairing the item ships with no description at all. - { - printf "# What's New in TablePro %s\n\n" "$VERSION" - cat release_highlights.md - printf '\n[View full changelog](https://docs.tablepro.app/changelog)\n' - } > "${staging}/TablePro-${VERSION}-${arch}.md" - - # Sparkle bypasses phasing for a critical item anyway, but passing both would be a contradiction - # in the feed rather than a belt and braces. - local flags=() - if [ "$CRITICAL_UPDATE" = "1" ]; then - # An empty --critical-update-version writes <sparkle:criticalUpdate/> with no version - # attribute, which SPUAppcastItemStateResolver treats as critical for every host. - flags+=(--critical-update-version "") - else - # Seven cohorts, so the interval times six is the tail: 21600 puts the last one 36 hours - # behind, which fits inside the median gap between releases. A user-initiated check is never - # phased, so Check for Updates always offers the newest build. - flags+=(--phased-rollout-interval 21600) - fi - - # --maximum-versions 1 states the invariant merge-appcast.py checks rather than leaving it as a - # consequence of the directory holding one archive. - "$SPARKLE_BIN/generate_appcast" \ - --ed-key-file "$KEY_FILE" \ - --download-url-prefix "$DOWNLOAD_PREFIX" \ - --embed-release-notes \ - --full-release-notes-url "https://docs.tablepro.app/changelog" \ - --maximum-versions 1 \ - "${flags[@]}" \ - "$staging" - - if [ ! -f "$staging/appcast.xml" ]; then - echo "::error::generate_appcast produced no feed for $arch" - return 1 - fi -} - -for arch in arm64 x86_64; do - generate_item "$arch" -done - -# --------------------------------------------------------------------------- -# 4. Keep both signed items, and merge once here as a gate -# --------------------------------------------------------------------------- -# The items are kept because the merge that actually ships runs later, in the commit step, against -# whatever appcast.xml main holds at that moment. Merging only here meant merging into a snapshot -# taken minutes earlier and then copying the result over main, so a withdrawal pushed in between -# was silently reverted and the pulled build was offered again. -# -# This merge still runs, against the snapshot, because it is the cheapest place to refuse a release: -# it happens before the GitHub Release publishes any artifact. Every invariant worth checking lives -# in merge-appcast.py, which scripts/ci/test_merge_appcast.py covers on the Linux runner, because a -# real release is the only thing that runs this script. -mkdir -p appcast/items -cp "$STAGING_ROOT/arm64/appcast.xml" appcast/items/arm64.xml -cp "$STAGING_ROOT/x86_64/appcast.xml" appcast/items/x86_64.xml - -python3 "$(dirname "$0")/merge-appcast.py" \ - --base "$BASE_APPCAST" \ - --version "$VERSION" \ - --arm64 appcast/items/arm64.xml \ - --x86-64 appcast/items/x86_64.xml \ - --download-prefix "$DOWNLOAD_PREFIX" \ - --out appcast/appcast.xml - -echo "✅ Appcast validated for $VERSION:" -head -c 4000 appcast/appcast.xml diff --git a/scripts/ci/test_appcast_feed.py b/scripts/ci/test_appcast_feed.py deleted file mode 100644 index 76ffb10665..0000000000 --- a/scripts/ci/test_appcast_feed.py +++ /dev/null @@ -1,160 +0,0 @@ -#!/usr/bin/env python3 -"""Regression tests for the text primitives both feed scripts share. - -merge-appcast.py and pull-release.py agree about where an item begins and ends only because they -call the same scanner. When that scanner is wrong, one of them moves the wrong bytes into the file -every install polls, and the result is still well-formed XML, so nothing downstream notices. - -Run: python3 scripts/ci/test_appcast_feed.py -""" - -import pathlib -import sys -import unittest - -sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent)) - -import appcast_feed # noqa: E402 - - -def feed(*items): - body = "".join(items) - return ( - '<?xml version="1.0" standalone="yes"?>\n' - '<rss xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle" version="2.0">\n' - " <channel>\n" - " <title>TablePro\n" - f"{body}" - " \n" - "\n" - ) - - -def item(version, body=""): - return ( - " \n" - f" {version}\n" - f"{body}" - " \n" - ) - - -class ItemSpanTests(unittest.TestCase): - def spans(self, text): - return [text[start:end] for start, end in appcast_feed.item_spans(text)] - - def test_finds_each_item_whole(self): - text = feed(item("0.75.0"), item("0.74.0")) - found = self.spans(text) - self.assertEqual(len(found), 2) - self.assertTrue(found[0].startswith(" ")) - self.assertTrue(found[0].rstrip().endswith("")) - - def test_a_comment_holding_an_open_tag_does_not_swallow_the_next_item(self): - """`` used to open a span that ran on until the real item's close tag, so - the first real item was consumed by a comment and silently moved with it.""" - text = feed(" \n", item("0.75.0")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertIn("0.75.0", found[0]) - self.assertNotIn("a note", found[0]) - - def test_a_comment_holding_a_close_tag_does_not_truncate_an_item(self): - """A comment inside an item used to close it early, so the rest of the item stayed behind - when the item was removed.""" - text = feed(item("0.75.0", " \n Keep\n")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertIn("Keep", found[0]) - - def test_an_element_whose_name_merely_starts_with_item_is_not_an_item(self): - text = feed(" 3\n", item("0.75.0")) - found = self.spans(text) - self.assertEqual(len(found), 1) - self.assertNotIn("itemCount", found[0]) - - def test_cdata_holding_either_tag_is_skipped_whole(self): - body = " and in the notes]]>\n" - text = feed(item("0.75.0", body), item("0.74.0")) - found = self.spans(text) - self.assertEqual(len(found), 2) - self.assertIn("in the notes", found[0]) - self.assertIn("0.74.0", found[1]) - - def test_an_unterminated_comment_is_refused(self): - with self.assertRaises(appcast_feed.FeedError) as caught: - appcast_feed.item_spans(feed(" ## [0.81.0] - 2026-10-16 @@ -5954,7 +5880,6 @@ TablePro is a native macOS database client built with SwiftUI and AppKit, design - Custom SQL query templates - Performance optimized for large datasets -[Unreleased]: https://github.com/TableProApp/TablePro/compare/v0.81.0...HEAD [0.81.0]: https://github.com/TableProApp/TablePro/compare/v0.80.0...v0.81.0 [0.80.0]: https://github.com/TableProApp/TablePro/compare/v0.79.0...v0.80.0 [0.79.0]: https://github.com/TableProApp/TablePro/compare/v0.78.0...v0.79.0 diff --git a/changelog.d/.gitkeep b/changelog.d/.gitkeep new file mode 100644 index 0000000000..e69de29bb2 diff --git a/changelog.d/3445.fixed.2.md b/changelog.d/3445.fixed.2.md new file mode 100644 index 0000000000..0077ab4ca1 --- /dev/null +++ b/changelog.d/3445.fixed.2.md @@ -0,0 +1 @@ +ClickHouse users with a read-only profile unable to connect. diff --git a/changelog.d/3445.fixed.3.md b/changelog.d/3445.fixed.3.md new file mode 100644 index 0000000000..3222574f87 --- /dev/null +++ b/changelog.d/3445.fixed.3.md @@ -0,0 +1 @@ +ClickHouse indexes, view definitions, functions, table comments and sizes failing to load on servers before 22.3. diff --git a/changelog.d/3445.fixed.4.md b/changelog.d/3445.fixed.4.md new file mode 100644 index 0000000000..924548688a --- /dev/null +++ b/changelog.d/3445.fixed.4.md @@ -0,0 +1 @@ +A ClickHouse query that failed partway showing its partial rows and the error text as a row instead of the error. diff --git a/changelog.d/3445.fixed.5.md b/changelog.d/3445.fixed.5.md new file mode 100644 index 0000000000..3714983d0b --- /dev/null +++ b/changelog.d/3445.fixed.5.md @@ -0,0 +1 @@ +A ClickHouse query with a `FORMAT` clause showing garbled rows in the grid and in an export. diff --git a/changelog.d/3445.fixed.6.md b/changelog.d/3445.fixed.6.md new file mode 100644 index 0000000000..2abc737804 --- /dev/null +++ b/changelog.d/3445.fixed.6.md @@ -0,0 +1 @@ +ClickHouse SQL export writing the current database's table structure for a table in another database. diff --git a/changelog.d/3445.fixed.7.md b/changelog.d/3445.fixed.7.md new file mode 100644 index 0000000000..91f6cccc3c --- /dev/null +++ b/changelog.d/3445.fixed.7.md @@ -0,0 +1 @@ +Opening a ClickHouse executable function's definition showing a syntax error. diff --git a/changelog.d/3445.fixed.8.md b/changelog.d/3445.fixed.8.md new file mode 100644 index 0000000000..4305b675b5 --- /dev/null +++ b/changelog.d/3445.fixed.8.md @@ -0,0 +1 @@ +SQL Server 2016 RTM offered a `CREATE OR ALTER VIEW` template it cannot run, and Azure SQL a drop-and-create one. diff --git a/changelog.d/3445.fixed.md b/changelog.d/3445.fixed.md new file mode 100644 index 0000000000..372c28e82a --- /dev/null +++ b/changelog.d/3445.fixed.md @@ -0,0 +1 @@ +Every ClickHouse query after connecting failing with `UNKNOWN_SETTING` on 23.8.0 to 23.8.8. diff --git a/changelog.d/3446.added.2.md b/changelog.d/3446.added.2.md new file mode 100644 index 0000000000..f256004406 --- /dev/null +++ b/changelog.d/3446.added.2.md @@ -0,0 +1 @@ +Edit > Set Value in the menu bar, with rows in Settings > Keyboard. diff --git a/changelog.d/3446.added.3.md b/changelog.d/3446.added.3.md new file mode 100644 index 0000000000..efe41eb712 --- /dev/null +++ b/changelog.d/3446.added.3.md @@ -0,0 +1 @@ +New UUID in a UUID primary key the server does not fill, on Add Row, Duplicate Row and Paste. diff --git a/changelog.d/3446.added.md b/changelog.d/3446.added.md new file mode 100644 index 0000000000..56f4369878 --- /dev/null +++ b/changelog.d/3446.added.md @@ -0,0 +1 @@ +Generate UUID and Generate UUIDv7 in Set Value, the row inspector and Fill Column, a new value for each cell. diff --git a/changelog.d/3446.changed.2.md b/changelog.d/3446.changed.2.md new file mode 100644 index 0000000000..1a31b68aa6 --- /dev/null +++ b/changelog.d/3446.changed.2.md @@ -0,0 +1 @@ +SQL functions in Set Value and the row inspector listed per database and only where they fit the column. diff --git a/changelog.d/3446.changed.3.md b/changelog.d/3446.changed.3.md new file mode 100644 index 0000000000..28908dfbc9 --- /dev/null +++ b/changelog.d/3446.changed.3.md @@ -0,0 +1 @@ +UUID functions in SQL Functions replaced by Generate UUID, which writes the value itself. diff --git a/changelog.d/3446.changed.4.md b/changelog.d/3446.changed.4.md new file mode 100644 index 0000000000..202fd11346 --- /dev/null +++ b/changelog.d/3446.changed.4.md @@ -0,0 +1 @@ +Text typed into a column that is not a date or time saved as typed, even when it reads like `now()`. diff --git a/changelog.d/3446.changed.5.md b/changelog.d/3446.changed.5.md new file mode 100644 index 0000000000..0fd52ff651 --- /dev/null +++ b/changelog.d/3446.changed.5.md @@ -0,0 +1 @@ +Set NULL and Set DEFAULT in the row inspector shown only where the column takes them, as in the data grid. diff --git a/changelog.d/3446.changed.6.md b/changelog.d/3446.changed.6.md new file mode 100644 index 0000000000..f6b8d566fe --- /dev/null +++ b/changelog.d/3446.changed.6.md @@ -0,0 +1 @@ +Default only on new rows on SQLite, libSQL, Turso, D1, ClickHouse, Databend, Snowflake, Trino, Cassandra. diff --git a/changelog.d/3446.changed.7.md b/changelog.d/3446.changed.7.md new file mode 100644 index 0000000000..0b12cf249e --- /dev/null +++ b/changelog.d/3446.changed.7.md @@ -0,0 +1 @@ +Pending SQL functions in the data grid drawn like a pending default. diff --git a/changelog.d/3446.changed.8.md b/changelog.d/3446.changed.8.md new file mode 100644 index 0000000000..102874b070 --- /dev/null +++ b/changelog.d/3446.changed.8.md @@ -0,0 +1 @@ +PluginKit ABI 39, additive: existing plugins keep loading. diff --git a/changelog.d/3446.changed.md b/changelog.d/3446.changed.md new file mode 100644 index 0000000000..b2eee69177 --- /dev/null +++ b/changelog.d/3446.changed.md @@ -0,0 +1 @@ +Set Value applies to every selected cell, or to the clicked column across the selected rows. diff --git a/changelog.d/3446.fixed.10.md b/changelog.d/3446.fixed.10.md new file mode 100644 index 0000000000..3de1acf839 --- /dev/null +++ b/changelog.d/3446.fixed.10.md @@ -0,0 +1 @@ +MariaDB `uuid` columns read as `CHAR`. diff --git a/changelog.d/3446.fixed.11.md b/changelog.d/3446.fixed.11.md new file mode 100644 index 0000000000..0dbd8f5afc --- /dev/null +++ b/changelog.d/3446.fixed.11.md @@ -0,0 +1 @@ +Empty field in the multi-row inspector after the selected rows were given different values. diff --git a/changelog.d/3446.fixed.2.md b/changelog.d/3446.fixed.2.md new file mode 100644 index 0000000000..1a5946c3b7 --- /dev/null +++ b/changelog.d/3446.fixed.2.md @@ -0,0 +1 @@ +SQL functions saved as text on SQL Server, Teradata, Spanner and BigQuery. diff --git a/changelog.d/3446.fixed.3.md b/changelog.d/3446.fixed.3.md new file mode 100644 index 0000000000..854eb5441e --- /dev/null +++ b/changelog.d/3446.fixed.3.md @@ -0,0 +1 @@ +`NOW()` offered on SQLite, Oracle, Cassandra and MongoDB, where it fails or saves the text. diff --git a/changelog.d/3446.fixed.4.md b/changelog.d/3446.fixed.4.md new file mode 100644 index 0000000000..d6bd75444a --- /dev/null +++ b/changelog.d/3446.fixed.4.md @@ -0,0 +1 @@ +SQLite's `datetime('now')` offered on DuckDB and `gen_random_uuid()` on Redshift. diff --git a/changelog.d/3446.fixed.5.md b/changelog.d/3446.fixed.5.md new file mode 100644 index 0000000000..64f5698583 --- /dev/null +++ b/changelog.d/3446.fixed.5.md @@ -0,0 +1 @@ +Stored text such as `CURRENT_DATE` turning into today's date on Duplicate Row or Restore Previous Values. diff --git a/changelog.d/3446.fixed.6.md b/changelog.d/3446.fixed.6.md new file mode 100644 index 0000000000..436f692a21 --- /dev/null +++ b/changelog.d/3446.fixed.6.md @@ -0,0 +1 @@ +Copy as INSERT and Copy as UPDATE quoting a staged SQL function. diff --git a/changelog.d/3446.fixed.7.md b/changelog.d/3446.fixed.7.md new file mode 100644 index 0000000000..052ec295c9 --- /dev/null +++ b/changelog.d/3446.fixed.7.md @@ -0,0 +1 @@ +Default on a saved row failing on SQLite, and storing `__DEFAULT__` on Teradata and DynamoDB. diff --git a/changelog.d/3446.fixed.8.md b/changelog.d/3446.fixed.8.md new file mode 100644 index 0000000000..6544d509f7 --- /dev/null +++ b/changelog.d/3446.fixed.8.md @@ -0,0 +1 @@ +Duplicate Row and Paste failing to save on a UUID primary key with no default. diff --git a/changelog.d/3446.fixed.9.md b/changelog.d/3446.fixed.9.md new file mode 100644 index 0000000000..13991df837 --- /dev/null +++ b/changelog.d/3446.fixed.9.md @@ -0,0 +1 @@ +Date functions and a date picker offered on Cassandra and ScyllaDB `timeuuid` columns. diff --git a/changelog.d/3446.fixed.md b/changelog.d/3446.fixed.md new file mode 100644 index 0000000000..b06bc3718e --- /dev/null +++ b/changelog.d/3446.fixed.md @@ -0,0 +1 @@ +`gen_random_uuid()`, `UUID()`, `NEWID()` and `datetime('now')` from SQL Functions saved as quoted text. diff --git a/changelog.d/3452.added.2.md b/changelog.d/3452.added.2.md new file mode 100644 index 0000000000..ff44c9880c --- /dev/null +++ b/changelog.d/3452.added.2.md @@ -0,0 +1 @@ +VS Code color theme import in Settings > Appearance, from the theme's `.json` file. diff --git a/changelog.d/3452.added.3.md b/changelog.d/3452.added.3.md new file mode 100644 index 0000000000..038da3f4e6 --- /dev/null +++ b/changelog.d/3452.added.3.md @@ -0,0 +1 @@ +Theme Editor window with a live preview, undo and Revert Changes. Open windows show each edit as you make it. diff --git a/changelog.d/3452.added.4.md b/changelog.d/3452.added.4.md new file mode 100644 index 0000000000..0ed6586c14 --- /dev/null +++ b/changelog.d/3452.added.4.md @@ -0,0 +1 @@ +iCloud sync of custom themes and the chosen light and dark themes, with Settings on in Settings > Sync. diff --git a/changelog.d/3452.added.5.md b/changelog.d/3452.added.5.md new file mode 100644 index 0000000000..f5bdb6564a --- /dev/null +++ b/changelog.d/3452.added.5.md @@ -0,0 +1 @@ +Theme import by dropping files on the Themes list, and export by dragging a theme to Finder. diff --git a/changelog.d/3452.added.md b/changelog.d/3452.added.md new file mode 100644 index 0000000000..f12d4a3c66 --- /dev/null +++ b/changelog.d/3452.added.md @@ -0,0 +1 @@ +Solarized, GitHub, Catppuccin, Gruvbox and Tokyo Night themes in light and dark, plus One Dark and One Light. diff --git a/changelog.d/3452.changed.2.md b/changelog.d/3452.changed.2.md new file mode 100644 index 0000000000..b7f4be3de7 --- /dev/null +++ b/changelog.d/3452.changed.2.md @@ -0,0 +1 @@ +Data Files, Compare & Sync and Open in Window value windows show the theme in the titlebar on macOS 26 and later. diff --git a/changelog.d/3452.changed.3.md b/changelog.d/3452.changed.3.md new file mode 100644 index 0000000000..0d293ea97d --- /dev/null +++ b/changelog.d/3452.changed.3.md @@ -0,0 +1 @@ +SQL aliases and parameters drawn in the editor text color instead of the NULL color. diff --git a/changelog.d/3452.changed.4.md b/changelog.d/3452.changed.4.md new file mode 100644 index 0000000000..2cf54f5c68 --- /dev/null +++ b/changelog.d/3452.changed.4.md @@ -0,0 +1 @@ +SQL previews in history, filters, maintenance and review sheets highlighted as in the editor, in the editor font. diff --git a/changelog.d/3452.changed.5.md b/changelog.d/3452.changed.5.md new file mode 100644 index 0000000000..a7be1d7ba9 --- /dev/null +++ b/changelog.d/3452.changed.5.md @@ -0,0 +1 @@ +Server Dashboard sessions and slow queries shown as data grids, with Cancel Query and Terminate Session in the header. diff --git a/changelog.d/3452.changed.6.md b/changelog.d/3452.changed.6.md new file mode 100644 index 0000000000..4e0ec6bb27 --- /dev/null +++ b/changelog.d/3452.changed.6.md @@ -0,0 +1 @@ +Terminate Session and Cancel Query confirmations name the session's process ID and user. diff --git a/changelog.d/3452.changed.7.md b/changelog.d/3452.changed.7.md new file mode 100644 index 0000000000..98ca5f3b81 --- /dev/null +++ b/changelog.d/3452.changed.7.md @@ -0,0 +1 @@ +Compare & Sync status bar shows the target connection's icon instead of a colored dot. diff --git a/changelog.d/3452.changed.8.md b/changelog.d/3452.changed.8.md new file mode 100644 index 0000000000..ab71034434 --- /dev/null +++ b/changelog.d/3452.changed.8.md @@ -0,0 +1 @@ +Themes move to the Trash instead of being deleted. Undo reverts a trash, rename, duplicate or import. diff --git a/changelog.d/3452.changed.9.md b/changelog.d/3452.changed.9.md new file mode 100644 index 0000000000..e12e3a46bc --- /dev/null +++ b/changelog.d/3452.changed.9.md @@ -0,0 +1 @@ +Editor tab strip over a themed window on macOS 26 and later drawn in a system fill instead of the theme color. diff --git a/changelog.d/3452.changed.md b/changelog.d/3452.changed.md new file mode 100644 index 0000000000..7a6e0beac9 --- /dev/null +++ b/changelog.d/3452.changed.md @@ -0,0 +1 @@ +Appearance (System, Light or Dark) kept per Mac instead of syncing. Older versions that sync settings switch to System. diff --git a/changelog.d/3452.fixed.10.md b/changelog.d/3452.fixed.10.md new file mode 100644 index 0000000000..fe65f4303d --- /dev/null +++ b/changelog.d/3452.fixed.10.md @@ -0,0 +1 @@ +System header band on the data grid and the Query Plan and Users & Roles trees under a theme with header colors. diff --git a/changelog.d/3452.fixed.11.md b/changelog.d/3452.fixed.11.md new file mode 100644 index 0000000000..8c7b248691 --- /dev/null +++ b/changelog.d/3452.fixed.11.md @@ -0,0 +1 @@ +System-colored alternate rows, header and booleans in a theme that sets only the grid background and text. diff --git a/changelog.d/3452.fixed.12.md b/changelog.d/3452.fixed.12.md new file mode 100644 index 0000000000..531ec9d8ba --- /dev/null +++ b/changelog.d/3452.fixed.12.md @@ -0,0 +1 @@ +Selected row text in white over a theme's selection color, or in the theme's color over the system selection. diff --git a/changelog.d/3452.fixed.13.md b/changelog.d/3452.fixed.13.md new file mode 100644 index 0000000000..fe68e81560 --- /dev/null +++ b/changelog.d/3452.fixed.13.md @@ -0,0 +1 @@ +Translucent theme selection, as in Dracula and Nord, in a different shade on rows, row numbers and column headers. diff --git a/changelog.d/3452.fixed.14.md b/changelog.d/3452.fixed.14.md new file mode 100644 index 0000000000..c206978d37 --- /dev/null +++ b/changelog.d/3452.fixed.14.md @@ -0,0 +1 @@ +No outline around a selected cell range in Dracula and Nord. diff --git a/changelog.d/3452.fixed.15.md b/changelog.d/3452.fixed.15.md new file mode 100644 index 0000000000..11d5aafcc6 --- /dev/null +++ b/changelog.d/3452.fixed.15.md @@ -0,0 +1 @@ +Shapes in the result Map view keeping the previous Accent color after it changes. diff --git a/changelog.d/3452.fixed.16.md b/changelog.d/3452.fixed.16.md new file mode 100644 index 0000000000..76e3047808 --- /dev/null +++ b/changelog.d/3452.fixed.16.md @@ -0,0 +1 @@ +Edit as Text in the array editor turning straight quotes and dashes into curly quotes and long dashes. diff --git a/changelog.d/3452.fixed.17.md b/changelog.d/3452.fixed.17.md new file mode 100644 index 0000000000..6afc32a86b --- /dev/null +++ b/changelog.d/3452.fixed.17.md @@ -0,0 +1 @@ +Theme editor bringing back a theme file deleted in Finder on the next change. diff --git a/changelog.d/3452.fixed.18.md b/changelog.d/3452.fixed.18.md new file mode 100644 index 0000000000..3ab2c13fb4 --- /dev/null +++ b/changelog.d/3452.fixed.18.md @@ -0,0 +1 @@ +Changes to a theme file made outside TablePro not shown until Settings > Appearance was opened. diff --git a/changelog.d/3452.fixed.19.md b/changelog.d/3452.fixed.19.md new file mode 100644 index 0000000000..8660028e1b --- /dev/null +++ b/changelog.d/3452.fixed.19.md @@ -0,0 +1 @@ +Theme files with a JSON syntax error reported without the line and column. diff --git a/changelog.d/3452.fixed.2.md b/changelog.d/3452.fixed.2.md new file mode 100644 index 0000000000..ffe0c76863 --- /dev/null +++ b/changelog.d/3452.fixed.2.md @@ -0,0 +1 @@ +No-match, invalid and NULL states in the JSON, PHP and array viewers drawn without the theme's background. diff --git a/changelog.d/3452.fixed.20.md b/changelog.d/3452.fixed.20.md new file mode 100644 index 0000000000..3dd695d1bb --- /dev/null +++ b/changelog.d/3452.fixed.20.md @@ -0,0 +1 @@ +Two themes with the same name impossible to tell apart in the theme pickers. diff --git a/changelog.d/3452.fixed.3.md b/changelog.d/3452.fixed.3.md new file mode 100644 index 0000000000..93eabe2870 --- /dev/null +++ b/changelog.d/3452.fixed.3.md @@ -0,0 +1 @@ +Agent result states and proposed SQL ignoring the theme. diff --git a/changelog.d/3452.fixed.4.md b/changelog.d/3452.fixed.4.md new file mode 100644 index 0000000000..a8589277cd --- /dev/null +++ b/changelog.d/3452.fixed.4.md @@ -0,0 +1 @@ +Connecting and Not connected panes, and Agent mode, in the system window color under a theme before macOS 26. diff --git a/changelog.d/3452.fixed.5.md b/changelog.d/3452.fixed.5.md new file mode 100644 index 0000000000..5ee9687031 --- /dev/null +++ b/changelog.d/3452.fixed.5.md @@ -0,0 +1 @@ +Selected text in SQL previews, such as query history, ignoring the theme's selection color. diff --git a/changelog.d/3452.fixed.6.md b/changelog.d/3452.fixed.6.md new file mode 100644 index 0000000000..14ad866d1a --- /dev/null +++ b/changelog.d/3452.fixed.6.md @@ -0,0 +1 @@ +Server Dashboard lists and the Query Insights chart ignoring the theme. diff --git a/changelog.d/3452.fixed.7.md b/changelog.d/3452.fixed.7.md new file mode 100644 index 0000000000..bfcc3b1736 --- /dev/null +++ b/changelog.d/3452.fixed.7.md @@ -0,0 +1 @@ +Compare & Sync row states and Data Files loading and failure states drawn without the theme's grid background. diff --git a/changelog.d/3452.fixed.8.md b/changelog.d/3452.fixed.8.md new file mode 100644 index 0000000000..6776e7a0a6 --- /dev/null +++ b/changelog.d/3452.fixed.8.md @@ -0,0 +1 @@ +Structure and Create Table code placeholders, Startup SQL in a link and the tunnel command preview ignoring the theme. diff --git a/changelog.d/3452.fixed.9.md b/changelog.d/3452.fixed.9.md new file mode 100644 index 0000000000..d13974a2f1 --- /dev/null +++ b/changelog.d/3452.fixed.9.md @@ -0,0 +1 @@ +Bars in query, history, filter, structure, plan, compare and other panes drawing a system band over a themed window. diff --git a/changelog.d/3452.fixed.md b/changelog.d/3452.fixed.md new file mode 100644 index 0000000000..37d6f54ad5 --- /dev/null +++ b/changelog.d/3452.fixed.md @@ -0,0 +1 @@ +Inline cell editor, the array editor's Edit as Text, and the inspector's hex dump and map pane ignoring the theme. diff --git a/changelog.d/3478.fixed.md b/changelog.d/3478.fixed.md new file mode 100644 index 0000000000..027b32d710 --- /dev/null +++ b/changelog.d/3478.fixed.md @@ -0,0 +1 @@ +No warning on a connection in the Welcome window whose driver plugin failed to load or is out of date. diff --git a/changelog.d/3491.changed.md b/changelog.d/3491.changed.md new file mode 100644 index 0000000000..29076adc78 --- /dev/null +++ b/changelog.d/3491.changed.md @@ -0,0 +1 @@ +Plugin download progress shown as the bytes arrive instead of jumping to full at the end. diff --git a/changelog.d/3497.changed.md b/changelog.d/3497.changed.md new file mode 100644 index 0000000000..825a6c6890 --- /dev/null +++ b/changelog.d/3497.changed.md @@ -0,0 +1 @@ +Welcome window and What's New show the build number after the version. diff --git a/scripts/ci/check-changelog.py b/scripts/ci/check-changelog.py new file mode 100755 index 0000000000..b7cf12308d --- /dev/null +++ b/scripts/ci/check-changelog.py @@ -0,0 +1,259 @@ +#!/usr/bin/env python3 + +import argparse +import datetime +import importlib.util +import json +import os +import re +import subprocess +import sys +import tempfile +from pathlib import Path +from typing import Callable, Dict, List, NamedTuple, Optional, Tuple + +ROOT = Path(__file__).resolve().parent.parent.parent +NOTES_SPEC = importlib.util.spec_from_file_location("notes", ROOT / "scripts" / "release" / "notes.py") +notes = importlib.util.module_from_spec(NOTES_SPEC) +NOTES_SPEC.loader.exec_module(notes) + +REPOSITORY = "TableProApp/TablePro" +MARKER = "" +FRAGMENTS = "changelog.d" +IGNORED = {".gitkeep"} +MAX_LINE = 120 +TYPE_DIRECTORY = re.compile(r'^\[\[tool\.towncrier\.type\]\]\ndirectory = "([a-z]+)"$', re.MULTILINE) +VERSION = r"(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)" +DATE = r"(?P[0-9]{4}-[0-9]{2}-[0-9]{2})" +OLD_HEADING = re.compile(rf"## \[(?P{VERSION})\] - {DATE}") +NEW_HEADING = re.compile( + rf"## \[(?P{VERSION})\]\(https://github\.com/{re.escape(REPOSITORY)}/releases/tag/v(?P=version)\)" + rf" - {DATE}(?: \[CRITICAL\])?" +) +# The forward-port to main keeps the release commit's subject, which ends in its (#N). +RELEASE_TITLE = re.compile(rf"chore\(release\): v{VERSION}(?: \(#[0-9]+\))?") +NEW_STATUSES = {"added", "copied"} + + +class Fragment(NamedTuple): + owner: str + kind: str + counter: Optional[str] + + +class ChangedFile(NamedTuple): + filename: str + status: str + previous_filename: Optional[str] = None + + +class CheckError(Exception): + pass + + +def fragment_types(towncrier_toml: str) -> List[str]: + return TYPE_DIRECTORY.findall(towncrier_toml) + + +def parse_fragment_name(name: str, types: List[str]) -> Optional[Fragment]: + match = re.fullmatch( + rf"(?P[0-9]+|\+[a-z0-9][a-z0-9-]*)\.(?P{'|'.join(types)})(?:\.(?P[1-9][0-9]*))?\.md", + name, + ) + if not match: + return None + return Fragment(match.group("owner"), match.group("kind"), match.group("counter")) + + +def fragment_problems(directory: Path, types: List[str]) -> List[str]: + if not directory.is_dir(): + return [] + problems = [] + for path in sorted(directory.iterdir()): + name = f"{FRAGMENTS}/{path.name}" + if path.name in IGNORED: + continue + if not path.is_file(): + problems.append(f"{name} is not a file") + continue + if parse_fragment_name(path.name, types) is None: + problems.append(f"{name} is not named ..md or ...md, with type one of " + + ", ".join(types)) + try: + text = path.read_text(encoding="utf-8") + except UnicodeDecodeError: + problems.append(f"{name} is not UTF-8") + continue + lines = [line.strip() for line in text.splitlines() if line.strip()] + if len(lines) != 1: + problems.append(f"{name} has {len(lines)} non-blank lines; a fragment is one line") + continue + line = lines[0] + if len(line) > MAX_LINE: + problems.append(f"{name} is {len(line)} characters, over the {MAX_LINE} limit") + if line.startswith("- "): + problems.append(f"{name} starts with `- `; towncrier adds the bullet") + if "by @" in line: + problems.append(f"{name} credits an author; notes.py adds the credit at release") + return problems + + +def _is_date(text: str) -> bool: + try: + datetime.date.fromisoformat(text) + except ValueError: + return False + return True + + +def changelog_problems(changelog: str) -> List[str]: + lines = changelog.split("\n") + markers = [index for index, line in enumerate(lines) if line == MARKER] + if len(markers) != 1: + return [f"CHANGELOG.md has {len(markers)} `{MARKER}` lines, expected 1"] + marker = markers[0] + problems = [] + + first = next((index for index in range(marker + 1, len(lines)) if lines[index].startswith("## [")), None) + if first is not None and lines[marker + 1:first] != [""]: + problems.append(f"CHANGELOG.md:{marker + 1}: only one blank line goes between the marker and the first ## [") + + versions: List[Tuple[int, ...]] = [] + seen: Dict[str, int] = {} + for index, line in enumerate(lines): + if not line.startswith("## "): + continue + number = index + 1 + if index < marker: + problems.append(f"CHANGELOG.md:{number}: `{line}` is above the marker") + continue + old, new = OLD_HEADING.fullmatch(line), NEW_HEADING.fullmatch(line) + heading = old or new + if heading is None: + problems.append(f"CHANGELOG.md:{number}: `{line}` is not `## [X.Y.Z]() - YYYY-MM-DD`") + continue + if not _is_date(heading.group("date")): + problems.append(f"CHANGELOG.md:{number}: {heading.group('date')} is not a date") + version = heading.group("version") + if version in seen: + problems.append(f"CHANGELOG.md:{number}: ## [{version}] repeats line {seen[version]}") + continue + seen[version] = number + parsed = tuple(int(part) for part in version.split(".")) + if versions and parsed >= versions[-1]: + problems.append(f"CHANGELOG.md:{number}: ## [{version}] is not below the version above it") + versions.append(parsed) + if new is not None: + problems += _new_section_problems(changelog, version, number) + return problems + + +def _new_section_problems(changelog: str, version: str, number: int) -> List[str]: + section = notes.find_section(changelog, version) + if section is None: + return [f"CHANGELOG.md:{number}: notes.py cannot find ## [{version}]"] + problems = [] + if len(section.lead) > notes.MAX_LEAD_LINES: + problems.append(f"CHANGELOG.md:{number}: the {version} lead block has {len(section.lead)} lines, " + f"over {notes.MAX_LEAD_LINES}") + with tempfile.TemporaryDirectory() as scratch: + source = Path(scratch, "CHANGELOG.md") + source.write_text(changelog, encoding="utf-8") + if not notes.whats_new(version, source, Path(scratch, "WhatsNew.md")): + problems.append(f"CHANGELOG.md:{number}: notes.py whats-new renders nothing for {version}") + return problems + + +def repository_problems(root: Path) -> List[str]: + types = fragment_types((root / "towncrier.toml").read_text(encoding="utf-8")) + if not types: + return ["towncrier.toml defines no [[tool.towncrier.type]]"] + return (changelog_problems((root / "CHANGELOG.md").read_text(encoding="utf-8")) + + fragment_problems(root / FRAGMENTS, types)) + + +def _fragment_in(path: Optional[str], types: List[str]) -> Optional[Fragment]: + if path is None or not path.startswith(f"{FRAGMENTS}/"): + return None + return parse_fragment_name(path[len(FRAGMENTS) + 1:], types) + + +def pull_request_problems(number: int, title: str, base: str, files: List[ChangedFile], + types: List[str]) -> List[str]: + problems = [] + expected = f"{FRAGMENTS}/{number}..md" + if any("CHANGELOG.md" in (changed.filename, changed.previous_filename) for changed in files) \ + and not RELEASE_TITLE.fullmatch(title): + problems.append(f"only a `chore(release): vX.Y.Z` pull request edits CHANGELOG.md. Put the entry in " + f"{expected}, with type one of {', '.join(types)} (.claude/rules/changelog.md)") + if base != "main": + return problems + + # A fragment deleted and added again under a new type keeps the pull request it is named for. + removed = {fragment.owner for fragment in + (_fragment_in(changed.filename, types) for changed in files if changed.status == "removed") + if fragment is not None} + for changed in files: + if changed.status not in NEW_STATUSES or not changed.filename.startswith(f"{FRAGMENTS}/"): + continue + if changed.filename[len(FRAGMENTS) + 1:] in IGNORED: + continue + fragment = _fragment_in(changed.filename, types) + if fragment is None: + problems.append(f"{changed.filename} is not a fragment name. Name it {expected}") + elif fragment.owner != str(number) and fragment.owner not in removed: + counter = f".{fragment.counter}" if fragment.counter else "" + problems.append(f"{changed.filename} is named for another pull request. " + f"Name it {FRAGMENTS}/{number}.{fragment.kind}{counter}.md") + return problems + + +def _gh(run: Callable, endpoint: str, jq: str, paginate: bool = False) -> List[str]: + command = ["gh", "api", *(["--paginate"] if paginate else []), endpoint, "--jq", jq] + result = run(command, capture_output=True, text=True) + if result.returncode != 0: + raise CheckError(f"gh api {endpoint}: {result.stderr.strip()}") + return [line for line in result.stdout.splitlines() if line.strip()] + + +def fetch_pull_request(number: int, repository: str, + run: Callable = subprocess.run) -> Tuple[str, str, List[ChangedFile]]: + endpoint = f"repos/{repository}/pulls/{number}" + details = json.loads(_gh(run, endpoint, "{title: .title, base: .base.ref} | @json")[0]) + entries = _gh(run, f"{endpoint}/files", ".[] | {filename, status, previous_filename} | @json", paginate=True) + files = [ChangedFile(entry["filename"], entry["status"], entry.get("previous_filename")) + for entry in map(json.loads, entries)] + return details["title"], details["base"], files + + +def workflow_command_data(text: str) -> str: + return text.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A") + + +def main(argv: Optional[List[str]] = None, run: Callable = subprocess.run) -> int: + parser = argparse.ArgumentParser(description="Check changelog.d/ and CHANGELOG.md.") + parser.add_argument("--pr", type=int, help="check the files this pull request changes instead") + parser.add_argument("--root", type=Path, default=ROOT) + args = parser.parse_args(argv) + + try: + if args.pr is None: + problems = repository_problems(args.root) + else: + types = fragment_types((args.root / "towncrier.toml").read_text(encoding="utf-8")) + repository = os.environ.get("GITHUB_REPOSITORY", REPOSITORY) + title, base, files = fetch_pull_request(args.pr, repository, run) + problems = pull_request_problems(args.pr, title, base, files, types) + except (CheckError, OSError, ValueError, KeyError, IndexError) as error: + problems = [str(error)] + + for problem in problems: + print(f"::error title=Changelog::{workflow_command_data(problem)}") + if problems: + return 1 + print("OK: changelog.d/ and CHANGELOG.md" if args.pr is None else f"OK: the changelog files of #{args.pr}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/ci/check-pr-title.py b/scripts/ci/check-pr-title.py index 0a54f3086b..bcb7785be2 100644 --- a/scripts/ci/check-pr-title.py +++ b/scripts/ci/check-pr-title.py @@ -1,13 +1,4 @@ #!/usr/bin/env python3 -"""Check a pull request title against the commit subject rules, because a squash merge uses it. - -The title must be a Conventional Commits subject, `()!: `, with the scope -and the `!` optional, and the title must fit in 72 characters. GitHub appends ` (#123)` when it -squashes; that suffix is not counted. - -Usage: - python3 scripts/ci/check-pr-title.py TITLE -""" import re import sys @@ -15,6 +6,8 @@ TYPES = ("feat", "fix", "refactor", "perf", "test", "docs", "build", "ci", "chore", "style", "revert") MAX_TITLE = 72 SUBJECT = re.compile(rf"^(?:{'|'.join(TYPES)})(?:\([a-z0-9][a-z0-9-]*\))?!?: \S") +# A pick into a release line keeps its subject from main, whose trailing (#N) names its origin. +PROVENANCE = re.compile(r" \(#[0-9]+\)$") def problems(title): @@ -26,8 +19,9 @@ def problems(title): + ", ".join(TYPES) + " and the scope is lowercase words joined by hyphens, for example `fix(editor): keep the caret`" ) - if len(title) > MAX_TITLE: - found.append(f"the title is {len(title)} characters, over the {MAX_TITLE} limit") + length = len(PROVENANCE.sub("", title, count=1)) + if length > MAX_TITLE: + found.append(f"the title is {length} characters, over the {MAX_TITLE} limit") return found diff --git a/scripts/ci/test_check_changelog.py b/scripts/ci/test_check_changelog.py new file mode 100644 index 0000000000..2b218e591e --- /dev/null +++ b/scripts/ci/test_check_changelog.py @@ -0,0 +1,318 @@ +#!/usr/bin/env python3 + +import importlib.util +import io +import json +import subprocess +import tempfile +import unittest +from contextlib import redirect_stdout +from pathlib import Path + +SPEC = importlib.util.spec_from_file_location("check_changelog", Path(__file__).with_name("check-changelog.py")) +check = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(check) + +TYPES = ["added", "changed", "deprecated", "removed", "fixed", "security"] +TOWNCRIER = "[tool.towncrier]\ndirectory = \"changelog.d\"\n" + "".join( + f"\n[[tool.towncrier.type]]\ndirectory = \"{kind}\"\nname = \"{kind.title()}\"\nshowcontent = true\n" + for kind in TYPES +) +LINK = "https://github.com/TableProApp/TablePro/releases/tag" +HEADER = "# Changelog\n\nUnreleased changes are files in `changelog.d/`.\n\n\n\n" +NEW = f"## [0.82.0]({LINK}/v0.82.0) - 2026-10-25\n\nOne lead line.\n\n### Added\n\n- A thing. ([#1](x))\n\n" +OLD = "## [0.81.0] - 2026-10-16\n\n### Fixed\n\n- A bug.\n\n## [0.80.0] - 2026-10-11\n\n### Added\n\n- Another.\n" + + +class TypesTests(unittest.TestCase): + def test_reads_the_fragment_types_and_not_the_directory(self): + self.assertEqual(check.fragment_types(TOWNCRIER), TYPES) + + +class FragmentTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.directory = Path(self.scratch.name, "changelog.d") + self.directory.mkdir() + (self.directory / ".gitkeep").write_text("", encoding="utf-8") + + def problems(self, name, text): + (self.directory / name).write_text(text, encoding="utf-8") + return check.fragment_problems(self.directory, TYPES) + + def test_a_one_line_fragment_passes(self): + self.assertEqual(self.problems("3446.added.md", "Generate UUID in Set Value.\n"), []) + + def test_counters_and_orphans_pass(self): + self.assertEqual(self.problems("3452.fixed.2.md", "A bug.\n"), []) + self.assertEqual(self.problems("+theme-import.changed.md", "A change.\n"), []) + + def test_an_empty_directory_and_a_missing_one_pass(self): + self.assertEqual(check.fragment_problems(self.directory, TYPES), []) + self.assertEqual(check.fragment_problems(self.directory / "missing", TYPES), []) + + def test_a_bad_name_is_refused(self): + for name in ("3446.feature.md", "3446.added.txt", "3446.added", "abc.added.md", "3446.md", + "3446.added.0.md", "+Theme.added.md"): + with self.subTest(name=name): + problems = self.problems(name, "A thing.\n") + (self.directory / name).unlink() + self.assertEqual(len(problems), 1) + self.assertIn("is not named", problems[0]) + + def test_more_than_one_line_is_refused(self): + self.assertIn("2 non-blank lines", self.problems("1.added.md", "One.\nTwo.\n")[0]) + + def test_a_blank_fragment_is_refused(self): + self.assertIn("0 non-blank lines", self.problems("1.added.md", "\n\n")[0]) + + def test_blank_lines_around_the_entry_pass(self): + self.assertEqual(self.problems("1.added.md", "\nA thing.\n\n"), []) + + def test_the_line_limit_is_120_characters(self): + self.assertEqual(self.problems("1.added.md", "x" * 120 + "\n"), []) + self.assertIn("121 characters", self.problems("1.added.md", "x" * 121 + "\n")[0]) + + def test_a_bullet_is_refused(self): + self.assertIn("`- `", self.problems("1.added.md", "- A thing.\n")[0]) + + def test_a_credit_is_refused(self): + self.assertIn("credit", self.problems("1.added.md", "A thing by @someone.\n")[0]) + + def test_a_directory_inside_is_refused(self): + (self.directory / "added").mkdir() + self.assertEqual(check.fragment_problems(self.directory, TYPES), ["changelog.d/added is not a file"]) + + +class ChangelogTests(unittest.TestCase): + def test_old_and_new_headings_pass(self): + self.assertEqual(check.changelog_problems(HEADER + NEW + OLD), []) + + def test_a_critical_new_heading_passes(self): + text = HEADER + NEW.replace("2026-10-25\n", "2026-10-25 [CRITICAL]\n", 1) + OLD + self.assertEqual(check.changelog_problems(text), []) + + def test_only_old_headings_pass(self): + self.assertEqual(check.changelog_problems(HEADER + OLD), []) + + def test_a_missing_or_repeated_marker_is_refused(self): + self.assertIn("0 `", check.changelog_problems(OLD)[0]) + self.assertIn("2 `", check.changelog_problems(HEADER + "\n" + OLD)[0]) + + def test_text_between_the_marker_and_the_first_version_is_refused(self): + text = HEADER + "- A stray entry.\n\n" + OLD + self.assertEqual(len(check.changelog_problems(text)), 1) + self.assertEqual(len(check.changelog_problems(HEADER + "\n" + OLD)), 1) + self.assertEqual(len(check.changelog_problems(HEADER.rstrip("\n") + "\n" + OLD)), 1) + + def test_a_heading_above_the_marker_is_refused(self): + text = "# Changelog\n\n## [Unreleased]\n\n" + HEADER.split("\n", 2)[2] + OLD + problems = check.changelog_problems(text) + self.assertEqual(len(problems), 1) + self.assertIn("above the marker", problems[0]) + + def test_a_heading_in_neither_grammar_is_refused(self): + for heading in ("## [Unreleased]", "## [0.82.0] 2026-10-25", "## [0.82.0-beta.1] - 2026-10-25", + f"## [0.82.0]({LINK}/v0.82.1) - 2026-10-25", + f"## [0.82.0](https://example.com/v0.82.0) - 2026-10-25", + "## [0.82.0] - 2026-10-25 [CRITICAL]"): + with self.subTest(heading=heading): + problems = check.changelog_problems(HEADER + heading + "\n\n" + OLD) + self.assertEqual(len(problems), 1) + self.assertIn("is not `## [X.Y.Z]", problems[0]) + + def test_a_heading_without_a_version_is_refused(self): + problems = check.changelog_problems(HEADER + OLD + "\n## Notes\n") + self.assertEqual(len(problems), 1) + self.assertIn("`## Notes` is not", problems[0]) + + def test_an_impossible_date_is_refused(self): + problems = check.changelog_problems(HEADER + OLD.replace("2026-10-16", "2026-02-30")) + self.assertEqual(problems, ["CHANGELOG.md:7: 2026-02-30 is not a date"]) + + def test_a_repeated_version_is_refused(self): + problems = check.changelog_problems(HEADER + OLD + "\n## [0.80.0] - 2026-10-01\n") + self.assertEqual(len(problems), 1) + self.assertIn("repeats", problems[0]) + + def test_versions_must_descend(self): + text = HEADER + "## [0.80.0] - 2026-10-11\n\n## [0.81.0] - 2026-10-16\n" + problems = check.changelog_problems(text) + self.assertEqual(len(problems), 1) + self.assertIn("not below", problems[0]) + + def test_versions_compare_as_numbers(self): + text = HEADER + "## [0.10.0] - 2026-10-11\n\n## [0.9.0] - 2026-10-01\n" + self.assertEqual(check.changelog_problems(text), []) + + def test_a_new_lead_block_takes_at_most_six_lines(self): + six = "\n".join(f"Lead {index}." for index in range(6)) + text = HEADER + NEW.replace("One lead line.", six) + OLD + self.assertEqual(check.changelog_problems(text), []) + problems = check.changelog_problems(HEADER + NEW.replace("One lead line.", six + "\nLead 6.") + OLD) + self.assertEqual(len(problems), 1) + self.assertIn("7 lines", problems[0]) + + def test_an_old_lead_block_is_not_limited(self): + lead = "\n".join(f"Lead {index}." for index in range(9)) + self.assertEqual(check.changelog_problems(HEADER + OLD.replace("### Fixed", lead + "\n\n### Fixed")), []) + + +class RepositoryTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.root = Path(self.scratch.name) + (self.root / "towncrier.toml").write_text(TOWNCRIER, encoding="utf-8") + (self.root / "CHANGELOG.md").write_text(HEADER + NEW + OLD, encoding="utf-8") + (self.root / "changelog.d").mkdir() + (self.root / "changelog.d" / "3446.added.md").write_text("A thing.\n", encoding="utf-8") + + def run_main(self, *argv): + captured = io.StringIO() + with redirect_stdout(captured): + status = check.main(["--root", str(self.root), *argv]) + return status, captured.getvalue() + + def test_a_good_tree_passes(self): + self.assertEqual(self.run_main(), (0, "OK: changelog.d/ and CHANGELOG.md\n")) + + def test_problems_from_both_files_are_annotations(self): + (self.root / "changelog.d" / "3446.added.md").write_text("- A thing.\n", encoding="utf-8") + (self.root / "CHANGELOG.md").write_text(HEADER + "## [Unreleased]\n\n" + OLD, encoding="utf-8") + status, out = self.run_main() + self.assertEqual(status, 1) + self.assertEqual(out.count("::error title=Changelog::"), 2) + + def test_a_config_with_no_types_fails(self): + (self.root / "towncrier.toml").write_text("[tool.towncrier]\n", encoding="utf-8") + status, out = self.run_main() + self.assertEqual(status, 1) + self.assertIn("no [[tool.towncrier.type]]", out) + + +def changed(filename, status="added", previous=None): + return check.ChangedFile(filename, status, previous) + + +class PullRequestTests(unittest.TestCase): + def problems(self, files, title="fix(editor): keep the caret", base="main", number=3600): + return check.pull_request_problems(number, title, base, files, TYPES) + + def test_a_fragment_named_for_the_pull_request_passes(self): + files = [changed("changelog.d/3600.fixed.md"), changed("changelog.d/3600.fixed.2.md"), + changed("TablePro/Editor.swift", "modified")] + self.assertEqual(self.problems(files), []) + + def test_editing_changelog_outside_a_release_names_the_fragment_to_add(self): + problems = self.problems([changed("CHANGELOG.md", "modified")]) + self.assertEqual(len(problems), 1) + self.assertIn("changelog.d/3600..md", problems[0]) + + def test_a_release_pull_request_edits_changelog(self): + files = [changed("CHANGELOG.md", "modified"), changed("changelog.d/3446.added.md", "removed")] + self.assertEqual(self.problems(files, "chore(release): v0.82.0", "release/0.82"), []) + self.assertEqual(self.problems(files, "chore(release): v0.82.1 (#3601)"), []) + + def test_titles_that_are_not_a_stable_release_cannot_edit_changelog(self): + for title in ("chore(release): v0.82.0-beta.1", "chore(release): 0.82.0", "chore: v0.82.0", + "chore(release): v0.82.0 and more", "chore(release): v0.82"): + with self.subTest(title=title): + self.assertEqual(len(self.problems([changed("CHANGELOG.md", "modified")], title)), 1) + + def test_renaming_changelog_away_counts_as_editing_it(self): + self.assertEqual(len(self.problems([changed("CHANGES.md", "renamed", "CHANGELOG.md")])), 1) + + def test_a_fragment_named_for_another_pull_request_prints_the_expected_name(self): + problems = self.problems([changed("changelog.d/3599.fixed.2.md")]) + self.assertEqual(len(problems), 1) + self.assertIn("Name it changelog.d/3600.fixed.2.md", problems[0]) + + def test_an_orphan_fragment_on_main_is_refused(self): + problems = self.problems([changed("changelog.d/+theme.fixed.md")]) + self.assertIn("Name it changelog.d/3600.fixed.md", problems[0]) + + def test_a_file_that_is_not_a_fragment_is_refused(self): + problems = self.problems([changed("changelog.d/notes.txt")]) + self.assertIn("Name it changelog.d/3600..md", problems[0]) + + def test_gitkeep_is_not_a_fragment(self): + self.assertEqual(self.problems([changed("changelog.d/.gitkeep")]), []) + + def test_a_pick_into_a_release_line_keeps_the_original_name(self): + self.assertEqual(self.problems([changed("changelog.d/3446.fixed.md")], "fix: keep it (#3446)", + "release/0.82"), []) + + def test_editing_or_renaming_another_pull_requests_fragment_passes(self): + files = [changed("changelog.d/3446.fixed.md", "modified"), + changed("changelog.d/3452.changed.md", "renamed", "changelog.d/3452.fixed.md")] + self.assertEqual(self.problems(files), []) + + def test_retyping_another_pull_requests_fragment_keeps_its_number(self): + files = [changed("changelog.d/3446.fixed.md", "removed"), changed("changelog.d/3446.changed.md")] + self.assertEqual(self.problems(files), []) + + +class FakeRun: + def __init__(self, *results): + self.results = list(results) + self.commands = [] + + def __call__(self, command, **_): + self.commands.append(command) + return self.results.pop(0) + + +def completed(stdout="", returncode=0, stderr=""): + return subprocess.CompletedProcess([], returncode, stdout, stderr) + + +class FetchTests(unittest.TestCase): + def setUp(self): + self.scratch = tempfile.TemporaryDirectory() + self.addCleanup(self.scratch.cleanup) + self.root = Path(self.scratch.name) + (self.root / "towncrier.toml").write_text(TOWNCRIER, encoding="utf-8") + + def run_main(self, run): + captured = io.StringIO() + with redirect_stdout(captured): + status = check.main(["--pr", "3600", "--root", str(self.root)], run) + return status, captured.getvalue() + + def test_reads_the_title_base_and_every_page_of_files(self): + files = "\n".join(json.dumps(entry) for entry in ( + {"filename": "CHANGELOG.md", "status": "modified", "previous_filename": None}, + {"filename": "changelog.d/1.added.md", "status": "renamed", "previous_filename": "changelog.d/1.fixed.md"}, + )) + run = FakeRun(completed(json.dumps({"title": "chore(release): v0.82.0", "base": "release/0.82"}) + "\n"), + completed(files + "\n")) + title, base, found = check.fetch_pull_request(3600, "TableProApp/TablePro", run) + self.assertEqual((title, base), ("chore(release): v0.82.0", "release/0.82")) + self.assertEqual(found, [changed("CHANGELOG.md", "modified"), + changed("changelog.d/1.added.md", "renamed", "changelog.d/1.fixed.md")]) + self.assertEqual(run.commands[0][:3], ["gh", "api", "repos/TableProApp/TablePro/pulls/3600"]) + self.assertEqual(run.commands[1][:4], ["gh", "api", "--paginate", "repos/TableProApp/TablePro/pulls/3600/files"]) + + def test_a_gh_failure_fails_the_check(self): + status, out = self.run_main(FakeRun(completed(returncode=1, stderr="HTTP 404"))) + self.assertEqual(status, 1) + self.assertIn("HTTP 404", out) + + def test_a_clean_pull_request_passes(self): + files = json.dumps({"filename": "changelog.d/3600.fixed.md", "status": "added"}) + run = FakeRun(completed(json.dumps({"title": "fix: x", "base": "main"})), completed(files)) + self.assertEqual(self.run_main(run), (0, "OK: the changelog files of #3600\n")) + + def test_an_annotation_cannot_inject_a_workflow_command(self): + files = json.dumps({"filename": "changelog.d/x\n::add-mask::secret", "status": "added"}) + run = FakeRun(completed(json.dumps({"title": "fix: x", "base": "main"})), completed(files)) + status, out = self.run_main(run) + self.assertEqual(status, 1) + self.assertEqual(len(out.splitlines()), 1) + self.assertIn("%0A::add-mask::secret", out) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/ci/test_check_pr_title.py b/scripts/ci/test_check_pr_title.py index dc176cfe43..f80ce884ca 100644 --- a/scripts/ci/test_check_pr_title.py +++ b/scripts/ci/test_check_pr_title.py @@ -1,8 +1,4 @@ #!/usr/bin/env python3 -"""Tests for check-pr-title.py. - -Run: python3 scripts/ci/test_check_pr_title.py -""" import importlib.util import io @@ -53,6 +49,24 @@ def test_the_title_alone_is_measured(self): def test_a_long_title_in_the_wrong_shape_reports_both(self): self.assertEqual(len(check.problems("Fix " + "x" * 100)), 2) + def test_a_picks_trailing_pull_request_number_is_not_counted(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ")) + self.assertEqual(check.problems(title + " (#3443)"), []) + self.assertEqual(check.problems("chore(release): v0.82.0 (#3601)"), []) + problems = check.problems(title + "x (#3443)") + self.assertEqual(len(problems), 1) + self.assertIn("73 characters", problems[0]) + + def test_only_one_trailing_pull_request_number_is_skipped(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ")) + self.assertIn("77 characters", check.problems(title + " (#1) (#2)")[0]) + + def test_a_number_that_is_not_at_the_end_is_counted(self): + title = "fix: " + "x" * (check.MAX_TITLE - len("fix: ") - len(" (#3443)")) + self.assertEqual(check.problems(title + " (#3443)"), []) + self.assertEqual(len(check.problems(title + " (#3443)x")), 1) + self.assertEqual(len(check.problems(title + "x (#3443) ")), 1) + class OutputTests(unittest.TestCase): def run_main(self, *argv): From cc8a140d3f5dcd7f2aa7219285f947b4f926aee4 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 09:46:54 +0700 Subject: [PATCH 06/10] ci(release): take versions from the tag, write What's New at build time --- Configs/Base.xcconfig | 5 +- Configs/Release.xcconfig | 7 +- Configs/Version.xcconfig | 6 - TablePro/Resources/WhatsNew.md | 9 -- TablePro/TablePro.entitlements | 6 +- .../Views/Welcome/WelcomeActionsPanel.swift | 16 +-- TablePro/Views/WhatsNew/WhatsNewNotice.swift | 16 +++ TablePro/Views/WhatsNew/WhatsNewView.swift | 20 ++-- .../Views/WhatsNewContentTests.swift | 76 ++++++++++-- project.yml | 108 ++++++------------ scripts/check-pluginkit-abi.sh | 28 ++--- scripts/generate-whats-new.sh | 38 ------ scripts/install-plugin-dev.sh | 4 +- 13 files changed, 147 insertions(+), 192 deletions(-) delete mode 100644 Configs/Version.xcconfig delete mode 100644 TablePro/Resources/WhatsNew.md create mode 100644 TablePro/Views/WhatsNew/WhatsNewNotice.swift delete mode 100755 scripts/generate-whats-new.sh diff --git a/Configs/Base.xcconfig b/Configs/Base.xcconfig index 323f91bffe..9f76303ed8 100644 --- a/Configs/Base.xcconfig +++ b/Configs/Base.xcconfig @@ -68,10 +68,7 @@ CODE_SIGN_STYLE = Automatic TABLEPRO_DEVELOPMENT_TEAM = D7HJ5TFYCU TABLEPRO_APP_BUNDLE_IDENTIFIER = com.TablePro -// Empty by default. scripts/build-release.sh passes a profile UUID on the xcodebuild -// command line; because only the app target reads this variable, the profile cannot -// leak into plugin or Swift package targets the way a bare -// PROVISIONING_PROFILE_SPECIFIER override would. +// Only the app target reads this, so a profile passed here never reaches a plugin or package target. TABLEPRO_PROVISIONING_PROFILE = #include? "Secrets.xcconfig" diff --git a/Configs/Release.xcconfig b/Configs/Release.xcconfig index 8aea3fe798..dc3d6989a0 100644 --- a/Configs/Release.xcconfig +++ b/Configs/Release.xcconfig @@ -5,10 +5,9 @@ ENABLE_NS_ASSERTIONS = NO MTL_ENABLE_DEBUG_INFO = NO SWIFT_COMPILATION_MODE = wholemodule -// scripts/build-release.sh used to pass these three on the xcodebuild command line, where an -// override reaches every target in the graph, SPM packages included. They belong to this project's -// Release configuration and nothing else. LLVM_LTO is the Clang knob: it reaches the nine C bridge -// modules, not Swift, which has its own SWIFT_LTO and leaves it off. +// Here rather than on the xcodebuild command line, where an override reaches every target in the +// graph, SPM packages included. LLVM_LTO is the Clang knob: it reaches the C bridge modules, not +// Swift, which has its own SWIFT_LTO and leaves it off. GCC_OPTIMIZATION_LEVEL = s SWIFT_OPTIMIZATION_LEVEL = -O LLVM_LTO = YES_THIN diff --git a/Configs/Version.xcconfig b/Configs/Version.xcconfig deleted file mode 100644 index fa8e3b186e..0000000000 --- a/Configs/Version.xcconfig +++ /dev/null @@ -1,6 +0,0 @@ -// Single source of truth for the macOS app version. -// The release skill and .github/workflows/build-plugin.yml read this file. -// Do not restate these values anywhere else. - -MARKETING_VERSION = 0.81.0 -CURRENT_PROJECT_VERSION = 140 diff --git a/TablePro/Resources/WhatsNew.md b/TablePro/Resources/WhatsNew.md deleted file mode 100644 index 020b26e0f8..0000000000 --- a/TablePro/Resources/WhatsNew.md +++ /dev/null @@ -1,9 +0,0 @@ -# TablePro 0.81.0 - -Cloud Firestore, Firebase Realtime Database and Google Cloud Bigtable connect through new registry plugins. - -Each database in the connections strip keeps its own tabs, and Oracle query tabs get a schema picker. - -Themes color every content view and the window background. Themes made for earlier versions need to be recreated. - -Plugins signed by other developers stop loading, and HTML and XML export are removed. diff --git a/TablePro/TablePro.entitlements b/TablePro/TablePro.entitlements index 1f880e34d9..e094e5eb3d 100644 --- a/TablePro/TablePro.entitlements +++ b/TablePro/TablePro.entitlements @@ -2,8 +2,6 @@ - com.apple.application-identifier - D7HJ5TFYCU.com.TablePro com.apple.developer.icloud-container-identifiers iCloud.com.TablePro @@ -14,15 +12,13 @@ com.apple.developer.icloud-container-environment Production - com.apple.developer.team-identifier - D7HJ5TFYCU com.apple.security.app-sandbox com.apple.security.cs.disable-library-validation keychain-access-groups - D7HJ5TFYCU.com.TablePro.shared + $(AppIdentifierPrefix)com.TablePro.shared diff --git a/TablePro/Views/Welcome/WelcomeActionsPanel.swift b/TablePro/Views/Welcome/WelcomeActionsPanel.swift index d58cb53b06..98b8ce8f84 100644 --- a/TablePro/Views/Welcome/WelcomeActionsPanel.swift +++ b/TablePro/Views/Welcome/WelcomeActionsPanel.swift @@ -21,7 +21,7 @@ struct WelcomeActionsPanel: View { .removeDuplicates() /// Captured once: the same appearance that reads the stored value overwrites it. - @State private var lastSeenAppVersion = "" + @State private var showsWhatsNew: Bool? var body: some View { VStack(spacing: 0) { @@ -54,7 +54,7 @@ struct WelcomeActionsPanel: View { .accessibilityIdentifier("welcome-check-for-updates-link") } - if showsWhatsNew { + if showsWhatsNew == true { Button(String(format: String(localized: "What's New in %@"), Bundle.main.appVersion)) { NSApp.sendAction(#selector(AppDelegate.openChangelog(_:)), to: nil, from: nil) } @@ -115,16 +115,8 @@ struct WelcomeActionsPanel: View { } private func recordAppVersion() { - let defaults = AppStorageEnvironment.shared.defaults - let key = PreferenceKeys.lastSeenAppVersion.name - let current = Bundle.main.appVersion - guard lastSeenAppVersion.isEmpty else { return } - lastSeenAppVersion = defaults.string(forKey: key) ?? current - defaults.set(current, forKey: key) - } - - private var showsWhatsNew: Bool { - !lastSeenAppVersion.isEmpty && lastSeenAppVersion != Bundle.main.appVersion + guard showsWhatsNew == nil else { return } + showsWhatsNew = WhatsNewNotice(bundle: .main, defaults: AppStorageEnvironment.shared.defaults).recordLaunch() } /// The badge follows entitlement and the support link follows whether anything has been paid, diff --git a/TablePro/Views/WhatsNew/WhatsNewNotice.swift b/TablePro/Views/WhatsNew/WhatsNewNotice.swift new file mode 100644 index 0000000000..015b08ea77 --- /dev/null +++ b/TablePro/Views/WhatsNew/WhatsNewNotice.swift @@ -0,0 +1,16 @@ +import Foundation + +internal struct WhatsNewNotice { + let bundle: Bundle + let defaults: UserDefaults + + func recordLaunch() -> Bool { + // A build without notes records nothing, so a Debug run never marks a release as seen. + guard WhatsNewContent.load(from: bundle) != nil else { return false } + let key = PreferenceKeys.lastSeenAppVersion.name + let current = bundle.appVersion + let previous = defaults.string(forKey: key) + defaults.set(current, forKey: key) + return previous.map { $0 != current } ?? false + } +} diff --git a/TablePro/Views/WhatsNew/WhatsNewView.swift b/TablePro/Views/WhatsNew/WhatsNewView.swift index 96446b2e3e..59cb699134 100644 --- a/TablePro/Views/WhatsNew/WhatsNewView.swift +++ b/TablePro/Views/WhatsNew/WhatsNewView.swift @@ -4,7 +4,7 @@ struct WhatsNewView: View { private let content: WhatsNewContent private let onViewFullChangelog: () -> Void - init(content: WhatsNewContent = .bundled, onViewFullChangelog: @escaping () -> Void) { + init(content: WhatsNewContent = .load() ?? .changelogPointer, onViewFullChangelog: @escaping () -> Void) { self.content = content self.onViewFullChangelog = onViewFullChangelog } @@ -63,13 +63,17 @@ struct WhatsNewContent { let title: String let paragraphs: [AttributedString] - static var bundled: WhatsNewContent { - guard let url = Bundle.main.url(forResource: "WhatsNew", withExtension: "md"), + static var changelogPointer: WhatsNewContent { + WhatsNewContent( + title: String(localized: "What's New"), + paragraphs: [AttributedString(String(localized: "Release notes are on the changelog."))] + ) + } + + static func load(from bundle: Bundle = .main) -> WhatsNewContent? { + guard let url = bundle.url(forResource: "WhatsNew", withExtension: "md"), let markdown = try? String(contentsOf: url, encoding: .utf8) else { - return WhatsNewContent( - title: String(localized: "What's New"), - paragraphs: [AttributedString(String(localized: "Release notes are on the changelog."))] - ) + return nil } return parse(markdown) } @@ -98,7 +102,7 @@ struct WhatsNewContent { } if body.isEmpty { - body = [AttributedString(String(localized: "Release notes are on the changelog."))] + body = changelogPointer.paragraphs } return WhatsNewContent(title: title, paragraphs: body) } diff --git a/TableProTests/Views/WhatsNewContentTests.swift b/TableProTests/Views/WhatsNewContentTests.swift index 7b6c7c1885..28d152fab4 100644 --- a/TableProTests/Views/WhatsNewContentTests.swift +++ b/TableProTests/Views/WhatsNewContentTests.swift @@ -54,13 +54,75 @@ struct WhatsNewContentTests { #expect(content.paragraphs.count == 1) } - /// The file is generated by scripts/generate-whats-new.sh and shipped in the bundle, so a - /// build where it is missing or unreadable must still open a usable window. - @Test("The bundled file parses into something displayable") - func bundledFileParses() { - let content = WhatsNewContent.bundled + @Test("A bundle with notes loads them") + func bundleWithNotesLoads() throws { + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n\nBeta updates.\n") { bundle in + let content = try #require(WhatsNewContent.load(from: bundle)) - #expect(!content.title.isEmpty) - #expect(!content.paragraphs.isEmpty) + #expect(content.title == "TablePro 0.82.0") + #expect(content.paragraphs.map { String($0.characters) } == ["Beta updates."]) + } + } + + @Test("A bundle without notes loads nothing") + func bundleWithoutNotesLoadsNothing() throws { + try withBundle(version: "0.0.0-dev", notes: nil) { bundle in + #expect(WhatsNewContent.load(from: bundle) == nil) + } + } + + @Test("Without notes the Welcome link stays hidden and the seen version is not written") + func noNotesNoLinkNoWrite() throws { + let suite = try EphemeralDefaults() + suite.defaults.set("0.81.0", forKey: PreferenceKeys.lastSeenAppVersion.name) + + try withBundle(version: "0.0.0-dev", notes: nil) { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(!showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.81.0") + } + } + + @Test("With notes a new version shows the Welcome link and is recorded as seen") + func notesShowLinkForNewVersion() throws { + let suite = try EphemeralDefaults() + suite.defaults.set("0.81.0", forKey: PreferenceKeys.lastSeenAppVersion.name) + + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n") { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.82.0") + } + } + + @Test("A first launch records the version without the Welcome link") + func firstLaunchHasNoLink() throws { + let suite = try EphemeralDefaults() + + try withBundle(version: "0.82.0", notes: "# TablePro 0.82.0\n") { bundle in + let showsLink = WhatsNewNotice(bundle: bundle, defaults: suite.defaults).recordLaunch() + + #expect(!showsLink) + #expect(suite.defaults.string(forKey: PreferenceKeys.lastSeenAppVersion.name) == "0.82.0") + } + } + + private func withBundle(version: String, notes: String?, _ body: (Bundle) throws -> Void) throws { + let root = FileManager.default.temporaryDirectory + .appendingPathComponent("WhatsNewContentTests-\(UUID().uuidString)", isDirectory: true) + defer { try? FileManager.default.removeItem(at: root) } + let bundleURL = root.appendingPathComponent("Probe.bundle", isDirectory: true) + let resources = bundleURL.appendingPathComponent("Contents/Resources", isDirectory: true) + try FileManager.default.createDirectory(at: resources, withIntermediateDirectories: true) + let info = ["CFBundleIdentifier": "com.TablePro.WhatsNewProbe", "CFBundleShortVersionString": version] + let plist = try PropertyListSerialization.data(fromPropertyList: info, format: .xml, options: 0) + try plist.write(to: bundleURL.appendingPathComponent("Contents/Info.plist")) + if let notes { + try Data(notes.utf8).write(to: resources.appendingPathComponent("WhatsNew.md")) + } + let bundle = try #require(Bundle(url: bundleURL)) + try body(bundle) } } diff --git a/project.yml b/project.yml index 4b1562726d..c386eb3bc6 100644 --- a/project.yml +++ b/project.yml @@ -31,6 +31,9 @@ settings: base: CLANG_ANALYZER_LOCALIZABILITY_NONLOCALIZED: YES DEAD_CODE_STRIPPING: YES + # Local placeholders. A release passes the tag's version and build: RELEASING.md, Versions. + MARKETING_VERSION: 0.0.0-dev + CURRENT_PROJECT_VERSION: 1 fileGroups: - Configs @@ -96,7 +99,6 @@ targetTemplates: settings: base: COMBINE_HIDPI_IMAGES: YES - CURRENT_PROJECT_VERSION: 1 DEVELOPMENT_TEAM: $(TABLEPRO_DEVELOPMENT_TEAM) ENABLE_HARDENED_RUNTIME: YES GENERATE_INFOPLIST_FILE: YES @@ -108,7 +110,6 @@ targetTemplates: LD_RUNPATH_SEARCH_PATHS: - $(inherited) - "@executable_path/../Frameworks" - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.${target_name} PRODUCT_NAME: $(TARGET_NAME) SKIP_INSTALL: YES @@ -251,36 +252,29 @@ targetTemplates: - package: TableProCore products: [TableProNumberFormatting, TableProTLSClientIdentity] - # The SAP HANA driver runs go-hdb in tablepro-hana-helper, which scripts/build-hana.sh builds - # and this phase copies into Contents/MacOS, where the plugin looks it up. A missing helper - # stops the build with "Build input file cannot be found" naming its path. The copy is signed - # in a directory from mktemp: the user-script sandbox lets the phase write its declared output - # and nothing beside it, codesign writes a temporary file next to whatever it signs, and it - # reads every parent of that file, which the sandbox refuses inside the build folders. - # scripts/release/build.sh passes OTHER_CODE_SIGN_FLAGS=--timestamp; other builds skip the timestamp server. + # The SAP HANA driver runs go-hdb in tablepro-hana-helper, which scripts/build-hana.sh builds. + # The plugin looks it up in Contents/MacOS, and the copy there signs it with the plugin's identity. HanaHelperEmbedding: - postBuildScripts: - - name: Embed tablepro-hana-helper + sources: + # A file type and optional, so a project generated before scripts/build-hana.sh runs still copies it. + - path: Native/HanaBridge/bin/tablepro-hana-helper + type: file + optional: true + buildPhase: + copyFiles: + destination: executables + attributes: [CodeSignOnCopy] + preBuildScripts: + - name: Check tablepro-hana-helper architectures inputFiles: - $(SRCROOT)/Native/HanaBridge/bin/tablepro-hana-helper - outputFiles: - - $(TARGET_BUILD_DIR)/$(EXECUTABLE_FOLDER_PATH)/tablepro-hana-helper + basedOnDependencyAnalysis: false script: | - set -eu - helper="$SCRIPT_INPUT_FILE_0" for arch in $ARCHS; do - if ! /usr/bin/lipo "$helper" -verify_arch "$arch"; then - echo "error: $helper has no $arch slice. Run scripts/build-hana.sh $arch, or both." >&2 - exit 1 - fi + /usr/bin/lipo "$SCRIPT_INPUT_FILE_0" -verify_arch "$arch" && continue + echo "error: $SCRIPT_INPUT_FILE_0 is missing or has no $arch slice. Run scripts/build-hana.sh $arch, or both." >&2 + exit 1 done - staging="$(mktemp -d)" - trap 'rm -rf "$staging"' EXIT - cp "$helper" "$staging/tablepro-hana-helper" - if [ "${CODE_SIGNING_ALLOWED:-NO}" = "YES" ]; then - /usr/bin/codesign --force --options runtime ${OTHER_CODE_SIGN_FLAGS:---timestamp=none} --sign "${EXPANDED_CODE_SIGN_IDENTITY:--}" "$staging/tablepro-hana-helper" - fi - cp -f "$staging/tablepro-hana-helper" "$SCRIPT_OUTPUT_FILE_0" targets: @@ -317,31 +311,13 @@ targets: # The MySQL plugin's socket key and path rules, so the app writes and checks the socket path # exactly as the driver reads it. - Plugins/MySQLDriverPlugin/MySQLLocalSocket.swift - configFiles: - Debug: Configs/Version.xcconfig - Release: Configs/Version.xcconfig dependencies: - target: TableProPluginKit embed: true codeSign: true removeHeaders: true - # Embedded rather than reached through an absolute rpath into the source tree. - # TablePro.debug.dylib and the MySQL, PostgreSQL and Redis plugins all link - # @rpath/libssl.3.dylib and @rpath/libcrypto.3.dylib, and both dylibs already carry an - # @rpath install name, so @executable_path/../Frameworks resolves them once they are here. - # Without this the app only launched from a checkout that had Libs/dylibs on disk, which is - # what made the sharded test jobs abort with SIGABRT before XCTest could connect (#2334). - # A release build was already self-contained: build-release.sh copies them in. - # - # codeSign matters as much as embed. create-openssl-dylibs.sh leaves them ad-hoc signed, and - # codesign refuses a bundle whose nested code carries a different identity from the app: - # "code object is not signed at all, in subcomponent libcrypto.3.dylib". A plain copy-files - # phase does not re-sign, so this built only until download-libs.sh next regenerated the - # dylibs. CI never saw it, because every CI build passes CODE_SIGNING_ALLOWED=NO. - # - # link: false keeps this to embedding and signing. A framework dependency links by default, - # which would put both into Link Binary With Libraries and change what the app links against; - # the app reaches them through @rpath from TablePro.debug.dylib and the plugins, as before. + # The app and the MySQL, PostgreSQL and Redis plugins load these through @rpath. Re-signed because + # create-openssl-dylibs.sh signs them ad hoc; link: false keeps them off the app's link line. - framework: Libs/dylibs/libssl.3.dylib embed: true codeSign: true @@ -417,6 +393,16 @@ targets: products: [TableProAnalytics, TableProConnectionLibrary, TableProGeometry, TableProGoogleCloud, TableProImport, TableProLogRedaction, TableProNumberFormatting, TableProRedisCore, TableProSQLGrammar, TableProSSHTransport, TableProSyncTransport, TableProTabular, TableProTabularIO, TableProTheme] - package: TableProMacCore products: [TableProAutocomplete, TableProMCP, TableProSSHConfig, TableProUtilities] + # Writes nothing for a version CHANGELOG.md has no section for, so a 0.0.0-dev build ships no notes. + postCompileScripts: + - name: Write WhatsNew.md + inputFiles: + - $(SRCROOT)/CHANGELOG.md + - $(SRCROOT)/scripts/release/notes.py + outputFiles: + - $(TARGET_BUILD_DIR)/$(UNLOCALIZED_RESOURCES_FOLDER_PATH)/WhatsNew.md + script: | + /usr/bin/python3 "$SCRIPT_INPUT_FILE_1" whats-new "$MARKETING_VERSION" --changelog "$SCRIPT_INPUT_FILE_0" --out "$SCRIPT_OUTPUT_FILE_0" settings: base: ASSETCATALOG_COMPILER_APPICON_NAME: AppIcon @@ -466,7 +452,6 @@ targets: Release: CODE_SIGN_ENTITLEMENTS: TablePro/TablePro.entitlements COPY_PHASE_STRIP: YES - DEPLOYMENT_POSTPROCESSING: YES # ─── Plugin ABI ─────────────────────────────────────────────────────────────── @@ -480,7 +465,6 @@ targets: base: # Swift Library Evolution. Removing this breaks every already-built plugin. BUILD_LIBRARY_FOR_DISTRIBUTION: YES - CURRENT_PROJECT_VERSION: 1 DEFINES_MODULE: YES DYLIB_COMPATIBILITY_VERSION: 1 DYLIB_CURRENT_VERSION: 1 @@ -490,7 +474,6 @@ targets: LD_RUNPATH_SEARCH_PATHS: - $(inherited) - "@executable_path/../Frameworks" - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProPluginKit PRODUCT_NAME: $(TARGET_NAME:c99extidentifier) SKIP_INSTALL: YES @@ -564,12 +547,10 @@ targets: products: [TableProMCPTestSupport] settings: base: - CURRENT_PROJECT_VERSION: 1 GENERATE_INFOPLIST_FILE: YES HEADER_SEARCH_PATHS: - $(inherited) - $(SRCROOT)/TablePro/Core/SSH/CLibSSH2/include - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProTests PRODUCT_NAME: $(TARGET_NAME) STRING_CATALOG_GENERATE_SYMBOLS: NO @@ -587,9 +568,7 @@ targets: - target: TablePro settings: base: - CURRENT_PROJECT_VERSION: 1 GENERATE_INFOPLIST_FILE: YES - MARKETING_VERSION: "1.0" PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.TableProUITests PRODUCT_NAME: $(TARGET_NAME) STRING_CATALOG_GENERATE_SYMBOLS: NO @@ -616,33 +595,18 @@ targets: templateAttributes: folder: SQLiteDriverPlugin principalClass: SQLitePlugin - settings: - base: - # Above plugin-sqlite-v1.0.1, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.2" ClickHouseDriver: templates: [DriverPlugin, ClickHouseLinkage] templateAttributes: folder: ClickHouseDriverPlugin principalClass: ClickHousePlugin - settings: - base: - # Above plugin-clickhouse-v1.0.19, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.20" RedisDriver: templates: [DriverPlugin, RedisLinkage] templateAttributes: folder: RedisDriverPlugin principalClass: RedisPlugin - settings: - base: - # Above plugin-redis-v1.0.19, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so the fixed built-in must replace a retained user copy. - MARKETING_VERSION: "1.0.20" CSVExport: templates: [DriverPlugin] @@ -738,10 +702,6 @@ targets: settings: base: PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.MQLExportPlugin - # Above plugin-mql-v1.0.7, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so at the template's 1.0 a retained user copy would keep - # writing server-provided names into exports unescaped. - MARKETING_VERSION: "1.0.8" SQLImport: templates: [DriverPlugin] @@ -751,10 +711,6 @@ targets: settings: base: PRODUCT_BUNDLE_IDENTIFIER: com.TablePro.SQLImportPlugin - # Above plugin-sqlimport-v1.0.5, the newest copy published to the registry. `selectWinners` - # compares versions numerically, so at the template's 1.0 a retained user copy would keep - # offering SQL on MongoDB. - MARKETING_VERSION: "1.0.6" JSONImport: templates: [DriverPlugin] diff --git a/scripts/check-pluginkit-abi.sh b/scripts/check-pluginkit-abi.sh index f1026ac933..cfd4cc50ad 100755 --- a/scripts/check-pluginkit-abi.sh +++ b/scripts/check-pluginkit-abi.sh @@ -6,16 +6,8 @@ set -euo pipefail # Builds TableProPluginKit at the current tree AND at a base ref with the SAME toolchain, then # diffs their public Swift interfaces. Comparing two builds from one compiler means Swift version # drift between a dev machine and CI cannot produce a false diff, so there is no committed baseline -# to keep in sync. A reported diff is a real ABI change to act on: -# -# Additive (a new requirement WITH a default implementation, a new field on a non-@frozen struct, -# a new case on a non-@frozen enum): bump currentPluginKitVersion in PluginManager.swift and -# TableProPluginKitVersion in every plugin Info.plist, leave minimumCompatiblePluginKitVersion -# alone, and re-publish nothing. Already-built plugins keep loading; the bump is what makes an -# older app refuse a plugin REBUILT against this API instead of failing its load. -# Breaking (changed/removed/renamed signature, a new case on a @frozen enum, a changed frozen -# layout): the same bumps, plus raise minimumCompatiblePluginKitVersion, then run -# scripts/release-all-plugins.sh . +# to keep in sync. A reported diff is a real ABI change; the rules printed at the end say what to +# raise in Configs/PluginKit.xcconfig. # # Both sides are generated from their project.yml first, so this regenerates # TablePro.xcodeproj in the working tree as well as in the base checkout. @@ -93,20 +85,16 @@ if diff -u "$work/base.txt" "$work/head.txt"; then exit 0 fi -# This is run by hand, per CLAUDE.md, not by a workflow. It used to have a branch that passed when -# ABI_ACKNOWLEDGED_ADDITIVE=1, set by a CI job that no longer exists, alongside instructions to add -# an `abi-additive` label and re-run. Nothing can set the variable and nothing reads the label, so -# both were telling the reader to do something that has no effect. cat <<'EOF' TableProPluginKit public ABI changed vs base (diff above). Decide additive vs breaking: Additive: a new requirement with a default, a reordering, or a field on a non-frozen transfer - struct. Bump currentPluginKitVersion and every plugin Info.plist - TableProPluginKitVersion; leave minimumCompatiblePluginKitVersion alone and re-publish - nothing. Shipped plugins keep loading; the bump makes an older app refuse a plugin - rebuilt against this API rather than failing its load. + struct. Raise TABLEPRO_PLUGINKIT_VERSION in Configs/PluginKit.xcconfig. Shipped plugins + keep loading; the bump makes an older app refuse a plugin rebuilt against this API + rather than failing its load. Breaking: a changed or removed requirement, a requirement without a default, a case on a @frozen - enum, or a frozen type's layout. The same bumps, plus raise - minimumCompatiblePluginKitVersion, then run scripts/release-all-plugins.sh . + enum, or a frozen type's layout. Raise TABLEPRO_PLUGINKIT_VERSION and + TABLEPRO_PLUGINKIT_MINIMUM_VERSION. + Registry plugins ship with the next app release, built against the new kit. EOF exit 1 diff --git a/scripts/generate-whats-new.sh b/scripts/generate-whats-new.sh deleted file mode 100755 index fd22089612..0000000000 --- a/scripts/generate-whats-new.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Writes TablePro/Resources/WhatsNew.md from a version's CHANGELOG lead block, which is the same -# text the Sparkle feed carries. Generated rather than hand-written so the window and the update -# dialog cannot disagree, and so shipping a release does not depend on anyone remembering to edit -# a second file. -# -# Run from the release runbook, after finalizing CHANGELOG.md and before committing. -# -# Usage: scripts/generate-whats-new.sh - -VERSION="${1:?Usage: generate-whats-new.sh }" -REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" -OUTPUT="$REPO_ROOT/TablePro/Resources/WhatsNew.md" - -cd "$REPO_ROOT" - -TEMP=$(mktemp) -trap 'rm -f "$TEMP"' EXIT - -# --require-lead-block, not the feed's fallback. Without a lead block the fallback is the whole -# section, and 0.74.0's is 270 entries: not something to compile into the app bundle. A release -# with no highlights ships a short pointer instead. -if bash scripts/ci/extract-release-notes.sh "$VERSION" \ - --highlights-only --require-lead-block --out "$TEMP" >/dev/null 2>&1; then - { - printf '# TablePro %s\n\n' "$VERSION" - cat "$TEMP" - } > "$OUTPUT" -else - echo "No lead block for $VERSION; writing a pointer to the changelog instead" - printf '# TablePro %s\n\nThis release is fixes and refinements. Every entry is in the changelog.\n' \ - "$VERSION" > "$OUTPUT" -fi - -echo "Wrote $OUTPUT:" -cat "$OUTPUT" diff --git a/scripts/install-plugin-dev.sh b/scripts/install-plugin-dev.sh index 1056c7f18d..2517927ade 100755 --- a/scripts/install-plugin-dev.sh +++ b/scripts/install-plugin-dev.sh @@ -22,9 +22,7 @@ TARGET="${1:-ElasticsearchDriverPlugin}" BUNDLE="${TARGET}.tableplugin" DEST_DIR="${HOME}/Library/Application Support/TablePro/Plugins" -# The newest build, not whichever one the filesystem happened to list first. This repo carries -# several worktrees and each gets its own DerivedData, so `head -n 1` routinely installed a stale -# bundle from somebody else's checkout and the mismatch only showed up as a load failure later. +# Each worktree has its own DerivedData, so take the newest build. CANDIDATES="$(find "${HOME}/Library/Developer/Xcode/DerivedData/TablePro-"*/Build/Products/Debug \ -maxdepth 1 -name "${BUNDLE}" -print0 2>/dev/null | xargs -0 stat -f '%m %N' 2>/dev/null | sort -rn)" SRC="$(printf '%s' "${CANDIDATES}" | head -n 1 | cut -d' ' -f2-)" From aee588f5350793d2ff2fb353c033a0dd0b3e9d98 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 10:18:42 +0700 Subject: [PATCH 07/10] ci(release): release from tags through release.yml --- .claude/rules/ci.md | 46 +-- .claude/rules/data-sync-storage.md | 1 + .claude/rules/native-bridges.md | 2 +- .claude/rules/plugin-system.md | 17 +- .claude/rules/theme.md | 2 +- .claude/skills/fix-issue/SKILL.md | 6 +- .claude/skills/fix-issue/references/lanes.md | 2 +- .../fix-issue/references/platform-apple.md | 1 + .claude/skills/release/SKILL.md | 282 ++++-------------- .github/ISSUE_TEMPLATE/bug_report.yml | 6 +- .github/pull_request_template.md | 4 +- .github/workflows/repo-hygiene.yml | 2 + AGENTS.md | 12 +- CONTRIBUTING.md | 10 +- Plugins/HanaDriverPlugin/README.md | 2 +- RELEASING.md | 272 ++++++++--------- 16 files changed, 238 insertions(+), 429 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 24d8319c79..f3317ba2f7 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -2,33 +2,40 @@ paths: - ".github/workflows/**/*" - ".github/actions/**/*" - - ".github/scripts/**/*" - ".github/actionlint.yaml" - ".github/*baseline*" - "scripts/ci/**/*" + - "scripts/release/**/*" - ".swiftlint.yml" - "**/.swiftlint.yml" --- # CI -The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. The release and plugin builds: `RELEASING.md`. +The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. Release tags: `RELEASING.md`. -## Rulesets and required checks +## Rulesets +| Ruleset | Refs | Rules | Bypass | +|---|---|---|---| +| 25308039 "tags never move" | tags `v*`, `libs-*` | update, deletion | none | +| 23959123 "release tags are created by admins" | tags `v*`, `libs-*`, `plugin-*` | creation | admins | +| 25308049 "release branches are created by admins" | `release/*` | creation | admins | +| 23957966 "main and release branches require green gates" | default branch, `release/*` | the five required checks, `do_not_enforce_on_create` | none | +| 25129799 "branches keep their history" | default branch, `release/*`, `feeds` | deletion, non-fast-forward | none | + +- `feeds` takes no required checks: CI pushes it with `GITHUB_TOKEN`, and the signed feed proves its content. - There are no gate jobs. A test job runs on `!cancelled() && needs.changes.outputs.run != 'false'`. A job its `if:` skips reports success, and a failed changes job runs the suite rather than skipping it. - `Changes (macOS)`, `Changes (iOS)` and `Changes (Release)` are not required checks. - A PR need not be up to date with main to merge (`strict_required_status_checks_policy: false`). -- A second ruleset blocks force-push and deletion of `main`, with no bypass. - A required check's name is its job's `name:`. Renaming a job changes the ruleset in the same change, and every open PR then merges main in before its checks report the new name. ## Runners and toolchain -- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. `build-plugin.yml` picks its runner from the ref it builds (`RELEASING.md`). +- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image. actionlint learns the label from `.github/actionlint.yaml`. A stable tag builds only on a GA Xcode: `setup-xcode` with `require-release` fails on a beta or Release Candidate. - The Xcode version is written once, in `.github/actions/setup-xcode/action.yml`, which runs `xcode-select --switch` and prints `xcodebuild -version`. A workflow never selects Xcode another way. - XcodeGen comes from `.github/actions/setup-xcodegen`, pinned by version and SHA-256. - `scripts/release/feed.py` runs Sparkle's `generate_appcast` and `sign_update` from `SPARKLE_BIN`, which `.github/actions/setup-sparkle` sets. The action reads the version and revision from the app's `Package.resolved` (`project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. -- The tag release (`build.yml`) signs the feed with the copy `scripts/ci/sign-and-appcast.sh` downloads, pinned by its own `SPARKLE_VERSION` and `SPARKLE_SHA256`. No check compares them with `project.yml`: a Sparkle bump changes all three. - The compilation cache is off on CI (`COMPILATION_CACHE_ENABLE_CACHING=NO`); `Configs/Base.xcconfig` turns it on locally. ## macOS Tests (`macos-tests.yml`) @@ -55,10 +62,16 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Push to main or `release/**`, and dispatch; never a PR. Each commit has its own concurrency group and is never cancelled. Not a required check. - `Changes (Release)` outputs `run` (detector `release`). `Release Build` runs on `xcode-27` with a 150-minute limit. - Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both`, `build-hana.sh both` and create-dmg. -- `scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set (the `.github/plugin-registry.json` entries not bundled) to `build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. +- `scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. - `scripts/release/package.sh app 0.0.0-dev --unsigned` writes the release assets: the DMG, the `.tar.xz` (extracted and compared with the app) and the dSYMs zip; `package.sh plugins` writes one zip per plugin and the plugin dSYMs zip. - The plugins build runs after an app failure too. The xcodebuild logs in `build/release/logs` upload only on failure. +## Release, Withdraw and Pages + +- A `v*` tag runs only `release.yml`; its jobs: `RELEASING.md`, What a tag runs. `withdraw.yml` runs only by dispatch, and `pages.yml` runs from both or by dispatch. Their environments allow only `v*` tags, so a dispatch names one with `--ref`. +- Jobs read the tag, version, build and channel only from `verify` outputs passed through `env:`, never through `${{ }}` inside `run:`. +- The default permission is `contents: read`, and each job lists every scope it needs. A job that calls `pages.yml` grants `pages: write` and `id-token: write`, since a called workflow can only lower what it receives. + ## Summaries (`scripts/ci/summarize-xcresult.sh`) - Runs `scripts/lib/xcresult-cases.py summary`. `verify.sh` reads its verdicts through the same parser, and `scripts/lib/test_xcresult_cases.py` tests it in Repo Hygiene. @@ -77,7 +90,7 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ ## PR title (`pr-title.yml`) - Runs on opened, edited, reopened and synchronize, so every head commit gets the check. -- Runs `scripts/ci/check-pr-title.py`, then `scripts/check-banned-words.sh` on the PR's added lines and on its body. The banned-words step runs both and exits with the worse result. +- Runs `scripts/ci/check-pr-title.py`, then `scripts/ci/check-changelog.py --pr` with `GH_TOKEN` (`pull-requests: read`), then `scripts/check-banned-words.sh` on the PR's added lines and on its body. The banned-words step runs both and exits with the worse result. ## Repo Hygiene (`repo-hygiene.yml`) @@ -85,14 +98,15 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - Steps, in order: - actionlint, which also shellchecks every inline `run:` block; - `shellcheck --severity=warning` on every `*.sh` under `scripts` and `.claude/skills`; - - the tooling tests: `scripts/ci/test_*.py`, `scripts/lib/test_*.py`, `scripts/test_localization.py`, `.github/scripts/test_update_registry.py` and two shell tests; + - the tooling tests: `scripts/ci/test_*.py`, `scripts/lib/test_*.py`, `scripts/test_localization.py` and `scripts/ci/test_libs_release.sh`; - `scripts/release/test_*.py` on Python 3.9 from `actions/setup-python`, in a venv that installs `scripts/release/requirements.txt` with `pip --require-hashes`; + - in the same venv, `check-changelog.py` and `towncrier build --draft --version 0.0.0`, which fails on a fragment name it cannot parse; - `check-lib-sourcing.py`, `check-plugin-manifest.py`, `check-deployment-target.sh`, `check-ios-shared-isolation.py`; - `localization.py plugins` and `plugins --target ios`; - `check-string-catalogs.py`, `check-license-inventory.py`, `check-entitlements.py`, `check-test-plans.py`; - `check-pluginkit-bump-cadence.py`; - `scripts/audit-refactor-health.sh --check`, `check-hub-ratchet.py`; - - the Go checks on the HANA helper, when the `hana` detector is true. + - the Go checks on the HANA helper, `govulncheck` among them, when the `hana` detector is true. New advisories between changes arrive as Dependabot alerts. - `audit-refactor-health.sh --check` fails a file defining a production `DatabaseType` outside `Packages/TableProCore/Sources/TableProCoreTypes/DatabaseType.swift`, unless `.github/duplicate-contract-baseline.txt` lists its path. Moving a listed file moves its baseline path in the same commit. Never add an entry to let a new copy pass. - `check-hub-ratchet.py` fails when an app hub grows past `.github/hub-ratchet-baseline.json`. `--update` only lowers a value, so growing one is a hand edit a reviewer sees. @@ -106,22 +120,20 @@ The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/ - `oracle` watches `Packages/TableProOracle/`; `hana` watches `Native/HanaBridge/`. Each suite also watches its own workflow. - `release` watches what the Release build reads: the app, plugin, package, native, config and libs folders, `project.yml`, the app's `Package.resolved`, `.github/plugin-registry.json`, `scripts/release/`, the build and CI scripts it runs and both setup actions. It skips `Plugins/*Tests/` and `Packages/*/Tests/`. - A path a suite newly reads joins that suite's pattern in the same change, or the suite skips on it. -- Release-only files never trigger a suite: `CHANGELOG.md`, `appcast.xml`, `TablePro/Resources/WhatsNew.md`, `.github/release-flags.json`, and `Configs/Version.xcconfig` while its diff touches only the two version keys. -- Every file the release commit stages (`.claude/skills/release/SKILL.md`, Commit, tag, push) is on that list; `appcast.xml` is `build.yml`'s own push. -- A release push that touches anything else runs the suites, and the tag waits for them. +- `CHANGELOG.md` and `changelog.d/` are on no suite's pattern: the What's New phase writes nothing at `0.0.0-dev`, and Repo Hygiene renders every fragment and section instead. ## What runs where - A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title, plus Release Build; never cancelled, `AllPlugins` always, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or release-only push skips the suites through the detector. -- A `v*` tag runs only `build.yml`; a `plugin-*-v*` tag runs only `build-plugin.yml` (`RELEASING.md`). The test workflows never run on a tag. -- `workflow_dispatch` exists on every workflow except `pr-title.yml`. +- A push to main or `release/**`: the same minus PR title, plus Release Build; never cancelled, `AllPlugins` always, the SPM cache saved on a miss. Every code push runs both plans in full; a docs-only or changelog-only push skips the suites through the detector. +- A `v*` tag runs only `release.yml`. The test workflows never run on a tag. +- `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. - `build-linux.yml` is on `main` only for its Monday schedule, which GitHub runs from the default branch. Until the Linux app's directory lands on `main`, every build job skips and `weekly-dispatch` starts the workflow on the branch the `LINUX_BRANCH` variable names (the linux branch when unset), whose testing guide describes the jobs. ## Caches and runner budget - Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `build.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other tags. +- `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. - `release-build.yml` restores it and leaves the save to macOS Tests. - There is no other cache: no libs or DerivedData cache, and none in the iOS, plugin or Ubuntu workflows. - CI fits the GitHub Free and Pro plans: no paid larger runner, no self-hosted Mac. At most 5 macOS jobs run at once, and main runs queue behind PR runs for those slots. diff --git a/.claude/rules/data-sync-storage.md b/.claude/rules/data-sync-storage.md index f6a31d2270..d9de4aa06b 100644 --- a/.claude/rules/data-sync-storage.md +++ b/.claude/rules/data-sync-storage.md @@ -14,6 +14,7 @@ paths: This path holds user data. A silent wrong answer here looks exactly like an empty database, so prove late-completion and cancellation behavior with a test. - **A synced CKRecord type or field reaches CloudKit Production before anything writes it.** Both apps pin the container to Production, where nothing is auto-created, and CloudKit rejects a whole record that carries an undeclared field. Write fields only through `record.fields(SomeSyncField.self)`, never `record["..."]`. To ship one: add it in Development, deploy to Production in the Console, run `scripts/export-cloudkit-schema.sh`, commit `CloudKit/production-schema.ckdb`, then add it to the matching `verifiedInProduction`. `ProductionSchemaParityTests` checks code against the snapshot in both directions, not against the server, so never edit the snapshot by hand: `scripts/export-cloudkit-schema.sh --check` compares it with Production. +- **A storage format stays readable by the previous stable.** Sparkle never downgrades, and a beta user can turn betas off (`RELEASING.md`, Channels). - **Persist, then notify.** `SyncChangeTracker.markDeleted()` runs after the storage write, because its notification can start a sync that re-uploads the record from the stale file. - **Concurrent schema loads await the one in-flight `loadTask`** in `SQLSchemaProvider`; never guard with a flag that returns without data. - **Every metadata read goes through `DatabaseManager.withMetadataDriver`**, never `MetadataConnectionPool.shared` directly, so an embedded engine with `supportsConnectionPooling = false` (DuckDB, PGlite) stays on its session driver; a second connection to it is a different, empty database. diff --git a/.claude/rules/native-bridges.md b/.claude/rules/native-bridges.md index 9345067f85..a77c344421 100644 --- a/.claude/rules/native-bridges.md +++ b/.claude/rules/native-bridges.md @@ -12,4 +12,4 @@ paths: - **First-party bridge code lives in `Native/Bridge`**, built by `scripts/build-.sh`, with output in a gitignored folder beside the source, never in `Libs/`. - **Rust may run in the app process; Go may not.** The Dameng staticlib is built with `panic = "unwind"`, and every export that does driver work wraps it in `catch_unwind`; a new export that does work takes the same guard. Go's `recover()` cannot catch a panic on a goroutine go-hdb starts, so HANA runs out of process as `tablepro-hana-helper`, one per driver session. - **Toolchains and upstream code are pinned**: `rust-toolchain.toml`, `Cargo.lock` with `--locked`, the `toolchain` line of `go.mod`, `go.sum`. A protocol change for Dameng lands on the [TableProApp/rust-dameng](https://github.com/TableProApp/rust-dameng) fork first, then the pinned `rev` moves. -- **Every nested Mach-O is signed with Developer ID, the hardened runtime and a timestamp** before the main binary; `scripts/build-plugin.sh` does it and checks it with `codesign -dvvv`. +- **Every nested Mach-O is signed with Developer ID, the hardened runtime and a timestamp** before the main binary. The `HanaHelperEmbedding` copy phase in `project.yml` signs the HANA helper with the plugin's identity, and `scripts/release/build.sh plugins` checks every executable with `codesign -dvvv`. diff --git a/.claude/rules/plugin-system.md b/.claude/rules/plugin-system.md index 0d52c96e8a..5dc3d157a4 100644 --- a/.claude/rules/plugin-system.md +++ b/.claude/rules/plugin-system.md @@ -8,7 +8,7 @@ paths: - "Packages/*/Package.swift" - "project.yml" - "TableProMobile/project.yml" - - ".github/workflows/build-plugin.yml" + - "Configs/PluginKit.xcconfig" - ".github/plugin-registry.json" - "scripts/*plugin*" --- @@ -24,10 +24,10 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke - Add a requirement only with a default implementation, then bridge it in `PluginDriverAdapter`. - Additive: a new requirement with a default, a new field on a non-`@frozen` struct, a new case on a non-`@frozen` enum, a reordering. - Breaking: a changed, removed or renamed requirement or signature, a requirement without a default, a new case on a `@frozen` enum, a change to a frozen type's layout. -- Every ABI change bumps `TABLEPRO_PLUGINKIT_VERSION` in `Configs/PluginKit.xcconfig`, at most once per release cycle: later changes in the cycle reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. +- Every ABI change bumps `TABLEPRO_PLUGINKIT_VERSION` in `Configs/PluginKit.xcconfig`, at most one past the newest `vX.Y.Z` tag: later changes reuse the pending number. `scripts/ci/check-pluginkit-bump-cadence.py` checks it in Repo Hygiene. A `release/*` line never bumps it. - The build stamps that value into `TableProPluginKitVersion` of the app and of every plugin. A plugin's `Info.plist` holds `TABLEPRO_PLUGINKIT_VERSION` and is preprocessed, because every shipped app reads the key with `as? Int`; the source file therefore fails `plutil -lint`. -- An additive change re-publishes nothing. Built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. -- A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`, and every registry plugin is re-released with `scripts/release-all-plugins.sh ` before the app tag, because the release's `Registry Readiness` job fails while a registry driver has no binary in the app's kit range (`RELEASING.md`, The release pipeline). +- After an additive change, built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. +- A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`. Every tag ships every registry plugin built against its own kit, so the release that carries the change carries plugins in its range. - Check with `verify.sh abi ` (`scripts/check-pluginkit-abi.sh`). It builds PluginKit at HEAD and at the base with the same toolchain and diffs the public `.swiftinterface`. A base with no interface passes; a diff prints the additive and breaking rules and exits 1. - It refuses uncommitted changes, so run it after the commit and before the push. No workflow runs it. - The diff is textual, so a `-` line can be a declaration that moved. Only a removed line with no matching added line, or a requirement added without a default, is breaking. This lists the removed lines that do not come back: @@ -65,12 +65,11 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke ## Structure - PluginKit-free logic a plugin shares with the iOS app lives in a TableProCore product, such as `TableProMySQLCore`, `TableProPostgreSQLCore` or `TableProRedisCore`. -- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, an entry in `.github/plugin-registry.json` keyed by its tag slug, a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a CHANGELOG entry. +- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, an entry in `.github/plugin-registry.json` keyed by its target, with `name` and `iconName`, a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a changelog fragment. - A new engine ships as a registry plugin (`AGENTS.md`, Scope). - Where a plugin's tests go, and how a test target is added in `project.yml`: `.claude/rules/tests.md`, Where a test goes and Adding a test target. -## Tags and shipped apps +## Releases -- Plugin tags are `plugin--v`, the slug a key in `.github/plugin-registry.json`. Check `git tag -l "plugin-*"` before creating one. -- A driver fix reaches users of a shipped app through `scripts/release-plugin-for-shipped-app.sh [appTag]`. Never relabel a binary built from `main` with an older kit version. -- How a plugin tag builds and publishes: `RELEASING.md`, Plugins. +- Registry plugins have no tags of their own: each app tag builds and publishes all of them at the app's version (`RELEASING.md`, Plugins). +- A driver fix reaches stable users through a patch on `release/`. diff --git a/.claude/rules/theme.md b/.claude/rules/theme.md index cd68b0cde5..2ed78301bb 100644 --- a/.claude/rules/theme.md +++ b/.claude/rules/theme.md @@ -17,7 +17,7 @@ paths: - The editor tab strip track draws `secondarySystemFill` over a themed backdrop on macOS 26 and later, where the track sits on the theme's window color, and its unthemed fill otherwise. Never a palette color. - **A code field takes the editor theme wherever it appears.** Startup Commands, Pre-Connect Script, a favorite's query, Rewind, Agent SQL, the deep-link Startup SQL and the tunnel command preview wear `editor.*`; the form or sheet around them stays system. - **The slot family follows what the surface shows.** Code, values, value trees and wells: `editor.*`, with tree tones from `editor.syntax.*`. Records, schema objects, diagram nodes, chart plots and states that stand in for a grid: `dataGrid.*`. Diff rows: `dataGrid.inserted`, `modified` and `deleted` over the editor background and text. A region behind panes: `window.background`. Identity, severity and status hues stay system. -- **Every site passes what it draws natively.** A themed call site names a slot and, as its fallback, the exact system color or style it drew before, so while the active theme leaves the slot out the pixels are an unthemed app's. A default changes only on purpose, named in the PR and the CHANGELOG. +- **Every site passes what it draws natively.** A themed call site names a slot and, as its fallback, the exact system color or style it drew before, so while the active theme leaves the slot out the pixels are an unthemed app's. A default changes only on purpose, named in the PR and in a changelog fragment. ## The model: `TableProTheme` diff --git a/.claude/skills/fix-issue/SKILL.md b/.claude/skills/fix-issue/SKILL.md index a32517cf71..7074c58ed2 100644 --- a/.claude/skills/fix-issue/SKILL.md +++ b/.claude/skills/fix-issue/SKILL.md @@ -41,7 +41,7 @@ Pick the platform from the report: the issue's Platform field, a `platform-*` la Invoking the skill authorizes the whole run: investigate, branch, implement, verify, commit, push, open the PR and keep it green. There is no approval gate. Ask only when the expected behavior has two reasonable readings or a finding needs a product decision: once, recommendation first, and keep working on what does not depend on the answer. Never ask about the environment (a peer's build, a wedged test runner, a full disk): handle it and mention it in the report. -Never, even in an authorized run: force-push, rewrite published history, touch release tags, publish plugins or libraries, deploy a CloudKit schema, or discard changes you did not make. When the fix needs one of these, build everything else and name what is left for the user. +Never, even in an authorized run: force-push, rewrite published history, touch release tags or `release/*` branches, publish plugins or libraries, deploy a CloudKit schema, or discard changes you did not make. When the fix needs one of these, build everything else and name what is left for the user. ## Phase 0: Intake @@ -79,7 +79,7 @@ Write `/.analysis//plan.md`, under 80 lines, opening with a progress - the design: the API or pattern, its ownership boundary, the design guideline; - the scope as a checklist: files in order, blast-radius cases, each verified related defect with `file:line`; - the tests that fail before the fix, in the targets the profile's Done section says to check first; -- the CHANGELOG, docs, localization and screenshots the change needs. +- the changelog fragment, docs, localization and screenshots the change needs. **Rules read.** List the files `plan.md` touches. Read `/AGENTS.md`, then every `/.claude/rules/*.md` whose row in its Area rules table matches one of them, and name them on a `Rules read:` line in `plan.md`. A file added later gets the same check before you edit it. The tree's copies are the ones that apply; a harness may have loaded another checkout's. @@ -90,7 +90,7 @@ In Full mode the `critic` lane attacks the plan; fold in what survives an eviden - When the change is visible on screen, build the fresh tree and capture the "before" shots first. - Follow the plan's order and do the refactor it calls for. Apply the guide's per-change checklist while you write, not afterwards. - **A large change may be split across implementers**: when the plan spans areas that share no files, give up to six subagents a disjoint file list each in `plan.md`. Each prompt names the tree, `plan.md` and the exact files the implementer owns, and says to edit nothing else, build nothing, and finish with the files it changed. You read every diff, integrate and verify. The harness file says which agent runs them. -- Competitors are research only: never name another client in code, commits, the PR, CHANGELOG or docs. +- Competitors are research only: never name another client in code, commits, the PR, a changelog fragment or docs. ## Phase 4: Verify diff --git a/.claude/skills/fix-issue/references/lanes.md b/.claude/skills/fix-issue/references/lanes.md index 2ca17404be..bf358364f5 100644 --- a/.claude/skills/fix-issue/references/lanes.md +++ b/.claude/skills/fix-issue/references/lanes.md @@ -56,7 +56,7 @@ For each one, read the cited code and its surroundings, and check that nothing u Attack `plan.md`. Report weaknesses, not a summary; a sound part gets one line. Cover three lenses: 1. **Patterns**: where the design fights an existing pattern, helper or convention in the repo (cite it), or breaks an invariant in the guide or its area rules. -2. **Scope**: callers that break, state or persistence that goes stale, and inputs the plan skips (empty, null, duplicate names, sentinel values, very large results, cancellation). Also a CHANGELOG or docs claim that says more than the change does. +2. **Scope**: callers that break, state or persistence that goes stale, and inputs the plan skips (empty, null, duplicate names, sentinel values, very large results, cancellation). Also a changelog fragment or docs claim that says more than the change does. 3. **Decision**: whether refactor-or-patch is right, whether a better documented API exists (name it), and whether the rejected alternative was rejected for a real reason. Each objection gets a severity (`blocking`, `material` or `minor`), evidence (`file:line`, an API ground-truth line or measured output, since reasoning alone is not evidence), and the smallest change to the plan that answers it. Read the files you cite: a confident wrong objection costs more than a missed one. diff --git a/.claude/skills/fix-issue/references/platform-apple.md b/.claude/skills/fix-issue/references/platform-apple.md index 4461e56466..40ab83906b 100644 --- a/.claude/skills/fix-issue/references/platform-apple.md +++ b/.claude/skills/fix-issue/references/platform-apple.md @@ -57,4 +57,5 @@ The macOS app and the iOS app, with everything they build from. Every profile ha ## Ship - Stage a string catalog only for the keys you added. +- A fix for a bug on the current beta or stable gets the label `backport release/`. - A probe worth keeping goes under `scripts/probes/`. diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index b1505356df..e1ab0c8136 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -1,261 +1,87 @@ --- name: release description: > - Ships a TablePro release end to end: bumps Configs/Version.xcconfig, finalizes CHANGELOG.md, - commits, tags, pushes and releases plugins. Use whenever the user says "release", "bump version", - "ship version", "tag a release", "cut a release", or gives a version number ("/release 0.67.0", - "/release plugin-oracle 1.0.0"). Not for the blog post, newsletter or X posts: those are the - announce skill in TableProApp/web. + Cuts a TablePro macOS release: a beta, a stable or a patch. Picks fixes onto a release/X.Y line, + curates the changelog fragments and the lead, decides whether a release is critical, opens the + release PR, hands the maintainer the tag command, watches the run and forward-ports the release. + Use whenever the user says "release", "cut a beta", "ship version", "tag a release", "promote the + beta", "backport", or gives a version number ("/release 0.82.0", "/release 0.82.0-beta.2"). Not for + the blog post, newsletter or X posts (the announce skill in TableProApp/web), and not for the iOS app. --- # Release -- You never push to this repository's `main`. Every push made with the release key (the release - commit, a withdrawal) is the user's to run in their own terminal, because that key never enters - ssh-agent (`RELEASING.md`, Keys and rulesets). Wait for them to confirm it landed. -- Tags are always annotated, `-a -m`: the owner's git has `tag.gpgSign=true`, and a plain - `git tag ` fails with "no tag message?". -- The release commit holds exactly the four files that Commit, tag, push stages. -- Once a release run is past **Publish the release**, only **Re-run failed jobs** - (`RELEASING.md`, When a release run fails). +`RELEASING.md` is the runbook and holds every command. This skill holds the judgment: each section is a decision, then names the runbook section that carries it out. -Run the stages in order, starting one only when the one before it is real: 1 release (always), 2 -plugins (when they changed). Announcing it is the `announce` skill in TableProApp/web -(`../tablepro-web`). +## 1. Ground rules -House rules relied on, not repeated: `.claude/rules/changelog.md`, `.claude/rules/plugin-system.md`, -the commit format in the top comment of `.github/pull_request_template.md`, and `RELEASING.md` for -the pipeline. +- Never push a tag, never create or delete a `release/*` branch, never push to `main` or `release/*`. The maintainer runs every block you hand over, in their own terminal, and tells you when it landed. +- A tag never moves. A fix after a tag is the next number. +- Release PRs and backport PRs go through the five required checks like any other PR. -## Stage 1: Release +## 2. Read the state -### Pre-flight +Before any decision, gather: -Check each. If one fails, stop and say what is wrong. +- the tags, newest first: `git -c versionsort.suffix=-beta tag -l 'v*' --sort=-v:refname | head`; +- the live lines: `git ls-remote --heads origin 'release/*'`; +- open regressions: `gh issue list --label regression --state open`; +- open Dependabot alerts: `gh api repos/TableProApp/TablePro/dependabot/alerts -q '.[] | select(.state == "open") | .security_advisory.summary'`; +- main's push runs: `gh run list --branch main --event push --limit 10`, with macOS Tests, iOS and Package Tests and Release Build green on the commit you would cut; +- whether `main` has the last stable's `## [X.Y.Z]` heading. If not, the forward-port is missing and the next tag will refuse to build: do that first (section 9). -1. The version is semver (`X.Y.Z`; a suffix like `-beta.1` is fine), newer than `MARKETING_VERSION` - in `Configs/Version.xcconfig`, and its tag is free (`git tag -l "v"`). Ask if missing. -2. The tree is clean (`git status --porcelain`), or the user says the changes belong in the - release. `[Unreleased]` has entries; if not, say the release has no notes. -3. `swiftlint lint --strict` is clean: the tag's checks do not cover SwiftLint. Fix findings - through their own pull request. -4. If `Plugins/TableProPluginKit/` changed since the last tag, do Stage 2, PluginKit, now, before - the tag. -5. `scripts/export-cloudkit-schema.sh --check` passes; it needs the cktool management token on this - Mac. A field in the snapshot but not in Production makes CloudKit reject every record carrying - it, so the user deploys it in CloudKit Console before the tag. -6. After `git fetch origin main`, you are on `main` and it equals `origin/main`. Report both suite - verdicts and stop when either is a failure. What the tag's `Tested on main` job checks on top of - this: `RELEASING.md`, The release pipeline. +## 3. Decide what ships - ```bash - for pair in 'macos-tests.yml|macOS Tests' 'ios-tests.yml|iOS and Package Tests'; do - for id in $(gh run list --workflow "${pair%%|*}" --branch main --event push --limit 20 \ - --json databaseId -q '.[].databaseId'); do - verdict=$(gh run view "$id" --json headSha,jobs \ - -q ".headSha[0:9] + \" \" + ([.jobs[] | select(.name == \"${pair#*|}\") | .conclusion][0] // \"\")") - case "$verdict" in *' ' | *skipped) continue ;; esac - echo "${pair#*|}: $verdict (run $id)" - break - done - done - ``` +- **beta.1 of X.Y**: `main` has features since the last X.Y.0. +- **beta.N**: the line has picks since its last beta. +- **Promote to X.Y.0**: the newest beta has no open `regression` issue. +- **Patch X.Y.Z**: the line has fixes only since its last stable. +- **X.0.0**: ask the maintainer. -7. The release key exists: `test -f ~/.ssh/tablepro_release`. If not, stop and give the user both - commands to run, the first with a passphrase: +Say which and why in one line, then wait for the maintainer to agree. Run: `RELEASING.md`, Cutting a beta, or Cutting a stable or patch. - ```bash - ssh-keygen -t ed25519 -f ~/.ssh/tablepro_release -C "release commit push" - gh repo deploy-key add ~/.ssh/tablepro_release.pub --allow-write --title "Release commit push" -R TableProApp/TablePro - ``` +## 4. Storage check -### Credit the entries +Before beta.1 and before a promotion, list the commits that touch user data since the previous stable: `git log --oneline v.. --` followed by every `paths:` entry of `.claude/rules/data-sync-storage.md`. Read each one for a changed storage or sync format. Sparkle never downgrades, so a format the previous stable cannot read blocks the release until it is fixed on `main`. Run: `RELEASING.md`, Channels. -Do this before rewording anything: the script reads each entry's commit with `git blame`, and a line -edited in the working tree blames to nothing. +## 5. Critical or not -```bash -python3 scripts/ci/changelog_credits.py --dry-run | tail -5 -python3 scripts/ci/changelog_credits.py -``` +Critical is only a security fix or data loss, never a beta. Name the fix and the reason, and let the maintainer confirm. A critical stable takes `--critical` on `notes.py assemble`. Run: `RELEASING.md`, Notes. -An outside contributor's entry ends `(#2905 by @digows)`, a maintainer's ends `(#2905)`. The output -ends with every contributor and every entry left bare. A bare entry came from a direct push: credit -it from `git log` or leave it, and never guess a handle. Blame credits the last commit to touch a -line, so check the contributors against `gh pr list --state merged --search "merged:>=" --json author` -and restore any handle a later rewording took. The script is idempotent. +## 6. Backport candidates -### Tighten the entries +- Candidates: merged `main` PRs labeled `backport release/`, plus `fix` PRs for a bug the line has. +- Never a `feat`, and after `X.(Y+1).0` ships, only security or data-loss fixes. +- Already picked: `gh pr list --base release/ --state merged --limit 200`. A pick's title keeps the original subject and its ` (#N)`. +- One fix per pick PR, from `backport/-`. A conflict that needs more than a mechanical resolution goes back to the maintainer. -Entries arrive one PR at a time and drift long. Measure them without their credit, then look for -the repair-story tells: +Run: `RELEASING.md`, Branches. -```bash -awk '/^## \[Unreleased\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep '^- ' \ - | sed -E 's/ \(#[0-9, #]+( by @[A-Za-z0-9-]+)?\)$//' \ - | awk '{ t+=length; n++; if (length>120) o++ } - END { if (!n) { print "no entries"; exit } print n" entries, avg "int(t/n)" chars, "o+0" over 120" }' -awk '/^## \[Unreleased\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep -nE '^- .*( now | no longer |, so )' -``` +## 7. Curate -Bare `length` is the line's length. Never write a dollar sign followed by a digit in this file: -Claude Code replaces it with a skill argument, so `/release 0.77.0` ran `length(0.77.0)`. +- Compare what merged with the fragments: `gh pr list --state merged --base main --search "merged:>=" --limit 1000 --json number,title,files`. A user-visible PR with no fragment needs one; a fragment that says more than its PR did needs rewording. +- Add or reword fragments through a PR to `main`, then pick it to the line. Never edit `CHANGELOG.md` by hand. +- For X.Y.0, write the lead per `.claude/rules/changelog.md`: at most 6 plain lines, what a reader would change their behavior over, in the reader's words. A release of fixes only can go without one. +- Check every fragment against `.claude/rules/changelog.md` and `scripts/check-banned-words.sh`. -`instead of` alone is fine when it describes the bug. Rewrite what runs long or matches to -`.claude/rules/changelog.md`, merging entries for one change and keeping every credit as it is. Diff -the `#` references and handles before and after to prove none were dropped. +Run: `RELEASING.md`, Notes, then Cutting a stable or patch. -### Bump the version +## 8. Watch -In `Configs/Version.xcconfig`, set `MARKETING_VERSION` to the new version and add 1 to -`CURRENT_PROJECT_VERSION`. Touch no other version: plugin and PluginKit versions live in `project.yml`, -and the iOS app's in `Configs/Version-iOS.xcconfig`. The release job checks both values -(`RELEASING.md`, The release pipeline). +- Follow the run with `gh run list --workflow release.yml --limit 3` and `gh run watch `. +- Sort a failure into infrastructure (runner, network, notary service, an expired certificate) or code (a build, test or check the tag's commit fails). Infrastructure is a re-run; code is a fix on `main`, a pick, and the next number. +- Confirm the result from the source, not the cached site: `gh api -H 'Accept: application/vnd.github.raw' 'repos/TableProApp/TablePro/contents/appcast.xml?ref=feeds' | grep -c ''`, and `gh release view v --json assets -q '.assets | length'`. -### Finalize CHANGELOG.md +Run: `RELEASING.md`, What a tag runs, and When a run fails. -1. Add `## [] - ` below `## [Unreleased]`, leaving Unreleased empty. In the - footer, point `[Unreleased]` at `v...HEAD` and add - `[]: https://github.com/TableProApp/TablePro/compare/v...v`. -2. Each `###` heading appears once, in the order Added, Changed, Deprecated, Removed, Fixed, - Security. Merge a repeated heading into the first and move an out-of-order block, then check - nothing was lost: +## 9. After a stable or patch - ```bash - awk '/^## \[\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep '^### ' - awk '/^## \[\]/{f=1;next} /^## \[/{f=0} f' CHANGELOG.md | grep -c '^- ' - grep -n '^## \[' CHANGELOG.md | head -5 - ``` +- Open the forward-port PR to `main` (and to a newer open line), and the docs changelog PR. +- For an X.Y.0, name the `announce` skill in TableProApp/web as the next step. +- Before the next stable, check `retainFromPluginKit` against the license dashboard. -3. Write the lead block: two or three lines under the version heading, before the first `###`, - naming the `### Added` items a reader would change their behavior over, in the reader's words. - The update window and the Sparkle feed show only this block. A release of pure fixes can skip it. -4. Regenerate the in-app notes so **Help > What's New** matches the feed: - `scripts/generate-whats-new.sh `. +Run: `RELEASING.md`, Cutting a stable or patch, and Plugins. -### Docs changelog +## 10. Withdraw -Add an `` block at the top of `changelog.mdx` in the docs repository (`../tablepro-docs`, -TableProApp/docs; confirm the path and `git pull --ff-only` first), right after the frontmatter. The -docs are English only, so there is no translated copy. - -```mdx - - ### New Features - - - **Feature Name**: what it does for the reader - -``` - -The headings are New Features, Improvements, Removed, Security and Bug Fixes, in that order, each -only when the version has one. Each line is one CHANGELOG entry written for a reader; a New Features -line may add one sentence on what the reader does with it. An outside contributor's line ends -`, by [@user](https://github.com/user)`; a maintainer's carries no credit. - -### Critical update flag - -`.github/release-flags.json` holds `criticalUpdate`. Leave it `false`: updates install in the -background and apply on quit, and the flag is what lets Sparkle interrupt the user. Set it `true` -only when the release fixes one of these, and say which: - -- Data loss or corruption of a database, saved connections or persisted tabs. -- A security issue with a `### Security` entry that users of the old build are exposed to. -- A crash or hang on a common path, or a failure to launch, connect or update. -- A regression from the previous release with no workaround. - -Nothing else; a handful a year at most. Set it in the release commit and back to `false` in the -next. A retag of the same release never gets it. A bad build is withdrawn (Withdraw a release) and -superseded by a corrective release with the flag. What the flag makes Sparkle do, and the release -job's check for a leftover flag: `RELEASING.md`, Staged rollout and critical updates. - -### Commit, tag, push - -The release key skips every check, so before the user pushes with it, prove the push carries the -release commit and nothing else; stop if the check prints nothing. Push the docs changelog in the -same sitting: the docs CI fails while the app's `CHANGELOG.md` carries a version the page does not. -It goes straight to the docs `main` under the admin bypass, and Mintlify deploys it. - -```bash -git add Configs/Version.xcconfig CHANGELOG.md .github/release-flags.json TablePro/Resources/WhatsNew.md -scripts/check-banned-words.sh --staged -git commit -m "release: v" -git fetch origin main -[ "$(git branch --show-current)" = main ] \ - && [ "$(git rev-parse HEAD^)" = "$(git rev-parse origin/main)" ] \ - && [ -z "$(git diff --name-only HEAD^ HEAD | grep -vxE 'CHANGELOG\.md|Configs/Version\.xcconfig|TablePro/Resources/WhatsNew\.md|\.github/release-flags\.json')" ] \ - && echo "only the release commit" -git tag -a v -m "v" -# The user runs this one: -GIT_SSH_COMMAND='ssh -i ~/.ssh/tablepro_release -o IdentitiesOnly=yes' git push origin main -git push origin v -# Then the docs changelog: -git -C ../tablepro-docs add changelog.mdx -git -C ../tablepro-docs commit -m "docs(changelog): v" -git -C ../tablepro-docs push origin main -``` - -- Keep the subject exactly `release: v`. -- Keep the release commit to the four files above. All four are on the change detector's - release-only list (`.claude/rules/ci.md`, Change detector), so the push skips both suites and the - tag takes the verdict of the code it ships. Any other file runs the suites on that push, and the tag waits for - them. A lint fix made on the way lands first, through its own PR. - -### Confirm it shipped - -A pushed tag is not a build: the workflow publishes from its last job. Check the run, the release -assets and the feed: - -```bash -gh run list --workflow build.yml --limit 1 -gh release view v --json tagName,assets -q '"\(.tagName) assets=\(.assets|length)"' -curl -s https://raw.githubusercontent.com/TableProApp/TablePro/main/appcast.xml | grep -o '[^<]*' | head -2 -``` - -That raw URL is the app's `SUFeedURL`; `tablepro.app/appcast.xml` does not exist. `shortVersionString` -is an element, not an attribute. A failed run: `RELEASING.md`, When a release run fails. - -### Withdraw a release - -Only when a build loses data, fails to launch or breaks connecting, and the fix is more than a few -minutes away. Commands and effects: `RELEASING.md`, Withdrawing a release; the user runs its push. -Users already on the build are reached by the corrective release with `criticalUpdate` set. - -## Stage 2: Plugins - -After the app tag is pushed, list the registry plugins with commits since their last tag: - -```bash -python3 .claude/skills/release/scripts/changed-plugins.py -``` - -Slugs, tag names and targets come from `.github/plugin-registry.json`, so nothing is mapped by hand. -`own` counts commits in the plugin's source paths (its folder, plus a package or `Native/` bridge it -builds from), `kit` counts PluginKit commits. Skip `bundled` rows: their changes ship inside the -app, which never applies a registry update to a bundled plugin. A row with only `kit` commits needs -a release only after a breaking bump, through `release-all-plugins.sh`. - -Show the user what changed and ask before tagging, suggesting a patch bump from the last tag. Tag -each approved plugin annotated (top of this file). The tag is the version, so a plugin needs no -version bump or changelog edit. - -```bash -git tag -a plugin--v -m "plugin--v" -git push origin plugin--v -``` - -- One tag per push: GitHub sends no push event for a push carrying more than three tags, so no - build starts. If several went at once, run `gh workflow run build-plugin.yml -f tags=","`. -- Wait for the app build first (`gh run list --workflow build.yml --limit 1`): plugin builds compete - with its `Build` jobs and `Create GitHub Release` for the same macOS runner slots - (`.claude/rules/ci.md`, Caches and runner budget). Then confirm a run started for each tag with - `gh run list --workflow build-plugin.yml --limit 10`. -- A published plugin release cannot take a rebuilt binary, and a rebuild never matches the - published bytes. A plugin run that fails after its release is published ships as the next patch. - -### PluginKit - -If the release touched `Plugins/TableProPluginKit/`, run `scripts/check-pluginkit-abi.sh v` -before publishing anything, and act on the diff as `.claude/rules/plugin-system.md` (ABI) says. -`scripts/release-all-plugins.sh` runs for a breaking bump only: each run uses a registry retention -slot for every plugin. One driver fix reaches users who have not updated through -`scripts/release-plugin-for-shipped-app.sh plugin--v v`. +Only when a build loses data, fails to launch or breaks connecting, and the fix is more than minutes away. The fix ships as the next number, critical when data was lost. Run: `RELEASING.md`, Withdrawing a release. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 177e2acf0f..667292e7aa 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -54,9 +54,9 @@ body: attributes: label: TablePro version description: >- - About TablePro shows it. On macOS, updates install in the background, so this is often not the newest. - On Linux, `tablepro --version` prints it too. - placeholder: "0.74.0" + About TablePro shows it, with the build number in parentheses and any -beta.N. On macOS, updates install + in the background, so this is often not the newest. On Linux, `tablepro --version` prints it too. + placeholder: "0.82.0 (820099)" validations: required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 7e1a85a819..1bdc68f1bf 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,6 +1,6 @@ + Fixes # diff --git a/.github/workflows/repo-hygiene.yml b/.github/workflows/repo-hygiene.yml index 54bf5e415b..2ecb7b2820 100644 --- a/.github/workflows/repo-hygiene.yml +++ b/.github/workflows/repo-hygiene.yml @@ -157,3 +157,5 @@ jobs: # go tool would build staticcheck for GOOS too, and Linux cannot run a darwin binary. go build -modfile=tools.mod -o "$RUNNER_TEMP/staticcheck" honnef.co/go/tools/cmd/staticcheck GOOS=darwin "$RUNNER_TEMP/staticcheck" ./... + GOBIN="$RUNNER_TEMP/bin" go install golang.org/x/vuln/cmd/govulncheck@v1.8.0 + GOOS=darwin "$RUNNER_TEMP/bin/govulncheck" ./... diff --git a/AGENTS.md b/AGENTS.md index 1c35aa67aa..86364dea00 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,7 +31,7 @@ Every capability is core, a module or a plugin. The first rule that fits decides ## Repository map - `TablePro/`: the app (`Core/`, `Views/`, `Models/`, `ViewModels/`, `Extensions/`, `Theme/`). -- `Plugins/`: drivers and import/export formats as `.tableplugin` bundles, plus `TableProPluginKit`. Bundled plugins are the app's `copy: { destination: plugins }` targets in `project.yml`; the rest ship through TableProApp/plugins, and the app never offers a registry install or update over a bundled one. +- `Plugins/`: drivers and import/export formats as `.tableplugin` bundles, plus `TableProPluginKit`. Bundled plugins are the app's `copy: { destination: plugins }` targets in `project.yml`; the rest ship as assets of each app release, and the app never offers a registry install or update over a bundled one. - `Packages/`: `TableProCore` (both apps), `TableProMacCore` (macOS-only modules; never in the iOS project), `TableProOracle`, `TableProEditor`, `TableProGrammars`. `TableProCore`'s PluginKit source is a symlink: edit `Plugins/TableProPluginKit/` only. Each manifest sets its own Swift language mode, so never pass `SWIFT_VERSION=` to `xcodebuild`. - Tests: `TableProTests/`, `TableProUITests/`, `Plugins/*Tests/`, `Packages/*/Tests/`, `TableProMobile/TableProMobileTests/`; plans in `TestPlans/`. Which holds what: `.claude/rules/tests.md`. - `Native/`: first-party Rust and Go bridges (`.claude/rules/native-bridges.md`). @@ -101,7 +101,7 @@ scripts/generate-project.sh [both|macos|ios] # needs the libs; rerun after edit ## Every change 1. **Tests.** Every testable change has a test in the target owning the code. Read `.claude/rules/tests.md` before creating a test file: a path rule loads only once a matching file is open. A UI flow only for a new or changed end-to-end flow (`.claude/rules/ui-tests.md`). -2. **CHANGELOG.md.** A user-visible change gets one fragment under `[Unreleased]`; format: `.claude/rules/changelog.md`. +2. **Changelog.** A user-visible change adds `changelog.d/..md`; only a release PR edits `CHANGELOG.md`. Format: `.claude/rules/changelog.md`. 3. **Localization.** `String(localized:)` outside SwiftUI literals, never interpolated: `String(format: String(localized: "Preview %@"), name)`. Technical terms stay unlocalized. A plugin or package string lives in the host app's catalog: add it with `python3 scripts/localization.py plugins --add`, plus `--target ios` for a package the iOS app links. Never hand-edit a `.xcstrings`. 4. **Docs.** A new shortcut, UI or settings change, or driver change gets a paired draft PR in TableProApp/docs naming this one; it merges after this one. `verify.sh docs` runs its checks here first. 5. **Lint** each changed Swift file with `verify.sh lint`. Run `verify.sh agent-docs` when `CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`, `RELEASING.md`, `.github/pull_request_template.md` or anything under `.claude/` changed. @@ -118,7 +118,7 @@ scripts/generate-project.sh [both|macos|ios] # needs the libs; rerun after edit ## CI -- A PR merges on five green checks: macOS Tests, iOS and Package Tests, SwiftLint, Repo Hygiene and PR title. Only release deploy keys bypass them (`RELEASING.md`, Keys and rulesets). +- A PR into `main` or `release/*` merges on five green checks: macOS Tests, iOS and Package Tests, SwiftLint, Repo Hygiene and PR title. Nothing bypasses them. - Read why a CI check failed before any re-run. - Workflows: `.claude/rules/ci.md`. Red checks: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. @@ -129,8 +129,8 @@ Claude Code loads a rule when it reads or edits a file the rule covers. Under Co | Rule | Read before editing | | --- | --- | | `ai-mcp-security.md` | `TablePro/Core/AI`, `TablePro/Core/MCP`, `Packages/TableProMacCore/Sources/TableProMCP` | -| `changelog.md` | `CHANGELOG.md` | -| `ci.md` | `.github/workflows`, `.github/actions`, `.github/scripts`, `.github/actionlint.yaml`, `.github/*baseline*`, `scripts/ci`, `.swiftlint.yml`, `**/.swiftlint.yml` | +| `changelog.md` | `CHANGELOG.md`, `changelog.d`, `towncrier.toml` | +| `ci.md` | `.github/workflows`, `.github/actions`, `.github/actionlint.yaml`, `.github/*baseline*`, `scripts/ci`, `scripts/release`, `.swiftlint.yml`, `**/.swiftlint.yml` | | `connection-window.md` | `TablePro/Core/Services/Infrastructure`, `TablePro/Models/Connection`, `TablePro/Core/Database/DatabaseManager*.swift`, `Packages/TableProCore/Sources/TableProDatabase`, `TablePro/Views/Main` | | `data-grid.md` | `TablePro/Views/Results`, `TablePro/Core/DataGrid`, `TableProTests/Views/Results` | | `data-sync-storage.md` | `TablePro/Core/**/*Storage*.swift`, `TablePro/Core/**/*Sync*.swift`, `TablePro/Core/Database`, `TablePro/Core/Plugins/PluginMetadataRegistry*.swift`, `TablePro/Core/Services/Query`, `Packages/TableProCore/Sources/TableProSyncTransport`, `CloudKit` | @@ -139,7 +139,7 @@ Claude Code loads a rule when it reads or edits a file the rule covers. Under Co | `mongodb-driver.md` | `Plugins/MongoDBDriverPlugin`, `Plugins/MQLExportPlugin` | | `mysql-driver.md` | `Plugins/MySQLDriverPlugin`, `Packages/TableProCore/Sources/TableProMySQLCore`, `TableProMobile/**/*MySQL*`, `TableProMobile/**/*MariaDB*` | | `native-bridges.md` | `Native`, `Plugins/DamengDriverPlugin`, `Plugins/HanaDriverPlugin`, `scripts/build-dameng.sh`, `scripts/build-hana.sh` | -| `plugin-system.md` | `Plugins`, `Packages/TableProCore/Sources/TableProPluginKit`, `Packages/TableProCore/Sources/TableProMySQLCore`, `Packages/TableProCore/Sources/TableProPostgreSQLCore`, `Packages/TableProCore/Sources/TableProRedisCore`, `Packages/*/Package.swift`, `project.yml`, `TableProMobile/project.yml`, `.github/workflows/build-plugin.yml`, `.github/plugin-registry.json`, `scripts/*plugin*` | +| `plugin-system.md` | `Plugins`, `Packages/TableProCore/Sources/TableProPluginKit`, `Packages/TableProCore/Sources/TableProMySQLCore`, `Packages/TableProCore/Sources/TableProPostgreSQLCore`, `Packages/TableProCore/Sources/TableProRedisCore`, `Packages/*/Package.swift`, `project.yml`, `TableProMobile/project.yml`, `Configs/PluginKit.xcconfig`, `.github/plugin-registry.json`, `scripts/*plugin*` | | `redis-driver.md` | `Plugins/RedisDriverPlugin`, `Packages/TableProCore/Sources/TableProRedisCore` | | `split-views.md` | `TablePro/**/*Split*.swift`, `TablePro/Core/Services/Infrastructure/WorkspacePane*.swift` | | `tests.md` | `TableProTests`, `TableProUITests`, `Plugins/*Tests`, `Packages/*/Tests`, `TableProMobile/TableProMobileTests`, `TestPlans`, `scripts/ci/check-test-plans.py`, `scripts/clean-test-residue.sh` | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ff3f0c45e6..03d1253aca 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,7 +39,7 @@ TABLEPRO_DEVELOPMENT_TEAM = TABLEPRO_APP_BUNDLE_IDENTIFIER = com..TablePro ``` -A free personal team works. Debug builds sign against `TablePro/TablePro.Debug.entitlements`, which drops iCloud, so sync turns itself off. Plugin bundles and the MCP helper sign ad hoc. The file is gitignored, so these settings never reach a commit and survive regenerating the project. Signing changed in the Xcode UI does not: the next `scripts/generate-project.sh` discards it. To check it, save a connection password, relaunch, and reopen the connection: the password is still there. +A free personal team works. Debug builds sign against `TablePro/TablePro.Debug.entitlements`, which drops iCloud, so sync turns itself off. Plugin bundles sign with the same team, and the MCP helper signs ad hoc. The file is gitignored, so these settings never reach a commit and survive regenerating the project. Signing changed in the Xcode UI does not: the next `scripts/generate-project.sh` discards it. To check it, save a connection password, relaunch, and reopen the connection: the password is still there. ## Building @@ -57,7 +57,7 @@ The app scheme compiles only the plugins the app embeds. The `AllPlugins` aggreg xcodebuild build -project TablePro.xcodeproj -scheme AllPlugins -destination 'platform=macOS' -skipPackagePluginValidation CODE_SIGNING_ALLOWED=NO ``` -`scripts/build-plugin.sh ` builds one signed, packaged bundle. Target names are the keys under `targets:` in `project.yml`, such as `OracleDriver`. +`scripts/release/build.sh plugins --unsigned` builds the registry plugins the way a release does, universal, into `build/release/plugins`: [`RELEASING.md`](RELEASING.md#reproducing-a-release-build). ### Clean builds @@ -113,14 +113,14 @@ The architecture and its invariants: [`AGENTS.md`](AGENTS.md), Architecture and ## Branch naming -Branch off `main`: `feat/add-cassandra-support`, `fix/query-editor-crash`. +Branch off `main`: `feat/add-cassandra-support`, `fix/query-editor-crash`. PRs target `main`; a `release/` line takes only cherry-picks of fixes already on `main` ([`RELEASING.md`](RELEASING.md#branches)). ## Pull requests One logical change per PR, described with the template the PR form opens with. Before you open it: - Tests added or updated. -- `CHANGELOG.md` updated under `[Unreleased]`, with the credit left off: the release adds it (`.claude/rules/changelog.md`). +- A changelog fragment in `changelog.d/` for a user-visible change (`.claude/rules/changelog.md`). - User-facing behavior documented in a paired pull request in [TableProApp/docs](https://github.com/TableProApp/docs) that links this one. - User-facing strings localized. - SwiftLint clean. @@ -154,7 +154,7 @@ The export builds the app first, so run `scripts/download-libs.sh` and `scripts/ | A new source file does not compile | Run `scripts/generate-project.sh` | | `Undefined symbols for architecture arm64` | `Libs/` is empty or stale: `scripts/download-libs.sh`, or `--force` to overwrite | | `Libs does not match the checksums committed in git.` | Re-run `scripts/download-libs.sh --force`. A library rebuilt on purpose is published as [`RELEASING.md`](RELEASING.md#static-libraries) describes | -| `Build input file cannot be found: '…/tablepro-hana-helper'`, or the helper has no slice for your architecture | Run `scripts/build-hana.sh` with no argument, which builds both architectures | +| `error: …/tablepro-hana-helper is missing or has no slice` | Run `scripts/build-hana.sh` with no argument, which builds both architectures | | Signing errors after a fresh clone | `Configs/Base.xcconfig` defaults to the maintainer's team and includes `Secrets.xcconfig` optionally, so a build without it still runs and fails to sign. Create the file with your team | | Package resolution fails | `Cmd+Shift+K` and reopen Xcode; then `rm -rf ~/Library/Developer/Xcode/DerivedData` | | `Timed out while enabling automation mode`, and UI tests run nothing | Answer the Enable UI Automation prompt with an admin password, or run `sudo automationmodetool enable-automationmode-without-authentication` once | diff --git a/Plugins/HanaDriverPlugin/README.md b/Plugins/HanaDriverPlugin/README.md index f78fc008df..0a7efd020c 100644 --- a/Plugins/HanaDriverPlugin/README.md +++ b/Plugins/HanaDriverPlugin/README.md @@ -4,7 +4,7 @@ This plugin connects TablePro to SAP HANA Cloud and SAP HANA 2.0 through [SAP/go The Go bridge in `Native/HanaBridge` is built by `scripts/build-hana.sh` into `tablepro-hana-helper`, an executable with no cgo. The plugin carries it in `Contents/MacOS`, starts one helper for each session, stops it when the session closes, and talks to it over stdin and stdout in length-prefixed JSON frames. The metadata connection pool opens drivers of its own, so one connection can run several helpers. A crash in go-hdb ends that driver's session, not TablePro. The helper owns the HANA session and returns JSON, and the Swift side writes every message the user reads. -Xcode copies the helper into the bundle and does not build it. Run `scripts/build-hana.sh` with no argument, which builds a universal arm64 and x86_64 helper, before building `HanaDriver` or `AllPlugins`. A thin build, `scripts/build-hana.sh arm64` or `x86_64`, only works in a build of that one architecture. `scripts/build-plugin.sh` signs the helper with Developer ID, the hardened runtime and a timestamp before it signs the plugin. +Xcode copies the helper into the bundle and does not build it. Run `scripts/build-hana.sh` with no argument, which builds a universal arm64 and x86_64 helper, before building `HanaDriver` or `AllPlugins`. A thin build, `scripts/build-hana.sh arm64` or `x86_64`, only works in a build of that one architecture. The copy phase signs the helper with the plugin's identity, so a release build gives it Developer ID, the hardened runtime and a timestamp. What the plugin does: diff --git a/RELEASING.md b/RELEASING.md index 03f003853a..ff8408722d 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,182 +1,150 @@ # Releasing -`scripts/build-release.sh` is the whole release build: it signs every nested binary (plugins, -helpers, frameworks) and the app bundle last, so signing again by hand undoes its work. +This runbook covers the macOS app, its registry plugins and the static libraries; the iOS app ships by hand. The `release` skill (`.claude/skills/release/SKILL.md`) decides what ships and points at the sections here. Every command below is the maintainer's to run. -Everything here needs the maintainer's credentials; a contributor wants `CONTRIBUTING.md`, Building. -The `release` skill in `.claude/skills/release` runs a release on top of this file and owns the -procedure and its decisions. +## Channels -## Release builds +- Stable and beta share one appcast, `https://updates.tablepro.app/appcast.xml`. A beta item carries `beta`. +- The app takes betas only while Settings > General > Get beta updates is on (`allowedChannels(for:)`). The key, `com.TablePro.updates.receivesBetaUpdates`, can be pinned by a configuration profile ([Managed updates](https://docs.tablepro.app/security/managed-updates)). +- Sparkle never downgrades: with the toggle off, a beta install keeps its beta until a stable passes it. So a beta never writes storage the previous stable cannot read (`.claude/rules/data-sync-storage.md`). +- A stable rolls out in phases of 21600 s. A beta is not phased, and on GitHub it is a prerelease. Latest goes only to the highest stable, so a patch on an older line publishes with `--latest=false`. +- Critical means a security fix or data loss, nothing else. It is ` [CRITICAL]` on the version's `CHANGELOG.md` heading, never on a beta. Every later stable item carries the highest critical build forward (`sparkle:criticalUpdate sparkle:version`). +- CI writes the orphan `feeds` branch, which holds `appcast.xml` and the v3 plugin manifest (`https://updates.tablepro.app/plugins/v3.json`), and `pages.yml` deploys it to `updates.tablepro.app`. The appcast is EdDSA-signed, and the app requires that (`SURequireSignedFeed`, `SUVerifyUpdateBeforeExtraction`). +- The legacy `appcast.xml` on `main` and `plugins.json` in TableProApp/plugins serve apps up to 0.81. Both freeze at v0.82.0, the one bridge release. -```bash -scripts/build-release.sh arm64 # or x86_64, or both (the default with no argument) -``` +## Versions -Output lands in `build/Release/TablePro-arm64.app` and `build/Release/TablePro-x86_64.app`. +- A tag is `vX.Y.Z` or `vX.Y.0-beta.N`, N from 1 to 98. There is no `-rc`. +- `MARKETING_VERSION` is the tag without the `v`. The build, `CURRENT_PROJECT_VERSION`, is `((MAJOR*1000+MINOR)*100+PATCH)*100+R`, where R is N for a beta and 99 for a stable: `v0.82.0-beta.1` is 820001, `v0.82.0` is 820099, `v0.82.1` is 820199. MINOR stays at most 999 and PATCH at most 99. +- A local build is `0.0.0-dev`, build 1 (`project.yml`, `settings.base`). `scripts/release/version.py parse ` prints a tag's version, build, channel and line. Every "is newer" check, in the app and in the plugin manifest, compares the build (`CFBundleVersion`). +- Every X.Y.0 ships at least one published `vX.Y.0-beta.N` first; patches go straight to stable. Promote the newest beta when it has no open `regression` issue. +- One universal build, arm64 and x86_64. It becomes arm64-only once the deployment target reaches macOS 27. -The script extracts the matching slice from the universal static libraries, copies every non-system -dynamic library the binaries reference into `Contents/Frameworks` and rewrites their install names, -strips the binaries, then signs with hardened runtime and `TablePro/TablePro.entitlements`. +## Branches -| Variable | Default | Purpose | -|---|---|---| -| `SIGN_IDENTITY` | the maintainer's Developer ID Application | codesign identity for every binary | -| `NOTARIZE` | `false` | `true` notarizes and staples after the build | -| `NOTARY_PROFILE` | `TablePro` | `notarytool` keychain profile holding the credentials | +- An admin cuts `release/` from a green `main` commit at beta.1. Only admins create `release/*` branches. +- Every fix lands on `main` first, then reaches the line through a cherry-pick PR from `backport/-`: one fix per PR, titled with the original subject and its ` (#N)`, which is the provenance. A release PR comes from `prep/v`. No PR head branch lives under `release/*`. +- A line takes patches until `X.(Y+1).0` ships, then only security or data-loss fixes. At most two lines are live. +- After a stable or patch publishes, a PR with the same title cherry-picks its release commit to `main`, and to a newer open line when one exists. The next tag refuses to build until it has. -Create the notary profile once with `xcrun notarytool store-credentials "TablePro"` plus -`--apple-id`, `--team-id` and `--password`. `scripts/lib/notarize.sh` zips, submits, waits and -staples. It reads both the exit status and the reported status, because `notarytool` has exited 0 -on a rejected submission. +## Notes -## Packaging a DMG +- Fragments in `changelog.d/` (`.claude/rules/changelog.md`) become a `CHANGELOG.md` section. The tooling needs towncrier: `python3 -m venv .venv && .venv/bin/pip install --require-hashes -r scripts/release/requirements.txt`. +- `python3 scripts/release/notes.py assemble X.Y.Z [--lead FILE] [--critical] [--date YYYY-MM-DD]` runs on `prep/v`. It refuses a beta, a branch with commits beyond `origin/release/` other than a revert, an empty `changelog.d/`, and a version `CHANGELOG.md` already has. It credits each fragment ` by @login` when its PR's author is not an OWNER or MEMBER (read with `gh`), runs `towncrier build`, which deletes the fragments, adds the lead and the critical marker, and stages `CHANGELOG.md`. +- The lead: X.Y.0 only, at most 6 plain lines, naming what a reader would change their behavior over. The Sparkle notes, What's New and the Telegram post show only the lead. +- A beta gets no section. Its GitHub release body and feed notes are a towncrier draft of the fragments at the tag. -`scripts/create-dmg.sh [source app]` packages one architecture. The source -app defaults to the one `build-release.sh` wrote for that architecture. An empty version reads -`MARKETING_VERSION` from `Configs/Version.xcconfig`, the only place the app version is declared. +## Cutting a beta + +beta.1 cuts the line at the newest `main` commit whose push runs are all green; beta.N tags the line's head once the picks are on it. ```bash -scripts/create-dmg.sh 0.80.0 arm64 # writes build/Release/TablePro-0.80.0-arm64.dmg +git fetch origin +git push origin :refs/heads/release/X.Y # beta.1 only +git tag -s vX.Y.0-beta.N -m vX.Y.0-beta.N origin/release/X.Y +git push origin vX.Y.0-beta.N ``` -It prefers `create-dmg` (`brew install create-dmg`) and falls back to `hdiutil`. `NOTARIZE=true` -notarizes the DMG too. +## Cutting a stable or patch -## Static libraries +```bash +git fetch origin +git switch -c prep/vX.Y.Z origin/release/X.Y +python3 scripts/release/notes.py assemble X.Y.Z --lead lead.txt # --lead on X.Y.0 only, --critical when it is +git commit -m "chore(release): vX.Y.Z" +git push -u origin prep/vX.Y.Z +gh pr create --base release/X.Y --title "chore(release): vX.Y.Z" --body "Release notes for vX.Y.Z." +``` -The `.a` files under `Libs/` and the xcframeworks under `Libs/ios/` live on the immutable -`libs-` GitHub release that `Libs/release` names, not in git. `scripts/download-libs.sh` fetches -it and verifies every file against the checksums at `HEAD` on every run. Publish a rebuilt library -through the script, then commit what it writes: +The PR changes only `CHANGELOG.md` and `changelog.d/`. Squash-merge it, then tag its merge commit: ```bash -scripts/publish-libs.sh libmongoc_arm64.a libmongoc_x86_64.a libmongoc_universal.a libmongoc.a -git add Libs/release Libs/checksums.sha256 Libs/ios/checksums.sha256 -git commit -m "build(libs): publish libs-" +git fetch origin +git tag -s vX.Y.Z -m vX.Y.Z "$(gh pr view --json mergeCommit -q .mergeCommit.oid)" +git push origin vX.Y.Z ``` -- Name every library you rebuilt, added or removed. The script verifies every other one against the - checksums at `HEAD`, creates the next `libs-` and rewrites `Libs/release` and both checksum files. -- `--dry-run` prints what it would publish. -- A published `libs-` never changes; a fix is the next ``. - -## The release pipeline - -`.github/workflows/build.yml` runs on a `v*` tag push or a manual dispatch, and a run is never -cancelled (concurrency group `build-tablepro-`). Every job except the two builds needs a `v*` -ref, so a dispatch from a branch only builds. The tag runs no test suite. - -- `Tested on main` (Ubuntu) fails when the tagged commit is not an ancestor of `origin/main`. -- For `macOS Tests` and `iOS and Package Tests` separately, it walks up to 50 first-parent commits - back from the tag. On each it takes the newest push run of that workflow and waits for it if it is - still going. A commit with no run, or whose job was skipped, is passed over; any conclusion other - than success fails the release. -- A passing run counts only when `scripts/ci/detect-changed-paths.sh --base ` - prints `false`, so nothing the suite reads changed between that commit and the tag. The step - summary names the covering run for each suite. -- The two suites may be covered by different commits. Requiring one commit for both would fail - whenever only one suite ran later. -- `Build arm64` and `Build x86_64` run on `xcode-27` with no fail-fast; x86_64 cross-compiles on an - arm64 runner. Each runs `scripts/build-release.sh` with notarization, then makes the DMG and the - tar.xz update archive. -- `Registry Readiness` (Ubuntu) runs `scripts/check-registry-readiness.py` with `--floor` and - `--current` set to the two kit versions in `TablePro/Core/Plugins/PluginManager.swift`. It fails - when a registry driver has no binary in that range, so the app never ships ahead of its plugins. -- `Create GitHub Release` needs all three and runs under the `release` environment. It fails when - the tag disagrees with `MARKETING_VERSION` (`tag v... does not match MARKETING_VERSION ...`). -- It also fails when `CURRENT_PROJECT_VERSION` is not above the newest published release's. Sparkle - compares that number as ``, so a flat one ships a release no install is offered. -- It signs both update archives with the Sparkle key and takes the notes from `CHANGELOG.md`. -- It checks that the appcast deploy key works before publishing anything. Then it puts both DMGs - and both update archives on a draft release, publishes the draft in one step, and only then - commits `appcast.xml`, whose download URLs 404 until the release is public. -- The appcast is merged with the tag's own copy of `scripts/ci/merge-appcast.py` and pushed with - `APPCAST_DEPLOY_KEY`, up to 3 attempts, each re-merging onto the current `origin/main`. -- `generate_appcast` sees only the new release. The merge script splices its items in and moves - published items byte for byte, because Sparkle rewrites an existing item's download URL and drops - its notes when a staged archive matches it. -- A tag containing `-alpha`, `-beta` or `-rc` publishes as a prerelease. - -## Keys and rulesets - -- `main` takes no direct push. Its ruleset requires the checks `AGENTS.md` lists under CI, and - deploy keys bypass it, so the appcast push and the release commit push need no check run. -- The appcast push uses `APPCAST_DEPLOY_KEY`, a write deploy key held by the `release` environment, - which only `v*` tags can use. -- The release commit and a withdrawal push with the release key, `~/.ssh/tablepro_release`, a second - write deploy key. The release skill's pre-flight step 7 creates it the first time a release needs - it. -- The release key has a passphrase and is never added to ssh-agent: agents run on the maintainer's - Mac and the key skips every check, so only the maintainer runs a push with it. -- The push command is in `.claude/skills/release/SKILL.md`, Commit, tag, push. -- The `v*` tag ruleset lets only admins create, move or delete a release tag. - -## When a release run fails - -- A published release is immutable: GitHub refuses to add, replace or delete an asset, or to move - the tag. -- Before **Publish the release**, only a draft exists. Fix forward on `main` through a pull request, - then move the tag to the commit it lands as. The next run reuses the draft. - - ```bash - git fetch origin main - git tag -f -a v -m "v" origin/main - git push origin v --force - ``` - -- After it (a failed `appcast.xml` push, say), use **Re-run failed jobs**: the release job finds the - same files already published and carries on. **Re-run all jobs** rebuilds, and the release job - refuses the new files. -- A published build that is wrong ships as a new version. - -## Staged rollout and critical updates - -- An ordinary release carries - `21600`. Sparkle splits installs - into seven cohorts by a per-install random id, so the last cohort is offered the build 36 hours - after the first. **Check for Updates…** is never phased. -- `"criticalUpdate": true` in `.github/release-flags.json` passes `--critical-update-version ""` to - `generate_appcast` instead. That writes `` with no version, critical for - every host: Sparkle skips phasing, hides Skip and Remind Me Later, retitles the alert, and shows - it even under automatic downloads. -- The release job fails when the flag is `true` and the file has not changed since the previous - tag, which is what a leftover flag looks like. -- When to set it: `.claude/skills/release/SKILL.md`, Critical update flag. +After the run publishes, forward-port the release commit, then write the docs changelog: -## Withdrawing a release +```bash +git fetch origin +git switch -c forward/vX.Y.Z origin/main +git cherry-pick +git push -u origin forward/vX.Y.Z +gh pr create --base main --title "chore(release): vX.Y.Z" --body "Forward-port of #." +python3 scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro-docs/changelog.mdx +``` + +- A cherry-pick conflict on a fragment reworded on `main`: delete the fragment; on `CHANGELOG.md`: keep both sections, in version order. The docs PR is `docs(changelog): vX.Y.Z` in TableProApp/docs, opened with the maintainer's own `gh` (a PR opened with `GITHUB_TOKEN` starts no workflows) and merged once the release is published. + +## What a tag runs + +A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. + +- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it. Then it needs macOS Tests, iOS and Package Tests and Release Build green on the tag commit or the nearest first-parent commit that ran them, waiting on a pending run. Last, it writes the notes. +- `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal, signature and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. +- `plugins` (`signing`): the Dameng and HANA bridges, `build.sh plugins` (each target `registry.py targets` prints, universal, signed with Developer ID, the hardened runtime and a timestamp), one notary submission for all, `package.sh plugins` (the zips, described as the manifest will read them). +- `github-release`: `publish.py draft` puts every asset on a draft and checks each digest, `attest-build-provenance` attests them, `publish.py release` publishes. +- `feed` (`release`): `feed.py add` downloads the archive and the zips from the release, checks their digests, runs `generate_appcast` and `registry.py add`, runs `feed.py check`, and pushes `feeds`, retrying from a fresh fetch up to 3 times. +- `pages` deploys `feeds`. `notify`, stable only and allowed to fail, posts the lead to Telegram. +- Assets: `TablePro-.dmg`, `TablePro-.tar.xz` (the Sparkle archive), `TablePro--dSYMs.zip`, `TablePro--Plugins-dSYMs.zip` and one `-.zip` per registry plugin. + +## When a run fails + +- Tags never move. Before `github-release` publishes, an infrastructure failure is a re-run of the failed jobs; a code failure is fixed on `main`, picked to the line, and ships as the next number. +- After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, or a feed item or manifest release with the same build and sha256. +- A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). +- No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. -Removing the version's items from `appcast.xml` is the only way to un-ship a Sparkle release. +## Withdrawing a release ```bash -python3 scripts/ci/pull-release.py --dry-run # reports what it would remove -GIT_SSH_COMMAND='ssh -i ~/.ssh/tablepro_release -o IdentitiesOnly=yes' \ - python3 scripts/ci/pull-release.py # edits, commits and pushes to main +gh workflow run withdraw.yml --ref v -f version=X.Y.Z ``` -- The second command pushes with the release key, so the maintainer runs it (Keys and rulesets). -- It refuses to push from any branch but `main`, or when local `main` is not `origin/main`, since - the key would carry any other commit past every check. It also refuses to empty the feed. -- The GitHub release stays up, so a direct download link still resolves. Users already on the bad - build are reached by shipping the fix. +- It removes the version's feed items and plugin releases, re-signs the feed, marks the highest stable left as Latest, and redeploys. It refuses to leave the feed with no stable item. The `--ref` is a tag because the `release` environment allows only `v*` tags. +- The GitHub release stays. The fix ships as the next number, critical when the bad build loses data. + +## Keys and environments + +| Environment | Refs | Secrets | +|---|---|---| +| `signing` | tags `v*` | `CERTIFICATES_P12`, `CERTIFICATES_PASSWORD`, `PROVISIONING_PROFILE`, `ASC_KEY_ID`, `ASC_ISSUER_ID`, `ASC_KEY_P8`, `ANALYTICS_HMAC_SECRET`, `CLOUDKIT_MANAGEMENT_TOKEN` | +| `release` | tags `v*` | `SPARKLE_PRIVATE_KEY`, `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_TOPIC_ID` | +| `github-pages` | tags `v*` | none | + +- No repo-level secret or deploy key is left, and no environment has required reviewers: admin-only, immutable tags are the gate. Rulesets: `.claude/rules/ci.md`, Rulesets. +- Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` stores as the `TablePro` notarytool profile. The action warns when the certificate or the provisioning profile expires within 60 days. +- An offline backup of the Sparkle EdDSA key exists and the maintainer holds it. The key never rotates without a signed transition, since every app from 0.82 verifies the feed with it. +- `feeds` takes pushes from `github-actions[bot]` with `GITHUB_TOKEN` and has no required checks; the signed feed proves its content. It was seeded once with the legacy `appcast.xml` and an empty v3 manifest. ## Plugins -- `.github/workflows/build-plugin.yml` runs on a `plugin-*-v*` tag, or on a dispatch with `tags` - (`tag[:pluginKitVersion],...`) and an optional `baseRef`. Runs are grouped per ref and per matrix - tag and never cancelled. -- `Resolve Plugin Tags` (Ubuntu) picks the runner: `macos-26` when the ref being built pins Xcode 26 - in its own `.github/actions/setup-xcode`, else `xcode-27`. -- It runs `.github/scripts/test_update_registry.py` and `scripts/ci/check-plugin-manifest.py`, and - validates each tag against `^plugin-[a-z0-9-]+-v[0-9]+\.[0-9]+\.[0-9]+$` and each kit version as - an integer. A tag given without a kit version takes `currentPluginKitVersion`. -- The build job checks out `baseRef` when one is given. It builds the Dameng bridge for - `plugin-dameng-*`, a universal HANA helper plus `govulncheck` for `plugin-hana-*`, then runs - `scripts/build-plugin.sh "$TARGET" both "$VERSION"` with notarization. -- Before publishing: `scripts/ci/verify-plugin-loads.sh` dlopens the arm64 bundle beside its - PluginKit framework; both zips must be stapled and accepted by Gatekeeper; the built - `TableProPluginKitVersion` must equal the release label. -- It publishes the release from a draft, then records the binary in the TableProApp/plugins registry - with `.github/scripts/update-registry.py`. The registry decides which binary a user gets: - [Plugin Registry](https://docs.tablepro.app/development/plugin-registry) covers the manifest, the - tag names and the bulk re-release after a PluginKit ABI change. +- One tag builds the app and every registry plugin, and the plugin zips are assets of the app's release. Each plugin's version and build are the app's. There are no plugin tags. +- `.github/plugin-registry.json` holds `retainFromPluginKit` and the publish set, keyed by target, with each plugin's `name` and `iconName`. Bundled plugins are not in it. +- The v3 manifest lists each plugin's releases: version, build, channel, PluginKit version, `minimumSystemVersion`, URL and sha256. The bundle id and `databaseTypeIds` come from the built bundle. +- Selection: a release in the running build's channel (a `-beta.N` or local build takes beta and stable, any other stable only), with its PluginKit version in the app's range and its `minimumSystemVersion` at or below the running macOS. The highest build wins. Installed registry plugins update after the app does. +- Retention per channel, kit and `minimumSystemVersion`: the two highest builds stay; a beta goes once a stable with the same key has a higher build; a kit below `retainFromPluginKit` goes. A release pruned on arrival is a logged no-op. +- `retainFromPluginKit` is the kit of the oldest app version still on at least 1% of the machines seen in the last 30 days (license dashboard, App Versions). Check it before each stable and raise it with a normal PR. The manifest keeps the higher of its own value and the tag's, so an older line never lowers it. +- Accepted risks: the manifest is trusted through HTTPS, the plugin's Team ID check and the sha256, so a rollback or freeze of the manifest goes undetected. +- PluginKit ABI rules: `.claude/rules/plugin-system.md`, ABI. + +## Static libraries + +The `.a` files under `Libs/` and the xcframeworks under `Libs/ios/` live on the immutable `libs-` GitHub release that `Libs/release` names, not in git; `scripts/download-libs.sh` fetches it and verifies every file against the checksums at `HEAD`. `scripts/publish-libs.sh` takes every library you rebuilt, added or removed, verifies the others against `HEAD`, creates the next `libs-` and rewrites `Libs/release` and both checksum files (`--dry-run` only prints). A published `libs-` never changes: a fix is the next ``. + +```bash +scripts/publish-libs.sh libmongoc_arm64.a libmongoc_x86_64.a libmongoc_universal.a libmongoc.a +git add Libs/release Libs/checksums.sha256 Libs/ios/checksums.sha256 && git commit -m "build(libs): publish libs-" +``` + +## Reproducing a Release build + +```bash +scripts/download-libs.sh && scripts/generate-project.sh macos && brew install create-dmg +scripts/release/build.sh app --unsigned && scripts/release/package.sh app 0.0.0-dev --unsigned +scripts/build-dameng.sh both && scripts/build-hana.sh both +scripts/release/build.sh plugins --unsigned && scripts/release/package.sh plugins 0.0.0-dev --unsigned +``` + +- `build.sh` writes `build/release/` (xcodebuild logs in `build/release/logs/`), `package.sh` writes `dist/`. Release Build (`release-build.yml`) runs the same on every push to `main` and `release/**`. Signed: drop `--unsigned`, set `VERSION` and `BUILD`, and have the Developer ID identity and the provisioning profile `ExportOptions.plist` names. `notarize.sh ...` needs the profile `xcrun notarytool store-credentials TablePro --key --key-id --issuer ` creates. From 7e16e6dc93c716fdb72fe24c1b75ee515786672e Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 10:41:23 +0700 Subject: [PATCH 08/10] ci(release): release from tags through release.yml --- scripts/ci/check-changelog.py | 3 +++ scripts/ci/test_check_changelog.py | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/scripts/ci/check-changelog.py b/scripts/ci/check-changelog.py index b7cf12308d..a197d28acc 100755 --- a/scripts/ci/check-changelog.py +++ b/scripts/ci/check-changelog.py @@ -180,6 +180,9 @@ def _fragment_in(path: Optional[str], types: List[str]) -> Optional[Fragment]: def pull_request_problems(number: int, title: str, base: str, files: List[ChangedFile], types: List[str]) -> List[str]: + # The pull request that creates changelog.d/ is the one that moves the old entries into it. + if any(changed.filename == f"{FRAGMENTS}/.gitkeep" and changed.status in NEW_STATUSES for changed in files): + return [] problems = [] expected = f"{FRAGMENTS}/{number}..md" if any("CHANGELOG.md" in (changed.filename, changed.previous_filename) for changed in files) \ diff --git a/scripts/ci/test_check_changelog.py b/scripts/ci/test_check_changelog.py index 2b218e591e..eb27db3f36 100644 --- a/scripts/ci/test_check_changelog.py +++ b/scripts/ci/test_check_changelog.py @@ -240,6 +240,12 @@ def test_a_file_that_is_not_a_fragment_is_refused(self): def test_gitkeep_is_not_a_fragment(self): self.assertEqual(self.problems([changed("changelog.d/.gitkeep")]), []) + def test_the_pull_request_that_creates_the_directory_moves_the_old_entries(self): + files = [changed("changelog.d/.gitkeep"), changed("changelog.d/3446.fixed.md"), + changed("CHANGELOG.md", "modified")] + self.assertEqual(self.problems(files, "ci(release): release from tags through release.yml"), []) + self.assertEqual(len(self.problems(files[1:], "ci(release): release from tags through release.yml")), 2) + def test_a_pick_into_a_release_line_keeps_the_original_name(self): self.assertEqual(self.problems([changed("changelog.d/3446.fixed.md")], "fix: keep it (#3446)", "release/0.82"), []) From 7834321b73c7261abf48099bdadabdfccfd5f9d8 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 11:00:35 +0700 Subject: [PATCH 09/10] ci(release): release from tags through release.yml --- RELEASING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RELEASING.md b/RELEASING.md index ff8408722d..bf3b904bc9 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -10,7 +10,7 @@ This runbook covers the macOS app, its registry plugins and the static libraries - A stable rolls out in phases of 21600 s. A beta is not phased, and on GitHub it is a prerelease. Latest goes only to the highest stable, so a patch on an older line publishes with `--latest=false`. - Critical means a security fix or data loss, nothing else. It is ` [CRITICAL]` on the version's `CHANGELOG.md` heading, never on a beta. Every later stable item carries the highest critical build forward (`sparkle:criticalUpdate sparkle:version`). - CI writes the orphan `feeds` branch, which holds `appcast.xml` and the v3 plugin manifest (`https://updates.tablepro.app/plugins/v3.json`), and `pages.yml` deploys it to `updates.tablepro.app`. The appcast is EdDSA-signed, and the app requires that (`SURequireSignedFeed`, `SUVerifyUpdateBeforeExtraction`). -- The legacy `appcast.xml` on `main` and `plugins.json` in TableProApp/plugins serve apps up to 0.81. Both freeze at v0.82.0, the one bridge release. +- The legacy `appcast.xml` on `main` and `plugins.json` in TableProApp/plugins serve apps up to 0.81, and no tag writes them. v0.82.0 is the one bridge: once it publishes, a PR copies its feed item verbatim into the legacy `appcast.xml` and a PR in TableProApp/plugins points `plugins.json` at its plugin zips. Both files then freeze. ## Versions From aac254bf509be40e1cb951321a5ae7cea0cdd40c Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Sun, 18 Oct 2026 12:34:27 +0700 Subject: [PATCH 10/10] ci(release): release from tags through release.yml --- .claude/rules/plugin-system.md | 2 +- .gitignore | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.claude/rules/plugin-system.md b/.claude/rules/plugin-system.md index ed334366b0..7b7d2090bf 100644 --- a/.claude/rules/plugin-system.md +++ b/.claude/rules/plugin-system.md @@ -65,7 +65,7 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke ## Structure - PluginKit-free logic a plugin shares with the iOS app lives in a TableProCore product, such as `TableProMySQLCore`, `TableProPostgreSQLCore` or `TableProRedisCore`. -- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, an entry in `.github/plugin-registry.json` keyed by its target, with `name` and `iconName`, a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), its `TableProProvides*` keys (below), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a changelog fragment. +- A new driver needs: its `project.yml` target and a `RegistryPlugins` entry (`BundledPlugins` and the app's `copy: { destination: plugins }` dependency for a bundled one), a `DatabaseType` constant, for a registry one an entry in `.github/plugin-registry.json` keyed by its target, with `name` and `iconName` (a bundled one stays out of it), a curated snapshot in `PluginMetadataRegistry` (without it the type never appears in the New Connection picker), its `TableProProvides*` keys (below), a row in `../tablepro-docs/databases/index.mdx` through a paired docs PR, and a changelog fragment. - A new engine ships as a registry plugin (`AGENTS.md`, Scope). - `PluginManager` registers a plugin from the `TableProProvidesDatabaseTypeIds`, `TableProProvidesExportFormatIds` and `TableProProvidesImportFormatIds` arrays in its `Info.plist` and loads its code on first use. A plugin with none of them loads at launch. The first database type id is the primary one. - `scripts/ci/check-plugin-provides.py` checks those arrays against the class's `databaseTypeId`, `additionalDatabaseTypeIds` and `formatId` in Repo Hygiene; `--write` stamps them. diff --git a/.gitignore b/.gitignore index c71089fc73..a78a15eff8 100644 --- a/.gitignore +++ b/.gitignore @@ -145,6 +145,9 @@ Derived/ *.xcarchive /dist/ +# towncrier venv for scripts/release/notes.py (RELEASING.md, Notes) +/.venv/ + # Static libraries (downloaded from GitHub Releases via scripts/download-libs.sh) Libs/*.a Libs/.downloaded