[Devops] Configure PyPI Trusted Publisher + pypi-release environment (manual, one-time)

[lowlydba]

Context

release-publish.yaml publishes to public PyPI via Trusted Publishing (OIDC) and supports a workflow_dispatch dry-run to Test PyPI. Neither side exists yet: PyPI has no trusted publisher registered for any of these packages, and none of the required GitHub Environments exist in this repo (gh api repos/OvertureMaps/schema/environments returns copilot, github-pages, staging). The workflow will fail closed until both are set up. Both are one-time manual steps (PyPI's UI, GitHub's repo settings), not something CI can provision.

Note

#650 renamed the six theme packages (overture-schema-<theme>-theme → overture-schema-theme-<theme>), and the PyPI distribution name changed with it, not just the package directory. The list below reflects the current names.

Important

A PyPI Trusted Publisher's identity is (repo, workflow filename, environment); it doesn't see release-publish.yaml's package input. A single shared pypi-release/test-pypi environment across all 12 packages means only the first package to register can ever bind to it: every other package's registration fails with A pending trusted publisher matching this configuration has already been registered for a different project name. Each package needs its own environment (pypi-release-<package>, test-pypi-<package>), which release-publish.yaml now computes per-package.

These environments scope the Trusted Publisher identity only, not an approval gate: with 12 of them, per-environment required reviewers would be unruly to maintain. The version-bump PR review is the approval; nothing further blocks the publish once a release exists.

Important

PyPI and Test PyPI each cap pending Trusted Publishers (ones registered before a project's first upload) at 3 per account. A pending publisher only converts to a real, uncapped one once its package's first publish succeeds. With 12 packages on each of two sites, this forces a batched rollout instead of registering all 24 up front:

1. Merge this PR's workflow changes to main (environments + per-package release-publish.yaml need to exist there first).
2. Register 3 packages' pending publishers, publish each once (dry-run to Test PyPI, real release to PyPI) to convert them, then register the next 3. Repeat until all 12 are done, on both sites.
3. Once every package has published at least once, all 24 Trusted Publisher configs are finalized and none are pending anymore.

Tasks

• Create 12 pypi-release-<package> GitHub Environments on OvertureMaps/schema (no required reviewers; see the approval-gate note above)
• Create 12 test-pypi-<package> GitHub Environments (mirrors overturemaps-py's test-pypi environment). release-publish.yaml's workflow_dispatch dry-run path targets these to exercise the build-and-publish pipeline without a real release or the production index.
• Register and publish the 12 packages in batches of 3 (per the pending-publisher cap above), on both pypi.org and test.pypi.org, pointing each at:
  • Repo: OvertureMaps/schema
  • Workflow: release-publish.yaml
  • Environment: pypi-release-<package> (pypi.org) or test-pypi-<package> (test.pypi.org)
  • Packages:
    • overture-schema
    • overture-schema-theme-addresses
    • overture-schema-theme-base
    • overture-schema-theme-buildings
    • overture-schema-cli
    • overture-schema-codegen
    • overture-schema-common
    • overture-schema-theme-divisions
    • overture-schema-theme-places
    • overture-schema-pyspark
    • overture-schema-system
    • overture-schema-theme-transportation
  • Use PyPI's pending publisher flow for each (no existing PyPI project yet); a package's slot frees up once its first publish converts the pending entry to a real one

Definition of done

• All 12 pypi-release-<package> environments exist
• All 12 test-pypi-<package> environments exist
• All 12 packages have published at least once to PyPI and Test PyPI, converting all 24 Trusted Publisher configs from pending to real
• One real release exercises the path end-to-end (first release covered by #509's open question on before: null first-release handling)

Parent: #509

[lowlydba]

Hit a snag where we can only queue up to 3 to-be-published packages before we have to start...publishing the package in PyPI - I had hoped we could just manually prep them all in Test/PyPI and then just point CI at them.

This will be blocked until we can actually trigger the publishes via a manual workflow_dispatch, ahead of automated releases to ensure everything works.