Skip to content

ci(release): publish stable Python wheels directly to PyPI - #5157

Merged
pimlock merged 3 commits into
mainfrom
ci/5156-direct-pypi-publishing/pimlock
Nov 3, 2026
Merged

pimlock merged 3 commits into
mainfrom
ci/5156-direct-pypi-publishing/pimlock

Conversation

@pimlock

@pimlock pimlock commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stable releases currently trigger Kitmaker through GitLab to publish Python wheels. Publish directly from release.yml to PyPI using Trusted Publishing and the pypi GitHub environment. Development and prerelease wheels remain GitHub Release assets without PyPI uploads.

Related Issue

Closes #5156

Changes

  • Upload the built wheel from a dedicated job in release.yml, with job-scoped id-token: write and a pinned PyPA publishing action. Check the stable tag's source commit, wheel name/version, and release checksum before uploading. Skip existing uploads on release retries.
  • Remove the stable and development Kitmaker triggers, the old wheel-publish.yml workflow, and GitLab trigger credential references.
  • Keep stable docs after PyPI publication, include PyPI and docs in the stable publication check and failure notification, and update the release guidance and regression tests.
flowchart LR
    B[Build stable artifacts] --> R[Publish release assets and other packages]
    R --> P[Verify wheel and publish to PyPI]
    P --> D[Publish stable docs]
Loading

The configured PyPI publisher trusts NVIDIA/OpenShell, workflow release.yml, and environment pypi. The GitHub environment currently allows main and has no required reviewers. Support release branches must be explicitly allowed before dispatching Release. No stored PyPI API token is required.

Release Notes

Testing

  • 35 existing release-tooling and updated docs-sync tests pass.
  • Ruff lint and format checks pass for the changed docs-sync test file.
  • Zizmor reports no high-severity findings in the four changed workflows.
  • Job dependencies and local reusable-workflow references resolve across all 58 workflows; no Kitmaker trigger references remain in the release workflows.
  • git diff --check passes.
  • Actionlint 1.7.12 reports two existing concurrency.queue syntax errors in release.yml and release-prerelease.yml. The unchanged versions produce the same errors; no new findings were reported. ShellCheck and Pyflakes were disabled for this check.
  • Live PyPI authentication and upload. No release workflow was dispatched during validation.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Release guidance and Fern workflow documentation updated

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
@pimlock pimlock added the area:sdk:python Python SDK label Nov 3, 2026
@pimlock
pimlock requested a review from SDAChess November 3, 2026 00:40
Comment thread .github/workflows/release-dev.yml Resolved
Comment thread .github/workflows/release.yml Resolved
SDAChess
SDAChess previously approved these changes Nov 3, 2026
Download the wheel and checksums from the stable GitHub Release instead of
the run's build artifact. A new Release run for the same tag rebuilds a
wheel whose bytes differ from the released asset, so publishing the run
artifact could not finish a failed PyPI upload after the artifact expired.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
@pimlock

pimlock commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator Author

a1cadbc changes publish-pypi to download the wheel and openshell-checksums-sha256.txt from the stable GitHub Release instead of the run's python-wheel build artifact.

Why:

  • Retries can finish a failed PyPI upload. A new Release run rebuilds the wheel, and the rebuilt bytes differ from the released asset because the build does not pin file timestamps. The release keeps its original assets (overwrite_files: false), so checking a rebuilt wheel against the release checksum fails and nothing could publish the wheel. The run artifact also expires after 5 days, and this PR removes wheel-publish.yml, which was the manual republish path. Publishing the release asset works from any later run for the same tag.
  • PyPI gets exactly what GitHub serves. PyPI never allows a published file to be replaced, so the uploaded bytes should be the ones users download from the release and the release checksum lists. The checksum check now verifies the published asset instead of comparing this run's build against what this run uploaded.
  • It matches the previous wheel path. wheel-publish.yml checked the wheel against the release's assets before Kitmaker published it. Only the GitLab step is removed.

The release is published before this job runs: publish-pypi needs publish, whose release job uploads release/*.whl and the checksum file, and whose check-publication job fails if that upload did not succeed.

This differs from the other publishers (snap and GHCR use run artifacts; npm rebuilds from the source commit). Those can be corrected or rebuilt from a fixed commit; a PyPI file cannot.

@pimlock
pimlock added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 669b6b6 Nov 3, 2026
84 of 85 checks passed
@pimlock
pimlock deleted the ci/5156-direct-pypi-publishing/pimlock branch November 3, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: publish stable Python wheels directly to PyPI

2 participants