Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .agents/skills/release-openshell/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,15 @@ Watch the stable release run and report its result.

## PyPI publishing

Stable releases publish the built Python wheel directly to PyPI from the `publish-pypi` job in `release.yml`, after the other release publishers succeed. The job downloads the wheel from the stable GitHub Release, not the run's build artifact, and checks the stable tag's source commit and the wheel's release checksum. It uses Trusted Publishing with the `pypi` GitHub environment. Configure the `openshell` PyPI project's GitHub publisher with owner `NVIDIA`, repository `OpenShell`, workflow `release.yml`, and environment `pypi`. No PyPI API token or GitLab trigger credentials are required.
Stable releases publish the Python wheel to PyPI from `wheel-publish.yml`, which the `publish-pypi` job in `release-publish.yml` calls after the GitHub Release is created. The workflow downloads the wheel from the stable GitHub Release, not a run's build artifact, and checks the stable tag's source commit and the wheel's release checksum. It uses Trusted Publishing with the `pypi` GitHub environment. PyPI matches the workflow file that contains the publishing job, so configure the `openshell` PyPI project's GitHub publisher with owner `NVIDIA`, repository `OpenShell`, workflow `wheel-publish.yml`, and environment `pypi`. No PyPI API token or GitLab trigger credentials are required.

The environment must allow the branch used to dispatch Release. For a support release, explicitly allow its support branch before dispatching. Required reviewers, if configured, must approve the PyPI job. Stable docs publish after PyPI succeeds. An existing wheel on PyPI is skipped when retrying Release; PyPI does not allow replacing a published wheel. Because the job publishes the release asset, a new Release run for the same tag can finish a failed PyPI upload.
The environment must allow the branch used to dispatch Release or `wheel-publish.yml`. For a support release, explicitly allow its support branch before dispatching. Required reviewers, if configured, must approve the PyPI job. An existing wheel on PyPI is skipped; PyPI does not allow replacing a published wheel.

If a stable release did not reach PyPI, publish the wheel without rerunning Release:

```shell
gh workflow run wheel-publish.yml -R NVIDIA/OpenShell --ref main \
-f release-tag=vX.Y.Z -f source-sha=<tag commit> -f python-version=X.Y.Z
```

Development and prerelease wheels remain GitHub Release assets and do not publish to PyPI.
1 change: 1 addition & 0 deletions .agents/skills/sync-agent-infra/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Run after adding, removing, or renaming a skill or crate; changing issue or PR c
| TUI behavior | `tui-development` |
| Release artifacts or smoke coverage | `test-release-canary` |
| Release cadence, tagging, or qualification workflows | `release-captain` |
| Release workflows or publishers | `release-openshell`, `release-captain`; keep the Release workflows table and the manual-trigger rule in `CI.md` current |
| CI diagnostics | `watch-github-actions` |
| Native Windows builds, runtime, or release branch guidance | `build-openshell-mxc-windows`; keep Windows implementation links on `windows` even in docs merged to `main` |
| Gator supervision | `launch-openshell-gator` |
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release-prerelease.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
concurrency:
group: release-prerelease-tags
cancel-in-progress: false
queue: max

Check failure on line 30 in .github/workflows/release-prerelease.yml

View workflow job for this annotation

GitHub Actions / Actionlint (informational)

unexpected key "queue" for "concurrency" section. expected one of "cancel-in-progress", "group"
outputs:
tag: ${{ steps.tag.outputs.tag }}
source-sha: ${{ steps.tag.outputs.source_sha }}
Expand Down Expand Up @@ -124,6 +124,7 @@
release-tag: ${{ needs.resolve.outputs.tag }}
source-sha: ${{ needs.build.outputs.source_sha }}
semver: ${{ needs.build.outputs.semver }}
python-version: ${{ needs.build.outputs.python_version }}
npm-version: ${{ needs.build.outputs.npm_version }}
is-prerelease: true
secrets:
Expand Down
20 changes: 19 additions & 1 deletion .github/workflows/release-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ on:
description: Release version without the v prefix
required: true
type: string
python-version:
description: Python package version
required: true
type: string
npm-version:
description: TypeScript package version
required: true
Expand Down Expand Up @@ -518,6 +522,19 @@ jobs:
buf push proto --label "${RELEASE_TAG}" \
--source-control-url "${SOURCE_CONTROL_URL}"

publish-pypi:
name: Publish Python Wheel
needs: release
if: ${{ !inputs.is-prerelease }}
permissions:
contents: read
id-token: write
uses: ./.github/workflows/wheel-publish.yml
with:
release-tag: ${{ inputs.release-tag }}
source-sha: ${{ inputs.source-sha }}
python-version: ${{ inputs.python-version }}

publish-snap:
name: Publish Snap
needs: release
Expand Down Expand Up @@ -583,6 +600,7 @@ jobs:
- tag-ghcr-release
- release-helm
- release
- publish-pypi
- publish-snap
- publish-prerelease-snap
- publish-sdk-typescript
Expand All @@ -602,7 +620,7 @@ jobs:
if [[ "${IS_PRERELEASE}" == "true" ]]; then
expected+=(publish-prerelease-snap)
else
expected+=(publish-sdk-typescript tag-sdk-go)
expected+=(publish-pypi publish-sdk-typescript tag-sdk-go)
if [[ "${MAKE_LATEST}" == "true" ]]; then
expected+=(publish-snap)
fi
Expand Down
62 changes: 4 additions & 58 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
concurrency:
group: ${{ inputs.notes-only && format('release-notes-preview-{0}', github.ref) || 'release-stable' }}
cancel-in-progress: false
queue: max

Check failure on line 25 in .github/workflows/release.yml

View workflow job for this annotation

GitHub Actions / Actionlint (informational)

unexpected key "queue" for "concurrency" section. expected one of "cancel-in-progress", "group"

jobs:
preview-notes:
Expand Down Expand Up @@ -153,71 +153,17 @@
release-tag: ${{ needs.promote.outputs.tag }}
source-sha: ${{ needs.build.outputs.source_sha }}
semver: ${{ needs.build.outputs.semver }}
python-version: ${{ needs.build.outputs.python_version }}
npm-version: ${{ needs.build.outputs.npm_version }}
is-prerelease: false
make-latest: ${{ needs.promote.outputs.make-latest == 'true' }}
secrets:
BUF_TOKEN: ${{ secrets.BUF_TOKEN }}
SNAPCRAFT_STORE_CREDENTIALS: ${{ secrets.SNAPCRAFT_STORE_CREDENTIALS }}

publish-pypi:
name: Publish Python Wheel to PyPI
needs: [promote, build, publish]
runs-on: ubuntu-latest
timeout-minutes: 10
environment:
name: pypi
url: https://pypi.org/project/openshell/
permissions:
contents: read
id-token: write
steps:
# Publish the wheel attached to the GitHub Release rather than this run's
# build artifact, so a later run can finish publishing the same bytes.
- name: Download and verify the released wheel
shell: bash
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ needs.promote.outputs.tag }}
SOURCE_SHA: ${{ needs.build.outputs.source_sha }}
PYTHON_VERSION: ${{ needs.build.outputs.python_version }}
run: |
set -euo pipefail
tag_sha=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${RELEASE_TAG}" --jq .sha)
if [[ "${tag_sha}" != "${SOURCE_SHA}" ]]; then
echo "${RELEASE_TAG} points to ${tag_sha}, not ${SOURCE_SHA}" >&2
exit 1
fi
gh release download "${RELEASE_TAG}" -R "${GITHUB_REPOSITORY}" \
--pattern '*.whl' --pattern openshell-checksums-sha256.txt --dir release
shopt -s nullglob
wheels=(release/*.whl)
if [[ "${#wheels[@]}" -ne 1 ]]; then
echo "Expected exactly one Python wheel on ${RELEASE_TAG}, found ${#wheels[@]}" >&2
exit 1
fi
wheel=$(basename "${wheels[0]}")
if [[ "${wheel}" != openshell-"${PYTHON_VERSION}"-*.whl ]]; then
echo "${wheel} is not an openshell ${PYTHON_VERSION} wheel" >&2
exit 1
fi
checksum=$(awk -v wheel="${wheel}" '$2 == wheel { print $1 " " $2 }' release/openshell-checksums-sha256.txt)
if [[ -z "${checksum}" ]]; then
echo "${wheel} is missing from the release checksums" >&2
exit 1
fi
(cd release && sha256sum -c - <<<"${checksum}")
mkdir dist
mv "release/${wheel}" dist/

- name: Publish wheel to PyPI
uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3
with:
skip-existing: true

publish-fern-docs:
name: Sync and Publish Fern Docs
needs: [build, publish, publish-pypi]
needs: [build, publish]
permissions:
contents: write
uses: ./.github/workflows/sync-docs.yml
Expand All @@ -234,7 +180,7 @@

check-publication:
name: Check Stable Publication
needs: [promote, build, release-notes, publish, publish-pypi, publish-fern-docs]
needs: [promote, build, release-notes, publish, publish-fern-docs]
if: ${{ always() && !inputs.notes-only }}
runs-on: ubuntu-latest
timeout-minutes: 2
Expand All @@ -250,7 +196,7 @@

notify-release-failure:
name: Notify Release Failure
needs: [promote, build, release-notes, publish, publish-pypi, publish-fern-docs, check-publication]
needs: [promote, build, release-notes, publish, publish-fern-docs, check-publication]
if: failure()
runs-on: ubuntu-latest
timeout-minutes: 2
Expand Down
92 changes: 92 additions & 0 deletions .github/workflows/wheel-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

name: Publish Wheel

# PyPI Trusted Publishing matches the `job_workflow_ref` claim, which names this
# file for both calls and dispatches. The `openshell` PyPI project trusts
# workflow `wheel-publish.yml` with environment `pypi`.
on:
workflow_call:
inputs:
release-tag:
required: true
type: string
source-sha:
required: true
type: string
python-version:
required: true
type: string
workflow_dispatch:
inputs:
release-tag:
description: Stable tag whose GitHub Release holds the wheel, e.g. v0.1.7
required: true
type: string
source-sha:
description: Commit the tag points to
required: true
type: string
python-version:
description: Python package version, e.g. 0.1.7
required: true
type: string

permissions:
contents: read

jobs:
publish:
name: Publish Python Wheel to PyPI
runs-on: ubuntu-latest
timeout-minutes: 10
environment:
name: pypi
url: https://pypi.org/project/openshell/
permissions:
contents: read
id-token: write
steps:
# Publish the wheel attached to the GitHub Release rather than a run's
# build artifact, so a retry or dispatch publishes the same bytes.
- name: Download and verify the released wheel
shell: bash
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ inputs.release-tag }}
SOURCE_SHA: ${{ inputs.source-sha }}
PYTHON_VERSION: ${{ inputs.python-version }}
run: |
set -euo pipefail
tag_sha=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${RELEASE_TAG}" --jq .sha)
if [[ "${tag_sha}" != "${SOURCE_SHA}" ]]; then
echo "${RELEASE_TAG} points to ${tag_sha}, not ${SOURCE_SHA}" >&2
exit 1
fi
gh release download "${RELEASE_TAG}" -R "${GITHUB_REPOSITORY}" \
--pattern '*.whl' --pattern openshell-checksums-sha256.txt --dir release
shopt -s nullglob
wheels=(release/*.whl)
if [[ "${#wheels[@]}" -ne 1 ]]; then
echo "Expected exactly one Python wheel on ${RELEASE_TAG}, found ${#wheels[@]}" >&2
exit 1
fi
wheel=$(basename "${wheels[0]}")
if [[ "${wheel}" != openshell-"${PYTHON_VERSION}"-*.whl ]]; then
echo "${wheel} is not an openshell ${PYTHON_VERSION} wheel" >&2
exit 1
fi
checksum=$(awk -v wheel="${wheel}" '$2 == wheel { print $1 " " $2 }' release/openshell-checksums-sha256.txt)
if [[ -z "${checksum}" ]]; then
echo "${wheel} is missing from the release checksums" >&2
exit 1
fi
(cd release && sha256sum -c - <<<"${checksum}")
mkdir dist
mv "release/${wheel}" dist/

- name: Publish wheel to PyPI
uses: pypa/gh-action-pypi-publish@7245906fb61fa33a13554f9d5fb856fe61758086 # v1.14.3
with:
skip-existing: true
6 changes: 4 additions & 2 deletions CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -694,21 +694,23 @@ The bot's full administrator documentation is internal to NVIDIA. The only comma

These workflows run after merge to publish dev/tagged artifacts and verify them. They are not PR-gated.

Each step that publishes to an external registry, store, or site lives in its own workflow with both `workflow_call` and `workflow_dispatch`. Release calls it, and a maintainer can dispatch it to finish one missing publication for an existing release without rerunning Release. Dispatch inputs identify an already-published release, such as its tag and source commit; the workflow verifies them before publishing and should skip what is already published. Keep the manual trigger when changing or moving a publisher.

| File | Role |
|---|---|
| `.github/workflows/release-dev.yml` | Publishes the rolling `dev` build on every push to `main`. Builds gateway, sandbox, and supervisor images and binaries, packages, wheels, and pushes the Helm chart as `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev` (plus an immutable `0.0.0-dev.<sha>` pin). Also dispatchable manually. |
| `.github/workflows/release.yml` | Manually promote the latest qualified prerelease for a version and branch, create its stable tag, then rebuild and publish. Support releases preserve shared latest channels. Publishes the prerelease's notes preview, retagged, or generates notes when it is unavailable. The notes-only mode generates a release notes preview artifact. Published artifacts are never overwritten: a rerun fills in what is missing, and a bad published artifact is fixed by releasing a new patch version. |
| `.github/workflows/release-auto-tag.yml` | Weekdays at 14:00 Europe/Zurich, tags the next `-pre.N` on `main` when commits landed since the latest one, then dispatches Pre-release. It skips when the train has no `-pre.1` or its version is already stable, so push each train's `-pre.1` tag by hand. |
| `.github/workflows/release-prerelease.yml` | Pre-release orchestrator, triggered by prerelease tag pushes or dispatched by Release Auto-Tag or manually. A tagless dispatch creates the next candidate on its branch; explicit tags select existing candidates. Builds, then publishes and qualifies in parallel. Qualification failure fails the run and sends the failure notification; published artifacts remain available. |
| `.github/workflows/release-build.yml` | Shared version computation, binary and image builds, and package builds. Validates that the tag matches the caller's release kind. |
| `.github/workflows/release-publish.yml` | Shared artifact assembly, OCI image tags and Helm chart publication, and package publication. Stable releases also publish release-labeled public schemas to Buf (`BUF_TOKEN` required), GitHub Releases, snap, npm, PyPI, and Fern. |
| `.github/workflows/release-publish.yml` | Shared artifact assembly, OCI image tags and Helm chart publication, and package publication. Stable releases also publish release-labeled public schemas to Buf (`BUF_TOKEN` required) and GitHub Releases, and call the PyPI, snap, npm, and Go SDK publishers. |
| `.github/workflows/release-notes.yml` | Shared prompt-driven release notes generation through Inference Hub, used by prerelease previews, notes-only previews, and stable releases. Stable releases reuse the promoted prerelease's preview when available. |
| `.github/workflows/release-qualification.yml` | Reusable qualification suites and result aggregation. Runs security scans, conformance and feature integration, Docker and VM E2E, protobuf compatibility, and advisory compatibility review. |
| `.github/workflows/snap-package.yml` | Builds snap and component artifacts for Release Dev, Release, and Pre-release without Store credentials or publication. |
| `.github/workflows/snap-publish.yml` | Uploads existing snap and component artifacts directly to a Store channel without rebuilding. Release Dev uses `latest/edge`, Pre-release uses `latest/beta`, and Release uses `latest/stable` for the stable rebuild. Uploads queue per channel and architecture, so tagged beta and stable jobs recheck repository tags after earlier publication finishes and before updating their floating channels. Manual dispatch uploads the `snap-linux-*` artifacts from `artifact-run-id`. |
| `.github/workflows/sdk-typescript-publish.yml` | Publishes the TypeScript SDK to GitHub Packages. Called by stable Release; manual dispatch publishes a given tag and npm version. |
| `.github/workflows/sdk-go-tag.yml` | Creates the `sdk/go/vX.Y.Z` tag and asks the Go module proxy to fetch it. Called by stable Release; manual dispatch tags a given version and commit. |
| `.github/workflows/wheel-publish.yml` | Triggers the GitLab pipeline that publishes wheels from a GitHub Release to PyPI. Called by stable Release; manual dispatch republishes a given tag. |
| `.github/workflows/wheel-publish.yml` | Publishes the wheel attached to a stable GitHub Release to PyPI with Trusted Publishing in the `pypi` environment. Called by stable Release; manual dispatch publishes a given tag. |
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The `ubuntu-snap-system-docker` job verifies a compatible system Docker lifecycle, while `ubuntu-snap-docker-preflight` tests fail-fast behavior when Docker is absent or supplied by the Docker snap. The positive snap lane also runs a local policy containment check with the packaged prover. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |

## Required status contexts
Expand Down
1 change: 0 additions & 1 deletion tasks/scripts/sync_docs_website_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ def test_release_workflows_sync_and_publish_docs_once() -> None:
assert tag_job["needs"] == [
"build",
"publish",
"publish-pypi",
]
assert tag_job["uses"] == "./.github/workflows/sync-docs.yml"
assert tag_job["with"]["channel"] == "stable"
Expand Down
Loading