Skip to content

Commit cb2a3f0

Browse files
committed
Publish dev pre-releases to PyPI on every merge to main ENG-234
Set up a testing deployment path for the client. Every merge to main now publishes a PEP 440 dev release (0.3.10.dev7) to PyPI, so the tip of main is always installable from the real index. Dev releases are pre-releases, so pip and uv skip them during resolution and ordinary installs keep getting the latest stable. ENG-234 originally proposed TestPyPI. Real PyPI is used instead because pre-releases there are already invisible to normal installs, while giving real dependency resolution and the real install path -- TestPyPI cannot, since the runtime dependencies do not exist on it. Versioning: hatch-vcs yields "0.3.10.dev16+gb710c2982" between tags, and the "+local" segment is rejected on upload. The workflow keeps setuptools-scm's next-version guess, drops the local segment and uses the run number as the dev counter, so versions are strictly increasing and re-runs never collide. A commit that is itself a release tag is skipped rather than published as X.Y.Z.dev<n>, which would sort below the release. Pre-release builds now default to https://api.test.codeplain.ai so an unreleased client is exercised against the unreleased backend. The default was hardcoded in three places; all three now read a single resolver keyed off the version baked in at build time. --api still overrides it. The PEP 440 check avoids the packaging library on purpose: it is only a dev-tool transitive here, so it is absent from a real user install.
1 parent b710c29 commit cb2a3f0

7 files changed

Lines changed: 296 additions & 5 deletions

File tree

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
name: Publish dev build to PyPI
2+
3+
# Every merge to main publishes a PEP 440 dev release (e.g. 0.3.10.dev7) to
4+
# PyPI, so the latest main is always installable from the real index with real
5+
# dependency resolution. Dev releases are pre-releases: pip and uv skip them
6+
# unless asked, so ordinary installs keep getting the latest stable.
7+
#
8+
# Tagged releases (including rc/beta/alpha) are handled by publish-to-pypi.yml.
9+
on:
10+
push:
11+
branches: [main]
12+
# Ad-hoc runs, e.g. re-publishing after a PyPI outage.
13+
workflow_dispatch:
14+
15+
# Least privilege by default: jobs get no GITHUB_TOKEN scopes unless they
16+
# opt in below.
17+
permissions: {}
18+
19+
# Serialize publishes so versions land in merge order. Queued runs are not
20+
# cancelled — each merge should still get its own dev build.
21+
concurrency:
22+
group: publish-dev-to-pypi
23+
cancel-in-progress: false
24+
25+
jobs:
26+
build:
27+
name: Build dev distribution
28+
runs-on: ubuntu-latest
29+
permissions:
30+
contents: read
31+
outputs:
32+
should-publish: ${{ steps.version.outputs.should-publish }}
33+
34+
steps:
35+
- uses: actions/checkout@v4
36+
with:
37+
# hatch-vcs derives the version from git tags, so we need full history.
38+
fetch-depth: 0
39+
40+
- name: Install uv
41+
# uv provisions its own Python (honoring requires-python), so no
42+
# separate actions/setup-python step is needed.
43+
run: |
44+
curl -LsSf https://astral.sh/uv/install.sh | sh
45+
echo "$HOME/.local/bin" >> $GITHUB_PATH
46+
47+
- name: Compute dev version
48+
id: version
49+
# Between releases hatch-vcs produces something like
50+
# "0.3.10.dev16+gb710c2982". The "+g..." local version segment is valid
51+
# PEP 440 but is rejected by PyPI on upload, so it has to go. We keep
52+
# setuptools-scm's own next-version guess (0.3.10 — the patch bump
53+
# after the latest tag) rather than re-deriving it here, and use the
54+
# run number as the dev counter: it is strictly increasing, so each
55+
# merge sorts after the last and re-runs never collide.
56+
run: |
57+
RAW=$(uvx --from setuptools-scm python -c \
58+
"from setuptools_scm import get_version; print(get_version())")
59+
echo "setuptools-scm reports: $RAW"
60+
61+
# On a commit that IS a release tag, setuptools-scm returns a clean
62+
# version with no .dev component (e.g. "0.3.9" or "0.4.0rc1").
63+
# Appending ".dev<run>" to that yields 0.3.9.dev42, which sorts BELOW
64+
# the released 0.3.9 — newer code would look older. A tagged commit
65+
# is a real release and is published by publish-to-pypi.yml, so there
66+
# is no dev build to make here.
67+
case "$RAW" in
68+
*.dev*) ;;
69+
*)
70+
echo "::notice::$RAW is a tagged release; skipping dev build."
71+
echo "should-publish=false" >> $GITHUB_OUTPUT
72+
exit 0
73+
;;
74+
esac
75+
76+
BASE=${RAW%%+*}
77+
BASE=${BASE%%.dev*}
78+
VERSION="${BASE}.dev${GITHUB_RUN_NUMBER}"
79+
echo "Derived version: $RAW -> $VERSION"
80+
echo "SETUPTOOLS_SCM_PRETEND_VERSION=$VERSION" >> $GITHUB_ENV
81+
echo "should-publish=true" >> $GITHUB_OUTPUT
82+
83+
- name: Build package
84+
if: steps.version.outputs.should-publish == 'true'
85+
# hatch-vcs reads SETUPTOOLS_SCM_PRETEND_VERSION in place of the tag.
86+
run: |
87+
uv build
88+
echo "Built distributions:"
89+
ls -l dist/
90+
91+
- name: Store the distribution packages
92+
if: steps.version.outputs.should-publish == 'true'
93+
uses: actions/upload-artifact@v4
94+
with:
95+
name: dev-package-distributions
96+
path: dist/
97+
98+
publish-dev:
99+
name: Publish dev build to PyPI
100+
needs: build
101+
if: needs.build.outputs.should-publish == 'true'
102+
runs-on: ubuntu-latest
103+
environment:
104+
name: pypi
105+
url: https://pypi.org/p/codeplain
106+
permissions:
107+
# Mint a short-lived OIDC token so PyPI Trusted Publishing can
108+
# authenticate this run — no long-lived API token needed.
109+
id-token: write
110+
111+
env:
112+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
113+
114+
steps:
115+
- name: Download distribution packages
116+
uses: actions/download-artifact@v4
117+
with:
118+
name: dev-package-distributions
119+
path: dist/
120+
121+
- name: Publish to PyPI
122+
uses: pypa/gh-action-pypi-publish@release/v1
123+
with:
124+
# PyPI uploads are immutable, so a re-run of an already-published
125+
# version should be a no-op, not a red build.
126+
skip-existing: true
127+
128+
# Success is deliberately quiet: this runs on every merge to main, which
129+
# is already announced elsewhere. Only failures ping.
130+
- name: Notify Slack (failure)
131+
if: ${{ failure() && env.SLACK_WEBHOOK_URL != '' }}
132+
run: |
133+
curl -s -X POST "$SLACK_WEBHOOK_URL" \
134+
-H 'Content-type: application/json' \
135+
--data "{\"text\":\"*Codeplain Client*: Publishing dev build to PyPI FAILED :x: (commit \`${{ github.sha }}\`). See <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|the run log>.\"}"

‎.github/workflows/publish-to-pypi.yml‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,10 +91,23 @@ jobs:
9191

9292
- name: Notify Slack (success)
9393
if: ${{ success() && env.SLACK_WEBHOOK_URL != '' }}
94+
env:
95+
# GitHub marks the release itself as a pre-release; the tag
96+
# (v0.4.0rc1) carries the same intent for PEP 440.
97+
IS_PRERELEASE: ${{ github.event.release.prerelease }}
98+
TAG: ${{ github.ref_name }}
9499
run: |
100+
VERSION="${TAG#v}"
101+
if [ "$IS_PRERELEASE" = "true" ]; then
102+
# pip and uv skip pre-releases by default, so "--upgrade" would
103+
# NOT pick this up. Tell people the command that actually works.
104+
TEXT="*Codeplain Client*: pre-release \`$TAG\` published to PyPI :test_tube:. Normal installs are unaffected — try it with \`pip install --pre codeplain==$VERSION\`."
105+
else
106+
TEXT="*Codeplain Client*: \`$TAG\` published to PyPI :white_check_mark:. Install with \`pip install --upgrade codeplain\`."
107+
fi
95108
curl -s -X POST "$SLACK_WEBHOOK_URL" \
96109
-H 'Content-type: application/json' \
97-
--data "{\"text\":\"*Codeplain Client*: \`${{ github.ref_name }}\` published to PyPI :white_check_mark:. Install with \`pip install --upgrade codeplain\`.\"}"
110+
--data "$(jq -n --arg t "$TEXT" '{text:$t}')"
98111
99112
- name: Notify Slack (failure)
100113
if: ${{ failure() && env.SLACK_WEBHOOK_URL != '' }}

‎CONTRIBUTING.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,3 +67,71 @@ Include:
6767
* the related issue, when applicable,
6868
* how the change was tested,
6969
* the `plainlang-examples` example used, when relevant.
70+
71+
## Pre-Releases
72+
73+
Everything below happens on real PyPI — there is no separate test index.
74+
Pre-releases are invisible to ordinary installs: pip and uv skip them during
75+
resolution, so `pip install codeplain` and `pip install --upgrade codeplain`
76+
always serve the latest *stable* release.
77+
78+
### Dev builds (automatic)
79+
80+
Every merge to `main` publishes a dev release such as `0.3.10.dev7`, so the tip
81+
of main is always installable from the real index:
82+
83+
```bash
84+
pip install --pre codeplain==0.3.10.dev7
85+
uv tool install --prerelease allow codeplain
86+
```
87+
88+
Versions are `<next-patch>.dev<run-number>`, so they sort after the last
89+
release and before the next one. See `.github/workflows/publish-dev-to-pypi.yml`.
90+
If that workflow is ever dispatched against a commit that is itself a release
91+
tag, it skips rather than publishing a dev build that would sort below the
92+
release.
93+
94+
### Release candidates (manual)
95+
96+
To get a specific build in front of people before it becomes the default
97+
install, cut a PEP 440 pre-release.
98+
99+
Tag and publish a GitHub release as usual, but with a pre-release version and
100+
the release marked "pre-release":
101+
102+
```
103+
v0.4.0a1 alpha
104+
v0.4.0b1 beta
105+
v0.4.0rc1 release candidate
106+
```
107+
108+
`publish-to-pypi.yml` handles these with no special casing: hatch-vcs reads the
109+
version straight off the tag (`v0.4.0rc1` -> `0.4.0rc1`).
110+
111+
Opt in the same way:
112+
113+
```bash
114+
pip install --pre codeplain==0.4.0rc1
115+
```
116+
117+
### Which API a build talks to
118+
119+
Pre-release builds default to the **test** API, so an unreleased client is
120+
exercised against the unreleased backend:
121+
122+
| build | default API |
123+
| --- | --- |
124+
| stable release (`0.3.9`) | `https://api.codeplain.ai` |
125+
| dev build (`0.3.10.dev7`) | `https://api.test.codeplain.ai` |
126+
| alpha/beta/rc (`0.4.0rc1`) | `https://api.test.codeplain.ai` |
127+
128+
`--api` always overrides this. Resolution lives in
129+
`system_config._resolve_default_api_url`, keyed off the version baked into the
130+
package at build time — so nothing needs configuring at install time.
131+
132+
A source checkout reports the nearest git tag as its version, so it is treated
133+
as stable and keeps pointing at production; use `--api` to aim it elsewhere.
134+
135+
Because every build above goes to real PyPI, dependency resolution and the
136+
install path are exactly what an end user gets — the main reason this is
137+
preferred over publishing to TestPyPI.

‎plain2code.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -345,7 +345,7 @@ def main(): # noqa: C901
345345
sys.exit(1)
346346

347347
if not args.api:
348-
args.api = "https://api.codeplain.ai"
348+
args.api = system_config.default_api_url
349349

350350
try:
351351
print_status(args.api_key, args.api, system_config.client_version)
@@ -389,7 +389,7 @@ def main(): # noqa: C901
389389
event_bus = EventBus()
390390

391391
if not args.api:
392-
args.api = "https://api.codeplain.ai"
392+
args.api = system_config.default_api_url
393393

394394
run_state = RunState(spec_filename=args.filename, replay_with=args.replay_with)
395395

‎plain2code_arguments.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
from plain2code_console import console
77
from plain2code_exceptions import AmbiguousConfigFileError
88
from plain2code_read_config import get_args_from_config
9+
from system_config import system_config
910

1011
# Attribute on the parsed Namespace mapping each argument dest to its source:
1112
# "cli" (explicit on the command line), "config" (from config.yaml), or
@@ -329,8 +330,10 @@ def create_parser():
329330
"--api",
330331
type=str,
331332
nargs="?",
332-
const="https://api.codeplain.ai",
333-
help="Alternative base URL for the API. Default: `https://api.codeplain.ai`",
333+
# Pre-release builds default to the test API; see
334+
# system_config._resolve_default_api_url.
335+
const=system_config.default_api_url,
336+
help=f"Alternative base URL for the API. Default: `{system_config.default_api_url}`",
334337
)
335338
_add_arg(
336339
parser,

‎system_config.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import importlib.resources
22
import os
3+
import re
34
import sys
45

56
import yaml
@@ -10,6 +11,15 @@
1011
PRODUCTION_ENV = "production"
1112
DEVELOPMENT_ENV = "development"
1213

14+
PRODUCTION_API_URL = "https://api.codeplain.ai"
15+
TEST_API_URL = "https://api.test.codeplain.ai"
16+
17+
# PEP 440 pre-release markers, matched against the end of the release segment:
18+
# alpha/beta/release-candidate (0.4.0a1, 0.4.0b2, 0.4.0rc1) or a dev release
19+
# (0.3.10.dev7, published on every merge to main). ".postN" alone is NOT a
20+
# pre-release and must not match.
21+
_PRERELEASE_RE = re.compile(r"(?:a|b|rc)\d+$|\.dev\d+$", re.IGNORECASE)
22+
1323

1424
def _resolve_version() -> str:
1525
"""Resolve the client version.
@@ -79,9 +89,33 @@ def _resolve_environment(installed: bool) -> str:
7989
return PRODUCTION_ENV if installed else DEVELOPMENT_ENV
8090

8191

92+
def _is_prerelease(version: str) -> bool:
93+
"""Return True if ``version`` is a PEP 440 pre-release (alpha, beta, rc or dev).
94+
95+
Deliberately dependency-free rather than using ``packaging``: that library
96+
is not a runtime dependency (it only reaches this checkout via dev tools),
97+
so it is absent from a real user install. The only versions that reach here
98+
are the ones our own build pipeline produces.
99+
"""
100+
# Drop any local version segment ("+gb710c298") before inspecting.
101+
return _PRERELEASE_RE.search(version.split("+", 1)[0]) is not None
102+
103+
104+
def _resolve_default_api_url(version: str) -> str:
105+
"""Resolve the API this client talks to when ``--api`` is not given.
106+
107+
Pre-release builds — the dev build published on every merge to main, and
108+
any alpha/beta/rc release — default to the test API, so an unreleased
109+
client is exercised against the unreleased backend. Stable releases default
110+
to production. ``--api`` always wins over this.
111+
"""
112+
return TEST_API_URL if _is_prerelease(version) else PRODUCTION_API_URL
113+
114+
82115
__version__ = _resolve_version()
83116
__installed__ = _resolve_installed()
84117
__environment__ = _resolve_environment(__installed__)
118+
__default_api_url__ = _resolve_default_api_url(__version__)
85119

86120

87121
class SystemConfig:
@@ -95,6 +129,7 @@ def __init__(self):
95129
self.client_version = __version__
96130
self.installed = __installed__
97131
self.environment = __environment__
132+
self.default_api_url = __default_api_url__
98133
self.error_messages = self.config["error_messages"]
99134

100135
def _load_config(self):

‎tests/test_system_config.py‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,11 @@
55
from system_config import (
66
DEVELOPMENT_ENV,
77
ENVIRONMENT_VAR,
8+
PRODUCTION_API_URL,
89
PRODUCTION_ENV,
10+
TEST_API_URL,
11+
_is_prerelease,
12+
_resolve_default_api_url,
913
_resolve_environment,
1014
system_config,
1115
)
@@ -56,3 +60,36 @@ def test_system_config_exposes_resolved_environment():
5660

5761
if __name__ == "__main__":
5862
pytest.main([__file__])
63+
64+
65+
@pytest.mark.parametrize(
66+
"version",
67+
[
68+
"0.3.10.dev7", # dev build published on every merge to main
69+
"0.3.10.dev16+gb710c2982", # local segment must not defeat detection
70+
"0.4.0a1",
71+
"0.4.0b2",
72+
"0.4.0rc1",
73+
"0.3.9.post1.dev2",
74+
],
75+
)
76+
def test_prerelease_versions_default_to_test_api(version):
77+
assert _is_prerelease(version) is True
78+
assert _resolve_default_api_url(version) == TEST_API_URL
79+
80+
81+
@pytest.mark.parametrize(
82+
"version",
83+
[
84+
"0.3.9",
85+
"1.0.0",
86+
"0.3.9.post1", # a post-release is NOT a pre-release
87+
],
88+
)
89+
def test_stable_versions_default_to_production_api(version):
90+
assert _is_prerelease(version) is False
91+
assert _resolve_default_api_url(version) == PRODUCTION_API_URL
92+
93+
94+
def test_system_config_exposes_default_api_url():
95+
assert system_config.default_api_url in (PRODUCTION_API_URL, TEST_API_URL)

0 commit comments

Comments
 (0)