Skip to content

Commit 1623ab5

Browse files
jsturtevantjsturtevant
authored
Add Windows support to Python sdk publishing (#21)
* Add Windows support to Python sdk Add wheel generation and tests the support for publishing Signed-off-by: James Sturtevant <jsturtevant@gmail.com> * Install required packages Signed-off-by: James Sturtevant <jsturtevant@gmail.com> * Add permissions and tests Signed-off-by: James Sturtevant <jsturtevant@gmail.com> * handle cache misses Signed-off-by: James Sturtevant <jsturtevant@gmail.com> --------- Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
1 parent 479d8e5 commit 1623ab5

11 files changed

Lines changed: 179 additions & 35 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 69 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -134,12 +134,10 @@ jobs:
134134
- name: Python SDK Benchmark
135135
run: just benchmark
136136

137-
python-wheelhouse:
138-
name: Python SDK wheelhouse test (${{ matrix.os }})
139-
runs-on: ${{ matrix.os }}
140-
strategy:
141-
matrix:
142-
os: [ubuntu-latest, windows-latest]
137+
# Build all Python packages on Linux (pure + backend wheels).
138+
python-wheelhouse-build-linux:
139+
name: Python wheelhouse build (Linux)
140+
runs-on: ubuntu-latest
143141
steps:
144142
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
145143

@@ -162,14 +160,73 @@ jobs:
162160
- name: Install just
163161
run: cargo install --locked just
164162

165-
- name: Install clang (Linux)
166-
if: runner.os == 'Linux'
163+
- name: Install clang
167164
run: sudo apt-get update && sudo apt-get install -y clang
168165

169-
- name: Install LLVM (Windows)
170-
if: runner.os == 'Windows'
166+
- name: Build all Python packages
167+
run: just python-dist
168+
169+
- uses: actions/upload-artifact@v4
170+
with:
171+
name: python-wheels-linux
172+
path: dist/pythonsdk/
173+
174+
# Build Windows-specific maturin backend wheels only.
175+
python-wheelhouse-build-windows:
176+
name: Python wheelhouse build (Windows)
177+
runs-on: windows-latest
178+
steps:
179+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
180+
181+
- uses: actions-rust-lang/setup-rust-toolchain@150fca883cd4034361b621bd4e6a9d34e5143606 # v1.15.4
182+
with:
183+
cache-key: release
184+
rustflags: ""
185+
186+
- uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
187+
188+
- name: Install Python
189+
run: uv python install 3.12
190+
191+
- name: Install just
192+
run: cargo install --locked just
193+
194+
- name: Install LLVM
171195
run: choco install llvm -y
172196

197+
- name: Build backend wheels
198+
run: just python-dist-backends
199+
200+
- uses: actions/upload-artifact@v4
201+
with:
202+
name: python-wheels-windows
203+
path: dist/pythonsdk/
204+
205+
# Download merged Linux + Windows wheels and run wheelhouse smoke tests.
206+
python-wheelhouse-test:
207+
name: Python wheelhouse test (${{ matrix.os }})
208+
needs: [python-wheelhouse-build-linux, python-wheelhouse-build-windows]
209+
runs-on: ${{ matrix.os }}
210+
strategy:
211+
matrix:
212+
os: [ubuntu-latest, windows-latest]
213+
steps:
214+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
215+
216+
- uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
217+
218+
- name: Install Python
219+
run: uv python install 3.12
220+
221+
- name: Install just
222+
run: cargo install --locked just
223+
224+
- uses: actions/download-artifact@v4
225+
with:
226+
path: dist/pythonsdk/
227+
merge-multiple: true
228+
pattern: python-wheels-*
229+
173230
- name: Enable KVM
174231
if: runner.os == 'Linux' && !env.ACT
175232
run: |
@@ -178,8 +235,8 @@ jobs:
178235
sudo udevadm trigger --name-match=kvm
179236
sudo chmod 666 /dev/kvm
180237
181-
- name: Build release wheels and run wheelhouse test
182-
run: just python-wheelhouse-test
238+
- name: Run wheelhouse tests
239+
run: just python python-wheelhouse-test
183240

184241
javascript-sandbox:
185242
name: JS Sandbox (${{ matrix.os }})

‎.github/workflows/publish.yml‎

Lines changed: 73 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,15 @@ on:
55
tags:
66
- "v*"
77

8+
permissions:
9+
contents: read
10+
811
jobs:
9-
publish:
12+
# Build all Python packages on Linux (pure + backend wheels).
13+
build-linux:
1014
if: ${{ !github.event.act }}
11-
name: Build & publish to PyPI
15+
name: Build Linux packages
1216
runs-on: ubuntu-latest
13-
environment:
14-
name: pypi
15-
permissions:
16-
id-token: write
17-
contents: read
1817
steps:
1918
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
2019

@@ -40,8 +39,70 @@ jobs:
4039
- name: Install clang
4140
run: sudo apt-get update && sudo apt-get install -y clang
4241

43-
- name: Build & Publish
44-
if: ${{ !env.ACT }}
45-
run: |
46-
just python-dist
47-
just python python-publish
42+
- name: Build all Python packages
43+
run: just python-dist
44+
45+
- uses: actions/upload-artifact@v4
46+
with:
47+
name: python-wheels-linux
48+
path: dist/pythonsdk/
49+
50+
# Build Windows-specific maturin backend wheels only.
51+
build-windows:
52+
if: ${{ !github.event.act }}
53+
name: Build Windows backend wheels
54+
runs-on: windows-latest
55+
steps:
56+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
57+
58+
- uses: actions-rust-lang/setup-rust-toolchain@150fca883cd4034361b621bd4e6a9d34e5143606 # v1.15.4
59+
with:
60+
cache-key: release
61+
rustflags: ""
62+
63+
- uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
64+
65+
- name: Install Python
66+
run: uv python install 3.12
67+
68+
- name: Install just
69+
run: cargo install --locked just
70+
71+
- name: Install LLVM
72+
run: choco install llvm -y
73+
74+
- name: Build backend wheels
75+
run: just python-dist-backends
76+
77+
- uses: actions/upload-artifact@v4
78+
with:
79+
name: python-wheels-windows
80+
path: dist/pythonsdk/
81+
82+
# Merge artifacts from both platforms and publish to PyPI.
83+
publish:
84+
if: ${{ !github.event.act }}
85+
name: Publish to PyPI
86+
needs: [build-linux, build-windows]
87+
runs-on: ubuntu-latest
88+
environment:
89+
name: pypi
90+
permissions:
91+
id-token: write
92+
contents: read
93+
steps:
94+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
95+
96+
- uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
97+
98+
- name: Install just
99+
run: cargo install --locked just
100+
101+
- uses: actions/download-artifact@v4
102+
with:
103+
path: dist/pythonsdk/
104+
merge-multiple: true
105+
pattern: python-wheels-*
106+
107+
- name: Publish to PyPI
108+
run: just python python-publish

‎Justfile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@ benchmark: python::python-sandbox-benchmark
4343

4444
python-dist: (wasm::build "release") (js::build "release") python::python-dist
4545

46+
python-dist-backends: wasm::_clean-stale-wasm wasm::guest-compile-wit js::_clean-stale
47+
cargo build --manifest-path src/wasm_sandbox/Cargo.toml --release
48+
cargo build --manifest-path src/javascript_sandbox/Cargo.toml --release
49+
just python python-dist-backends
50+
4651
python-wheelhouse-test: python-dist python::python-wheelhouse-test
4752

4853
examples target=default-target: (wasm::examples target) (js::examples target) python::examples

‎src/sdk/python/Justfile‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,15 @@ python-dist: python-sync-guest-resources
5858
cd {{repo-root}}/src/sdk/python/wasm_guests/python_guest && uv run python -m build --outdir {{python-guest-dist}}
5959
cd {{repo-root}}/src/sdk/python/wasm_guests/javascript_guest && uv run python -m build --outdir {{javascript-guest-dist}}
6060

61+
# Build only the platform-specific maturin backend wheels.
62+
# Used by CI to produce Windows wheels without needing guest binaries.
63+
# Requires sandbox-world.wasm to exist (run `just wasm guest-compile-wit` first).
64+
python-dist-backends: python-sync-env
65+
-{{rmrf}} {{wasm-wheels}}
66+
-{{rmrf}} {{hyperlight-js-wheels}}
67+
cd {{repo-root}}/src/sdk/python/wasm_backend && uv run maturin build --release --out {{wasm-wheels}}
68+
cd {{repo-root}}/src/sdk/python/hyperlight_js_backend && uv run maturin build --release --out {{hyperlight-js-wheels}}
69+
6170
python-publish repository="pypi":
6271
uv publish {{ if repository != "pypi" { "--publish-url https://test.pypi.org/legacy/" } else { "" } }} {{wasm-wheels}}/*
6372
uv publish {{ if repository != "pypi" { "--publish-url https://test.pypi.org/legacy/" } else { "" } }} {{hyperlight-js-wheels}}/*
@@ -74,7 +83,13 @@ python-wheelhouse-test:
7483
--find-links={{wasm-wheels}} \
7584
--find-links={{python-guest-dist}} \
7685
--with "hyperlight-sandbox[wasm,python_guest]" \
77-
python {{repo-root}}/src/sdk/python/tests/wheelhouse_wasm.py
86+
python {{repo-root}}/src/sdk/python/tests/wheelhouse_wasm_python.py
87+
uv run --no-project --no-index \
88+
--find-links={{core-dist}} \
89+
--find-links={{wasm-wheels}} \
90+
--find-links={{javascript-guest-dist}} \
91+
--with "hyperlight-sandbox[wasm,javascript_guest]" \
92+
python {{repo-root}}/src/sdk/python/tests/wheelhouse_wasm_js.py
7893
uv run --no-project --no-index \
7994
--find-links={{core-dist}} \
8095
--find-links={{hyperlight-js-wheels}} \

‎src/sdk/python/core/pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ classifiers = [
1717
"Programming Language :: Python :: 3",
1818
"Programming Language :: Rust",
1919
"Operating System :: POSIX :: Linux",
20+
"Operating System :: Microsoft :: Windows",
2021
]
2122
keywords = ["hyperlight", "wasm", "sandbox", "isolation", "code-execution"]
2223

‎src/sdk/python/hyperlight_js_backend/pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ classifiers = [
1515
"Programming Language :: Python :: 3",
1616
"Programming Language :: Rust",
1717
"Operating System :: POSIX :: Linux",
18+
"Operating System :: Microsoft :: Windows",
1819
]
1920

2021
[tool.maturin]

‎src/sdk/python/tests/wheelhouse_js.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
from hyperlight_sandbox import Sandbox
22

33
sandbox = Sandbox(backend="hyperlight-js")
4-
result = sandbox.run('console.log("wheelhouse install ok")')
4+
result = sandbox.run('console.log("wheelhouse hyperlight-js install ok")')
55
assert result.success, result.stderr
66
print(result.stdout.strip())

‎src/sdk/python/tests/wheelhouse_wasm.py‎

Lines changed: 0 additions & 9 deletions
This file was deleted.

‎src/sdk/python/tests/wheelhouse_wasm_js.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from hyperlight_sandbox import Sandbox
2+
3+
sandbox = Sandbox(backend="wasm", module="javascript_guest.path")
4+
result = sandbox.run('console.log("wheelhouse wasm javascript install ok")')
5+
assert result.success, result.stderr
6+
print(result.stdout.strip())

‎src/sdk/python/tests/wheelhouse_wasm_python.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from hyperlight_sandbox import Sandbox
2+
3+
sandbox = Sandbox(backend="wasm", module="python_guest.path")
4+
result = sandbox.run('print("wheelhouse wasm python install ok")')
5+
assert result.success, result.stderr
6+
print(result.stdout.strip())

0 commit comments

Comments
 (0)