ci: publish a source-only OpenVMM release - #4090
Draft
benhillis wants to merge 2 commits into
Draft
Conversation
OpenVMM is meant to be built and packaged by Linux distributions from a source archive, which makes that configuration a shipping interface. It is also the one configuration nothing in CI covers: every other build provisions native dependencies through `.packages/`, which a packager cannot use. A change that only resolves through `.packages/` therefore breaks downstream packagers silently. Add a job that does what a packager does, in order: 1. assemble a source archive from the commit under test, plus a `SHA256SUMS` covering it; 2. `sha256sum --check --strict`; 3. unpack outside the repository, so nothing can reach back into the checkout; 4. assert there is no `.git` inside; 5. assert the archive's embedded `.openvmm-release.json` matches what it was assembled as; 6. `cargo build --release --locked -p openvmm` with `PROTOC` and `OPENSSL_NO_VENDOR=1`, matching what a spec file does; 7. assert the binary's own `NEEDED` entries name the system OpenSSL. Building the checkout rather than an archive would let this pass on a tree a packager cannot reproduce, since a packager has no `.git` directory and no untracked files. The archive also has to carry its identity in a file, for the same reason. Step 7 is what keeps the build honest: if something acquired a vendored or static OpenSSL the build would still succeed, but the packaged binary would no longer be one the distribution can service. It reads `readelf -d` rather than `ldd` because `ldd` reports the whole transitive closure, and so would be satisfied by an unrelated library pulling OpenSSL in. Assembly is reproducible -- `git archive` is deterministic for a commit, `gzip -n` omits the timestamp, and `tar.umask` is pinned rather than inherited from the machine. Measured at 5m35s on a GitHub-hosted runner, against an existing 11m x64-linux baseline. Dropping debug info from the release profile and disabling incremental compilation is what keeps it there. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Add the pipeline that turns a pushed `openvmm-v*` tag into a GitHub release carrying the source archive and its `SHA256SUMS`, both attested with build provenance. The release is created as a **draft**: nothing becomes public without a human looking at it first. The pipeline is two jobs. The first is the distribution build gate that already runs on every pull request, and the second publishes. Handing the gate an `IdentitySource` is what makes that reuse meaningful: on a release it assembles the tag's own identity, so the archive it proves buildable is byte-identical to the one that ships, rather than a lookalike built under a snapshot version. Both jobs resolve the identity through the same function, so neither can grow a rule the other lacks. Assembly is deterministic, so no artifact has to be passed between them. Resolving the tag in the first job also means a malformed tag fails in seconds rather than after a full build. `parse_release_tag` requires canonical three-component decimal versions; note that `u16::from_str` accepts a leading sign, so `openvmm-v1.2.+3` has to be rejected explicitly or it reaches the archive prefix and the release title verbatim. `publish_gh_release` gains an explicit policy for an existing release, because the two consumers genuinely differ. `vmgstool` publishes from every push to `main` against a tag derived from a version in the tree, so an existing release is its steady state: it gets `Skip`, exactly the behavior it has always had. The OpenVMM release runs once per tag, so an existing release means an earlier attempt already got this far: it gets `Fail`, since quietly reporting success would leave a half-finished release looking like a good one. Assets are never replaced automatically in either case. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ba1c0df-faf0-4a0e-b39d-c3eb0e363fce
Contributor
There was a problem hiding this comment.
🟡 Not ready to approve
There are concrete reliability/operational issues in the new release publish and archive assembly paths (error handling and memory usage) that should be addressed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
Adds a Flowey-driven “source-only” OpenVMM release pipeline that triggers on openvmm-v* tags, assembling a deterministic source tarball + SHA256SUMS, gating it with a distro-style build, and then publishing a draft GitHub release with build provenance attestation.
Changes:
- Introduces a shared “assemble OpenVMM source release” node (deterministic
git archive+gzip -n) and a distro-build CI gate that builds from the assembled archive. - Adds a new GitHub Release pipeline (
release-openvmm) that assembles, attests, and publishes a draft release onopenvmm-v*tags. - Extends the shared
publish_gh_releasenode to support configurable release notes and explicit behavior when a release already exists.
File summaries
| File | Description |
|---|---|
| flowey/flowey_lib_hvlite/src/lib.rs | Exposes the new source-release assembly module. |
| flowey/flowey_lib_hvlite/src/assemble_openvmm_source_release.rs | New reusable node to assemble deterministic source tarball + checksums and parse/validate release tags. |
| flowey/flowey_lib_hvlite/src/_jobs/publish_vmgstool_gh_release.rs | Updates vmgstool publishing to use new publish_gh_release parameters. |
| flowey/flowey_lib_hvlite/src/_jobs/publish_openvmm_gh_release.rs | New job to assemble, attest, and publish OpenVMM source-only GitHub releases. |
| flowey/flowey_lib_hvlite/src/_jobs/mod.rs | Registers new distro-build and OpenVMM release jobs. |
| flowey/flowey_lib_hvlite/src/_jobs/check_distro_build.rs | New CI gate that verifies/distributes build by unpacking and building the assembled source archive. |
| flowey/flowey_lib_common/src/publish_gh_release.rs | Adds release-note control + “on existing release” policy; adjusts gh release view/create flow. |
| flowey/flowey_lib_common/src/lib.rs | Exposes the new provenance attestation node. |
| flowey/flowey_lib_common/src/attest_build_provenance.rs | New node using actions/attest@v4 to generate provenance for release assets. |
| flowey/flowey_hvlite/src/pipelines/release_openvmm.rs | New pipeline definition for tag-triggered OpenVMM source release publish. |
| flowey/flowey_hvlite/src/pipelines/mod.rs | Wires the new release-openvmm CLI/pipeline into Flowey. |
| flowey/flowey_hvlite/src/pipelines/checkin_gates.rs | Adds the distro-build configuration gate to the checkin pipelines. |
| ci-flowey/openvmm-pr.yaml | Autogenerated ADO pipeline updated to include the new distro-build gate job. |
| .github/workflows/openvmm-release.yaml | Autogenerated GitHub Actions workflow for tag-triggered OpenVMM releases. |
| .github/workflows/openvmm-pr.yaml | Autogenerated PR workflow updated to include distro-build job. |
| .github/workflows/openvmm-pr-release.yaml | Autogenerated PR “release” workflow updated to include distro-build job. |
| .github/workflows/openvmm-ci.yaml | Autogenerated CI workflow updated for new/shifted job numbering/artifacts due to added distro-build job. |
| .flowey.toml | Registers the new openvmm-release.yaml generation mapping to ci release-openvmm. |
Review details
- Files reviewed: 18/18 changed files
- Comments generated: 2
- Review effort level: Low
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Comment on lines
+163
to
+167
| let source_archive = output_dir.join(identity.archive_name()); | ||
| let compressed = flowey::shell_cmd!(rt, "gzip -n --best --stdout {source_tar}") | ||
| .output()?; | ||
| fs_err::write(&source_archive, compressed.stdout)?; | ||
| fs_err::remove_file(source_tar)?; |
Comment on lines
+181
to
188
| } else { | ||
| log::debug!( | ||
| "assuming no release exists for tag {tag} in repo {repo}; \ | ||
| `gh release view` exited {} with: {}", | ||
| output.status, | ||
| String::from_utf8_lossy(&output.stderr).trim(), | ||
| ); | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns a pushed
openvmm-v*tag into a GitHub release carrying the source archive and itsSHA256SUMS, both attested with build provenance.Draft on purpose. Split out of #4075 at @jstarks' request:
Agreed, and that discussion is what this PR is for. Stacked on #4075; until that lands, the diff here includes its commit too.
Shape
The first job is the distribution build gate from #4075, unchanged except for one parameter. Handing it an
IdentitySourceis what makes the reuse meaningful: on a release it assembles the tag's own identity, so the archive it proves buildable is byte-identical to the one that ships, rather than a lookalike built under a snapshot version. Both jobs resolve that identity through the same function, so neither can drift by growing a rule the other lacks — and because assembly is deterministic, no artifact has to be passed between them.Resolving the tag in the first job also means a malformed tag fails in seconds instead of after a full build.
What it publishes
openvmm-<VERSION>-source.tar.gzandSHA256SUMS, as a draft release with generated notes. Nothing becomes public without a human looking at it first, which is the current answer to "be intentional about what creates a release" — though probably not the whole answer.Open questions, for the bikeshed
workflow_dispatchonly, or gated behind a protected environment with a required reviewer?openvmm-v*tag? Right now that is the entire trigger, and the release being a draft is the only thing standing behind it.openvmm-v*the tag scheme we want, given it has to coexist withvmgstool-v*and whatever else the repo releases later?Happy to reshape any of this — the assembly and the gate are the parts I would like to keep regardless of how the trigger question lands.
Notes
u16::from_straccepts a leading sign, soopenvmm-v1.2.+3has to be rejected explicitly or it reaches the archive prefix and the release title verbatim.publish_gh_releasegains an explicit policy for an existing release, because the two consumers genuinely differ.vmgstoolpublishes from every push tomainagainst a tag derived from a version in the tree, so an existing release is its steady state — it getsSkip, exactly the behavior it has today. The OpenVMM release runs once per tag, so an existing release means an earlier attempt already got this far — it getsFail, since quietly reporting success would leave a half-finished release looking like a good one. Assets are never replaced automatically in either case.