Skip to content

Commit dc76bfe

Browse files
committed
add major versions HIP
Signed-off-by: Yosh <github@yosh.is>
1 parent 83849ab commit dc76bfe

1 file changed

Lines changed: 128 additions & 0 deletions

File tree

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# HIP 0007 - Major Versions
2+
3+
## Elevator pitch
4+
5+
We are proposing the acceptance criteria for Hyperlight to reach 1.0 status, what
6+
major versions mean for Hyperlight, as well as discuss the process for future
7+
major Hyperlight releases.
8+
9+
## Status quo
10+
11+
Hyperlight is currently at version 0.18.0, with monthly releases happening and
12+
every release including breaking changes. While we do try and minimize the
13+
breakage by deprecating APIs before we remove them entirely, we still cannot
14+
guarantee that users can always upgrade from one version to another.
15+
16+
Versions also carry a social signal. When a project is "pre-1.0" that tends to
17+
imply it is unstable, experimental, or "beta"-quality. We have heard this from
18+
multiple potential users. Though whether this perspective is justified is a
19+
separate discussion.
20+
21+
## What we propose to do about it
22+
23+
We're proposing for Hyperlight to reach 1.0 status and start making API
24+
compatibility guarantees between versions. Within a major version, minor and
25+
patch releases will preserve compatibility for stable public APIs in the host
26+
and guest crates. Upgrading may require rebuilding guest binaries and
27+
regenerating snapshots, but should not require source changes.
28+
29+
What is explicitly excluded from the stability guarantees however is ABI
30+
stability. When upgrading from one minor version to another (e.g. 1.0.0 to 1.1.0)
31+
users will be expected to recompile their guest binaries and regenerate
32+
snapshots. This is necessary to allow Hyperlight to keep evolving the guest-host
33+
communication boundary.
34+
35+
Hyperlight 1.0.0 will not be the last major version we're releasing either. On
36+
some cadence, possibly ~yearly, we will be releasing new major versions of
37+
Hyperlight. This gives us an opportunity to make breaking changes to the API as
38+
needed, for example to add multi-threading support to Hyperlight. This HIP does
39+
not lock us into any specific cadence, all it declares is the intent to release
40+
new major versions beyond 1.0.0.
41+
42+
## Implementation details and plan
43+
44+
### Requirements
45+
46+
There are six areas of work we need to finish before shipping Hyperlight 1.0:
47+
48+
* **aarch64 support:** support ARM64 on macOS, Linux, and Windows, with guest
49+
crashdumps on at least Linux.
50+
* **Snapshot-based lifecycle:** finish the snapshot-first lifecycle, including
51+
creating sandboxes from snapshots and persisting snapshots to disk.
52+
* **IO queue support:** finish integrating packed virtqueues for guest-host IO,
53+
including their behavior across snapshot and restore, and validate the design
54+
by integrating it in hyperlight-wasm.
55+
* **Transport refactor:** finish replacing Flatbuffers with a Serde-based
56+
encoding, keeping serialization details out of the public API.
57+
* **Security, testing, & fuzzing:** complete a security review, address
58+
release-blocking findings, and cover the new IO and snapshot paths with tests
59+
and fuzzing.
60+
* **Public API:** remove deprecated APIs and review the host and guest interfaces
61+
for consistency before committing to supporting them throughout 1.x.
62+
63+
All of these items will be tracked separately in Hyperlight, and some may even
64+
require their own follow-up proposals.
65+
66+
### Release Process
67+
68+
The release itself should be relatively straight-forward. If we do things right,
69+
it shouldn't be too different from most other releases. With the sole exception
70+
that we'll probably want to write a slightly larger release post instead of our
71+
usual one.
72+
73+
Depending on the timing we may want to time the release alongside a CNCF/LF
74+
event, to try and create a moment. However, that shouldn't come at the cost of
75+
quality and readiness, so this is an opportunity only.
76+
77+
### Guaranteeing compatibility
78+
79+
We're already quite careful in how we replace APIs. However, we might want to
80+
take extra care by introducing additional guard rails by for example running
81+
[cargo semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) in CI.
82+
Native guest binaries and snapshots must match the exact Hyperlight version
83+
they run on, including the patch version. We should version-tag both and reject
84+
mismatches explicitly, guarding against ABI compatibility assumptions across
85+
versions (see also: [hyrum's law](https://www.hyrumslaw.com/)).
86+
87+
We'll want to think about all the ways to ensure we can guarantee stability,
88+
while also ensuring we keep the flexibility in all the places we do want it.
89+
And of course also clearly documenting what we do and don't guarantee.
90+
91+
## Frequently Asked Questions
92+
93+
### Why not commit to ABI stability?
94+
95+
Because we've seen what happens when you do. C++ has effectively frozen its ABI,
96+
and it's been paying for it ever since. `std::regex` is famously slow and can't
97+
be fixed, `std::unique_ptr` can't be passed in a register, and when `std::string`
98+
needed to change in C++11 it took a split "dual ABI" in libstdc++ and years of
99+
pain to roll out. Titus Winters' paper
100+
[*ABI: Now or Never*](https://wg21.link/p1863) lays this out well: once you promise
101+
ABI stability, every past mistake becomes permanent. We don't want that for
102+
Hyperlight, especially not while the guest-host boundary is still moving.
103+
104+
So instead we only promise API stability. If you upgrade from 1.0 to 1.1 you'll
105+
need to recompile your guests and regenerate your snapshots, but you shouldn't
106+
need to change any code. Guests targeting WebAssembly through `hyperlight-wasm`
107+
can retain their Wasm binaries across Hyperlight upgrades. The native runtime
108+
still needs to be rebuilt, and any Hyperlight snapshots regenerated.
109+
110+
### Why are you planning to release more major versions?
111+
112+
Because 1.0 isn't the end. It means we're happy enough with the API to commit to
113+
it, not that we think we got everything right. There are things we already know
114+
we want that won't fit in the current design. Multi-threading is the obvious
115+
one: a lot of today's APIs assume a single guest thread, and changing that is
116+
going to break things.
117+
118+
Major versions give us a place to put those changes. Minor releases stay boring
119+
and compatible, and breaking changes get batched up and shipped together when
120+
there's a good reason to. That might be yearly, it might be less often. What we
121+
don't want is to end up stuck with a 1.0 we can never move away from.
122+
123+
### What does Hyperlight 1.0 mean for Guests?
124+
125+
What we're proposing here only affects the `hyperlight-dev/hyperlight` crates.
126+
Hyperlight Wasm, Hyperlight Unikraft, and all other guests are not required to
127+
reach 1.0 status when Hyperlight does. Though a follow-up goal will of course be
128+
to have those reach 1.0 status too.

0 commit comments

Comments
 (0)