feat!: replace shared-memory init data with virtqueue buffers

Remove the peb backed init-data region and APIs. Use retained virtqueue
ByteChunks for startup data. After removing the init section PEB no
longer contains any data so it can be removed as well.

Author: andreiltd

CHANGELOG.md

@@ -4,6 +4,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Prerelease] - Unreleased
 
+### Changed
+* **Breaking:** Snapshots use ABI 10 and config schema v6. Guests must be rebuilt
+  and existing snapshots regenerated. Guest entry takes the random seed, host
+  page size, and log level.
+
+### Removed
+* **Breaking:** `SandboxBuilder::init_data`, `GuestEnvironment`, `GuestBlob`,
+  `GuestHandle`, `HyperlightPEB`, and `read_n_bytes_from_user_memory`.
+  Pass startup data through a host function and retain its `ByteChunks` reply.
+  `UninitializedSandbox::new` takes a `GuestBinary`.
+
 ## [v0.19.0] - 2026-11-04
 
 ### Added

docs/how-to-build-a-hyperlight-guest-binary.md

@@ -119,6 +119,20 @@ If you mix the manual form with `#[guest_function]`, registrations from the
 macro still happen automatically. Your `hyperlight_main` only needs to do
 whatever extra setup the macros do not cover.
 
+### Startup data
+
+Register a host function before `SandboxBuilder::build` or
+`UninitializedSandbox::evolve`. Call it from `hyperlight_main` and keep its
+`Vec<Bytes>` reply in guest state. Transport is ready before `hyperlight_main`.
+The reply's `ByteChunks` buffers remain valid across calls, snapshots,
+restores, and forks.
+
+For data needed after startup, pass `Vec<Bytes>` to a guest function and retain
+it there. A `Vec<u8>` parameter or return value copies into guest-owned memory.
+Retained buffers occupy transport pool slots. Size the pools for retained data
+plus later calls, or copy large inputs into guest memory as you fetch chunks.
+Retained buffers do not provide custom page permissions.
+
 ### Troubleshooting
 
 #### "duplicate lang item `panic_impl`" error

docs/snapshot-oci-format.md

@@ -32,7 +32,7 @@ Four blob kinds per tag:
 * **manifest** (`application/vnd.oci.image.manifest.v1+json`). Tiny JSON
   pointer record selected via `index.json`. References one config,
   the snapshot blobs, and one transport layer by digest.
-* **config** (`application/vnd.hyperlight.snapshot.config.v5+json`). The
+* **config** (`application/vnd.hyperlight.snapshot.config.v6+json`). The
   snapshot descriptor: arch, hypervisor, CPU vendor, ABI version,
   resume address and captured registers, memory and transport layout,
   registered host functions, the start address, data length, and page-table

docs/snapshot-versioning.md

@@ -14,8 +14,8 @@ A snapshot carries four independently evolvable version markers:
   [src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs](../src/hyperlight_host/src/sandbox/snapshot/file/media_types.rs)).
   This is what the host reads back from a snapshot: the `OutBAction`
   and `VmAction` port numbers, the virtqueue transport layout, the
-  offset and size of each memory region (including the `HyperlightPEB`
-  size), and the calling convention for guest function entry. A change
+  offset and size of each memory region, and the calling convention for
+  guest function entry. A change
   to any of these breaks older snapshots unless the loader adds a
   compat path.
 * **Snapshot blob encoding**, `MT_SNAPSHOT_V2`
@@ -27,16 +27,17 @@ A snapshot carries four independently evolvable version markers:
   (`application/vnd.hyperlight.snapshot.transport.v1`), aliased as
   `MT_TRANSPORT_CURRENT`. This is the binary encoding of canonical
   virtqueue state stored outside the memory layer.
-* **Config schema**, `MT_CONFIG_V5`
-  (`application/vnd.hyperlight.snapshot.config.v5+json`), aliased as
+* **Config schema**, `MT_CONFIG_V6`
+  (`application/vnd.hyperlight.snapshot.config.v6+json`), aliased as
   `MT_CONFIG_CURRENT`. This is the JSON shape of the config blob:
   field names, types, required vs optional, the descriptors the loader
   needs in order to reconstruct the sandbox (memory sizes, buffer
   sizes, `abi_version`, `hyperlight_version`, etc.). Renaming a field,
   changing its type, or adding a required field is a schema change and
   bumps this constant. Version 4 describes snapshot blobs, the blob with
   the page tables, and transport geometry, and requires a transport
-  layer. Version 5 drops the heap size. Config v1 to v4 are incompatible
+  layer. Version 5 drops the heap size. Version 6 contains code and scratch
+  geometry without init-data fields. Config v1 to v5 are incompatible
   with the current ABI.
 
 ABI 6 requires checkpoint readiness after guest initialization and permits
@@ -52,6 +53,9 @@ regenerated.
 ABI 9 removes the heap region from snapshot memory. Earlier snapshots must
 be regenerated.
 
+ABI 10 contains no PEB or init-data region. Guest entry takes the random seed,
+host page size, and log level. Earlier snapshots must be regenerated.
+
 The `OCI_LAYOUT_VERSION` constant is pinned by the OCI image-layout
 spec at `1.0.0`.
 
@@ -83,7 +87,7 @@ Compile-time tripwires in
 [src/hyperlight_host/src/sandbox/snapshot/tripwires.rs](../src/hyperlight_host/src/sandbox/snapshot/tripwires.rs)
 hold a copy of every value that defines the format:
 `SNAPSHOT_ABI_VERSION`, the snapshot and config media-type strings, the
-OCI layout version, the `HyperlightPEB` size, every `OutBAction` and
+OCI layout version, every `OutBAction` and
 `VmAction` discriminant, and `BASE_ADDRESS`. If the source value
 drifts from the copy in `tripwires.rs`, the crate fails to compile.
 

src/hyperlight_common/src/lib.rs

@@ -24,9 +24,6 @@ pub mod layout;
 // cbindgen:ignore
 pub mod log_level;
 
-/// cbindgen:ignore
-pub mod mem;
-
 /// cbindgen:ignore
 pub mod outb;
 

src/hyperlight_common/src/mem.rs

@@ -19,7 +19,3 @@ pub mod paging;
 pub mod prim_alloc;
 pub mod transport;
 pub mod types;
-
-pub mod guest_handle {
-    pub mod handle;
-}

src/hyperlight_guest/src/guest_handle/handle.rs

@@ -56,22 +56,16 @@ unsafe fn init_vbar() {
 /// Machine-specific initialisation; calls [`crate::generic_init`]
 /// once VBAR and the main stack have been set up
 #[unsafe(no_mangle)]
-pub extern "C" fn entrypoint(peb_address: u64, seed: u64, ops: u64, max_log_level: u64) -> ! {
+pub extern "C" fn entrypoint(seed: u64, ops: u64, max_log_level: u64) -> ! {
     unsafe {
         init_vbar();
         let stack_top = crate::init::init_stack();
-        pivot_stack(peb_address, seed, ops, max_log_level, stack_top);
+        pivot_stack(seed, ops, max_log_level, stack_top);
     }
 }
 
 unsafe extern "C" {
-    unsafe fn pivot_stack(
-        peb_address: u64,
-        seed: u64,
-        ops: u64,
-        max_log_level: u64,
-        stack_top: u64,
-    ) -> !;
+    unsafe fn pivot_stack(seed: u64, ops: u64, max_log_level: u64, stack_top: u64) -> !;
 }
 
 core::arch::global_asm!("
@@ -83,7 +77,7 @@ core::arch::global_asm!("
     msr SPSel, #1\n
     mov sp, x5\n
     msr SPSel, #0\n
-    mov sp, x4\n
+    mov sp, x3\n
     bl {generic_init}\n
     ldr x1, ={hlt_addr}\n
     str x0, [x1]\n

src/hyperlight_guest/src/lib.rs

@@ -94,7 +94,7 @@ unsafe fn init_tss(pc: *mut ProcCtrl) {
 /// Machine-specific initialisation; calls [`crate::generic_init`]
 /// once stack, CoW, etc have been set up.
 #[unsafe(no_mangle)]
-pub extern "C" fn entrypoint(peb_address: u64, seed: u64, ops: u64, max_log_level: u64) {
+pub extern "C" fn entrypoint(seed: u64, ops: u64, max_log_level: u64) {
     unsafe {
         // Allocate a VA for processor control structures which must
         // survive snapshotting at the same VA.
@@ -108,18 +108,12 @@ pub extern "C" fn entrypoint(peb_address: u64, seed: u64, ops: u64, max_log_leve
         // Architecture early init is complete! We pivot now to
         // executing on the main stack, and jump into generic
         // initialisation code in lib.rs
-        pivot_stack(peb_address, seed, ops, max_log_level, stack_top);
+        pivot_stack(seed, ops, max_log_level, stack_top);
     }
 }
 
 unsafe extern "C" {
-    unsafe fn pivot_stack(
-        peb_address: u64,
-        seed: u64,
-        ops: u64,
-        max_log_level: u64,
-        stack_top: u64,
-    ) -> !;
+    unsafe fn pivot_stack(seed: u64, ops: u64, max_log_level: u64, stack_top: u64) -> !;
 }
 
 // Mark this as the bottom-most frame for debuggers:
@@ -135,7 +129,7 @@ core::arch::global_asm!("
     pivot_stack:\n
     .cfi_startproc\n
     .cfi_undefined rip\n
-    mov rsp, r8\n
+    mov rsp, rcx\n
     xor ebp, ebp\n
     call {generic_init}\n
     mov dx, {halt_port}\n