Skip to content

Commit 5c441b1

Browse files
committed
feat(host): add namespaced snapshot metadata
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
1 parent 83bbc9c commit 5c441b1

7 files changed

Lines changed: 217 additions & 2 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
55
## [Prerelease] - Unreleased
66

77
### Added
8+
* Namespaced application metadata on immutable snapshots.
89
* Per-direction virtqueue configuration through `SandboxConfiguration` and
910
`SandboxBuilder`, with allocations included in scratch sizing.
1011
* Shared virtqueue framing with a 12-byte `MsgHeader` and external byte values.

‎docs/snapshot-oci-format.md‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,8 @@ Four blob kinds per tag:
3535
* **config** (`application/vnd.hyperlight.snapshot.config.v3+json`). The
3636
snapshot descriptor: arch, hypervisor, CPU vendor, ABI version,
3737
resume address and captured registers, memory and transport layout,
38-
registered host functions, and snapshot generation counter. Loaded
39-
eagerly and fully parsed.
38+
registered host functions, snapshot generation counter, and namespaced
39+
application metadata. Loaded eagerly and fully parsed.
4040
* **layer / memory** (`application/vnd.hyperlight.snapshot.memory.v1`).
4141
The raw guest memory image, exactly `memory_size` bytes. mmap'd on
4242
restore.
@@ -51,6 +51,18 @@ The runtime queue protocol and canonical checkpoint are described in
5151
Blob filenames are the sha256 of the blob bytes, so identical blobs
5252
across tags are stored once.
5353

54+
## Application metadata
55+
56+
`Snapshot::with_metadata` creates a snapshot that shares the source snapshot's
57+
sandbox state and stores a serializable value under an application-owned
58+
namespace. `Snapshot::metadata` deserializes the value for that namespace.
59+
The source snapshot remains unchanged.
60+
61+
Metadata is stored in the config blob as a JSON object with namespaces as
62+
keys. Snapshots without metadata omit the field. The loader treats a missing
63+
field as an empty map, so snapshots written before metadata support remain
64+
loadable. Metadata counts toward the config blob size limit.
65+
5466
## Transport framing
5567

5668
The transport layer is at most 2 MiB. Integers are unsigned and little-endian.

‎docs/snapshot-versioning.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,9 @@ The config blob also records `hyperlight_version`, the `CARGO_PKG_VERSION`
5050
of the host crate at write time. This is informational only. The loader
5151
records it for diagnostics and does not gate loading on it.
5252

53+
The optional `metadata` field stores application-owned values by namespace.
54+
Writers omit an empty map, and readers treat a missing field as empty.
55+
5356
## Compatibility cleanup
5457

5558
Record compatibility paths here when a future hard snapshot break can remove

‎src/hyperlight_host/src/sandbox/snapshot/file/config.rs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
// SPDX-License-Identifier: Apache-2.0
22
// Copyright 2025 The Hyperlight Authors.
33

4+
use std::collections::BTreeMap;
5+
46
use hyperlight_common::flatbuffer_wrappers::function_types::{ParameterType, ReturnType};
57
use hyperlight_common::flatbuffer_wrappers::host_function_definition::HostFunctionDefinition;
68
use hyperlight_common::vmem::PAGE_SIZE;
@@ -199,6 +201,9 @@ pub(super) struct OciSnapshotConfig {
199201
/// `SCRATCH_TOP_SNAPSHOT_GENERATION_OFFSET` is continuous across
200202
/// save/load.
201203
pub(super) snapshot_generation: u64,
204+
/// Application-owned metadata keyed by namespace.
205+
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
206+
pub(super) metadata: BTreeMap<String, serde_json::Value>,
202207
}
203208

204209
/// Sizes and permissions of the regions inside the snapshot blob,
@@ -847,6 +852,7 @@ mod tests {
847852
memory_size: PAGE_SIZE as u64,
848853
host_functions: Vec::new(),
849854
snapshot_generation: 0,
855+
metadata: BTreeMap::new(),
850856
}
851857
}
852858

‎src/hyperlight_host/src/sandbox/snapshot/file/mod.rs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -653,6 +653,7 @@ impl Snapshot {
653653
memory_size: self.state.memory.mem_size() as u64,
654654
host_functions,
655655
snapshot_generation: self.state.snapshot_generation,
656+
metadata: self.metadata.clone(),
656657
})
657658
}
658659

@@ -965,6 +966,7 @@ impl Snapshot {
965966
host_functions,
966967
virtq: Some(virtq),
967968
}),
969+
metadata: cfg.metadata,
968970
})
969971
}
970972
}

‎src/hyperlight_host/src/sandbox/snapshot/file_tests.rs‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
use std::sync::Arc;
99

1010
use hyperlight_testing::{c_simple_guest_as_pathbuf, simple_guest_as_pathbuf};
11+
use serde::{Deserialize, Serialize};
1112
use serde_json::Value;
1213
use sha2::{Digest as _, Sha256};
1314

@@ -51,6 +52,12 @@ fn create_snapshot() -> Arc<Snapshot> {
5152
sbox.snapshot().unwrap()
5253
}
5354

55+
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
56+
struct TestMetadata {
57+
format_version: u32,
58+
runtime: String,
59+
}
60+
5461
/// `Result::unwrap_err` requires `T: Debug`, but `Snapshot` is not
5562
/// `Debug`. This wrapper is the test-side equivalent.
5663
#[track_caller]
@@ -144,6 +151,45 @@ fn from_snapshot_in_memory_pre_init() {
144151
assert_eq!(result, 0);
145152
}
146153

154+
#[test]
155+
fn snapshot_metadata_is_immutable_and_shares_state() {
156+
let snapshot = create_snapshot();
157+
let metadata = TestMetadata {
158+
format_version: 1,
159+
runtime: "test".to_string(),
160+
};
161+
162+
let with_metadata = snapshot
163+
.with_metadata("snapshot-metadata-namespace-v1", &metadata)
164+
.unwrap();
165+
166+
assert!(Arc::ptr_eq(&snapshot.state, &with_metadata.state));
167+
assert_eq!(
168+
snapshot
169+
.metadata::<TestMetadata>("snapshot-metadata-namespace-v1")
170+
.unwrap(),
171+
None
172+
);
173+
assert_eq!(
174+
with_metadata
175+
.metadata::<TestMetadata>("snapshot-metadata-namespace-v1")
176+
.unwrap(),
177+
Some(metadata)
178+
);
179+
}
180+
181+
#[test]
182+
fn snapshot_metadata_namespaces_are_independent() {
183+
let snapshot = create_snapshot();
184+
let first = snapshot.with_metadata("first", &1_u32).unwrap();
185+
let second = first.with_metadata("second", &2_u32).unwrap();
186+
let replaced = second.with_metadata("first", &3_u32).unwrap();
187+
188+
assert_eq!(second.metadata::<u32>("first").unwrap(), Some(1));
189+
assert_eq!(replaced.metadata::<u32>("first").unwrap(), Some(3));
190+
assert_eq!(replaced.metadata::<u32>("second").unwrap(), Some(2));
191+
}
192+
147193
// Round-trip via OCI layout on disk.
148194

149195
#[test]
@@ -164,6 +210,59 @@ fn round_trip_save_load_call() {
164210
assert_eq!(result, "hello\n");
165211
}
166212

213+
#[test]
214+
fn snapshot_metadata_round_trip() {
215+
let metadata = TestMetadata {
216+
format_version: 1,
217+
runtime: "test".to_string(),
218+
};
219+
let snapshot = create_snapshot()
220+
.with_metadata("snapshot-metadata-namespace-v1", &metadata)
221+
.unwrap();
222+
let dir = tempfile::tempdir().unwrap();
223+
let path = dir.path().join("metadata");
224+
225+
snapshot
226+
.save(&path, &OciTag::new("latest").unwrap())
227+
.unwrap();
228+
let config: Value =
229+
serde_json::from_slice(&std::fs::read(find_config_blob(&path)).unwrap()).unwrap();
230+
assert_eq!(
231+
config["metadata"],
232+
serde_json::json!({
233+
"snapshot-metadata-namespace-v1": {
234+
"format_version": 1,
235+
"runtime": "test",
236+
}
237+
})
238+
);
239+
let loaded = Snapshot::checked_load(&path, OciTag::new("latest").unwrap()).unwrap();
240+
241+
assert_eq!(
242+
loaded
243+
.metadata::<TestMetadata>("snapshot-metadata-namespace-v1")
244+
.unwrap(),
245+
Some(metadata)
246+
);
247+
}
248+
249+
#[test]
250+
fn snapshot_without_metadata_omits_config_field() {
251+
let snapshot = create_snapshot();
252+
let dir = tempfile::tempdir().unwrap();
253+
let path = dir.path().join("metadata");
254+
255+
snapshot
256+
.save(&path, &OciTag::new("latest").unwrap())
257+
.unwrap();
258+
let config: Value =
259+
serde_json::from_slice(&std::fs::read(find_config_blob(&path)).unwrap()).unwrap();
260+
let loaded = Snapshot::checked_load(&path, OciTag::new("latest").unwrap()).unwrap();
261+
262+
assert!(config.get("metadata").is_none());
263+
assert_eq!(loaded.metadata::<Value>("missing").unwrap(), None);
264+
}
265+
167266
/// A pre-existing snapshot blob with the right length but wrong
168267
/// bytes (corruption, partial copy, foreign tool) must be detected
169268
/// and replaced by `save`, not silently trusted.

‎src/hyperlight_host/src/sandbox/snapshot/mod.rs‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ pub enum NextAction {
5959
/// An immutable snapshot of sandbox state.
6060
pub struct Snapshot {
6161
state: Arc<SnapshotState>,
62+
// Stable key order keeps config bytes deterministic.
63+
metadata: BTreeMap<String, serde_json::Value>,
6264
}
6365

6466
/// Immutable sandbox state held by a snapshot.
@@ -409,6 +411,7 @@ impl Snapshot {
409411
},
410412
virtq: None,
411413
}),
414+
metadata: BTreeMap::new(),
412415
})
413416
}
414417

@@ -601,9 +604,98 @@ impl Snapshot {
601604
host_functions,
602605
virtq,
603606
}),
607+
metadata: BTreeMap::new(),
604608
})
605609
}
606610

611+
/// Deserializes the metadata stored under `namespace` as `T`.
612+
///
613+
/// Returns `None` if the namespace has no metadata.
614+
///
615+
/// # Errors
616+
///
617+
/// Returns an error if the JSON metadata cannot be deserialized as `T`.
618+
///
619+
/// # Examples
620+
///
621+
/// ```no_run
622+
/// # use std::sync::Arc;
623+
/// # use hyperlight_host::sandbox::snapshot::Snapshot;
624+
/// # use serde::{Deserialize, Serialize};
625+
/// #
626+
/// #[derive(Debug, PartialEq, Serialize, Deserialize)]
627+
/// struct Metadata {
628+
/// version: u32,
629+
/// }
630+
///
631+
/// # fn example(snapshot: Arc<Snapshot>) -> Result<(), Box<dyn std::error::Error>> {
632+
/// let snapshot =
633+
/// snapshot.with_metadata("snapshot-metadata-namespace-v1", &Metadata { version: 1 })?;
634+
/// let metadata = snapshot
635+
/// .metadata::<Metadata>("snapshot-metadata-namespace-v1")?
636+
/// .expect("metadata should exist");
637+
/// assert_eq!(metadata, Metadata { version: 1 });
638+
/// # Ok(())
639+
/// # }
640+
/// ```
641+
pub fn metadata<T>(&self, namespace: &str) -> Result<Option<T>>
642+
where
643+
T: serde::de::DeserializeOwned,
644+
{
645+
self.metadata
646+
.get(namespace)
647+
.cloned()
648+
.map(serde_json::from_value)
649+
.transpose()
650+
.map_err(Into::into)
651+
}
652+
653+
/// Adds `metadata` to this snapshot and returns the result as a new
654+
/// snapshot. Metadata already stored under `namespace` is replaced.
655+
///
656+
/// This snapshot remains unchanged.
657+
/// Metadata is saved and loaded with the snapshot.
658+
///
659+
/// # Errors
660+
///
661+
/// Returns an error if `metadata` cannot be serialized as JSON.
662+
///
663+
/// # Examples
664+
///
665+
/// ```no_run
666+
/// # use std::sync::Arc;
667+
/// # use hyperlight_host::sandbox::snapshot::Snapshot;
668+
/// # use serde::{Deserialize, Serialize};
669+
/// #
670+
/// #[derive(Debug, PartialEq, Serialize, Deserialize)]
671+
/// struct Metadata {
672+
/// version: u32,
673+
/// }
674+
///
675+
/// # fn example(snapshot: Arc<Snapshot>) -> Result<(), Box<dyn std::error::Error>> {
676+
/// let snapshot =
677+
/// snapshot.with_metadata("snapshot-metadata-namespace-v1", &Metadata { version: 1 })?;
678+
/// let metadata = snapshot
679+
/// .metadata::<Metadata>("snapshot-metadata-namespace-v1")?
680+
/// .expect("metadata should exist");
681+
/// assert_eq!(metadata, Metadata { version: 1 });
682+
/// # Ok(())
683+
/// # }
684+
/// ```
685+
pub fn with_metadata<T>(&self, namespace: impl Into<String>, metadata: &T) -> Result<Arc<Self>>
686+
where
687+
T: serde::Serialize,
688+
{
689+
let namespace = namespace.into();
690+
let metadata = serde_json::to_value(metadata)?;
691+
let mut metadata_by_namespace = self.metadata.clone();
692+
metadata_by_namespace.insert(namespace, metadata);
693+
Ok(Arc::new(Self {
694+
state: Arc::clone(&self.state),
695+
metadata: metadata_by_namespace,
696+
}))
697+
}
698+
607699
/// Generation number assigned to this snapshot when it was taken.
608700
pub(crate) fn snapshot_generation(&self) -> u64 {
609701
self.state.snapshot_generation

0 commit comments

Comments
 (0)