From e9cfcc138af1f63f0122e106647dafe4bf9fcf35 Mon Sep 17 00:00:00 2001 From: Yosh Date: Mon, 2 Nov 2026 11:11:53 +0100 Subject: [PATCH 1/2] Add notes for 2026-10-26 community meeting Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Yosh --- 2026/2026-10-26.md | 522 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 521 insertions(+), 1 deletion(-) diff --git a/2026/2026-10-26.md b/2026/2026-10-26.md index 5132cd6..824d20a 100644 --- a/2026/2026-10-26.md +++ b/2026/2026-10-26.md @@ -2,8 +2,528 @@ ## Attendees -## Announcements +- James Sturtevant +- Yosh Wuyts +- Magnus Kulke +- Tomasz Andrzejak +- Ludvig Liljenberg +- Dan Chiarlone +- Simon Davies +- Lucy Menon +- Shailesh Vashishth ## Design Meeting: Streams over Virtqueues (Tomasz) - https://github.com/hyperlight-dev/hyperlight/pull/1969 + +**Status:** Draft. Stream support and execution-driving APIs are proposed. + +## Vocabulary + +CM: Component Model + +## Summary + +Add streams to Hyperlight's host and guest APIs over the existing virtqueues. Support stream +arguments and results through `Sandbox::call`, guest function registration and WIT binding +generation. Follow the WebAssembly Component Model (CM) stream ownership rules. Optional WIT +bindings map `stream` to these endpoints. + +A function can return a stream before producing its contents. Rust applications use async stream +operations. A host driver and a small `no_std` guest task runner make a stream progress. + +MVP scope: byte streams only (`u8` elements). Other stream payloads, CM `future`, and CM async +functions are later work. + +## Motivation + +Today `VecBytes` and `ByteChunks` transport complete byte values. Streams let a producer generate +data as they become available. This supports request bodies, generated output, and in general +transformations larger than the shared-memory pools. Transport memory stays bounded regardless of +the total stream length. + +## Proposal + +### Relationship to the Component Model + +In the [Component Model][component-model], a stream has a readable end and a writable end: + +```text +producer keeps writer ---- bytes ----> receiver owns reader +``` + +- Only the readable end crosses a function boundary. Following the CM rule for `stream` arguments + and results, a call moves the reader to the receiver while the producer keeps its writer. +- The call and stream have separate lifetimes: a call can return while the producer still has work + to do. Neither endpoint is clonable. +- A stream buffers a fixed window of unread bytes. A write completes once its bytes enter the + window, and the reader returns [credit][linux-credits] as it consumes them. A CM write completes + only when a read takes its bytes. The window moves data without a round trip per read. Operations + can make partial progress. +- The reader and writer each allow one pending [operation][stream-operations], so a read and a write + can be pending together. Passing or returning a reader requires it to be [idle][stream-transfer], + with no read pending. Unread bytes stay with the stream and reach the reader's new owner. + +The MVP supports byte-stream readers in function parameters and results for both host and guest +producers. WIT bindings map these readers to `stream`, including inside supported aggregates +such as `result` and `option`. Other payloads, CM `future`, and CM async functions are deferred. + +## Design Details + +### Type system + +The user-facing types are small: + +| Concept | Rust API | +| -------------------------------- | ----------------------------------------------------- | +| Native stream argument or result | Owned `StreamReader`. | +| Private producer | Owned `StreamWriter`. It stays with the producer. | +| read, write, close | Methods returning Rust operation futures. | +| Drive execution | `Sandbox::block_on` or `Sandbox::run_until`. | + +These Rust futures represent pending operations, so awaiting `read_chunk(n)` does not require CM +futures or an async function ABI. + +Later CM `future` support can specialize the stream core for a single element, sharing its +transport and operation machinery while keeping a distinct type, one-shot completion, and +future-specific drop rules. + +The examples use proposed stream APIs. Their task and driving functions are similar to +[`wit_bindgen::spawn_local`][wit-spawn-local] and [`wit_bindgen::block_on`][wit-block-on]: +scheduling local tasks and driving async work from synchronous code. + +**Native Rust guest.** Stream/task admission errors use the existing guest-call error reply. + +```rust +#[hyperlight_guest_bin::guest_function] +fn hello() -> hyperlight_guest::error::Result> { + let (reader, mut writer) = stream::new::()?; + + task::spawn_local(async move { + writer + .write_all(Bytes::from_static(b"hello, world\n")) + .await?; + + writer.close().await + })?; + + Ok(reader) +} +``` + +`spawn_local` queues a runtime-owned task for a later poll, letting `hello` return the reader while +the task keeps the writer. Since the task can outlive the call, it must own its state without +borrowing from the call frame, and the runtime must report any errors it produces. + +**Native Rust host.** Use the core `Sandbox` directly. + +```rust +let mut sandbox = SandboxBuilder::from_file("guest.bin").build()?; +let mut body: StreamReader = sandbox.call("hello", ())?; + +sandbox.block_on(async move { + while let Some(bytes) = body.read_chunk(4096).await? { + std::io::stdout().write_all(&bytes)?; + } + + body.close().await?; + Ok::<(), anyhow::Error>(()) +})?; +``` + +`read_chunk(n)` returns an owned chunk of up to `n` bytes or `None` at EOF, with `n > 0` required +for every read. On the writing side, `write_all` handles partial writes. The sandbox drives these +operations through `block_on`, which advances the supplied future and the VM and propagates errors +from either. Async hosts use `sandbox.run_until(work).await` to drive the same work. + +Host producers use `sandbox.new_stream::()` to create the same reader/writer pair, bound to that +sandbox's session. They move the reader into a guest call and drive the writer's work. + +This keeps `hello` synchronous: handlers return a reader and schedule their stream work for later. +Waiting for stream data before returning requires the later async-call design. + +### C example + +Proposed native C API + +```c +// Runs at a later Progress entry, outside transport locks. +static void hello_written(void *writer, hl_write_result result) { + hl_task_report_write(result); + hl_stream_writer_drop(writer); +} + +// Register as "hello" with return type hl_ReturnType_StreamReader. +static hl_ReturnValue *hello(const hl_FunctionCall *call) { + (void)call; + const uint8_t message[] = "hello, world\n"; + hl_stream_reader *reader; + hl_stream_writer *writer; + hl_status status = hl_stream_new_u8(&reader, &writer); + + if (status != HL_OK) { + goto error; + } + + // Copies the stack buffer. On success, the callback owns writer cleanup. + status = hl_stream_write_all_copy( + writer, message, sizeof(message) - 1, hello_written, writer); + + if (status != HL_OK) { + hl_stream_reader_drop(reader); + hl_stream_writer_drop(writer); + goto error; + } + + // Moves the reader into the RPC result. + return hl_result_from_StreamReader(reader); + +error: + hl_set_error(hl_ErrorCode_GuestError, hl_status_message(status)); + return NULL; +} +``` + +### WIT bindings + +WIT can describe the same operation: + +```wit +package demo:hello; + +interface greeter { + hello: func() -> result, string>; +} + +world hello { + export greeter; +} +``` + +Here the WIT `result` carries stream/task admission errors. The shared bindings map `stream` to +the native reader type, including inside `result`, `option` etc. + +`host_bindgen!` already generates a world-specific wrapper around a sandbox. The proposed +stream-valued bindings and forwarding methods for `new_stream`, `block_on`, and `run_until` extend +that wrapper while sharing the core runtime and driver. + +This synchronous `hello` uses Hyperlight's runtime-owned tasks. A `wit-bindgen` Wasm producer that +keeps working after returning uses a CM async export task and [`async func`][wit-async]. That +async-function integration is out of scope. + +### Stream lifecycle and driving + +**The `Sandbox` driver owns progress.** Endpoint methods queue work without entering the VM. + +- `spawn_local`: queue guest tasks for later polling. +- `block_on`: poll host work, pump queues, and run guest on the calling thread. +- `run_until`: expose async driving that a host task can await on an executor like Tokio. + +```text +return reader -> write fills window -> data -> read -> credit reopens window -> close +``` + +Release endpoints after both ends close and all pending operations are finished. + +Each guest dispatch starts at the saved stack top, so a task that outlives its call needs storage +outside that call's stack frame. The runtime keeps task futures pinned on the heap and uses Rust +[wakers][rust-future-poll] to mark them ready for another poll. It releases its queue and +task-registry locks or borrows before polling tasks or invoking callbacks, since that code can +re-enter the runtime to queue more work. + +Before waiting for more work, the host driver follows [Linux's notification ordering][linux-ring]: +arm queue notifications, apply the required memory barrier, then recheck the queues. This catches +work that arrived while notifications were disabled, preventing the driver from waiting with work +already queued. The wait mechanism must also preserve wakeups arriving between the final check and +the start of the wait. + +`wit-bindgen` keeps `block_on` running until its spawned tasks finish. Hyperlight's `block_on` +returns as soon as its supplied future completes, so any remaining work needs a later drive or +cleanup during sandbox teardown. Stream progress depends on returning control to the driver: +callbacks must not block on stream operations, and duplex streams must be fed and drained +concurrently. Turn budgets are cooperative, which means guest code that never yields requires +interruption. + +### Cancellation and failure + +This is modeled after CM. Cancellation completes locally, without a round trip. + +| Event | What happens | +| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| Drop a read future | The read ends and takes no bytes. | +| Cancel or drop a write | The write ends at once and reports how many bytes were accepted. Accepted bytes stay readable. Cancelling does not close the endpoint. | +| Close the reader early | Tell the producer to stop and discard unread bytes. Pending writes report that the reader closed and how many bytes were accepted. | +| Drop a reader or writer | Close it without waiting. Keep driving the sandbox to finish cleanup. | +| Guest failure, kill during a guest turn, or sandbox destruction | Fail pending operations and wake waiting tasks. Cleanup must work without running the guest again. | +| Kill between guest turns | End the drive. The session and pending operations stay valid. | +| Restore a snapshot | End the old session and reject its handles. Late results must not affect the restored sandbox. | +| Take a snapshot | Reject until all endpoints are released and all operations and tasks finish, even for cached snapshots. Stream activity makes cached state outdated. | + +The transport keeps EOF, cancellation, resource-limit errors, and transport failures separate. EOF +only says that the writer closed its end, so a producer that can fail after returning must also +report whether its work succeeded. The MVP can carry this final status in application framing within +the byte stream, with a separate CM future as a later option. + +### Flow control and VM transitions + +Streams use credit-based flow control after [Linux virtio-vsock][linux-vsock] and +[vhost-vsock][linux-vhost]. The writer's side sends data while the stream's window has room. The +reader's side returns [credit updates][linux-credits] as it consumes data, which reopens the window. +A stream advances up to one window per round trip. + +Stream records travel in existing `SubmitBatch` submissions with packed-ring event suppression. +Control records go first, so data never delays credit or closure. Records depend only on queue +order, so the protocol carries over to a host and guest running on separate threads. + +The MVP keeps Hyperlight's halt-based result handoff. After each halt the host drains the queues and +re-enters the guest only to deliver records or poll ready guest tasks. Synchronous host calls keep +an immediate `outb` and reply. + +**Example.** Suppose the `hello` task writes 40 KiB. With a 16 KiB window, the call and the host's +`block_on` loop take five guest entries. Right arrows carry host records in at an entry. Left +arrows carry guest records back at the halt. `(n)` counts the 4 KiB queue buffers that carry the +data. + +```text +Host (reader) Guest (writer task) + +== entry 1, call ========================================================================= + Request -------------------> hello() returns the reader +takes the reader <------ Response, task ready + +== entry 2 =============================================================================== + no records ----------------> task writes, 16K enters the window +holds 16K <-------------- DATA 16K (5) write waits for credit +reads 16K, owes credit + +== entry 3 =============================================================================== + CREDIT 16K ----------------> next 16K enters the window +holds 16K <-------------- DATA 16K (5) write waits for credit +reads 16K, owes credit + +== entry 4 =============================================================================== + CREDIT 16K ----------------> last 8K enters, write completes, + task closes the writer +holds 8K and EOF <---------- DATA 8K (3), END +reads 8K and EOF, closes + +== entry 5 =============================================================================== +close completes CLOSE_READER --------------> stream released +``` + +Entry 2 runs because the guest reported a ready task. Entries 3 and 4 run because the host owes +credit, and each brings back the next window. Credit never needs a round trip of its own. When +data flows both ways, as in an echo of stream A into stream B, credit shares buffers with data. +Each bracket is one buffer: + +```text +entry --> [CREDIT B | DATA A] [DATA A] [DATA A] ... +halt <-- [CREDIT A | DATA B] [DATA B] [DATA B] ... +``` + +**Future work.** A guest running on its own thread, idle halts with wakeups, +[eventfd-based notifications][linux-ioeventfd], and [coalesced I/O][linux-coalesced-io] could +remove most userspace exits. + +## Implementation History + +- 2026-10-20: Draft proposal. No implementation under this HIP. + +## References + +- CM specification references use revision `0220da84209397d9067c19cac83dae969aa85a7b`. +- Linux references use revision `b95f03f04d475aa6719d15a636ddf32222d55657`. + +- [Canonical ABI][canonical-abi] and [Python reference][canonical-reference]. +- [Component type validity][component-binary]. +- [Concurrency and stream ownership][concurrency], including [stream readiness][stream-readiness]. +- [Stream reads and writes][stream-operations] and [reader transfer][stream-transfer]. +- [wit-bindgen async binding rules][wit-async]. +- [Hyper body streams][hyper-body] and [Tokio blocking workers][tokio-blocking]. +- [HIP 0001: Virtio-inspired ring buffer][hip-0001]. +- [HIP 0002: Serialized envelope and external byte stream][hip-0002]. +- Linux [virtio-vsock][linux-vsock], [vhost-vsock][linux-vhost], and [virtio-net][linux-net]. +- Linux [notification ordering][linux-ring] and [vsock credit updates][linux-credits]. +- KVM [eventfd routing][linux-ioeventfd] and [coalesced I/O][linux-coalesced-io]. + +[hip-0001]: https://github.com/hyperlight-dev/hyperlight/blob/main/proposals/0001-rng-buf/README.md +[hip-0002]: https://github.com/hyperlight-dev/hyperlight/blob/main/proposals/0002-envelope-external-byte-stream/README.md +[component-model]: https://component-model.bytecodealliance.org/ +[canonical-abi]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/CanonicalABI.md +[canonical-reference]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/canonical-abi/definitions.py +[component-binary]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/Binary.md +[concurrency]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/Concurrency.md +[stream-operations]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/Explainer.md#-streamread-and-streamwrite +[stream-transfer]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/CanonicalABI.md?plain=1#L2422-L2448 +[stream-readiness]: + https://github.com/WebAssembly/component-model/blob/0220da84209397d9067c19cac83dae969aa85a7b/design/mvp/Concurrency.md#stream-readiness +[wit-async]: + https://github.com/bytecodealliance/wit-bindgen/blob/4f7496b180e4c0eaf787b0bc5ef4a3ac5c68151a/crates/core/src/async_.rs#L26-L28 +[wit-spawn-local]: https://docs.rs/wit-bindgen/0.63.0/wit_bindgen/fn.spawn_local.html +[wit-block-on]: https://docs.rs/wit-bindgen/0.63.0/wit_bindgen/fn.block_on.html +[rust-future-poll]: https://doc.rust-lang.org/core/future/trait.Future.html#tymethod.poll +[hyper-body]: https://hyper.rs/guides/1/server/echo/#body-streams +[tokio-blocking]: https://docs.rs/tokio/latest/tokio/task/fn.spawn_blocking.html +[linux-vsock]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/net/vmw_vsock/virtio_transport.c#L151-L205 +[linux-vhost]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/drivers/vhost/vsock.c +[linux-net]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/drivers/net/virtio_net.c#L3336-L3393 +[linux-ring]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/drivers/virtio/virtio_ring.c#L3138-L3217 +[linux-credits]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/net/vmw_vsock/virtio_transport_common.c#L451-L643 +[linux-ioeventfd]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/Documentation/virt/kvm/api.rst#L2093-L2137 +[linux-coalesced-io]: + https://github.com/torvalds/linux/blob/b95f03f04d475aa6719d15a636ddf32222d55657/Documentation/virt/kvm/api.rst#L5000-L5028 + +## Tomasz + +Tomasz: Some justification: I'm referencing the component model and referencing the linux v-sock implementation. CM is giving us useful semantics for ownership of the stream. While vsock impl is how we're already structuring our virtqueue layout; vsock is giving us a performant version of the stream. This combines the two. + +Lucy: would caution against using the CM semantics too closely. They were informed by being able to follow both completion based and submission based semantics, and to support moving and forwarding stream ends between many (more than 2) components. We don't need to follow that. + +## Vibe Check + +*Write your initial impressions here. Briefly, ideally max one paragraph*. + +### Magnus + +Magnus: Without being to familiar w/ HP guest/host contracts, the stream interface makes sense to me. One aspect that wasn't obvious to me was the failure modes for snapshot/restore, e.g. "Take a snapshot: Reject until all endpoints are released", what is being rejected and is the orchestration of the shutdown/release managed by the Host component? + +Tomasz: that's probably the simplest solution. Would suggest we treat a stream which is live the same as a function call in progress today. You can finish the stream, and then the snapshot is allowed. At least for MVP. Not sure if there's a use case for snapshotting streams in the middle. + +Magnus: this will eventually come up, will feel arbitrary - not obvious that this is going on. + +Lucy: not sure that's true, how would I get a hold of a handle when we do a restore? If we did support snapshots with livestreams, how would we recover? + +### Ludvig + +Looks great! Put 2 question below + +### Dan + +Overall, I think the HIP is pretty solid and the mechanism seems sound! + +(questions moved to the bottom) + +### Simon + +Lucy: Looks good, all the examples seem are guest producer and host consumer, assume both are supported. We should porbably add a threat model to the HIP + +Simon: there are no examples where the producer and consumer are the other way around. The guest is the producer in all the examples, not sure if implied or unsupported - but we should be clear. + +### Lucy + +Lucy: Overall, I think this is about the right size/shape for a first feature driving towards asynchronous I/O. I appreciate the comparisons to the Component Model streams, but wonder if it would also benefit from comparisons to io_uring/IOCP/virtio-ethernet, etc.---since at its core it is really a similar completion-based I/O paradigm to those, rather than attempting to capture the full many-things-to-many-runtimes generality of component runtimes. + +Lucy: would be useful to have a little discussion about the other IO paradigms we're basing this off of and setting the stage. Also would be good to call out the buffering semantics, basically about the clarity. + +### Doru + +[Technical problems] + +### James + +James: I like that this is scoped and makes it so that we have something that we can incrementally build on + +### Yosh + +Like others have said: I think this scopes the work nicely. It takes a single core primitive, and builds around that. I don't know I quite got everything I wanted from this doc, but it's EOD for me here so that might just be me. + +## Discussion + +### API Examples + +Yosh: the HIP mentions `block_on`, `run_until` APIs - I would love to see the signatures of that even if you omit the implementation. I had a bit of trouble following the stream lifecyle section - maybe that's me since I'm at the end of my day here, but I'm wondering if some more examples / breaking it down a bit further would help? + +### Buffering semantics + +Lucy: I misunderstood at first and thought that this proposal usesdthe same buffering semantics as the CM where read/write synchronise rather than buffer. Maybe the difference could be a little bit clearer and called out---though I expect it is probably the right one. + +Lucy: on second read, you're saying that a write will complete as-is? I think that's correct, but would appreciate it if that was clearer in the doc. I have related questions: + +- how do you decide the window size? +- up to the max buffer size, do you support pipelining multiple stream ops with the same vm exit? or do you need to wait for switching first? + +Tomasz: window size is configurable by linux vsock? How do we want to configure it? Per stream? Per sandbox? Window size in the vsock doesn't have anything to do with the buffer size in the virtual queue. You can have a queue which spans multiple buffers [???]. We can come up with some default or expose this config. Didn't think much about multiplexing though. + +Lucy: one exit has a "create stream" op, another has a "read from stream" op. Can we combine these into a single exit? + +Tomasz: in this proposal the stream reader is only the movable end. If you're moving from the host to the guest, with the function call you will move the readable end to the guest. The reader will return e.g. admission errors, and then the stream can start. + +Lucy: the writes can ack before the reads complete [???] - instead of having to exit multiple times they can be combined. + +Tomasz: should be doable, yes! + +Lucy: how will streams be represented on the wire? multiple threads may be running on both end, syncing IDs may be complex. Should be possible without round tripping but it's not easy. + +Tomasz: we should be able to do that. We have headers which track that data. + +Lucy: not quite what I meant tho. but let's follow up offline. How do you allocate stream IDs? + +### Function types in the guest + +Lucy: I disagree with having the guest functions use the implicit Result<> that is currently on every guest function for stream errors. I think that result is itself a misfeature on the guest side and should go away. + +Lucy: minor nit. hopefully that result will eventually go away, we should follow CM semantics here. + +Tomasz: ok makes sense, feedback in the PR would be helpful. + +### Performance tradeoff + +James: I see we are doing 5 vm entries to write sizable data. There must be a trade of around the cost to do those 5 vm entries/exits vs doing a full write? Would be nice to see this called out explicitly. + +James: what's the use case we're trying to solve for? Do we get more perf if we chunk vs exiting multiple times? We're not allowing the guest to continue doing stuff, so it's not async that way? + +Tomasz: you could imagine streaming HTTP body, for example? one goes out of the guest, the other does not. For a long-running guest, you don't really know what the amount of the data is that you're going to receive? If you just have some bounded amount of the data, and you just need the response back? The tradeoff is that you just pay this cost once, and then the running stream implementation should be more efficient than those function calls. + +James: makes sense. Would be useful to add to this scenario. + +Lucy: the number looks big here, but only because the buffer window size is configured to be smaller than the buffer. Stream close notification could be batched with next operation if it is not urgent. Pipelining allows you to coalesce the stream create/function return/first stream write some of the operation. So hopefully it can be done with no extra overhead in the best case. + +Magnus: you mentioned you're thinking about eventfd optimizations? have you thought about that? + +Tomasz: I haven't. Unsure about feasability/benefits. + +Magnus: for regular virtualization it's what you always want. Might be different if you don't have regular interrupt flows. + +Tomasz: we don't have the same guest/host driving. + +### Fire and forget + +James: I don't see this listed as a scenario. Is this something we could enable with this or not yet + +### What workloads are expected to be unlocked/improved by streams + +Ludvig: + +### I'm curious what hostile guests will be able to with these streams and potentally affect the host + +### Dan: Semantic difference w/ CM + +In the HIP, it's highligthed that there's a difference between the CM and HL wrt when a write completes. In HL, it completes when the bytes enter a window vs. in CM it completes when the reader takes the bytes? What are the consequences of this to integrate w/ HL-Wasm (i.e., hl-wasm and wasmtiem CM interop)? + +### Dan: Consider hluk's `step` + +In hluk, we have a step call alongside run that executes a guest before until it goes idle. Maybe consider this together w/ block_on? Detailed notes: https://github.com/hyperlight-dev/hyperlight-unikraft/blob/main/docs/execution.md + +## Next steps + +James: are you going to take the feedback here from the hip, and then put a two-week period for approval? + +Tomasz: for the feedback, yeah for sure. Maybe we can revisit in the next meeting? + +James: maybe we can review it in the next meeting, and then file major concerns next week? + +Lucy: if nothing urgent comes up, maybe we can merge it next week? + +James: sooner the better! From c7be5b804239a8e27f8413a4fa2afddf7a46d81e Mon Sep 17 00:00:00 2001 From: Yosh Date: Mon, 2 Nov 2026 11:14:04 +0100 Subject: [PATCH 2/2] Fix typos in October 26 meeting notes Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Yosh --- 2026/2026-10-26.md | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/2026/2026-10-26.md b/2026/2026-10-26.md index 824d20a..55f9b58 100644 --- a/2026/2026-10-26.md +++ b/2026/2026-10-26.md @@ -387,7 +387,7 @@ remove most userspace exits. ## Tomasz -Tomasz: Some justification: I'm referencing the component model and referencing the linux v-sock implementation. CM is giving us useful semantics for ownership of the stream. While vsock impl is how we're already structuring our virtqueue layout; vsock is giving us a performant version of the stream. This combines the two. +Tomasz: Some justification: I'm referencing the Component Model and referencing the Linux vsock implementation. CM is giving us useful semantics for ownership of the stream. While vsock impl is how we're already structuring our virtqueue layout; vsock is giving us a performant version of the stream. This combines the two. Lucy: would caution against using the CM semantics too closely. They were informed by being able to follow both completion based and submission based semantics, and to support moving and forwarding stream ends between many (more than 2) components. We don't need to follow that. @@ -397,17 +397,17 @@ Lucy: would caution against using the CM semantics too closely. They were inform ### Magnus -Magnus: Without being to familiar w/ HP guest/host contracts, the stream interface makes sense to me. One aspect that wasn't obvious to me was the failure modes for snapshot/restore, e.g. "Take a snapshot: Reject until all endpoints are released", what is being rejected and is the orchestration of the shutdown/release managed by the Host component? +Magnus: Without being too familiar with Hyperlight guest/host contracts, the stream interface makes sense to me. One aspect that wasn't obvious to me was the failure modes for snapshot/restore, e.g. "Take a snapshot: Reject until all endpoints are released", what is being rejected and is the orchestration of the shutdown/release managed by the host component? Tomasz: that's probably the simplest solution. Would suggest we treat a stream which is live the same as a function call in progress today. You can finish the stream, and then the snapshot is allowed. At least for MVP. Not sure if there's a use case for snapshotting streams in the middle. Magnus: this will eventually come up, will feel arbitrary - not obvious that this is going on. -Lucy: not sure that's true, how would I get a hold of a handle when we do a restore? If we did support snapshots with livestreams, how would we recover? +Lucy: not sure that's true, how would I get a hold of a handle when we do a restore? If we did support snapshots with live streams, how would we recover? ### Ludvig -Looks great! Put 2 question below +Looks great! Put 2 questions below. ### Dan @@ -417,7 +417,7 @@ Overall, I think the HIP is pretty solid and the mechanism seems sound! ### Simon -Lucy: Looks good, all the examples seem are guest producer and host consumer, assume both are supported. We should porbably add a threat model to the HIP +Lucy: Looks good, all the examples seem to be guest producer and host consumer, assume both are supported. We should probably add a threat model to the HIP. Simon: there are no examples where the producer and consumer are the other way around. The guest is the producer in all the examples, not sure if implied or unsupported - but we should be clear. @@ -443,18 +443,18 @@ Like others have said: I think this scopes the work nicely. It takes a single co ### API Examples -Yosh: the HIP mentions `block_on`, `run_until` APIs - I would love to see the signatures of that even if you omit the implementation. I had a bit of trouble following the stream lifecyle section - maybe that's me since I'm at the end of my day here, but I'm wondering if some more examples / breaking it down a bit further would help? +Yosh: the HIP mentions `block_on`, `run_until` APIs - I would love to see the signatures of those even if you omit the implementation. I had a bit of trouble following the stream lifecycle section - maybe that's me since I'm at the end of my day here, but I'm wondering if some more examples / breaking it down a bit further would help? ### Buffering semantics -Lucy: I misunderstood at first and thought that this proposal usesdthe same buffering semantics as the CM where read/write synchronise rather than buffer. Maybe the difference could be a little bit clearer and called out---though I expect it is probably the right one. +Lucy: I misunderstood at first and thought that this proposal used the same buffering semantics as the CM where read/write synchronise rather than buffer. Maybe the difference could be a little bit clearer and called out---though I expect it is probably the right one. Lucy: on second read, you're saying that a write will complete as-is? I think that's correct, but would appreciate it if that was clearer in the doc. I have related questions: - how do you decide the window size? -- up to the max buffer size, do you support pipelining multiple stream ops with the same vm exit? or do you need to wait for switching first? +- up to the max buffer size, do you support pipelining multiple stream ops with the same VM exit? or do you need to wait for switching first? -Tomasz: window size is configurable by linux vsock? How do we want to configure it? Per stream? Per sandbox? Window size in the vsock doesn't have anything to do with the buffer size in the virtual queue. You can have a queue which spans multiple buffers [???]. We can come up with some default or expose this config. Didn't think much about multiplexing though. +Tomasz: window size is configurable by Linux vsock? How do we want to configure it? Per stream? Per sandbox? Window size in the vsock doesn't have anything to do with the buffer size in the virtual queue. You can have a queue which spans multiple buffers [???]. We can come up with some default or expose this config. Didn't think much about multiplexing though. Lucy: one exit has a "create stream" op, another has a "read from stream" op. Can we combine these into a single exit? @@ -464,7 +464,7 @@ Lucy: the writes can ack before the reads complete [???] - instead of having to Tomasz: should be doable, yes! -Lucy: how will streams be represented on the wire? multiple threads may be running on both end, syncing IDs may be complex. Should be possible without round tripping but it's not easy. +Lucy: how will streams be represented on the wire? multiple threads may be running on both ends, syncing IDs may be complex. Should be possible without round tripping but it's not easy. Tomasz: we should be able to do that. We have headers which track that data. @@ -472,7 +472,7 @@ Lucy: not quite what I meant tho. but let's follow up offline. How do you alloca ### Function types in the guest -Lucy: I disagree with having the guest functions use the implicit Result<> that is currently on every guest function for stream errors. I think that result is itself a misfeature on the guest side and should go away. +Lucy: I disagree with having the guest functions use the implicit `Result<>` that is currently on every guest function for stream errors. I think that result is itself a misfeature on the guest side and should go away. Lucy: minor nit. hopefully that result will eventually go away, we should follow CM semantics here. @@ -480,7 +480,7 @@ Tomasz: ok makes sense, feedback in the PR would be helpful. ### Performance tradeoff -James: I see we are doing 5 vm entries to write sizable data. There must be a trade of around the cost to do those 5 vm entries/exits vs doing a full write? Would be nice to see this called out explicitly. +James: I see we are doing 5 VM entries to write sizable data. There must be a trade-off around the cost to do those 5 VM entries/exits vs doing a full write? Would be nice to see this called out explicitly. James: what's the use case we're trying to solve for? Do we get more perf if we chunk vs exiting multiple times? We're not allowing the guest to continue doing stuff, so it's not async that way? @@ -488,11 +488,11 @@ Tomasz: you could imagine streaming HTTP body, for example? one goes out of the James: makes sense. Would be useful to add to this scenario. -Lucy: the number looks big here, but only because the buffer window size is configured to be smaller than the buffer. Stream close notification could be batched with next operation if it is not urgent. Pipelining allows you to coalesce the stream create/function return/first stream write some of the operation. So hopefully it can be done with no extra overhead in the best case. +Lucy: the number looks big here, but only because the buffer window size is configured to be smaller than the buffer. Stream close notification could be batched with the next operation if it is not urgent. Pipelining allows you to coalesce some of the stream create/function return/first stream write operations. So hopefully it can be done with no extra overhead in the best case. Magnus: you mentioned you're thinking about eventfd optimizations? have you thought about that? -Tomasz: I haven't. Unsure about feasability/benefits. +Tomasz: I haven't. Unsure about feasibility/benefits. Magnus: for regular virtualization it's what you always want. Might be different if you don't have regular interrupt flows. @@ -506,19 +506,19 @@ James: I don't see this listed as a scenario. Is this something we could enable Ludvig: -### I'm curious what hostile guests will be able to with these streams and potentally affect the host +### I'm curious what hostile guests will be able to do with these streams and potentially affect the host ### Dan: Semantic difference w/ CM -In the HIP, it's highligthed that there's a difference between the CM and HL wrt when a write completes. In HL, it completes when the bytes enter a window vs. in CM it completes when the reader takes the bytes? What are the consequences of this to integrate w/ HL-Wasm (i.e., hl-wasm and wasmtiem CM interop)? +In the HIP, it's highlighted that there's a difference between the CM and HL wrt when a write completes. In HL, it completes when the bytes enter a window vs. in CM it completes when the reader takes the bytes? What are the consequences of this to integrate w/ HL-Wasm (i.e., hl-wasm and Wasmtime CM interop)? ### Dan: Consider hluk's `step` -In hluk, we have a step call alongside run that executes a guest before until it goes idle. Maybe consider this together w/ block_on? Detailed notes: https://github.com/hyperlight-dev/hyperlight-unikraft/blob/main/docs/execution.md +In hluk, we have a `step` call alongside `run` that executes a guest until it goes idle. Maybe consider this together with `block_on`? Detailed notes: https://github.com/hyperlight-dev/hyperlight-unikraft/blob/main/docs/execution.md ## Next steps -James: are you going to take the feedback here from the hip, and then put a two-week period for approval? +James: are you going to take the feedback here from the HIP, and then put a two-week period for approval? Tomasz: for the feedback, yeah for sure. Maybe we can revisit in the next meeting?