Skip to content

HIP 0008: Postcard serialization for host-guest messages - #1999

Merged
ludfjig merged 2 commits into
hyperlight-dev:mainfrom
ludfjig:ll/hip-postcard
Nov 10, 2026
Merged

ludfjig merged 2 commits into
hyperlight-dev:mainfrom
ludfjig:ll/hip-postcard

Conversation

@ludfjig

@ludfjig ludfjig commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig added A-docs Related to documentation C-enhancement For PRs adding features, improving functionality, docs, tests, etc. labels Oct 29, 2026
Copilot AI balanced review requested due to automatic review settings October 29, 2026 23:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The proposal leaves handling of events larger than 4 KiB undefined and omits the snapshot ABI bump and regeneration requirements needed to prevent incompatible snapshots from being resumed.

2 open issues
What changed in this PR

HIP 0008 proposes replacing FlatBuffers with serde and postcard for Hyperlight’s host-guest control messages. This PR contains the design proposal only, not an implementation.

Changes:

  • Defines message types, a control-data length field, and trace batching while preserving external byte payloads.
  • Describes removing FlatBuffers tooling and generated code.
  • Presents preliminary performance results, drawbacks, and alternatives.
File Description
proposals/​0008-postcard-serialization/​README.md Documents the serialization proposal, wire format, benchmarks, and tradeoffs.

Review effort: Balanced

andreiltd
andreiltd previously approved these changes Nov 4, 2026

@andreiltd andreiltd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was an easy read. Postcard seems like a good candidate for the wire format. IIUC the proposal keeps byte payloads outside the control plane using external bytes, so serialization/deserialization performance isn't much of a concern, as all control messages are expected to be relatively small.

I wonder if you considered the timing for landing this work and fixing: #1987 ?

@ludfjig

ludfjig commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

I wonder if you considered the timing for landing this work and fixing: #1987 ?

I think it makes sense to do the serde change first and then #1987 on top, so #1987 doesn't have to change FlatBuffers code that this removes. #1987 then removes the temporary chunked flag along with VecBytes.

@jsturtevant jsturtevant left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple questions but I like the motivation section, helps understand why we would want to do this!

yoshuawuyts
yoshuawuyts previously approved these changes Nov 9, 2026

@yoshuawuyts yoshuawuyts left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question, but otherwise this seems good to me. Thanks @ludfjig!

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig
ludfjig dismissed stale reviews from yoshuawuyts and andreiltd via 6f56424 November 9, 2026 19:34

@jprendes jprendes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks for putting it together @ludfjig !
In the future I would like to decouple all the data-plane stuff from the control-plane, using external bytes and a customizable serialization.
This proposal to change to serde + postcard is a big enablement for that work!

@ludfjig
ludfjig merged commit e3a3966 into hyperlight-dev:main Nov 10, 2026
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-docs Related to documentation C-enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants