Speed up full-window MSR restore audit by jprendes · Pull Request #1905 · hyperlight-dev/hyperlight

jprendes · 2026-10-08T14:38:30Z

Summary

probe MSR readability in bounded guest-side batches
recover expected general protection faults without poisoning and restoring the sandbox
preserve write and restore verification for every readable MSR
cover allowed and denied batched reads on KVM

The audit probes 147,456 indices. Restoring the complete sandbox after every denied read dominates its runtime on filterless Hyper-V backends.

Performance

On my KVM host, the discovery phase improved from 45.683 seconds to 123.748 milliseconds. This is a 369x speedup. WHP and MSHV CI will provide the representative end-to-end measurements.

Validation

just fmt-apply
just guests
just clippy
focused batched MSR probe test
unit, isolated, integration, documentation, and tool suites with the independently failing host-dependent CET test skipped

Probe MSR readability in bounded guest-side batches and recover expected general protection faults without restoring the sandbox after every denied read. Preserve the existing write and restore checks for each readable MSR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>

Copilot

🟢 Approval recommended

The bounded fault-recovery path preserves the audit’s verification behavior and has focused coverage.

0 open findings

What changed in this PR

Speeds up full-window MSR restore auditing by recovering expected guest faults during batched readability probes.

Changes:

Adds bounded guest-side MSR probing with recoverable #GP handling.
Uses probe bitmaps to audit only readable MSRs.
Tests allowed and denied KVM reads without sandbox poisoning.

File	Description
`src/tests/rust_guests/simpleguest/src/main.rs`	Implements batched MSR probing and fault recovery.
`src/hyperlight_host/src/sandbox/initialized.rs`	Integrates probes into the audit and adds KVM coverage.

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

ludfjig · 2026-10-08T17:52:06Z

nice! This probably got even worse recentlyd ue to the restore regression on windows!

Give the transport buffer release test enough heap headroom for guest function metadata and its 32 KiB response allocation. Keep the global sandbox default unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>

hyperlight-gh-bot · 2026-10-08T20:15:47Z

Benchmark Results

Measured commit: 13c5bf98c604
Baseline commit: 73f8f3ef9d3d

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`788.97 ns` (➖ 1.00x slower)
`vec_bytes`	`583.92 ns` (➖ 1.01x faster)
		`372.57 µs` (➖ 1.00x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`529.29 ns` (➖ 1.01x slower)
`65536`	`147.37 ns` (➖ 1.04x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`78.69 ms` (➖ 1.03x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`7.81 ns` (➖ 1.01x slower)	`7.72 ns` (➖ 1.00x faster)	`7.77 ns` (➖ 1.01x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`93.25 µs` (➖ 1.00x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.01 µs` (➖ 1.25x faster)	`6.99 µs` (➖ 1.05x faster)
`65536`	`2.06 µs` (➖ 1.07x slower)	`1.93 µs` (➖ 1.08x faster)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`6.20 µs` (➖ 1.03x faster)	`6.20 µs` (➖ 1.17x faster)
`8192`	`1.07 µs` (➖ 1.05x faster)	`1.06 µs` (➖ 1.01x faster)
`262144`		`28.02 µs` (➖ 1.03x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`840.84 ns` (➖ 1.03x slower)
`vec_bytes`	`610.79 ns` (➖ 1.01x faster)
		`835.54 µs` (➖ 1.05x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`604.97 ns` (➖ 1.00x slower)
`65536`	`164.98 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`82.99 ms` (➖ 1.02x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`8.32 ns` (➖ 1.00x slower)	`8.30 ns` (➖ 1.00x slower)	`8.35 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`52.55 µs` (➖ 1.02x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.08 µs` (➖ 1.00x slower)	`9.03 µs` (➖ 1.00x faster)
`65536`	`2.55 µs` (➖ 1.00x slower)	`2.55 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.50 µs` (➖ 1.00x slower)	`8.45 µs` (➖ 1.02x faster)
`8192`	`892.54 ns` (➖ 1.01x slower)	`866.45 ns` (➖ 1.02x faster)
`262144`		`35.76 µs` (➖ 1.01x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`958.18 ns` (➖ 1.02x slower)
`vec_bytes`	`709.53 ns` (➖ 1.00x slower)
		`324.81 µs` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`674.47 ns` (➖ 1.00x slower)
`65536`	`183.59 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`61.84 ms` (➖ 1.14x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`9.92 ns` (➖ 1.00x faster)	`9.91 ns` (➖ 1.00x faster)	`9.91 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`87.04 µs` (➖ 1.00x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.23 µs` (➖ 1.03x slower)	`9.22 µs` (➖ 1.00x slower)
`65536`	`2.64 µs` (➖ 1.02x faster)	`2.56 µs` (➖ 1.04x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`7.97 µs` (➖ 1.03x faster)	`8.01 µs` (➖ 1.02x faster)
`8192`	`1.35 µs` (➖ 1.00x faster)	`1.32 µs` (➖ 1.01x slower)
`262144`		`34.77 µs` (➖ 1.02x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`995.17 ns` (➖ 1.09x slower)
`vec_bytes`	`659.11 ns` (➖ 1.01x slower)
		`802.40 µs` (➖ 1.19x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`623.40 ns` (➖ 1.01x slower)
`65536`	`163.41 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`67.84 ms` (➖ 1.06x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`9.54 ns` (➖ 1.01x faster)	`10.08 ns` (➖ 1.15x slower)	`8.78 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`46.11 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.93 µs` (➖ 1.01x faster)	`7.95 µs` (➖ 1.01x faster)
`65536`	`2.28 µs` (➖ 1.01x faster)	`2.28 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`7.50 µs` (➖ 1.00x faster)	`7.53 µs` (➖ 1.01x faster)
`8192`	`933.94 ns` (➖ 1.01x faster)	`914.98 ns` (➖ 1.00x slower)
`262144`		`40.87 µs` (➖ 1.04x slower)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.12 µs` (➖ 1.06x faster)
`vec_bytes`	`746.10 ns` (➖ 1.00x faster)
		`1.92 ms` (➖ 1.18x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`788.65 ns` (➖ 1.14x faster)
`65536`	`231.77 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`73.81 ms` (➖ 1.09x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.58 ns` (➖ 1.05x faster)	`10.61 ns` (➖ 1.01x faster)	`10.64 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`497.46 µs` (➖ 1.45x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`9.43 µs` (➖ 1.08x slower)	`9.00 µs` (➖ 1.05x slower)
`65536`	`2.56 µs` (➖ 1.01x slower)	`2.59 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.74 µs` (➖ 1.02x slower)	`8.92 µs` (➖ 1.01x slower)
`8192`	`1.56 µs` (➖ 1.04x faster)	`1.44 µs` (➖ 1.04x slower)
`262144`		`38.42 µs` (➖ 1.02x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.22 µs` (➖ 1.00x faster)
`vec_bytes`	`793.79 ns` (➖ 1.01x faster)
		`3.17 ms` (➖ 1.02x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`740.40 ns` (➖ 1.01x faster)
`65536`	`222.42 ns` (➖ 1.06x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`108.27 ms` (➖ 1.05x faster)

slot_pool

	`alloc_dealloc_1500`	`alloc_dealloc_4096`	`alloc_dealloc_128`
	`10.36 ns` (➖ 1.01x slower)	`10.01 ns` (➖ 1.00x slower)	`10.62 ns` (➖ 1.01x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`574.93 µs` (➖ 1.02x faster)

virtq_readonly

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`262144`	`7.81 µs` (➖ 1.01x slower)	`7.79 µs` (➖ 1.01x slower)
`65536`	`2.58 µs` (➖ 1.12x slower)	`2.35 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented_fragmented`	`slot_pool_segmented`
`65536`	`8.38 µs` (➖ 1.01x faster)	`8.50 µs` (➖ 1.01x faster)
`8192`	`1.37 µs` (➖ 1.08x faster)	`1.12 µs` (➖ 1.02x slower)
`262144`		`39.92 µs` (➖ 1.02x faster)

_{Reported by cargo ci bench-report --candidate run:37832355682 --config-file bench_report.toml.}

Copilot AI balanced review requested due to automatic review settings October 8, 2026 14:38

jprendes requested review from andreiltd, danbugs, dblnz, devigned, jsturtevant, ludfjig, simongdavies, squillace and syntactically as code owners October 8, 2026 14:38

jprendes added C-enhancement For PRs adding features, improving functionality, docs, tests, etc. A-perf Addresses performance A-testing Related to tests or testing infrastructure S-needs-review PR is ready for (re-)review labels Oct 8, 2026

Copilot started reviewing on behalf of jprendes October 8, 2026 14:39

View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

ludfjig previously approved these changes Oct 8, 2026

View reviewed changes

jprendes dismissed ludfjig’s stale review via 13c5bf9 October 8, 2026 19:29

jprendes commented Oct 8, 2026

View reviewed changes

Comment thread src/hyperlight_host/tests/sandbox_host_tests.rs Resolved

ludfjig approved these changes Oct 8, 2026

View reviewed changes

jprendes merged commit 4374594 into hyperlight-dev:main Oct 8, 2026
74 of 80 checks passed

github-actions Bot removed the S-needs-review PR is ready for (re-)review label Oct 8, 2026

Conversation

jprendes commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

This comment has been minimized.

ludfjig commented Oct 8, 2026

Uh oh!

Uh oh!

hyperlight-gh-bot Bot commented Oct 8, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jprendes commented Oct 8, 2026 •

edited

Loading