Expose one CPU feature set to MSHV and WHP guests by ludfjig · Pull Request #2028 · hyperlight-dev/hyperlight

ludfjig · 2026-11-05T22:49:15Z

Draft until #2022 is merged.

MSHV and WHP use different default CPU features. Both expose features with guest state, such as speculation control, TSX, CET, and TSC_ADJUST. WHP guests also share the host process's speculation domain.

This sets one feature list for both backends in partition_features.rs and gives WHP guests their own speculation domain. The list keeps AVX through AVX-512. Hyperlight does not support AVX, but a guest can enable it. Restore resets XCR0 to x87 only.

Breaking: guests that use a removed feature fault on MSHV and WHP. See docs/guest-cpu.md for the list.

Fixes #1685

On WHP hosts with TscAdjustSupport, restore does not rewind the guest TSC. Writing TSC shifts TSC_ADJUST by the same amount, and writing TSC_ADJUST shifts TSC back, so restoring both keeps the time since the snapshot. Clear TscAdjustSupport in the partition's processor feature banks, as the MSHV driver does. The probe then drops TSC_ADJUST from the reset set, and restore_rewinds_tsc runs on WHP too. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

Copilot

🟡 Changes recommended

Failures in configure_partition can return before WhpVm owns the partition, bypassing cleanup and leaking partition handles on repeated creation attempts.

1 open issue

Clean up partition when configuration fails · New

What changed in this PR

This PR gives MSHV and WHP a shared CPU feature policy, addressing differing backend defaults. It also isolates WHP guest speculation and enables TSC rewind coverage on WHP.

Changes:

Apply shared processor and XSAVE feature masks while retaining AVX through AVX-512.
Give WHP guests a separate speculation domain.
Add feature-exposure tests and document the guest compatibility changes.

File	Description
`src/tests/rust_guests/simpleguest/src/main.rs`	Adds a guest CPUID query for tests.
`src/hyperlight_host/src/sandbox/initialized.rs`	Tests feature exposure and TSC rewind on WHP.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/x86_64.rs`	Configures feature masks and a separate security domain.
`src/hyperlight_host/src/hypervisor/virtual_machine/partition_features.rs`	Defines shared processor and XSAVE feature masks.
`src/hyperlight_host/src/hypervisor/virtual_machine/mshv/x86_64.rs`	Applies shared masks during partition creation.
`src/hyperlight_host/src/hypervisor/virtual_machine/mod.rs`	Registers the shared feature module.
`src/hyperlight_host/src/hypervisor/regs/x86_64/msrs.rs`	Removes a TSC_ADJUST comment. No functional change.
`docs/README.md`	Links CPU feature and MSR documentation.
`docs/msr.md`	Aligns MSR documentation with the feature policy.
`docs/guest-cpu.md`	Documents exposed features and exclusions.
`CHANGELOG.md`	Records feature restrictions, speculation isolation, and the TSC fix.

_{Review effort: Balanced}

ludfjig · 2026-11-05T23:49:30Z

+                },
+            )?;
+
+            Self::configure_partition(p)?;


The leak is older than this PR. On main, every failure after WHvCreatePartition in WhpVm::new leaks the handle. We should fix it in a separate PR.

hyperlight-gh-bot · 2026-11-05T23:47:43Z

Benchmark Results

Measured commit: cc462d0712b0
Baseline commit: a03088470d82

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`760.09 ns` (➖ 1.01x faster)
`vec_bytes`	`591.78 ns` (➖ 1.01x slower)
		`376.63 µs` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`525.30 ns` (➖ 1.01x faster)
`65536`	`143.81 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.36 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`7.67 ns` (➖ 1.00x faster)	`7.69 ns` (➖ 1.00x slower)	`7.72 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`99.87 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.14 µs` (➖ 1.17x slower)	`7.18 µs` (➖ 1.05x faster)
`65536`	`1.92 µs` (➖ 1.08x faster)	`2.05 µs` (➖ 1.07x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`26.83 µs` (➖ 1.03x faster)
`65536`	`6.33 µs` (➖ 1.10x faster)	`6.94 µs` (➖ 1.04x faster)
`8192`	`1.03 µs` (➖ 1.01x slower)	`1.05 µs` (➖ 1.04x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`683.93 ns` (➖ 1.01x slower)
`vec_bytes`	`505.64 ns` (➖ 1.01x faster)
		`685.71 µs` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`497.17 ns` (➖ 1.01x faster)
`65536`	`154.79 ns` (➖ 1.02x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`15.13 ms` (➖ 1.00x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`6.79 ns` (➖ 1.01x faster)	`6.80 ns` (➖ 1.00x faster)	`6.80 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`48.72 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.58 µs` (➖ 1.00x slower)	`7.71 µs` (➖ 1.01x slower)
`65536`	`2.12 µs` (➖ 1.01x faster)	`2.13 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`30.23 µs` (➖ 1.03x slower)
`65536`	`7.05 µs` (➖ 1.01x faster)	`7.11 µs` (➖ 1.01x faster)
`8192`	`720.12 ns` (➖ 1.01x slower)	`720.99 ns` (➖ 1.08x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`979.23 ns` (➖ 1.01x slower)
`vec_bytes`	`714.08 ns` (➖ 1.04x faster)
		`267.83 µs` (➖ 1.27x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`731.88 ns` (➖ 1.00x faster)
`65536`	`192.54 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`63.16 ms` (➖ 1.02x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`11.64 ns` (➖ 1.19x slower)	`9.51 ns` (➖ 1.06x faster)	`9.61 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`105.54 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.64 µs` (➖ 1.02x faster)	`9.90 µs` (➖ 1.04x faster)
`65536`	`2.25 µs` (➖ 1.01x slower)	`2.41 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`35.86 µs` (➖ 1.06x faster)
`65536`	`8.46 µs` (➖ 1.12x faster)	`9.65 µs` (➖ 1.01x slower)
`8192`	`1.23 µs` (➖ 1.01x slower)	`1.22 µs` (➖ 1.03x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`967.03 ns` (➖ 1.03x slower)
`vec_bytes`	`654.63 ns` (➖ 1.02x faster)
		`643.29 µs` (➖ 1.04x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`663.53 ns` (➖ 1.01x slower)
`65536`	`260.59 ns` (➖ 1.00x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`84.80 ms` (➖ 1.19x slower)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`8.72 ns` (➖ 1.10x faster)	`8.78 ns` (➖ 1.01x slower)	`8.70 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`54.99 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`8.10 µs` (➖ 1.01x slower)	`7.97 µs` (➖ 1.00x faster)
`65536`	`2.30 µs` (➖ 1.01x slower)	`2.28 µs` (➖ 1.00x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`39.22 µs` (➖ 1.02x slower)
`65536`	`7.61 µs` (➖ 1.02x slower)	`7.52 µs` (➖ 1.00x slower)
`8192`	`917.88 ns` (➖ 1.02x slower)	`903.36 ns` (➖ 1.01x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.19 µs` (➖ 1.03x slower)
`vec_bytes`	`768.44 ns` (➖ 1.01x slower)
		`2.20 ms` (➖ 1.03x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`793.50 ns` (➖ 1.00x slower)
`65536`	`241.54 ns` (➖ 1.04x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`10.89 ms` (➖ 1.16x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`10.68 ns` (➖ 1.08x faster)	`10.58 ns` (➖ 1.00x faster)	`10.66 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`807.99 µs` (➖ 1.03x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`8.61 µs` (➖ 1.09x faster)	`8.94 µs` (➖ 1.04x slower)
`65536`	`2.53 µs` (➖ 1.10x faster)	`2.64 µs` (➖ 1.04x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`38.14 µs` (➖ 1.29x faster)
`65536`	`9.03 µs` (➖ 1.06x faster)	`9.05 µs` (➖ 1.04x slower)
`8192`	`1.33 µs` (➖ 1.05x faster)	`1.56 µs` (➖ 1.00x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.23 µs` (➖ 1.04x slower)
`vec_bytes`	`779.27 ns` (➖ 1.03x slower)
		`3.11 ms` (➖ 1.04x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`751.52 ns` (➖ 1.01x faster)
`65536`	`207.67 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.91 ms` (➖ 1.08x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.72 ns` (➖ 1.00x slower)	`10.43 ns` (➖ 1.16x faster)	`9.70 ns` (➖ 1.01x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`667.99 µs` (➖ 1.03x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.51 µs` (➖ 1.01x faster)	`7.67 µs` (➖ 1.00x faster)
`65536`	`2.46 µs` (➖ 1.06x slower)	`2.31 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`40.23 µs` (➖ 1.25x faster)
`65536`	`8.62 µs` (➖ 1.00x slower)	`8.37 µs` (➖ 1.02x slower)
`8192`	`1.06 µs` (➖ 1.00x faster)	`1.40 µs` (➖ 1.03x faster)

_{Reported by cargo ci bench-report --candidate run:42127848766 --config-file bench_report.toml.}

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

ludfjig requested review from danbugs, dblnz, devigned and jsturtevant as code owners November 5, 2026 22:49

ludfjig added the C-enhancement For PRs adding features, improving functionality, docs, tests, etc. label Nov 5, 2026

ludfjig requested review from andreiltd, jprendes, simongdavies, squillace and syntactically as code owners November 5, 2026 22:49

Copilot AI balanced review requested due to automatic review settings November 5, 2026 22:49

Copilot started reviewing on behalf of ludfjig November 5, 2026 22:55

View session

ludfjig marked this pull request as draft November 5, 2026 22:59

Copilot AI reviewed Nov 5, 2026

View reviewed changes

ludfjig force-pushed the unify-partition-features branch from 0b47648 to cc462d0 Compare November 5, 2026 23:12

Expose one CPU feature set to MSHV and WHP guests

f2e1995

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

ludfjig force-pushed the unify-partition-features branch from cc462d0 to f2e1995 Compare November 6, 2026 19:10

Conversation

ludfjig commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

ludfjig Nov 5, 2026

Choose a reason for hiding this comment

Uh oh!

hyperlight-gh-bot Bot commented Nov 5, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ludfjig commented Nov 5, 2026 •

edited

Loading