Restore the guest TSC exactly on WHP by ludfjig · Pull Request #2022 · hyperlight-dev/hyperlight

ludfjig · 2026-11-04T00:26:29Z

On WHP hosts with TscAdjustSupport, restore does not rewind the guest TSC. Writing TSC shifts TSC_ADJUST by the same amount, and writing TSC_ADJUST shifts TSC back, so restoring both keeps the time since the snapshot.

This clears TscAdjustSupport when creating the partition, as the MSHV driver does. restore_rewinds_tsc now runs on WHP. CI hosts lack the feature, so I tested on a host that has it. The test fails without the fix and passes with it.

Fixes #2021

Copilot

🔵 Needs a closer look

The hard-coded SDK bit controlling WHP partition feature exposure remains unverified because authoritative header retrieval failed and independent runtime validation requires a Windows host with TscAdjustSupport.

No open issues 🎉

What changed in this PR

This PR makes snapshot restore rewind the guest time-stamp counter (TSC) on WHP hosts with TscAdjustSupport. It also includes the KVM restore fix from #2015, aligning guest-time behavior across Hyperlight’s backends.

Changes:

Disable TscAdjustSupport during WHP partition creation.
Include KVM’s decoy TSC write and regression tests for rewind and zero-value restore.
Document TSC restore behavior and record both fixes in the changelog.

File	Description
`src/tests/rust_guests/simpleguest/src/main.rs`	Add a guest function for reading the TSC.
`src/hyperlight_host/src/sandbox/initialized.rs`	Test TSC rewind and KVM zero-value restore.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/x86_64.rs`	Disable TSC adjustment support during partition creation.
`src/hyperlight_host/src/hypervisor/virtual_machine/kvm/x86_64.rs`	Prepend a decoy write to force TSC restoration.
`src/hyperlight_host/src/hypervisor/regs/x86_64/msrs.rs`	Remove an inaccurate comment about Hyper-V TSC storage.
`docs/msr.md`	Explain TSC rewind behavior and disabled adjustment support.
`CHANGELOG.md`	Record the KVM and WHP TSC restore fixes.

_{Review effort: Balanced}

On WHP hosts with TscAdjustSupport, restore does not rewind the guest TSC. Writing TSC shifts TSC_ADJUST by the same amount, and writing TSC_ADJUST shifts TSC back, so restoring both keeps the time since the snapshot. Clear TscAdjustSupport in the partition's processor feature banks, as the MSHV driver does. The probe then drops TSC_ADJUST from the reset set, and restore_rewinds_tsc runs on WHP too. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

hyperlight-gh-bot · 2026-11-05T21:10:29Z

Benchmark Results

Measured commit: 3245e4248354
Baseline commit: a03088470d82

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`780.34 ns` (➖ 1.02x slower)
`vec_bytes`	`595.37 ns` (➖ 1.01x slower)
		`374.47 µs` (➖ 1.00x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`526.61 ns` (➖ 1.01x slower)
`65536`	`146.56 ns` (➖ 1.04x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.64 ms` (➖ 1.02x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`7.69 ns` (➖ 1.00x faster)	`7.68 ns` (➖ 1.00x faster)	`7.68 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`99.31 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.67 µs` (➖ 1.01x slower)	`7.07 µs` (➖ 1.03x faster)
`65536`	`1.94 µs` (➖ 1.10x faster)	`2.05 µs` (➖ 1.05x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`28.29 µs` (➖ 1.04x slower)
`65536`	`6.12 µs` (➖ 1.07x faster)	`6.17 µs` (➖ 1.02x faster)
`8192`	`1.03 µs` (➖ 1.00x faster)	`1.14 µs` (➖ 1.12x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`758.35 ns` (➖ 1.12x slower)
`vec_bytes`	`503.17 ns` (➖ 1.06x faster)
		`679.83 µs` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`495.69 ns` (➖ 1.00x slower)
`65536`	`155.52 ns` (➖ 1.00x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`15.05 ms` (➖ 1.03x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`6.76 ns` (➖ 1.00x faster)	`6.78 ns` (➖ 1.01x faster)	`6.81 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`48.59 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.53 µs` (➖ 1.00x slower)	`7.50 µs` (➖ 1.01x slower)
`65536`	`2.12 µs` (➖ 1.00x slower)	`2.12 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`29.02 µs` (➖ 1.09x faster)
`65536`	`7.01 µs` (➖ 1.01x faster)	`7.02 µs` (➖ 1.00x faster)
`8192`	`713.96 ns` (➖ 1.01x slower)	`710.64 ns` (➖ 1.02x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`911.44 ns` (➖ 1.00x slower)
`vec_bytes`	`702.67 ns` (➖ 1.01x slower)
		`315.95 µs` (➖ 1.02x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`672.68 ns` (➖ 1.00x faster)
`65536`	`182.11 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`70.04 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.91 ns` (➖ 1.00x faster)	`9.86 ns` (➖ 1.21x faster)	`9.88 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`111.40 µs` (➖ 1.04x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.23 µs` (➖ 1.00x slower)	`9.22 µs` (➖ 1.02x slower)
`65536`	`2.47 µs` (➖ 1.00x slower)	`2.67 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`34.73 µs` (➖ 1.01x faster)
`65536`	`7.94 µs` (➖ 1.03x faster)	`8.01 µs` (➖ 1.02x faster)
`8192`	`1.26 µs` (➖ 1.01x slower)	`1.32 µs` (➖ 1.03x slower)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`915.99 ns` (➖ 1.02x faster)
`vec_bytes`	`654.81 ns` (➖ 1.01x faster)
		`663.82 µs` (➖ 1.03x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`662.17 ns` (➖ 1.00x faster)
`65536`	`241.19 ns` (➖ 1.07x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`56.64 ms` (➖ 1.46x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`8.69 ns` (➖ 1.00x faster)	`8.69 ns` (➖ 1.00x faster)	`8.71 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`54.83 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.97 µs` (➖ 1.00x slower)	`7.98 µs` (➖ 1.00x slower)
`65536`	`2.31 µs` (➖ 1.01x slower)	`2.30 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`39.38 µs` (➖ 1.02x slower)
`65536`	`7.41 µs` (➖ 1.02x faster)	`7.53 µs` (➖ 1.06x faster)
`8192`	`902.70 ns` (➖ 1.02x faster)	`901.93 ns` (➖ 1.02x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.19 µs` (➖ 1.02x slower)
`vec_bytes`	`803.74 ns` (➖ 1.02x slower)
		`2.05 ms` (➖ 1.05x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`784.83 ns` (➖ 1.00x slower)
`65536`	`229.56 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`12.81 ms` (➖ 1.04x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`10.21 ns` (➖ 1.03x faster)	`9.99 ns` (➖ 1.03x faster)	`9.89 ns` (➖ 1.05x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`756.30 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.25 µs` (➖ 1.21x faster)	`9.36 µs` (➖ 1.04x faster)
`65536`	`2.31 µs` (➖ 1.10x faster)	`2.36 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`38.81 µs` (➖ 1.01x faster)
`65536`	`8.86 µs` (➖ 1.03x faster)	`9.30 µs` (➖ 1.01x slower)
`8192`	`1.28 µs` (➖ 1.05x faster)	`1.32 µs` (➖ 1.01x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.31 µs` (➖ 1.10x slower)
`vec_bytes`	`754.02 ns` (➖ 1.09x faster)
		`3.13 ms` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`742.08 ns` (➖ 1.02x faster)
`65536`	`208.71 ns` (➖ 1.02x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.69 ms` (➖ 1.01x slower)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.72 ns` (➖ 1.00x slower)	`10.25 ns` (➖ 1.02x slower)	`9.64 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`674.55 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.76 µs` (➖ 1.01x slower)	`7.85 µs` (➖ 1.05x slower)
`65536`	`2.39 µs` (➖ 1.03x slower)	`2.39 µs` (➖ 1.05x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`42.73 µs` (➖ 1.06x slower)
`65536`	`8.83 µs` (➖ 1.06x slower)	`8.23 µs` (➖ 1.01x faster)
`8192`	`1.15 µs` (➖ 1.07x slower)	`1.11 µs` (➖ 1.05x slower)

_{Reported by cargo ci bench-report --candidate run:42104899724 --config-file bench_report.toml.}

ludfjig requested a review from danbugs as a code owner November 4, 2026 00:26

ludfjig added the C-bug For PRs that fix bugs label Nov 4, 2026

ludfjig requested review from andreiltd, dblnz, devigned, jprendes, jsturtevant, simongdavies, squillace and syntactically as code owners November 4, 2026 00:26

Copilot AI balanced review requested due to automatic review settings November 4, 2026 00:26

ludfjig added the C-bug For PRs that fix bugs label Nov 4, 2026

Copilot started reviewing on behalf of ludfjig November 4, 2026 00:30

View session

Copilot AI reviewed Nov 4, 2026

View reviewed changes

ludfjig force-pushed the whp-tsc-restore branch from 7fa6952 to f9412e1 Compare November 4, 2026 00:49

This comment has been minimized.

Sign in to view

ludfjig force-pushed the whp-tsc-restore branch from f9412e1 to 3245e42 Compare November 5, 2026 20:36

ludfjig mentioned this pull request Nov 5, 2026

Expose one CPU feature set to MSHV and WHP guests #2028

Draft

jsturtevant reviewed Nov 11, 2026

View reviewed changes

Comment thread src/hyperlight_host/src/hypervisor/virtual_machine/whp/x86_64.rs Resolved

jsturtevant approved these changes Nov 11, 2026

View reviewed changes

ludfjig merged commit 6635b69 into hyperlight-dev:main Nov 11, 2026
63 checks passed

Conversation

ludfjig commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

This comment has been minimized.

hyperlight-gh-bot Bot commented Nov 5, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ludfjig commented Nov 4, 2026 •

edited

Loading