Fix flaky reset_vcpu_simple and log cpu in CI by ludfjig · Pull Request #2018 · hyperlight-dev/hyperlight

ludfjig · 2026-11-03T19:19:20Z

reset_vcpu_simple fails intermittently (#1341). It sets FPU state and checks that it reads back unchanged, including the x87 error pointers FOP, FIP and FDP. The test leaves FSW.ES clear, so those pointers are undefined, and AMD CPUs save them as 0 in that state. The test now skips comparing them.

build-test jobs also log the OS and hypervisor versions and the CPU model and features before building, to help debug failures that depend on the runner.

Fixes #1341

reset_vcpu_simple sets dirty FPU state and checks that it reads back unchanged. On a Windows AMD runner, FOP, FIP and FDP read back as 0. The test's dirty FSW leaves ES clear, so these pointers are undefined. AMD CPUs save them as 0 while no x87 exception is pending, and the Hyper-V register API passes x87 state through XSAVE. Stop comparing them. Fixes hyperlight-dev#1341 Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

Print the OS and hypervisor versions and the CPU model and features before building, to help debug failures that depend on the runner. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

hyperlight-gh-bot · 2026-11-03T21:21:20Z

Benchmark Results

Measured commit: 0da1537108a3
Baseline commit: 35dd23664883

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`375.57 µs` (➖ 1.01x slower)
`byte_chunks`		`753.46 ns` (➖ 1.01x faster)
`vec_bytes`		`580.29 ns` (➖ 1.03x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`143.36 ns` (➖ 1.01x slower)
`262144`	`567.47 ns` (➖ 1.09x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.25 ms` (➖ 1.05x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`7.71 ns` (➖ 1.04x faster)	`7.71 ns` (➖ 1.00x slower)	`7.91 ns` (➖ 1.03x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`101.61 µs` (➖ 1.03x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`1.95 µs` (➖ 1.07x faster)	`2.07 µs` (➖ 1.09x slower)
`262144`	`7.09 µs` (➖ 1.00x slower)	`7.03 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.03 µs` (➖ 1.00x slower)	`1.02 µs` (➖ 1.00x faster)
`65536`	`6.17 µs` (➖ 1.05x faster)	`6.19 µs` (➖ 1.02x faster)
`262144`	`28.36 µs` (➖ 1.03x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`671.01 µs` (➖ 1.06x faster)
`byte_chunks`		`697.56 ns` (➖ 1.07x faster)
`vec_bytes`		`525.90 ns` (➖ 1.15x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`159.83 ns` (➖ 1.11x faster)
`262144`	`515.48 ns` (➖ 1.09x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.56 ms` (➖ 1.08x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`7.03 ns` (➖ 1.08x faster)	`7.03 ns` (➖ 1.09x faster)	`7.05 ns` (➖ 1.09x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`49.99 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.17 µs` (➖ 1.10x faster)	`2.19 µs` (➖ 1.10x faster)
`262144`	`7.79 µs` (➖ 1.10x faster)	`7.79 µs` (➖ 1.08x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`759.92 ns` (➖ 1.05x faster)	`742.67 ns` (➖ 1.10x faster)
`65536`	`7.35 µs` (➖ 1.07x faster)	`7.25 µs` (➖ 1.10x faster)
`262144`	`30.98 µs` (➖ 1.08x faster)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`337.91 µs` (➖ 1.04x slower)
`byte_chunks`		`963.61 ns` (➖ 1.02x slower)
`vec_bytes`		`728.62 ns` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`65536`	`192.93 ns` (➖ 1.00x faster)
`262144`	`729.14 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`66.16 ms` (➖ 1.07x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`9.63 ns` (➖ 1.01x faster)	`10.39 ns` (➖ 1.07x slower)	`9.64 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`105.66 µs` (➖ 1.00x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.26 µs` (➖ 1.01x faster)	`2.43 µs` (➖ 1.04x faster)
`262144`	`10.27 µs` (➖ 1.00x slower)	`10.41 µs` (➖ 1.02x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.29 µs` (➖ 1.03x slower)	`1.30 µs` (➖ 1.03x slower)
`65536`	`8.34 µs` (➖ 1.20x faster)	`8.29 µs` (➖ 1.18x faster)
`262144`	`35.83 µs` (➖ 1.06x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`688.85 µs` (➖ 1.06x slower)
`byte_chunks`		`943.56 ns` (➖ 1.01x slower)
`vec_bytes`		`652.33 ns` (➖ 1.00x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`281.02 ns` (➖ 1.10x slower)
`262144`	`658.46 ns` (➖ 1.23x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`99.08 ms` (➖ 1.05x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`8.70 ns` (➖ 1.00x faster)	`8.70 ns` (➖ 1.04x faster)	`8.70 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`65.48 µs` (➖ 1.15x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.29 µs` (➖ 1.00x slower)	`2.29 µs` (➖ 1.00x slower)
`262144`	`7.92 µs` (➖ 1.01x faster)	`7.94 µs` (➖ 1.04x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`913.60 ns` (➖ 1.00x faster)	`908.75 ns` (➖ 1.00x faster)
`65536`	`7.48 µs` (➖ 1.01x faster)	`7.56 µs` (➖ 1.00x slower)
`262144`	`40.29 µs` (➖ 1.01x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`1.91 ms` (➖ 1.00x slower)
`byte_chunks`		`1.22 µs` (➖ 1.05x slower)
`vec_bytes`		`795.89 ns` (➖ 1.09x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`279.16 ns` (➖ 1.21x slower)
`262144`	`979.45 ns` (➖ 1.25x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`11.20 ms` (➖ 1.01x slower)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`11.48 ns` (➖ 1.14x slower)	`11.10 ns` (➖ 1.04x slower)	`11.13 ns` (➖ 1.09x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`778.72 µs` (➖ 1.17x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.42 µs` (➖ 1.04x slower)	`2.49 µs` (➖ 1.08x slower)
`262144`	`9.59 µs` (➖ 1.04x slower)	`9.11 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.30 µs` (➖ 1.01x slower)	`1.34 µs` (➖ 1.05x slower)
`65536`	`8.97 µs` (➖ 1.06x faster)	`9.16 µs` (➖ 1.04x slower)
`262144`	`45.03 µs` (➖ 1.03x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`decode_vec_bytes_copy`	`encode_control`
	`3.10 ms` (➖ 1.00x slower)
`byte_chunks`		`1.21 µs` (➖ 1.03x faster)
`vec_bytes`		`754.61 ns` (➖ 1.03x faster)

payload_allocation

	`slot_pool_segmented`
`65536`	`214.00 ns` (➖ 1.01x faster)
`262144`	`752.55 ns` (➖ 1.02x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.94 ms` (➖ 1.12x faster)

slot_pool

	`alloc_dealloc_4096`	`alloc_dealloc_128`	`alloc_dealloc_1500`
	`10.43 ns` (➖ 1.03x slower)	`9.83 ns` (➖ 1.02x faster)	`9.93 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`632.71 µs` (➖ 1.11x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`65536`	`2.47 µs` (➖ 1.07x slower)	`2.29 µs` (➖ 1.00x slower)
`262144`	`7.57 µs` (➖ 1.01x faster)	`7.67 µs` (➖ 1.02x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`8192`	`1.09 µs` (➖ 1.05x slower)	`1.22 µs` (➖ 1.13x slower)
`65536`	`8.29 µs` (➖ 1.01x faster)	`8.37 µs` (➖ 1.00x faster)
`262144`	`43.59 µs` (➖ 1.16x faster)

_{Reported by cargo ci bench-report --candidate run:41708627118 --config-file bench_report.toml.}

Copilot

🟢 Approval recommended

No active review issues block the FPU assertion adjustment and best-effort runner diagnostics.

No open issues 🎉

What changed in this PR

This PR addresses the intermittent reset_vcpu_simple failure reported in #1341 and adds runner diagnostics to help investigate hardware-dependent CI failures.

Changes:

Skip comparisons of x87 error pointers when FSW.ES is clear, while retaining the other FPU checks.
Log platform-specific OS, hypervisor, and CPU details before building. Diagnostic failures do not fail the job.

File	Description
`src/hyperlight_host/src/hypervisor/hyperlight_vm/x86_64.rs`	Adjust FPU assertions to tolerate hardware-dependent x87 error-pointer values.
`.github/workflows/dep_build_test.yml`	Add best-effort runner diagnostics for Linux, Windows, and macOS.

_{Review effort: Balanced}

ludfjig added C-bug For PRs that fix bugs A-testing Related to tests or testing infrastructure labels Nov 3, 2026

ludfjig added 2 commits November 3, 2026 12:00

Log the host CPU in build-test jobs

0da1537

Print the OS and hypervisor versions and the CPU model and features before building, to help debug failures that depend on the runner. Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>

ludfjig force-pushed the fix-fpu-pointer-flake branch from be6699e to 0da1537 Compare November 3, 2026 20:00

ludfjig marked this pull request as ready for review November 3, 2026 21:24

ludfjig changed the title ~~(test) Fix flaky reset_vcpu_simple and log the host CPU in CI~~ Fix flaky reset_vcpu_simple and log the host CPU in CI Nov 3, 2026

ludfjig requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, simongdavies, squillace and syntactically as code owners November 3, 2026 21:24

Copilot AI balanced review requested due to automatic review settings November 3, 2026 21:24

Copilot started reviewing on behalf of ludfjig November 3, 2026 21:28

View session

ludfjig mentioned this pull request Nov 3, 2026

Benchmark PR comment tables are broken #2020

Closed

Copilot AI reviewed Nov 3, 2026

View reviewed changes

ludfjig changed the title ~~Fix flaky reset_vcpu_simple and log the host CPU in CI~~ Fix flaky reset_vcpu_simple and log cpu in CI Nov 3, 2026

danbugs approved these changes Nov 3, 2026

View reviewed changes

ludfjig merged commit b8169a2 into hyperlight-dev:main Nov 4, 2026
62 checks passed

Conversation

ludfjig commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hyperlight-gh-bot Bot commented Nov 3, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ludfjig commented Nov 3, 2026 •

edited

Loading