feat(aarch64): add GDB debugging for Linux KVM by cshung · Pull Request #2016 · hyperlight-dev/hyperlight

cshung · 2026-11-03T00:34:05Z

ARM64 Linux KVM GDB

KVM guests support register and memory access, two hardware breakpoints,
BRK software breakpoints, and hardware single-step.
KVM reports debug stops through KVM_EXIT_DEBUG.
Debug state survives vCPU reset, and register access respects the active stack pointer.

Commit layers

This PR is C, one KVM-specific commit above shared support A in #2050.
WHP support in #1907 is an independent sibling, not a prerequisite.

KVM-only comparison.

Shared page-table translation, guest-page-sized memory access, snapshot
recovery, cancellation and crash-detach handling belong to #2050.
This layer contains KVM integration, platform enablement and native GDB CI.

The graph and update rules are in
docs/gdb-commit-layers.md.
The full PR diff includes the unmerged shared prerequisite.

Validation

Formatting and ARM64 GDB-enabled cross-Clippy pass with default and KVM-only features.
KVM backend source matches the tested combined tree.
Shared support passes native ARM64 MSHV formatting, Clippy, build, the full
just test recipe, 26 debugger/backend tests and seven real GDB client tests.
Earlier Raspberry Pi 4 validation passed 16 targeted native unit tests and six
real GDB client tests. Native KVM ARM64 runtime tests were not repeated for these
shared fixes.

Pi validation uses a separate, uncommitted ARMv8.0 allocator fix and guests built
with RUSTFLAGS="-C target-feature=-lse". Pi compatibility changes are outside this PR.

Copilot

🟡 Changes recommended

Unresolved critical flaws allow guest modification of executable trampoline code and debugger memory corruption, while breakpoint cleanup and snapshot-ordering issues can leave sandboxes unrecoverable.

5 open issues

What changed in this PR

Adds ARM64 guest debugging on Linux/KVM, alongside the shared debugger infrastructure and Windows/WHP support from #1907.

Changes:

Supports ARM64 register and memory access, software and hardware breakpoints, and hardware single-step.
Adds instruction-cache synchronization, debug-state preservation across reset, and snapshot safeguards.
Adds debugger tests and documents platform capabilities.

File	Description
`src/hyperlight_host/src/sandbox/uninitialized_evolve.rs`	Initializes ARM64 debug vectors during VM setup.
`src/hyperlight_host/src/sandbox/snapshot/memory/backing.rs`	Exposes backing ranges for cache synchronization.
`src/hyperlight_host/src/sandbox/initialized.rs`	Guards snapshots and restores against active software breakpoints.
`src/hyperlight_host/src/mem/mgr.rs`	Exposes checked host memory ranges to the debugger.
`src/hyperlight_host/src/mem/exe.rs`	Carries debug-vector addresses in load metadata.
`src/hyperlight_host/src/mem/elf.rs`	Finds and relocates the debug-vector symbol.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/aarch64.rs`	Implements WHP debugging, cache synchronization, and tests.
`src/hyperlight_host/src/hypervisor/virtual_machine/mshv/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/mod.rs`	Adds ARM64 debug exits and initialization errors.
`src/hyperlight_host/src/hypervisor/virtual_machine/kvm/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/kvm/aarch64.rs`	Implements KVM debugging and active-stack-pointer handling.
`src/hyperlight_host/src/hypervisor/regs/aarch64/whp_reg.rs`	Defines WHP debug register identifiers.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/x86_64.rs`	Uses shared debugger handling and breakpoint storage.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs`	Integrates ARM64 stops, stepping, and mapping safeguards.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/debug.rs`	Provides shared requests and software-breakpoint lifecycle handling.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/aarch64.rs`	Connects ARM64 VM setup and execution to GDB.
`src/hyperlight_host/src/hypervisor/gdb/target.rs`	Adds ARM64 register conversion to the shared target.
`src/hyperlight_host/src/hypervisor/gdb/mod.rs`	Adds page-table translation and cache-flush support.
`src/hyperlight_host/src/hypervisor/gdb/event_loop.rs`	Reports ARM64 initial stops to GDB.
`src/hyperlight_host/src/hypervisor/gdb/arch/x86_64.rs`	Houses x86 breakpoint constants and stop classification.
`src/hyperlight_host/src/hypervisor/gdb/arch/aarch64.rs`	Defines ARM64 software breakpoints and alignment checks.
`src/hyperlight_host/src/hypervisor/gdb/arch.rs`	Selects architecture-specific debugger helpers.
`src/hyperlight_host/examples/guest-debugging/main.rs`	Adds ARM64 breakpoint, stepping, and code-write integration tests.
`src/hyperlight_host/build.rs`	Enables GDB for supported ARM64 platforms.
`src/hyperlight_guest_bin/src/arch/aarch64/exception/entry.rs`	Exports debug vectors and forwards synchronous exceptions.
`src/hyperlight_common/src/arch/aarch64/exn.rs`	Defines the shared debug-exception HVC immediate.
`docs/how-to-debug-a-hyperlight-guest.md`	Documents ARM64 capabilities and guest-symbol requirements.
`CHANGELOG.md`	Records Linux/KVM and Windows/WHP ARM64 debugging support.

_{Review effort: Balanced}

Architecture adapters, software breakpoints, page-sized memory access and recovery handling are shared by backend layers. Guest debug vectors and common tests support ARM64. Backend layers control platform enablement. The commit-layer document defines dependencies and shared-update validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

KVM debug exits provide hardware breakpoints and single-step. Register access respects the active stack pointer. Native ARM64 GDB tests run in debug CI. Shared debugger behavior belongs to the parent commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

hyperlight-gh-bot · 2026-11-12T19:19:28Z

Benchmark Results

Measured commit: 68a8704b5f6b
Baseline commit: 01df99173865

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`764.19 ns` (➖ 1.01x slower)
`vec_bytes`	`597.80 ns` (➖ 1.01x slower)
		`374.01 µs` (➖ 1.01x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`521.29 ns` (➖ 1.01x faster)
`65536`	`141.21 ns` (➖ 1.02x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`13.43 ms` (➖ 1.01x slower)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`7.69 ns` (➖ 1.00x slower)	`7.68 ns` (➖ 1.00x slower)	`7.68 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`99.12 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.27 µs` (➖ 1.04x slower)	`7.20 µs` (➖ 1.01x faster)
`65536`	`1.92 µs` (➖ 1.09x faster)	`2.03 µs` (➖ 1.06x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`27.00 µs` (➖ 1.01x faster)
`65536`	`6.18 µs` (➖ 1.01x faster)	`6.16 µs` (➖ 1.03x faster)
`8192`	`1.01 µs` (➖ 1.03x faster)	`1.03 µs` (➖ 1.01x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`797.14 ns` (➖ 1.02x slower)
`vec_bytes`	`586.43 ns` (➖ 1.02x faster)
		`734.39 µs` (➖ 1.01x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`576.17 ns` (➖ 1.01x faster)
`65536`	`199.38 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`15.31 ms` (➖ 1.02x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`7.95 ns` (➖ 1.00x faster)	`7.94 ns` (➖ 1.01x faster)	`7.94 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`54.38 µs` (➖ 1.02x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`8.72 µs` (➖ 1.01x faster)	`8.72 µs` (➖ 1.00x faster)
`65536`	`2.47 µs` (➖ 1.00x slower)	`2.43 µs` (➖ 1.02x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`33.95 µs` (➖ 1.01x faster)
`65536`	`8.16 µs` (➖ 1.01x faster)	`8.23 µs` (➖ 1.02x faster)
`8192`	`834.13 ns` (➖ 1.00x faster)	`850.21 ns` (➖ 1.00x slower)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`929.77 ns` (➖ 1.01x faster)
`vec_bytes`	`706.63 ns` (➖ 1.00x slower)
		`320.88 µs` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`673.78 ns` (➖ 1.00x slower)
`65536`	`182.19 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`67.20 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.87 ns` (➖ 1.02x faster)	`9.86 ns` (➖ 1.00x faster)	`9.92 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`109.56 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.00 µs` (➖ 1.04x faster)	`9.22 µs` (➖ 1.00x faster)
`65536`	`2.50 µs` (➖ 1.00x slower)	`2.65 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`34.65 µs` (➖ 1.02x faster)
`65536`	`7.99 µs` (➖ 1.01x faster)	`8.08 µs` (➖ 1.14x faster)
`8192`	`1.39 µs` (➖ 1.11x slower)	`1.29 µs` (➖ 1.00x faster)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`933.86 ns` (➖ 1.00x slower)
`vec_bytes`	`659.29 ns` (➖ 1.00x slower)
		`668.69 µs` (➖ 1.06x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`663.37 ns` (➖ 1.00x slower)
`65536`	`252.34 ns` (➖ 1.02x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`54.72 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`8.70 ns` (➖ 1.00x slower)	`8.69 ns` (➖ 1.15x faster)	`8.74 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`54.00 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.90 µs` (➖ 1.01x faster)	`8.04 µs` (➖ 1.02x slower)
`65536`	`2.28 µs` (➖ 1.00x slower)	`2.29 µs` (➖ 1.01x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`37.79 µs` (➖ 1.02x slower)
`65536`	`7.54 µs` (➖ 1.04x faster)	`7.50 µs` (➖ 1.01x faster)
`8192`	`903.73 ns` (➖ 1.02x faster)	`893.66 ns` (➖ 1.01x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.18 µs` (➖ 1.02x slower)
`vec_bytes`	`743.81 ns` (➖ 1.00x faster)
		`2.27 ms` (➖ 1.02x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`939.35 ns` (➖ 1.19x slower)
`65536`	`230.82 ns` (➖ 1.03x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.21 ms` (➖ 1.11x slower)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`10.59 ns` (➖ 1.00x slower)	`10.62 ns` (➖ 1.10x faster)	`10.60 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`653.75 µs` (➖ 1.04x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`8.60 µs` (➖ 1.01x slower)	`8.60 µs` (➖ 1.00x slower)
`65536`	`2.49 µs` (➖ 1.00x faster)	`2.51 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`38.73 µs` (➖ 1.01x slower)
`65536`	`8.99 µs` (➖ 1.02x faster)	`8.79 µs` (➖ 1.01x slower)
`8192`	`1.35 µs` (➖ 1.04x faster)	`1.35 µs` (➖ 1.02x faster)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.22 µs` (➖ 1.00x faster)
`vec_bytes`	`830.46 ns` (➖ 1.06x slower)
		`3.16 ms` (➖ 1.00x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`819.25 ns` (➖ 1.09x slower)
`65536`	`228.76 ns` (➖ 1.09x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.78 ms` (➖ 1.06x slower)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`10.05 ns` (➖ 1.00x faster)	`10.40 ns` (➖ 1.14x faster)	`9.90 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`677.74 µs` (➖ 1.02x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.87 µs` (➖ 1.02x slower)	`7.63 µs` (➖ 1.00x faster)
`65536`	`2.37 µs` (➖ 1.03x slower)	`2.31 µs` (➖ 1.02x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`40.84 µs` (➖ 1.09x faster)
`65536`	`8.29 µs` (➖ 1.03x faster)	`8.39 µs` (➖ 1.04x faster)
`8192`	`1.07 µs` (➖ 1.12x faster)	`1.11 µs` (➖ 1.04x slower)

_{Reported by cargo ci bench-report --candidate run:43277989649 --config-file bench_report.toml.}

cshung added C-enhancement For PRs adding features, improving functionality, docs, tests, etc. S-needs-review PR is ready for (re-)review labels Nov 3, 2026

Copilot AI balanced review requested due to automatic review settings November 3, 2026 00:34

cshung requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, ludfjig, simongdavies, squillace and syntactically as code owners November 3, 2026 00:34

cshung marked this pull request as draft November 3, 2026 00:36

cshung removed the S-needs-review PR is ready for (re-)review label Nov 3, 2026

Copilot started reviewing on behalf of cshung November 3, 2026 00:37

View session

Copilot AI reviewed Nov 3, 2026

View reviewed changes