VirtualBox driver: MachineDebugger and VBoxVMI access methods (phase 1: VM state access) #269

Problem Statement

VMI users who run their guests on VirtualBox cannot introspect them with libmicrovmi on any supported VirtualBox release.

The only VirtualBox access method libmicrovmi has today is FDP, the icebox protocol. It requires icebox's patched VirtualBox 5.2. VirtualBox 5.2 has been end-of-life since 2020, and icebox has had no activity since 2022. Stock VirtualBox offers no memory access either: IMachineDebugger's four memory methods (read/write × physical/virtual) have returned "not implemented" for over a decade, up to and including current main (issue #216, VirtualBox ticket #10222).

The existing driver also has correctness bugs that silently return wrong data:

• It reports RAM size as the max physical address, which truncates guests with more than ~3.5 GiB because their RAM above 4 GiB is never read.
• It hardcodes the vCPU count to 1.
• It fills only the segment bases, leaving out RFLAGS, MSRs, TR and LDT.

Building it also drags an unpinned clone of icebox into CI, the Python Dockerfile and the Gitpod Dockerfile.

Solution

The VirtualBox Driver gains two access methods against current VirtualBox, both providing full VM state access. Phase 1 covers physical memory read/write, registers read/write, pause/resume, vCPU count, and the max physical address derived from RAM ranges.

• MachineDebugger method: uses VirtualBox's public IMachineDebugger COM API, after a small patch that implements its four memory methods. The patch is designed for upstream inclusion in VirtualBox, so that eventually no patched VirtualBox is needed. Slower, which is fine for forensics.
• VBoxVMI method: speaks VBoxVMI, a new protocol served from inside a patched VirtualBox VM process over a Unix domain socket. Faster, and the foundation for VMI events in phase 2. It replaces FDP and is not compatible with it.

The FDP access method and every icebox dependency are removed. The user selects the access method at init time, and a single virtualbox Cargo feature enables both.

User Stories

1. As a forensics analyst, I want to read the physical memory of a VM running on stock VirtualBox, so that I can analyse it without rebuilding VirtualBox once the upstream patch lands.
2. As a forensics analyst, I want to run Volatility3 against a live VirtualBox VM through libmicrovmi's Python bindings, so that I can investigate a running system without taking a full memory dump first.
3. As a forensics analyst, I want a full physical memory dump of a VirtualBox guest with more than 4 GiB of RAM to include memory above 4 GiB, so that no evidence is silently missing.
4. As a forensics analyst, I want unreadable regions (MMIO holes) to be reported as unreadable rather than aborting the whole read, so that a dump tool can zero-fill them and keep going.
5. As a VMI developer, I want to read all vCPU registers (general purpose, RIP, RFLAGS, control registers, segment registers with base/limit/selector, GDTR/IDTR, and the MSRs libmicrovmi exposes) in one call, so that register dumps are fast and complete.
6. As a VMI developer, I want to write vCPU registers, so that I can modify guest execution state.
7. As a VMI developer, I want to write guest physical memory, so that I can patch the guest from outside.
8. As a VMI developer, I want the real vCPU count of the VM, so that I can iterate over every vCPU.
9. As a VMI developer, I want the max physical address to be the end of the highest RAM range, so that it behaves the same as the Xen, KVM and memflow drivers.
10. As a VMI developer, I want pausing to be synchronous, so that registers and memory I read immediately after pausing are consistent.
11. As a VMI developer, I want pause to be idempotent, so that pausing an already-paused VM does not error or require two resumes.
12. As a VMI developer, I want to pick the access method in the VirtualBox driver's init parameters, so that the same code works with either method.
13. As a VMI developer, I want the VBoxVMI method to be fast enough for page-by-page reads of whole guest RAM, so that tools that read one 4 KiB page per call (Volatility, mem-dump) remain practical.
14. As a VMI developer, I want batched multi-page reads in VBoxVMI, so that bulk reads amortise the per-request cost.
15. As a VMI developer, I want a clear error if the VBoxVMI server is missing, disabled or speaks a different protocol version, so that I can tell configuration problems from bugs.
16. As a VMI developer, I want a clear error if the MachineDebugger memory methods are not implemented by the installed VirtualBox, so that I know I need a patched or newer VirtualBox.
17. As a VirtualBox user, I want VBoxVMI to be off unless I enable it per VM, so that VMs I don't introspect expose nothing new.
18. As a VirtualBox user, I want the VBoxVMI socket to be accessible only by my user, so that other local users cannot introspect my VMs.
19. As a VirtualBox user, I want my VM to resume and be left clean if a VMI tool crashes or disconnects, so that a buggy tool never freezes my guest.
20. As a VirtualBox user, I want only one VBoxVMI client connected per VM at a time, so that two tools cannot corrupt each other's session.
21. As a VirtualBox user, I want a VM paused by a VMI tool to show as Paused in VirtualBox, so that I understand its state.
22. As a VirtualBox user, I want to configure the VBoxVMI socket path per VM, with a sensible default under my VirtualBox user home, so that I can control where it lives.
23. As a VirtualBox maintainer, I want the IMachineDebugger memory patch to be small, follow existing VMM function-table conventions, and use the existing debugger memory API, so that it is easy to review and merge.
24. As a VirtualBox maintainer, I want the four memory methods documented in the API reference, so that SDK users know their semantics and limits.
25. As a VirtualBox maintainer, I want the reasoning for not gating the memory API documented, so that I can evaluate the security trade-off.
26. As a VirtualBox SDK user (Python, C), I want readPhysicalMemory and its siblings to work, so that I can build my own tools without libmicrovmi.
27. As a libmicrovmi maintainer, I want the FDP access method, the fdp dependency and every icebox build step removed, so that CI no longer depends on an unmaintained, unpinned external repository.
28. As a libmicrovmi maintainer, I want separate virtualbox-vmi and virtualbox-api Cargo features, with virtualbox enabling both, so that existing --features virtualbox builds keep working and platforms can opt into one method.
29. As a libmicrovmi maintainer, I want the Windows CI job to build the MachineDebugger method only, so that Windows support keeps compiling while VBoxVMI is Linux-only.
30. As a libmicrovmi maintainer, I want the VirtualBox driver documentation to describe both access methods, their requirements and configuration, so that users can set them up.
31. As a libmicrovmi maintainer, I want a reproducible way to build and run a development VirtualBox, so that I and contributors can test changes without installing VirtualBox build dependencies on the host.
32. As a libmicrovmi maintainer, I want the integration test suite to run against both VirtualBox access methods, so that regressions in either are caught.
33. As a libmicrovmi maintainer, I want tests to verify that bytes read are the guest's real bytes and that bytes written really land in guest memory, so that the API working is not mistaken for the data being right.
34. As a future contributor, I want a written VBoxVMI protocol specification and design notes, so that I can write another client or extend the server for phase 2.
35. As a future contributor, I want the protocol to carry a version and a request id on every message, so that it can evolve and so that phase-2 events can share the connection with commands.

Implementation Decisions

Vocabulary follows CONTEXT.md: Driver, access method, VBoxVMI, VBoxVMI method, MachineDebugger method, FDP (legacy), RAM ranges, VM state access, VMI events.

VirtualBox repository (developed in a fork; both branches are based on VirtualBox main and are combined in a local, unpushed integration branch for testing):

• Upstream branch — IMachineDebugger memory methods.

  • Implements readPhysicalMemory, writePhysicalMemory, readVirtualMemory and writeVirtualMemory with one shared helper built on the debugger memory API (DBGFR3MemRead/DBGFR3MemWrite). That API transparently runs the access on the vCPU's thread and bypasses MMIO access handlers, so reads never trigger device side effects.
  • The two missing entry points (DBGFR3MemWrite, DBGFR3AddrFromPhys) are added to the VMM function table by consuming reserved slots in the debugger section. This is the established, ABI-compatible convention and needs no table version bump.
  • No gate (no build flag or per-VM setting): any API client able to reach a VM can already dump its memory, inject NMIs and rewrite registers. The rationale goes in the PR description.
  • Commits carry Signed-off-by for the Oracle Contributor Agreement.
  • Docs: method documentation in the API definition (which generates the SDK reference) and a user-manual note.
  • Known cost: reads on a running VM wait for the target vCPU's next VM exit. That is acceptable for this method.

• VBoxVMI branch — server.

  • A new ring-3 module compiled into the VMM library behind a build flag (VBOX_WITH_VMI), started at VM creation the same way the debugger console is, and enabled per VM via VBoxInternal/VMI/... extradata. It must live inside the VMM because it uses VMM-internal APIs.
  • Transport: a Unix domain socket, SOCK_SEQPACKET. The path comes from per-VM config, defaulting to the VirtualBox user home under vmi/<vm-uuid>.sock. The socket is mode 0600, the peer uid is checked on connect, and one client is allowed at a time.
  • Messages: a fixed header carrying size, request id and command or event code, with replies matched by id. The first exchange is a handshake with magic and protocol version.
  • Phase-1 commands:
    • Batched physical read and write: multiple ranges per request, a per-page status in the reply, and an unreadable page never fails the whole request. Served on the server thread through the external physical-access PGM functions, so RAM accesses don't hop to an EMT.
    • Read and write the full register set of one vCPU in one round trip: a single worker on that vCPU's EMT that reads or writes the CPU context directly, including the MSRs libmicrovmi exposes. Needed because DBGF's batch register query is compiled out and its writes are per-register.
    • Pause and resume using VirtualBox's standard suspend and resume. This is visible to the GUI and VBoxManage. Pause is synchronous and idempotent.
    • vCPU count.
    • RAM ranges: the list of guest-physical RAM ranges.
  • On disconnect, for any reason, the server resumes anything it paused and removes any intercepts.
  • Docs: protocol specification, design notes with decisions, and a user-manual section on enabling VBoxVMI.

New Rust crates (unpublished during development, consumed by path):

• vboxapi: a minimal binding to VirtualBox's C API glue, loaded at runtime from the installed VirtualBox. It covers only what the MachineDebugger method needs: attach a shared session to a running VM by name, the console pause/resume, the machine debugger's memory and register methods, and the machine's CPU count.
• vboxvmi: a pure-Rust VBoxVMI client. It replaces the C++ libFDP and the fdp crate.

libmicrovmi:

• One DriverType::VirtualBox. VBoxInitParams, an empty enum today, gains one variant per access method: MachineDebugger (VM name) and VBoxVMI (socket path). The Python and C bindings expose the same choice.
• Cargo features: virtualbox-vmi, virtualbox-api, and virtualbox which enables both.
• The FDP access method, the fdp dependency, and all icebox clone and libFDP build steps in CI, the Python Dockerfile and the Gitpod Dockerfile are deleted.
• Fixes in the VirtualBox driver for both access methods:
  • max physical address = end of the highest RAM range,
  • real vCPU count,
  • complete register set: RFLAGS, MSRs, TR, LDT, and segment limit and selector,
  • multi-page reads issued as batches where the access method supports it.
• Linux hosts only for the VBoxVMI method. The Windows CI job builds virtualbox-api only.

Development environment: VirtualBox is built in an Ubuntu 24.04 container, headless only (no Qt) and with hardening disabled, against the host's stock 6.8.0-*-generic kernel headers. The build is run from the build output directory on the host, and the kernel modules are loaded with insmod. A hardened build is checked once before the upstream PR.

ADRs will record: VBoxVMI over a Unix socket rather than shared memory; no gate on the upstream memory API; removal of the FDP access method.

Testing Decisions

• One seam: the Introspectable trait, exercised through the existing integration test harness. The harness already runs driver-agnostic tests against a real VM through a per-driver context: setup restores a snapshot, init creates the driver, teardown cleans up. KVM is the prior art.
• A VirtualBox context is added and runs once per access method. Setup restores a VBoxManage snapshot of a small Linux guest (Alpine) and starts it headless. Teardown powers it off. The existing generic tests run unchanged against both methods.
• Good tests check observable behaviour against an independent source of truth, not internal calls. Agreement between the two access methods alone is not sufficient, because both could share a bug. So VirtualBox-specific tests in the same harness check:
  • Read correctness against ground truth: with the VM paused, every RAM page read through each method is byte-identical to the same page in a VBoxManage debugvm dumpvmcore core file. The core dump uses VirtualBox's separate core-dump path.
  • Write correctness:
    • Write a random pattern to selected guest RAM pages through one method.
    • Verify it in a fresh core dump, through the other method, and after a resume/re-pause cycle.
    • Restore the original bytes afterwards.
    • Writes that silently do nothing or land elsewhere fail.
  • Semantic sanity:
    • Walking the guest page tables from CR3 translates RIP and the IDT base to mapped guest-physical pages.
    • The Linux kernel banner string is found in guest RAM.
    • Registers from both methods are identical while paused.
  • Coverage: on a guest configured with more than 4 GiB of RAM, the max physical address exceeds 4 GiB and reads above 4 GiB match the core dump.
  • vCPU count matches the VM's configured count on a multi-vCPU guest.
  • Pause semantics: pause is idempotent, and a paused VM's registers do not change between two reads.
  • Robustness (VBoxVMI): killing a client while it holds the VM paused leaves the VM running. A second concurrent client is refused.
  • Error paths: connecting to a VM with VBoxVMI disabled, or using the MachineDebugger method against an unpatched VirtualBox, yields a clear error.
• The vboxvmi and vboxapi crates and the in-VirtualBox server are covered through this seam. No unit tests against a fake server.
• The upstream PR additionally ships a short VirtualBox SDK (Python) script, so Oracle reviewers can verify the methods without Rust.
• CI continues to build only (including --features virtualbox). Integration tests run on a developer host with VirtualBox, as for KVM today.

Out of Scope

• Phase 2, VMI events: breakpoints, CR/MSR write events, page-access (EPT) events, singlestep, page-access get/set commands, and any pause invisible to the GUI. Known constraints, recorded for later:
  • CR/MSR events need either DBGF's exclusive debugger attach or new HM/NEM hooks.
  • Execute-only page traps need new PGM/ring-0 work.
  • Breakpoints can use DBGF's breakpoint-owner API.
• VBoxVMI on Windows or macOS hosts, and ARM guests.
• Shared-memory or zero-copy data paths. They can be added later over the socket if profiling shows the need.
• Exposing RAM ranges through the Introspectable trait. They are used internally for the max physical address.
• Publishing the new crates, upstreaming VBoxVMI, and contacting VirtualBox maintainers.
• Running integration tests in CI.

Further Notes

• Prior discussion: issue Native VirtualBox driver via builtin debugger interface #216, and a 2025 vbox-dev thread in which an Oracle developer agreed to raise VMI with VirtualBox's lead developer (no outcome yet). That thread proposed a compile-time flag, which this spec deliberately drops (see Implementation Decisions).
• The FDP protocol and icebox's VirtualBox 5.2 patch were studied as reference. Its in-debugger-console server and ring-0 VT-x hooks are not ported.
• Suggested order:
  1. upstream patch,
  2. vboxapi and the MachineDebugger method, which proves build, guest and harness end to end on the smallest change,
  3. VBoxVMI server, vboxvmi crate and the VBoxVMI method,
  4. docs alongside each step.