Validate disk sector ranges in backends, not frontends - #4040
Conversation
Storage frontends hand guest-supplied sector numbers to disk backends, and several paths do so with no range check at all -- virtio-blk and the GED VMGS handlers among them. A read near u64::MAX therefore reaches disk_layered's sector bitmap, where `start_sector + bits.len()` panics with overflow checks enabled and silently wraps without them. Both are guest-reachable, making the panic an availability bug and the wrap a data-corruption one. Adding the checks back to the frontends would not fix this properly, because sector_count() is dynamic for several backends: a frontend that checks and then issues the I/O can be beaten by a resize in between. Only the backend can validate atomically against its own state. This change establishes that division of responsibility -- backends own range validation -- and gives them the one guarantee that makes their existing checks sound. That guarantee is arithmetic well-formedness. Disk now rejects requests whose end byte offset would exceed i64::MAX, the real limit imposed by pread64/pwrite64 and the Windows file APIs. It is deliberately not a range check: it never consults sector_count(), so it cannot mask a backend that fails to validate, and it cannot be invalidated by a resize. It is nonetheless enough to repair the checks backends already have, since sector arithmetic can no longer overflow -- disk_file's byte-offset comparison no longer discards high bits in the shift, and disk_striped's end_sector can no longer wrap to a small in-range value. The contract is documented on DiskIo and LayerIo. Disk::unmap additionally short-circuits disks reporting UnmapBehavior::Ignored, after validating the range. This is the one place Disk range checks, and it is sound because it is also the one place Disk does not delegate: with the request never reaching the backend, there is no backend check for it to be redundant with or to mask. Such disks previously returned success for an out-of-range unmap, which would become an SBC conformance violation once the frontend checks are removed. A reusable conformance suite in storage_tests exercises the awkward (sector, length) pairs -- the sector count boundary, u64::MAX, the i64::MAX byte boundary, and sector numbers that wrap when shifted into a byte offset -- against read, write and unmap. It is split into the representability cases, which every disk must pass, and the range cases, which only the backend can enforce. Every case asserts DiskError::IllegalBlock specifically, since that variant becomes guest-visible once frontends stop checking. Finally, disklayer_sqlite gains rusqlite's bundled feature as a dev-dependency so that its tests link without a system libsqlite3 development package.
The initial change covered four backends. This extends the conformance suite to twelve, adding the blob backend in both its raw and fixed-VHD1 forms, VHD1, SQLite, striped, and the crypt, delay and persistent-reservation wrappers, along with multi-layer and read-cache layered configurations. The covered set is declared as a table rather than as a run of near-identical test functions, so that which backends are actually exercised is visible at a glance and cannot drift from what runs. Two backends failed, both in ways their existing tests could not see. disk_blob presents a disk that is strictly smaller than its blob when the blob is a fixed VHD1, because of the trailing footer. A read one sector past the end of the disk therefore landed inside the footer and succeeded, returning data that is not part of the disk at all. Delegating the range check to the blob is only sound when the backing object's bounds coincide with the disk's, which is exactly what a trailing footer breaks, so the backend now checks the range itself. disklayer_sqlite had no range check in write_maybe_overwrite, and SQLite will happily insert rows for sectors beyond the end of the disk, so nothing downstream would have caught it either. Finally, disklayer_sqlite gains a bundled feature that builds SQLite from source. Test crates enable it so that they link without a system libsqlite3 development package; it is off by default and not intended for production builds.
Adds an emulated NVMe controller to the conformance table. This is the most load-bearing entry in it so far: NvmeDisk performs no range check of its own, so the test can only pass if the controller returns LBA_OUT_OF_RANGE and map_nvme_error converts it to IllegalBlock. Removing the NVMe frontend's own range check, later in this series, depends on exactly that behaviour, which up to now had been assumed from reading the code rather than demonstrated. Sharing the controller setup between the new test and the existing SCSI DVD tests meant untangling the crate's test layout first. tests.rs declared scsidvd_nvme and storvsc as modules, but both files also sat directly in tests/, where Cargo auto-discovers them as test targets in their own right. Every test in them was consequently compiled and run twice. It also made the setup impossible to share: a file included both as a crate root and as a submodule cannot resolve a `mod common;` consistently, because the path is relative to a different directory in each case. The test modules now live in a subdirectory alongside a single main.rs, which Cargo discovers as one target while leaving its siblings alone. Each module, and the harness they share, is compiled exactly once. Test names are unchanged from the ones the aggregator already produced.
HttpBlob built its range header as `offset + buf.len() - 1`. HTTP byte ranges are inclusive at both ends, so a zero-length read made that `offset + 0 - 1`, which underflows: a panic with overflow checks enabled, and in release a request for `bytes=0-18446744073709551615`, that is, the entire blob. Nothing upstream filters the request out, since BlobDisk passes an empty buffer straight through, so a guest issuing a zero-length read to an HTTP-backed disk could bring down the host. FileBlob is unaffected, which is why this only surfaced once a second Blob implementation was exercised. The conformance suite now runs against a blob served over HTTP, in both its raw and fixed-VHD1 forms, alongside a set of deliberately misbehaving responses: a body shorter than requested, a server that ignores the Range header, a 416, and a connection dropped part way through the body. These matter because BlobDisk cannot detect a short read itself and depends entirely on the Blob contract to report one as UnexpectedEof rather than as partial success, so the short-body case asserts that specific error rather than merely that the read failed. The server is hand-rolled on std::net::TcpListener. disk_blob depends on hyper with only its client features, so using a real server would mean new dependencies, and none of the responses above is one a real server would produce anyway. Also adds VHDMP to the conformance table, gated to Windows. It cannot be run here, but it type-checks and lints clean against the x86_64-pc-windows-gnu target.
The macOS CI job lints the whole workspace, excluding a handful of packages that cannot be cross compiled for it -- disk_crypt among them, because there is no crypto backend there. Cargo's --exclude keeps a package from being linted, not from being built as a dependency of something that is, so adding disk_crypt and a bundled-SQLite build to storage_tests' dev-dependencies broke that job. Both are now limited to non-macOS targets, along with the two fixtures that use them, which is narrower than excluding the whole crate and leaves everything else linted there. While confirming this, the comment explaining why the emulated NVMe tests are limited to Windows and Linux turned out to be wrong. The harness itself is portable: user_driver, user_driver_emulated_mock, page_pool_alloc, nvme and nvme_driver all compile for aarch64-apple-darwin. The actual constraint is disk_nvme, which needs pal::get_cpu_number to select a per-CPU submission queue. That is implemented with sched_getcpu on Linux and GetCurrentProcessorNumber on Windows, and macOS exposes no equivalent. The comment now says so. Finally, drop the inner cfg attribute from scsidvd_nvme.rs. It was load-bearing while that file was also compiled as a test binary of its own, but now that it is only ever a module of the single test binary, the cfg on its `mod` declaration is what gates it.
The suite previously lived entirely in storage_tests, which meant that crate had
to know each backend's platform constraints. It accumulated three
target-specific dev-dependency sections and about a dozen cfg attributes, all
restating what the backends already declare for themselves: disk_vhdmp is
Windows-only, disk_get_vmgs is Linux-only, disk_nvme is neither on macOS,
disk_crypt has no crypto backend there. Duplicating that is how the macOS lint
job got broken earlier in this series, twice.
Each backend now runs the suite from its own tests, so those constraints are
expressed once, by the crate that owns them. All three target-specific sections
in storage_tests are now empty and have been removed. Coverage is still legible,
but as a query rather than a maintained list, which cannot drift from what
actually runs:
cargo nextest list -E 'test(sector_range)'
Two backends genuinely cannot do this and stay in storage_tests. disk_nvme needs
an emulated controller and PCI device to construct, which live in crates it does
not depend on, so that composition only exists at integration level. disk_layered
is subtler: storage_tests depends on it, so dev-depending back would put a second
copy of disk_layered in the graph, and a layer's LayerIo impl -- built against
the non-test instance -- would not satisfy the trait as seen from the test
instance. Dev-dependency cycles are fine in general, but not ones that pass
through the crate under test.
Also fixes a latent break in disklayer_sqlite, whose bundled SQLite build was an
unconditional dev-dependency. Building SQLite from source needs a C toolchain for
the target, which the macOS cross-compile lint job does not have; that job only
lints and never links, so it does not need the library at all.
The fuzz target drives a RAM-backed layered disk with arbitrary (sector, count) pairs, which is the shape of the bug that started this work: a read near u64::MAX overflowed while building the layer's sector bitmap. Because the disk is RAM-backed and cannot fail for any other reason, the expected outcome of every operation is known exactly -- a request is within the disk and must succeed, or it is not and must be rejected with IllegalBlock. That is a stronger oracle than "did not panic", and in particular it also catches an out-of-range request quietly succeeding, which is the more dangerous of the two failures. Reads and writes are forced to cover at least one sector. A zero-length transfer is uninteresting here and would make the oracle wrong, since LayeredDisk builds an empty bitmap and returns success without consulting the range at all; the conformance suite covers that case directly. The loop-device test covers the one path disk_blockdevice's existing tempfile test cannot. For a real block device the kernel enforces the bounds and this backend delegates to it rather than checking the range itself, so it is the delegation that is under test. Attaching a loop device needs CAP_SYS_ADMIN, so the test is #[ignore]d rather than silently skipped: it stays visible in the test list and can be run deliberately. The device is detached by a guard on drop so that a panicking test does not leak it.
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Disk::buffer_sectors divided by the sector size on every read and write, when sector_shift is already cached beside it for exactly this. It also rounded up, which was defending against nothing. The result feeds only the representability guard and never drives an access, and rounding down is sound regardless: max_sector is i64::MAX rounded down to a sector, so for a power-of-two sector size it leaves exactly sector_size - 1 bytes of slack below i64::MAX, which is enough to cover a partial trailing sector.
There was a problem hiding this comment.
Pull request overview
Establishes a backend-owned contract for sector-range validation by having Disk reject requests whose end byte offset is not representable (≤ i64::MAX), documenting the new expectations on DiskIo/LayerIo, and adding reusable conformance tests (plus a fuzzer) to ensure backends reject out-of-range requests with DiskError::IllegalBlock without panicking.
Changes:
- Add
Disk-level “representability” validation (nosector_count()consult) and document backend/layer sector-range responsibilities. - Introduce
storage_tests::sector_rangeconformance suites and wire them into multiple backends’ test modules; add cross-crate integration tests and an NVMe emulation harness. - Add a layered-disk fuzz target and expand backend-specific tests to cover previously-missed edge cases (HTTP blob behavior, striped-disk end-sector wrap, sqlite layer writes past end, etc.).
Reviewed changes
Copilot reviewed 40 out of 42 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| vm/devices/storage/storage_tests/tests/tests/storvsc.rs | New integration test for storvsc/storvsp request/response flow. |
| vm/devices/storage/storage_tests/tests/tests/scsidvd_nvme.rs | Refactor NVMe-backed SCSI DVD test to use shared EmulatedNvme harness. |
| vm/devices/storage/storage_tests/tests/tests/main.rs | New test-binary entrypoint to avoid per-file Cargo auto-discovery duplication. |
| vm/devices/storage/storage_tests/tests/tests/disk_sector_range.rs | Cross-crate sector-range conformance runs for layered disk + NVMe composition. |
| vm/devices/storage/storage_tests/tests/tests/common.rs | Shared NVMe emulation harness for tests needing a real NvmeDisk. |
| vm/devices/storage/storage_tests/tests/tests.rs | Remove old single-file integration test entrypoint. |
| vm/devices/storage/storage_tests/src/sector_range.rs | New reusable sector-range conformance suites for disks and layers. |
| vm/devices/storage/storage_tests/src/lib.rs | New storage_tests library crate exporting shared test helpers. |
| vm/devices/storage/storage_tests/Cargo.toml | Split storage_tests into a real library + integration-test crate with deps. |
| vm/devices/storage/disklayer_sqlite/src/lib.rs | Add missing write range check; add conformance tests for disk + layer paths. |
| vm/devices/storage/disklayer_sqlite/Cargo.toml | Add dev-deps for conformance tests; enable bundled sqlite for tests (non-macOS). |
| vm/devices/storage/disklayer_ram/src/lib.rs | Add conformance tests for disk + 4K representability + direct layer coverage. |
| vm/devices/storage/disklayer_ram/Cargo.toml | Add storage_tests dev-dependency for conformance suites. |
| vm/devices/storage/disk_vhdmp/src/lib.rs | Add disk conformance test to validate backend behavior. |
| vm/devices/storage/disk_vhdmp/Cargo.toml | Add storage_tests dev-dependency for conformance suites. |
| vm/devices/storage/disk_vhd1/src/lib.rs | Add disk conformance test to validate backend behavior. |
| vm/devices/storage/disk_vhd1/Cargo.toml | Add storage_tests dev-dependency for conformance suites. |
| vm/devices/storage/disk_striped/src/lib.rs | Add conformance test + explicit regression test for end-sector wrap. |
| vm/devices/storage/disk_striped/Cargo.toml | Add storage_tests dev-dependency for conformance suites. |
| vm/devices/storage/disk_prwrap/src/lib.rs | Add conformance test for reservation-wrapping disk. |
| vm/devices/storage/disk_prwrap/Cargo.toml | Add dev-deps for conformance test wiring. |
| vm/devices/storage/disk_layered/src/lib.rs | Document layer sector-range/representability contract on LayerIo. |
| vm/devices/storage/disk_layered/fuzz/fuzz_disk_layered.rs | New fuzz target for (sector,count) sector-range handling. |
| vm/devices/storage/disk_layered/fuzz/Cargo.toml | Add new fuzz target crate config and deps. |
| vm/devices/storage/disk_get_vmgs/src/lib.rs | Add disk conformance test for GET VMGS disk. |
| vm/devices/storage/disk_get_vmgs/Cargo.toml | Add storage_tests dev-dependency on Linux for conformance tests. |
| vm/devices/storage/disk_file/src/lib.rs | Add disk conformance tests + regression for sector->byte shift wrap. |
| vm/devices/storage/disk_file/Cargo.toml | Add dev-deps needed by new disk_file tests. |
| vm/devices/storage/disk_delay/src/lib.rs | Add disk conformance test for delay wrapper disk. |
| vm/devices/storage/disk_delay/Cargo.toml | Add dev-deps for conformance test wiring. |
| vm/devices/storage/disk_crypt/src/lib.rs | Add disk conformance test for crypt wrapper disk. |
| vm/devices/storage/disk_crypt/Cargo.toml | Add storage_tests dev-dependency for conformance suites. |
| vm/devices/storage/disk_blockdevice/src/lib.rs | Refactor helper construction; add conformance tests for file + loop-device paths. |
| vm/devices/storage/disk_blockdevice/Cargo.toml | Add storage_tests dependency for conformance suites. |
| vm/devices/storage/disk_blob/src/tests/mod.rs | New blob backend tests covering conformance + fixed-VHD footer + HTTP behaviors. |
| vm/devices/storage/disk_blob/src/tests/http_server.rs | New hand-rolled HTTP server for HttpBlob contract/edge-case testing. |
| vm/devices/storage/disk_blob/src/lib.rs | Add explicit disk-size range check for reads (fixed-VHD footer protection). |
| vm/devices/storage/disk_blob/src/blob/http.rs | Fix zero-length read handling to avoid offset underflow for HTTP ranges. |
| vm/devices/storage/disk_blob/Cargo.toml | Add dev-deps for blob backend tests (HTTP server, fixed VHD1 creation, suites). |
| vm/devices/storage/disk_backend/src/lib.rs | Document new DiskIo sector-range validation contract; add representability enforcement + unmap short-circuit for Ignored. |
| Cargo.toml | Add storage_tests as a workspace dependency; add layered-disk fuzz crate to members. |
| Cargo.lock | Lockfile updates for new dev-deps and new fuzz target crate. |
These returned impl Future because they were bare delegations to the backing object's future, with no body of their own. Adding the representability check turned them into async blocks wrapped in a function, which is what async fn is for -- as clippy pointed out for unmap, which was converted at the time while its two neighbours were not. The explicit signature is not carrying anything here. Send still holds by auto-trait leakage, and is still checked at a definition site: DiskAsLayer::read implements LayerIo::read, which declares impl Future + Send, and awaits Disk::read_vectored in its body. The use<'a> capture list was narrower than what async fn infers, but no caller holds the future beyond the borrow.
| sector_shift: sector_size.trailing_zeros(), | ||
| max_sector: (i64::MAX as u64) >> sector_size.trailing_zeros(), |
There was a problem hiding this comment.
Is it really worth storing these as fields instead of making little const helper fns? Same question for all the calls on disk below, but those could have more involved behavior I suppose.
| self.0.disk.unmap(sector, count, block_level_only) | ||
| ) -> Result<(), DiskError> { | ||
| self.check_representable(sector, count)?; | ||
| if self.0.unmap_behavior == UnmapBehavior::Ignored { |
There was a problem hiding this comment.
| if self.0.unmap_behavior == UnmapBehavior::Ignored { | |
| if self.unmap_behavior() == UnmapBehavior::Ignored { |
| /// ignore unmap: a disk that does no work still has to reject a request that | ||
| /// names sectors it does not have, or the guest gets a success status for an | ||
| /// operation on sectors past the end of the disk. | ||
| async fn expect_rejected(disk: &Disk, mem: &GuestMemory, what: &str, sector: u64, count: u64) { |
There was a problem hiding this comment.
Should more of the functions in here have track_caller?
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| //! Shared harness for tests that need a real `NvmeDisk`. |
There was a problem hiding this comment.
If this file is nvme specific it should have an nvme-specific name
| } | ||
|
|
||
| /// Two layers, so the paths that consult the layer below are exercised. | ||
| #[async_test] |
There was a problem hiding this comment.
Is there anything that would catch us if we added a new disk type but didn't have proper range checks and test coverage?
Storage frontends hand guest-supplied sector numbers to disk backends, and several paths do so with no range check at all — virtio-blk and the GED VMGS handlers among them. A read near
u64::MAXreachesdisk_layered's sector bitmap, wherestart_sector + bits.len()panics with overflow checks enabled and silently wraps without them. Both are guest-reachable.Putting the checks in the frontends doesn't fix it:
sector_count()is dynamic for several backends, so a frontend that checks and then issues the I/O can be beaten by a resize in between. Only the backend can validate atomically against its own state. This PR establishes that split and gives backends the guarantee that makes their existing checks sound. Removing the now-redundant frontend checks is a follow-up; nothing guest-visible changes here.Disknow rejects requests whose end byte offset would exceedi64::MAX, the real limit imposed bypread64/pwrite64and the Windows file APIs. This is deliberately not a range check — it never consultssector_count(), so it can't mask a backend that fails to validate, and can't be invalidated by a resize. But it's enough to repair the checks backends already had, since sector arithmetic can no longer overflow. The contract is documented onDiskIoandLayerIo.Disk::unmapalso short-circuits disks reportingUnmapBehavior::Ignored, after validating the range. That's the one placeDiskrange checks, and it's sound because it's also the one placeDiskdoesn't delegate: the backend never sees the request, so there's no check to be redundant with or to mask.Five defects turned up, three guest-reachable. Besides the bitmap overflow,
HttpBlob::readunderflowed computingoffset + buf.len() - 1on a zero-length read; out-of-range unmaps returned success on any disk reportingIgnored;disk_blobreturned VHD footer bytes for a read past the end of a fixed-VHD1 blob, which is strictly larger than the disk it presents; anddisklayer_sqlitehad no write range check at all.Testing
A conformance suite lives in
storage_testsand each backend runs it from its own crate, so platform constraints stay with the crate that has them. It covers the awkward(sector, length)pairs against read, write and unmap, and assertsDiskError::IllegalBlockspecifically, since that variant becomes guest-visible once the frontends stop checking. Layers also run aLayerIo-level suite, becauseLayeredDiskreaches neitherwrite_no_overwritenor both values ofunmap'snext_is_zero.cargo nextest list -E 'test(sector_range)'is the coverage statement.disk_nvmeanddisk_layeredrun fromstorage_testsinstead: the former needs an emulated controller from crates it doesn't depend on, and the latter can't dev-depend onstorage_testsbecausestorage_testsdepends on it, which would put two copies ofdisk_layeredin the graph.disk_blobis tested over HTTP as well as a file, against a hand-rolled server that injects a short body, an ignoredRange, a 416, and a mid-body disconnect. That's what found theHttpBlobpanic —FileBlobis unaffected by it, so testing one implementation of the trait said nothing about the other.A fuzz target drives a RAM-backed layered disk with arbitrary
(sector, count)pairs. The disk can't fail for any other reason, so the oracle is exact rather than "didn't panic": in range must succeed, out of range must beIllegalBlock.disk_vhdmphas been run on Windows.disk_blockdeviceis covered over a temporary file and over a real loop device; the latter needsCAP_SYS_ADMINso it's#[ignore]d, but it has been run and passes.Discovered via https://github.com/weltling/virtio-villain.
Fixes #4046.