Repository navigation
vm driver: the image's sandbox account is never read for a prepared image (passwd read at the disk root, not under /image-rootfs) #4665
Description
User Story
I run OpenShell's VM driver on a Mac for local agents. My sandbox image carries its own sandbox account at uid/gid 10001, and its creation policy sets process.run_as_user: "10001". The runtimes docs say a MicroVM uses "the image's sandbox account, otherwise 1000", so I expected the workload to run as 10001.
Problem Statement
For any image the driver prepares (a registry image or a rootfs tar, i.e. anything other than the bootstrap image), the workload identity always resolves to 1000:1000. The image's own sandbox account is ignored. With an explicit selector the create then fails (error under Logs).
Cause, as far as I can read it (0.1.5-dev.49+ga4c848adb; the same lines are on main today):
- The owner identity is read from the prepared image disk:
owner_source_disk = image_disk.as_ref().unwrap_or(&root_disk)(crates/openshell-driver-vm/src/driver.rs). sandbox_guest_user_ids_from_imagerunsdebugfs -R "cat /etc/passwd"on that disk (crates/openshell-driver-vm/src/rootfs.rs).- A prepared disk keeps the unpacked image under
PREPARED_IMAGE_ROOTFS_DIR=/image-rootfs. Its root holds onlylost+found,config,upper,workandimage-rootfs. - So the read finds no
/etc/passwd, the parser returnsNone, and the identity falls back toDEFAULT_SANDBOX_UID(1000). The prepared-image cache entry is still named…:image-account:…, which suggests the image's account was meant to be used.
Impact / Why This Matters
Images built for a non-1000 sandbox user cannot use their own account on the VM driver, although the docs say they can. The same image runs under one uid on Docker or Podman and another on MicroVM. The workaround is gateway-wide [openshell.drivers.vm] sandbox_uid/sandbox_gid. It works (the configured identity wins before any image read), but it applies to every sandbox on that gateway, so images with different accounts cannot share one gateway.
Acceptance Criteria
- For a prepared image disk, the driver reads
<PREPARED_IMAGE_ROOTFS_DIR>/etc/passwd(and still reads/etc/passwdfor the bootstrap root disk), so an image withsandbox:10001resolves to10001:10001with no gateway setting. - A creation policy with
run_as_user: "10001"then reaches Ready for that image, andWorkloadIdentityreports 10001:10001. - Images with no
sandboxaccount still default to 1000, as documented. - A test builds a prepared disk with the account only under
/image-rootfsand asserts the identity.
Reproduction Steps
VM driver; no sandbox_uid or sandbox_gid set:
- Create a sandbox from an image whose
/etc/passwdhassandbox:x:10001:10001::/sandbox:/bin/sh, with a policy settingprocess.run_as_user: "10001"andrun_as_group: "10001". The create fails as shown under Logs. - Inspect the cached prepared disk (
<state_dir>/images/sandbox-prepared-rootfs-ext4-umoci-v4-…-image-account-…/rootfs.ext4):debugfs -R "cat /etc/passwd" rootfs.ext4→File not found by ext2_lookupdebugfs -R "cat /image-rootfs/etc/passwd" rootfs.ext4→ containssandbox:x:10001:10001::/sandbox:/bin/sh
- Reading the code, the same image without the selector would resolve to 1000:1000, and guest init would rewrite the account to 1000 and re-own
/sandboxto match. (I have not run that case.)
Suggested UX (if applicable)
No response
Environment
macOS on Apple Silicon; OpenShell CLI, gateway and openshell-driver-vm 0.1.5-dev.49+ga4c848adb (the dev release); sandbox from a rootfs tar (--from ./rootfs.tar); e2fsprogs 1.47.4.
Logs
ProvisioningFailed: prepare guest overlay disk failed: VM run_as_user '10001' conflicts with the resolved workload identity 1000:1000; omit the selector or request the driver-owned identity
Activity
letv1nnn commentedon Oct 26, 2026
Classification: confirmed bug
The User Story gives first-hand use: the VM driver on macOS with a rootfs tar image whose
sandboxaccount is 10001. It also covers impact, the limits of the gateway-wide workaround, and observable acceptance criteria. No duplicate was found. #2331 fixed the equivalent behavior for Docker and Podman only.What the source confirms (
mainateeba0e79)driver.rs:1382picksowner_source_disk = image_disk.as_ref().unwrap_or(&root_disk). For any prepared image, that is the prepared disk.sandbox_owner_identity_from_image(driver.rs:6217) callssandbox_guest_user_ids_from_image(rootfs.rs:810), which always reads/etc/passwd. ANoneresult falls back toDEFAULT_SANDBOX_UID(1000).PREPARED_IMAGE_ROOTFS_DIR(/image-rootfs,driver.rs:211). The preparation check atdriver.rs:3980and the guest init lower-layer selection (openshell-vm-sandbox-init.sh:273) both depend on that layout.docs/how-it-works/sandboxes/runtimes.mdx:314: "MicroVM | The image'ssandboxaccount, otherwise1000."Reproduction (unit level, Linux, e2fsprogs)
I built an ext4 image with the prepared-disk layout (
config/,upper/,work/, andimage-rootfs/etc/passwdcontainingsandbox:x:10001:10001). I then ran the existing readers against it in a temporary test:So any prepared image resolves to 1000:1000, which matches the reported
run_as_user '10001' conflicts with the resolved workload identity 1000:1000. I did not run a full libkrun boot.Notes for whoever implements this
persisted_sandbox_owner_identity(driver.rs:6159) also reads identity from a cached prepared disk through the same function. The fix should cover that caller too, not only the fresh-provision path.passwdtake precedence today. The fix should keep that precedence so an upgrade does not change the uid of an existing sandbox.ext4_image_has_directory_...test inrootfs.rsshows how to build the prepared-layout image that the acceptance criteria call for.Next human decision: whether to accept this fix.
TaylorMutch commentedon Nov 2, 2026
Reassessed the existing triage and PR #5131 at
810b7ea002fd7346dc633f2d0fcbce50228a0857. This is a confirmed bug and warrants acceptance under the maintainer's instruction in this session: the reported first-hand workflow fails to honor the documented image-owned sandbox identity, and the gateway-wide override cannot accommodate images with different accounts.The previous assessment correctly identifies the lookup mismatch, the shared persisted-image caller, and the need to preserve recorded owner/overlay-passwd precedence. I independently reproduced the disk-level result with real ext4 fixtures and the reader functions extracted from the PR: the old
/etc/passwdlookup returnsNonefor the nested fixture; the proposed reader returns10001:10001. Additional checks passed for a top-level account (998:997), no sandbox account (None, retaining the caller's default), malformed nested account rejection, and unreadable ext4 rejection. This was an isolated reader harness, not the full crate test suite or a libkrun boot; Ready/WorkloadIdentity and upgrade behavior still need runtime validation.Two qualifications to the existing assessment: the defect affects disks with the
/image-rootfslayout, rather than every image preparation path (host-prepared images can have a top-level rootfs); and the newer #5073 describes the same root cause with additional first-hand performance evidence. Its reported timing improvement has not been independently measured here.PR #5131 fixes the shared reader, including persisted-image lookups, and leaves owner-state and overlay-passwd precedence unchanged. Existing docs already describe the intended behavior. Applying
state:acceptedand removing the obsoletestate:triage-neededassessment label.