Skip to content

vm driver: the image's sandbox account is never read for a prepared image (passwd read at the disk root, not under /image-rootfs) #4665

Open

Description

@cloudlesssmarthome

User Story

I run OpenShell's VM driver on a Mac for local agents. My sandbox image carries its own sandbox account at uid/gid 10001, and its creation policy sets process.run_as_user: "10001". The runtimes docs say a MicroVM uses "the image's sandbox account, otherwise 1000", so I expected the workload to run as 10001.

Problem Statement

For any image the driver prepares (a registry image or a rootfs tar, i.e. anything other than the bootstrap image), the workload identity always resolves to 1000:1000. The image's own sandbox account is ignored. With an explicit selector the create then fails (error under Logs).

Cause, as far as I can read it (0.1.5-dev.49+ga4c848adb; the same lines are on main today):

  • The owner identity is read from the prepared image disk: owner_source_disk = image_disk.as_ref().unwrap_or(&root_disk) (crates/openshell-driver-vm/src/driver.rs).
  • sandbox_guest_user_ids_from_image runs debugfs -R "cat /etc/passwd" on that disk (crates/openshell-driver-vm/src/rootfs.rs).
  • A prepared disk keeps the unpacked image under PREPARED_IMAGE_ROOTFS_DIR = /image-rootfs. Its root holds only lost+found, config, upper, work and image-rootfs.
  • So the read finds no /etc/passwd, the parser returns None, and the identity falls back to DEFAULT_SANDBOX_UID (1000). The prepared-image cache entry is still named …:image-account:…, which suggests the image's account was meant to be used.

Impact / Why This Matters

Images built for a non-1000 sandbox user cannot use their own account on the VM driver, although the docs say they can. The same image runs under one uid on Docker or Podman and another on MicroVM. The workaround is gateway-wide [openshell.drivers.vm] sandbox_uid/sandbox_gid. It works (the configured identity wins before any image read), but it applies to every sandbox on that gateway, so images with different accounts cannot share one gateway.

Acceptance Criteria

  • For a prepared image disk, the driver reads <PREPARED_IMAGE_ROOTFS_DIR>/etc/passwd (and still reads /etc/passwd for the bootstrap root disk), so an image with sandbox:10001 resolves to 10001:10001 with no gateway setting.
  • A creation policy with run_as_user: "10001" then reaches Ready for that image, and WorkloadIdentity reports 10001:10001.
  • Images with no sandbox account still default to 1000, as documented.
  • A test builds a prepared disk with the account only under /image-rootfs and asserts the identity.

Reproduction Steps

VM driver; no sandbox_uid or sandbox_gid set:

  1. Create a sandbox from an image whose /etc/passwd has sandbox:x:10001:10001::/sandbox:/bin/sh, with a policy setting process.run_as_user: "10001" and run_as_group: "10001". The create fails as shown under Logs.
  2. Inspect the cached prepared disk (<state_dir>/images/sandbox-prepared-rootfs-ext4-umoci-v4-…-image-account-…/rootfs.ext4):
    • debugfs -R "cat /etc/passwd" rootfs.ext4 → File not found by ext2_lookup
    • debugfs -R "cat /image-rootfs/etc/passwd" rootfs.ext4 → contains sandbox:x:10001:10001::/sandbox:/bin/sh
  3. Reading the code, the same image without the selector would resolve to 1000:1000, and guest init would rewrite the account to 1000 and re-own /sandbox to match. (I have not run that case.)

Suggested UX (if applicable)

No response

Environment

macOS on Apple Silicon; OpenShell CLI, gateway and openshell-driver-vm 0.1.5-dev.49+ga4c848adb (the dev release); sandbox from a rootfs tar (--from ./rootfs.tar); e2fsprogs 1.47.4.

Logs

ProvisioningFailed: prepare guest overlay disk failed: VM run_as_user '10001' conflicts with the resolved workload identity 1000:1000; omit the selector or request the driver-owned identity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions