Skip to content

drm/asahi: keep GEM_USERPTR pins within the user's RLIMIT_MEMLOCK across fork and exit - #2777

Open
funkzy wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
funkzy:pr/asahi-userptr-user-limit
Open

funkzy wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
funkzy:pr/asahi-userptr-user-limit

Conversation

@funkzy

@funkzy funkzy commented Nov 22, 2026

Copy link
Copy Markdown
Contributor

Problem

GEM_USERPTR counts the pages it pins in the calling mm's pinned_vm and compares that count with RLIMIT_MEMLOCK.
The object belongs to the DRM file, not to the mm. A forked child starts with nothing counted, its pins land in the
file it shares with its parent, and they stay pinned after it exits, counted against an mm that no longer runs
anything. So the limit doesn't bound what one user can pin.

Why this shape

The limit was checked in one place, UserPages::pin(), against a count that belongs to an mm, while the pinned
object belongs to the DRM file and can outlive the process that made it. The per-mm count (mm->pinned_vm, as RDMA
umem and amdxdna use it) has that property wherever the pinned object outlives the process; counting per DRM file
instead would still give each file of the user its own full limit. The change uses the count the kernel already keeps
for pins owned by a file rather than a process: user_struct::locked_vm, which io_uring (__io_account_mem()) and
iommufd charge the same way. Every process of the user then draws on one limit, a pin is charged until its object is
freed, and a CAP_IPC_LOCK caller is not charged at all, as in both. The mm's pinned_vm still counts the pages for
/proc. Only the G17 backend creates user-memory objects, so this is the only place they are charged.

Change

  • asahi_user_account() and asahi_user_unaccount() in user_pages.c, beside the driver's other pin helpers: the
    same arithmetic as __io_account_mem(), with iommufd's underflow warning on release. UserPages holds the charged
    user with a reference until it is dropped.
  • <linux/sched/user.h>: the driver joins the users of user_struct::locked_vm, as iommufd did.
  • The UAPI comment for DRM_IOCTL_ASAHI_GEM_USERPTR says what is counted.
  • A process with a smaller page size (the fragment pins of CONFIG_MM_SUBPAGE, drm/asahi: GEM_USERPTR in processes with a smaller page size #2226) is charged the same native
    pages.
  • For applications: Borealis's VK_EXT_external_memory_host imports are the userspace that creates these objects.
    They now share one RLIMIT_MEMLOCK across all of the user's processes, where each process had its own before. Under
    systemd's default (8 MiB for a user session) the sum of a user's host imports is what is limited; an import past
    it fails with VK_ERROR_OUT_OF_DEVICE_MEMORY, as one that took a single process past its own limit did before.
    The Gallium driver doesn't create these objects, and under virtio (agx_has_userptr()) they aren't used.

Testing

  • KUnit, new case pins_count_against_the_user in asahi_gem_user_memory. A caller with CAP_IPC_LOCK is not
    charged, while its mm still counts the pages. With CAP_IPC_LOCK dropped and RLIMIT_MEMLOCK set just above what the
    test's user holds, pins within the limit are charged and given back. Once pages are charged to the user as another
    of its processes would charge them, the next pin is refused, although the test's own mm has none. On
    integration/aurora cacf355 plus this change only, 361 of 363 cases pass and 2 skip, as on integration alone,
    where the removed memlock_limit_policy case runs instead.
  • J775D (Mac Studio, M5 Ultra), on integration/aurora cacf355 plus what the J775D's GPU needs to run at all:
    drm/asahi: t6050: identify the K114/K116 27.2 (26B8125) G17S firmware (on #1173) #1650 (the 27.2 firmware's M5 InitData), arm64: dts: apple: t6050-j775d: Give the Mac Studio (M5 Ultra) a working GPU, and its M and P core capacities (754/1024) to the scheduler #1973 (the J775D GPU node), and two commits of ours not yet posted,
    "drm/asahi: t6050: Run the Mac Studio's (M5 Ultra) GPU under the 27.2 boot stub: admit its 26B123 G17D firmware"
    and "drm/asahi: t6050: Give the M5 Ultra's 27.2 firmware the power-controller status config of its own 26B123
    layout". One boot with this change and one without, through the 27.2 firmware, default kernel command line,
    read-only root.
    • With this change, pins of an unprivileged user beyond its RLIMIT_MEMLOCK are refused with ENOMEM across that
      user's processes, children that have exited included, and closing the file gives the charge back.
    • Userspace at the default limit (8 MiB, systemd's DefaultLimitMEMLOCK, which every process of the desktop
      session has), with a Borealis build that advertises VK_EXT_external_memory_host:
      dEQP-VK.memory.external_memory_host.* passes 4 and reports 8 not supported (image formats) of 12, 0 failed,
      with and without this change. Among the installed applications, FFmpeg's Vulkan upload doesn't import host
      memory (no pinned pages while it runs), and the desktop session holds no pinned pages on either kernel. No
      installed workload was found that imports host memory, so none is affected.
    • Both kernels bring up the same co-processors and the desktop on the GPU, with 2 failed units on both (the same
      two) and no new kernel error at err level and above. ALARM has no SELinux, so there is no AVC count.
  • Builds: on that base, with integration's own config (defconfig plus asahi.config), and as the J775D kernel; no
    warning in the files this change touches.
  • run-host-tests.py in the build VM: 451 of 451 pass with this change. The base's one failure in the same run is a
    timeout of an unrelated test's mutation run.
  • checkpatch --strict: no errors.

Still to run

  • The fragment-pin path of a process with a smaller page size on hardware: the J775D kit has no CONFIG_MM_SUBPAGE.
    The user4k/gem-userptr selftest on a subpage kit covers it, and its VmPin checks still hold.
  • Rows from other G17 Macs are welcome.

Assisted-by: LLM (Claude:Opus 5.5)

…oss fork and exit

GEM_USERPTR counts the pages it pins in the calling mm's pinned_vm and
compares that count with RLIMIT_MEMLOCK. The object belongs to the DRM
file, not to the mm. A forked child starts with nothing counted, its
pins land in the file it shares with its parent, and they stay pinned
after it exits, counted against an mm that no longer runs anything. The
limit therefore does not bound what one user can pin.

Charge the pins to the user instead, in user_struct::locked_vm, as
io_uring does for its registered buffers (__io_account_mem()) and
iommufd for its pinned pages: every process of the user shares the
count, and a pin stays charged until its object is freed. As in both,
a CAP_IPC_LOCK caller is not charged. The mm's pinned_vm still counts
the pages, as it did. The UAPI comment says so, and the driver joins
the users of user_struct::locked_vm in <linux/sched/user.h>, as
iommufd did.

Only the G17 backend takes user-memory objects
(File::supports_user_memory()), so other GPUs do not run this code. A
process with a smaller page size is charged the same native pages. A
KUnit case checks that a CAP_IPC_LOCK caller is not charged; that,
without it, pages charged to the test's user as another process of that
user would charge them make the next pin fail although the test's own
mm has none; and that dropped pins are given back.

Fixes: ecc94a2 ("drm/asahi: GEM_USERPTR, buffer objects over the caller's memory")
Reported-by: Erik Bourget <erik@ebourget.net>
Suggested-by: Erik Bourget <erik@ebourget.net>
Signed-off-by: Jeff Queisser <funkzy@users.noreply.github.com>
@RyanTheTide

Copy link
Copy Markdown
Contributor

The accounting looks correct to me.

  • drivers/gpu/drm/asahi/user_pages.c:39-65 matches __io_account_mem(): CAP_IPC_LOCK is checked in init_user_ns (so root inside a user namespace is still charged), the cmpxchg loop is overflow-checked, and the release drops the reference it took. The charge is tied to the user_struct reference, not to the mm or the task. A child that exits keeps its pins charged until the object is freed, and a setuid() after pinning still uncharges the original user.
  • gem.rs:505-530 and :605-608: the charge is taken before UserPages exists, and every failure after that (the reserve(), the GUP or fragment pin) goes through Drop, which gives back exactly counted pages to the stored user, once. I don't see a leak or a double uncharge.
  • The pool is now shared with io_uring fixed buffers, iommufd and SO_ZEROCOPY, so one of a user's processes using those leaves less room for Borealis host imports, and the reverse. That's the usual kernel model, and the UAPI comment says so. Worth knowing for anyone who hits ENOMEM from VK_EXT_external_memory_host under the default 8 MiB.
  • Minor: the KUnit helpers at user_pages.c:67-100 (override_creds(), writing rlim_cur) are built whenever CONFIG_KUNIT is set, not only when this suite is. The test also asserts exact locked_vm values for the root user. If the suite runs as a module on a live system, root's io_uring or perf mmaps could make it flaky. Not a blocker.

The only hardware row is the J775D. Rows from the other G17 Macs (A18 Pro, M4 Pro) would be good: dEQP-VK.memory.external_memory_host.* at the default limit, plus a normal desktop session, at this head.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-PR checklist review on 464023d790812a1522cdf7b3d8eb8aa1c790c309 (CI: host-tests green; note SEP source checks / rust-build / kunit workflows did not trigger for drivers/gpu/drm/asahi/ paths).

Blockers (REQUEST_CHANGES)

  1. Build failure with CONFIG_KUNIT=m (-Dwarnings / dead_code):

    • #[kernel::macros::kunit_tests(asahi_gem_user_memory)] expands to #[cfg(CONFIG_KUNIT = "y")] (rust/macros/kunit.rs:44), so mod user_memory_tests is compiled out when CONFIG_KUNIT=m (standard on distro kernels).
    • However, drivers/gpu/drm/asahi/gem.rs:1075 gates the four private extern "C" KUnit helpers with #[cfg(CONFIG_KUNIT)], which Kbuild defines for both =y and =m (scripts/kconfig/confdata.c:799). Under CONFIG_KUNIT=m, all four extern "C" declarations have no callers and fail the build with error: function ... is never used under -Dwarnings (dead_code).
    • Every other KUnit helper in drivers/gpu/drm/asahi/ (g17/kick.rs, pairs.rs, preparation.rs, queue/admission.rs, runtime.rs) uses #[cfg(CONFIG_KUNIT = "y")].
    • Matching that on the C side, drivers/gpu/drm/asahi/user_pages.c:67 and user_pages.h:21 should use #ifdef CONFIG_KUNIT (i.e. IS_BUILTIN(CONFIG_KUNIT)) instead of #if IS_ENABLED(CONFIG_KUNIT), so the test-only asahi_test_drop_ipc_lock() / asahi_test_user_locked_vm_add() / asahi_test_set_memlock() helpers are not compiled into production aurora.ko when CONFIG_KUNIT=m.
  2. KUnit test cleanup underflows global root_user->locked_vm on early error return (drivers/gpu/drm/asahi/gem.rs:887-914):

    • Inside the cleanup closure in pins_count_against_the_user(), UserPages::pin(addr, 2 * PAGE_SIZE, true)? (line 889) runs before asahi_test_user_locked_vm_add(held) (line 897).
    • If that ? returns Err (before held is added), line 911 still unconditionally executes asahi_test_user_locked_vm_add(-held), subtracting 1024 from the global root_user->locked_vm that was never added and leaving it negative for the rest of the kernel run.
    • Track whether held was added (e.g., let mut held_added = false;) and only subtract -held if it was actually added.

Code Review of Core Accounting Path (Looks Good)

  • asahi_user_account() / asahi_user_unaccount() in user_pages.c: Matches io_uring (ns_capable_noaudit(&init_user_ns, CAP_IPC_LOCK) + atomic cmpxchg loop on user->locked_vm, plus check_add_overflow which is strictly safer) and iommufd's underflow WARN_ON while still calling free_uid(user).
  • UserPages::pin() / Drop: user and mm accounting unwind cleanly if pages.reserve(), asahi_pin_user_fragments(), or pin_user_pages_fast() fail after this is constructed.
  • include/linux/sched/user.h: Adding IS_ENABLED(CONFIG_DRM_ASAHI) to user_struct::locked_vm matches IS_ENABLED(CONFIG_IOMMUFD).
  • asahi_test_drop_ipc_lock() / asahi_test_restore_creds(): prepare_creds() + override_creds() + put_cred(revert_creds()) refcounting is correct for this kernel's rcu_replace_pointer-based override_creds/revert_creds.

Pre-PR Checklist (§1–§6)

  • §1 No model gating: Pass — applies to all GPUs supporting user-memory objects (File::supports_user_memory()).
  • §2 Production-ready: Pass — default command line, no hacks.
  • §3 Safety: Pass — uses standard RLIMIT_MEMLOCK + user_struct::locked_vm.
  • §4 Clean-room & attribution: Pass — clean human commit identity and Signed-off-by; Assisted-by: in PR body.
  • §5 Evidence: Strong J775D A/B row (including Borealis dEQP-VK.memory.external_memory_host.* and cross-process fork/exit ENOMEM verification), though as noted in ## Still to run the CONFIG_MM_SUBPAGE fragment-pin path (user4k/gem-userptr) and other G17 Macs remain to be run; also please re-verify a build with CONFIG_KUNIT=m after fixing the cfg gate above.
  • §6 Naming: Pass (correctly uses Borealis).

Assisted-by: LLM (Gemini 4 Neon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants