Repository navigation
drm/asahi: g16: Keep compute commands in flight; one 64 MiB PB heap per VM (J773g) - #2419
DjDeveloperr wants to merge 8 commits into
Conversation
Every G16 render gets a fresh parameter buffer epoch at SUBMIT and an InitBM ahead of its TA work, so the firmware rebuilds the allocator state for each render. Reusing the epoch instead cuts a small triangle render at queue depth 1 on a base M4 MacBook Pro (J604) from 1121 to 492 us. With this change, the next render of the same VM on a pass slot gets the slot's owner epoch, without a reseed or InitBM, if that epoch's render retired normally and was exposed. InitBM binds the device-wide slot to one epoch and VM, so this happens only while no other epoch was handed to the slot since; after another VM's render on the slot, after a failed or aborted render, or after any firmware recovery, the next render starts fresh. SUBMIT still allocates a fresh epoch for every render, so one is always at hand. Renders are bound, published and retired one at a time, so the slot's owner is the epoch it is bound to whenever reuse is allowed. The reused epoch is the slot's existing owner lease, so this holds no epoch, VM or admission permit beyond what the lease already holds: a slot binding is tracked by a counter, not by a reference. A host test covers when an epoch may be reused and that nothing else holds it. aurora.g16_pb_reuse=0 restores a fresh epoch and InitBM per render. The stats line counts reused and fresh bindings. Assisted-by: LLM Signed-off-by: Friedrich <261542475+aeries0x@users.noreply.github.com>
G16 published one command at a time and built the next only after the previous one fully retired on the host, so every compute command paid the whole firmware and host round trip: on the J773g Mac mini a 1.5 us dispatch cost ~270 us per submission at any queue depth, and dependent work (one command buffer per dispatch, as Geekbench records it) was bound by it. Queue compute commands behind in-flight compute commands on the one firmware compute queue: - A VM gets more retained compute units, each in its own hardware UMA slot (unit k in slot 1 + k). All units in slot 1, as before, is not safe once commands are queued: the firmware starts the next command while the previous one still runs, and with long commands two owners of one slot then corrupt each other (firmware recovery within seconds on queued ALU work). With one slot per unit, ~115k queued commands ran without a recovery. - The displacement ledger generalizes to N owners: owners retire in publication order, and an owner is reusable once any later publication retired (the firmware finished it before starting that one). Publication stays serialized for the render pass slots. - Every queued command posts its own submit record and doorbell (the firmware does not pick up a WRITE advance without one) behind unconsumed ones, with room checks before anything is visible. - A flight with later compute flights behind it retires on "passed" receipts: its stamp, queue READ cursor and the submit pipe's cursors have moved past it. The newest flight keeps the exact receipt. - A compute command may follow its queue's published (not yet retired) compute predecessors: the firmware compute queue runs commands in publication order, each to its Finalize. Renders and every cross-engine dependency still wait for completion, and nothing is published behind a render or a failed, doomed or recovering flight. - Only the same VM's commands queue behind each other: with six dEQP clients, compute queued behind another VM's lost writes (image and sampler cases failed) and stalled the firmware queue. Nothing queues behind a command with user timestamps either: the host writes them when the command retires, and a query copy queued behind it ran before they existed and never finished. - Recovery matches a snapshot entry against every flight and marks every unfinished one; a failed flight dooms the later flights of its queue (their commands would follow a hole). - Two units per VM as before; a VM that submits compute back to back (three submissions within 2 ms each, or three compute commands in one submission) gets four more, mapped at 0x7090000000, so ordinary clients keep the old memory cost. - Up to 16 jobs per logical queue reach the engine (was 2). - The engine statistics and the compute owner ledger move to the heap: `Firmware` is built on the probe's stack, and its larger frame overflowed the 32 KiB kernel stack. - `aurora-g16/tune` (debugfs, root) sets the depth (compute_flights; 1 = the serialized engine), the units of a growing VM and the completion repoll interval for measurements. J773g Mac mini (G16G B0, 25G83), gpubench, before -> after (macOS 27.2 on the same machine): compute.light d8 268 -> 136 us/submit, d64 269 -> 125 (gap between commands 266 -> 16.5 us; macOS 19.7), compute.heavy alu d4 874 -> 557, d64 871 -> 554 (GPU time 554; macOS 540), compute.heavy bw d16 1667 -> 1275 (macOS 1437). Signed-off-by: Diljit Singh <hey@djdev.me>
Every rendering VM pinned two 256 MiB parameter-buffer heaps, one per render allocator (pass slot), so each rendering process cost ~0.5 GiB of unevictable memory and a few dozen clients (a dEQP-EGL run at six jobs) exhausted a 24 GiB Mac mini. macOS 27.2 on the same machine adds ~24 MiB of wired memory per rendering process. A render is published only while nothing else is in flight and retires with its PB blocks returned to its manager, so the two pass slots' PB managers never hand out heap blocks at the same time. Back both with one heap at the first slot's address. Signed-off-by: Diljit Singh <hey@djdev.me>
A hardware slot's last owner stays bound until a later owner displaces it, because the firmware may still write that owner's descriptor then. The leases held the owner's whole VM state, and with it every arena of the VM: a closed client whose compute unit was the last in some UMA slot kept its 256 MiB PB heap and its compute pools pinned until another VM happened to use that slot. With more compute units, slots used only by clients that queue compute are displaced rarely, and closed clients stayed pinned. Each lease now holds exactly the arena of its unit (compute) or pass slot (render) beside the VM state, and the VM's cache lets go of its arenas when the client closes the VM. What jobs and leases still need, they hold themselves; the rest is freed at close. Signed-off-by: Diljit Singh <hey@djdev.me>
Each render submission allocated a fresh PB epoch: three firmware objects, their GPU and firmware mappings and a shared-L2 synchronisation. With PB epoch reuse most renders never expose that epoch, and its deferred teardown unmaps it again. Render submission cost ~390 us of CPU per vkQueueSubmit on the J773g Mac mini (compute ~25 us). Return a retired job's never-exposed epochs to its VM (up to four, under a spinlock: a job may drop in any context) and hand them to the next render submissions before allocating. A spare keeps the budget permit it was created with. A spare holds its VM, which holds the cache, so the cache drops its spares, and takes no more, when the client closes the VM. Signed-off-by: Diljit Singh <hey@djdev.me>
The parameter-buffer heap was the whole 256 MiB window, fully backed and pinned for every rendering VM. The 25G83 firmware copes with a PB that runs out without the host: with only 64 blocks (8 MiB) posted, heavy and tiler-overflow renders still complete, no growth event arrives, and counters in the PB manager move only for the scenes that overran it. Back 64 MiB of the window by default and name only the backed part in the page and block lists. `aurora-g16/tune` sets the heap of VMs created later (pb_heap_mb, 8..256) and caps the blocks a fresh epoch posts (pb_blocks), for measurements. Signed-off-by: Diljit Singh <hey@djdev.me>
Nothing is published behind a render, so a job submitted while one is in flight waits for the poll pass after its retirement anyway. With more jobs admitted per queue, each such submit still ran a full publication pass (ring drain, health and power checks) under the device lock and delayed the retire-to-publish path: gpubench render.light at depth 8 took 389-396 us per submission. Enqueue only; the next poll publishes it (344 us; stock 350, macOS 47). Signed-off-by: Diljit Singh <hey@djdev.me>
A firmware recovery marks every unfinished flight suspect, and a suspect the recovery passed without a full receipt is aborted. Its units were quarantined for good. Once every unit a VM may use was quarantined, every compute job of every client was cancelled until a reboot, and rendering the same once both pass slots were. One client's fault with compute in flight could do that; with compute kept in flight, one recovery can abort all six units at once. An aborted owner is now abandoned instead: - Its job fails and its VM's sticky error is latched under the device lock before anything else is published, so no job of that VM is published again and nothing loads the aborted storage. The engine retains the job, which keeps that storage mapped for good. - Its unit returns to service for other storage as a fresh owner. It is not a retired owner, so it proves no displacement: an owner published before it still needs a later full retirement. The recovery passed the aborted command, so nothing runs in its UMA slot when the next owner is published, and a render's next epoch is fresh (the recovery invalidated every PB binding). Quarantine stays for storage that cannot be trusted: a corrupt descriptor identity or a failed retirement. Measured on the J773g (M4, 25G83 firmware, kernel 7.3.0-ARCH-m4drv8), 2026-11-19: four forced firmware recoveries in a row, each aborting compute. Every recovery's owner was abandoned (quarantined 0), compute passed a host-checked dispatch test in a new process after each one with no reboot, and render and compute ran in other processes afterwards. Without this change the same trigger left compute cancelled for every client until a reboot. Signed-off-by: Diljit Singh <hey@djdev.me>
|
@DjDeveloperr, tested on a J604 (MacBook Pro 14" M4, 10-core G16G B0, GPU firmware 25G83). Each rendering process now pins 177.6 MiB instead of 625.5 MiB, a closed render+compute client no longer leaves 625 MiB behind, and the output is identical with no GPU faults. Both rows are one boot each of the same tree: alarm-aurora's pkgrel 38 recipe on Setup for both boots:
Method:
This is the problem I reported in #1943 for this Mac: with this PR a terminal window costs 178 MiB instead of 626 MiB. Disclosure: an LLM (Claude) ran these boots and measurements on my Mac and drafted this. I reviewed it and am responsible for it. Assisted-by: LLM |
|
Blockers: 2. I checked the N-owner ledger and the slot code at the head (slots.rs claim/hold/publish/retire/abandon/release, which keep
The PR body ends with the "Generated with Claude Code" line rather than an HW row: J773g and J604 on the final 7-commit module, default command line: a compute client that queues back to back (6 units) under the Vulkan CTS 1/16 sample at 6 jobs, 0 Fail/0 Crash, no GPU recovery. Then 20 forced recoveries in a row with Assisted-by: LLM (Claude:Opus 5.5) |
On the J773g Mac mini (M4, G16G B0, 25G83 firmware), G16 had two structural gaps against macOS 27.2 on the same machine. It published one GPU command at a time and built the next only after the previous one retired on the host, so a 1.5 µs dispatch cost ~270 µs per submission at any depth. And every rendering process pinned ~0.5 GiB: two 256 MiB parameter-buffer heaps plus its arenas.
Stacking: the first commit is #1543 (@aeries0x, PB epoch reuse), included because commits 2-6 build on its epoch handling. Merge #1543 first; the six commits after it are this PR. Hardware results are with #2239 (J773g S10) as well, as on the mini.
Commits
Firmwaremoved its new state to the heap: it is built on the probe's stack, and the larger frame overflowed the 32 KiB stack.aurora-g16/tuneexposes depth, units and the repoll interval.vkQueueSubmit(the epoch allocation, mappings and L2 sync were wasted with epoch reuse). The cache drops spares at close; a spare holds its VM.pb_heap_mb.Results (J773g Mac mini, kernel 7.3.0-ARCH-m4drv8; macOS 27.2 on the same machine)
gpubench, µs per submit; stock = the kernel's own
aurora.ko, run back to back with this PR's module.Memory (unevictable delta per process, settled): 8 × render.light 158 MB (stock 551, macOS 60); 4 × render.heavy 100 / 124 MB (stock 324 / 348, macOS 111 / 95). Idle desktop after boot: 0.47 GB unevictable (before 2.75 GB). Compute-only process: 52 MB with 2 units, 149 MB with the 6 units a pipelining client gets.
Regressions against stock (same kernel)
aurora-g16/tunecompute_units=2 keeps 52 MB but gives back the pipelining.Tests
tools/testing/selftests/drivers/gpu/asahi-m4/run-host-tests.py: 23/23. Harness mocks were updated for the new calls, and host cases were added for the N-owner ledger, the published frontier, in-order "passed" retirement and cache close.Recovery: a firmware-aborted owner's unit returns to service (commit 7,
4b15d49)The issue it fixes. This branch had it before commit 7, and stock has it too:
The fix. An aborted owner is now abandoned:
Hardware, J773g, 2026-11-19. Four forced firmware recoveries in a row (event type 4), each aborting compute:
With the quarantine rule, the same trigger left compute cancelled for every client until a reboot.
Trigger and module. The recoveries were forced with an experimental tune of my follow-up branch (
compute_pool_share=1, which trips the firmware every time). The module was that branch plus this commit, whosepoll.rs/slots.rs/lifetime.rsare byte-identical to this PR's. A write through an unmapped device address does not fault on G16G (it completes), so it can't serve as the trigger.Host tests:
run-host-tests.py23/23.test-recovery.pynow covers six pipelined units with a recovery aborting all of them every third round.Bisect of the six commits (J773g, 2026-11-19, same boot state for all; 3 runs per module plus an ABAB repeat)
Not done
🤖 Generated with Claude Code