Repository navigation
drm/asahi: g16: Wave 2–5 GPU submit, concurrency, DVFS, ZLS/prefixes, and hang-recovery fixes - #2787
Open
DjDeveloperr wants to merge 17 commits into
Open
DjDeveloperr wants to merge 17 commits into
DjDeveloperr wants to merge 17 commits into
Conversation
Contributor
|
Reviewed the 16 commits on top of #2741 (65a87c4..9ce6f75). No gates, knobs or taint wording found. Concerns, by commit:
Testing: all hardware results are from the J773g Mac mini, which has a fan. Rows still needed at 9ce6f75: J713 power/thermal as above, J713 vk-mustpass-short and gles-smoke, and a GPU-fault recovery run under concurrent compute+render on either M4. Assisted-by: LLM (Claude:Opus 5.5) |
…ll overhead Pipelined submissions previously never retired completed flights inside submit_locked(), forcing a workqueue round-trip whenever in-flight slots filled and producing hold[0] stalls and 300-500 us tail bubbles. At the same time, every EP 0x20 interrupt triggered full housekeeping in submit_locked(), every publish_checked() ran Session::check() (including the cross-core UAT handoff lock and full TTB scan), every TA (0x70) completion triggered a 50 us EVENT_RETIRE_POLLS busy-spin while FS was still running, every retire_once() polled all nine log sub-rings and the trace ring, and retired timestamps were re-read from WC memory for both accounting and empty timestamp outputs. Deferring the full Session::check() checks (ASC_RUN, pstate > top, and check_adopted_bindings()) to the 1 ms housekeeping tick is safe because: - publish_checked() still runs Session::transport() on every submit so RTKit crashes and latched transport errors stop publication immediately; - host VM bind/unbind updates ttbs[slot] and owned_user_roots[slot] atomically under uat.inner.lock() before submit/publish, so check_adopted_bindings() detects only unexpected firmware/external TTBAT corruption rather than normal host binding races; - the host never writes pstate > top (policy::limit() bounds every published selector to 1..=top), avoiding per-submit PPL handoff lock contention; - full Session::check() still runs on every >=1 ms housekeeping tick, on the first submit after >=1 ms idle, and whenever recovery_pending() is true. Fix the hot submit and retirement paths: - Eagerly retire completed flights inside submit_locked() when an EP 0x20 event arrived, housekeeping/recovery is due, or the oldest flight's terminal fw_stamp has reached its target, leaving evented cleared when work retires so queued poll work does not spin. - Separate 1 ms housekeeping (Session::check, object health, power, auxiliary log/trace ring drain) from per-submit fast publication, using Session::transport() in publish_checked(). - Track fragment completion events (0x71) alongside stage events so TA-only (0x70) interrupts do not trigger receipt-tail busy-spins or short-repoll storms while FS is still executing, and step the initial receipt-tail backoff from 1 us -> 2 us -> 5 us. - Split channel draining into fast event+stats draining on retirement and auxiliary log/trace draining on 1 ms housekeeping, skip ktime_get() on empty event rings, reuse retired_snapshot() GPU ticks in stat_retire(), and skip WC timestamp reads when timestamp_outputs is empty. Signed-off-by: Diljit Singh <hey@djdev.me>
…ork writes With RENDER_SLOTS = 8, render pipelining capped out at 7 in-flight passes (stalling depth 8 and 16 workloads), while each render pass recomputed Geometry::new() three times and rewrote the untouched zero middle gaps of both ta_work (0x360..0x740, 992 bytes) and fs_work (0x460..0x780, 800 bytes) into write-combining memory on every clean same-VM reuse. Both WORK_STRIDE and RENDER_SLOTS are host client-VA layout choices rather than firmware bounds: firmware receives explicit 64-bit VAs for each pass object rather than indexing by stride. Each pass slot spans +0x0..+0x1c00000 (28 MiB: TILEMAP at +0x1400000 plus 8 MiB TILEMAP_SIZE), which rounds to a 32 MiB power-of-two stride (4 MiB guard) and yields 16 slots inside the 512 MiB window 0x7020000000..0x7040000000 (matching COMPUTE_UNITS = 16). - Halve WORK_STRIDE / PASS_STRIDE from 64 MiB to 32 MiB and expand RENDER_SLOTS from 8 to 16 inside the existing 0x7020000000..0x7040000000 window, raising pipelined render depth from 7 to 15. - Keep full 16 MiB TPC / 8 MiB tilemap capacity on the two base pass slots (0..BASE_RENDER_SLOTS) and size extension slots (2..16) to 2 MiB TPC / 1 MiB tilemap so bursting VMs support 15 concurrent passes up to 4K multi-layer and 8K without inflating per-VM RAM. - Track RenderOwners::last_full (mirroring ComputeOwners::last_full) and write only the active head and tail spans of ta_work and fs_work on clean same-VM reuse, saving 1,792 bytes of WC writes per render pass while full-scrub claims still write the complete images. - Compute render Geometry once per publication and update only layer_metadata after claiming the pass slot. Signed-off-by: Diljit Singh <hey@djdev.me>
Every SUBMIT ioctl currently takes 10-40 separate gpuvm.lock_inner() holds across command root validation, attachment checks, and per-record stream-gate walks, decodes the command stream twice, calls ktime_get() on every submit even after extension arenas are allocated, and unconditionally notifies the standby condvar even with zero waiters. - Batch page-aligned VM range checks in g16/command.rs through a 48-entry stack SpanBatch calling vm.covers_ranges_batch() once per render or compute command, preserving exact per-span EINVAL/ENOENT/EACCES error reporting on failure. - Collapse command::prepare()'s two stream decode passes into one. - Reuse page coverage across 256-byte window cache hits in g16/gate.rs instead of calling cover() on every record in the same page. - Skip ktime_get() burst tracking in g16/resources.rs once extension arenas are already allocated, and skip standby_available.notify_all() when no waiter is blocked. - Add a host unit test in test-stream-gate.py verifying window-cached coverage reuse. Signed-off-by: Diljit Singh <hey@djdev.me>
…s-engine barriers Previously, G16 serialized compute and render work strictly: can_queue_behind() rejected any mixed-class overlap, Order::ready() forced cross-engine dependencies to wait for full host retirement, and poll.rs only retired flights[0], leaving the compute and TA/FS hardware pipelines unable to overlap. Enable concurrent same-VM compute and render flights with per-class retirement, per-class hang deadlines, drain fairness, and hardware ring barriers: - Allow healthy same-VM non-InitBM compute and render commands to queue concurrently up to their respective per-class flight caps, while preserving cross-VM, InitBM, and query_access serialization. - Gate alt-class fallback selection on !needs_engine_drain(&job) so class-cap overlap runs concurrently on primed same-VM queues while drain-requiring head jobs (unprimed same-VM InitBM render, cross-VM switch, failed VM, or doomed/recovery/InitBM flight in progress) are never starved; advance queue fairness turns only after publish_prefix() succeeds so transient EAGAIN does not burn a turn. - Emit hardware Hold ring barriers for same-queue cross-engine dependencies (fs2c0 on the compute ring using the FS fw_stamp, and c2ta0 on the TA ring using the compute fw_stamp via a dedicated PassSlot::compute_barrier FwObject), walking all commands of a flight so multi-command compute batches resolve their exact member stamp and defer with EAGAIN if an unretired predecessor is missing. - Retire the oldest compute flight and oldest render flight independently in poll.rs, track progress and administrative/warning deadlines per class (render deadline_ns vs compute_deadline_ns) so an advancing class never masks a hung flight on the other class, and convert Job::full_retired to a 64-bit command bitmask so mixed-class jobs can retire out of class order while preserving completion proofs. - Read only the published engine's WC qstate WRITE register(s) during publication instead of touching both compute and render rings. - Add host selftests covering concurrent compute+render publication, drain fairness and EAGAIN turn preservation, fs2c0/c2ta0 barriers across multi-command flights, out-of-order per-class retirement, and independent per-class progress/timeout enforcement. Signed-off-by: Diljit Singh <hey@djdev.me>
…rnor On depth-1 submission loops, the engine briefly drains between consecutive jobs, causing Power::report_work(false) to toggle apple_clpc_gpu_set_active() on every job boundary (triggering mod_delayed_work(..., 0) flushes in drivers/soc/apple/clpc.c even though CLPC itself already holds rails for APPLE_CLPC_HOLD_MS = 32 ms on a 4 ms sampling cadence) and forcing the utilization governor to wait a full 20 ms window before stepping up from FLOOR. Add short 2 ms CLPC idle-hold hysteresis and an asymmetric 5 ms fast-ramp upward path to the G16 power governor: - Hold the CLPC active hint for 2 ms (CLPC_IDLE_HOLD_NS, half of clpc.c's APPLE_CLPC_SAMPLE_MS = 4 ms) across brief sub-ms depth-1 inter-job gaps via policy::ClpcHold, releasing it from service_power() once expired and re-arming run_poll() while held idle. This suppresses redundant mod_delayed_work flushes without holding across 60 Hz (16.67 ms) or 120 Hz (8.33 ms) frames. - Release CLPC immediately via Power::release_clpc_now() on fail-stop (fail_flights) and system sleep commit (sleep_down). - Add a 5 ms upward-only ramp window (RAMP_WINDOW_NS = SAMPLE_NS, aligned with the existing PMP sampling cadence) and ramp_demand() matching demand()'s upward thresholds (>= 70% -> top, >= 40% -> +1) so sustained bursts reach top within 5 ms while light 60 Hz desktop frames stay near FLOOR and 20 ms slow decay is preserved. Clamp stored Governor::demand to the active thermal/user limit on the fast-ramp path so cooling relaxation steps cleanly instead of jumping. - Add unit tests in g16/power/policy.rs and wiring checks in test-thermal-policy.py covering fast ramp, 60 Hz non-pumping, decay preservation, clamped cooling ceilings, and CLPC idle-hold expiry/reset. Signed-off-by: Diljit Singh <hey@djdev.me>
Every render pass claim previously ran PassSlot::scrub() synchronously while holding the device mutex (Shared::state), stalling concurrent submissions and retirement behind WC tilemap/heap/preemption zeroing. Move clean same-VM pass slot scrubbing outside Shared::state with a race-free synchronous fallback: - Record clean render retirements in Slots::note_render_retired() and stage remaining idle displaced, cleanly retired render pass slots (never Published or undisplaced Retired slots) right before dropping Shared::state via stage_and_take_scrubs(), only when the engine is neither failed nor frozen. Staging after publication avoids self-blocking immediate slot reuse within the same lock hold and skips staging on pure compute workloads. - Run staged scrubs in run_poll(), submit(), and try_submit_inline() after dropping Shared::state and after signalling retired completion fences via job::complete_all(). - Coordinate slot selection (pass_available, pick_pass, pick_pass_in) and off-lock scrub completion via SeqCst scrubbing/waiter atomics so in-flight scrubbing slots are skipped, pre-cleaned slots are preferred, and blocked waiters trigger an immediate poll_now(). - Skip synchronous WC fills in claim_pass_in()/claim_pass_for() when a same-VM slot was pre-cleaned, while falling back synchronously on first use, cross-VM switch, recovery invalidation, error/cancel, or any failed/dropped off-lock scrub. - Wait for active off-lock scrubs to drain in sleep_quiet() before entering system sleep. - Add host selftests in test-resources.py covering displaced-only staging, post-publish staging without blocking immediate reuse, off-lock scrub + zero-fill claim, waiter wakeup, dropped batch cleanup, and synchronous fallback on error/recovery/cross-VM. Signed-off-by: Diljit Singh <hey@djdev.me>
…imelines off-lock Ready submissions with already-signalled inputs previously still allocated and armed a drm_sched job and woke the scheduler kworker even when inline publication succeeded, and `try_submit_inline()` used `try_lock()` on `Shared::state`, bouncing bursts during brief poll-lock windows onto `drm_sched` wakeups and causing 350-520 us gap95 tail stalls on multi-depth workloads. In addition, `Timeline::complete()` held `Timeline::pending` across `dma_fence_signal()` callbacks. Bypass `drm_sched` allocation and kworker wakeup in `Scheduler::push()` when `prev_started` is true and `Job::try_start_inline()` succeeds, rolling back `accepted` and falling through to `drm_sched` otherwise. Track active firmware recovery in `Shared::recovering` (guarded via RAII in `Firmware::recover()`) so `Shared::try_submit_inline()` can bail out only during sleep or recovery and otherwise acquire `Shared::state` directly. Finally, split `Timeline::complete()` so ready head fences are drained in fixed stack batches under `signaller -> pending` and signalled without holding `Timeline::pending`. Extend `test-job-failure.py` and `test-fence-signalling.py` to cover `Scheduler::push()` inline bypass/fallback rollback, recovery bailout, and off-pending FIFO batch signalling. Signed-off-by: Diljit Singh <hey@djdev.me>
Mirror the G15 logical-queue contract on G16: - Add params::queue_limits() (max_queues = MAX_QUEUES = 58, max_in_flight_per_queue = FLIGHTS = 32) and return it from Gpu::queue_limits(), which enables DRM_ASAHI_FEATURE_INDEPENDENT_QUEUES and DRM_ASAHI_PARAM_GROUP_QUEUE_LIMITS in the shared frontend. - Track live logical queues per open DRM file in usage::Client via an RAII QueueCredit guard (ENOSPC at MAX_QUEUES), stored after Scheduler in Queue so capacity returns only after scheduler drain. Because G16 logical queues own no firmware execution contexts, keeping QueueCredit per-file while file.rs enforces per-process queue_share::Share bounds ensures a client that opens MAX_QUEUES queues cannot starve a later process (such as a compositor) from creating its own queue. - Extend test-publication.py and usage.rs unit tests to cover queue limits, INDEPENDENT_QUEUES, per-file QueueCredit isolation and release, and multi-queue compute+render concurrency on one VM. Signed-off-by: Diljit Singh <hey@djdev.me>
Allow compute-only barrier-free command runs within a job and across consecutive ready compute-only jobs on the same queue and VM to share one firmware publication and one doorbell, bounded by free compute units, available compute ring slots, and the compute flight budget. Each batched compute member claims its own hardware compute unit, writes its own submitted counter before the shared barrier, and records its own completion stamp and work descriptor so retirement verifies every claimed unit receipt, publishes timestamps for every grouped job, and lets cross-engine waits and firmware recovery snapshots resolve interior stamps. Signed-off-by: Diljit Singh <hey@djdev.me>
Avoid redundant uncached write-combining reads and barriers during poll and retirement: - Track per-class check state across retire_once() so retiring a burst of one class does not re-poll an already pending flight of the other class or double-poll a mixed flight. - Collapse oldest_stamp_reached() and progress() into single flight passes that touch only active classes, and skip the empty-ring rmb() in event and statistics draining. - Check FS firmware stamp before TA, and check notifier/descriptor receipts before reading scene or ITS timestamps so unsettled polls fail fast. Trim hot-path SUBMIT ioctl heap allocations without unsafe code: - Add inline 0..2 storage (Commands, Orders, Resolved, InputFences) that spills to KVec only above two entries, eliminating per-submit kmalloc traffic on single- and two-command submissions. - Move attachment lists into the final render or compute command of a stream via core::mem::take instead of duplicating them with copy_list(). Signed-off-by: Diljit Singh <hey@djdev.me>
When pipelining a healthy same-VM non-InitBM render behind an in-flight render (class_behind), prefix_render() encoded both prev_barrier (TA->TA under early tiling or TA->FS otherwise) and fs_dep (FS->FS under early tiling) with BarrierType::Hold (type 1), holding each consecutive TA/FS stage for the predecessor software-job teardown even though own_barrier on the same FS ring already uses BarrierType::Stamp (type 0). Use BarrierType::from_stamp_only(!init_buffer) for both same-class render->render barriers (prev_barrier and fs_dep), matching own_barrier and G15's proven RENDER_INTERNAL_HOLD = 0 policy while keeping cross-engine barriers (fs2c and c2ta) strictly BarrierType::Hold on G16's class-2 queues. G16 uses firmware-initiated phase recovery and never writes G15 kill controls, and any failed/suspect/cross-VM/InitBM transition still drains the engine first. Signed-off-by: Diljit Singh <hey@djdev.me>
Wire the HAL200 compressed ZLS metadata registers (`fs::SEL[44..52]`) and the four `JobParameters3` partial-render mirrors (`0x958`, `0x970`, `0x988`, `0x9a0`) in `encode_fs_for()`, all of which lie inside the clean-reuse active work spans (`0..fs::HEAD_ACTIVE_SIZE` and `fs::TAIL_ACTIVE_START..fs::WORK_SIZE`). Widen `zls_ctrl` admission in `g16/command.rs` to admit the four load and store compression bits (`0x154`, mask `0x00000000060cc1fe`), enforce symmetric load/store compression bits per plane, remove the `EOPNOTSUPP` refusal on `comp_base`/`comp_stride`, and validate compressed metadata alignment, stride encoding, and read-write VM span coverage through the layer count via `SpanBatch`. Add host selftests covering 1x, 2x, and layered compressed depth/stencil admission, invalid control/alignment/span rejection, 25G83/26B123 FS encoding parity, and clean-reuse scrubbing of compressed ZLS fields. Signed-off-by: Diljit Singh <hey@djdev.me>
Accept the 480-byte drm_asahi_cmd_render_m5 render payload when DRM_ASAHI_RENDER_STAGE_PREFIXES is set, decode the 32-byte drm_asahi_render_stage_prefixes tail with compile-time layout assertions, and record per-stage TA compute and render wait-through prefixes on Order via narrow_tiling() (refusing any TA wait later than the FS wait, nonzero reserved words, or combination with DRM_ASAHI_RENDER_FRAGMENT_BARRIERS). Add per-pass ta_fs_barrier and fs_compute_barrier objects to PassSlot, resolve independent TA and FS compute/render barrier stamps in prefix_render() via plan_render_barriers(), and advertise DRM_ASAHI_FEATURE_RENDER_STAGE_PREFIXES_V2 in G16 params so Mesa can emit independent TA and FS waits on G16G. Signed-off-by: Diljit Singh <hey@djdev.me>
Multi-pass render frames (shadow maps, deferred G-buffer + lighting, post-processing, and Zink multi-pass submits) currently pay per-pass VM span coverage locks, per-pass VDM root reads, and per-pass TA/FS ring publications with individual SubmitRecords and UMA submitted counter updates. Batch validation and publication across consecutive render passes within a single SUBMIT without delaying fence completion for separate jobs: - Lift SpanBatch to Prepared::validate() so render passes within a SUBMIT deduplicate shared VM spans and flush coverage in one lock acquisition, and replace gate::vdm() with gate::vdm_batch() to verify all render VDM root words under a single Vm::read_lock() with a 256-byte intra-page read window. - Add ordered multi-pass slot tracking (Claims::pass_count), replace PoolSet::publish() with PoolSet::publish_batch() (submitted += 2 * renders), and add Job::can_fold_render_cmd() so consecutive ready non-InitBM render passes in one SUBMIT fold into a single Flight, one TA/FS producer advance, one SubmitRecord kick, and one UMA submitted update while preserving exact per-pass barriers (wait_stamp_for_with), recovery matching (r_ta_stamp/r_fs_stamp), per-pass scene ITS verification, and ordered pass-slot retirement. - Add host selftests covering batched VDM root admission, submit-wide span deduplication, ordered wrapped multi-pass retirement and publish_batch(), same-batch render wait stamps, can_fold_render_cmd, and interior render-stamp recovery preparation. Signed-off-by: Diljit Singh <hey@djdev.me>
Cache::prepare() previously gated RenderExtArena allocation on compute's 2 ms BURST_GAP_NS and 3-command BURST_COMMANDS thresholds. Heavy multi-millisecond pipelined renders arrive more than 2 ms apart and would reset render_burst to 0 on every submit, staying capped at BASE_RENDER_SLOTS (2 slots), whereas simply widening the idle gap to cover 60/120 Hz frame periods would force every steady depth-1 compositor or window client to allocate 14 extension pass slots it never uses. Track live prepared render jobs per VM in Cache::active_renders (incremented on returning Prepared and decremented in Prepared::drop), keep the 2 ms BURST_GAP_NS for back-to-back single-pass bursts, and expand RenderExtArena when a render overlaps an active in-flight render (active_renders != 0), submits at least BASE_RENDER_SLOTS passes in one SUBMIT, or sustains a sub-2 ms burst. Add a host unit test covering steady 60/120 fps depth-1 non-expansion, multi-ms overlap, sub-2 ms bursts, and multi-pass submits. Signed-off-by: Diljit Singh <hey@djdev.me>
… slot is quarantined Fresh user VMs start with only BASE_COMPUTE_UNITS (2) compute units and BASE_RENDER_SLOTS (2) render pass slots, allocating extension slots lazily on bursts, overlap, or multi-pass submits. After a hang recovery quarantines one base compute unit (0 or 1) while the other base unit is the newest-retired undisplaced owner, compute_exhausted_within(2) becomes true even though 14+ extension compute units remain healthy. Every subsequent fresh single-command compute client then submits with only 2 units, hits compute_exhausted_within(2) in engine publication, and fails immediately with ECANCELED without ever running on an extension unit to displace the surviving base unit. The same deadlock occurs for fresh single-pass render clients when a base pass slot is quarantined. Track base-slot quarantine state in Lifetime::any_quarantined_within(), publish lockless hints on Shared whenever Slots::quarantine() runs, and pass them via PrepareHints into Cache::prepare() so fresh VMs expand compute_ext / render_ext immediately once a base slot is quarantined. Add host unit tests covering both compute and render base-slot quarantine recovery and displacement. Signed-off-by: Diljit Singh <hey@djdev.me>
…ender hangs Physical render PB/UMA allocators (hardware UMA slot 33) were previously capped at two fixed slots (parameter_id 0 and 1). Once both slots had been used across VMs (for example compositor plus benchmark), a single render hang on the active slot called pools.invalidate() and pools.quarantine_flight(), leaving the other slot undisplaced and immediately tripping Pools::exhausted() -> render_failed=1, degraded=1, and device wedged even with 14+ healthy render pass slots remaining. Expand the client-VA render allocator window to PB_SLOTS = 6 (parameter_id 0..6 in a 64 MiB stride across 0x7008000000..0x7020000000, disjoint from base compute units, render pass slots, dynamic PB page lists, and extension compute units). Keep non-fault VMs at BASE_PB_SLOTS = 2 with zero extra RAM overhead; lazily allocate per-slot PoolExtArena recovery allocators sharing the VM's existing PB heap only after a render hang quarantines an allocator slot, signaled locklessly via Shared::needed_pb_slots. Generalize Pools selection, displacement, and exhaustion checks to PB_SLOTS while preserving the two-slot working set in normal cross-VM switching and per-job limit checks (pick_slot_within / can_pipeline_within / exhausted_for_within) so pre-hang jobs cancel cleanly instead of spinning on EAGAIN. Add host unit tests covering the exact cross-VM single-render-hang recovery and survivor displacement path, multi-hang survival up to four quarantines, and clean exhaustion at the slot limit. Signed-off-by: Diljit Singh <hey@djdev.me>
DjDeveloperr
force-pushed
the
djdev/g16-perf-wave2-5
branch
from
November 22, 2026 15:46
9ce6f75 to
5610a84
Compare
Contributor
Author
|
@RyanTheTide Thanks for the thorough review — pushed R2 (
Assisted-by: LLM (Gemini:4 Neon) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Seventeen-commit R2 (
5610a84b974e5) G16 (M4 / T8132,26B123firmware) GPU performance, concurrency, and hang-recovery series stacked cleanly on#2741(djdev/g16-perf-272-prat65a87c442d99), introducing zero module knobs or debugfs switches and addressing all 10 review items from@RyanTheTide(#issuecomment-7078017919):bbd2af5c36515Inline retirement & poll trim: retire completed flights inline on submit and trim poll-loop overhead (documents why 1 msSession::check()deferral is safe whilepublish_checked()keeps per-submitSession::transport()).042b94b1ea2fb16 render pass slots & clean-reuse work trims: expand render pass slots to 16 inside0x7020000000..0x7040000000(32 MiB stride = 28 MiB pass layout + 4 MiB guard) and trim clean-reuse WC writes.752c66d26f295Batched VM coverage checks: batch VM range checks and trim per-SUBMITCPU overhead.612dabbe9f4feConcurrent compute + render flights: allow concurrent same-VM compute and render flights with cross-engineHoldbarriers (walking all commands of multi-command flights), independent per-class progress/hang deadlines (deadline_nsvscompute_deadline_ns), and squashed alt-class drain-fairness guard (!needs_engine_drain(&job)) +EAGAINturn preservation so every commit bisects cleanly.1cad4da5fb6e7CLPC idle-hold hysteresis & fast-ramp DVFS governor:2 msCLPC_IDLE_HOLD_NS(half ofclpc.c's4 mssample cadence),5 msRAMP_WINDOW_NS(== SAMPLE_NS),>= 70% / >= 40%thresholds matchingdemand(), and clampedGovernor::demandunder thermal limits.7b54ab53b397bOff-lock slot pre-scrubbing: pre-scrub displaced render pass slots off-lock.6d376201cd880Ready inlinedrm_schedbypass & off-lock timeline signaling: bypassdrm_schedon ready inline submits and signal timeline fences off-lock.058731de1f0afIndependent queue limits: expose per-class queue limits with per-fileQueueCreditisolation inusage::Clientso one client cannot starve other processes.70756169505cdBarrier-free compute horizon batching: batch barrier-free compute horizons into one doorbell while tracking per-member completion stamps for cross-engine waits and recovery snapshots.70c082c6d03a7Inactive WC poll skip & submit allocation trims: skip inactive WC poll reads (hold[0]poll storm-93.2%) and trim submit allocations.379a034591e92Stamp-only barriers for pipelined same-VM renders: use stamp-only barriers for pipelined same-VM renders while keeping cross-engine barriersHold.469144bdac255Compressed ZLS depth/stencil support: enable compressed ZLS depth and stencil attachments.33a7c5223bd63DRM_ASAHI_RENDER_STAGE_PREFIXES_V2: support v2 render stage prefixes (feature bit 33).f5a4612e16f69Batched multi-pass render validation & publication: validate and publish multi-pass render batches together.c4e5905be6fc0Lazy render extension slots on overlap or burst: replace25 msburst gap withCache::active_rendersoverlap tracking +2 mssub-ms burst / multi-pass expansion so steady 60/120 Hz depth-1 clients stay on 2 base slots.ca433600f9e7bQuarantine-triggered extension expansion (live-caught fix): expand extension compute/render slots when a base slot is quarantined so fresh single-command clients do not deadlock onECANCELED.5610a84b974e5Lazy recovery PB/UMA allocator slots (PB_SLOTS = 6, live-caught fix): keep non-fault VMs atBASE_PB_SLOTS = 2(zero extra RAM) and lazily allocate per-slot recoveryPoolExtArenaallocators only after a render hang quarantines an allocator slot, preventing prematurerender_failed=1 / degraded=1 / device wedgedafter a single render hang.Verification
Host checks (
5610a84b974e5)scripts/checkpatch.pl --strict --ignore FILE_PATH_CHANGES:0 errors, 0 warnings, 0 checksacross all 17 commits.run-host-tests.py:25 / 25PASS (including new tests for per-class hang deadlines, multi-command cross-engine barriers, 60 Hz DVFS non-pumping & clamped cooling ceiling, per-fileQueueCredit, 60/120 Hz render non-expansion, base-slot quarantine expansion, and multi-hang PB/UMA recovery).0newrustcwarnings across the full kernel build.Live hardware (
Apple M4 (G16G B0),J773g,0x8132,26B123firmware)dcp77-v15-recovery-proof,7.3.0-ARCH-m4prod15):render.heavy d=4):recoveries 0 -> 1(526.9 ms), concurrent renderrc=0(33,622submits), immediate freshcompute.heavy d=1/compute.light d=1/render.heavy d=1allrc=0with0ECANCELED.compute.heavy d=4):recoveries 1 -> 2(113.6 ms), concurrent computerc=0(16,179submits),render_failed=0,degraded=0, no[drm] device wedged, immediate freshrender.heavy d=1+render.postfx(CRC=EXACT 668d6f0a) +compute.heavy d=1allrc=0.recoveries 2 -> 3(128.9 ms), concurrent computerc=0(16,170submits),render_failed=0,degraded=0,pb: bad 0, post-3-recoveryvk-mustpass-short87/87PASS,0kernelBUG/WARN/Oops.5610a84b974e5differs from thedcp77-tested commitaf25205059308solely by adding#[allow(dead_code)]onPbOwnerArenainresources.rs:293-300, zero codegen change.)dcp71/dcp72/dcp73a/dcp77):gpubenchWave-157/57 OK(0.8027×M4 Metal),concurrent.sram10/10 PASS(1.00–1.04×Metal,EXACTCRC),hold[0]poll-storm-93.2%.vk-mustpass-short87/87 PASS,dEQP-VK.draw.*indirect*3,710/0 PASS,gles-smoke3,086/0 PASS, real Deep Sleep S2R (apple-sleep: back from sleep, boot object resumes 0 -> 1) + post-wakevk-mustpass-short87/87 PASS.Stacked on
#2741(djdev/g16-perf-272-pr); will retarget ontointegration/auroraonce#2741lands.Assisted-by: LLM (Gemini:4 Neon)