Skip to content

drm/asahi: g16: Wave 7 — barrier elision, timestamp-free render microsequences, and 48 render pass slots - #2852

Open
DjDeveloperr wants to merge 3 commits into
djdev/g16-perf-wave6from
djdev/g16-perf-wave7
Open

DjDeveloperr wants to merge 3 commits into
djdev/g16-perf-wave6from
djdev/g16-perf-wave7

Conversation

@DjDeveloperr

Copy link
Copy Markdown
Contributor

Summary

Three-commit G16 (M4 / T8132, 26B123 firmware) Wave 7 series stacked directly on #2826 (djdev/g16-perf-wave6 at f63168347300c):

  1. 720cbd169a5cf drm/asahi: g16: elide redundant same-ring TA and independent FS barriers
    • Resolves exact in-flight render wait stamps in prefix_render() / plan_render_barriers(), removes unused PassSlot::prev_barrier (same-ring TA→TA barriers), emits ta_fs_barrier / fs_dep only when needed and not already dominated, and pre-checks wait_stamp_for_with() in the batch fold loop. Independent multi-pass renders drop from 3 barrier records (5 ring entries) to 1 (own_barrier, 3 ring entries).
  2. 124f9c4145d55 drm/asahi: g16: omit render microsequence timestamps when unused
    • Adds compact timestamp-free TA/FS microsequence variants (SEQ_SIZE_NOTS, FINALIZE_NOTS, END_NOTS) keeping WaitForIdle (1) and rebased FINALIZE/has_attachments offsets, tracks emission in Flight::stage_ts_mask, writes only active microsequence length to WC memory, and skips per-pass retirement timestamp WC reads when neither userspace timestamps nor Debug1 profiling are active.
  3. 71daf4f3f7ea0 drm/asahi: g16: raise render pass slots to 48 and tighten slot-wait repoll
    • Raises hw::sizes::RENDER_SLOTS from 32 to 48 within the existing 0x7020000000..0x7040000000 client VA window (80 MiB spare), decouples params::render_flights() from hw::sizes::FLIGHTS (up to 47 passes in flight across up to 32 flights), widens pass bitmasks to 64-bit, extends demand-grown Tier 2 to slots 18..48 (shallow/moderate VMs stay on Tier 0 / Tier 1 with zero Tier-2 backing), and arms engine.short_repoll when pending work blocks on slot headroom.

Verification (exact head 71daf4f3f7ea0836ffbf9f3c17791f8069362755)

Host gates

  • run-host-tests.py: 25 / 25 PASS (including check-abi-26b123.rs, test-stamp-wrap.py, test-publication.py, test-poll-retire.py, test-resources.py).
  • scripts/checkpatch.pl --strict --ignore FILE_PATH_CHANGES: 0 errors, 0 warnings, 0 checks.

Live hardware (Apple M4 (G16G B0), J773g / T8132, 7.3.0-ARCH-m4prod18, /opt/mesa-v10-tc)

Vulkan CTS (1,104 Pass / 233 NotSupported / 0 Fail / 0 Crash / 0 Timeout / 0 GPU errors)

Subset Cases Pass NotSupported Fail
vk-mustpass-short 87 87 0 0
*timestamp* (strided of 1,094) 350 340 10 0
dEQP-VK.query_pool.statistics_query.* (strided of 18,775) 250 250 0 0
dEQP-VK.synchronization.op.single_queue.* (strided of 7,324) 300 291 9 0
dEQP-VK.renderpasses.renderpass2.* (strided of 32,680) 350 136 214 0

gpubench-vk (v18 Wave 7 R2 vs v17 Wave 6 R2 f63168347300c on identical /opt/mesa-v10-tc; 0 CRC/status fails across 44 rows)

  • render.light(kind=tri): d=1 273.82 -> 262.61 us (+4.27%), d=2 209.73 -> 196.08 us (+6.96%), d=4 173.13 -> 157.83 us (+9.70%), d=8 138.27 -> 127.10 us (+8.79%), d=16 132.56 -> 121.48 us (+9.12%).
  • render.postfx (6 rows): +7.91% geomean (1080p msaa=4 d=2 1388.89 -> 1269.04 us +9.44%, 1440p msaa=1/4 d=2 1428.57 -> 1322.75 us +8.00%).
  • compress.bw(kind=array,res=1440p) (2 rows): +8.11% geomean (d=1 +6.67%, d=4 886.52 -> 809.06 us +9.57%).
  • 24-row 4K@120Hz: +1.68% geomean vs v17 (0.8773× M4 Metal).
  • overlap.passes (4 rows): 1.1853× M4 Metal (+0.54% vs v17; render_render d=2 1.38 -> 1.44 +4.24%).
  • render.deferred (8 rows): +1.20% geomean vs v17 (0.7149× M4 Metal).
  • Live debugfs: render_owners: fresh 3 retired 45 quarantined 0 reusable 47 (48 slots), slot_sync 0, ct_sync 0, pass_sync 0, pb: bad 0, type0: fs2c 759 c2ta 7454.

Real Deep Sleep S2R

  • apple-sleep: back from sleep, boot object resumes 0 -> 1, pcie-apple port 0 & 2 link restored, GPU firmware back in 5083 us, ZERO_S2R_ERRORS, post-wake vk-mustpass-short 87 / 87 PASS.

Assisted-by: LLM (Gemini:4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Review at 71daf4f.

Review at 71daf4f (stacked on #2826 at f631683).

  1. g16/poll.rs:1065 + 1091: the TA/FS start/end check (ta_start != 0 && ta_end > ta_start ...) was the only per-pass proof that a render actually ran. recovery_prepare_suspects() (engine.rs:840) keeps unfinished flights of other VMs as suspects "until either its full receipt or its passed-abort proof". If the firmware aborts such a render during recovery, its stamps and queue pointers still pass. Before, the missing timestamps sent it to Retirement::Aborted after 20 ms. Now passes with no stage_ts_mask bit skip that check, so an aborted render retires as Full, i.e. reported as success. Please keep some execution evidence in the compact sequence, or fail suspect flights whose passes have no timestamp evidence, and rerun the render-hang recovery row (Dir A/B) at this head.
  2. g16/poll.rs:1564 / 1072 (stat_retire): the same timestamps feed queue.usage, which is fdinfo drm-engine-vertex/drm-engine-fragment (file.rs:470). With emit_ts false for every render that requests no timestamps (engine.rs:2322), normal apps report 0 ns render time in gputop/nvtop. Please keep fdinfo accounting working, for example from the FS end event time, or keep timestamps at least on the last pass of a flight.
  3. g16/poll.rs:599: short_repoll is now set whenever engine.pending is non-empty and flights are active, not only when pending work waits on slot headroom as the commit message says. Pending work that is blocked for a different reason, such as a cross-VM job (can_queue_behind refuses other VMs), the compute_flights() cap, or PB pool exhaustion, now repolls every 15 us (g16_repoll_us default) for the whole length of the in-flight render. A compositor plus one app hits this on every frame. Please key it on the slot or flight headroom condition that actually blocked the work.
  4. g16/resources.rs:96 + hw.rs:730: with 48 slots, Tier 2 (18..48) is 30 slots, +107 MiB allocated in one step. That is the same eager 30-slot allocation item 1 of the drm/asahi: g16 + shared SUBMIT: Wave 6 R2 — demand-grown 32 render slots / 18 compute units, fast SUBMIT path, stamp-only barriers, and batched render release #2826 review objected to. The worst case is about 214 MiB of driver-private memory per VM, and nothing in the tier code shrinks it. Please split Tier 2 (for example 18..32 and 32..48) and give the idle and steady footprint numbers.
  5. params.rs:54: render_flights() is no longer capped by FLIGHTS, so up to 47 renders can be in flight. The drm/asahi: g16 + shared SUBMIT: Wave 6 R2 — demand-grown 32 render slots / 18 compute units, fast SUBMIT path, stamp-only barriers, and batched render release #2826 follow-up question is still open: what firmware-published capacity (initdata or ring sizes) shows the firmware accepts 47 TA/FS works in flight? A host VA budget does not answer that.
  6. engine.rs:2425: the comment justifies removing TA->TA barriers by saying "the TA ring is FIFO". FIFO dispatch does not by itself mean TA(N) starts only after TA(N-1)'s FINALIZE. Before this change, two TAs sharing a PB never overlapped. Please say what orders them, or keep the barrier when both passes share a PB epoch. The cross-queue case looks sound: elision now applies only to same-queue wait_through, and other queues are ordered through signalled syncobjs.
  7. Still open from drm/asahi: g16 + shared SUBMIT: Wave 6 R2 — demand-grown 32 render slots / 18 compute units, fast SUBMIT path, stamp-only barriers, and batched render release #2826 and made larger by 48 slots: poll.rs:346 while let Some(..) = self.scrub_queue.lock().pop() still holds the spinlock across the scrub, and there is still no write barrier between scrub completion and slot reuse (slots.rs run_one).
  8. Rows at this head: the only sync coverage is synchronization.op.single_queue (300 strided). Needed: dEQP-VK.synchronization.op.multi_queue.*, synchronization2.* and memory_model.* at 71daf4f; a render and compute hang-recovery row (item 1); an fdinfo engine-time check (item 2); idle CPU wakeups or powertop with compositor plus app (item 3); and the J713 smoke row still owed from drm/asahi: g16 + shared SUBMIT: Wave 6 R2 — demand-grown 32 render slots / 18 compute units, fast SUBMIT path, stamp-only barriers, and batched render release #2826. This PR changes only g16/, but its base still carries the shared mmu.rs/file.rs changes, so the G13/G14/G15/G17 smoke rows owed on drm/asahi: g16 + shared SUBMIT: Wave 6 R2 — demand-grown 32 render slots / 18 compute units, fast SUBMIT path, stamp-only barriers, and batched render release #2826 stay open.

Previously prefix_render() unconditionally emitted a same-ring
FS(N-1)->FS(N) barrier (fs_dep) on every pipelined early render pass,
even when vdm_barrier was NONE or the render dependency had already
retired or been satisfied by TA.

Resolve the exact in-flight render wait stamp (fs_wait_render) in
prefix_render() and plan_render_barriers():
- Keep the TA(N-1)->TA(N) prev_barrier (matching G15) whenever a prior
  render is in flight and plan.ta_fs_wait does not already serialize TA
  behind prev_fs (which dominates prev_ta via own_barrier): consecutive
  pipelined renders share one parameter-buffer / UMA epoch
  (RENDER_UMA_SLOT), so TA(N) must still wait on TA(N-1) before writing
  into the shared PB.
- Emit cross-ring FS->TA (ta_fs_barrier) only when plan.ta_fs_wait
  names an in-flight FS stamp (legacy dependent renders, conservative
  early_tiling=false, or staged TA waits).
- Emit FS->FS (fs_dep) only when plan.fs_render_wait names an
  in-flight FS dependency not already dominated by plan.ta_fs_wait.
- Pre-check wait_stamp_for_with() for o2.wait_through[RENDER] in the
  multi-pass render batch fold loop so prefix_render() never defers
  after claiming a pass slot.

Independent multi-pass renders drop fs_dep (from 3 hardware barrier
records / 5 ring entries per pass to 2 barrier records / 4 ring entries)
while preserving TA->TA serialization on the shared PB epoch, exact
FS->FS coherence on dependent passes, and cross-VM / InitBM boundaries.

Expand test-publication.py to cover independent multi-pass renders,
dependent FRAGMENT_BARRIERS/STAGE_PREFIXES renders (including
TA-dominated waits and older-FS waits), legacy renders,
early_tiling=false, and cross-VM/InitBM boundaries.

Signed-off-by: Diljit Singh <hey@djdev.me>
Every G16 render pass unconditionally emitted four firmware Timestamp
microsequence opcodes (TA start, TA end, FS start, FS end) into the
microsequence buffer and performed two write-combining scene reads at
retirement even when userspace requested no timestamp outputs and G16
profiling (DebugFlags::Debug1) was disabled.

Add compact timestamp-free TA/FS microsequence variants (SEQ_SIZE_NOTS,
FINALIZE_NOTS, END_NOTS) that keep the WaitForIdle (1) opcode, rebase
FINALIZE and its negative restart branch offset, place has_attachments
at the rebased offset, and record exact per-pass timestamp emission in
Flight::stage_ts_mask. When userspace requests no timestamp outputs and
Debug1 profiling is off, encode the compact microsequences, write only
the active microsequence length to WC memory, and skip the per-pass
scene timestamp WC reads on retirement while keeping two safety and
accounting invariants intact:
- In retired_snapshot_at(), a recovery-suspect flight (recovery_suspect
  set) whose render pass omitted microsequence timestamps never retires
  as Retirement::Full; lacking stage-execution proof to distinguish a
  completed pass from a firmware-aborted pass that still advanced stamps,
  it retires as Retirement::Aborted (marking the flight doomed) after
  the 20 ms grace window.
- Record Flight::published_ticks via counter_ticks() (24 MHz CNTPCT_EL0,
  matching firmware's timestamp clock domain) in stat_publish(), and in
  stat_retire() fall back to published_ticks..counter_ticks() through
  queue.usage::Lane whenever firmware timestamps were omitted so DRM
  fdinfo engine-time counters remain active and overlapping pipelined
  flights still account their exact union with zero WC reads.

Verify both V25G83 and V26B123 compact microsequence layouts,
timestamp-free retirement WC read counts, recovery-suspect abort
handling, and fdinfo host-counter fallback/union accounting in the
host selftests.

Signed-off-by: Diljit Singh <hey@djdev.me>
…epoll

Deep multi-pass render benchmarks (such as 11-pass render.postfx at
depth 3, 7-pass compress.bw array at depth 5, and 4-pass render.shadow
at depth 8) saturated the 32-slot pass cap (31 active + 1 displacement)
and stalled between passes waiting for the 100 us resnap timer to free
retired pass slots.

- Raise hw::sizes::RENDER_SLOTS from 32 to 48 (47 in-flight passes + 1
  displacement slot) and widen pass_busy/pass_mask/clean_mask/scrubbing
  in g16/slots.rs from u32 to u64. Both TA and FS work rings hold
  QUEUE_CAP = 256 entries (255 usable); at worst-case 4 ring entries per
  pipelined pass (MAX_RENDER_RING_ENTRIES = 4), 47 in-flight passes
  consume at most 188 entries (67 spare), bounded by compile-time
  assertion in hw.rs and clamped in params::render_flights() to
  (QUEUE_CAP - 1) / MAX_RENDER_RING_ENTRIES (63), well within
  AHEAD_RECORDS = 128, RING_SLOTS = 256, FLIGHTS = 32, and firmware
  recovery snapshot_entries = 448.
- Split EXT_RENDER_TIERS in g16/resources.rs into four demand-grown
  tiers so 20-31 pass workloads do not pay for 48 slots upfront (per
  base slot = 25,184 KiB / 24.59 MiB; per ext slot = 3,680 KiB /
  3.59 MiB):
  - Base   (0..2,   2 slots):  50,368 KiB (49.19 MiB), ext       0 KiB
  - Tier 0 (2..8,  +6 ext):   +22,080 KiB (+21.56 MiB), ext  21.56 MiB
  - Tier 1 (8..18, +10 ext):  +36,800 KiB (+35.94 MiB), ext  57.50 MiB
  - Tier 2 (18..32, +14 ext): +51,520 KiB (+50.31 MiB), ext 107.81 MiB
  - Tier 3 (32..48, +16 ext): +58,880 KiB (+57.50 MiB), ext 165.31 MiB
- Add Engine::slot_wait and Firmware::blocked_on_slot_headroom() so
  poll_step() arms short_repoll (repoll_us) when pending work is
  specifically blocked on same-VM flight, pass/compute slot, or
  work-ring headroom, without arming short_repoll for cross-VM drains,
  InitBM/PB-unprimed waits, external sync fences, or CPU timestamp
  waits.

Update host selftests in test-resources.py, test-publication.py, and
test-poll-retire.py to verify 48 pass slots, all four demand-grown tier
boundaries and exact byte footprints, blocked_on_slot_headroom()
filtering, and slot_wait-gated short_repoll behavior.

Signed-off-by: Diljit Singh <hey@djdev.me>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-checked at abb9cd7 against the 71daf4f review.

Fixed:

  • Suspect flights without timestamps now never retire as Full (g16/poll.rs:1067-1076).
  • The short repoll is keyed on real slot/flight/ring headroom through blocked_on_slot_headroom() (g16/engine.rs:1199-1240, poll.rs:602-606).
  • Tier 2 is split at 32 (g16/resources.rs:96-107).
  • render_flights() is clamped by the work-ring capacity (g16/params.rs:51-64).
  • The TA(N-1)->TA(N) prev_barrier is back whenever TA does not already wait on prev_fs (g16/engine.rs:221, :2507-2519).
  • The scrub lock scope and the wmb are fixed in the base.

New in this revision:

  • Innocent renders now fail on any recovery. recovery_prepare_suspects() (g16/engine.rs:844-866) marks every unfinished flight on every VM as suspect.
  • fdinfo fallback clock (g16/poll.rs:1608-1638). The fallback interval comes from CNTPCT_EL0 (counter_ticks(), engine.rs:63-75), but add_interval converts it with the fixed COMMAND_TIMESTAMP_HZ of 24 MHz (poll.rs:1535). Elsewhere the driver reads CNTFRQ_EL0 for the counter rate (mod.rs:714-719).
    • Please convert with CNTFRQ, or use now_ns() deltas.
    • The window also includes ring queueing and poll latency, and it is charged in full to both vertex and fragment. A short comment saying it is an upper bound would help.
    • A gputop/fdinfo comparison with timestamps on and off at this head would confirm it.
  • PR body. It still lists the 71daf4f commits, says prev_barrier is removed and gives "3 barrier records -> 1". Please update it.

Still open: rows. Nothing is posted at abb9cd7, and all three commits changed. Needed at this head:

  • vk-mustpass-short and gpubench;
  • dEQP-VK.synchronization.op.multi_queue.*, synchronization2.* and memory_model.*;
  • the hang-recovery row above;
  • the fdinfo check;
  • the idle-wakeup check with a compositor plus one app.

The J713 smoke row and the other-generation rows owed on #2826 still apply here.

Assisted-by: LLM (Claude:Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants