Skip to content

drm/asahi: g17: grow a busy render pair's tiler heap ahead of the render - #1902

Merged
RyanTheTide merged 1 commit into
integration/aurorafrom
gpuperf/g17-busy-tvb-grow
Nov 21, 2026
Merged

RyanTheTide merged 1 commit into
integration/aurorafrom
gpuperf/g17-busy-tvb-grow

Conversation

@DjDeveloperr

@DjDeveloperr DjDeveloperr commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

On the A18 Pro (G17P), the host grows a render pair's tiler heap ahead of a render only when the render is submitted to an idle pair: host_growth_target() returns early while the pair has work in flight. A pair that never goes idle, because renders are queued back to back, keeps the heap its first render left. The next tiling pass then waits for the previous scene's pages to be freed instead of growing, so tiling and the previous fragment pass run one after the other. One drain of the pair is enough to size the heap for two scenes, and from then on they overlap.

The firmware grow path already grows a busy pair safely: it appends blocks to the inventory ring and advances the committed cursor, and leaves the firmware-owned page list, PB state and PB descriptor alone. This change:

  • grows a busy pair's heap ahead of the render through that same path (prepare_growth() without a descriptor snapshot, then commit_growth()). The target is the layout's blocks (with the mesh floor), or the automatic target from the page high-water, capped at the heap's capacity. The high-water is read without requesting its reset, so the idle growth check and the shrink period keep their samples;
  • skips the growth while the device's GPU memory, or the client's, would come within an eighth of its limit. A growth that cannot be prepared or committed keeps the heap and publishes the render unchanged, and either case retries a shrink period later. A render never fails because of it;
  • lets the idle shrink give busy growth back down to the size the heap had before it (new busy_base), since that growth only serves renders in flight together;
  • applies to every heap of the block geometry the A18 Pro and the M4 Pro and Max share (BlockGeometry::NEO). M5 heaps have their own geometry and are never shrunk, so they keep growing on idle submissions only;
  • reports every tiler heap size change through the asahi_g17_message tracepoint as kind 3: source << 56 | changed << 48 | blocks before << 24 | blocks after. Source 1 is an idle host growth, 2 a busy host growth (changed 0 means refused or failed), 3 a granted firmware request, 4 a refused or capped one, and 5 a shrink.

Changes since the first version (review of 11-19)

  • Rebased onto integration/aurora cc14570 (conflicts with the mesh floor in host_growth_target(), the Makefile and the M5 Variant).
  • No module parameter: aurora.g17_busy_tvb_grow and busy_tvb_grow.c are gone.
  • No firmware-build gate: Variant::busy_heap_growth is gone. Busy growth follows the heap geometry, so the M4 Pro and Max (25G83 and 26B123) get it too. drm/asahi: g17: grow busy tiler heaps on the 25G83 firmware too #1986 ran it on the J616c (25G83) for about 13 hours without GPU faults, recoveries or timeouts, and jgoyvaerts measured it on the J616s (25G83) taking a never-idle 10,000-triangle frame from 109 ms to 27.5 ms there (see drm/asahi: g17: grow busy tiler heaps on the 25G83 firmware too #1986). 26B123 has not run it yet.

Testing

Neo (J700, A18 Pro, G17P B1), same-machine kernel A/B, 2026-11-21.

  • Kernels, built together on the fedora offload (rustc 1.99.0, clang 22) from the kernel-aurora-core-7.3.0-43 RPM config + DEBUG_INFO_NONE, one-shot GRUB boots, the RPM default untouched:
    • A = e0ee46a (the Edge alpha.2 kernel source), 7.3.0.pg1902a #1 SMP PREEMPT_DYNAMIC Sat Nov 21 03:41:03 EST 2026, aurora.ko sha256 c3abe521…;
    • B = A + this commit (8af03f0e, the same change as 6d406e1 on e0ee46a), 7.3.0.pg1902b #1 SMP PREEMPT_DYNAMIC Sat Nov 21 03:29:50 EST 2026, aurora.ko sha256 5518a109….
  • Mesa: mesa-aurora 26.3.0-35 (git-15fe5010c1, signed RPMs) in a private prefix, libvulkan_asahi.so sha256 767f3e17… in both arms.
  • gpubench 8679dd1 (suite/plan.txt + rt/plan.txt lines), 2 passes per boot, a cool-down before every chunk, rows started on a settled boot (load < 1, no coredump processing). W = Linux vs macOS 26.6.2 (25G83) on the same Neo, > 1 is Linux faster; A/B ratio > 1 is B faster for fps rows and slower for µs rows.
row macOS A B B vs A
rt.scene raster 1080p d2 (fps) 160.6 148.8 (W 0.927) 177.1 (W 1.103) +19.0 %
rt.scene light 720p d1 / d2 (fps) 540.9 / 663.6 622.5 / 850.9 622.3 / 833.7 0.0 % / -2.0 % (A spread 4.2 %)
rt.scene medium 1080p d1 / d2 / d3 (fps) 63.29 / 64.89 / 62.91 64.75 / 66.71 / 66.20 64.78 / 66.14 / 65.78 0.0 / -0.9 / -0.6 % (spreads 0.6-2.9 %)
rt.scene sweep:blas static d2 (fps) 69.31 67.31 67.33 0.0 %
rt.scene sweep:instances 4096 d2 (fps) 9.005 8.989 8.989 0.0 %
render.light tri d1 / d8 / d32 (µs) 239.0 / 29.92 / 26.82 122.8 / 29.06 / 27.41 121.8 / 28.94 / 27.22 -0.8 / -0.4 / -0.7 %
render.light clear d1 / d8 (µs) 238.8 / 29.50 102.7 / 16.66 102.7 / 16.65 0.0 %
render.heavy fill 1080p d1 / d8 (µs) 825.1 / 661.4 695.4 / 615.8 695.4 / 625.8 0.0 / +1.6 %
render.tvb 32768 / 131072 d1, d4 (µs) 0.0 % (all four)

render.heavy fill d2-d64 reads 1.0-1.8 % slower in B. That is thermal drift across the slot, not the change: the passes ran A1, A2, B1, B2 and fill d8 went 609.8, 621.9, 621.9, 629.7 µs, while the GPU time of each render stayed at 610.8-611.3 µs. The raster frame went from 6.72 to 5.64 ms with the raster pass's own GPU time unchanged (6762 vs 6768 µs): the gain is overlap, as intended.

Health over both boots (25 min of rows each): kernel GPU error lines 0 (no faults, recoveries, timeouts or lost devices; one kernel line per window, not GPU), AVC 0, failed units 0, GPU liveness (vulkaninfo + a dEQP-VK compute dispatch) passed at the start and end of each boot.

Still to run before merge: a CTS render subset on B (Neo, next slot), and the M4 Pro rows on 26B123 (Ryan's offer in the review).

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for this. Growing a busy pair's heap ahead of the render looks like the right shape.

Two points, in line with how we select behaviour elsewhere in the tree:

  • drivers/gpu/drm/asahi/g17/buffer.rs:1006 returns early unless self.tvb.geometry == BlockGeometry::NEO, and drivers/gpu/drm/asahi/g17/fw/variant.rs:184 keeps 25G83 off with "Not measured on this firmware". If no hardware or firmware difference keeps the M4 parts out, could this be the default for every G17 geometry that supports in-flight growth? Otherwise, a comment naming that difference would help. drm/asahi: g17: grow busy tiler heaps on the 25G83 firmware too #1986 already reports the 25G83 path running cleanly on a J616c.
  • drivers/gpu/drm/asahi/g17/busy_tvb_grow.c:176-177 adds a writable module parameter that changes production behaviour. Could it be dropped, or kept only in a debug build?

We can run rows on our A18 Pro and M4 Pro for the wider default. The branch also needs a rebase.

The host grows a render pair's tiler heap ahead of a render only when
the render is submitted to an idle pair: host_growth_target() returns
early while the pair has work in flight, and leaves growth to the
firmware. A pair that is never idle, because renders are queued back
to back, keeps the heap its first render left. The next tiling pass
then waits for the previous scene's pages to be freed instead of
growing, so tiling and the previous fragment pass run one after the
other. One drain of the pair is enough to size the heap for two
scenes, after which they overlap.

Grow a busy pair's heap ahead of the render as well, through the path
a firmware grow request already takes while work is in flight:
- the target is the layout's blocks (with the mesh floor), or the
  automatic target from the page high-water (three times the
  high-water once twice it no longer fits), capped at the heap's
  capacity. The high-water is read without requesting its reset, so
  the idle growth check and the shrink period keep their samples;
- prepare_growth() without a descriptor snapshot, then
  commit_growth(): new blocks are appended to the inventory ring after
  the committed cursor and published by advancing it. The
  firmware-owned page list, PB state and PB descriptor are left alone
  and existing blocks never move. The firmware takes the new blocks
  without a grow reply;
- the growth is skipped while the device's GPU memory, or the
  client's, would come within an eighth of its limit. A growth that
  cannot be prepared or committed keeps the heap and publishes the
  render as before. Either case retries a shrink period later;
- the idle shrink may give busy growth back down to the size the heap
  had before it. That growth only serves renders in flight together.

This applies to every heap of the block geometry the A18 Pro and the
M4 Pro and Max share. M5 heaps have their own geometry and are never
shrunk, so they keep growing on idle submissions only.

Report every tiler heap size change through the asahi_g17_message
tracepoint as kind 3: source << 56 | changed << 48 | blocks before << 24
| blocks after, with source 1 an idle host growth, 2 a busy host growth
(changed 0: refused or failed), 3 a granted firmware request, 4 a
refused or capped one and 5 a shrink.

On the A18 Pro, rt.scene raster at queue depth 2 ran about 18 % faster
in the first measurement of this change: the heap grows within the
first frames instead of waiting for a drain. The light and medium
presets were unchanged.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr
DjDeveloperr force-pushed the gpuperf/g17-busy-tvb-grow branch from 7771d2f to 6d406e1 Compare November 21, 2026 09:18
@DjDeveloperr

Copy link
Copy Markdown
Contributor Author

@RyanTheTide thanks, both points are addressed in 6d406e1 (rebased onto integration/aurora cc14570):

  • The firmware-build gate is gone (Variant::busy_heap_growth removed). Busy growth now follows the heap geometry: every BlockGeometry::NEO heap, so the A18 Pro and the M4 Pro and Max (25G83 and 26B123) get it. M5 heaps stay out by their own geometry, because they are never shrunk and busy growth would never be given back. I found no firmware difference that should keep the M4 parts out: drm/asahi: g17: grow busy tiler heaps on the 25G83 firmware too #1986 ran the same path on the J616c (25G83) for about 13 h, and jgoyvaerts measured it on the J616s.
  • The module parameter is gone (busy_tvb_grow.c removed).

Neo A/B of the reworked change (same kernel source, built together, one-shot boots): rt.scene raster d2 148.8 -> 177.1 fps (+19.0 %, W vs macOS 0.927 -> 1.103), the other rt.scene presets and render rows within spread, 0 GPU error lines. Details in the description. Still open: a CTS render subset on the Neo (next slot) and, if your offer stands, the M4 Pro rows on 26B123.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Looks good; approve once the CTS render row and the 26B123 row are posted.

I checked the busy path from start to finish. host_growth_target() (queue/render.rs:1826-1836) skips quarantined pairs. busy_target() (buffer.rs:999-1031) reads max_pages without a reset and caps at max_blocks(). prepare.rs passes no descriptor snapshot, so prepare_growth() takes the firmware-request branch and builds no rebuilt page list. commit_growth() still checks blocks() == previous and the cursor capacity under the device mutex. A failed prepare or commit only defers the growth and leaves the prepared mappings for the caller to drop. The firmware grow handler now traces before and after without changing its logic. I found no correctness problem.

  1. Nit, buffer.rs:1046-1060: once busy_base is set, shrink_floor() ignores the recent peak until an idle growth or a shrink to busy_base clears it. That includes blocks a firmware grow request granted later for a single render, not only growth for renders in flight together. The next idle growth restores the size, so this costs a rebuild, not correctness. If that is intended, say so in the comment at buffer.rs:1051. Otherwise clear busy_base when a firmware request is granted.

HW row: Neo (J700) kernel B (this head on e0ee46a): a dEQP-VK/GLES render subset with pass/fail counts equal to kernel A. J616s or J616c on 26B123: the same rt.scene raster d2 A/B with 0 GPU faults, recoveries and timeouts. Both rows also need 0 AVC, 0 failed units and 0 new kernel errors.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide
RyanTheTide merged commit 773debe into integration/aurora Nov 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants