Skip to content

drm/asahi: g17: tile clustered on the 32-core M4 Max (T6041 25G83) - #2021

Open
AxxzyWasTaken wants to merge 3 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:pr/t6041-clustered-tiling
Open

AxxzyWasTaken wants to merge 3 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:pr/t6041-clustered-tiling

Conversation

@AxxzyWasTaken

@AxxzyWasTaken AxxzyWasTaken commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

T6041 runs its 25G83 tiling passes in TA mode 1 (0x1c890 = 1), which tiles non-clustered: one of the four clusters does the tiling and vertex work of every pass. Vertex shaders reach a quarter of the GPU's ALU rate (2.95 vs 12.3 TFLOPS on a 32-core part), so geometry-heavy games are capped by it.

TA mode 1 came from my report in #525: it stopped reason-0 hangs on the M4 Max. Those hangs had more than one cause. This series fixes them instead of avoiding clustered tiling on the 32-core M4 Max (J616C).

Commits

  1. drm/asahi: g17: scale the render pool floor from GPU shape:
    • With all four clusters running vertex work beside fragment work, the 44-block USC render pool runs out and Roblox (Vulkan) hangs about once a minute (reason-0 no-progress recovery, no MMU fault, nearly empty tiler heap). Because G17 firmware has no grow-request event for the USC render pool (unlike TVB growth) and a published render pool cannot be resized in place, size the floor up front in g17::fw::render_rt::required_blocks().
    • One- and two-cluster GPUs keep the 44-block floor (RENDER_BLOCKS), leaving A18 Pro and M4 Pro unchanged. Above two clusters, the base scales at 3 blocks (3 MiB) per enabled core, floored at 44 (empirically fitted from one 32-core J616C measurement, 96 / 32; 40-core J614C = 120 blocks is untested). Ray-query scratch is added on top.
    • WIDE_LOWER (~3.19 GiB) capacity: queue creation allocates no VA/backing; backing is placed lazily on first render (~98 MiB span on J616C), reducing simultaneously resident pools from 58 to ~33 on J616C (~26 on J614C). When WIDE_LOWER fills, wait_render_capacity() -> reclaim_render_capacity() evicts idle pools via FreeListRelease and transparently re-arms them on next use; only >33 (J616C) / >26 (J614C) simultaneously active render contexts held past QUEUE_WAIT_NS fail submit with EBUSY.
    • Because this keys on GPU shape, 25F84, 40-core 25G83, and 26B123 on T6041 keep ta_mode_1 = true and only inherit the larger render pool floor.
  2. drm/asahi: g17: tile clustered on the 32-core M4 Max (T6041 25G83):
    • Splits BUILD_25G83_32CORE so clustered tiling (ta_mode_1 = false and full-table tvb_normal_blocks = NORMAL_MAX_BLOCKS_FOUR_CLUSTERS) is enabled strictly on the tested 32-core T6041 25G83 configuration (J616C). 40-core T6041 parts (J614C, J575C), 25F84, and 26B123 stay on ta_mode_1 = true and the default NORMAL_MAX_BLOCKS heap maximum until tested on hardware.
    • Clustering metadata sized for >2 clusters (0x1180 metadata 2 per macOS 26.6.2 capture; note 25F84/40-core 25G83/26B123 share RegisterPrograms::V25G83 and inherit the larger metadata sizing while remaining in TA mode 1).
    • Four-cluster segment words from fw::m5::target::cluster_control() (0x280540, 4) and v25g83_clustered_free_pages(clusters) = 28 + 16 * clusters (0x3c for 2 clusters, 0x5c for 4 clusters).
  3. drm/asahi: g17: a 16 KiB floor for four-cluster metadata 1:
    • Conservative 16 KiB floor (META1_MIN_SIZE = 0x4000) for four-cluster metadata 1 (formula yields 0x100–0x180 on 3456x2234 Retina and 0x1100 on 10240x10240) with a KUnit layout test.

Testing

  • Host checks on 2ba7a68b14a5 (rebased onto integration/aurora 71abea7fa7c7):

    • tools/asahi/run-host-tests.py --only g17: 90/90 passed (CI host-tests SUCCESS).
    • scripts/checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks across all 3 commits (ce99a9090138, 0ffbc70ac5c6, 2ba7a68b14a5).
  • Exact-head J616C (32-core M4 Max, 25G83) boot & light 3D check at 2ba7a68b14a5 (7.3.0-j616c-pr2021+):

    • BUILD_25G83_32CORE probes cleanly; glmark2-es2-drm at 3456x2234 (zink/Borealis Vulkan 1.4) Score 2479 (build 3622 FPS, shading=phong 3495 FPS, terrain 324 FPS, exit 0); 0 GPU faults, 0 timeouts/resets, FAILED_UNITS=0.
    • @RyanTheTide APPROVED on 2ba7a68b14a5 (2026-11-22T10:30:51Z).
    • @RyanTheTide (11:54Z): the approval covers code + exact-head boot; glmark2 is too light to fill the old 44-block pool. @DjDeveloperr CHANGES_REQUESTED (10:35Z, re-affirmed 14:39Z) on §5 evidence (below).
  • J616s (M4 Pro, T6040, 25G83, two clusters) non-regression row: 2ba7a68b14a5 merged on integration/aurora a4631125 with unrelated #2549/#1906/#2784 (posted by @RyanTheTide):

    • gpubench: render.heavy d1/d4 272.3 / 127.8 vs 276.4 / 127.8 us/submit on prod; compute.heavy ALU d4 7511 vs 7504 GFLOPS; render.light d1/d16 121.5 / 24.83 vs 116.6 / 24.01 on prod; G17 LowerVa capacity refusals 2 vs 1.
    • dEQP-VK 1.4.7.1 smoke (16541 cases): 8951 Pass / 7590 NotSupported / 0 DeviceLost (vs 8950 / 7590 / 1 on prod).
    • 0 GPU faults/timeouts, 0 failed units, 0 AVC denials, 0 kernel errors.
  • Earlier daily-tree measurements on J616C (25G83, Hyprland) with the same clustered tiling and 96-block pool (that tree also carries other changes: zeroed clustering pointers, a TA PTC cap, unclustering under 256 tiles; off-screen, identical images in both modes):

    Test Non-clustered Clustered
    Vertex-shader ALU 2.95 TFLOPS 9.8-10 TFLOPS
    Sodium-style terrain, 16k draws 1.93 ms 0.82 ms
    32.8M vertices at 3456x2234 13.9 ms 4.65 ms

    Minecraft (Fabulously Optimized, Sodium, OpenGL) at RD32 went from 105 to ~300 fps standing; Roblox in Vulkan ran 11+ minutes with 0 recoveries (vs 1-2 minute hangs at 44 blocks).

  • Remaining §5 scope on exact head 2ba7a68b14a5 (J616C): heavy-load stability soak exercising >44 render blocks and >0x45e TVB growth (Roblox Vulkan 15+ min + Minecraft RD32 with 0 GPU recoveries), same-machine A/B vs base 71abea7fa7c7 (vertex-ALU/terrain + dEQP-VK/GLES pass counts unchanged), and explicit 0 SELinux denials + 0 new kernel errors counts.

Not tested on hardware: 40-core M4 Max (J614C/J575C — kept on ta_mode_1 = true), 25F84, and 26B123.

Assisted-by: LLM (Anthropic:Claude Opus 5.5, Google:Gemini 4 Neon)

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Added a fourth commit: clustering metadata 1 on four-cluster passes now has a 16 KiB floor (META1_MIN_SIZE). The old floor was 0x80, the two-cluster M4 Pro size. The size per block of tiles (0x44) stays the M2 Pro/Max formula. The 16 KiB floor is my own choice, not a macOS value: the capture doesn't show metadata 1's size. Metadata 3 and 4 already matched my tree.

The floor has been in my daily module since 16 Nov, but not tested on its own. This branch is built (M=drivers/gpu/drm/asahi), not booted as-is; run-host-tests.py --only g17 90/90.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested on my M4 Max as described.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

A gentle ping: this is ready for review. Head 6096eb7 merges cleanly into current integration/aurora and CI passes. Clustered tiling has been in the J616C's daily kernel since 16 Nov. #2170, #2171 and #2172 build on it.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described.

@AxxzyWasTaken
AxxzyWasTaken force-pushed the pr/t6041-clustered-tiling branch from 6096eb7 to 1e62e2c Compare November 21, 2026 09:49
@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 2.

  1. drivers/gpu/drm/asahi/g17/fw/freelist.rs:15 and hw/t6041.rs:198: the 96-block render pool is the depth that stopped the Roblox hang on one Mac and one build, keyed by chip and firmware build. The cause you describe is pool demand from vertex work running on every cluster at once. That scales with what required_blocks() (fw/render_rt.rs:38) already gets: active cores, or the cluster count the pair carries. Derive the minimum from that, as the ray-query scratch already is. Then the 40-core J614C on the same BUILD_25G83 (it shares this variant through CONFIG_25G83) and any later four-cluster part get a size that follows their shape instead of a 32-core number. If 96 cannot be derived, the commit message should say why, and the value should be proven on the 40-core part as well.
  2. The exact head has not been booted. The numbers come from a daily tree that also carries other changes (zeroed clustering pointers, a TA PTC cap, unclustering under 256 tiles). The PR flips the default tiling mode for every T6041 on 25G83, including the 40-core part, which has not been tested. It needs one run of this head before it can merge.

Nits:

  • g17/fw/render.rs:766-768: the comment says the four-cluster arm uses "32 + 16 x clusters free pages", but the arm drops cluster_control()'s page count and writes T6041_CLUSTERED_FREE_PAGES (0x5c) for any GPU above two clusters on V25G83 programs. Fix the comment, and either name the constant for what it is (four-cluster V25G83) or tie it to the cluster count.
  • g17/buffer/target.rs:46: META1_MIN_SIZE (16 KiB) is a chosen floor with no measurement behind it. Oversizing only costs memory, so it is safe, but please note in the comment what a heavy four-cluster frame actually used, so the floor can be checked later.

What I checked: every queue gets the larger pool up front, because queue_create always calls reserve_render_scs() (file.rs:1738). So the blocks > render_blocks() check in enqueue (g17/queue/mod.rs:226) cannot start refusing renders on T6041 25G83. normal_blocks is bounded by geometry.max_blocks() before it reaches the graph (g17/buffer.rs:775). 25F84 and 26B123 keep TA mode 1 (hw/t6041.rs:113, :136), so machines on 27.2 firmware see none of this change.

HW row: J616C (32-core) on this exact head, 25G83: Roblox (Vulkan) 15+ min and Minecraft RD32 with 0 GPU recoveries; the vertex-ALU A/B; dEQP-VK/GLES pass counts unchanged against the base; 0 AVC denials, 0 failed units, 0 new kernel errors. Plus one J614C (40-core) 25G83 boot with a short clustered workload and 0 recoveries, or keep the 40-core configuration on TA mode 1 until that run exists.

Assisted-by: LLM (Claude:Opus 5.5)

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Rebased onto integration/aurora cacf3554 and reshaped into a 3-commit bisectable series (0409ba099c6b) addressing Blocker #1 and both Nits (note: commits now carry human Signed-off-by only, with no Assisted-by: trailers in commits):

  1. 6f8cf5a8d4a3 — drm/asahi: g17: scale the render pool floor from GPU shape (moved first so clustered tiling never lands on a 44-block pool):
    • Removed the chip/firmware-gated RENDER_BLOCKS_T6041_25G83 / Variant::render_pool_blocks; derived base_blocks() from GPU shape inside g17::fw::render_rt::required_blocks() (44 for <= 2 clusters; (active_cores * 3).max(44) for > 2 clusters, with ray-query scratch added on top).
    • Gives 96 blocks on 32-core J616C (measured Roblox/SCP:SL fix) and 120 blocks on 40-core J614C (untested), leaving A18 Pro / M4 Pro / M5 pool paths unchanged. Documents the WIDE_LOWER context-capacity cost (~33 contexts on J616C, ~26 on J614C) and that 25F84/26B123 on T6041 inherit the shape-derived floor. Added asahi_g17_render_pool KUnit tests.
  2. 2575ca2b096c — drm/asahi: g17: tile clustered on the four-cluster M4 Max (T6041 25G83) (folded the metadata 2 / free-pages fix so bisect never hits the 0xc00 overflow):
    • Replaced the dropped page count + T6041_CLUSTERED_FREE_PAGES with v25g83_clustered_free_pages(clusters) = 28 + 16 * clusters (0x3c for 2 clusters, 0x5c for 4, 0x9c for 8) passed into cluster_control(), with KUnit tests.
    • Accurately notes that 25F84 and 26B123 keep ta_mode_1 = true (non-clustered) while sharing RegisterPrograms::V25G83's four-cluster metadata sizing.
  3. 0409ba099c6b — drm/asahi: g17: a 16 KiB floor for four-cluster metadata 1:
    • Documented the M2 Pro/Max formula sizes (0x100/0x180 on 3456x2234, 0x1100 on 10240x10240) and the unmeasured high-water-mark caveat; added four_cluster_v25g83_metadata_layout KUnit test.

Checks: checkpatch.pl --strict 0/0/0 on all 3 commits; aurora.o compiles clean with W=1 (default + CONFIG_KUNIT=y); run-host-tests.py --only g17 90/90 pass. Exact-head J616C boot row (Blocker #2) is queued next, and a 40-core J614C row is requested before merge.

Assisted-by: LLM (Google:Gemini 4 Neon)

@AxxzyWasTaken
AxxzyWasTaken force-pushed the pr/t6041-clustered-tiling branch from 1e62e2c to 0409ba0 Compare November 21, 2026 22:27
@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-reviewed at 0409ba0. The chip/firmware-keyed RENDER_BLOCKS_T6041_25G83 is gone and the pool now follows the GPU's shape, and both nits are done (free pages are 28 + 16 x clusters with tests, the metadata-1 floor says what it was sized against). Thanks for the bisectable split.

What is still open on blocker 1 is where the numbers come from:

  • MULTI_CLUSTER_RENDER_BLOCKS_PER_CORE = 3 is 96 / 32, the one depth that stopped the hang on the J616C, written per core. It is the same measurement as before, now extrapolated: nothing says demand is linear in cores rather than in clusters, tiles or vertex output, and the 40-core part gets 120 without a run. The step at two clusters makes that visible: a 2-cluster 20-core part gets 44 (2.2 per core), a 3-cluster 24-core part would jump to 72.
  • The cost is real: the commit itself says WIDE_LOWER context capacity drops to ~33 contexts on the J616C and ~26 on the J614C. What does the 34th GPU context see, an error at queue creation? On a desktop with many GL/Vulkan clients that is a new failure mode.
  • The shape that avoids picking a number is growing the pool when the firmware reports it is short (the no-progress / reason-0 event you already see), as drm/asahi: g18: grow the parameter buffer when the firmware asks (10 s GPU stalls on the M6 Mac mini) #2242 does on G18 for the parameter buffer. If that is too big for this PR, keep the floor but say in the commit message that 3/core is fitted from one Mac, and get the 40-core row before the default flips for it.

Also worth stating in the description: on T6041 only 25G83 turns clustered tiling on (ta_mode_1 stays true on 25F84 and 26B123), so Macs on the 27.2 firmware keep non-clustered tiling and only get the bigger pool.

Blocker 2 is unchanged until the exact-head J616C row lands (you say it is queued).

Assisted-by: LLM (Claude:Opus 5.5)

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed re-review on 0409ba099c6b. Traced both questions in Blocker 1 directly against the G17/G18 code:

  1. Why G18 #2242-style dynamic growth doesn't port to the G17P USC render pool:

    • #2242 grows the parameter buffer (TVB) in response to firmware's TvbGrowEventRecord (event type 6) / TvbGrowReply (device-control opcode 8). G17 already implements that TVB growth protocol (g17/fw/channels.rs:136, g17/runtime/events.rs:111, g17/runtime/render/grow.rs:1-46).
    • For the USC render pool, G17 firmware has no grow-request event (Event in g17/fw/channels.rs:133-148 only carries TvbGrow and host-initiated FreeListGrow completions). When the 44-block pool exhausts on T6041 25G83, firmware stalls straight into a generic no-progress recovery (reason 0). Non-M5 RenderBacking is also a single contiguous KernelObject in WIDE_LOWER (g17/freelist.rs:543-552) and Context::reserve_render_blocks() rejects resizing once published (EBUSY, g17/context.rs:558-562), so post-recovery growth would still drop the victim frame and disrupt concurrent clients.
  2. Exact 34th-context WIDE_LOWER behaviour (traced in code):

    • queue_create (file.rs:1738 → reserve_render_scs() → reserve_render_blocks()) only stores an atomic block count and allocates zero VA/backing.
    • Backing is allocated lazily on first render in WIDE_LOWER (3,264 MiB; hw/t6041.rs:46). When map_into_range() hits ENOSPC, place_lower() raises LOWER_REFUSED (g17/freelist.rs:310-317) and submit.rs:501-526 enters wait_render_capacity() → reclaim_render_capacity() (g17/runtime/render/submit.rs:343-385), which sends FreeListRelease and vacates idle render pools (vacate_idle_pools() / reclaim_idle_render_pools(), g17/runtime/render.rs:1407-1440). Evicted idle desktop contexts transparently re-arm via pool.ensure_backing() (g17/context.rs:585) on next use.
    • Only when >33 (J616C) / >26 (J614C) distinct rendering contexts are simultaneously active (non-idle) for longer than preparation::QUEUE_WAIT_NS does wait_render_capacity() time out and fail that submission with EBUSY (g17/runtime/render/submit.rs:324-328)—not ENOSPC and not at queue creation, leaving existing contexts intact.

Next step: Keeping 0409ba099c6b steady while your J616s regression row runs; next update will:

  • Take the fallback path on Blocker 1: explicitly document in commit 6f8cf5a8d4a3 that MULTI_CLUSTER_RENDER_BLOCKS_PER_CORE = 3 is empirically fitted from one 32-core J616C (96 / 32), correct the commit's ENOSPC wording to the reclaim_render_capacity() / EBUSY path above, and clarify in the PR description that 25F84 and 26B123 keep ta_mode_1 = true and only inherit the larger pool.
  • Keep 40-core T6041 (J614C) on ta_mode_1 = true unless we get a 40-core hardware row before flipping it, and post the exact-head J616C hardware row.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for tracing both. That settles blocker 1 for me:

  • No grow event: agreed. G17 firmware only signals TVB growth, the USC render pool has no request to answer, and a published pool can't be resized. So the drm/asahi: g18: grow the parameter buffer when the firmware asks (10 s GPU stalls on the M6 Mac mini) #2242 shape doesn't port, and a documented floor is the right tool here.
  • 34th context: also fine. Idle pools are reclaimed through reclaim_render_capacity() and re-armed on next use, and only more than ~33 simultaneously active rendering contexts get EBUSY at submit, after the queue wait, with existing contexts untouched. That's a sane failure mode.

So blocker 1 is resolved once the update you describe lands: the commit says 3 blocks/core is fitted from one 32-core J616C, the ENOSPC wording becomes the reclaim/EBUSY path, the description notes that 25F84/26B123 keep ta_mode_1 and only get the bigger pool, and the 40-core part stays on ta_mode_1 = true until it has a row. With that and the exact-head J616C row, this is fine from our side.

Assisted-by: LLM (Claude:Opus 5.5)

@AxxzyWasTaken
AxxzyWasTaken force-pushed the pr/t6041-clustered-tiling branch from 0409ba0 to 6177db6 Compare November 22, 2026 06:30
@AxxzyWasTaken AxxzyWasTaken changed the title drm/asahi: g17: tile clustered on the four-cluster M4 Max (T6041 25G83) drm/asahi: g17: tile clustered on the 32-core M4 Max (T6041 25G83) Nov 22, 2026
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Pushed 6177db61a671 with all four Blocker 1 updates (human Signed-off-by: preserved on all 3 commits):

  1. 79528dc53181 (commit 1): Updated the commit message to state that 3 blocks/core is empirically fitted from one 32-core J616C (96 / 32) and replaced the ENOSPC sentence with the traced wait_render_capacity() -> reclaim_render_capacity() (FreeListRelease) / EBUSY path.
  2. c9a3512e9de6 (commit 2): Split BUILD_25G83_32CORE in hw/t6041.rs and hw/t6041_25g83.rs so 40-core T6041 (J614C/J575C) stays on ta_mode_1 = true and NORMAL_MAX_BLOCKS until tested on 40-core hardware, enabling clustered tiling strictly on 32-core T6041 25G83 (J616C), with updated unit tests in g17/fw/render.rs.
  3. 6177db61a671 (commit 3): Unchanged 16 KiB META1_MIN_SIZE floor + layout test.
  4. PR description: Updated to explicitly note that 25F84, 40-core 25G83, and 26B123 keep ta_mode_1 = true and only inherit the larger render pool floor.

Verified scripts/checkpatch.pl --strict (0 errors / 0 warnings / 0 checks on all 3 commits) and tools/asahi/run-host-tests.py --only g17 (90/90 passed). Exact-head J616C hardware row on 6177db61a671 is queued next once the test box reconnects.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-checked at 6177db6: the update lands as described. Commit 1 says the 3 blocks/core is fitted from one 32-core J616C and describes the reclaim/EBUSY path. Commit 2 turns clustered tiling on only for the 32-core 25G83 image and keeps the 40-core images (J614C, J575C) and 25F84/26B123 on TA mode 1, selected by which identified firmware build is loaded rather than by board. Commit 3 is unchanged. Blocker 1 is resolved. The exact-head J616C row is the last gate.

Assisted-by: LLM (Claude:Opus 5.5)

With four-cluster tiling the 32-core M4 Max runs vertex shaders on every
cluster at once beside fragment work. Roblox (Vulkan) then hangs the GPU
about once a minute: the firmware kills a small depth/stencil render as
no-progress (reason 0), with no MMU fault and a nearly empty tiler heap.
The same hang showed up on this Mac with the earlier driver until its
render pool had 96 blocks. G17 firmware has no grow-request event for
the USC render pool (unlike TVB growth) and a published render pool
cannot be resized in place, so the pool must be sized up front.

Rather than gating a fixed 96-block pool on T6041 and firmware build
25G83, derive the render pool minimum from GPU shape (active cores and
cluster count) inside g17::fw::render_rt::required_blocks(), before
adding ray-query scratch extras:

- One- and two-cluster GPUs keep the 44-block floor (RENDER_BLOCKS),
  preserving the A18 Pro shared pool window invariant and the M4 Pro's
  narrow pool window unchanged.
- Above two clusters, the base pool scales at 3 blocks (3 MiB) per
  enabled core, floored at RENDER_BLOCKS (44). Note that 3 blocks/core
  is an empirical fit from one 32-core J616C measurement (96 / 32), not
  a hardware-derived bound; the 40-core J614C value (120 blocks) has not
  been run on hardware yet. Because it keys on GPU shape, every
  >2-cluster G17 firmware build (including 25F84 and 26B123 on T6041)
  gets this floor, not only 25G83.
- Ray-query scratch is added on top of this base instead of competing
  with it.

Capacity cost in WIDE_LOWER (~3.19 GiB, shared with other lower-window
allocations): queue creation allocates no VA or backing; backing is
placed lazily on first render (~98 MiB span on J616C, ~122 MiB span on
J614C, vs ~46 MiB before), reducing the simultaneously resident pool
budget from 58 (MAX_EXECUTION_CONTEXTS) to roughly 33 on J616C and 26 on
J614C. When WIDE_LOWER fills, submit enters wait_render_capacity() ->
reclaim_render_capacity(), which evicts idle render pools via
FreeListRelease and transparently re-arms them on next use. Only when
more than ~33 (J616C) / ~26 (J614C) distinct contexts are simultaneously
active for longer than QUEUE_WAIT_NS does a new submission fail with
EBUSY, leaving existing contexts intact. J616C has run with 96 blocks
daily since 16 Nov without hitting the cap.

Resulting base render pool blocks (pages = [0, 0]):

  Part                     Clusters  Cores  Base blocks
  A18 / A18 Pro (T8140)           1   5/6            44  (unchanged)
  M4 (T8132, uses g16)            1  8/10            44  (if on g17)
  M4 Pro (T6040)                  2 16/20            44  (unchanged)
  M4 Max 32-core (J616C)          4    32            96  (measured fix)
  M4 Max 40-core (J614C)          4    40           120  (untested)
  M5 (T8142, M5 pool path)        1  8/10            44  (unchanged)
  M5 Pro (T6050, M5 pool)         2 16/20            44  (unchanged)
  M5 Max (T6050, M5 pool)         4 32/40        96/120  (M5 uses own)
  M5 Ultra (T6050, M5 pool)       8 64/80       192/240  (M5 uses own)

Add KUnit tests (asahi_g17_render_pool) covering these shapes, ray-query
scratch addition, and invalid/oversized rejections.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
T6041 ran its 25G83 tiling passes in TA mode 1 (0x1c890 = 1), which
tiles non-clustered: one of the four clusters runs the tiling and
vertex work of a pass. Vertex shaders reached a quarter of the ALU rate
of the GPU (2.95 vs 12.3 TFLOPS), and geometry-bound games were capped
by it.

Tile clustered instead on the 32-core T6041 (J616C) 25G83 build, while
keeping the untested 40-core T6041 parts (J614C, J575C) on TA mode 1
until they have a hardware row. Clustered four-cluster passes receive:

- Clustering metadata sized for more than two clusters, with metadata 2
  at 0x1180 as macOS 26.6.2 writes on the 32-core M4 Max (metadata 2 at
  the M2 Pro/Max 0xc00 size allowed the tiler to write past it into
  metadata 3; metadata 3 and 4 keep their larger sizes). Because 25F84,
  40-core 25G83, and 26B123 share RegisterPrograms::V25G83, four-cluster
  targets on those builds also receive the 0x1180 metadata 2 sizing
  while keeping TA mode 1 (non-clustered); that oversizing has not been
  booted on 25F84/26B123.
- The four-cluster segment words of cluster_control(): 0x1c890 =
  0x280540, 0x1c918 = 4, and v25g83_clustered_free_pages(clusters) =
  28 + 16 * clusters free parameter-buffer pages in 0x1c9a8 (0x3c for
  two clusters, 0x5c for four, matching the macOS 26.6.2 capture). Two
  clusters keep the M4 Pro capture.
- The full block table for Variant::tvb_normal_blocks on 32-core 25G83,
  because a clustered RD32 Minecraft frame needs about 1600-2200 blocks
  and never starts at the 0x45e default.

25F84, 40-core 25G83, and 26B123 keep TA mode 1 and the default heap
maximum. Every other chip and build keeps its values.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Four-cluster passes size clustering metadata 1 as the M2 Pro/Max path
does, 0x44 bytes per block of tiles, with 0x80, the two-cluster M4 Pro
size, as the floor. On a 3456x2234 Retina frame (Minecraft RD32) that
formula yields 0x100 (32x32 utiles) or 0x180 (16x16), and on 10240x10240
(point_size_10000, 16x16 utiles) it yields 0x1100.

Because metadata 1 is the last sub-buffer in the TPC object, neither
macOS's allocation size nor the hardware's written high-water mark was
visible in the J616C 26.6.2 capture. Give it a conservative 16 KiB floor
until measured, document the formula sizes on those frames, and add a
KUnit test covering the four-cluster V25G83 metadata layout. (Because
this keys on RegisterPrograms::V25G83 && clusters > 2, 25F84 and 26B123
on T6041 also get this conservative floor.)

The per-block size stays the M2 Pro/Max formula; the floor is our own
choice, not a macOS value.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken
AxxzyWasTaken force-pushed the pr/t6041-clustered-tiling branch from 6177db6 to 2ba7a68 Compare November 22, 2026 07:17
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Rebased onto integration/aurora (71abea7fa7c7) as 2ba7a68b14a5 (preserving Signed-off-by: on all 3 commits), resolving the drivers/gpu/drm/asahi/hw/t6041.rs doc-comment conflict with merged #2082 (dda522bb9748). The newly admitted 40-core J614C 25G83 image under iBoot 27.2 stays on BUILD_25G83 (ta_mode_1 = true, NORMAL_MAX_BLOCKS), while 32-core J616C uses BUILD_25G83_32CORE. Verified scripts/checkpatch.pl --strict (0/0/0 on all 3 commits) and tools/asahi/run-host-tests.py --only g17 (90/90 pass). Exact-head J616C row on 2ba7a68b14a5 is queued once the test machine is back online.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

J616C (M4 Max 16c/32g) exact-head boot & 3D render verification at 2ba7a68b14a5 (7.3.0-j616c-pr2021+, Stage 2 1.0.0b14-g6ac0495f):

  • GPU probe & 32-core BUILD_25G83_32CORE split (dmesg):
    • aurora 28a600000.gpu: GPU firmware build: Apple GFX 25G83 for T6041 (normalized loaded image, 32-core J616C)
    • aurora 28a600000.gpu: GPU firmware: macOS 26.6.2 (25G83, iBoot mBoot-18000.161.10), 32 cores
    • [drm] Initialized aurora 1.0.0 for 28a600000.gpu on minor 1
  • 3D render workload (glmark2-es2-drm at 3456x2234, zink / Borealis Vulkan 1.4):
    • build: 3622 FPS (0.276 ms)
    • shading=phong: 3495 FPS (0.286 ms)
    • terrain: 324 FPS (3.087 ms)
    • Score: 2479, exit 0.
  • Health: 0 GPU faults, 0 firmware timeouts/resets, FAILED_UNITS=0, wlan0 UP.

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head J616C row at 2ba7a68: the 32-core 25G83 split is selected and renders run. With the fitted floor documented and the 40-core parts left on TA mode 1, our blocker is resolved. Approving.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full review of 2ba7a68b14a5072cd77fa22736f9f09d89f4a7a6 against pre-pr-checklist.md (§1–§6): REQUEST_CHANGES (code clean; held strictly on §5 evidence).

Code & checklist audit (§1–§4, §6) — clean

  • §1 No model gating: Render pool floor (g17/fw/render_rt.rs::base_blocks(active_cores, clusters)), four-cluster metadata sizing (g17/buffer/target.rs, clusters > 2), and segment words (v25g83_clustered_free_pages(clusters) = 28 + 16 * clusters fed into cluster_control()) all derive cleanly from GPU shape with checked arithmetic and KUnit coverage (asahi_g17_render_pool, four_cluster_v25g83_metadata_layout, four_clusters_take_the_four_cluster_segment_words). Splitting BUILD_25G83_32CORE in hw/t6041.rs / hw/t6041_25g83.rs keeps untested 40-core T6041 (J614C/J575C), 25F84, and 26B123 on ta_mode_1 = true and NORMAL_MAX_BLOCKS, matching @RyanTheTide's agreed scope.
  • §2 Production-ready / §3 Safety: Works on default command line with no module params or magic waits; normal_blocks is bounds-checked against geometry.max_blocks() in g17/buffer.rs; the conservative 16 KiB META1_MIN_SIZE floor only over-allocates safely; WIDE_LOWER reclaim/EBUSY behavior is accurately traced and documented.
  • §4 Clean-room & attribution / §6 Naming: 3 clean bisectable commits (ce99a9090138, 0ffbc70ac5c6, 2ba7a68b14a5), all human author/committer + human Signed-off-by: only, zero AI trailers in commits; PR body carries Assisted-by:; Borealis/Aurora naming clean.
  • CI: host-tests SUCCESS; MERGEABLE (CLEAN), non-draft.

Remaining blocker before approval (§5 Evidence)

The exact-head J616C boot row at 2ba7a68b14a5 (7.3.0-j616c-pr2021+) confirms BUILD_25G83_32CORE probes cleanly and glmark2-es2-drm at 3456x2234 passes (terrain 324 FPS, Score 2479, 0 GPU faults, FAILED_UNITS=0). However, glmark2-es2-drm alone does not yet close §5 (and @RyanTheTide's gate) for the specific fixes and performance claims in this PR:

  1. Heavy-load stability at exact head 2ba7a68b14a5: Run a workload that actually exercises the 96-block render pool and >0x45e TVB heap growth (e.g., Roblox Vulkan 15+ min soak and Minecraft RD32) confirming 0 GPU recoveries on this exact head (the PR body numbers are from an older daily tree with additional patches).
  2. Same-machine A/B + conformance unchanged: For the PR's performance claims, provide a same-machine A/B at 2ba7a68b14a5 vs base 71abea7fa7c7 (e.g., vertex-ALU / terrain A/B) and confirm conformance (dEQP-VK/GLES pass counts unchanged against base).
  3. Full regression check: Confirm 0 SELinux denials and 0 new kernel errors alongside FAILED_UNITS=0.

(Non-blocking nit: update the stale 6177db61a671 SHA references in the PR description once the final 2ba7a68b14a5 row is ready.)

Assisted-by: LLM (Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

To add to the approval: it covered the code and the exact-head boot. The glmark2 run shows the 32-core image is selected and renders, but it is too light to fill the old 44-block pool, so it doesn't exercise the hang the 96-block floor fixes. Before merge, Roblox (Vulkan) for 15+ minutes and Minecraft RD32 at 2ba7a68 with 0 GPU recoveries are still wanted, plus dEQP pass counts against base 71abea7f and a check for AVC denials and new kernel errors, as in the earlier row list.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

J616s (MacBook Pro 16" M4 Pro, T6040, 25G83, two clusters) no-regression row at 2ba7a68, merged on integration/aurora a463112 with unrelated CLPC/Kconfig PRs (#2549, #1906, #2784), fleet78 config, Mesa 26.3.0-30. This shape keeps the 44-block render pool floor and TA mode as before. The row checks that the shared G17 paths (lazy render-pool placement, reclaim) don't regress a two-cluster part.

production fleet78 this PR
gpubench render.heavy d1 / d4 (us/submit) 276.4 / 127.8 272.3 / 127.8
render.light d1 / d16 116.6 / 24.01 121.5 / 24.83
compute.heavy ALU d4 (GFLOPS) 7504 7511
dEQP-VK 1.4.7.1 smoke (16541 cases) Pass / NotSupported / DeviceLost 8950 / 7590 / 1 8951 / 7590 / 0
G17 LowerVa capacity refusals in the run 1 2
GPU faults / timeouts 0 0

The one DeviceLost on production (dEQP-VK.ray_query.acceleration_structures.empty.traditional_structures.tess_control_shader.gpu_built.index_uint16.no_primitives_bottom, a LowerVa refusal → ENOSPC) predates this PR and is intermittent on this Mac. With the PR the same list passed. 0 failed units, 0 AVC, 0 kernel errors. No regression on the J616s.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 2ba7a68b14a5072cd77fa22736f9f09d89f4a7a6 following @RyanTheTide's J616s (M4 Pro) no-regression row: keeping REQUEST_CHANGES strictly on exact-head J616C (four-cluster M4 Max) §5 evidence.

Confirmed clean

  • Code, policy, attribution & CI (§1–§4, §6): Unchanged since prior review — all 3 commits (ce99a9090138, 0ffbc70ac5c6, 2ba7a68b14a5) carry human author/committer + human Signed-off-by: only, zero AI commit trailers, host-tests SUCCESS, MERGEABLE (CLEAN), non-draft.
  • Two-cluster G17 no-regression row (J616s, T6040 25G83, 2ba7a68b): Great coverage from @RyanTheTide confirming shared G17 paths do not regress two-cluster parts (gpubench render/compute on par, dEQP-VK 1.4.7.1 smoke 8951 pass / 7590 NotSupported / 0 DeviceLost vs 8950 / 7590 / 1 on prod, 0 GPU faults/timeouts, 0 AVC denials, 0 failed units, 0 kernel errors).

Remaining §5 blockers on affected hardware (J616C, 32-core M4 Max)

Because J616s is a two-cluster part, it keeps RENDER_BLOCKS (44), GTP_FIXED_SIZES, NORMAL_MAX_BLOCKS (0x45e), and two-cluster VDM_SEGMENT_CONTROL — none of the four-cluster clustered tiling / 96-block render pool / full TVB block-table / 0x1180+16 KiB metadata paths execute on J616s. As @RyanTheTide also noted on 2ba7a68b, the exact-head J616C glmark2-es2-drm row alone is still too light to exercise those paths:

  1. Heavy-load stability at exact head 2ba7a68b14a5 on J616C: Run workloads exercising >44 render blocks and >0x45e TVB heap growth (Roblox Vulkan 15+ min soak and Minecraft RD32) confirming 0 GPU recoveries on this exact head.
  2. Same-machine J616C A/B + conformance unchanged (§5): Provide a same-machine A/B at 2ba7a68b14a5 vs base 71abea7fa7c7 for the PR's performance claims (vertex-ALU / terrain) and confirm dEQP-VK/GLES pass counts unchanged against base on J616C.
  3. Full J616C regression counts: Confirm 0 SELinux denials (AVC) and 0 new kernel errors alongside FAILED_UNITS=0 on J616C.

Once that exact-head J616C row lands, this is ready for immediate approval.

Assisted-by: LLM (Gemini 4 Neon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants