Repository navigation
drm/asahi: g17: tile clustered on the 32-core M4 Max (T6041 25G83) - #2021
AxxzyWasTaken wants to merge 3 commits into
Conversation
8448d29 to
b83eb43
Compare
|
Added a fourth commit: clustering metadata 1 on four-cluster passes now has a 16 KiB floor ( The floor has been in my daily module since 16 Nov, but not tested on its own. This branch is built ( Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested on my M4 Max as described. |
b83eb43 to
6096eb7
Compare
|
A gentle ping: this is ready for review. Head 6096eb7 merges cleanly into current integration/aurora and CI passes. Clustered tiling has been in the J616C's daily kernel since 16 Nov. #2170, #2171 and #2172 build on it. Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described. |
6096eb7 to
1e62e2c
Compare
|
Blockers: 2.
Nits:
What I checked: every queue gets the larger pool up front, because HW row: J616C (32-core) on this exact head, 25G83: Roblox (Vulkan) 15+ min and Minecraft RD32 with 0 GPU recoveries; the vertex-ALU A/B; dEQP-VK/GLES pass counts unchanged against the base; 0 AVC denials, 0 failed units, 0 new kernel errors. Plus one J614C (40-core) 25G83 boot with a short clustered workload and 0 recoveries, or keep the 40-core configuration on TA mode 1 until that run exists. Assisted-by: LLM (Claude:Opus 5.5) |
|
Rebased onto
Checks: Assisted-by: LLM (Google:Gemini 4 Neon) |
1e62e2c to
0409ba0
Compare
|
Re-reviewed at 0409ba0. The chip/firmware-keyed What is still open on blocker 1 is where the numbers come from:
Also worth stating in the description: on T6041 only 25G83 turns clustered tiling on ( Blocker 2 is unchanged until the exact-head J616C row lands (you say it is queued). Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks for the detailed re-review on
Next step: Keeping
|
|
Thanks for tracing both. That settles blocker 1 for me:
So blocker 1 is resolved once the update you describe lands: the commit says 3 blocks/core is fitted from one 32-core J616C, the Assisted-by: LLM (Claude:Opus 5.5) |
0409ba0 to
6177db6
Compare
|
Pushed
Verified |
|
Re-checked at 6177db6: the update lands as described. Commit 1 says the 3 blocks/core is fitted from one 32-core J616C and describes the reclaim/ Assisted-by: LLM (Claude:Opus 5.5) |
With four-cluster tiling the 32-core M4 Max runs vertex shaders on every cluster at once beside fragment work. Roblox (Vulkan) then hangs the GPU about once a minute: the firmware kills a small depth/stencil render as no-progress (reason 0), with no MMU fault and a nearly empty tiler heap. The same hang showed up on this Mac with the earlier driver until its render pool had 96 blocks. G17 firmware has no grow-request event for the USC render pool (unlike TVB growth) and a published render pool cannot be resized in place, so the pool must be sized up front. Rather than gating a fixed 96-block pool on T6041 and firmware build 25G83, derive the render pool minimum from GPU shape (active cores and cluster count) inside g17::fw::render_rt::required_blocks(), before adding ray-query scratch extras: - One- and two-cluster GPUs keep the 44-block floor (RENDER_BLOCKS), preserving the A18 Pro shared pool window invariant and the M4 Pro's narrow pool window unchanged. - Above two clusters, the base pool scales at 3 blocks (3 MiB) per enabled core, floored at RENDER_BLOCKS (44). Note that 3 blocks/core is an empirical fit from one 32-core J616C measurement (96 / 32), not a hardware-derived bound; the 40-core J614C value (120 blocks) has not been run on hardware yet. Because it keys on GPU shape, every >2-cluster G17 firmware build (including 25F84 and 26B123 on T6041) gets this floor, not only 25G83. - Ray-query scratch is added on top of this base instead of competing with it. Capacity cost in WIDE_LOWER (~3.19 GiB, shared with other lower-window allocations): queue creation allocates no VA or backing; backing is placed lazily on first render (~98 MiB span on J616C, ~122 MiB span on J614C, vs ~46 MiB before), reducing the simultaneously resident pool budget from 58 (MAX_EXECUTION_CONTEXTS) to roughly 33 on J616C and 26 on J614C. When WIDE_LOWER fills, submit enters wait_render_capacity() -> reclaim_render_capacity(), which evicts idle render pools via FreeListRelease and transparently re-arms them on next use. Only when more than ~33 (J616C) / ~26 (J614C) distinct contexts are simultaneously active for longer than QUEUE_WAIT_NS does a new submission fail with EBUSY, leaving existing contexts intact. J616C has run with 96 blocks daily since 16 Nov without hitting the cap. Resulting base render pool blocks (pages = [0, 0]): Part Clusters Cores Base blocks A18 / A18 Pro (T8140) 1 5/6 44 (unchanged) M4 (T8132, uses g16) 1 8/10 44 (if on g17) M4 Pro (T6040) 2 16/20 44 (unchanged) M4 Max 32-core (J616C) 4 32 96 (measured fix) M4 Max 40-core (J614C) 4 40 120 (untested) M5 (T8142, M5 pool path) 1 8/10 44 (unchanged) M5 Pro (T6050, M5 pool) 2 16/20 44 (unchanged) M5 Max (T6050, M5 pool) 4 32/40 96/120 (M5 uses own) M5 Ultra (T6050, M5 pool) 8 64/80 192/240 (M5 uses own) Add KUnit tests (asahi_g17_render_pool) covering these shapes, ray-query scratch addition, and invalid/oversized rejections. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
T6041 ran its 25G83 tiling passes in TA mode 1 (0x1c890 = 1), which tiles non-clustered: one of the four clusters runs the tiling and vertex work of a pass. Vertex shaders reached a quarter of the ALU rate of the GPU (2.95 vs 12.3 TFLOPS), and geometry-bound games were capped by it. Tile clustered instead on the 32-core T6041 (J616C) 25G83 build, while keeping the untested 40-core T6041 parts (J614C, J575C) on TA mode 1 until they have a hardware row. Clustered four-cluster passes receive: - Clustering metadata sized for more than two clusters, with metadata 2 at 0x1180 as macOS 26.6.2 writes on the 32-core M4 Max (metadata 2 at the M2 Pro/Max 0xc00 size allowed the tiler to write past it into metadata 3; metadata 3 and 4 keep their larger sizes). Because 25F84, 40-core 25G83, and 26B123 share RegisterPrograms::V25G83, four-cluster targets on those builds also receive the 0x1180 metadata 2 sizing while keeping TA mode 1 (non-clustered); that oversizing has not been booted on 25F84/26B123. - The four-cluster segment words of cluster_control(): 0x1c890 = 0x280540, 0x1c918 = 4, and v25g83_clustered_free_pages(clusters) = 28 + 16 * clusters free parameter-buffer pages in 0x1c9a8 (0x3c for two clusters, 0x5c for four, matching the macOS 26.6.2 capture). Two clusters keep the M4 Pro capture. - The full block table for Variant::tvb_normal_blocks on 32-core 25G83, because a clustered RD32 Minecraft frame needs about 1600-2200 blocks and never starts at the 0x45e default. 25F84, 40-core 25G83, and 26B123 keep TA mode 1 and the default heap maximum. Every other chip and build keeps its values. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Four-cluster passes size clustering metadata 1 as the M2 Pro/Max path does, 0x44 bytes per block of tiles, with 0x80, the two-cluster M4 Pro size, as the floor. On a 3456x2234 Retina frame (Minecraft RD32) that formula yields 0x100 (32x32 utiles) or 0x180 (16x16), and on 10240x10240 (point_size_10000, 16x16 utiles) it yields 0x1100. Because metadata 1 is the last sub-buffer in the TPC object, neither macOS's allocation size nor the hardware's written high-water mark was visible in the J616C 26.6.2 capture. Give it a conservative 16 KiB floor until measured, document the formula sizes on those frames, and add a KUnit test covering the four-cluster V25G83 metadata layout. (Because this keys on RegisterPrograms::V25G83 && clusters > 2, 25F84 and 26B123 on T6041 also get this conservative floor.) The per-block size stays the M2 Pro/Max formula; the floor is our own choice, not a macOS value. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
6177db6 to
2ba7a68
Compare
|
Rebased onto |
|
J616C (M4 Max 16c/32g) exact-head boot & 3D render verification at
|
RyanTheTide
left a comment
There was a problem hiding this comment.
Exact-head J616C row at 2ba7a68: the 32-core 25G83 split is selected and renders run. With the fitted floor documented and the 40-core parts left on TA mode 1, our blocker is resolved. Approving.
Assisted-by: LLM (Claude:Opus 5.5)
DjDeveloperr
left a comment
There was a problem hiding this comment.
Full review of 2ba7a68b14a5072cd77fa22736f9f09d89f4a7a6 against pre-pr-checklist.md (§1–§6): REQUEST_CHANGES (code clean; held strictly on §5 evidence).
Code & checklist audit (§1–§4, §6) — clean
- §1 No model gating: Render pool floor (
g17/fw/render_rt.rs::base_blocks(active_cores, clusters)), four-cluster metadata sizing (g17/buffer/target.rs,clusters > 2), and segment words (v25g83_clustered_free_pages(clusters) = 28 + 16 * clustersfed intocluster_control()) all derive cleanly from GPU shape with checked arithmetic and KUnit coverage (asahi_g17_render_pool,four_cluster_v25g83_metadata_layout,four_clusters_take_the_four_cluster_segment_words). SplittingBUILD_25G83_32COREinhw/t6041.rs/hw/t6041_25g83.rskeeps untested 40-core T6041 (J614C/J575C),25F84, and26B123onta_mode_1 = trueandNORMAL_MAX_BLOCKS, matching@RyanTheTide's agreed scope. - §2 Production-ready / §3 Safety: Works on default command line with no module params or magic waits;
normal_blocksis bounds-checked againstgeometry.max_blocks()ing17/buffer.rs; the conservative 16 KiBMETA1_MIN_SIZEfloor only over-allocates safely;WIDE_LOWERreclaim/EBUSYbehavior is accurately traced and documented. - §4 Clean-room & attribution / §6 Naming: 3 clean bisectable commits (
ce99a9090138,0ffbc70ac5c6,2ba7a68b14a5), all human author/committer + humanSigned-off-by:only, zero AI trailers in commits; PR body carriesAssisted-by:; Borealis/Aurora naming clean. - CI:
host-testsSUCCESS;MERGEABLE(CLEAN), non-draft.
Remaining blocker before approval (§5 Evidence)
The exact-head J616C boot row at 2ba7a68b14a5 (7.3.0-j616c-pr2021+) confirms BUILD_25G83_32CORE probes cleanly and glmark2-es2-drm at 3456x2234 passes (terrain 324 FPS, Score 2479, 0 GPU faults, FAILED_UNITS=0). However, glmark2-es2-drm alone does not yet close §5 (and @RyanTheTide's gate) for the specific fixes and performance claims in this PR:
- Heavy-load stability at exact head
2ba7a68b14a5: Run a workload that actually exercises the 96-block render pool and>0x45eTVB heap growth (e.g., Roblox Vulkan 15+ min soak and Minecraft RD32) confirming0GPU recoveries on this exact head (the PR body numbers are from an older daily tree with additional patches). - Same-machine A/B + conformance unchanged: For the PR's performance claims, provide a same-machine A/B at
2ba7a68b14a5vs base71abea7fa7c7(e.g., vertex-ALU / terrain A/B) and confirm conformance (dEQP-VK/GLES pass counts unchanged against base). - Full regression check: Confirm
0 SELinux denialsand0 new kernel errorsalongsideFAILED_UNITS=0.
(Non-blocking nit: update the stale 6177db61a671 SHA references in the PR description once the final 2ba7a68b14a5 row is ready.)
Assisted-by: LLM (Gemini 4 Neon)
|
To add to the approval: it covered the code and the exact-head boot. The glmark2 run shows the 32-core image is selected and renders, but it is too light to fill the old 44-block pool, so it doesn't exercise the hang the 96-block floor fixes. Before merge, Roblox (Vulkan) for 15+ minutes and Minecraft RD32 at 2ba7a68 with 0 GPU recoveries are still wanted, plus dEQP pass counts against base Assisted-by: LLM (Claude:Opus 5.5) |
|
J616s (MacBook Pro 16" M4 Pro, T6040, 25G83, two clusters) no-regression row at 2ba7a68, merged on integration/aurora a463112 with unrelated CLPC/Kconfig PRs (#2549, #1906, #2784), fleet78 config, Mesa 26.3.0-30. This shape keeps the 44-block render pool floor and TA mode as before. The row checks that the shared G17 paths (lazy render-pool placement, reclaim) don't regress a two-cluster part.
The one DeviceLost on production ( Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Re-review at 2ba7a68b14a5072cd77fa22736f9f09d89f4a7a6 following @RyanTheTide's J616s (M4 Pro) no-regression row: keeping REQUEST_CHANGES strictly on exact-head J616C (four-cluster M4 Max) §5 evidence.
Confirmed clean
- Code, policy, attribution & CI (§1–§4, §6): Unchanged since prior review — all 3 commits (
ce99a9090138,0ffbc70ac5c6,2ba7a68b14a5) carry human author/committer + humanSigned-off-by:only, zero AI commit trailers,host-testsSUCCESS,MERGEABLE(CLEAN), non-draft. - Two-cluster G17 no-regression row (J616s, T6040 25G83,
2ba7a68b): Great coverage from@RyanTheTideconfirming shared G17 paths do not regress two-cluster parts (gpubenchrender/compute on par,dEQP-VK 1.4.7.1smoke8951 pass / 7590 NotSupported / 0 DeviceLostvs8950 / 7590 / 1on prod,0GPU faults/timeouts,0AVC denials,0failed units,0kernel errors).
Remaining §5 blockers on affected hardware (J616C, 32-core M4 Max)
Because J616s is a two-cluster part, it keeps RENDER_BLOCKS (44), GTP_FIXED_SIZES, NORMAL_MAX_BLOCKS (0x45e), and two-cluster VDM_SEGMENT_CONTROL — none of the four-cluster clustered tiling / 96-block render pool / full TVB block-table / 0x1180+16 KiB metadata paths execute on J616s. As @RyanTheTide also noted on 2ba7a68b, the exact-head J616C glmark2-es2-drm row alone is still too light to exercise those paths:
- Heavy-load stability at exact head
2ba7a68b14a5on J616C: Run workloads exercising>44render blocks and>0x45eTVB heap growth (Roblox Vulkan 15+ min soak and Minecraft RD32) confirming0GPU recoveries on this exact head. - Same-machine J616C A/B + conformance unchanged (§5): Provide a same-machine A/B at
2ba7a68b14a5vs base71abea7fa7c7for the PR's performance claims (vertex-ALU / terrain) and confirm dEQP-VK/GLES pass counts unchanged against base on J616C. - Full J616C regression counts: Confirm
0 SELinux denials(AVC) and0 new kernel errorsalongsideFAILED_UNITS=0on J616C.
Once that exact-head J616C row lands, this is ready for immediate approval.
Assisted-by: LLM (Gemini 4 Neon)
T6041 runs its 25G83 tiling passes in TA mode 1 (
0x1c890 = 1), which tiles non-clustered: one of the four clusters does the tiling and vertex work of every pass. Vertex shaders reach a quarter of the GPU's ALU rate (2.95 vs 12.3 TFLOPS on a 32-core part), so geometry-heavy games are capped by it.TA mode 1 came from my report in #525: it stopped reason-0 hangs on the M4 Max. Those hangs had more than one cause. This series fixes them instead of avoiding clustered tiling on the 32-core M4 Max (J616C).
Commits
drm/asahi: g17: scale the render pool floor from GPU shape:g17::fw::render_rt::required_blocks().RENDER_BLOCKS), leaving A18 Pro and M4 Pro unchanged. Above two clusters, the base scales at 3 blocks (3 MiB) per enabled core, floored at 44 (empirically fitted from one 32-core J616C measurement,96 / 32; 40-core J614C = 120 blocks is untested). Ray-query scratch is added on top.WIDE_LOWER(~3.19 GiB) capacity: queue creation allocates no VA/backing; backing is placed lazily on first render (~98 MiB span on J616C), reducing simultaneously resident pools from 58 to ~33 on J616C (~26 on J614C). WhenWIDE_LOWERfills,wait_render_capacity()->reclaim_render_capacity()evicts idle pools viaFreeListReleaseand transparently re-arms them on next use; only >33 (J616C) / >26 (J614C) simultaneously active render contexts held pastQUEUE_WAIT_NSfail submit withEBUSY.ta_mode_1 = trueand only inherit the larger render pool floor.drm/asahi: g17: tile clustered on the 32-core M4 Max (T6041 25G83):BUILD_25G83_32COREso clustered tiling (ta_mode_1 = falseand full-tabletvb_normal_blocks = NORMAL_MAX_BLOCKS_FOUR_CLUSTERS) is enabled strictly on the tested 32-core T6041 25G83 configuration (J616C). 40-core T6041 parts (J614C, J575C), 25F84, and 26B123 stay onta_mode_1 = trueand the defaultNORMAL_MAX_BLOCKSheap maximum until tested on hardware.0x1180metadata 2 per macOS 26.6.2 capture; note 25F84/40-core 25G83/26B123 shareRegisterPrograms::V25G83and inherit the larger metadata sizing while remaining in TA mode 1).fw::m5::target::cluster_control()(0x280540,4) andv25g83_clustered_free_pages(clusters) = 28 + 16 * clusters(0x3cfor 2 clusters,0x5cfor 4 clusters).drm/asahi: g17: a 16 KiB floor for four-cluster metadata 1:META1_MIN_SIZE = 0x4000) for four-cluster metadata 1 (formula yields0x100–0x180on 3456x2234 Retina and0x1100on 10240x10240) with a KUnit layout test.Testing
Host checks on
2ba7a68b14a5(rebased ontointegration/aurora71abea7fa7c7):tools/asahi/run-host-tests.py --only g17: 90/90 passed (CIhost-testsSUCCESS).scripts/checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks across all 3 commits (ce99a9090138,0ffbc70ac5c6,2ba7a68b14a5).Exact-head J616C (32-core M4 Max, 25G83) boot & light 3D check at
2ba7a68b14a5(7.3.0-j616c-pr2021+):BUILD_25G83_32COREprobes cleanly;glmark2-es2-drmat3456x2234(zink/BorealisVulkan 1.4) Score 2479 (build3622 FPS,shading=phong3495 FPS,terrain324 FPS, exit 0); 0 GPU faults, 0 timeouts/resets,FAILED_UNITS=0.@RyanTheTideAPPROVEDon2ba7a68b14a5(2026-11-22T10:30:51Z).@RyanTheTide(11:54Z): the approval covers code + exact-head boot;glmark2is too light to fill the old 44-block pool.@DjDeveloperrCHANGES_REQUESTED(10:35Z, re-affirmed14:39Z) on §5 evidence (below).J616s (M4 Pro, T6040, 25G83, two clusters) non-regression row:
2ba7a68b14a5merged onintegration/auroraa4631125with unrelated#2549/#1906/#2784(posted by@RyanTheTide):gpubench:render.heavyd1/d4272.3 / 127.8vs276.4 / 127.8us/submit on prod;compute.heavyALU d47511vs7504GFLOPS;render.lightd1/d16121.5 / 24.83vs116.6 / 24.01on prod; G17 LowerVa capacity refusals2vs1.dEQP-VK 1.4.7.1smoke (16541 cases):8951 Pass / 7590 NotSupported / 0 DeviceLost(vs8950 / 7590 / 1on prod).Earlier daily-tree measurements on J616C (25G83, Hyprland) with the same clustered tiling and 96-block pool (that tree also carries other changes: zeroed clustering pointers, a TA PTC cap, unclustering under 256 tiles; off-screen, identical images in both modes):
Minecraft (Fabulously Optimized, Sodium, OpenGL) at RD32 went from 105 to ~300 fps standing; Roblox in Vulkan ran 11+ minutes with 0 recoveries (vs 1-2 minute hangs at 44 blocks).
Remaining §5 scope on exact head
2ba7a68b14a5(J616C): heavy-load stability soak exercising >44 render blocks and >0x45eTVB growth (Roblox Vulkan 15+ min + Minecraft RD32 with 0 GPU recoveries), same-machine A/B vs base71abea7fa7c7(vertex-ALU/terrain + dEQP-VK/GLES pass counts unchanged), and explicit0 SELinux denials+0 new kernel errorscounts.Not tested on hardware: 40-core M4 Max (J614C/J575C — kept on
ta_mode_1 = true), 25F84, and 26B123.Assisted-by: LLM (Anthropic:Claude Opus 5.5, Google:Gemini 4 Neon)