Repository navigation
drm/asahi: g17: one zeroed clustering-metadata copy per in-flight render - #2170
AxxzyWasTaken wants to merge 5 commits into
Conversation
|
The production boot stub pins at macOS 27.2 (26B123), so please retarget this to 27.2. The clustering metadata layout here comes from the 26.6.2 (25G83) GPU firmware; please take it from the T6041 GPU image the 27.2 stub loads. |
|
Thanks. As far as I can see, 26.6.2 is still what M4 Max installs get today, so I'd like to keep this on 25G83 for now and port it once there's a 27.2 base for the T6041:
When a T6041 board moves to 26B123 in the installer, I'll port this and the four-cluster series (#2021, #2171, #2172) onto the 27.2 base and re-test on my J616c. If you'd rather hold these until then, that's fine too. Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); facts checked against the repos named above. |
|
I found an overflow in this PR's metadata-copy commit and pushed a fix as a new commit on top ("drm/asahi: g17: hold the last tile map in the shared TPC object"). The earlier commits are unchanged.
Testing on my J616C (32-core M4 Max, 25G83):
#2171 and #2172 aren't affected: they sit on #2021 alone, where the TPC object holds the whole parameter block and is sized to its end. Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described. |
2ee4e1a to
8e7877b
Compare
8e7877b to
17a8d6f
Compare
T6041 ran its 25G83 tiling passes in TA mode 1 (0x1c890 = 1), which tiles non-clustered: one of the four clusters runs the tiling and vertex work of a pass. Vertex shaders reached a quarter of the ALU rate of the GPU (2.95 vs 12.3 TFLOPS), and geometry-bound games were capped by it. Tile clustered instead, with what a four-cluster pass needs: - Clustering metadata sized for more than two clusters, as the M2 Pro/Max path sizes it (meta1 per block of tiles, meta2-4 fixed). - The four-cluster segment words of cluster_control(): 0x1c890 = 0x280540, 0x1c918 = 4, and 32 + 16 x clusters = 0x60 free parameter buffer pages in 0x1c9a8. The two-cluster capture (0x200440 / 4 / 0x3c) stays for two clusters. - A per-build tiler heap normal maximum: the four-cluster 25G83 build may use every slot of the block table. A clustered RD32 Minecraft frame needs about 1600-2200 blocks; at the 0x45e normal maximum the pass never starts and the firmware recovers with reason 0. 25F84 keeps TA mode 1, and every other build keeps its values. Measured on the J616C (32-core M4 Max, 25G83): vertex-shader ALU 2.95 -> 10-11.5 TFLOPS; Sodium-style terrain 1.93 -> 0.68 ms per frame with identical images; Minecraft RD32 105 -> ~300 fps, with no GPU recoveries over several sessions, heavy and concurrent stress included. G17 host tests 31/31. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
With four-cluster tiling the 32-core M4 Max runs vertex shaders on every cluster at once. Roblox (Vulkan) then hangs the GPU about once a minute: the firmware kills a small depth/stencil render as no-progress (reason 0), with no MMU fault and a nearly empty tiler heap. The same hang showed up on this Mac with the earlier driver until its render pool had 96 blocks. Give firmware variants a smallest render pool, 44 blocks as before, and 96 for T6041 25G83. Owners grow to it on their first render, as they grow for ray-query scratch. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…them macOS 26.6.2 on the 32-core M4 Max (25G83), captured under a hypervisor while running a game, spaces a clustered pass's tiler metadata 2, 3 and 4 0x1180, 0x1400 and 0x1600 apart and writes 0x5c free parameter-buffer pages. Metadata 2 was 0xc00, the M2 Pro/Max size, so the tiler could write past it into metadata 3. Use 0x1180 and 0x5c; metadata 3 and 4 keep their larger sizes. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Four-cluster passes size clustering metadata 1 as the M2 Pro/Max path does, 0x44 bytes per block of tiles, with 0x80, the two-cluster M4 Pro size, as the floor. Metadata 1 is the last sub-buffer, and the macOS 26.6.2 capture on the M4 Max does not show its size. Give it a 16 KiB floor instead. The per-block size stays the M2 Pro/Max formula; the floor is our own choice, not a macOS value. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
|
Looks good; approve once this exact head has booted on the M4 Max and on a two-cluster M4 Pro.
Checked and correct: the shared/per-render split in gtp_va() (target.rs:364). Offsets below HW row: this head on #2021, booted as-is (not a daily tree with other work). On the J616C (M4 Max, 25G83): Minecraft with the shader pack near 3456x2234 for 10 min and SCP:SL for 10 min, 0 GPU recoveries. On a T6040 (J616s or J614s): the same pair, or the G17 Vulkan CTS render subset, with 0 recoveries. Also 0 AVC, 0 failed units and no new kernel errors on both. Assisted-by: LLM (Claude:Opus 5.5) |
A render target keeps one TPC object, with the 25G83 clustering metadata (GTP 4-7) after it, for all of its in-flight renders, and never clears the metadata; only the per-slot records are zeroed. Clustered tiling can then read metadata left by another render, or written by one still in flight. Give each render slot its own copy of the clustering metadata, as the records have, and zero it before the render uses it, as G15 gives each clustered pass a zeroed buffer. Only the metadata is copied: the TPC and the tile maps stay shared among the target's renders. A copy of the whole object per slot took 47 MiB for a 1728x1117 target with 32 renders in flight, and SCP: Secret Laboratory filled the 320 MiB target window within minutes (target growth failed with ENOSPC until the render submit gave up with EBUSY). The metadata copy is 0x70000 bytes at 1728x1117 (metadata 4, GTP 6, has room for 64 partial renders) and starts at a page-aligned base, so every sub-buffer keeps its alignment. M5 is unchanged. The shared TPC object, which holds the TPC and GTP 1-3, is sized to GTP 4 rounded up to a page, not down. The tile maps (GTP 1) run up to GTP 4 itself, so when GTP 4 is not page aligned their end would lie past the object. With 32x32 utiles at 3456x2234 the TPC is 0x46000 bytes and the last 8 KiB of the tile maps would fall into the unmapped guard page after it; that hung the GPU (reason 3, no MMU fault reported) within about 35 seconds of Minecraft with a shader pack in a window near that size. gtp_va() checks the bound. The TPC itself has to stay shared by the target's renders: two kernels that also gave each render its own copy of the TPC hung the GPU in SCP: Secret Laboratory (reason 0). The tests check that every sub-buffer lies in its object with its alignment across sizes, utile shapes, layer counts and cluster counts, including 3456x2234 with 32x32 utiles. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
17a8d6f to
6d2f59d
Compare
Depends on #2021. This branch sits on #2021's branch (
1e62e2c25c38), so GitHub shows #2021's four commits first; only the last commit belongs to this PR. I'll rebase it once #2021 lands.A render target keeps one TPC object for all of its in-flight renders. The 25G83 clustering metadata (GTP 4-7) sits at its end and is never cleared; only the per-slot records are zeroed. Clustered tiling can then read metadata left by another render, or written by one still in flight. G15 gives each clustered pass a zeroed buffer.
This gives each render slot its own copy of the clustering metadata, zeroed before the render uses it. The TPC and the tile maps stay in one object shared by the target's renders.
My first version of this in my tree copied the whole TPC object per slot. With 32 renders in flight that took 47 MiB for a 1728x1117 target, and grown targets keep their old copies until those renders retire. SCP: Secret Laboratory filled the client's 320 MiB target window within minutes: target growth failed with ENOSPC, then the render submit gave up with EBUSY and the game saw a lost device. This version copies only the metadata:
0x70000bytes (448 KiB) per render at that size, 14 MiB for 32 renders, from a page-aligned base so every sub-buffer keeps its alignment.The shared TPC object (holding the TPC and GTP 1-3) is sized to GTP 4 rounded up to a page, not down, so the tile maps never spill past the object when GTP 4 is not page-aligned (for example 32x32 utiles at 3456x2234). Unit tests check that every sub-buffer lies in its object with its alignment across sizes, utile shapes, layer counts and cluster counts. (The earlier separate overflow-fix commit has been folded into this single commit for bisectability.)
The TPC has to stay shared. Two kernels that gave each render its own TPC copy as well, one of them with the overflow fixed, hung the GPU in SCP: Secret Laboratory (reason 0).
The two-cluster M4 Pro (T6040) uses the same 25G83 layout, so it gets the per-render copy too. I couldn't test it there. M5 is unchanged.
Testing
All on a J616C (32-core M4 Max), 25G83, under Hyprland, as the daily module in my tree on top of other work that isn't in this PR:
6d2f59d650ff):aurora.ocompiles clean withW=1andCONFIG_KUNIT=y, andtools/asahi/run-host-tests.py --only g17passes 90/90. Not booted as-is yet.It conflicts in one line of
queue/render.rs(thegtp_vacall) with the PR that tiles small passes unclustered. Whichever lands second needs a one-line rebase.Assisted-by: LLM (Google:Gemini 4 Neon)