Repository navigation
drm/asahi: g18: grow the parameter buffer when the firmware asks (10 s GPU stalls on the M6 Mac mini) - #2242
Conversation
|
Pushed a second commit, Why this shape, from runs on the J873g:
J873g rows (asyncbench, headless, run on the M6 tree with the same constants; this branch compiles with them):
The cost is memory: the PB stays with the session, so after the 5M scene 572 MiB stay allocated until reboot (the same holds for the 127 MiB before). If that's too much, a lower cap would mean lowering the ring the firmware is given ( |
|
Blockers: 1. I checked the grow path end to end. The event 6 fields are read in initdata/mod.rs and answered after the drain.
If an LLM helped with the series, the PR body's last line should be HW row: J873, this head on the integration tip. Run the 5M-triangle headless scene, then a 20000-triangle row, and record Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review of f8b3b9e: No blockers. Earlier points:
HW row: J873, this head on the integration tip, default command line: the 5M-triangle headless scene and two clients alternating 600000-triangle frames, with Assisted-by: LLM (Claude:Opus 5.5) |
f8b3b9e to
400ec54
Compare
|
@RyanTheTide rows on the J873g, plus one fix they turned up. Head Fix found by the refusal row: the limits were never set on G18. memacct's Setup:
Boot 1 (admission, limits at their default, before the probe fix):
Boot 2 (
So a refused growth costs every client the stall, about 10 s per refused frame, which is how it already behaved at the block limit. Only the client that asked loses work. Shrinking: still not done. The commit message says why: grown blocks are not given back, because there is no known message that makes the firmware drop them. Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review of 400ec54: No blockers. Since f8b3b9e the third commit gained one change: drivers/gpu/drm/asahi/g18/probe.rs:69 now calls Earlier points:
HW row: J873, this exact head on the integration tip rather than the port onto the desktop tree, default command line: the 5M-triangle scene plus a 20000-triangle client, with Assisted-by: LLM (Claude:Opus 5.5) |
400ec54 to
e8f4481
Compare
|
@RyanTheTide thanks. Head
Assisted-by: LLM (Claude:Opus 5.5) |
|
@RyanTheTide here are the two rows I promised, run on the port of head Default limits (device 10346 MiB, per client 7721 MiB):
This matches the rows on Assisted-by: LLM (Claude:Opus 5.5) |
RyanTheTide
left a comment
There was a problem hiding this comment.
Re-reviewed at e8f4481: both points are fixed. grow() refuses with no client (client.ok_or(ENOMEM)?) before anything is allocated, so every grown block is admitted, and the commit message now says the limits apply to every G18 GEM allocation. The rows on the desktop-tree port are accepted for the reason given (the tip can't run the G18 firmware without #1652).
One follow-up from your own limit row, not blocking this PR: a refused growth "fails only that render", but the render only fails when the 10 s watchdog fires, and each of those stalls holds the GPU, so the other client averaged 65 fps instead of ~490 over the test. A client at its limit should have its render failed as soon as the growth is refused rather than left for the watchdog. Worth a separate change; please note it in the PR body so it isn't lost.
Nit: the new Rust in graphics_application.rs isn't rustfmt-formatted (several statements per line); rustfmt on the file would make the next review easier.
Approve.
Assisted-by: LLM (Claude:Opus 5.5)
|
@RyanTheTide thanks.
Assisted-by: LLM (Claude:Opus 5.5) |
|
@RyanTheTide the fail-fast follow-up is #2758, stacked on this PR. A refused growth now ends its render within milliseconds: the 20k client next to the refused scene ran at 494 fps instead of 65, with 0 watchdogs. The default-limit row is unchanged. Assisted-by: LLM (Claude:Opus 5.5) |
The G18 render path backs the tiler's parameter buffer with 8 blocks (4 MiB) and never grows it. When a scene needs more, the firmware posts event type 6, a TVB grow request in the M5 TvbGrowEventRecord layout, and the tiler waits for the host. Nothing answered, so the frame stalled until the 10 s job watchdog requested a recovery, and the client's submit failed with EIO. Zooming into Google Maps in Firefox or Chromium on the J873 (M6 Mac mini) hits this. Answer the request as the M5 driver does for a firmware-requested growth (Manager::commit_growth without a descriptor): back 16 more blocks as one object, write their first page numbers into the block table at the ring slots after the total cursor, advance total and committed, and reply with a TvbGrowReply (device-control opcode 8, grew = 1) plus the 0x84/0x11 control doorbell. The firmware builds the page-list entries itself. A "not grown" reply leaves the tiler stalled, so it is only sent when growth fails or the buffer is at its limit. The PB grows to at most 254 blocks (127 MiB): the block table before the shared header holds 256 ring slots. The graphics-pb VA window grows from 16 to 128 MiB for that; its biased half stays inside the COMPACT_BIAS + [32 MiB, 2 GiB) that Mesa leaves to the kernel. The new blocks are homed in the PB's storage VM like the initial ones and are aliased into the running client's VM; a later client maps every block. Two per-frame checks assumed the PB and the control ring never change: the PB row's page count may now grow up to the backed blocks, and the outer-ring publication expects the primary control ring past the grow replies published since boot. A headless GLES test drawing N triangles of about 200 px into 2560x1440 stalled at 60000 triangles before. Now 150000 and 600000 triangles grow the PB on their first frame and render in 10 and 44 ms afterwards; the images match llvmpipe apart from single edge pixels, as small scenes do. One million triangles reach the limit and still stall. Two clients alternating 600000-triangle frames render all frames. Tiny frames of two alternating clients take the same time with an 8-block and a 254-block PB (2.66-2.88 s vs 2.71-2.77 s for 2x2000 frames). Zooming into Google Maps in Firefox now grows the PB to 40 blocks (20 MiB) without a stall. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
…e is given The PB's ring is described to the firmware with PB_CAPACITY (1650) slots, but growth stopped at 254 blocks, because the block table in front of the shared header held 256 entries. A scene that needs more than 127 MiB of PB, about 1.1M triangles of the headless test, made the firmware ask again after a refused grow and then wait until the job watchdog: a 10 s stall and EIO, as before the growth support. Telling the firmware a lower maximum (state +0x7c) changed nothing; it does not render partially there. Make room for the whole ring instead. The block table stays at the start of the PM graph and holds PB_CAPACITY entries; the shared header, the counter and the scene records move up behind it (0x3400, 0x3500, 0x4000). Placing the table behind the scene records, at +0x8000 of a larger graph, faulted the first frame. PB_MAX_BLOCKS follows PB_CAPACITY, and the graphics-pb window grows to 832 MiB to hold that many blocks, still inside the COMPACT_BIAS + [32 MiB, 2 GiB) range Mesa leaves to the kernel. The job's ring capacity now comes from PB_CAPACITY rather than a second 1650. The PB is retained with the session, as before: a scene that grows it to N blocks keeps N * 512 KiB allocated. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
A PB growth was a kernel object: the device and per-client memory limits never applied to it, and its bytes were counted as nobody's. The PB lives with the session and is shared by later clients, so one client drawing a large scene could pin up to 1649 blocks (about 824 MiB) until reboot. Admit each growth against the requesting client's limit and the device limit first, as g17 admits its pools, and keep the blocks as that client's held memory (a user charge on a kernel object). A refused admission fails the growth with ENOMEM: the firmware gets "not grown", and only the render that asked fails, as at the block limit. A growth with no client to charge is refused the same way, so nothing grows unadmitted. The G18 driver's probe never configured the limits (memacct's configure() ran only in the other generations' probe), so on the J873 both read 0, "none", and nothing could be refused. Configure them there too. That applies the limits to every G18 GEM allocation, not only to PB growth: at the limit, GEM_CREATE fails with ENOMEM as it does on the other generations. Grown blocks are not given back. The firmware's ring is only ever extended: it asks with event 6, and the host answers by writing the new table slots and advancing total and committed. No message is known that makes the firmware drop blocks it was given. Lowering total behind its back would take away slots it may still hold, so until such a message is found the grown blocks stay with the session, held by the client that grew them. The limits bound how much that can be. The ring keeps one slot empty to tell a full ring from an empty one, so PB_MAX_BLOCKS is PB_CAPACITY - 1. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
The parameter buffer growth code added to this file in the previous commits kept several statements per line, like the rest of the file. Format the whole file with rustfmt (the tree's .rustfmt.toml), so the next change to it can be read and reviewed line by line. No functional change; rustfmt --check passes and the driver builds with the same warnings as before. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
4c7d852 to
c48d68d
Compare
|
Rebased onto integration/aurora
The review was at Assisted-by: LLM (Claude:Opus 5.5) |
…rol ring reply_tvb_grow() placed the TvbGrowReply at the 26A428 control ring offset, a fixed constant. The shared-memory layout is per firmware build since the InitData profiles, and the IdlePowerOff record next to it already takes the ring from self.layouts. With 26B123, which the J873g loads today, the reply went to the wrong place: the tiler waited for it and every render that needed a second parameter buffer block stalled until the watchdog. Take the ring from the admitted profile, as IdlePowerOff does. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
|
@RyanTheTide @DjDeveloperr a new commit on this head, now Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review of 83ccd92. The four earlier commits are unchanged from c48d68d. The new commit replaces the last fixed 26A428 offset on this path: Assisted-by: LLM (Claude:Opus 5.5) |
|
@RyanTheTide the J873g before/after rows at Assisted-by: LLM (Claude:Opus 5.5) |
83ccd92
into
aurora-silicon:integration/aurora
Supersedes #2241, whose fixed 30-block parameter buffer was not enough for Google Maps.
On the J873 (M6 Mac mini), zooming into Google Maps in Firefox or Chromium stalled the GPU for 10 s, and the browser's submit then failed with EIO:
Event type 6 is the firmware's TVB grow request. Its record has the M5
TvbGrowEventRecordlayout: VM slot, buffer slot, counter, subpipe, and the halt count, which is the recovery epoch. The G18 parameter buffer had 8 blocks (4 MiB) and nothing answered the request.This PR answers it the way the M5 driver handles a firmware-requested growth (
Manager::commit_growthwithout a descriptor):total, then advancetotalandcommitted.grew = 1) and the 0x84/0x11 control doorbell. The firmware builds the page-list entries itself.The PB grows up to the ring the firmware is given, PB_CAPACITY (1650) slots less the one kept empty: 1648 blocks, 824 MiB (second commit). Before, growth stopped at 254 blocks (127 MiB), because the block table in front of the shared header held 256 entries; a scene needing more (about 1.1M triangles) stalled 10 s and failed with EIO. The table stays at the start of the PM graph with room for every ring slot, and the shared header, counter and scene records move up behind it. Placing the table behind the scene records instead faulted the first frame. Telling the firmware a lower maximum (state +0x7c) did not help: the firmware does not render partially when a grow is refused. To hold the blocks, the graphics-pb window grows from 16 MiB to 832 MiB. Its biased half stays inside the COMPACT_BIAS + [32 MiB, 2 GiB) range that Mesa leaves to the kernel. The PB is retained with the session, so a scene that grows it to N blocks keeps N × 512 KiB allocated.
Two per-frame checks assumed the PB never changes:
A
grew = 0reply alone leaves the tiler stalled, so it is only sent when growth fails or the PB is at its limit.Tests on the M6 (J873 #10 stack). The port to the integration tip compiles; integration has no
layoutsfield, so the control ring offset comes fromG18G_26A428_CONTROL_RING_BUNDLE_OFFSETS.The integration tip cannot run the G18 GPU on this M6: its macOS 27.2 (26B123) firmware needs #1652, and without it the probe stops at stage 10 ("needs /chosen aurora,g18g-gpu-firmware-profile"). So the hardware rows, including the admission rows in the comments, ran on the port of these three commits onto the J873 desktop tree.
Headless GLES test, N triangles of about 200 px into 2560x1440:
The frame that grows, including three grows within one frame, matches llvmpipe except for single edge pixels. Small scenes show the same.
Two clients alternating 600000-triangle frames render all 40 frames.
Tiny frames of two alternating clients take as long with a 254-block PB as with 8 blocks: 2.66-2.88 s against 2.71-2.77 s for 2x2000 frames. That needed one object per grow step; per-block aliases cost about 0.3 ms per client switch.
Zooming into Google Maps in Firefox grows the PB to 40 blocks (20 MiB) and no longer stalls.
Follow-up (separate change, from the limit row): a refused growth fails only the requesting client's render, but not until the 10 s job watchdog fires, and each such stall holds the GPU for every client (the other client averaged 65 fps instead of ~490 during the limit row). A client at its limit should have its render failed as soon as the growth is refused. Done in #2758, which is stacked on this PR.
Not part of this PR: Maps still draws wrong colours on G18G, with the original module too. Mesa refuses translucent 4x MSAA draws ("draw exceeds the native triangle profile: multisampled"), which aurora-silicon/mesa#407 works around.
Assisted-by: LLM (Claude:Opus 5.5)