Skip to content

drm/asahi: g17: cut allocations and copies in the submit path - #2228

Open
jgoyvaerts wants to merge 3 commits into
aurora-silicon:integration/aurorafrom
jgoyvaerts:m4pro/g17-submit-path-allocs
Open

jgoyvaerts wants to merge 3 commits into
aurora-silicon:integration/aurorafrom
jgoyvaerts:m4pro/g17-submit-path-allocs

Conversation

@jgoyvaerts

@jgoyvaerts jgoyvaerts commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Three changes that remove per-submit allocations and copies on the G17 submit path. They don't change what is submitted or where it runs.

  • Reuse each queue's publication batch. Every publication allocated a DeferredBatch with room for two rings of completions: 64 entries, about 3.5 KB through kvmalloc. It freed the batch when the publication ended, so the direct compute and render paths did a kvmalloc and a kvfree in every submit ioctl.
    • The backend now keeps the batch of its last finished publication and lends it to the next one.
    • A batch that still holds work is dropped as before.
    • Concurrent publishers of one queue each get their own batch, and the first one back is kept. A new batch is allocated without the spare's lock held.
  • Keep the submit ioctl's command vectors in the queue. prepare_submission collected the validated commands and their VM pins in two vectors allocated and freed by every ioctl. The queue now keeps both, empty between ioctls. Whatever an error leaves unsubmitted is released at the same point as before.
  • Build each packet in its allocation. Packet::new built the packet, with its 1.3 KB validated command, on the stack and then moved it into the Arc. The compiler kept three stack copies of the command for that. The packet is now allocated first and initialised in place, so the command is copied once.

Tested on the J616s (M4 Pro, 48 GiB, 25G83)

These have been in our daily kernel since 11-18. Before and after are separate boots, compared with gpubench and kmem tracing, with 0 GPU recoveries. The one result that holds across boots is the allocation count.

Allocations (kmem trace, compute.light pinned at depth 1, 139 submits). Per submit, 12 before (11 kmalloc + 1 kmem_cache_alloc) and 9.3 after. The per-ioctl temporaries are gone: the DeferredBatch publication (4096 B), and prepare_submission's pending vector (2048 B) plus its VM-guard vector (192 B). What is left is objects that outlive the ioctl, plus a drm_exec in 2% of submits.

Submit ioctl CPU time, compute.light, in us at depth 1 / 8 / 64. Only rows with the same P-cluster frequency floor compare:

Run Before After
Hardware-minimum floors (1.26 GHz), unpinned 11.0 / - / 4.92 16.29 / - / 6.00
P floors 4.5 GHz, unpinned 9.50 / - / - 10.17 / 8.58 / 4.67
P floors 4.5 GHz, pinned to cpu5 - 9.75 / 7.54 / 4.25
P floors 2.35 GHz, pinned to cpu5 8.75 / 7.58 / 7.71 13.31 / 8.04 / 4.58
P floors 2.35 GHz, pinned to cpu11 10.38 / 8.21 / 6.62 13.93 / 9.12 / 6.81

So the change in submit CPU time is not measurable across boots: depth 64 is lower on cpu5, level on cpu11 and higher unpinned. The differences follow each boot's cluster load, not this code. The same floor gave 8.75 and 13.31 us at depth 1 on the two boots. Depth 1 is set by core type, cluster and frequency, which #2217 and #2222 address. What these commits measurably do is remove three per-submit allocations.

Not an A/B for these commits:

On this branch: KUnit 697 of 698 passed, one skipped (needs CONFIG_UCLAMP_TASK). That includes the new a_publication_batch_comes_back_only_when_idle. tools/asahi/run-host-tests.py --only g17 passed 90/90.

Not done

  • Not tested on the M4, the M4 Max or the M5.
  • A kept vector stays at its largest size until the queue closes: at most 64 commands, about 96 KB per queue.

Assisted-by: LLM (Claude:Opus 5.5)

Every publication allocated a DeferredBatch with room for two rings of
completions (64 entries, about 3.5 KB through kvmalloc) and freed it when
the publication ended: on the direct compute and render paths this was a
kvmalloc and a kvfree in every submit ioctl.

The backend now keeps the batch of its last publication once it is
finished and lends it to the next one. A batch that still holds work is
dropped as before, and concurrent publishers of one queue (the two
engines' scheduler workers and the ioctl) each get their own: the first
one back is kept. The batches are created and used exactly as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
prepare_submission collected the validated commands and their VM pins in
two vectors allocated by every submit ioctl and freed at its end.

The queue now keeps both, empty between ioctls, so their room is allocated
once. Commands are drained from them in the same order, and whatever an
error leaves unsubmitted is released at the same point as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Packet::new built the packet, with its 1.3 KB validated command, on the
stack and moved it into the Arc. The compiler kept three stack copies of
the command for that, so every submitted command was copied four times
here.

Allocate the packet first and initialise it in place: the command is
copied once. A failed allocation still drops the command unpublished.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
@jgoyvaerts
jgoyvaerts force-pushed the m4pro/g17-submit-path-allocs branch from 954fc9b to 1c33065 Compare November 21, 2026 06:14
@jgoyvaerts

Copy link
Copy Markdown
Contributor Author

Commit messages only, for the pre-PR checklist: the commits no longer carry Assisted-by (item 4), and the disclosure is now the PR body's last line. The tree is identical to 954fc9be6f (git diff 954fc9be6f 1c33065812 is empty). New head 1c33065812.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Looks good; approve once the HW row below is in.

I checked the three changes. PublicationBatch (drivers/gpu/drm/asahi/g17/runtime.rs:917-925, 1586-1597) takes the spare before allocating, so nothing is allocated under the mutex. It returns a batch only when is_idle() holds, after finish() has drained every vector and cleared has_source_completions, and it is dropped at the end of each publisher with no other lock held. No path replaces a lent batch's vectors, so the room checks (render/tvb/vacated) behave as before. In queue/mod.rs the scratch vectors are taken out of the queue for the ioctl and cleared before finished.clear(), the same point the old vectors were dropped, and the VM pins now outlive the leftover commands on the error path. Packet::new keeps its error behaviour.

  1. Nit: drivers/gpu/drm/asahi/g17/job.rs:154-169 can be a single Arc::init(init!(Self { ... }), GFP_KERNEL), like the Arc::pin_init users elsewhere in g17. That drops the UniqueArc round trip and the irrefutable let Ok(..).

HW row: one more G17 Mac other than the J616s (M4 or M5) on a kernel with only these commits on top of integration/aurora: gpubench compute/render rows plus a deqp-vk subset with no new failures and 0 GPU recoveries, and a regression row on the J616s (0 SELinux denials, 0 failed units, 0 new kernel errors).

Assisted-by: LLM (Claude:Opus 5.5)

@jgoyvaerts

Copy link
Copy Markdown
Contributor Author

I tried the nit and left the code as it is; the head stays 1c33065812.

With the arm64 defconfig + asahi.config build (LLVM=1), both Arc::init(init!(Self { ... }), GFP_KERNEL) and Arc::try_init(try_init!(...), GFP_KERNEL) compile cleanly, but they make Packet::new worse. The stack frame grows from sub sp, #0x4f0 to #0xd20, and memcpy calls go from 4 to 7. The InPlaceInit path moves the initializer, together with the ~1.3 KB command it captured, through extra closures by value. That brings back the copies this commit removes, so job.rs:154-169 keeps UniqueArc::new_uninit + init_with.

The J616s regression row will follow the next kernel build of our tree. We'd take your offer to run the other G17 Mac row (M4 or M5) in your lab.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review at 1c3306581218 against the pre-PR checklist.

No code, CI, attribution, or policy blockers found:

  • CI & Attribution/Policy (§1–§4, §6): host-tests is green on 1c3306581218; all three commits carry human author/committer and human Signed-off-by: only; PR body disclosure is clean; no model gating or naming issues.
  • PublicationBatch reuse (drivers/gpu/drm/asahi/g17/runtime.rs:917-925, 1429-1436, 1571-1597): spare_batch is a leaf mutex never held across allocation (DeferredBatch::publication()?), kvfree (when a competing batch drops because *spare is already populated, the guard drops before batch), or shared.state. DeferredBatch::is_idle() checks every field (has_source_completions, values, bindings, vacated, render, tvb), and reserve_replays is never called on a publication batch so lent capacities stay bounded.
  • Queue scratch vectors (drivers/gpu/drm/asahi/g17/queue/mod.rs:108-115, 447-530, 587-645): Moving self.scratch out with core::mem::take and clearing pending then guards before restoring self.scratch preserves error-path cleanup semantics. (If outputs.publish() fails after guards.drain_all(), guards drains before scratch.pending.clear(), which matches the pre-existing guards.into_iter() drop order.)
  • In-place Packet::new (drivers/gpu/drm/asahi/g17/job.rs:152-169): UniqueArc::new_uninit + init_with(init!(...)) is sound, avoids the Arc::init closure-move stack/memcpy regression you measured under LLVM=1, and preserves allocation-failure cleanup.

Remaining before approval (§5)

  • Clean same-head HW & regression run on J616s (default command line, without the unrelated tiler-heap PRs / unincluded 4th patch noted in the PR body): 0 SELinux denials, 0 failed units, 0 new kernel errors, 0 GPU recoveries.
  • One additional G17 Mac (M4 or M5) on top of integration/aurora with gpubench + a deqp-vk subset showing no new failures and 0 GPU recoveries.

Assisted-by: LLM (Gemini 4 Neon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants