Repository navigation
drm/asahi: g18: let queues opt in to returning before a submission retires - #2679
Conversation
|
The opt-in design is the safe one. Old Mesa never sets the flag and keeps returning only after retirement, which avoids the stale indirect-size read. Other generations reject the flag through the default
Assisted-by: LLM (Claude:Opus 5.5) |
5a65ae0 to
cc0b7b3
Compare
|
Thanks. Updated to cc0b7b3, and aurora-silicon/mesa#601 to cbe72672cb4.
|
|
Re-review of cc0b7b3: No blockers. The only change since our last comment is the rename and the uapi text; the code is otherwise identical to the previous head. All three points are resolved:
Assisted-by: LLM (Claude:Opus 5.5) |
cc0b7b3 to
558aa0e
Compare
|
I rebased this onto integration/aurora
Assisted-by: LLM (Claude:Opus 5.5) |
|
Re-review of 558aa0e: LGTM. The range-diff against cc0b7b3 shows only the drivers/gpu/drm/asahi/Makefile line next to #2071's Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Adversarial review at 558aa0efab374edb433b239178b3703ffbb6ae63: REQUEST_CHANGES (1 code blocker + 1 §4 PR body disclosure blocker; CI host-tests: pass; mergeable/clean).
Blockers
-
Container input fence errors are silently dropped (
drivers/gpu/drm/asahi/g18/drm.rs:388-395,drivers/gpu/drm/asahi/g18/sync_input.c:15-31)G18Job::execute()checksdma_fence_get_status(fence.raw())directly on each outerself.inputsfence.- Because
wait_foreign_inputs()unwrapsdma_fence_chainanddma_fence_arraycontainers viadma_fence_unwrap_for_each()and either waits on foreign leaf fences or skips G18's own leaf fences,dma_fence_enable_signaling()is never called on the outerdma_fence_array(unlike the previous synchronoussubmit_one(), which calleddma_fence_wait_timeout()on the outer fence). - When
G18Job::execute()later callsdma_fence_get_status()on an un-enableddma_fence_array,dma_fence_array_signaled()(drivers/dma-buf/dma-fence-array.c:103-140) takes the lazy scan path, verifies every leaf is signaled, jumps tosignal:, and callsdma_fence_array_clear_pending_error()— clearingPENDING_ERRORto0without propagating any leaff->error.dma_fence_get_status()therefore returns1(success) even when a contained G18 or foreign leaf fence signaled with an error (e.g.-EINVAL/-EIO). - Similarly, for timeline syncobjs (
dma_fence_chain, which Mesa uses for timeline points),drivers/dma-buf/dma-fence-chain.cnever copies leaff->errorontochain->base.error, sodma_fence_get_status(chain)always returns1even when a contained fence failed. - Fix: Add an unwrap-based status helper in
drivers/gpu/drm/asahi/g18/sync_input.cusingdma_fence_unwrap_for_each()(withdma_fence_put(cursor.chain)on early return) that checksdma_fence_get_status(leaf)on every unwrapped leaf fence and returns the first negative error, and call it fromG18Job::execute().
-
§4 Attribution (hard rule): missing
Assisted-by:line at the end of the PR body- Commit
558aa0efitself is clean (human author/committerThomas Glanzmann <thomas@glanzmann.de>,Signed-off-by:, no agent trailers). - However, an agent assisted on this PR (disclosed in the
558aa0efrebase comment asAssisted-by: LLM (Claude:Opus 5.5)), andpre-pr-checklist.md§4 requires a plainAssisted-by:disclosure line at the end of the PR body itself.
- Commit
Other checks (§1–§3, §6) & verification notes
- §1 / §2 / §3 / §6: Pass. Feature-bit + queue-flag opt-in (
DRM_ASAHI_FEATURE_ASYNC_SUBMITbit 39 /DRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMITbit 0) is generation-neutral, default-cmdline safe, and keeps existing userspace behavior unchanged. - Verified sound:
OutputPublication::accept()installs output fences underJobQueue::stateso cross-queue ordering holds;sync_input.cproperly putscursor.chainon early exit;G18JobholdsVmJobGuardsoVm::drop_mappingsdefers unmaps even on non-PF_EXITINGfd/queue close; dropping an unpublishedUserFenceonERESTARTSYSis safe (dma_fence_releaseonly warns whencb_listis non-empty). - Non-blocking nit:
G18DrmGpudoes not explicitly flushJobQueueon unbind/drop (currently safe becauseg18::probe::Driver::suppress_bind_attrs()istrueandApplicationOwner::shutdown()stops the session under the mutex before returning, causing queued jobs to exit viais_crashed()).
§5 HW evidence to include with the fix
- On J873g (M6) at the fixed head: add the §5 regression counts (0 SELinux denials, 0 failed systemd units, 0 new kernel errors) and an error-propagation test case using a timeline syncobj (
dma_fence_chain) /dma_fence_arraywrapping a failed fence. - Because
file.rsandqueue/mod.rsare shared paths, include a non-G18 sanity check confirmingDRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMITis rejected with-EINVALand existing queues are unaffected.
Assisted-by: LLM (Gemini 4 Neon)
…tires The G18 adapter ran every submission inside the submit ioctl: it handed the job to a kworker and polled for its retirement, so every flush blocked the caller for the whole GPU job (0.3 to 10 ms on the M6). Accepted submissions now run one at a time on a job worker, in submission order across every queue of the device, as the GPU runs them anyway. A queue created with the new DRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMIT flag (advertised by DRM_ASAHI_FEATURE_ASYNC_SUBMIT) returns from a submission with output syncs once it is accepted; the outputs signal when it retires. Every other submission still returns once it has retired, so userspace that reads GPU results right after a submit (Mesa's G18G indirect dispatch sizes, before its fix) is unaffected. - The output fence is installed under the job queue's lock, once the job has its place: a submission that finds it in a syncobj runs after the job that signals it. A submission that fails before it is accepted signals its fence before installing it, after the queue's earlier fences, as before. - The submit ioctl waits, interruptibly, for every input fence that no G18 job signals, inside chains and arrays too (g18/sync_input.c). The worker waits for no fence: one it waited for would hold every queue of the device. The adapter's own fences belong to earlier jobs, which retire first. - Before it runs, a job fails with the first error of any input fence, inside chains and arrays too (asahi_g18_fence_error()): a timeline point (dma_fence_chain) never carries its fences' errors, and a dma_fence_array whose signalling was never enabled clears its pending error when it is found signalled. - A queue has at most 32 accepted jobs; a further submit waits for the oldest. - An accepted job that fails after its submit returned signals its outputs with the error and is logged with the submitting process, and the queue's next submission fails with it: Mesa learns of a GPU fault only from a submit error. - The worker runs a job on the session itself (ApplicationOwner::run) instead of handing it to a second kworker, and is_crashed() no longer blocks on the session lock that a running job holds. - G18_SYNC_ONLY logs at debug level: with the Mesa fix it would log every indirect dispatch. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
558aa0e to
302cdf9
Compare
|
@DjDeveloperr both points. New head
The J873g rows you list come when the J873g is free:
Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Re-review against pre-pr-checklist.md (§1–§6) at verified live head 302cdf9905bd40e75ae18d27854666416cf05ab0: 0 code/policy/attribution/naming/CI blockers remain (host-tests: pass, mergeable/clean).
Prior blockers — resolved
- Container input fence error propagation (
drivers/gpu/drm/asahi/g18/sync_input.c:76-91,g18/drm.rs:390-394): Fixed.asahi_g18_fence_error()walks every leaf fence viadma_fence_unwrap_for_each()and checksdma_fence_get_status(fence)on each leaf, returning the first negative errno.- Verified against
drivers/dma-buf/dma-fence-unwrap.c:dma_fence_unwrap_first()takes a reference solely oncursor->chain(cursor->arrayis borrowed fromdma_fence_chain_contained()), sodma_fence_put(cursor.chain)on early return is exact with no leak or over-put. G18Job::execute()now callsasahi_g18_fence_error(fence.raw())per input, catching leaf errors insidedma_fence_chaintimeline points and un-enableddma_fence_arraycontainers as well as plain fences.
- §4 Attribution: Fixed. Commit
302cdf9905bdcarries human author/committer (Thomas Glanzmann <thomas@glanzmann.de>) andSigned-off-by:only, and the PR body now ends withAssisted-by:.
Checklist summary (§1–§4, §6)
- §1 / §2 / §3 / §6: Pass. Generation-neutral
DRM_ASAHI_FEATURE_ASYNC_SUBMIT(bit 39) /DRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMIT(bit 0), default command line, output fences installed underJobQueue::statelock, and non-blockingis_crashed().
Holding strictly for §5 HW rows on 302cdf9905bd
As noted in your update, holding APPROVE only for the queued hardware runs on this head:
- J873g (M6) §5 regression counts (
0 SELinux denials, 0 failed units, 0 new kernel errors); - J873g error-propagation rows with a timeline syncobj (
dma_fence_chain) and adma_fence_arraywrapping a failed fence; - Non-G18 sanity check confirming
DRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMITreturns-EINVALand existing queues are unaffected.
Ready to upgrade to APPROVE as soon as those rows are posted.
Assisted-by: LLM (Gemini 4 Neon)
|
@DjDeveloperr the J873g rows for Build.
Regression (the "with" boot, which also ran the #2652/#2680/#2758 rows):
Errors inside containers.
In every run A was still pending when it was wrapped (sync file status 0, Why the wrapped fence has to be pending. An earlier variant wrapped the already-failed fence itself. That test can't tell the two modules apart, because the error is gone before the driver sees the input:
Still to do: the non-G18 refusal ( Assisted-by: LLM (Claude:Opus 5.5) |
|
@DjDeveloperr the non-G18 row, on the M5 (J813, T8142, G17). Build.
Test. A small client calls
Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Final §5 re-review against pre-pr-checklist.md (§1–§6) at verified live head 302cdf9905bd40e75ae18d27854666416cf05ab0: APPROVE (0 blockers; host-tests: pass; MERGEABLE / CLEAN).
§5 Hardware Verification — Complete
- J873g (M6 / G18G) on
302cdf9905bd(7.3.0-aurora-j873-avd-usb4-perf+, default command line):- Container-fence error propagation A/B (3 runs/module, pending fence A failing with
-22): bothdma_fence_chain(timeline syncobj point) anddma_fence_array(2-fence merged sync file) now propagate-22(vs.1/ error lost prior toasahi_g18_fence_error()), with the queue's next submits returning-22once then0. - Baseline regression:
0failed systemd units,0SELinux AVC denials,0new kernel errors (only the 3 pre-existing boot lines), and0job failures from this PR's paths. - Combined with the earlier J873g async/sync A/B (
glFlushsubmit3.43 ms→0.020 ms, frame time6.48 ms→3.24 ms,synctest5×50/50, foreignsw_syncbinary/chain non-blocking isolation,SIGSTOP/SIGCONTrestart,SIGKILLteardown), G18 coverage is thorough.
- Container-fence error propagation A/B (3 runs/module, pending fence A failing with
- Non-G18 M5 (J813 / T8142 / G17) (
7.3.0-aurora-j813-s2r-usb4+, sharedfile.rs/queue/mod.rs/ UAPI hunks):DRM_ASAHI_FEATURE_ASYNC_SUBMIT(bit 39) is not advertised.QUEUE_CREATEwithDRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMITreturns-EINVAL(and unknown flag bit 1 returns-EINVAL), cleanly drops the rejected queue, and leaves the fd healthy for subsequentflags=0queue creation and destruction (0GPU job failures).
UAPI & Merge Readiness
- Verified against
integration/auroraHEAD(17075e3ed26a): highest defined feature bit remains bit 38 (DRM_ASAHI_FEATURE_RENDER_MESH_OUTPUT) andstruct drm_asahi_queue_create::flagshas no other flags, so feature bit 39 and queue-create flag bit 0 have no collisions. - Prior §1–§4 and §6 checks at
302cdf9905bdremain clean (host-tests:SUCCESS,mergeStateStatus:CLEAN).
Assisted-by: LLM (Gemini 4 Neon)
|
Merge-train adversarial review (MERGETRAIN-LINUX3) at 302cdf9: no blockers. Merges clean on integration/aurora bb9f8ec, and together with #2680 (1012615); The delta vs our earlier LGTM at 558aa0e is Dj's container-error fix only ( Nits (not blocking)
Rows: G18 (J873g) and G17 (J813) are posted; G13/G14/G14X/G15/G16 take the refusal path by code but have no hardware row. |
eb6d9ec
into
aurora-silicon:integration/aurora
|
Thanks. I'll keep 302cdf9 as it is for the train and put the four nits into one follow-up PR on top as soon as this lands:
The G13 to G16 rows stay as you noted: refusal path by code, no hardware row. Assisted-by: LLM (Claude:Opus 5.5) |
On the J873g (M6 Mac mini), the G18 adapter runs every submission inside the submit ioctl. It hands the job to a kworker and polls until it retires, so each
glFlushblocks the caller for the whole GPU job: 0.3–1 ms usually, up to 10 ms. CPU and GPU never overlap within a client. 4K60 video in mpv drops frames, and every GL client spends its frame budget waiting in the ioctl.Change.
DRM_ASAHI_QUEUE_CREATE_ASYNC_SUBMITflag (advertised byDRM_ASAHI_FEATURE_ASYNC_SUBMIT) returns from a submission with output syncs once it is accepted. The outputs signal when it retires.Ordering and fences.
g18/sync_input.c, like the G17 one.G18 job failed: submission N pid P (comm): err). The queue's next submission then fails with that error: Mesa learns of a GPU fault only from a submit error, and treats the context as lost.Session.
ApplicationOwner::run) instead of handing it to a second kworker and polling.is_crashed()no longer blocks on the session lock that a running job holds.G18_SYNC_ONLYlogs at debug level: with the Mesa fix, every indirect dispatch issues a sync-only submission.J873g rows. This commit on the M6's current stack (#2623, #2242, #2652), with the system Mesa and with a dev Mesa carrying aurora-silicon/mesa#601:
glFenceSync+glFlush), asyncbench 20000 triangles + 3 ms CPU per frameOther driver's fence, plain and inside a timeline point.
sw_syncfence as input, from a test module. The fence signals after 2 s. Meanwhile a second process renders and records its longest frame.dma_fence_chainaround the fence)Error delivery.
-EINVAL. The kernel logsG18 job failed: submission N pid P (fencetest): Error(22). The queue's next submission returns-EINVALonce; the one after succeeds.Teardown. Five GL clients were killed with SIGKILL while they had queued jobs (200000-triangle frames). There were no faults, and the next client rendered normally.
Earlier coverage, all queues async (the first version of this patch, without the opt-in; on the M6 desktop since 11-19):
Checks.
sync_input.c.tools/asahi/run-host-tests.py) pass, apart frompmp-floor-tests --mutant class-twice, which fails without this change too (J873g: report G18 GPU work to the CLPC (memory and fabric floor, +34% GPU-bound throughput) #2623's branch fails it the same way).queue-priority-testsis updated: it extracts thequeue_creategate, which now lets the async flag through.cc @DjDeveloperr @RyanTheTide @Acelogic @LiamRay10
Assisted-by: LLM (Claude:Opus 5.5)