Repository navigation
irqchip/apple-aic: Use the SIQ_CFG routing on every AICv2+ controller - #2222
RyanTheTide merged 2 commits into
Conversation
|
Thanks for tracking this down; the IRQ-thread wakeup numbers are a clear win. We select behaviour by a hardware property rather than by the list of SoCs it was tested on, and this adds a second per-SoC entry for the same mechanism: SIQ_CFG is the AICv2-and-later system register that says which CPUs accept IRQs, so the effective-affinity read is a property of the controller version (and of running at EL2, which the code already checks), not of T6040/T6041. Could you set If you make it the default, we can run rows on our M1 Pro, M2, M4 Pro and A18 Pro. |
With SIQ_CFG routing, each AICv2+ IRQ reports the CPUs that accept IRQs as its effective affinity. Each CPU's SIQ_CFG is read when that CPU starts, and an IRQ copies the set only when its affinity is set. An IRQ keeps the set it was given. So a CPU that starts later and accepts IRQs is missing from every IRQ requested before it came up, and a CPU that goes offline stays in every mask. That is harmless while only the boot CPU accepts IRQs, but not when several CPUs do. Read SIQ_CFG in a CPU's online callback, drop the CPU from the set in its offline callback, and set the affinity of every requested AIC IRQ again whenever the set changes. genirq then updates the IRQ's effective affinity and its thread's CPU mask. CPUs that do not change the set cost nothing beyond the register read. Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
SIQ_CFG, the per-CPU register that says whether a CPU accepts AIC IRQs, is part of AICv2 and later. m1n1 and aura write it on every CPU of the M1, M2 and M3 families, and the sleep driver saves and restores it on the M5 parts. Reporting the accepting CPUs as the effective affinity of AICv2+ IRQs is therefore a property of the controller version, plus the EL2 check that is already there. It was enabled only for T8140, through a per-SoC flag. T6040 and T6041 take every AIC IRQ on CPU 0, like T8140: after 3 hours of uptime /proc/interrupts counts none of the 72 AIC IRQs on any other CPU. Their device tree lists "apple,t6041-aic3" and "apple,t8122-aic3", which matches the generic AIC3 entry, so their IRQs reported no effective affinity and genirq let the threaded handlers run anywhere. A SCHED_FIFO IRQ thread stays on whichever CPU it last ran on, and when that CPU is idle in WFI the hard IRQ on CPU 0 must wake it with an IPI before the handler can run. For the G17 GPU completion IRQ on T6040 (gpubench compute.light at depth 1, ftrace, about 10,000 IRQs per run, five boots), the time from the hard IRQ to the thread running is: thread's CPU idle (99% of IRQs on an idle desktop) 4.4-9.6 us thread's CPU running another task 1.3-2.4 us thread on CPU 0 1.3-1.5 us The one boot that measured 2 us had a compiler busy on every CPU during the run, so the thread's CPU was almost never idle. Drop the flag and use the routed irqchip on every AICv2+ controller at EL2. At EL1 nothing changes. Where the boot loader lets every CPU accept IRQs (m1n1 and aura on M1 to M3), each IRQ now reports all of them, which the previous commit keeps up to date across CPU hotplug. Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
41fd5db to
7d5104b
Compare
|
Done in 7d5104b, as two commits:
Tested: arm64 build of Yes please, rows on the M1 Pro, M2, M4 Pro and A18 Pro would help most. On M1 Pro and M2, |
|
One gap I couldn't close: I found nothing showing that T8152 (M6) implements SIQ_CFG. It takes the generic AIC3 entry, so if the register isn't there, the EL2 read in the online callback would fault at boot. No M6 board is enabled today. If you know it lacks the register, I'll give it its own entry with that reason in a comment. |
|
Thanks for 7d5104b, and for adding the hotplug tracking rather than leaving it as a comment. Since this now runs on every AICv2+ machine and has not booted yet, we will run rows on our M1 Pro, M2, M4 Pro and A18 Pro, including |
|
Row from our M2 (MacBook Air, T8112, AIC2), as offered: production kernel (fleet74) vs the same tree plus this PR's two commits, one-shot boot, same machine.
Rows on our M4 Pro (T6040) and A18 Pro are queued; they also cover hotplug. |
|
Row from our A18 Pro (MacBook Neo, T8140, AIC2): production 7.3.0-40 vs the same tree plus this PR, one-shot boot, same machine.
Together with the M2 row above, this covers both AIC2 cases we have. An M4 Pro row is still queued. |
|
Row from our M4 Pro (MacBook Pro 16-inch, J616s, T6040): production fleet kernel vs the same tree plus this PR, one-shot boot.
With the M2 and A18 Pro rows above, that is every AIC2 case in our lab: PASS on all three. |
7d5104b to
c2e95d3
Compare
|
Commit messages only, for the pre-PR checklist: the commits no longer carry Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Review of head c2e95d38b7160796d4992956f4e2767ccc22fb92 against the pre-PR checklist.
Code, policy, CI & evidence
- 0 blockers. CI (
host-tests) is green. - §1–§3: Removes the per-SoC
siq_cfg_routinggate in favour of controller version (version == 2 || version == 3) +is_kernel_in_hyp_mode(). MovingSIQ_CFGtracking toCPUHP_AP_ONLINE_DYN(aic_irq_cpu_online/aic_irq_cpu_offline) keepsaic_irq_cpusaccurate across CPU bringup and hotplug, refreshes requested IRQs viairq_set_affinity()(safe in hotplug context — takesdesc->lockonly, nocpus_read_lock()), and gracefully falls back tocpu_possible_maskifcpuhp_setup_state()fails oraic_irq_cpusis empty. - §4 & §6: Both commits (
d3d404c0e448,c2e95d38b716) carry human author/committer and humanSigned-off-by:only, with PR-body disclosure present. - §5: Hardware rows from @RyanTheTide on the identical code diff (
7d5104b9->c2e95d38, commit messages only) cover all three AICv2+ classes — M2 (T8112, AIC2: 62/77 active IRQs0-7, s2idle x2 + NVMe, Enforcing, 0 failed units, 0 AVC, 0 kernel errors), A18 Pro (T8140, AIC3: 74/860, P+E hotplug + NVMe, s2idle x2, Enforcing, 0 failed units, 0 AVC), and M4 Pro (J616s, T6040, AIC3: 115/1370, s2idle x2, 0 new kernel errors).
Non-blocking notes
- In
aic_irq_cpus_update()(drivers/irqchip/irq-apple-aic.c:1123),cpumask_copy(&aff, irq_get_affinity_mask(virq))assumesirq_get_affinity_mask(virq)is non-NULL right afterirq_has_action(virq). Static DT AIC IRQs are rarely disposed at runtime, but checking for NULL (or guarding withrcu_read_lock()) would eliminate any TOCTOU window. - On a multi-CPU-accepting part, if an IRQ's saved
smp_affinitywere ever pinned solely to a CPU that is currently offline,irq_set_affinity(virq, &aff)(force=false) would return-EINVALwithout refreshingeffective_affinity; not hit under normal masks, just worth keeping in mind for any follow-up. - The PR body's "Not done" section still says "Not booted yet", which is now superseded by the M2, A18 Pro, and M4 Pro hardware rows in the thread.
Assisted-by: LLM (Gemini 4 Neon)
|
Merge-train adversarial review (MERGETRAIN-LINUX2) at c2e95d3: NO BLOCKERS. With Dj's approval at this head, that makes the joint verdict MERGE. Before the merge we will boot it on our Macs, as one test kernel of integration 24dc422 plus #2222 and #2225: M1 Pro, M2, J616s, Neo and the J615. Results will follow here. Full review#2222 irqchip/apple-aic: Use the SIQ_CFG routing on every AICv2+ controller @c2e95d38VERDICT: NO BLOCKERS Commits: d3d404c (hotplug tracking), c2e95d3 (drop What I checked and found to hold:
Blockers: none. Nits:
Missing rows:
Commit hygiene: both commits are by Jonathan Goyvaerts (author and committer) with a human Signed-off-by, and there are no agent trailers in the commits. The disclosure is on the PR body's last line. Provenance: SIQ_CFG is the public m1n1 register name already used in the tree, and there are no kext names. |
|
Hardware rows (MERGETRAIN-LINUX2), test kernel 7.3.0-aurora-mt3 = integration/aurora 24dc422 + #2222 c2e95d3 + #2225 6643e60, on the default fleet cmdline. Each Mac got 3 one-shot warm boots and went back to its default:
Evidence: traces/kernel/20261122-mt3-fleetboot (lab store). |
|
J616s (M4 Pro, T6040 AIC) row: PASS. This is the row the merge train was still waiting for. Kernel
The same kernel passed earlier on the M1 Pro, M2, Neo and J615 (posted above). With Dj's APPROVE and our review at this head, the train now merges it. |
cc5c09b
into
aurora-silicon:integration/aurora
#447 made AICv2+ IRQs report the CPUs that SIQ_CFG says accept them, but only for T8140, through the per-SoC
siq_cfg_routingflag. On the M4 Pro (T6040), whose device tree listsapple,t6041-aic3and thenapple,t8122-aic3, the controller matched the genericaic3_info. Every IRQ had an empty effective affinity, and genirq had no CPU to bind the irq threads to.The hard IRQ is always taken on CPU0. An irq thread stays on whichever CPU it last ran on, so when that CPU is idle, CPU0 has to wake it first. For the GPU's mailbox IRQ (irq/70), that wakeup is on the path of every GPU completion.
SIQ_CFG is part of AICv2 and later, not of one SoC. m1n1 and aura write it on every CPU of the M1, M2 and M3 families (value 2, "Enable IRQs (at least necessary on t600x)"), the sleep driver saves and restores it on the M5 parts, and #447 and this PR read it on T8140 and T6040. Two commits:
aic_irq_cpus_update()).siq_cfg_routingflag is gone. AICv2 and AIC3 at EL2 useaic2_routed_chip(aic_irq_domain_map()), and the EL2 check stays. Noaic3_t6041_infoentry is added.aic3_t8140_infostays because it still carriesfw_suspend. At EL1 nothing changes.Tested on the J616s (M4 Pro, 48 GiB, 25G83)
These numbers come from the first revision, which had a T6041-only entry. The behaviour on T6040 is the same, because only CPU0 accepts IRQs there and the new hotplug path does nothing. The kernel runs at EL2 (CTXR window).
effective_affinity_list= 0.Cpus_allowed_listis 0, and all of its interrupts land on CPU0.GPU IRQ to irq-thread wakeup, gpubench
compute.lightat depth 1 (trimmed mean):Depth-1 latency went from 75.1 to 68.7 us (P0), 91.9 to 86.1 us (P1) and 86.7 to 79.2 us (unpinned). In the full gpubench record,
idle.wakeis 8-14% faster andrender.lightclear at depth 1 is 14% faster. The before and after kernels were separate boots with our pending G17 changes (including #2217). The after kernel also had a drm_sched placement change that we later dropped. It does not touch IRQ routing or the irq thread.This revision:
drivers/irqchip/irq-apple-aic.obuilds for arm64 with W=1 and no warnings after each commit; checkpatch gives 0 errors and 0 warnings on both.Hardware
RyanTheTide posted rows on the same code (
7d5104b9->c2e95d38, commit messages only) for all three AICv2+ classes: M2 (T8112, AIC2), A18 Pro (T8140, AIC3) and M4 Pro (J616s, T6040, AIC3), with s2idle, CPU hotplug where available, 0 failed units and 0 new kernel errors. See the thread for the numbers.Not done
Assisted-by: LLM (Claude:Opus 5.5)