Skip to content

cpufreq: apple-soc: verify every SoC's transitions lazily on fast switch - #2302

Open
RyanTheTide wants to merge 1 commit into
integration/aurorafrom
sweep/cpufreq-lazy
Open

RyanTheTide wants to merge 1 commit into
integration/aurorafrom
sweep/cpufreq-lazy

Conversation

@RyanTheTide

@RyanTheTide RyanTheTide commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Lazy transition verification on the fast-switch path is enabled by a lazy_verify flag that only soc_t8140_info sets (drivers/cpufreq/apple-soc-cpufreq.c:300, from b11a5e9 "cpufreq: apple: verify T8140 transitions on the next request"). The other SoCs with verify_transition (T8142, T8152, T6050) get fast_switch_possible = false (line 1540). On them every schedutil frequency change wakes the governor kthread, which then busy-waits for the transition.

Change

Drop lazy_verify and key the lazy path on verify_transition, the match-data property that says the command register's PS1 field reads back the accepted state. Every policy now sets fast_switch_possible = true.

The lazy path needs nothing beyond what verify_transition already requires: the BUSY bit and the PS1 readback. Its deadline is the per-SoC transition timeout, so T8152 and T6050 keep their 2 ms bound. It still stops all requests when the accepted state is not the one last written, so an unexpected owner is caught as before, one request later. Waited-for requests (initial check, static governors, suspend) are unchanged.

The lazy verification scheme is by Diljit Singh (b11a5e9 and follow-ups); this only extends it to the SoCs that already verify transitions.

Testing

  • Build: drivers/cpufreq/apple-soc-cpufreq.o with W=1 (arm64, LLVM=1, release config): builds, no warnings.

  • SoCs that gain fast switch (T8142, T8152, T6050) are not bootable in our lab; the safety argument is from code and DT review below.

  • Hardware (one-shot test kernel 7.3.0-aurora-swl1 = integration 0abc2f3 + this and the other open sweep PRs, 2026-11-20):

    • T8112 J413 and T6000 J316s (no verify_transition, no change expected): boot + 20 s idle + 3 s2idle cycles (rtcwake, ~20 s each, rc=0): 0 kernel errors, 0 SError, 0 failed units, 0 AVC; apple-cpufreq on every policy, no 'stopping requests'.
    • T6040 J616s: same, 3 s2idle clean.

Still to run

  • Neo (T8140, J700): no-regression row. cat /sys/devices/system/cpu/cpufreq/policy*/scaling_driver = apple-cpufreq; schedutil active; cpu_frequency trace events under load from task context (no sugov: kthread wakeups); no "stopping requests" in dmesg.
  • J616s (T6040), M1 Pro (T6000), M2 (T8112): no-regression only. These SoCs have no verify_transition and already used fast switch.
Per-SoC impact and safety
SoC Boards (DT) verify_transition Before After
T8140 t8140.dtsi (Neo) yes lazy, fast switch same
T8142 t8142-j704, t8142-j813-j815 yes waited, governor thread lazy, fast switch
T8152 t8152-j873g yes waited, governor thread lazy, fast switch
T6050 t6050-j716s, t6050-dieX yes waited, governor thread lazy, fast switch
T8103/T600x, T8112/T6041, T8132, S5L8960X, generic many no fast switch same
  • T8142: firmware keeps CLPC/PMP ownership. A firmware change to the request is still detected as an unexpected state and latches the failure, as before.
  • T8152: P and M CPUs share one performance domain (m6_pm_cpufreq in t8152-j873g.dts), so one policy owns each command register. It has a fast die-temperature limit; a lowered limit is now also applied to the cluster at once, as on T8140.
  • T6050: firmware owns thermal limits (apple,firmware-managed-thermal); the resume path that re-reads the state after a firmware sleep already ran for every verify_transition SoC.
  • Suspend on these SoCs now settles a pending request before the generic suspend, as on T8140.
Not changed here
  • Boost off by default on T8112 (of_machine_is_compatible("apple,t8112") in module init): measured, turbo states do not raise the delivered P-core clock there. Its reason is in the comment.
  • T8132 online/offline lifecycle in module init: tied to needs_thermal_owner, which only T8132 sets. It could be derived from match data, but those lines are the context of the open boost-control change (cpufreq: apple-soc: offer no boost control without turbo-mode OPPs #2199), so it is left for after that lands.

Assisted-by: LLM (Claude:Opus 5.5)

Lazy verification was enabled by a separate lazy_verify flag set only on
T8140. The other SoCs whose command register reads back the accepted
state (T8142, T8152, T6050) still waited about a transition per request
on every frequency change, and so kept schedutil off the fast-switch
path: every change woke the governor kthread, which then busy-waited on
the register.

Nothing in the lazy scheme is specific to T8140. It needs what
verify_transition already describes: a command register whose PS1 field
reads back the accepted state and a BUSY bit. It uses the per-SoC
transition timeout for its deadline, so T8152 and T6050 keep their 2 ms
bound, and it still stops all requests when the accepted state is not
the one last written, so an unexpected owner is caught as before, one
request later. Waited-for requests (initial check, static governors)
are unchanged; suspend first settles a pending request, as on T8140.

Drop lazy_verify and key the lazy path on verify_transition. Every
policy can now switch from the scheduler, and a lowered fast
die-temperature limit is applied to the cluster at once on all of them,
as on T8140. The lazy scheme itself is Diljit Singh's.

Fixes: b11a5e9 ("cpufreq: apple: verify T8140 transitions on the next request")
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide
RyanTheTide marked this pull request as ready for review November 19, 2026 21:28
@RyanTheTide

Copy link
Copy Markdown
Contributor Author

M2 (J413, T8112) no-regression row: pass.

This PR's patch on the production fleet74 driver source (1f710d2), built as a module and swapped in at runtime, compared with the stock module on the same boot:

stock this PR
driver / governor, both policies apple-cpufreq / schedutil apple-cpufreq / schedutil
sugov kthreads (0 = fast switch) 0 0
cpu_frequency events in a 6 s single-thread run / from sugov threads 324 / 0 364 / 0
one busy P core, add chain, median GHz 2.523 2.525
all 8 cores, total GHz 18.11 18.08
cpufreq errors, "stopping requests" 0 0

s2idle x2 on rtc1 with this PR's module: 2/2 clean, no cpufreq messages after, one busy P core at 2.529 GHz after resume. Stock module restored.

@satchlj

satchlj commented Nov 20, 2026

Copy link
Copy Markdown
Member

Claude Opus 5.5 on @satchlj's behalf: a T6050 row for this PR, and a stacked follow-up.

  • What it replaces: our cpufreq: apple: switch T6050 frequencies from the scheduler #2484, which added .lazy_verify = true to T6050 alone. That is now closed in favour of this PR, which gives T6050 the same path through verify_transition.
  • T6050 input (J775c, Mac Studio M5 Max, earlier lobata kernel, 2026-11-05): a transition between 300 MHz or 1.3 GHz and the fastest state takes 12–75 µs at p50 and at most about 100 µs (userspace governor). The sugov kthread ran for 30–58 µs per update in a sched_switch trace. That is the kthread hop this PR removes.
  • Not yet run on this head. lobata's next candidate carries this PR. I'll post the J775c result here.
  • Stacked follow-up: arm64: dts: apple: t6050: describe CPU OPP transition latency (stacked on #2302) #2516 reports that measured 100 µs as T6050's transition latency, so schedutil's rate limit becomes 150 µs instead of 6 ms.

🤖 Generated with Claude Code

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Looks good; approve once T6050 (J775c) has a row on 544c498.

  1. drivers/cpufreq/apple-soc-cpufreq.c:84 and 678: the pending-request retry is scheduled at write_time + APPLE_DVFS_TRANSITION_NS (50 us). After that the verify timer re-arms every APPLE_DVFS_RETRY_NS (4 us, hard-irq, line 635) for as long as BUSY reads set. The 50 us figure is the T8140 transition time (the comment at line 531 says so), and this commit now applies it to T8142, T8152 and T6050. T8152 and T6050 are given a 2 ms transition bound, and on T6050 transitions were reported at up to about 100 us. On those SoCs a request that lands behind a busy transition turns into a 4 us hard-irq poll from about 50 us to the end of the transition, on every burst. Take the retry point from the SoC: from policy->cpuinfo.transition_latency when the OPP table gives one, or from a per-SoC field next to transition_timeout_us. At minimum, fix the line 531 comment, which now states a T8140 number for every verify_transition SoC.

The rest holds up. lock, verify_timer, pending_pstate and cap_pstate are set up for every policy (lines 1505-1510). The lazy deadline uses apple_soc_cpufreq_timeout(), so T8152 and T6050 keep 2 ms. An unexpected PS1 still latches transition_failed (lines 588-590 and 648). Suspend drains and settles a pending request before cpufreq_generic_suspend(). Resume re-reads the state on the same verify_transition condition as before. T8103, T8112, T8132 and the default info have verify_transition false, so nothing changes for them. The posted M2 module-swap row confirms that. CI host-tests pass and the branch merges cleanly.

HW row: J775c (T6050) on 544c498 + integration, the run already offered on this PR. It should show schedutil on every policy with no sugov: kthreads, cpu_frequency events from task context under load, no "stopping requests" or "DVFS busy timeout" through a 10 min mixed load and 3 s2idle cycles, and 0 new kernel errors, 0 AVC and 0 failed units. Neither T8142 nor T8152 has a row. Say so in the PR and ask for one from anyone who has those machines. Also needed: a J700 (T8140) no-regression boot on this head.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Merge-train adversarial review (MERGETRAIN-LINUX2) at 544c498: BLOCKERS.

Not merging until the blockers below are fixed or answered; the details give file:line and a failure case.

Full review

#2302 cpufreq: apple-soc: verify every SoC's transitions lazily on fast switch @544c4981

VERDICT: BLOCKERS

The mechanics are sound. The timer and lock are initialised for every policy (drivers/cpufreq/apple-soc-cpufreq.c:1505-1508). ->exit drains unconditionally (:1572). The lazy deadline uses apple_soc_cpufreq_timeout() (:404-407, :596-597), so T8152/T6050 keep 2 ms and T8142 keeps 400 us. An unexpected PS1 still latches transition_failed (:588-590, :648). No lazy_verify user remains after a merge with b241e35. Non-verify SoCs keep fast_switch_possible = true (:1542), as before.

Blockers:

  1. apple-soc-cpufreq.c:84, :678 and :633-636: a request that finds the cluster busy is retried at write_time + 50 us (the T8140 transition time), then every 4 us from a hard-irq hrtimer until BUSY clears. T8152 and T6050 are described with a 2 ms transition bound, and T6050 was measured at up to ~100 us. On T8152, a transition that really takes ~1-2 ms produces ~250-500 hard-irq timer fires per pending burst per cluster. Each fire takes a raw spinlock with IRQs off and does an MMIO read. That can easily cost more than the sugov kthread hop this PR removes. The retry point should come from the SoC (policy->cpuinfo.transition_latency or a field next to transition_timeout_us). The reviewer raised this on this exact head (RyanTheTide 11-21) and it is unaddressed, including the :531 comment that now states the T8140 50 us figure for every verify_transition SoC.
  2. No row on any SoC whose behaviour changes (T8142, T8152, T6050). The rows posted (J413, J316s, J616s) are on SoCs without verify_transition, where the diff does nothing. The reviewer's own approval condition, a T6050/J775c row on 544c498, has not been met. T8152 also gains the immediate fast-die cap path (:1025), which is a thermal-safety path, without a row.

Nits:

  • The body and the existing comment at :309 say "CLPC/PMP". CLPC is the macOS internal name for the performance controller. The comment predates this PR, but don't repeat it in new text.
  • Neo T8140: the code path is identical (it only swaps the flag for its alias), so the missing Neo row is low risk. It is still listed as "Still to run".

Missing rows: T6050 (J775c, from satchlj, pending), T8152 (J873, an M6 owner) with a transition-latency and hard-irq count under schedutil load, T8142 (J704/J813, an M5 owner), and the Neo T8140 no-regression row.

@satchlj

satchlj commented Nov 22, 2026 •

Copy link
Copy Markdown
Member

Claude Opus 5.5 on @satchlj's behalf. J775c (Mac Studio M5 Max, T6050) row for this PR's drivers/cpufreq/ at 544c4981. This is the run offered on 11-20.

What was booted: this is not an exact-head tree.

  • The kernel is lobata's cand-15 (2ed7977aa07c = integration/aurora plus our open PRs, which carried this PR plus our stacked T6050 latency commit afba7661), with afba7661 reverted (eb4a446cfbe2).
  • git diff --stat eb4a446cfbe2 544c49819597 -- drivers/cpufreq/ is empty, so the cpufreq directory is identical to this head. The rest of the tree differs from the PR branch.
  • Default cmdline, one one-shot boot (004beb8c), nothing installed.

Results

Check Result
Driver / governor apple-cpufreq / schedutil on all three policies (0-5, 6-11, 12-17)
sugov: kthreads 0 (fast switch)
cpuinfo_transition_latency / schedutil rate_limit_us 4,000,000 ns / 6000. There's no OPP clock-latency-ns in this DT, so it is derived from the 2 ms timeout
6 s bursty single-thread window, cpu_frequency events 1,218. Emitting context: 180 task (d..3.), the rest from scheduler hooks in hard or soft IRQ (d.h*, d.s*). By comm: <idle> 1,026, dd 96, bash 18, others ≤12. None from a sugov thread
Mixed load, 600 s 7-Zip at 1, 4 and 18 threads in turns, plus a bursty dd thread with idle gaps
"stopping requests", "DVFS busy timeout", "DVFS completion timeout", "unexpected DVFS state" 0 through the load and after each s2idle
s2idle 3/3 rtcwake -m freeze -s 12, rc 0. After each, a 5 s burst moves all three policies, with 0 lines
7-Zip 1T after the cycles 13,546
AVC / failed units / new kernel errors 0 / 0 / 0. The 9 error lines are the known DCP lane-count/clock and speakers-off lines

Blocker 1 cost on T6050. I counted fires of apple_soc_cpufreq_verify (the hard-IRQ hrtimer callback) with a kprobe (p:apsverify apple_soc_cpufreq_verify, kprobe_profile hit counts). The candidate has CONFIG_KPROBE_EVENTS; lobata's default kernel does not.

  • 6 s window: 202 fires for 1,218 frequency events, so 0.16 fires per event. 160 of them in hard IRQ.
  • Re-arms: only 3 fires came within 10 µs of a previous fire on the same CPU, which is the 4 µs APPLE_DVFS_RETRY_NS chain.
  • 600 s mixed load: 23,822 fires, about 39 per second across 18 CPUs.
  • Reading: on T6050 most requests find the cluster idle, so the 50 µs retry point rarely turns into a poll chain. T6050 transitions measured 12–75 µs at p50 and at most about 100 µs.
  • Not covered: this says nothing about T8152. If its transitions take the 1–2 ms its bound allows, the per-SoC retry point you asked for still matters there.

Evidence is in lobata's session notes (claude-cand15-2302-*).

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

The J775c (T6050) row is in and clean. Still needed: take the pending-request retry point from each SoC's transition latency instead of the T8140 50 us (and fix the comment that states it for every SoC), then T8152 and T8142 rows under schedutil load with the verify-timer fire count, plus the Neo no-regression row.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants