Repository navigation
clpc/g17: busy-share GPU memory floor from the DT - #1369
AxxzyWasTaken wants to merge 4 commits into
Conversation
1257b68 to
37559c1
Compare
RyanTheTide
left a comment
There was a problem hiding this comment.
@AxxzyWasTaken thanks for the rebase and the power numbers. Three things before this can go in:
- apple_clpc_hw_t6040 is also the M4 Pro's table. The CPU map change (DCS 1,1,3,4 and fabric 0,0,0,0 at the top, and level_khz for the E cluster) lowers the memory floor a busy CPU gets on the M4 Pro, where the comment above the table measures 237-242 GB/s at DCS 5 / fabric 2. Please give T6041 its own table and leave T6040 as it is, or add M4 Pro measurements. We can run the M4 Pro side if you split it.
- The light-effort governor is in the shared G17 code and on by default, so it also changes the A18 Pro and M4 Pro. A 120 fps game reads about a third busy, under the 350 entry point, and stays under the 750 exit point once it runs at half effort. Please scope it per SoC (match data, T6041 first) until it has frame-time results on games, not only power.
- Please replace the five g17_light_* module parameters with per-SoC values, as was done for the CLPC thresholds.
|
Thanks. All three are in 33db416, on top of a merge of the current integration/aurora (e6588ee, clean).
Tested: Image and modules build at |
RyanTheTide
left a comment
There was a problem hiding this comment.
@AxxzyWasTaken thanks, this addresses all three points. The T6040 table matches the base again, the T6041 table is picked by machine compatible, and light effort is off everywhere except T6041 with no module parameters. Two things before merge:
- Please boot this head (33db416) on the M4 Max and post the scrolling power and a GPU smoke result. The light-effort and table code moved since your last numbers.
- Small race in report_busy(): two submitters that both see elapsed >= BUSY_WINDOW_NS each swap BUSY_WINDOW_TOTAL, and the second one reports about 0 permille for the same window, which on T6041 also counts as a quiet window. A compare_exchange on BUSY_WINDOW_START (only the winner reports) closes it. Optional: clearing busy_valid when the GPU goes inactive, so a new burst does not start from the share it had before a long idle.
We will run the M4 Pro and A18 Pro side to confirm nothing changes there.
|
Thanks. The race is fixed in 2a87da9: Boot of this head on the M4 Max (J616c), release
|
|
Merged the current integration/aurora (677be39) in 08860ee, so this merges again. The only conflict was the Built with Still open from your 13 Nov review: the scrolling power on the M4 Max. I'll post it from a boot that has the backlight. |
08860ee to
38e1273
Compare
|
Rebased without the two merge commits: the four commits now sit directly on Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested on my M4 Max as described. |
|
Tested on the J614c (16-core CPU / 40-core GPU M4 Max), at 38e1273, with #2173 for the GPU link, on top of #1664. It boots, the GPU registers with the CLPC, and there are no new warnings. The GNOME desktop and YouTube in Firefox (software decode, about 10 W for the whole machine at brightness 54) work as before. I don't have a GPU-heavy workload, so I didn't do an A/B power test and can't confirm the savings. It conflicts with my #1796 (the J614c's own Low Power ceiling, 2616 MHz instead of 2808). #1796 replaces 🤖 Generated with Claude Code |
|
Thanks for this, and for the careful power measurements on the M4 Max. One structural point before it can go in. We select behaviour by a hardware property, not by the machine it was measured on, and improvements that come from how the GPU works should apply to every chip with that GPU:
If you make the light effort the default, we can supply A/B rows on our M1 Pro, M2, M4 Pro and A18 Pro. |
|
Thanks. Both points are addressed in four new commits on top.
The @joske, about the order with #1796: now that #1369 has no T6041 table, there is nothing to convert to your per-machine low-power table. I test-merged #1796's head (25ab204) onto this one. The only conflict is in Tested:
Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described. |
|
Thanks for the rework: the light effort is now the default on every G17 configuration and the CLPC memory states come from the DT instead of the machine compatible, which is exactly the shape we were after. We will run the A/B rows on our M4 Pro and A18 Pro against this head. |
|
Thanks for f4adbb5. Reporting the busiest engine's share applies to every G17 and needs no per-SoC gate. The CS2 numbers explain it well: summed concurrent engine time pinned the share at 1000. Dropping the full-effort doubling is backed by the J616C measurement that the effort word does not move the clock under load. This change reaches every G17 that feeds the CLPC, so we will run the A/B rows we offered on our M4 Pro and A18 Pro at f4adbb5 rather than 9be5ff2. Each row will cover one GPU-bound and one mixed load, with power, and we will post them here. |
|
New commit f4adbb5: "drm/asahi: g17: busy share from the busiest engine". The busy share given to the CLPC was the window's summed engine time across vertex, fragment and compute, doubled at full effort. Those engines overlap, so the sum isn't a share of the window: in CS2 on the J616c, fragment and compute are 73 % and 83 % busy at once, so every game read 1000 ‰ and T6041's memory ladder sat in its top column whenever the GPU had work. It now reports the busiest engine's share, unscaled. The doubling assumed that full effort does the same work in about half the time, but on the J616c (25G83) the effort word doesn't change the clock under load. vkalu runs at 10.8 TFLOPS, and the firmware reports its top performance state (statistics tag 0x1b), at every effort from 0x10000 down to 0x1. The light-effort governor keeps its summed measure, so nothing changes off T6041. Rows on the J616c (old and new switchable in one test module):
At FIFO 120 the SoC and memory power don't change, because the P-cluster votes keep DCS at 4 in that scene. gltxf and gldepthbench give the same frame times and power in both: with a single busy engine, the old and new shares read the same. |
|
Thanks for the f4adbb5 write-up; reporting the busiest engine's share instead of the overlapping sum makes sense, and the CS2 rows show the ladder dropping from 3 to 1-2 with no fps change. integration/aurora has moved and GitHub now reports this branch as conflicting. Could you rebase it? We'll run the M4 Pro and A18 Pro A/B rows on the rebased head, so they match what would merge. |
f4adbb5 to
02edec4
Compare
|
Thanks for the push. The 02edec4 series still sits on a base from 2026-11-16 (676 commits behind integration/aurora), so GitHub still reports a conflict and CI has not run. Could you rebase it onto the current integration/aurora? The busy-share and light-effort changes look the same as on f4adbb5, so nothing else is needed from the code side. We'll run the M4 Pro and A18 Pro A/B rows once the rebased head is clean. |
The CLPC driver's per-SoC table says which DCS and fabric state reported GPU work gets. On SoCs whose memory latency and power at each state have been measured, scaling that floor by the GPU's busy share lets light GPU work use lower memory states while busy GPU work keeps the top states. Describe the optional apple,gpu-busy-memory-states and apple,gpu-busy-permille properties on the performance-controller node. Both properties are required together; without them, reported GPU work gets the compatible's default floor. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
02edec4 to
306bfb9
Compare
|
Rebased onto
Assisted-by: LLM (Google:Gemini 4 Neon) |
|
Blockers: 2. I checked the earlier points at 306bfb9. They are resolved: the T6040 and T6041 tables are untouched, there are no module parameters, and the light-effort governor is gone. The window report is claimed with a compare_exchange, and
Commits: 4/4 have a human Signed-off-by and no agent trailers, and the PR body ends with an Assisted-by line. CI is still pending at 306bfb9. HW row: at 306bfb9 (W=1 build, default cmdline), with 0 SELinux denials, 0 failed units and 0 new kernel errors on each Mac. Assisted-by: LLM (Claude:Opus 5.5) |
… the DT On SoCs where the GPU driver reports work in flight to the CLPC, any reported GPU work holds the top memory and fabric floor. At 120 Hz a scrolling browser keeps the GPU reported active most of the time for a few percent of real execution, so light desktop use pays for the top DCS and fabric states. Let the GPU driver report its busy share per window through apple_clpc_gpu_set_busy() and, when a performance-controller node provides apple,gpu-busy-memory-states and apple,gpu-busy-permille (apple,pmc.yaml), pick the GPU floor from that four-column ladder. A non-zero top DCS state enables the ladder; otherwise, and on every node without these DT properties, reported GPU work keeps the compatible's default top floor unchanged. Each DT state is bounded by its rail's maximum in the compatible table (max(map[rail][top], gpu[rail]), matching apple_clpc_ane_states_set()) as well as the 4-bit PMC state field. Because the optional ladder only lowers a floor, malformed DT properties (bad cell count, missing partner, out-of-range rail state or unordered thresholds) log a warning and fall back to the compatible's static table unchanged instead of failing CLPC probe. A higher busy level applies at once. A lower one applies at the first CLPC sample at least 400 ms after the last sample that asked for the current level or a higher one. Clearing GPU activity forgets the last reported share so a new burst after idle starts at the top column, as does a GPU driver that reports no share. The debugfs status reports the last busy share and, when a ladder is active, the current level without advancing it. Extend tools/asahi/clpc-gpu-demand-tests.py to cover the DT parser bounds, safe fallback on malformed properties, the busy-share ladder, the no-share and no-DT fallbacks, the 400 ms down-step hold and negative controls. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Track the engine time charged across all clients per engine (vertex, fragment and compute) in usage.rs, track each active QID's oldest unretired command timestamp in g17/qos.rs, and report the busiest engine's share of each 200 ms window to CLPC through apple_clpc_gpu_set_busy(). The three engines run in parallel, so summing their times is not a share of the window: in CS2 on the J616c the fragment and compute engines run 73 % and 83 % busy concurrently, which saturated a summed share at 1000 permille and pinned the memory ladder at its top column. Taking the busiest engine's share avoids that double-counting; overlapping work across multiple clients on the same engine still clamps to 1000 permille, safely erring toward the higher floor. Counting only retired stages misses a long compute or render command in flight alongside a light submitter such as a compositor, which would otherwise keep reporting the compositor's low share until the long command retired. Difference (retired_ns + in_flight_age) per engine across window boundaries so in-flight commands count while they run without being double-counted when they retire, and lift immediately to 1000 permille (including mid-window and via idle_check rechecks when no other client submits) once any unretired command reaches 50 ms. Reading ENGINE_NS with Acquire ordering before locking qos pairs with Client::add()'s Release store after qos_complete() so a concurrent retirement cannot be counted twice. Also notify port GPU activity on ComputeHost::note_submission() to match the render and direct paths. Add KUnit tests in g17/power.rs and extend tools/asahi/clpc-gpu-demand-tests.py to drive the production CLPC ladder with the production g17 BusyShare logic for a long compute command running beside a light compositor, along with negative controls. Measured on the J616c (busiest-engine retired share): - CS2, fixed de_dust2 scene, FIFO 120: busy share reads 480-620 permille instead of 1000, ladder steps to levels 1-2, 120.0 fps unchanged. - CS2 uncapped: 176.6-177.1 fps against 176.1-176.5 fps. - gltxf (3456x2234) and gldepthbench: identical 3.84 ms frame times and power (single busy engine reads the same). Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…n the J616c Give the M4 Max's CLPC node the GPU busy-share memory states measured on the 32-core J616c: DCS 1, 2, 4, 5 and fabric 0, 0, 0, 3, split at 160, 500 and 800 permille of the busy share reported by the GPU driver. The top column matches apple_clpc_hw_t6041's DCS 5 / fabric 3 floor, so busy GPU work keeps full memory bandwidth while light work such as scrolling a browser drops to the lower memory states. These values were measured on the 32-core J616c only; placing them in t6041.dtsi also applies them to the J614c and J575c (which have not been measured yet). With a 3 GB/s copy on the J616c: DCS 0 / fabric 0 draws 5.46 W at 309 ns DRAM latency, DCS 3 draws 6.22 W at 166 ns, DCS 4 draws 6.92 W at 114 ns, and DCS 5 draws 7.04 W at 113 ns. Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
|
Pushed
Validated with |
306bfb9 to
e0b93b2
Compare
|
Thanks for e0b93b2. Checked against the four points from the last review:
A note only: in-flight age counts from publication, so a command that waits a long time on a dependency after it is published also reads as busy and gets the top floor. That is the old behaviour (top floor while active), so the cost is a missed saving, not a regression. A sentence in the Still open: the J616c rows at this exact head, including the scroll saving with the CPU map now asking for DCS 5 / fabric 3 ( Assisted-by: LLM (Claude:Opus 5.5) |
|
J616C (M4 Max 16c/32g) exact-head boot & live CLPC GPU busy-share memory floor verification at
|
RyanTheTide
left a comment
There was a problem hiding this comment.
Exact-head J616C row at e0b93b2 shows the DT ladder driving the CLPC GPU memory floor live. All four earlier points were fixed in code before. Approving.
Assisted-by: LLM (Claude:Opus 5.5)
DjDeveloperr
left a comment
There was a problem hiding this comment.
Full review of e0b93b21f2142629c1908747ece397bf0523c167 against pre-pr-checklist.md (§1–§6): REQUEST_CHANGES (code & CI clean; held strictly on §5 evidence).
Code & checklist audit (§1–§4, §6) — clean
- §1 No model gating: Ladder selection is driven strictly by optional DT properties (
apple,gpu-busy-memory-statesandapple,gpu-busy-permilleinapple,pmc.yaml,dependentRequired). Static tables for T6040/T6041/T6050/T8140/T8132 are untouched, and nodes without the properties keep their exact default floor.t6041.dtsicommit message transparently notes the states were measured on J616c only and also apply to J614c/J575c. - §2 Production-ready / §3 Safety: Default cmdline, zero module parameters.
apple_clpc_dt_map()bounds every DT state against bothmax(hw->map[rail][LAST], hw->gpu[rail])and15;apple_clpc_dt_apply()warns and falls back cleanly to the static compatible table on any malformed/partial property so CLPC probe can never fail. Ing17/qos.rs,g17/port.rs,g17/power.rs, andusage.rs:ENGINE_NSRelease store afterqos_complete()paired with Acquire load beforeqos.lock()prevents retirement double-counting.self.qosandself.busylocks are never nested (report_busy()releasesqosbefore lockingbusy), avoiding lock-ordering hazards.- Telescoping window accounting (
delta + age - prev_age), immediate1000permille lift for unretired commands>= 50 ms(with soloidle_check()recheck), andclear_active()window reset after idle all handle long commands and burst transitions safely.
- §4 Clean-room & attribution / §6 Naming: 4 clean bisectable commits (
86801ace6525,225afc294fcb,5819ed4b3308,e0b93b21f214), all human author/committer + humanSigned-off-by:only, zero AI trailers in commits; PR body carriesAssisted-by:; naming clean. - CI: 8/8 checks
SUCCESS(dt-checks,host-tests,source,rust-build,kunit, and all 3DWC3/ATCbuild matrices);MERGEABLE(CLEAN), non-draft.
Remaining blocker before approval (§5 Evidence)
The exact-head J616C boot row at e0b93b21f214 (7.3.0-j616c-pr1369+) cleanly confirms the DT properties are parsed live and apple-pmc agent1 floor transitions 0x0 -> 0x4 -> 0x35 -> 0x0 across glmark2-es2-drm terrain (267 FPS, 0 GPU/CLPC/PMC errors, FAILED_UNITS=0). However, the remaining §5 items requested by @RyanTheTide at e0b93b21 are still owed before merge:
- Light desktop / browser scroll A/B at
e0b93b21f214vs base: Confirm power savings on current base (where#1951set T6041's CPU map to DCS 5 / fabric 3 withsustain=0) withclpcdebugfslast_reasonshowingHOST_GPUwithout CPU at the lower ladder columns. - Live long-compute-beside-light-compositor check: Confirm on hardware that a
>= 50 mscompute job alongside a light desktop liftsagent1 floorto0x35(DCS 5 / fabric 3) while in flight (plus no-regression on GPU copy / CPU DRAM bandwidth). - Full regression check: Explicit
0 SELinux denialsand0 new kernel errorsconfirmation alongsideFAILED_UNITS=0(and M4 Pro J616s / A18 Pro J700 no-regression rows once posted).
(Non-blocking nit: the PR description still references 306bfb98ef7c and CAS reporting; refreshing it to describe e0b93b21f214's spinlocked in-flight BusyShare accounting would keep it accurate.)
Assisted-by: LLM (Gemini 4 Neon)
|
A18 Pro (J700, T8140) no-regression row at e0b93b2: PASS, no change in behaviour, as expected. The Neo's CLPC node ( A = fleet81 (integration/aurora 85e255d + open heads, 4bf9f69d), B = the same tree + this PR. Config identical, one boot each, silent. CLPC debugfs status was sampled every 0.2 s during GPU load (gpubench, a render-pass sweep, a dEQP-VK subset). A ran twice in the same boot, B once.
Throughput: every gpubench and render-pass value for B falls inside the spread between the two A runs. Queued, multi-process and multi-queue rows are within -3.0% to +2.2% of the A mean. Depth-1 rows are noisy; the two A runs alone differ by 9-34%. Render passes, median ms (A1 / A2 / B): 16 passes 0.395 / 0.379 / 0.381, 100 passes 3.327 / 3.408 / 3.369, 200 passes 7.073 / 7.020 / 7.062. No new build warnings. How it was run
Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
§5 verification re-review at e0b93b21f2142629c1908747ece397bf0523c167: keeping prior CHANGES_REQUESTED hold in place pending the remaining J616C §5 hardware rows.
Newly verified since prior review (10:35Z)
- A18 Pro (J700, T8140) no-regression row (comment) — PASS: Same-machine A/B at
e0b93b21cleanly proves the non-DT-ladder path (host_gpu busy: 8..1000 permille (none), default floor unchanged,gpubench+vkpasseswithin A1/A2 spread,dEQP-VK1139-case subset802 pass, 0 failunchanged,0 SELinux denials / 0 failed units / 0 kernel errors / 0 oops / 0 GPU firmware crashes / 0 CLPC errors, 0 new build warnings). - PR description: Refreshed to accurately describe
e0b93b21f214and spinlockedBusyShareaccounting (prior non-blocking nit resolved). - Code (§1–§4, §6) & CI: Unchanged and clean (
8/8CI checksSUCCESS,MERGEABLE/CLEAN).
Remaining §5 blockers on J616C (affected T6041 hardware)
@AxxzyWasTaken's 10:05Z J616C boot & glmark2-es2-drm terrain floor-transition row (comment) was already evaluated in the 10:35Z review. As also noted as queued in the updated PR body, the remaining J616C §5 items are still owed before approval:
- Light desktop / browser scroll A/B at
e0b93b21f214vs base: Same-machine A/B confirming power savings on the current#1951base (CPU map at DCS 5 / fabric 3,sustain=0), withclpcdebugfslast_reasonshowingHOST_GPUwithout CPU at the lower ladder columns. - Live long-compute-beside-light-compositor check: Confirm on hardware that a
>= 50 mscompute job alongside a light desktop liftsagent1 floorto0x35(DCS 5 / fabric 3) while in flight (plus no-regression on GPU copy / CPU DRAM bandwidth). - Full J616C §5 regression check: Explicit
0 SELinux denialsand0 new kernel errorsconfirmation alongsideFAILED_UNITS=0(currently10:05Zonly reports0 GPU/CLPC/PMC errorsandFAILED_UNITS=0).
Once those J616C rows land, this is ready for immediate approval.
Assisted-by: LLM (Gemini 4 Neon)
Scales the CLPC's GPU memory/fabric floor with the GPU's reported busy share, driven purely by DT properties (
apple,gpu-busy-memory-statesandapple,gpu-busy-permille). Rebased ontointegration/auroracacf3554(live heade0b93b21f214) as a 4-commit bisectable series:dt-bindings: power: apple,pmc: the CLPC's GPU busy-share memory states— addsapple,gpu-busy-memory-states(4-bit PMC states,maximum: 15) andapple,gpu-busy-permille(maximum: 1000) withdependentRequired.soc: apple: clpc: scale the GPU memory floor with its busy share from the DT— reads the GPU busy-share ladder from DT whenever present (keyed strictly on the DT properties, not on T6041 code paths; static T6040/T6041 tables unchanged, compatible default floor used when absent or when top DCS state is 0).drm/asahi: g17: report the busiest engine's share to CLPC— tracks per-engine retired time inusage.rsand each active QID's oldest unretired command timestamp ing17/qos.rs, reporting the busiest engine's share (retired plus in-flight age via spinlockedBusyShareaccounting, with Acquire/Release ordering againstqos_complete(), immediate 1000-permille lift once any unretired command reaches 50 ms, andidle_check()rechecks) over each 200 ms window to CLPC.arm64: dts: apple: t6041: the GPU busy-share memory states measured on the J616c— adds<1 2 4 5>, <0 0 0 3>and<160 500 800>to&clpcint6041.dtsi(keeping soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951's DCS 5 / fabric 3 top states for both CPU and busy GPU work).Rebase & scope updates (after #1951 and #2607 merged)
#2607(cacf3554) supersedes it (setsactive_effort=0x55af/0x6442<0x8000on non-M5 and0on M5 streaming). Keeping light effort on top of#2607would have clearedbusy_validviaset_effort_full(false)and regressed#2607'sbound/waitedboost. Off T6041 (M4 Pro / A18 Pro / M5), firmware effort is now completely untouched, and CLPC behavior is unchanged where DT does not specifyapple,gpu-busy-*.apple,cpu-memory-states,apple,cpu-level-mhz) so CPU keeps soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951's DCS 5 / fabric 3, and raised the GPU busy-share top column to DCS 5 / fabric 3 (<1 2 4 5>, <0 0 0 3>).checkpatch.pl --strictwith 0 errors / 0 warnings / 0 checks.Testing
ARCH=arm64 LLVM=1 W=1):drivers/soc/apple/clpc.o(at commit 2 and HEAD),drivers/gpu/drm/asahi/g17/clpc.o,drivers/gpu/drm/asahi/aurora.o,apple/t6041-j616c.dtb, andapple/t6040-j616s.dtbbuild cleanly with zero new warnings.checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on all 4 commits.clpc-gpu-demand-tests.pyPASS (including long-compute-beside-light-compositor and negative controls),clpc-high-power-cleanup-tests,usage-tests,g17-firmware-selection-tests,g15-queue-credit-tests, and KUnit tests ing17/power.rs.e0b93b21f214(7.3.0-j616c-pr1369+): verified DT properties parsed live andapple-pmcagent1 floortransitions0x0 -> 0x4 -> 0x35 -> 0x0acrossglmark2-es2-drmterrain(267 FPS, 0 GPU/CLPC/PMC errors,FAILED_UNITS=0). Follow-up §5 hardware rows queued: light desktop/browser scroll power A/B vs base, live>= 50 mscompute-beside-light-compositor check, and explicit0 SELinux denials/0 new kernel errorsconfirmation. Note:t6041.dtsiapplies the measured J616c GPU busy-share table to T6041 boards (J614c/J616c/J575c).Assisted-by: LLM (Google:Gemini 4 Neon)