Skip to content

clpc/g17: busy-share GPU memory floor from the DT - #1369

Open
AxxzyWasTaken wants to merge 4 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:max-gpu-power-v2
Open

AxxzyWasTaken wants to merge 4 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:max-gpu-power-v2

Conversation

@AxxzyWasTaken

@AxxzyWasTaken AxxzyWasTaken commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Scales the CLPC's GPU memory/fabric floor with the GPU's reported busy share, driven purely by DT properties (apple,gpu-busy-memory-states and apple,gpu-busy-permille). Rebased onto integration/aurora cacf3554 (live head e0b93b21f214) as a 4-commit bisectable series:

  1. dt-bindings: power: apple,pmc: the CLPC's GPU busy-share memory states — adds apple,gpu-busy-memory-states (4-bit PMC states, maximum: 15) and apple,gpu-busy-permille (maximum: 1000) with dependentRequired.
  2. soc: apple: clpc: scale the GPU memory floor with its busy share from the DT — reads the GPU busy-share ladder from DT whenever present (keyed strictly on the DT properties, not on T6041 code paths; static T6040/T6041 tables unchanged, compatible default floor used when absent or when top DCS state is 0).
  3. drm/asahi: g17: report the busiest engine's share to CLPC — tracks per-engine retired time in usage.rs and each active QID's oldest unretired command timestamp in g17/qos.rs, reporting the busiest engine's share (retired plus in-flight age via spinlocked BusyShare accounting, with Acquire/Release ordering against qos_complete(), immediate 1000-permille lift once any unretired command reaches 50 ms, and idle_check() rechecks) over each 200 ms window to CLPC.
  4. arm64: dts: apple: t6041: the GPU busy-share memory states measured on the J616c — adds <1 2 4 5>, <0 0 0 3> and <160 500 800> to &clpc in t6041.dtsi (keeping soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951's DCS 5 / fabric 3 top states for both CPU and busy GPU work).

Rebase & scope updates (after #1951 and #2607 merged)

  • Dropped G17 light-effort governor: merged #2607 (cacf3554) supersedes it (sets active_effort = 0x55af/0x6442 < 0x8000 on non-M5 and 0 on M5 streaming). Keeping light effort on top of #2607 would have cleared busy_valid via set_effort_full(false) and regressed #2607's bound/waited boost. Off T6041 (M4 Pro / A18 Pro / M5), firmware effort is now completely untouched, and CLPC behavior is unchanged where DT does not specify apple,gpu-busy-*.
  • Reconciled with soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951: dropped the unused CPU DT override properties (apple,cpu-memory-states, apple,cpu-level-mhz) so CPU keeps soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951's DCS 5 / fabric 3, and raised the GPU busy-share top column to DCS 5 / fabric 3 (<1 2 4 5>, <0 0 0 3>).
  • Cleaned commit series: squashed intermediate fixups and dropped T6040 static-table edits so every commit compiles cleanly and passes checkpatch.pl --strict with 0 errors / 0 warnings / 0 checks.

Testing

  • Compile (ARCH=arm64 LLVM=1 W=1): drivers/soc/apple/clpc.o (at commit 2 and HEAD), drivers/gpu/drm/asahi/g17/clpc.o, drivers/gpu/drm/asahi/aurora.o, apple/t6041-j616c.dtb, and apple/t6040-j616s.dtb build cleanly with zero new warnings.
  • checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on all 4 commits.
  • CI 8/8 green; host & KUnit tests: clpc-gpu-demand-tests.py PASS (including long-compute-beside-light-compositor and negative controls), clpc-high-power-cleanup-tests, usage-tests, g17-firmware-selection-tests, g15-queue-credit-tests, and KUnit tests in g17/power.rs.
  • Live J616c hardware boot at exact head e0b93b21f214 (7.3.0-j616c-pr1369+): verified DT properties parsed live and apple-pmc agent1 floor transitions 0x0 -> 0x4 -> 0x35 -> 0x0 across glmark2-es2-drm terrain (267 FPS, 0 GPU/CLPC/PMC errors, FAILED_UNITS=0). Follow-up §5 hardware rows queued: light desktop/browser scroll power A/B vs base, live >= 50 ms compute-beside-light-compositor check, and explicit 0 SELinux denials / 0 new kernel errors confirmation. Note: t6041.dtsi applies the measured J616c GPU busy-share table to T6041 boards (J614c/J616c/J575c).

Assisted-by: LLM (Google:Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

This no longer merges into integration/aurora: drivers/soc/apple/clpc.c conflicts with #1405 (T6050 CPU floor, merged as 1682b11). Please merge the current integration/aurora into the branch (or rebase) and keep both the T6050 floor and the T6041 busy-share path.

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AxxzyWasTaken thanks for the rebase and the power numbers. Three things before this can go in:

  1. apple_clpc_hw_t6040 is also the M4 Pro's table. The CPU map change (DCS 1,1,3,4 and fabric 0,0,0,0 at the top, and level_khz for the E cluster) lowers the memory floor a busy CPU gets on the M4 Pro, where the comment above the table measures 237-242 GB/s at DCS 5 / fabric 2. Please give T6041 its own table and leave T6040 as it is, or add M4 Pro measurements. We can run the M4 Pro side if you split it.
  2. The light-effort governor is in the shared G17 code and on by default, so it also changes the A18 Pro and M4 Pro. A 120 fps game reads about a third busy, under the 350 entry point, and stays under the 750 exit point once it runs at half effort. Please scope it per SoC (match data, T6041 first) until it has frame-time results on games, not only power.
  3. Please replace the five g17_light_* module parameters with per-SoC values, as was done for the CLPC thresholds.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks. All three are in 33db416, on top of a merge of the current integration/aurora (e6588ee, clean).

  1. T6041 has its own CLPC table. apple_clpc_hw_t6040 is back to exactly what integration/aurora has, so the M4 Pro keeps DCS 1,2,5,5 / fabric 0,1,2,2 and its ANE cluster floor. The M4 Max shares the apple,t6041-clpc compatible, so apple_clpc_machine_hw() picks the new apple_clpc_hw_t6041 on an "apple,t6041" machine. That table carries the CPU map, level_khz, the busy-share ladder, the Low Power Mode values and the ANE rails, with the J616c measurements in its comment. It leaves out the ANE cluster floor, which was measured on the M4 Pro only.
  2. Light effort is per SoC. The G17 Config has a new light_effort: Option<power::LightEffort>. Only T6041's base config sets it, which covers both of its firmware builds. T6040, T6050, T8140 and T8142 have None, so every submission runs at full effort there and the busy share is reported unscaled.
  3. No module parameters. The five g17_light_* parameters are gone. The thresholds come from that per-SoC value, and power::configure_light() loads them when the GPU starts.

Tested: Image and modules build at W=1 with no new warnings in drivers/soc/apple or drivers/gpu/drm/asahi, and tools/asahi/clpc-gpu-demand-tests.py passes, with the busy-share test now on the T6041 table. Not run on hardware yet, and there are no frame-time results for light effort yet, so it stays limited to T6041 until I have those.

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AxxzyWasTaken thanks, this addresses all three points. The T6040 table matches the base again, the T6041 table is picked by machine compatible, and light effort is off everywhere except T6041 with no module parameters. Two things before merge:

  1. Please boot this head (33db416) on the M4 Max and post the scrolling power and a GPU smoke result. The light-effort and table code moved since your last numbers.
  2. Small race in report_busy(): two submitters that both see elapsed >= BUSY_WINDOW_NS each swap BUSY_WINDOW_TOTAL, and the second one reports about 0 permille for the same window, which on T6041 also counts as a quiet window. A compare_exchange on BUSY_WINDOW_START (only the winner reports) closes it. Optional: clearing busy_valid when the GPU goes inactive, so a new burst does not start from the share it had before a long idle.
    We will run the M4 Pro and A18 Pro side to confirm nothing changes there.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks. The race is fixed in 2a87da9: report_busy() now does a compare_exchange on BUSY_WINDOW_START, and only the submitter that wins it swaps the total and reports. I also took the optional part: apple_clpc_gpu_set_active(false) clears busy_valid, so a burst after idle starts at the top of the ladder.

Boot of this head on the M4 Max (J616c), release 7.3.0-j616c-pr1369, built with the daily kernel's config:

  • Boots. The CLPC status shows the T6041 table picked by machine compatible (dcs map 1,1,3,4, soc map 0,0,0,0). Light effort engages when the GPU is quiet (effort_full=false at idle).
  • GPU smoke. vulkan-msaa-mrt (2x, including a 6-target spilling layout), vulkan-msaa-xfer and aurora-tile-visit all PASS, plus 6 repeated tile-visit runs. No G17 errors or GPU recovery in the kernel log. aurora-r11g11b10f fails only with the Mesa build installed on the Max; with the test build it was written against, it passes on this kernel too. So it's userspace, not this PR.
  • Scrolling power: not done yet. The run was cut short when the Max was rebooted onto its daily kernel by other work. Note also that this head has no backlight device (the daily's display patches aren't in it), so its total power wouldn't be directly comparable to the earlier numbers anyway. I'll post the scrolling result from a boot that has the backlight.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Merged the current integration/aurora (677be39) in 08860ee, so this merges again. The only conflict was the apple_clpc_hw kernel-doc comment, where integration's new @high_power_machine and this branch's busy-share fields both went in; both are kept, and the T6050 floor and High Power code are untouched.

Built with W=1 at that head (drivers/soc/apple/ and drivers/gpu/drm/asahi/): no errors, no warnings in those directories. Not booted at this head.

Still open from your 13 Nov review: the scrolling power on the M4 Max. I'll post it from a boot that has the backlight.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Rebased without the two merge commits: the four commits now sit directly on integration/aurora 677be39, the base of the last merge. The final tree is identical to the previous head 08860ee, so CI's results still apply.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested on my M4 Max as described.

@joske

joske commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Tested on the J614c (16-core CPU / 40-core GPU M4 Max), at 38e1273, with #2173 for the GPU link, on top of #1664.

It boots, the GPU registers with the CLPC, and there are no new warnings. The GNOME desktop and YouTube in Firefox (software decode, about 10 W for the whole machine at brightness 54) work as before. I don't have a GPU-heavy workload, so I didn't do an A/B power test and can't confirm the savings.

It conflicts with my #1796 (the J614c's own Low Power ceiling, 2616 MHz instead of 2808). #1796 replaces .low_power_khz / .low_power_machine with a per-machine table. To combine them, I set the new T6041 table here to .low_power_machines = apple_clpc_t6041_low_power and adjusted the CLPC test script to match; both test scripts pass. Whichever lands second needs that rebase; if this one goes first, I'll rebase #1796.

🤖 Generated with Claude Code

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for this, and for the careful power measurements on the M4 Max.

One structural point before it can go in. We select behaviour by a hardware property, not by the machine it was measured on, and improvements that come from how the GPU works should apply to every chip with that GPU:

  • drivers/gpu/drm/asahi/hw/t6041.rs:57-64: the light effort is Some(...) only for T6041, while t6040.rs:114, t8140.rs:63, t8142.rs:51 and t6050.rs get None. Nothing about the quiet-window logic is specific to the M4 Max. Could it be the default for every G17 configuration? If a generation really needs it off, please add a comment that names the hardware reason.
  • drivers/soc/apple/clpc.c:1808-1809 (and .low_power_machine = "apple,t6041" at :1802): the table is picked with of_machine_is_compatible("apple,t6041"). A distinct CLPC compatible or a DT property on the CLPC node would describe this better than the SoC name.

If you make the light effort the default, we can supply A/B rows on our M1 Pro, M2, M4 Pro and A18 Pro.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks. Both points are addressed in four new commits on top.

  1. Light effort on every G17 ("drm/asahi: g17: light effort for quiet work on every G17"). power::LIGHT_DEFAULT has the values T6041 had, and every G17 config takes it: T6040, T6041, the three T6050 configs, T8140 and T8142. light_effort stays an Option, so a SoC that must run at full effort can still set None. The field's doc asks for the hardware reason at that entry; no SoC has one today. For your rows, keep in mind:

    • The values were measured on the M4 Max only, as power. There are no frame-time results from games yet.
    • On M5, the firmware's effort word gets the value only for submissions the host waits on, as before (InitData::submission_effort()).
  2. No machine compatible in the CLPC (the other three commits). The CLPC states are measurements of a SoC's memory, so they are now optional properties of the performance-controller node in apple,pmc.yaml:

    • apple,cpu-memory-states
    • apple,cpu-level-mhz
    • apple,gpu-busy-memory-states
    • apple,gpu-busy-permille

    The driver takes the compatible's table and puts the node's states in its place. apple_clpc_hw_t6041 and apple_clpc_machine_hw() are gone, and the J616c values and their measurements are in t6041.dtsi. The rest of the old T6041 table was the same as the T6040 table, so T6041 gets the same floors. The ANE cluster floor stays M4 Pro only through the T6040 table's existing ane_cluster_machine. The M4 Pro's node has none of the properties, so it keeps its own states.

    I chose properties over a new compatible because the M4 Pro and the M4 Max share the CLPC block: what differs is what was measured on each. Another SoC can now take the busy-share floors from its DT once it has rows, with no driver change.

The .low_power_machine = "apple,t6041" you pointed at (:1802) was in the T6041 table and went with it. The one left is integration's, in the T6040 table, and #1796 replaces it with apple,low-power-frequency-max-mhz.

@joske, about the order with #1796: now that #1369 has no T6041 table, there is nothing to convert to your per-machine low-power table. I test-merged #1796's head (25ab204) onto this one. The only conflict is in apple,pmc.yaml, where both PRs add properties at the same spot, and keeping both resolves it. clpc.c merges on its own. The merged clpc.o builds with W=1 with no warnings, and the CLPC demand test passes on the merged tree. So it no longer matters much which one lands first. If #1796 goes first, I'll rebase this one onto it. Thanks for testing on the J614c.

Tested:

  • drivers/soc/apple/clpc.o and drivers/gpu/drm/asahi/g17/clpc.o build with LLVM=1 W=1 with no warnings. aurora.ko builds with no warnings in the files these commits touch.
  • tools/asahi/clpc-gpu-demand-tests.py builds the T6041 table from the t6041.dtsi values through the driver's parsers and checks that they are the J616c values. The busy-share test runs on that table, with two new negative controls for unchecked cells. tools/asahi/run-host-tests.py passes, 383/383.
  • The J614c, J616c and J575c DTBs build. CHECK_DTBS gives no new warnings against the previous head, and each DTB has the four properties. The M4 Pro and T6050 DTBs have none of them. dt_binding_check passes for apple,pmc.yaml.
  • I haven't booted this head.

Disclosure: prepared with an AI tool (the commits carry Assisted-by: LLM); tested as described.

@AxxzyWasTaken AxxzyWasTaken changed the title soc: apple: clpc, drm/asahi: g17: busy-share memory floor and light effort for T6041 clpc/g17: busy-share memory floor from the DT + light effort on every G17 Nov 19, 2026
@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for the rework: the light effort is now the default on every G17 configuration and the CLPC memory states come from the DT instead of the machine compatible, which is exactly the shape we were after. We will run the A/B rows on our M4 Pro and A18 Pro against this head.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for f4adbb5. Reporting the busiest engine's share applies to every G17 and needs no per-SoC gate. The CS2 numbers explain it well: summed concurrent engine time pinned the share at 1000. Dropping the full-effort doubling is backed by the J616C measurement that the effort word does not move the clock under load.

This change reaches every G17 that feeds the CLPC, so we will run the A/B rows we offered on our M4 Pro and A18 Pro at f4adbb5 rather than 9be5ff2. Each row will cover one GPU-bound and one mixed load, with power, and we will post them here.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

New commit f4adbb5: "drm/asahi: g17: busy share from the busiest engine".

The busy share given to the CLPC was the window's summed engine time across vertex, fragment and compute, doubled at full effort. Those engines overlap, so the sum isn't a share of the window: in CS2 on the J616c, fragment and compute are 73 % and 83 % busy at once, so every game read 1000 ‰ and T6041's memory ladder sat in its top column whenever the GPU had work.

It now reports the busiest engine's share, unscaled. The doubling assumed that full effort does the same work in about half the time, but on the J616c (25G83) the effort word doesn't change the clock under load. vkalu runs at 10.8 TFLOPS, and the firmware reports its top performance state (statistics tag 0x1b), at every effort from 0x10000 down to 0x1. The light-effort governor keeps its summed measure, so nothing changes off T6041.

Rows on the J616c (old and new switchable in one test module):

test old new
CS2 fixed scene, FIFO 120: share 1000 480-620
CS2 fixed scene, FIFO 120: ladder level 3 1-2
CS2 fixed scene, FIFO 120: fps 120.0 120.0
CS2 uncapped: fps 176.1-176.5 175.8-177.1

At FIFO 120 the SoC and memory power don't change, because the P-cluster votes keep DCS at 4 in that scene. gltxf and gldepthbench give the same frame times and power in both: with a single busy engine, the old and new shares read the same.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for the f4adbb5 write-up; reporting the busiest engine's share instead of the overlapping sum makes sense, and the CS2 rows show the ladder dropping from 3 to 1-2 with no fps change.

integration/aurora has moved and GitHub now reports this branch as conflicting. Could you rebase it? We'll run the M4 Pro and A18 Pro A/B rows on the rebased head, so they match what would merge.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for the push. The 02edec4 series still sits on a base from 2026-11-16 (676 commits behind integration/aurora), so GitHub still reports a conflict and CI has not run. Could you rebase it onto the current integration/aurora? The busy-share and light-effort changes look the same as on f4adbb5, so nothing else is needed from the code side. We'll run the M4 Pro and A18 Pro A/B rows once the rebased head is clean.

The CLPC driver's per-SoC table says which DCS and fabric state reported
GPU work gets. On SoCs whose memory latency and power at each state have
been measured, scaling that floor by the GPU's busy share lets light GPU
work use lower memory states while busy GPU work keeps the top states.

Describe the optional apple,gpu-busy-memory-states and
apple,gpu-busy-permille properties on the performance-controller node.
Both properties are required together; without them, reported GPU work
gets the compatible's default floor.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken AxxzyWasTaken changed the title clpc/g17: busy-share memory floor from the DT + light effort on every G17 clpc/g17: busy-share GPU memory floor from the DT Nov 21, 2026
@AxxzyWasTaken

AxxzyWasTaken commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor Author

Rebased onto integration/aurora cacf3554 and reshaped into a clean 4-commit bisectable series (306bfb98ef7c):

Assisted-by: LLM (Google:Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 2.

I checked the earlier points at 306bfb9. They are resolved: the T6040 and T6041 tables are untouched, there are no module parameters, and the light-effort governor is gone. The window report is claimed with a compare_exchange, and apple_clpc_gpu_set_active(false) clears busy_valid. The new code has no machine or SoC gate (apple_clpc_machine_hw() is gone, and the ladder is enabled only by the node's properties). On a node without the properties, apple_clpc_gpu_input_demand() computes the same max(map top, gpu) as the base, so the M4 Pro (J616s) and A18 Pro (J700) only see an extra debugfs line and one spinlocked store every 200 ms.

  1. drivers/gpu/drm/asahi/g17/power.rs:629-649 with drivers/gpu/drm/asahi/usage.rs:43-51: the share counts engine time only when a stage retires (Client::add is "called exactly once per retired stage"). It is reported only when something is submitted (port.rs:567). A long command in flight therefore counts as 0 until it completes. One case: a compositor keeps submitting at a few percent and keeps busy_valid true at level 0, while another client runs a compute or render command lasting hundreds of ms to seconds. The T6041 GPU then sits at DCS 1 / fabric 0 for that whole command; the T6041 table comment says that is ~43 GB/s, against ~230 GB/s at DCS 5 / fabric 3. Today it gets the top floor while it is active. When the command retires, its whole time lands in one window: the share clamps to 1000, and after 400 ms (clpc.c:81) the ladder drops again for the next long command. Fix: count in-flight time. For example, report 1000 when the port has a command in flight that started before the window start; the feed already tracks the busy run (feed.busy_run()). Or charge running stages up to now_ns when the window closes. Please add this case (a long command next to a light submitter) to clpc-gpu-demand-tests.py, plus a hardware check.

  2. drivers/soc/apple/clpc.c:1470: the ladder states from the DT are only checked against 15. Nothing between apple_clpc_gpu_input_demand() and apple_pmc_floor_update() (clpc.c:1073) limits them to the rail's range, which on T6041 is DCS 0-5 and fabric 0-3. Until now the values could only come from the static tables. Now a DT typo such as <1 2 4 7> goes straight to the PMC. Please reject (or clamp) states above max(map[rail][top], gpu[rail]) of the compatible's table in apple_clpc_dt_hw(). That is the same bound apple_clpc_ane_states_set() uses at clpc.c:1218. Better, use the machine's own maximum: the ADT CLPC node has fabric-mem-separation-max (cited at clpc.c:1843), and aura already copies a CLPC ADT value into this node (src/kboot.c:10700, dt_set_clpc_low_power()). The permille thresholds are policy, not machine data, so carrying them in the DT is fine.

  3. clpc.c:1548-1551: a malformed optional ladder fails the whole CLPC probe, which drops every CPU, ANE and GPU floor and the Low Power profile on that Mac. The ladder only lowers a floor, so the safe fallback is to warn and use the compatible's table unchanged (the behaviour without the properties).

  4. arch/arm64/boot/dts/apple/t6041.dtsi:356-357: the DCS/fabric states belong to the SoC's memory, so putting them in t6041.dtsi is defensible. But the values are applied to the J614c and J575c, which have not been measured, and that includes the binned 384-bit (6-AMCC) parts. Those have less bandwidth headroom at the lower columns. Please either measure one of them or say in the commit that only the J616c is measured.

  5. Interaction with soc: apple: clpc, arm64: dts: t6041: M4 Max CLPC policy, GPU/DCS wiring, and macsmc fan lock fix #1951: T6041's CPU map is DCS 5 / fabric 3 in every column with sustain 0. Any CPU cluster sample at or above APPLE_CLPC_BUSY_PCT asks for the top (clpc.c:638), so the ladder only lowers the floor when no cluster is busy. The f4adbb5 note that "P-cluster votes keep DCS at 4" predates that. The scrolling saving needs to be shown at this head, where the CPUs now ask for DCS 5 / fabric 3, with last_reason showing HOST_GPU without CPU.

Commits: 4/4 have a human Signed-off-by and no agent trailers, and the PR body ends with an Assisted-by line. CI is still pending at 306bfb9.

HW row: at 306bfb9 (W=1 build, default cmdline), with 0 SELinux denials, 0 failed units and 0 new kernel errors on each Mac.
(a) J616c A/B against cacf355: browser scroll power at 60 and 120 Hz; CS2 (or another GPU-bound game) fps; GPU copy GB/s; CPU DRAM read GB/s; a debugfs trace of clpc status (busy permille, level, last_floor, last_reason) during each load; and one long compute command running next to the desktop, where the floor must reach DCS 5 / fabric 3 while it runs.
(b) A J614c or J575c smoke of the same loads.
(c) M4 Pro J616s and A18 Pro J700 no-regression: a GPU bench and idle power against cacf355 that match within noise, and a clpc status trace showing no ladder level and an unchanged GPU floor.

Assisted-by: LLM (Claude:Opus 5.5)

… the DT

On SoCs where the GPU driver reports work in flight to the CLPC, any
reported GPU work holds the top memory and fabric floor. At 120 Hz a
scrolling browser keeps the GPU reported active most of the time for a
few percent of real execution, so light desktop use pays for the top
DCS and fabric states.

Let the GPU driver report its busy share per window through
apple_clpc_gpu_set_busy() and, when a performance-controller node
provides apple,gpu-busy-memory-states and apple,gpu-busy-permille
(apple,pmc.yaml), pick the GPU floor from that four-column ladder. A
non-zero top DCS state enables the ladder; otherwise, and on every node
without these DT properties, reported GPU work keeps the compatible's
default top floor unchanged.

Each DT state is bounded by its rail's maximum in the compatible table
(max(map[rail][top], gpu[rail]), matching apple_clpc_ane_states_set())
as well as the 4-bit PMC state field. Because the optional ladder only
lowers a floor, malformed DT properties (bad cell count, missing
partner, out-of-range rail state or unordered thresholds) log a warning
and fall back to the compatible's static table unchanged instead of
failing CLPC probe.

A higher busy level applies at once. A lower one applies at the first
CLPC sample at least 400 ms after the last sample that asked for the
current level or a higher one. Clearing GPU activity forgets the last
reported share so a new burst after idle starts at the top column, as
does a GPU driver that reports no share. The debugfs status reports the
last busy share and, when a ladder is active, the current level without
advancing it.

Extend tools/asahi/clpc-gpu-demand-tests.py to cover the DT parser
bounds, safe fallback on malformed properties, the busy-share ladder,
the no-share and no-DT fallbacks, the 400 ms down-step hold and
negative controls.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Track the engine time charged across all clients per engine (vertex,
fragment and compute) in usage.rs, track each active QID's oldest
unretired command timestamp in g17/qos.rs, and report the busiest
engine's share of each 200 ms window to CLPC through
apple_clpc_gpu_set_busy().

The three engines run in parallel, so summing their times is not a share
of the window: in CS2 on the J616c the fragment and compute engines run
73 % and 83 % busy concurrently, which saturated a summed share at 1000
permille and pinned the memory ladder at its top column. Taking the
busiest engine's share avoids that double-counting; overlapping work
across multiple clients on the same engine still clamps to 1000 permille,
safely erring toward the higher floor.

Counting only retired stages misses a long compute or render command in
flight alongside a light submitter such as a compositor, which would
otherwise keep reporting the compositor's low share until the long
command retired. Difference (retired_ns + in_flight_age) per engine
across window boundaries so in-flight commands count while they run
without being double-counted when they retire, and lift immediately to
1000 permille (including mid-window and via idle_check rechecks when no
other client submits) once any unretired command reaches 50 ms. Reading
ENGINE_NS with Acquire ordering before locking qos pairs with
Client::add()'s Release store after qos_complete() so a concurrent
retirement cannot be counted twice. Also notify port GPU activity on
ComputeHost::note_submission() to match the render and direct paths.

Add KUnit tests in g17/power.rs and extend
tools/asahi/clpc-gpu-demand-tests.py to drive the production CLPC ladder
with the production g17 BusyShare logic for a long compute command
running beside a light compositor, along with negative controls.

Measured on the J616c (busiest-engine retired share):
- CS2, fixed de_dust2 scene, FIFO 120: busy share reads 480-620 permille
  instead of 1000, ladder steps to levels 1-2, 120.0 fps unchanged.
- CS2 uncapped: 176.6-177.1 fps against 176.1-176.5 fps.
- gltxf (3456x2234) and gldepthbench: identical 3.84 ms frame times and
  power (single busy engine reads the same).

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…n the J616c

Give the M4 Max's CLPC node the GPU busy-share memory states measured on
the 32-core J616c: DCS 1, 2, 4, 5 and fabric 0, 0, 0, 3, split at 160,
500 and 800 permille of the busy share reported by the GPU driver. The
top column matches apple_clpc_hw_t6041's DCS 5 / fabric 3 floor, so busy
GPU work keeps full memory bandwidth while light work such as scrolling
a browser drops to the lower memory states.

These values were measured on the 32-core J616c only; placing them in
t6041.dtsi also applies them to the J614c and J575c (which have not been
measured yet).

With a 3 GB/s copy on the J616c: DCS 0 / fabric 0 draws 5.46 W at 309 ns
DRAM latency, DCS 3 draws 6.22 W at 166 ns, DCS 4 draws 6.92 W at 114 ns,
and DCS 5 draws 7.04 W at 113 ns.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Pushed e0b93b21f214 (4 bisectable commits on cacf35543b4d5) resolving all 4 items from the 21 Nov review:

  1. In-flight busy-share accounting: Per-QID active_qids/qid_since_ns in g17/qos.rs with telescoping window math in BusyShare (retired delta + age - prev_age, Release/Acquire ordering so retired work is never double-counted), immediate 1000 permille lift for unretired work >= 50 ms, solo long-command recheck in idle_check(), and window reset in clear_active() so a new burst after an idle gap always starts at the top column.
  2. Rail-specific DT state bounds: apple_clpc_dt_map() checks each rail against max(hw->map[rail][LAST], hw->gpu[rail]) (matching apple_clpc_ane_states_set()) as well as 15.
  3. Safe fallback on invalid DT properties: apple_clpc_dt_apply() parses onto a copy and warns + restores the compatible's static table on any error instead of failing CLPC probe.
  4. DTS scope note: Commit 4 clarifies that states/thresholds were measured on J616c only and also apply to J614c/J575c via t6041.dtsi.

Validated with checkpatch --strict (0/0/0 across all 4 commits), W=1 build, KUnit tests, and tools/asahi/clpc-gpu-demand-tests.py (all pass including Rust/C in-flight, idle-burst, rail-bound, and fallback negative controls). Exact-head J616c hardware rows (a)-(c) are queued for our next idle test boot.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for e0b93b2. Checked against the four points from the last review:

  1. In-flight work now counts. BusyShare::sample() (drivers/gpu/drm/asahi/g17/power.rs) charges each engine's oldest unretired command by age and differences it across windows, so a command spanning a window is not counted twice when it retires, and any command running 50 ms or more lifts the share to 1000 at once. The idle-check recheck covers a lone long command with no further submissions. The KUnit cases include the long command beside a light submitter, and tools/asahi/clpc-gpu-demand-tests.py passes at this head.
  2. apple_clpc_dt_map() now rejects DT states above the compatible table's rail maximum. Taking the bound from the ADT's fabric-mem-separation-max through the boot loader can be a follow-up.
  3. A malformed or half-present ladder now warns and keeps the compatible's table, so CLPC probe no longer fails.
  4. The commit message says the states were measured on the J616c only.

A note only: in-flight age counts from publication, so a command that waits a long time on a dependency after it is published also reads as busy and gets the top floor. That is the old behaviour (top floor while active), so the cost is a missed saving, not a regression. A sentence in the BusyShare comment would be enough.

Still open: the J616c rows at this exact head, including the scroll saving with the CPU map now asking for DCS 5 / fabric 3 (last_reason showing HOST_GPU without CPU), and one long compute command next to the desktop reaching DCS 5 / fabric 3 while it runs. We will post the M4 Pro (J616s) and A18 Pro (J700) no-regression rows at e0b93b2.

Assisted-by: LLM (Claude:Opus 5.5)

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

J616C (M4 Max 16c/32g) exact-head boot & live CLPC GPU busy-share memory floor verification at e0b93b21f214 (7.3.0-j616c-pr1369+, Stage 2 1.0.0b14-g6ac0495f):

  • DT properties (performance-controller@1):
    • apple,gpu-busy-memory-states = <1 2 4 5 0 0 0 3>
    • apple,gpu-busy-permille = <160 500 800>
  • Live /sys/kernel/debug/apple-pmc/5048aa000.power-management/status progression across 5s glmark2-es2-drm terrain workload (3456x2234, 267 FPS, exit 0):
    • Idle before: agent1 floor: 0x00000000 honoured: 0x00000000
    • Ramp (t+2.0s): agent1 floor: 0x00000004 honoured: 0x00000004 (DCS state 4)
    • Full GPU load (t+3.5s): agent1 floor: 0x00000035 honoured: 0x00000035 (DCS state 5 + SoC state 3 from the DT table)
    • Idle after (t+2.0s): agent1 floor: 0x00000000 honoured: 0x00000000 (released cleanly)
  • Health: PMP running (status 0x1), GPU 25G83 32-core initialized cleanly, 0 GPU/CLPC/PMC errors, FAILED_UNITS=0, wlan0 UP.

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head J616C row at e0b93b2 shows the DT ladder driving the CLPC GPU memory floor live. All four earlier points were fixed in code before. Approving.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full review of e0b93b21f2142629c1908747ece397bf0523c167 against pre-pr-checklist.md (§1–§6): REQUEST_CHANGES (code & CI clean; held strictly on §5 evidence).

Code & checklist audit (§1–§4, §6) — clean

  • §1 No model gating: Ladder selection is driven strictly by optional DT properties (apple,gpu-busy-memory-states and apple,gpu-busy-permille in apple,pmc.yaml, dependentRequired). Static tables for T6040/T6041/T6050/T8140/T8132 are untouched, and nodes without the properties keep their exact default floor. t6041.dtsi commit message transparently notes the states were measured on J616c only and also apply to J614c/J575c.
  • §2 Production-ready / §3 Safety: Default cmdline, zero module parameters. apple_clpc_dt_map() bounds every DT state against both max(hw->map[rail][LAST], hw->gpu[rail]) and 15; apple_clpc_dt_apply() warns and falls back cleanly to the static compatible table on any malformed/partial property so CLPC probe can never fail. In g17/qos.rs, g17/port.rs, g17/power.rs, and usage.rs:
    • ENGINE_NS Release store after qos_complete() paired with Acquire load before qos.lock() prevents retirement double-counting.
    • self.qos and self.busy locks are never nested (report_busy() releases qos before locking busy), avoiding lock-ordering hazards.
    • Telescoping window accounting (delta + age - prev_age), immediate 1000 permille lift for unretired commands >= 50 ms (with solo idle_check() recheck), and clear_active() window reset after idle all handle long commands and burst transitions safely.
  • §4 Clean-room & attribution / §6 Naming: 4 clean bisectable commits (86801ace6525, 225afc294fcb, 5819ed4b3308, e0b93b21f214), all human author/committer + human Signed-off-by: only, zero AI trailers in commits; PR body carries Assisted-by:; naming clean.
  • CI: 8/8 checks SUCCESS (dt-checks, host-tests, source, rust-build, kunit, and all 3 DWC3/ATC build matrices); MERGEABLE (CLEAN), non-draft.

Remaining blocker before approval (§5 Evidence)

The exact-head J616C boot row at e0b93b21f214 (7.3.0-j616c-pr1369+) cleanly confirms the DT properties are parsed live and apple-pmc agent1 floor transitions 0x0 -> 0x4 -> 0x35 -> 0x0 across glmark2-es2-drm terrain (267 FPS, 0 GPU/CLPC/PMC errors, FAILED_UNITS=0). However, the remaining §5 items requested by @RyanTheTide at e0b93b21 are still owed before merge:

  1. Light desktop / browser scroll A/B at e0b93b21f214 vs base: Confirm power savings on current base (where #1951 set T6041's CPU map to DCS 5 / fabric 3 with sustain=0) with clpc debugfs last_reason showing HOST_GPU without CPU at the lower ladder columns.
  2. Live long-compute-beside-light-compositor check: Confirm on hardware that a >= 50 ms compute job alongside a light desktop lifts agent1 floor to 0x35 (DCS 5 / fabric 3) while in flight (plus no-regression on GPU copy / CPU DRAM bandwidth).
  3. Full regression check: Explicit 0 SELinux denials and 0 new kernel errors confirmation alongside FAILED_UNITS=0 (and M4 Pro J616s / A18 Pro J700 no-regression rows once posted).

(Non-blocking nit: the PR description still references 306bfb98ef7c and CAS reporting; refreshing it to describe e0b93b21f214's spinlocked in-flight BusyShare accounting would keep it accurate.)

Assisted-by: LLM (Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

A18 Pro (J700, T8140) no-regression row at e0b93b2: PASS, no change in behaviour, as expected.

The Neo's CLPC node (apple,t8140-clpc) has no apple,gpu-busy-memory-states / apple,gpu-busy-permille. With this PR the driver reports the busy share (host_gpu busy: N permille (none)) and keeps the default floor path.

A = fleet81 (integration/aurora 85e255d + open heads, 4bf9f69d), B = the same tree + this PR. Config identical, one boot each, silent. CLPC debugfs status was sampled every 0.2 s during GPU load (gpubench, a render-pass sweep, a dEQP-VK subset). A ran twice in the same boot, B once.

A run 1 A run 2 B
floor 0x34 / 0x0 / 0x4 / 0x3 (samples) 222 / 35 / 25 / 18 210 / 41 / 21 / 19 213 / 47 / 27 / 22
main votes (0x34 streaming GPU, with and without CPU) 123 + 94 117 + 89 104 + 102
busy share reported no no 8..1000 permille, mean 227
dEQP-VK subset (1139) 802 pass, 0 fail same same
SELinux denials / failed units / kernel errors / oops / GPU firmware crashes / CLPC errors 0 0 0

Throughput: every gpubench and render-pass value for B falls inside the spread between the two A runs. Queued, multi-process and multi-queue rows are within -3.0% to +2.2% of the A mean. Depth-1 rows are noisy; the two A runs alone differ by 9-34%. Render passes, median ms (A1 / A2 / B): 16 passes 0.395 / 0.379 / 0.381, 100 passes 3.327 / 3.408 / 3.369, 200 passes 7.073 / 7.020 / 7.062. No new build warnings.

How it was run
  • Kernels: 7.3.0-aurora-fleet81 (refs/fleet/fleet81 4bf9f69d) and 7.3.0-aurora-f81p1369 = that tree + the net diff of the clean merge of e0b93b2 (the same +/- lines as e0b93b2~4..e0b93b2). Incremental build, identical config. The asahi Rust warnings are the same 15 as the base; 0 C warnings.
  • Each kernel was installed non-default and booted once (one-shot with a dead-man), on the production Stage 2, then removed. The Mac is back on its default.
  • Rows: 12 gpubench rows (render.light/heavy, compute.light, queue/procs/queues, depths 1..64), vkpasses 16..200 render passes per command buffer with 2 frames in flight, and a dEQP-VK render-pass/dynamic-rendering/query subset (1139 cases). The CLPC status agent ... floor, last_vote ... reason_bits and host_gpu busy lines were read every 0.2 s throughout.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

§5 verification re-review at e0b93b21f2142629c1908747ece397bf0523c167: keeping prior CHANGES_REQUESTED hold in place pending the remaining J616C §5 hardware rows.

Newly verified since prior review (10:35Z)

  • A18 Pro (J700, T8140) no-regression row (comment) — PASS: Same-machine A/B at e0b93b21 cleanly proves the non-DT-ladder path (host_gpu busy: 8..1000 permille (none), default floor unchanged, gpubench + vkpasses within A1/A2 spread, dEQP-VK 1139-case subset 802 pass, 0 fail unchanged, 0 SELinux denials / 0 failed units / 0 kernel errors / 0 oops / 0 GPU firmware crashes / 0 CLPC errors, 0 new build warnings).
  • PR description: Refreshed to accurately describe e0b93b21f214 and spinlocked BusyShare accounting (prior non-blocking nit resolved).
  • Code (§1–§4, §6) & CI: Unchanged and clean (8/8 CI checks SUCCESS, MERGEABLE / CLEAN).

Remaining §5 blockers on J616C (affected T6041 hardware)

@AxxzyWasTaken's 10:05Z J616C boot & glmark2-es2-drm terrain floor-transition row (comment) was already evaluated in the 10:35Z review. As also noted as queued in the updated PR body, the remaining J616C §5 items are still owed before approval:

  1. Light desktop / browser scroll A/B at e0b93b21f214 vs base: Same-machine A/B confirming power savings on the current #1951 base (CPU map at DCS 5 / fabric 3, sustain=0), with clpc debugfs last_reason showing HOST_GPU without CPU at the lower ladder columns.
  2. Live long-compute-beside-light-compositor check: Confirm on hardware that a >= 50 ms compute job alongside a light desktop lifts agent1 floor to 0x35 (DCS 5 / fabric 3) while in flight (plus no-regression on GPU copy / CPU DRAM bandwidth).
  3. Full J616C §5 regression check: Explicit 0 SELinux denials and 0 new kernel errors confirmation alongside FAILED_UNITS=0 (currently 10:05Z only reports 0 GPU/CLPC/PMC errors and FAILED_UNITS=0).

Once those J616C rows land, this is ready for immediate approval.

Assisted-by: LLM (Gemini 4 Neon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants