Skip to content

arm64: dts: apple: t6050: give each P core a private L2 in front of the cluster L3 - #2574

Open
satchlj wants to merge 1 commit into
integration/aurorafrom
satchlj/t6050-cpu-caches-20261120
Open

satchlj wants to merge 1 commit into
integration/aurorafrom
satchlj/t6050-cpu-caches-20261120

Conversation

@satchlj

@satchlj satchlj commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Claude Opus 5.5 on @satchlj's behalf.

arm64: dts: apple: t6050: give each P-tier core a private L2 in front of a 16 MiB L3 shared by its cluster. This is rebased onto integration/aurora, which since #2654 describes one 16 MiB L2 per P cluster.

Why the P cores have three levels

Evidence What it shows
CLIDR_EL1 three cache levels on a P core (apple,everest: L1D, L1I, L2, L3), two on an M core (apple,sawtooth)
J775c contention sweep: a dependent-load chase pinned to cpu12 (P), alone, then with cpu13 (same P cluster) or cpu2 (M cluster) running the same chase private up to 512 KiB: 2.79 → 2.84 ns at 256 KiB, 2.83 → 2.88 ns at 512 KiB with cpu13 running. Shared from 8 MiB: 8.04 → 12.87 ns at 8 MiB, 8.55 → 27.71 ns at 12 MiB with cpu13, which is half of a shared 16 MiB. An M-cluster core changes neither (8.04 / 8.58 ns).
ADT /cpus/cpuN l2-cache-size 16 MiB on the P cores and 8 MiB on the M cores, l2-cache-id 0 for all. The ADT's "L2" on a P core is the shared cluster cache.
cacheinfo, earlier J775c boots with this layout (codex-2574-*) cpu12: L1D 128K, L1I 192K, L2 unified (size unknown) private to cpu12, L3 unified 16384K shared by 12-17

With two levels in the DT and three in CLIDR, init_cache_level() takes the larger count. cache_setup_of_node() then finds no node for the third level, and cacheinfo falls back to CLIDR alone, which drops the DT's sharing for every CPU.

Change

  • t6050-base.dtsi: T6050_P_L2(label, name, l3) is a private unified L2 whose next-level-cache is the cluster's L3. Its size is not established, so it is left out. The new T6050_P_L3 is a 16 MiB unified level-3 cache. The "Caches" comment gives the evidence.
  • t6050-max.dtsi: cpu12-17 each get their own L2, all in front of l3_cache_pc.
  • t6050-ultra.dtsi: the same for cpu12-16 and cpu30-34, one L3 per die.
  • Unchanged: the M-tier clusters and the L1 sizes.

Testing

  • All 11 T6050 DTBs build.
  • dtbs_check (dtschema 2026.9): nothing about the cache nodes.
  • checkpatch --strict: clean.
  • The decompiled J775c DTB has l3-cache-pcluster (level 3, 16 MiB) and l2-cache-pcore0..5 (level 2, next-level-cache → L3).
  • The sweep above ran on lobata's default kernel.
  • Scope of the evidence: the contention sweep measured one pair of cores, cpu12 and cpu13 in lobata's (J775c, t6050 max) only P cluster. The ultra layout (two dies, cpu12-16 and cpu30-34) is build-checked only. The sweep shows only that the private level reaches at least 512 KiB; where it ends is not measured, which is why the L2 nodes carry no size.
  • Not run: a boot of this exact head. The tip's J775c DTB is an inventory stub that doesn't boot. The cacheinfo row is from the same layout in lobata's candidate DTB.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

For the record: #2654 describes the same T6050 caches. It agrees on the L2 sizes, adds L1 sizes, and leaves out the level-3 (system level cache) node that this PR adds. The comparison is on #2654. Only one of the two should land.

Assisted-by: LLM (Claude:Opus 5.5)

@satchlj satchlj changed the title arm64: dts: apple: t6050: describe the CPU caches beyond L1 arm64: dts: apple: t6050: describe the CPU caches Nov 21, 2026
@satchlj
satchlj force-pushed the satchlj/t6050-cpu-caches-20261120 branch from 46d6842 to da98bf2 Compare November 21, 2026 04:40
@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Gemini 4 Neon on @satchlj's behalf:

Updated #2574 to fold in the verified L1 instruction and data cache sizes (128 KiB I / 64 KiB D on M-tier, 192 KiB I / 128 KiB D on P-tier) while keeping the 3-level P-core hierarchy required by CLIDR_EL1 (and updated commits/body for DCO sign-off and PR checklist compliance).

Clarification vs #2654: The cache-level = <3> node here is not the system-level cache (SLC). CLIDR_EL1 on T6050 P-cores (apple,everest) reports 3 cache levels (4 leaves: L1D, L1I, private L2, cluster-shared 16 MiB L3), whereas M-cores (apple,sawtooth) report 2 levels. If a DT stops at level 2 on P-cores (as #2654 does), drivers/base/cacheinfo.c:cache_setup_of_node() returns -ENOENT on CPU 12 and cache_setup_properties() flips the file-static global use_arch_info = true, collapsing non-L1 cache sharing back to system-wide for all 18 CPUs.

Verified on J775c (lobata, boot_id=bc6bcaf9-a41d-48b4-86fa-3ed8d88e4299, matched Image.gz 42e841af75c2... / t6050-j775c.dtb 3321a5c804c2..., 0 failed units):

  • cpu0–5: L1D=64K/<cpu> L1I=128K/<cpu> L2U=8192K/0-5
  • cpu6–11: L1D=64K/<cpu> L1I=128K/<cpu> L2U=8192K/6-11
  • cpu12–17: L1D=128K/<cpu> L1I=192K/<cpu> L2U=?/<cpu> L3U=16384K/12-17

All 11 T6050 board DTBs on this branch compile with zero dtc warnings.

@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

The 18-CPU sysfs capture came from the standalone cand-11 board DTS with the L1 properties applied. The shared include on this PR has 11-board compile coverage and still needs an exact-head hardware boot. The P-tier L2 was private per CPU, with its size unavailable.

Lobata returned to the known-good Aura kernel with a fresh boot ID. Both one-shot markers are spent, and the Thunderbolt peer answered over the direct link.

@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

J775c integration row for shared-cache definitions at da98bf2d5104. I matched the exact head's compiled cache properties to the bootable cand-11 board by CPU MPIDR, compatible and cache phandle topology, and verified that the rest of the DTB was unchanged. This is a cand-11 integration boot. The standalone PR-head J775c inventory still lacks the devices needed to bring up NVMe root and collect sysfs.

Boot d0b2d07e-e8cf-40fe-8782-49c74790382d. Image SHA-256 f08f19b670582c25027e2eee8068a9eb762b9ad3ca08cb89d4019fdf1165972b. Integrated DTB SHA-256 7b6a1eec08eae5734eb4cf5dd4577c6cd54a08b3debb45fd64e821a5d3384a4f.

  • CPU 0–11 each reported private 64 KiB L1D and 128 KiB L1I, with 8 MiB L2 shared by 0–5 and 6–11 respectively.
  • CPU 12–17 each reported private 128 KiB L1D and 192 KiB L1I, private L2 with size unavailable, and 16 MiB L3 shared by 12–17.
  • All 18 CPU rows matched. Zero failed units.

Warm reboot returned to known-good Aura with a fresh boot ID, spent trial marker, original default hashes, and clean ext4.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-review of da98bf2: No blockers.

Our earlier point, that only one of this PR and #2654 should land: settled in favour of this one. The level-3 node is the P cluster's L3 that CLIDR_EL1 reports. Without it, cache_setup_of_node() fails on the P cores and the global use_arch_info drops the DT sharing for every CPU. #2654's L1 sizes are now folded in here (da98bf2). #2654 is still open, so please close it as superseded.

What I checked at this head: every M CPU points at its cluster's L2 (t6050-max.dtsi:30-41, t6050-ultra.dtsi:30-54). Every P CPU points at its own L2 through __concat(l2_cache_, label) (t6050-base.dtsi:105-106), and that L2 points at its die's L3 (l3_cache_p or l3_cache_p_die1; t6050-max.dtsi:51, t6050-ultra.dtsi:63-76). On the Ultra the die-1 P cores 30-34 use l3_cache_p_die1 and the die-1 M cores use the _die1 L2s, so neither die's LLC spans the other. A P L2 node of a core the boot loader prunes is left unreferenced, which cacheinfo ignores.

HW row: one boot with the exact head's t6050-j775c.dtb (the shared t6050-max.dtsi, not the cand-11 board file), plus one Ultra boot (J775d). Neither the Ultra file nor its die-1 labels have booted yet. Pass: /sys/devices/system/cpu/cpu*/cache/index*/shared_cpu_list per cluster as in your J775c listing, with die-1 CPUs sharing only die-1 caches; 0 failed units, 0 new kernel errors.

Assisted-by: LLM (Claude:Opus 5.5)

@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

Supplemental J775c cache row. The unmodified Image from PR head da98bf2d51049d51cfef62c84812f57afd464b3e booted with the same bootable cand-11 cache carrier DTB as the earlier integration row. The standalone inventory DTB retains build coverage.

  • Boot ID 78a91e6d-acc4-4b3a-a4ca-73a7059b4f5e, Image SHA-256 1a3778061e08721e1a79938e4b5627251559ed62fd2ac1be176826e8209d8d13, carrier DTB SHA-256 7b6a1eec08eae5734eb4cf5dd4577c6cd54a08b3debb45fd64e821a5d3384a4f.
  • All 18 CPUs matched the expected L1 sizes and cache sharing. CPU 0–11 reported 64 KiB L1D and 128 KiB L1I, with 8 MiB L2 per six-core cluster. CPU 12–17 reported 128 KiB L1D and 192 KiB L1I, private L2 size unavailable, and 16 MiB shared L3.
  • The cache-only trial used initcall_blacklist=sdhci_driver_init to avoid the known SD power-up path on this Image. GL9755 enumerated and stayed unbound. The single failed unit was ac-power-mode.service. Its journal records permission denied writing the SMC ac_power_mode attribute.
  • Warm return to default Aura verified the original payloads, both spent markers, clean offline ESPs and ext4, zero failed units, and 2/2 direct Thunderbolt pings.

@DjDeveloperr

Copy link
Copy Markdown
Contributor

#2654 duplicated this PR and landed first (a84637d): we missed that this one was open, sorry. It describes the T6050 caches with two levels on every CPU (macOS's view, hw.perflevel0.l2cachesize = 16 MiB with cpusperl2 = 6), so it conflicts with the P-core part here.

Measured on J714s (M5 Pro) with #2654's two-level DT, on the same 7.3.0 kernel and DTB with only the cache nodes added. cpu12 is a P core:

sysfs before (no DT caches) sysfs after
cpu0 index2 level 2, shared 0-17, no size index2 8192K, shared 0-5
cpu6 index2 level 2, shared 0-17, no size index2 8192K, shared 6-11
cpu12 index2 level 2, shared 0-17; index3 level 3, shared 12-17 (CLIDR's three levels) index0-index2 only: 128K D / 192K I, then 16384K shared 12-17

So with a DT that stops at level 2 on the P cores, per-cluster sharing stays intact on this kernel. init_of_cache_level() takes the DT's levels and leaves, the CLIDR path doesn't run, and use_arch_info stays false. There were no cacheinfo warnings, and the kernel error set was the same on both boots.

The private P-core level is real, and both of our curves show it. Dependent-load latency on J714s cpu12 (Linux), with macOS-HV on the same machine:

Range Linux macOS-HV
L1, ≤128 KiB 0.71 ns 0.66 ns
Private level, 160-768 KiB 2.25-2.83 ns 2.09-2.63 ns
Cluster cache, 1.5-10 MiB 7.7-8.8 ns 7.1-8.8 ns

Your description matches CLIDR. If you want the P cores described as a private L2 plus a 16 MiB cluster L3, rebasing this PR as a P-core-only delta on top of #2654 would be the change. That's your call. The M-tier part and the L1 sizes are already in.

…he cluster L3

The P-tier (apple,everest) cores are described with one 16 MiB L2 per
cluster. The hardware has one more level there:

- CLIDR_EL1 reports three cache levels on a P core (L1D, L1I, L2, L3)
  and two on an M-tier core.
- On J775c a dependent-load sweep pinned to one P core is not slowed by
  a second P core of the same cluster running the same sweep up to
  512 KiB, so that level is private, but is from 8 MiB on (12.9 vs
  8.0 ns at 8 MiB, 27.7 vs 8.6 ns at 12 MiB), half of a shared 16 MiB.
  An M-tier core running the sweep slows neither.
- The ADT's l2-cache-size of 16 MiB on the P cores is that shared
  cluster cache.

With two levels in the DT and three in CLIDR, cacheinfo counts the
larger, finds no DT node for the third and falls back to CLIDR alone,
which drops the DT's sharing for every CPU. Describe a private unified
L2 per P core (size not established, left out) whose next level is a
16 MiB unified L3 per P cluster, in the shared T6050 dtsi and in
t6050-max.dtsi and t6050-ultra.dtsi (both dies). The M-tier clusters
are unchanged.

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj
satchlj force-pushed the satchlj/t6050-cpu-caches-20261120 branch from da98bf2 to 84983be Compare November 22, 2026 13:03
@satchlj satchlj changed the title arm64: dts: apple: t6050: describe the CPU caches arm64: dts: apple: t6050: give each P core a private L2 in front of the cluster L3 Nov 22, 2026
@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-review of 84983be, now a single commit on top of #2654's merged nodes. The structure reads right. M-tier cores keep their 8 MiB cluster L2. Each P core gets a private cache-level = <2> node, without a size since none is established, whose next-level-cache is a 16 MiB cache-level = <3> node per P cluster. That is changed consistently in t6050-base, t6050-max and both dies of t6050-ultra. With three DT levels on the P cores, cacheinfo's level count matches CLIDR, so the DT sharing is no longer thrown away for every CPU. The sweep numbers in the message (private to 512 KiB, shared from 8 MiB) support the split.

Since #2654 landed with the two-level view, please post one sysfs row at this head on a T6050 machine (J775c or J714s). For one M core and one P core, give index*/level, type, size and shared_cpu_list, plus dmesg | grep -i cacheinfo. I'd expect P cores to show index2 private, index3 shared with their cluster, and no fallback message. With that row this is good to go from my side.

Assisted-by: LLM (Claude:Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants