Skip to content

sched/fair: start new threads and processes on an idle CPU of the creator's LLC (stacked on #2480) - #2481

Open
satchlj wants to merge 2 commits into
satchlj/sched-brief-wake-20261119from
satchlj/sched-fork-exec-llc-20261119
Open

satchlj wants to merge 2 commits into
satchlj/sched-brief-wake-20261119from
satchlj/sched-fork-exec-llc-20261119

Conversation

@satchlj

@satchlj satchlj commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Stacked on #2480. This PR adds two commits for fork placement.

With FORK_LOCAL enabled, try an idle CPU in the creator's last-level cache domain when forking a thread or process. Skip CPUs on asymmetric systems that cannot fit the new task, and fall back to normal fork balancing if no suitable local CPU is idle. The feature defaults off while the 12–16% posix_spawn regression and throughput across architectures remain open.

T6050 needs the per-cluster next-level-cache descriptions from #2574 for cluster-local placement. The T6050 DT on this base groups J775c's CPUs into a single LLC. T8103 and T8132 already describe per-cluster L2 caches and need testing with the feature enabled. The heuristic follows the scheduler's LLC domains on other architectures as well.

Evidence

Earlier J775c kernel with the cache DT, same-boot feature toggles on 2026-11-05.

Workload FORK_LOCAL off FORK_LOCAL on
fork + exit + wait 184.4 / 157.4 µs 50.6 / 50.5 µs
posix_spawn 166.2 / 150.9 µs 185.2 / 175.5 µs

Thread create+join went from 25–32 µs before the series to 8.4–9.8 µs with the combined changes. Its isolated result remains unmeasured.

Testing

  • ARM64 kernel/sched/fair.o built with W=1.
  • The revised head is untested on hardware. Compare the feature on and off on J775c with the cache DT and on one M1 or M4 Mac. Measure fork, thread and spawn latency, kernel build wall time, and hackbench. Throughput also needs qualification across architectures.

Assisted-by: LLM (OpenAI:GPT-6.2 Sol)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 3.

  1. kernel/sched/fair.c:9989-10002: fork_local_idle_cpu() takes the first idle CPU in the creator's LLC and ignores CPU capacity. The path it short-circuits does not: update_sg_wakeup_stats() (:12720-12723) marks groups the task does not fit on SD_ASYM_CPUCAPACITY systems, and wake_short_local() in the same series checks task_fits_cpu() (:9969). On Apple Silicon every cluster is its own LLC, so a child forked from an E-core with an inherited uclamp_min now stays on E-cores. Skip i when sched_asym_cpucap_active() && !task_fits_cpu(p, i), as wake_short_local() does.
  2. kernel/sched/features.h (FORK_LOCAL, true) and fair.c:10078-10083: this changes fork placement by default for every architecture and topology. That includes multi-node NUMA, where the widest SD_BALANCE_FORK domain spans nodes, and threads, which get no exec balancing to correct them. On this tree the T6050 DT has no next-level-cache (t6050-base.dtsi: 0 hits), so the only machine measured gets no effect. Meanwhile T8103 and T8132 do describe per-cluster L2 (t8103.dtsi:87 onward, t8132.dtsi: 10 next-level-cache), so the feature is live there and untested. A throughput run is also needed, because this trades spreading for locality: a make -jN or a thread-pool start, on at least one other Mac. The known 12-16% posix_spawn regression is still unexplained. Either find its cause or keep the feature off by default until it is.
  3. Both commits carry Co-Authored-By: Claude Opus 5.5. Keep only the human Signed-off-by. The disclosure goes on the PR body's last line as Assisted-by: LLM (<Vendor>:<Model>), not "Generated with Claude Code".

HW row: this head on integration/aurora, built on its own, default command line, on J775c with the T6050 cache DT and on one M1 or M4 Mac. Same boot, FORK_LOCAL on/off: osbench fork/thread/spawn, a make -j$(nproc) kernel build wall time, and hackbench. The pass criteria are no throughput loss beyond noise and an explained or fixed posix_spawn cost, plus 0 failed units, 0 AVC and 0 new kernel errors.

Assisted-by: LLM (Claude:Opus 5.5)

Fork balancing starts a new task in the idlest group of the widest domain
with SD_BALANCE_FORK. With a creating thread busy in one cluster, that is
nearly always another cluster, often one that is idle and clocked down.

On an M5 Max (T6050, three six-core clusters), traced pthread_create() +
pthread_join() iterations take 11 us from the new thread's wake-up to its
creator running again when the thread starts in its creator's cluster,
and 16 us when it starts in another one (p50 of ~10k iterations): the
thread starts later and runs longer on a cold cache, and the wake-ups
between it and its creator cross clusters. The whole create+join loop
costs 25-32 us against 12 us on macOS.

Start a new thread, a task sharing its creator's mm, on an idle CPU in the
creating CPU's LLC domain when there is one. A thread that runs for long
is still spread by load balancing; most short-lived threads finish before
that. New processes keep the usual fork and exec balancing.

On asymmetric systems, skip CPUs that cannot fit the new task,
including its utilization clamps. Fall back to normal fork balancing
if no suitable local CPU is idle.

FORK_LOCAL is disabled by default pending throughput tests across
architectures. The heuristic follows the scheduler's LLC domains.
T6050 needs next-level-cache links to describe its per-cluster caches;
without those links this base groups the J775c CPUs into one LLC.
T8103 and T8132 already describe per-cluster L2 caches and need
hardware qualification when the feature is enabled.

Signed-off-by: Satya Benson <satya@benson.earth>
FORK_LOCAL starts a new thread on an idle CPU of its creator's LLC; a new
process still goes to the idlest group of the widest domain, on an M5 Max
(T6050) nearly always another cluster, often idle and clocked down. A
fork()+_exit()+waitpid() loop (osbench) takes 171-198 us that way on
J775c, and 39 us while one cluster other than the parent's is kept busy at
full clock: most of the cost is the child running on a cold cluster, and
the parent waking from it across clusters.

Most new processes exec at once, and exec balancing then places the program
in the idlest group as before; until then the child shares its parent's
pages and cache, as a thread does. Start new processes like new threads.
Both paths use the opt-in FORK_LOCAL feature. A same-boot J775c
comparison with the cache topology present found a 12–16% posix_spawn
slowdown with the feature enabled. Keep it disabled by default until
that cost is explained and throughput is qualified across architectures.

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj
satchlj force-pushed the satchlj/sched-fork-exec-llc-20261119 branch from c0d8efc to 23e3003 Compare November 21, 2026 12:33
@satchlj
satchlj force-pushed the satchlj/sched-brief-wake-20261119 branch from dd4abcc to c86e358 Compare November 21, 2026 12:33
@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

The idle LLC scan now skips CPUs that cannot fit the child when asymmetric CPU capacity is active. FORK_LOCAL defaults off while the 12–16% posix_spawn regression and throughput across architectures are unresolved.

The PR now documents the T6050 dependency on the next-level-cache links in #2574 and the T8103/T8132 scope. Restacked both commits on the revised #2480, removed their co-author trailers, and updated the PR footer.

ARM64 kernel/sched/fair.o built with W=1. The revised head is untested on hardware. The J775c and M1/M4 latency and throughput comparisons and regression checks are pending.

@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf

J775c same-boot A/B on exact 23e300314bbe, immediate parent 4d45f47255c8, both above #2480 c86e358c4a4c. The underlying integration ancestor 26294e82e9c87a0162a7973c5e23c599ceaea22f was 370 commits behind a84637dca209 on Nov 21. The stack's changed files merge cleanly in order in a text-only check against that tip. Runtime on the combined tip is untested.

The J775c carrier DTB included the exact #2574 cache properties. Cache records covered all 18 CPUs and the private P-core L2 size is unknown. The trial disabled PCI SDHCI and used arm64.nomte console=tty0 console=ttyDC0. FORK_LOCAL booted off and was restored to off. WAKE_SHORT_LOCAL stayed on. sched_energy_aware=1 read back, but the Energy Model debugfs directory was empty and the boot journal had no registration lines.

Three independent runs per state, in off/on/on/off/off/on order for each suite. Values are medians and ranges.

Workload FORK_LOCAL off FORK_LOCAL on
Thread create/join, µs 29.642 (27.116–30.379) 12.000 (11.712–12.246)
Fork/exit/wait, µs 144.358 (136.420–151.538) 68.087 (65.307–70.094)
posix_spawn true, µs 260.399 (249.892–268.636) 220.185 (219.822–228.213)
hackbench -p -g 8, s 0.016 (0.013–0.020) 0.020 (0.019–0.023)
Clean kernel Image.gz, -j18 wall s 108.91 (108.80–110.35) 108.95 (103.36–109.27)

The earlier posix_spawn slowdown did not reproduce in this row. Its cause remains open. hackbench's median rose with overlapping ranges and 1 ms output resolution. All six clean builds passed on the same source and config. Their wall-time ranges overlapped. Build start temperatures were 42.0–56.7 °C off and 51.2–60.5 °C on.

0 AVC denials, one failed unit (ac-power-mode.service) and three boot error lines absent from Aura (two for Gen4 SPMI IRQ rejection and one from AOP probe), also seen on #2480. Two other DCP errors were also present on Aura. Verified warm handback to the default kernel, original payload hashes, both selectors and spent markers, clean ext4 and journal, both ESPs passing offline and Thunderbolt peer 2/2. Non-T6050 hardware remains untested here.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-review of 23e3003: No blockers.

Points from the last review:

  1. Capacity fit: resolved. kernel/sched/fair.c:10017-10018 skips CPUs that fail task_fits_cpu() when sched_asym_cpucap_active(), which matches wake_short_local().
  2. Default-on for every topology: resolved. kernel/sched/features.h:109 is SCHED_FEAT(FORK_LOCAL, false), and the commit messages give the T6050 next-level-cache dependency (arm64: dts: apple: t6050: give each P core a private L2 in front of the cluster L3 #2574) and the T8103/T8132 scope.
  3. Co-author trailers: resolved. Both commits carry only the human Signed-off-by, and the body ends with the Assisted-by: line.

One new point:

  1. fair.c:10014-10021: fork_local_idle_cpu() returns the first idle CPU without sched_core_cookie_match(). The fork path it short-circuits, sched_balance_find_dst_group_cpu() (fair.c:8724), skips CPUs whose core runs a different cookie. With core scheduling on (SMT systems only, so not Apple Silicon), a child forked with a cookie can now land on a core whose sibling runs a different cookie, and that sibling is then forced idle. Add if (!sched_core_cookie_match(cpu_rq(i), p)) continue; next to the fit check.

The J775c same-boot row is useful: thread create/join 29.6 -> 12.0 us, fork/exit/wait 144 -> 68 us, the earlier posix_spawn slowdown did not reproduce (260 -> 220 us), and kernel build wall time did not change. hackbench rose from 0.016 to 0.020 s, but the ranges overlap at 1 ms output resolution. While the feature is off by default this code does nothing on a stock boot. Once the row below is in, turning it on here, or in a follow-up that cites it, is what makes the win reach users.

HW row: this stack merged onto the current integration/aurora tip on the default command line, on J775c with #2574 and on one M1 or M4 Mac (where per-cluster LLCs already exist), FORK_LOCAL on/off in the same boot: osbench thread/fork/spawn, hackbench -p -g 8 -l 10000, and make -j$(nproc) wall time. Pass criteria: no throughput loss beyond noise, 0 AVC, 0 failed units, 0 new kernel errors. The trial's ac-power-mode.service failure and its SPMI/AOP error lines should be gone on the tip.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-PR checklist review (§1–§6) on head 23e300314bbe05686f3455d32453ba97e6f20294 (stacked on #2480 c86e358c4a4c).

Blockers

  1. Core-scheduling cookie violation in fork_local_idle_cpu() (kernel/sched/fair.c) — Still unfixed on 23e300314bbe. The fork path bypassed here (sched_balance_find_dst_group_cpu()) checks if (!sched_core_cookie_match(rq, p)) continue; before choosing a CPU. fork_local_idle_cpu() returns the first idle CPU in sd_llc without checking sched_core_cookie_match(cpu_rq(i), p). On SMT systems with CONFIG_SCHED_CORE, a newly forked task carrying a cookie can land on a core whose sibling runs an incompatible cookie and force-idle that sibling.
  2. SD_BALANCE_FORK domain boundary not respected in fork_local_idle_cpu() (kernel/sched/fair.c) — select_task_rq_fair() enters this slow path with sd equal to the widest domain carrying SD_BALANCE_FORK, then fork_local_idle_cpu() unconditionally scans per_cpu(sd_llc, cpu) without checking sd_llc->flags & SD_BALANCE_FORK or intersecting with sched_domain_span(sd). If SD_BALANCE_FORK is cleared on sd_llc (or a cpuset/domain boundary restricts fork balancing below LLC), this heuristic violates the sched-domain flag.
  3. Missing CI (statusCheckRollup is empty) — Zero check runs have executed on 23e300314bbe because the PR base is satchlj/sched-brief-wake-20261119 (unmerged #2480, 370+ commits behind integration/aurora) rather than integration/aurora. Once #2480 lands, rebase/retarget onto integration/aurora so host-tests runs and passes.
  4. §2 (Default command line / production readiness) & §5 (Cross-Mac hardware evidence):
    • SCHED_FEAT(FORK_LOCAL, false) is currently default-off while the earlier 12–16% posix_spawn slowdown remains unexplained (note that glibc posix_spawn uses clone(CLONE_VM | CLONE_VFORK) and shares mm, so both commits affect it) and cross-architecture throughput is unqualified. Under §2, shipping a default-off heuristic that requires manual debugfs toggling is not a production-ready user-facing improvement.
    • The J775c A/B row (23e300314bbe) ran on a stale base (370 commits behind integration/aurora), with non-default cmdline/config (arm64.nomte console=tty0 console=ttyDC0, PCI SDHCI disabled, out-of-tree #2574 cache DT), and had 1 failed unit (ac-power-mode.service) and 3 new kernel error lines (Gen4 SPMI IRQ + AOP), which does not meet §2/§5.
    • hackbench -p -g 8 median regressed from 0.016 s to 0.020 s (+25%, at 1 ms resolution); this needs a longer run (e.g. hackbench -p -g 8 -l 10000) to distinguish noise from real contention.
    • Zero non-T6050 Mac testing: M1/M2/M3/M4 Macs already describe per-cluster next-level-cache LLCs and have asymmetric E/P clusters. Note that in wake_up_new_task(), select_task_rq() runs before post_init_entity_util_avg(p), so task_util_est(p) is 0 when task_fits_cpu(p, i) runs — meaning task_fits_cpu() enforces inherited uclamp_min / capacity pressure, while unclamped children forked from an E-core parent stay on the E-cluster until load balancing or misfit migration moves them. Cross-Mac A/B on at least one asymmetric Mac (M1/M2/M4) is essential.

Non-blocking Notes

  • fork_local_idle_cpu() returns the first idle CPU in wrap order (choose_idle_cpu()), bypassing sched_balance_find_dst_group_cpu()'s shallowest-idle-state / most-recently-idled (idle->exit_latency / rq->idle_stamp) selection; worth verifying on deeper idle states.

Checklist Summary

  • §1 No model gating: Pass (uses generic sd_llc).
  • §2 Production-ready / default cmdline: Fail (default-off feature with unexplained posix_spawn regression; J775c test used arm64.nomte + disabled PCI SDHCI).
  • §3 Safety: Pass.
  • §4 Clean-room & attribution: Pass (both commits have human author/committer + human Signed-off-by: only; PR body discloses Assisted-by:).
  • §5 Evidence: Fail (stale base, 1 failed unit + 3 kernel errors, no non-T6050 M1/M2/M4 row, unresolved hackbench/spawn questions).
  • §6 Naming: Pass.

Assisted-by: LLM (Gemini 4 Neon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants