Repository navigation
sched/fair: start new threads and processes on an idle CPU of the creator's LLC (stacked on #2480) - #2481
sched/fair: start new threads and processes on an idle CPU of the creator's LLC (stacked on #2480)#2481satchlj wants to merge 2 commits into
Conversation
|
Blockers: 3.
HW row: this head on Assisted-by: LLM (Claude:Opus 5.5) |
Fork balancing starts a new task in the idlest group of the widest domain with SD_BALANCE_FORK. With a creating thread busy in one cluster, that is nearly always another cluster, often one that is idle and clocked down. On an M5 Max (T6050, three six-core clusters), traced pthread_create() + pthread_join() iterations take 11 us from the new thread's wake-up to its creator running again when the thread starts in its creator's cluster, and 16 us when it starts in another one (p50 of ~10k iterations): the thread starts later and runs longer on a cold cache, and the wake-ups between it and its creator cross clusters. The whole create+join loop costs 25-32 us against 12 us on macOS. Start a new thread, a task sharing its creator's mm, on an idle CPU in the creating CPU's LLC domain when there is one. A thread that runs for long is still spread by load balancing; most short-lived threads finish before that. New processes keep the usual fork and exec balancing. On asymmetric systems, skip CPUs that cannot fit the new task, including its utilization clamps. Fall back to normal fork balancing if no suitable local CPU is idle. FORK_LOCAL is disabled by default pending throughput tests across architectures. The heuristic follows the scheduler's LLC domains. T6050 needs next-level-cache links to describe its per-cluster caches; without those links this base groups the J775c CPUs into one LLC. T8103 and T8132 already describe per-cluster L2 caches and need hardware qualification when the feature is enabled. Signed-off-by: Satya Benson <satya@benson.earth>
FORK_LOCAL starts a new thread on an idle CPU of its creator's LLC; a new process still goes to the idlest group of the widest domain, on an M5 Max (T6050) nearly always another cluster, often idle and clocked down. A fork()+_exit()+waitpid() loop (osbench) takes 171-198 us that way on J775c, and 39 us while one cluster other than the parent's is kept busy at full clock: most of the cost is the child running on a cold cluster, and the parent waking from it across clusters. Most new processes exec at once, and exec balancing then places the program in the idlest group as before; until then the child shares its parent's pages and cache, as a thread does. Start new processes like new threads. Both paths use the opt-in FORK_LOCAL feature. A same-boot J775c comparison with the cache topology present found a 12–16% posix_spawn slowdown with the feature enabled. Keep it disabled by default until that cost is explained and throughput is qualified across architectures. Signed-off-by: Satya Benson <satya@benson.earth>
c0d8efc to
23e3003
Compare
dd4abcc to
c86e358
Compare
|
Codex GPT-6.2 Sol on @satchlj's behalf The idle LLC scan now skips CPUs that cannot fit the child when asymmetric CPU capacity is active. The PR now documents the T6050 dependency on the ARM64 |
|
Codex GPT-6.2 Sol on @satchlj's behalf J775c same-boot A/B on exact The J775c carrier DTB included the exact #2574 cache properties. Cache records covered all 18 CPUs and the private P-core L2 size is unknown. The trial disabled PCI SDHCI and used Three independent runs per state, in off/on/on/off/off/on order for each suite. Values are medians and ranges.
The earlier posix_spawn slowdown did not reproduce in this row. Its cause remains open. hackbench's median rose with overlapping ranges and 1 ms output resolution. All six clean builds passed on the same source and config. Their wall-time ranges overlapped. Build start temperatures were 42.0–56.7 °C off and 51.2–60.5 °C on. 0 AVC denials, one failed unit ( |
|
Re-review of 23e3003: No blockers. Points from the last review:
One new point:
The J775c same-boot row is useful: thread create/join 29.6 -> 12.0 us, fork/exit/wait 144 -> 68 us, the earlier HW row: this stack merged onto the current Assisted-by: LLM (Claude:Opus 5.5) |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Pre-PR checklist review (§1–§6) on head 23e300314bbe05686f3455d32453ba97e6f20294 (stacked on #2480 c86e358c4a4c).
Blockers
- Core-scheduling cookie violation in
fork_local_idle_cpu()(kernel/sched/fair.c) — Still unfixed on23e300314bbe. The fork path bypassed here (sched_balance_find_dst_group_cpu()) checksif (!sched_core_cookie_match(rq, p)) continue;before choosing a CPU.fork_local_idle_cpu()returns the first idle CPU insd_llcwithout checkingsched_core_cookie_match(cpu_rq(i), p). On SMT systems withCONFIG_SCHED_CORE, a newly forked task carrying a cookie can land on a core whose sibling runs an incompatible cookie and force-idle that sibling. SD_BALANCE_FORKdomain boundary not respected infork_local_idle_cpu()(kernel/sched/fair.c) —select_task_rq_fair()enters this slow path withsdequal to the widest domain carryingSD_BALANCE_FORK, thenfork_local_idle_cpu()unconditionally scansper_cpu(sd_llc, cpu)without checkingsd_llc->flags & SD_BALANCE_FORKor intersecting withsched_domain_span(sd). IfSD_BALANCE_FORKis cleared onsd_llc(or a cpuset/domain boundary restricts fork balancing below LLC), this heuristic violates the sched-domain flag.- Missing CI (
statusCheckRollupis empty) — Zero check runs have executed on23e300314bbebecause the PR base issatchlj/sched-brief-wake-20261119(unmerged #2480, 370+ commits behindintegration/aurora) rather thanintegration/aurora. Once #2480 lands, rebase/retarget ontointegration/aurorasohost-testsruns and passes. - §2 (Default command line / production readiness) & §5 (Cross-Mac hardware evidence):
SCHED_FEAT(FORK_LOCAL, false)is currently default-off while the earlier 12–16%posix_spawnslowdown remains unexplained (note that glibcposix_spawnusesclone(CLONE_VM | CLONE_VFORK)and sharesmm, so both commits affect it) and cross-architecture throughput is unqualified. Under §2, shipping a default-off heuristic that requires manual debugfs toggling is not a production-ready user-facing improvement.- The J775c A/B row (
23e300314bbe) ran on a stale base (370 commits behindintegration/aurora), with non-default cmdline/config (arm64.nomte console=tty0 console=ttyDC0, PCI SDHCI disabled, out-of-tree #2574 cache DT), and had 1 failed unit (ac-power-mode.service) and 3 new kernel error lines (Gen4 SPMI IRQ + AOP), which does not meet §2/§5. hackbench -p -g 8median regressed from0.016 sto0.020 s(+25%, at 1 ms resolution); this needs a longer run (e.g.hackbench -p -g 8 -l 10000) to distinguish noise from real contention.- Zero non-T6050 Mac testing: M1/M2/M3/M4 Macs already describe per-cluster
next-level-cacheLLCs and have asymmetric E/P clusters. Note that inwake_up_new_task(),select_task_rq()runs beforepost_init_entity_util_avg(p), sotask_util_est(p)is0whentask_fits_cpu(p, i)runs — meaningtask_fits_cpu()enforces inheriteduclamp_min/ capacity pressure, while unclamped children forked from an E-core parent stay on the E-cluster until load balancing or misfit migration moves them. Cross-Mac A/B on at least one asymmetric Mac (M1/M2/M4) is essential.
Non-blocking Notes
fork_local_idle_cpu()returns the first idle CPU in wrap order (choose_idle_cpu()), bypassingsched_balance_find_dst_group_cpu()'s shallowest-idle-state / most-recently-idled (idle->exit_latency/rq->idle_stamp) selection; worth verifying on deeper idle states.
Checklist Summary
- §1 No model gating: Pass (uses generic
sd_llc). - §2 Production-ready / default cmdline: Fail (default-off feature with unexplained
posix_spawnregression; J775c test usedarm64.nomte+ disabled PCI SDHCI). - §3 Safety: Pass.
- §4 Clean-room & attribution: Pass (both commits have human author/committer + human
Signed-off-by:only; PR body disclosesAssisted-by:). - §5 Evidence: Fail (stale base, 1 failed unit + 3 kernel errors, no non-T6050 M1/M2/M4 row, unresolved hackbench/spawn questions).
- §6 Naming: Pass.
Assisted-by: LLM (Gemini 4 Neon)
Stacked on #2480. This PR adds two commits for fork placement.
With
FORK_LOCALenabled, try an idle CPU in the creator's last-level cache domain when forking a thread or process. Skip CPUs on asymmetric systems that cannot fit the new task, and fall back to normal fork balancing if no suitable local CPU is idle. The feature defaults off while the 12–16%posix_spawnregression and throughput across architectures remain open.T6050 needs the per-cluster
next-level-cachedescriptions from #2574 for cluster-local placement. The T6050 DT on this base groups J775c's CPUs into a single LLC. T8103 and T8132 already describe per-cluster L2 caches and need testing with the feature enabled. The heuristic follows the scheduler's LLC domains on other architectures as well.Evidence
Earlier J775c kernel with the cache DT, same-boot feature toggles on 2026-11-05.
Thread create+join went from 25–32 µs before the series to 8.4–9.8 µs with the combined changes. Its isolated result remains unmeasured.
Testing
kernel/sched/fair.obuilt withW=1.Assisted-by: LLM (OpenAI:GPT-6.2 Sol)