Skip to content

PCI: apple: hold T6050 PCIe-C power domain across s2idle - #2739

Merged
RyanTheTide merged 1 commit into
integration/aurorafrom
djdev/t6050-pciec-s2idle-hold-domain
Nov 22, 2026
Merged

RyanTheTide merged 1 commit into
integration/aurorafrom
djdev/t6050-pciec-s2idle-hold-domain

Conversation

@DjDeveloperr

Copy link
Copy Markdown
Contributor

Summary

On T6050 PCIe-C tunnel hosts (1c70000000.pcie), ps_atcN_pcie is not always-on (power_retained is false) and the sibling DART is bound as cio_dart for native CIO sleep transactions.

During PM_SUSPEND_TO_IDLE (s2idle), the NHI leaves the tunnel running and does not start a CIO sleep transaction, expecting the host to retain its link through apple_pcie_keep_link(). Because t6050_pciec_hw omitted .s2idle_hold_domain = true (unlike t8132_pciec_hw and t6040_pciec_hw), apple_pcie_keep_link() returned false without holding the power domain, causing apple_pcie_suspend_noirq() to abort late with -EBUSY (genpd_suspend_noirq returns -16) after endpoints had already entered D3hot.

This PR:

  1. Sets .s2idle_hold_domain = true on t6050_pciec_hw in drivers/pci/controller/pcie-apple.c so s2idle vetoes GENPD_NOTIFY_PRE_OFF on ps_atcN_pcie and releases the hold on resume, leaving PM_SUSPEND_MEM (deep) CIO sleep transactions unchanged.
  2. Extends apple_pcie_prepare() to refuse early with -EBUSY when a cio_dart host without an NHI sleep transaction has neither a retained power domain nor s2idle_hold_domain, matching the condition under which apple_pcie_keep_link() cannot retain the link in suspend_noirq.
  3. Extends tools/asahi/pciec-policy-tests.py to cover T6050 s2idle domain holding/release, deep-sleep pass-through, and early refusal in apple_pcie_prepare() with negative controls.

Verification

Host checks (head 075bd14d3dec)

  • scripts/checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks.
  • python3 tools/asahi/pciec-policy-tests.py: PASS (including 6/6 negative controls killed).
  • arm64 build of drivers/pci/controller/pcie-apple.o: 0 warnings.

Hardware row — Apple M5 Pro (J714s / T6050)

  • Before this patch (stock 7.3.0-m5-sship7 cc145705c592 with active PCIe-C tunnel): 3/3 rtcwake -m freeze cycles aborted in noirq with pcie-apple 1c70000000.pcie: PM: dpm_run_callback(): genpd_suspend_noirq returns -16.
  • With this patch (075bd14d3dec) on base cc145705c592 + minimal T6050 s2idle stack (arm64: dts: apple: t6050: wire the PMIC RTC alarm interrupt #2007 36ef4f8b60cf incl. [Joshua's agent] spmi: apple: gen4 interrupts and the gen4 mask bank (draft) #1743, arm64: dts: apple: t6050: keep the ATC USB power domains on in system sleep #2592 a142752404ed, PCI: apple: keep a boot-trained adopted link in D0 through suspend #2557 f90f377c8c5e, phy: apple: atc-t6050: keep active combo PHY domains powered across s2idle #2664 6fb5e78f9b46, rebuilt t6050-j714s.dtb with MTE enabled and no_console_suspend ignore_loglevel for serial capture; Dell Thunderbolt dock attached on active PCIe-C tunnel):
    • s2idle (rtcwake -d rtc1 -m freeze -s 15): 3/3 PASS
      • Cycle 1: rc=0, asleep 14.5 s (rtc_delta=20 s), macsmc-rtc IRQ 118 0 -> 1, pciec_ebusy=0, refusal=0, disp=3/3, usb=20, wifi=1, bt=1, nvme=1, aer=0, link_err=0, oops=0, dart=0, avc=0, gpu_lost=0.
      • Cycle 2: rc=0, asleep 14.0 s (rtc_delta=20 s), macsmc-rtc IRQ 118 1 -> 2, pciec_ebusy=0, refusal=0, disp=3/3, usb=20, wifi=1, bt=1, nvme=1, aer=0, link_err=0, oops=0, dart=0, avc=0, gpu_lost=0.
      • Cycle 3: rc=0, asleep 14.6 s (rtc_delta=20 s), macsmc-rtc IRQ 118 2 -> 3, pciec_ebusy=0, refusal=0, disp=3/3, usb=20, wifi=1, bt=1, nvme=1, aer=0, link_err=0, oops=0, dart=0, avc=0, gpu_lost=0.
    • deep regression (rtcwake -d rtc0 -m mem -s 15): 1/1 PASS
      • Cycle 1: rc=0, asleep 13.1 s (rtc_delta=23 s), pcie-apple 1c70000000.pcie: CIO PCIe-C stop (T6050 order) -> PME_Turn_Off L2 reported -> CIO PCIe-C link and DMA restored, pciec_ebusy=0, refusal=0, disp=3/3, usb=20, wifi=1, bt=1, nvme=1, aer=0, oops=0, dart=0, avc=0, gpu_lost=0.
    • Regression line: suspend_stats success=0 -> 4, fail=0; GPU LIVE (vulkaninfo + compute dispatch) before and after all 4 cycles; 0 kernel oops/WARN, 0 DART faults, 0 SELinux AVCs, 0 new failed systemd units (only pre-existing offline tailscaled.service).

Assisted-by: LLM (Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

An adversarial read of 075bd14. The fix is right: T6050's PCIe-C host was the one tunnel host without s2idle_hold_domain, so apple_pcie_keep_link() gave up before even trying the hold. The before/after (3/3 noirq -EBUSY, then 3/3 s2idle plus a clean deep sleep with the dock) shows it. s2idle_keep_link defaults on, so this works on the default command line.

One gap, the same shape as #2729: on a cio_dart host, a false from apple_pcie_keep_link() still ends in a late -EBUSY from suspend_noirq, after the endpoints are in D3hot. The new .prepare check covers only the power-domain term. Several of the other terms are known before any device suspends:

  • !(kernel_init || tunnel_cold_init), i.e. a PCIe-C host whose tunnel never came up, such as a J714s with nothing on its Thunderbolt ports;
  • bus_stopped / resume_failed.

These could go into the same .prepare refusal through a shared helper, leaving only the hold failure and the live link-health read to noirq. Can you add one row with no dock attached (no PCIe-C tunnel ever brought up)? It would show whether s2idle on an undocked J714s refuses in .prepare, refuses late, or succeeds. That's the configuration most users will sleep in.

Minor, non-blocking: s2idle_hold_domain is a per-SoC hw_info field, consistent with T8132/T6040. Whether the domain may be held is arguably a property of the tunnel's power domain that the DT could carry, which is a candidate for later gate cleanup rather than this PR.

Assisted-by: LLM (Claude:Opus 5.5)

On T6050 PCIe-C tunnel hosts, the ps_atcN_pcie power domain is not
always-on (power_retained is false) and the sibling DART is bound as
cio_dart for native CIO sleep transactions.

During suspend-to-idle (PM_SUSPEND_TO_IDLE), the NHI leaves the tunnel
running and does not start a CIO sleep transaction, expecting the host
to keep its link via apple_pcie_keep_link(). Because t6050_pciec_hw
omitted .s2idle_hold_domain = true (unlike t8132_pciec_hw and
t6040_pciec_hw), apple_pcie_keep_link() returned false without holding
the domain, causing apple_pcie_suspend_noirq() to abort late with
-EBUSY after endpoints had already entered D3hot.

Set .s2idle_hold_domain = true on t6050_pciec_hw so s2idle vetoes
GENPD_NOTIFY_PRE_OFF on ps_atcN_pcie and releases the hold on resume,
leaving PM_SUSPEND_MEM (deep sleep) CIO transactions untouched.

Factor the pre-suspend link-retention policy out of apple_pcie_keep_link()
into apple_pcie_can_keep_link() (checking s2idle_keep_link,
pm_suspend_no_platform(), kernel_init || tunnel_cold_init,
!bus_stopped && !resume_failed, and power_retained || s2idle_hold_domain)
and share it with apple_pcie_prepare() and apple_pcie_suspend_noirq().
That refuses early with -EBUSY before any endpoint suspends whenever a
cio_dart host has no NHI sleep transaction and cannot retain its link,
leaving only the domain hold and live link-health check to noirq.

Extend tools/asahi/pciec-policy-tests.py to cover T6050 s2idle domain
holding and release, deep-sleep pass-through, and every pre-suspend
refusal term in apple_pcie_can_keep_link() / apple_pcie_prepare(), with
negative controls.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr
DjDeveloperr force-pushed the djdev/t6050-pciec-s2idle-hold-domain branch from 075bd14 to aedd024 Compare November 21, 2026 22:08
@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks. Checked aedd024: apple_pcie_can_keep_link() now gives .prepare, apple_pcie_keep_link() and the noirq CIO-DART check one set of terms, leaving only the hold and the live link read to noirq. That closes the late-refusal path.

One consequence to decide on deliberately: with the tunnel never brought up (!(kernel_init || tunnel_cold_init), i.e. an undocked J714s), a cio_dart host now refuses every s2idle in .prepare. That is safer than the late noirq -EBUSY it replaces, but there's no link to keep and no endpoint below that could be lost, so refusing the whole sleep for it looks stricter than needed. If an idle PCIe-C host can simply be left to genpd (or skipped) when no tunnel ever came up, undocked s2idle would work. The no-dock row would show which behaviour you get today. If it refuses, a sentence saying that is intended (and why) would help users who see no CIO sleep transaction from the NHI; refusing sleep on an undocked laptop.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr

Copy link
Copy Markdown
Contributor Author

Tested head aedd024e0b74 (with the shared apple_pcie_can_keep_link() helper; CI 2/2 green: Host tests/host-tests + MT7932 KUnit/kunit, checkpatch --strict 0/0/0, pciec-policy-tests.py PASS with 10/10 negative controls killed) on Apple M5 Pro (J714s / T6050) across all four scenarios:

1. Why an undocked J714s does not refuse in .prepare

On T6050, pcie-tunnel-{0,1,2} are plain container nodes without compatible; of_platform_default_populate() never descends into them at boot. apple_cio_probe() registers typec_thunderbolt_switch without populating pcie-tunnel-N. Only when a USB4/Thunderbolt PCIe tunnel is activated does apple_cio_populate_pcie_tunnel() call of_platform_populate(acio->pcie_tunnel_np, ...), and on unplug apple_cio_remove_children() depopulates it.

Therefore, on an undocked J714s (no Thunderbolt dock / no PCIe tunnel brought up), zero apple,t6050-pciec platform devices exist (pciec_bound=0), apple_pcie_prepare() never runs for a PCIe-C host, and s2idle succeeds. The newly shared terms in apple_pcie_can_keep_link() fire only on a bound cio_dart host whose link cannot be kept (!(kernel_init || tunnel_cold_init) bootloader handoff, bus_stopped after a deactivated PCIe tunnel while the cable remains connected, or resume_failed), turning late noirq -EBUSY into an early .prepare refusal.

2. J714s 4-row hardware proof on exact head aedd024e0b74

Stack: 7.3.0 (cc145705c592) + #2739 (aedd024e0b74) + #2007 (36ef4f8b60cf, incl. #1743) + #2592 (a142752404ed) + #2557 (f90f377c8c5e) + #2664 (6fb5e78f9b46), rebuilt t6050-j714s.dtb (1fbcc4fe2c55, MTE enabled, no_console_suspend ignore_loglevel for serial capture). Note 2a9ac0000.acio (acio1) is natively idle-undocked throughout all four rows (1c60000000.pcie never bound).

Row Configuration Command Result Key evidence
A (Docked active PCIe-C) 1c70000000.pcie bound, active TB5 dock tunnel (pciec_bound=1) rtcwake -d rtc1 -m freeze -s 15 PASS (rc=0) asleep=14.2 s (rtc_delta=20 s), macsmc-rtc IRQ 118 0 -> 1, pciec_ebusy=0, disp=3/3, usb=20, wifi=1, bt=1, nvme=1, 0 AER/link_err/oops/DART/AVC/gpu_lost
B (Bound cio_dart + bus_stopped=true) echo 0 > /sys/bus/thunderbolt/devices/0-1/authorized (1c70000000.pcie still bound, quiesced) rtcwake -d rtc1 -m freeze -s 15 PASS (early .prepare -EBUSY) pcie-apple 1c70000000.pcie: no CIO sleep transaction from the NHI; refusing sleep -> device_prepare(): genpd_prepare returns -16, last_failed_errno=-16, last_failed_step=prepare, noirq_abort=0, 0 oops/DART/AVC/gpu_lost
C (Undocked / zero PCIe-C hosts bound) 381ac0000.acio unbound via sysfs (pciec_bound=0; 2a9ac0000.acio idle-undocked; note dock stays physically plugged in remote lab so acio2 was unbound in software and cleanly rebound after wake) rtcwake -d rtc1 -m freeze -s 15 PASS (rc=0) unbind_rc=0, pciec_bound=0, asleep=16.0 s, macsmc-rtc IRQ 118 1 -> 2, suspend_stats success=2 -> 3, pciec_ebusy=0, refusal=0, bind_rc=0, 0 oops/DART/AVC/gpu_lost
D (deep regression) Docked (pciec_bound=1) rtcwake -d rtc0 -m mem -s 15 PASS (rc=0) asleep=13.5 s (rtc_delta=23 s), disp=3/3, usb=20, wifi=1, bt=1, nvme=1, 0 AER/oops/DART/AVC/gpu_lost
  • Regression line: GPU LIVE (vulkaninfo + compute dispatch) before and after all 4 rows; 0 kernel oops/WARN, 0 DART faults, 0 SELinux AVCs, 0 new failed systemd units (only pre-existing offline tailscaled.service).

Assisted-by: LLM (Gemini 4 Neon)

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at aedd024.

  • The undocked case is answered: the pcie-tunnel-N nodes have no compatible, so a PCIe-C host only exists after apple_cio_populate_pcie_tunnel(), and an undocked machine has no apple,t6050-pciec device for apple_pcie_prepare() to refuse on. Row C exercises the same depopulate path through the ACIO unbind and sleeps cleanly; rows A, B and D cover the kept link, the early refusal and deep.
  • apple_pcie_can_keep_link() now gives prepare and noirq the same answer, so the late noirq -EBUSY after endpoints have suspended is gone for the static cases. The only thing left in noirq is the domain hold and the live link check, which is where it belongs.
  • Small follow-up, not blocking: the prepare warning still says "no CIO sleep transaction from the NHI" when the actual reason is bus_stopped or resume_failed (row B prints it after a de-authorised tunnel). Naming the reason would save the next person a detour.

Approve.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide
RyanTheTide merged commit 9fcf638 into integration/aurora Nov 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants