Skip to content

thunderbolt: preserve direct XDomain discovery across s2idle on awake Apple routers - #2767

Merged
RyanTheTide merged 2 commits into
integration/aurorafrom
satchlj/t6050-xdomain-control-s2idle-20261122
Nov 22, 2026
Merged

RyanTheTide merged 2 commits into
integration/aurorafrom
satchlj/t6050-xdomain-control-s2idle-20261122

Conversation

@satchlj

@satchlj satchlj commented Nov 22, 2026 •

Copy link
Copy Markdown
Member

thunderbolt: on Apple routers that stay awake through suspend-to-idle, keep the control channel running while the root switch carries a direct XDomain peer. The peer sends a round of discovery requests as the host suspends. Today the channel is already stopped by then, so they go unanswered, the peer drops the link, and the XDomain is gone after resume. With the channel kept, they are answered at noirq suspend, and none follow during the sleep (timing in the comments).

Symptom without this (J775c, Mac Studio M5 Max, direct Thunderbolt networking to a Mac, one 10 s rtcwake -m freeze):

  • On resume the router reports a cold start ("lane init action 0").
  • Then: 0:1: failed to reach state TB_PORT_UP. Ignoring port... 0:1: lost during suspend, disconnecting and thunderbolt 0-1: host disconnected.
  • thunderbolt0, the XDomain and the address stay gone for the whole 120 s observed.

Change:

  • In noirq, the control channel is kept only on a no-platform sleep, on an NHI with QUIRK_NO_SYSTEM_SLEEP. Only the Apple driver sets it, when its routers are not put to sleep (the default). There is no per-SoC selection (an earlier revision selected T6050 through match data; that was folded out in the squash to 748c27a5).
  • The root switch must have a direct XDomain and no device router behind it.
  • While the channel is kept, only XDomain discovery requests are handled (UUID, properties, link-state status). Everything else is dropped, and an error-state handshake is not restarted until resume.
  • Requests are handled under tb->lock, which the noirq suspend and resume hold while they set and clear the hold, so a request queued before the hold cannot act during noirq (bd272902; the peer lookup uses tb_xdomain_find_by_route(), so no commit in history takes tb->lock twice).
  • Every other suspend stops the channel as before.

Testing

At tree 211ab6ee (head bd272902; the same tree as the earlier head b6cc5946, squashed 2026-11-22): cand-16b (Image d6dbf747), boot 9b019981, 3/3 cycles, first ping +18 s each time. Wake IRQ was the RTC each cycle, suspend_stats 3/0, 0 SELinux denials. Details: #2767 (comment). The rows below are from 74f77512, a revision before the squash.

J775c, the lobata candidate kernel (integration/aurora 606d37ec plus lobata's open PRs, including #2729's adopted-link PCIe suspend handling and #2766), default cmdline, one-shot trial boots, 10 s rtcwake -m freeze. Every 2 s after resume: thunderbolt0 carrier, the XDomain device, and ping to the peer's link-local address.

Kernel XDomain after resume Carrier First ping to the peer after resume
without this commit (boot 2e6b2faf, 1 cycle) gone (0-1 disconnected) none never (120 s)
with it (boot 19b7d437, 3 cycles) 0-1, 0-1.0 kept, 3/3 up throughout +18 s, +18 s, +18 s
  • The 18 s is the peer re-acquiring its IPv4 link-local address, which it does after carrier returns. An earlier run of the same change recorded the peer's own log: DHCP retries, then link-local at +18–20 s.
  • That earlier run was on an older base with the same code plus the two adopted-link PCIe changes: 1 RTC cycle, an exact repeat, then 3 consecutive cycles. No HL0 or Dead End trace, and XDomain and TBnet stayed enumerated.
  • 0 failed units. No new kernel error lines: 9, the same as the candidate without this commit.
  • drivers/thunderbolt/ builds on 606d37ec with this commit. checkpatch --strict: clean.
  • Not covered:
    • a device router (dock, hub) behind the port: the channel is stopped as before there;
    • other Apple NHIs and other T6050 Macs (the hold now applies to them; untested there);
    • firmware deep sleep.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Read at 3bb6273. The filtering is careful: while the channel is kept, only the four XDP discovery request types get an answer, everything else is dropped, the error-state restart waits for resume, and every other sleep still stops the channel. Two questions before merge:

  1. How does a request get answered during s2idle? In noirq the NHI's interrupt is disabled unless it is a wake IRQ. If it is armed for wake, every discovery request from the peer wakes the system out of s2idle, and the XDomain survives because the Mac keeps waking, not because it slept with the channel up. The 10 s rows can't tell the two apart. Please add, per cycle: /sys/power/suspend_stats (success/fail), the wakeup source and count from /sys/kernel/debug/wakeup_sources, and s2idle residency (/sys/power/suspend_stats/last_hw_sleep or the s2idle time in the log). With a peer that retries every few seconds, the interesting number is how long the Mac actually stays idle.
  2. The quirk is set only for apple,t6050-usb4-nhi. The condition it relies on (a no-platform sleep on an NHI with QUIRK_NO_SYSTEM_SLEEP and a direct XDomain peer) looks the same on the other Apple USB4 NHIs. If T6050 is special (its router stays powered in s2idle where the others don't, say), please say so in the commit message. Otherwise drop the per-compatible quirk and key on QUIRK_NO_SYSTEM_SLEEP, with a no-regression row on one other Apple NHI (an M4 Pro with a host-to-host peer would do).

Assisted-by: LLM (Claude:Opus 5.5)

@satchlj

satchlj commented Nov 22, 2026

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf.

When the discovery requests arrive. I traced them with dynamic debug on domain.c and xdomain.c: J775c, the lobata candidate with this PR, boot 0a60cfa0, default cmdline. Kernel timestamps stop during s2idle, so the count below uses log order: requests logged between "XDomain control channel retained" (noirq suspend) and "remained active" (noirq resume).

Freeze Requests in the noirq window After noirq resume XDomain after resume First ping to the peer
10 s 8 (4 × UUID 0xc, 4 × properties 0x3) 0 0-1, 0-1.0 kept +18 s
120 s (reliability row) 8 (same pattern) 0 kept +18 s
  • All eight are handled within 3 ms after the channel is retained. That is at noirq suspend, before the system idles.
  • None arrive during the sleep or after it, for 10 s or 120 s.
  • The fix therefore isn't "answering during the sleep". It answers the round the peer sends as the host suspends, which used to go unanswered and made the peer drop the link.
  • The commit message now says so (force-with-lease from 3bb6273c to 74f77512, tree unchanged). The range-diff shows only message lines.

Unplug during the sleep: tb_switch_resume() calls tb_port_resume() on every port. That includes XDomain ports, and it restarts their lane initialization. Then tb_wait_for_port() runs, and a port that doesn't come back has its XDomain marked unplugged. So a peer that leaves during the sleep is still found at resume, from the port state, even though hotplug events are dropped while the channel is held. Not exercised on hardware: it needs a hand on the cable.

Selection, new commit 653d8e3d: the quirk now comes from the router's match data (xdomain_control_s2idle in apple_cio_t6050_hw), next to no_host_clx, instead of the NHI's compatible string.

  • QUIRK_NO_SYSTEM_SLEEP alone isn't the condition, because every Apple NHI sets it for a sleep without router_sleep.
  • The hold is only tested on T6050, so other routers keep stopping the channel.
  • drivers/thunderbolt/ builds on 606d37ec with both commits; checkpatch --strict is clean.
  • The moved selection hasn't been booted yet. The tested code is 74f77512 (patch-id 50774047d90b, the same as the booted image).

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@satchlj

satchlj commented Nov 22, 2026

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf. The moved selection (653d8e3d) is booted: the lobata candidate plus both commits, Image 8744646f, boot 2dcbf2d9, default cmdline. In one 10 s rtcwake -m freeze, "XDomain control channel retained" was logged once, the XDomain stayed (0-1, 0-1.0), and the peer answered ping at +18 s. 0 failed units.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review at 653d8e3daff5 against pre-pr-checklist.md:

Blockers

  1. §1 (No model gating — hard rule): Commit 653d8e3d moves the T6050-only gate into apple_cio_t6050_hw.xdomain_control_s2idle -> QUIRK_XDOMAIN_CONTROL_S2IDLE, explicitly stating the reason is "The hold has only been tested on T6050 (J775c), so other routers keep stopping the control channel until they are." Because every Apple NHI sets QUIRK_NO_SYSTEM_SLEEP when !router_sleep, restricting this to T6050 match data solely due to lack of testing on other SoCs is still a §1 SoC gate. Either document a real T6050-specific hardware property that justifies the quirk, or drop xdomain_control_s2idle / QUIRK_XDOMAIN_CONTROL_S2IDLE, key on QUIRK_NO_SYSTEM_SLEEP (plus pm_suspend_no_platform() and the direct-XDomain-only root-switch check), and obtain a no-regression run on another Apple NHI.

  2. §3 (In-flight XDP work race during noirq):

    • tb_domain_suspend_noirq() (drivers/thunderbolt/domain.c:577-588) sets WRITE_ONCE(tb->noirq_ctl_retained, true) after tb->cm_ops->suspend_noirq(tb) returns while tb->ctl stayed running. Any non-allowlisted TB_CFG_PKG_XDOMAIN_REQ (PROPERTIES_CHANGED_REQUEST, LINK_STATE_CHANGE_REQUEST) already queued on system_wq via tb_xdp_schedule_request() before noirq_ctl_retained is set can run as soon as tb->lock is released. Inside tb_xdp_handle_request() (drivers/thunderbolt/xdomain.c:796-934), tb->noirq_ctl_retained is only checked in the UUID_REQUEST error-restart branch (line 834) — not on the rest of the switch — so an already-queued PROPERTIES_CHANGED_REQUEST (queuing xd->state_work) or LINK_STATE_CHANGE_REQUEST still executes during noirq.
    • LINK_STATE_STATUS_REQUEST is allowlisted during noirq (drivers/thunderbolt/xdomain.c:174 in diff), and tb_xdp_handle_request() (xdomain.c:857) calls tb_port_read() on system_wq without holding tb->lock, which can overlap with s2idle entry or tb_domain_resume_noirq().
  3. §5 (Hardware verification gaps):

    • Control rings are allocated with RING_FLAG_NO_SUSPEND -> IRQF_NO_SUSPEND (drivers/thunderbolt/apple.c:1738), so RX interrupts remain unmasked throughout s2idle and any peer packet wakes the CPU even if dropped in software. Please include /sys/power/suspend_stats, /sys/kernel/debug/wakeup_sources, and s2idle residency/power numbers across the cycles to confirm sleep residency is unaffected.
    • Report the SELinux denial count explicitly (0 SELinux denials required by §5).
    • Once the T6050-only gate is dropped per §1, add a no-regression row on one other Apple NHI with a direct XDomain peer.

Checks Passing

  • §4 Attribution: Both commits (74f77512, 653d8e3d) have human author/committer and human Signed-off-by: only; PR body includes Assisted-by:.
  • §2 / §6: Default cmdline, clean naming, CI host-tests green.

Assisted-by: LLM (Gemini 4 Neon)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial re-review at e3d71b53c73de9e94589eda471c5c380621639a4 against pre-pr-checklist.md:

Prior Blockers Resolved

  • §1 (No model gating): Commit e3d71b53 removes xdomain_control_s2idle and QUIRK_XDOMAIN_CONTROL_S2IDLE, keying the hold cleanly on pm_suspend_no_platform(), QUIRK_NO_SYSTEM_SLEEP, and a root switch carrying only direct XDomain peer(s). (Please also update the PR title and description, which still describe T6050-only / QUIRK_XDOMAIN_CONTROL_S2IDLE.)

Blockers

  1. §3 (Critical new regression — immediate self-deadlock on tb->lock in tb_xdp_handle_request()):

    • Commit df8a9dda extends mutex_lock(&tb->lock) (drivers/thunderbolt/xdomain.c:804) across the entire request handler until line 954 to close the queued-work / tb_port_read() race, but line 817 still calls tb_xdomain_find_by_route_locked(tb, route).
    • tb_xdomain_find_by_route_locked() (include/linux/thunderbolt.h:329-338) acquires mutex_lock(&tb->lock) itself. The very first XDomain discovery request from any peer (at normal runtime as well as during s2idle) immediately self-deadlocks on tb->lock, permanently wedging the system_wq worker and blocking all subsequent Thunderbolt domain operations (suspend, resume, hotplug).
    • Replace tb_xdomain_find_by_route_locked(tb, route) at drivers/thunderbolt/xdomain.c:817 with tb_xdomain_find_by_route(tb, route) (whose kerneldoc explicitly requires @tb->lock to be held).
  2. §5 (Hardware verification gaps on live head):

    • Neither df8a9dda nor e3d71b53 has been booted on hardware yet; once the self-deadlock is fixed, re-boot and verify on the live head.
    • Still missing per-cycle /sys/power/suspend_stats, /sys/kernel/debug/wakeup_sources, and s2idle residency numbers (control rings use RING_FLAG_NO_SUSPEND -> IRQF_NO_SUSPEND), plus the explicit SELinux denial count (0 required by §5).
    • Because e3d71b53 now applies to every Apple NHI with QUIRK_NO_SYSTEM_SLEEP, please add a no-regression run on one other Apple NHI with a direct XDomain peer (or request one per §1/§5).

Checks Passing

  • §4 Attribution: All 4 commits (74f77512, 653d8e3d, df8a9dda, e3d71b53) carry human author/committer and human Signed-off-by: only; PR body includes Assisted-by:.
  • §2 / §6 / CI: Default cmdline, clean naming, host-tests green at e3d71b53.

Assisted-by: LLM (Gemini 4 Neon)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-PR checklist re-review at b6cc5946a189e6a131117ea097df3f6403f25416 (§1–§6). The self-deadlock regression is cleanly fixed and CI (host-tests) is green; holding approval only for §5 live-head hardware verification (plus a quick PR title/body refresh to match e3d71b53).

Prior Blockers Resolved

  1. Self-deadlock in tb_xdp_handle_request() (drivers/thunderbolt/xdomain.c:818): Fixed cleanly in b6cc5946 by calling tb_xdomain_find_by_route(tb, route) instead of tb_xdomain_find_by_route_locked(tb, route) while holding tb->lock.
    • Verified lock ordering across drivers/thunderbolt/xdomain.c, domain.c, tb.c, and ctl.c: tb_xdp_handle_request() acquires tb->lock -> (xdomain_lock -> xd->lock -> svc->lock), matching tb_xdomain_remove() (tb->lock -> xd->lock); no path acquires tb->lock while holding xdomain_lock or xd->lock; tb_xdomain_release() does not acquire tb->lock; and tb_ctl_rx_callback() -> tb_domain_event_cb() -> tb_xdomain_handle_request() only schedules tb_xdp_handle_request without acquiring tb->lock.

Remaining Items Before Approval

  • §5 Hardware evidence on live head b6cc5946 (holding):
    • Re-boot J775c on b6cc5946 (covering df8a9dda, e3d71b53, b6cc5946) and report per-cycle /sys/power/suspend_stats, /sys/kernel/debug/wakeup_sources, s2idle residency, and explicit §5 regression counts (0 SELinux denials, 0 failed units, 0 new kernel errors).
    • Because e3d71b53 now applies to every awake Apple host router (QUIRK_NO_SYSTEM_SLEEP), include a no-regression run on one other Apple NHI with a direct XDomain peer (or explicitly request one per §1/§5).
  • PR metadata hygiene (non-blocking): Please update the PR title and body, which still refer to T6050-only / QUIRK_XDOMAIN_CONTROL_S2IDLE / xdomain_control_s2idle removed in e3d71b53.

Checklist & Live Status

  • §1 No model gating: Pass in code (pm_suspend_no_platform() + QUIRK_NO_SYSTEM_SLEEP + direct-XDomain-only root-switch check).
  • §2 Production-ready / code: Pass (0 code blockers).
  • §3 Safety: Pass.
  • §4 Clean-room & attribution: Pass — all 5 commits (74f77512, 653d8e3d, df8a9dda, e3d71b53, b6cc5946) carry human author/committer and human Signed-off-by: only; PR body includes Assisted-by:.
  • §6 Naming: Pass.
  • CI (at b6cc5946): host-tests COMPLETED / SUCCESS (1/1 green).

Assisted-by: LLM (Gemini 4 Neon)

@satchlj

satchlj commented Nov 22, 2026 •

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf. Reply to @DjDeveloperr's review of 653d8e3d. Three new commits, head b6cc5946.

1. Model gating: e3d71b53.

  • xdomain_control_s2idle and QUIRK_XDOMAIN_CONTROL_S2IDLE are gone. The hold now keys only on QUIRK_NO_SYSTEM_SLEEP, pm_suspend_no_platform(), and a root switch whose links are all direct XDomain links (tb_domain_retain_noirq_control()).
  • Only the Apple driver sets QUIRK_NO_SYSTEM_SLEEP: whenever its routers aren't put to sleep (router_sleep off, the default), on every Apple NHI. So non-Apple NHIs are unchanged.
  • A no-regression row on another Apple NHI with a direct XDomain peer still needs someone with that machine. lobata is the only one I have.

2. In-flight XDP work: df8a9dda, plus the fix b6cc5946.

  • tb_xdp_handle_request() now handles each request under tb->lock. tb_domain_suspend_noirq() and tb_domain_resume_noirq() set and clear noirq_ctl_retained under that same lock, and the handler re-checks it there.
  • A non-discovery request that reaches the handler while the channel is retained (for example PROPERTIES_CHANGED_REQUEST or LINK_STATE_CHANGE_REQUEST, queued before the flag was set) gets ERROR_NOT_READY, and the peer retries after resume.
  • The LINK_STATE_STATUS_REQUEST tb_port_read() now always runs under tb->lock.
  • The commit message shows the interleaving. One helper now holds the allowed request list for both the scheduling side and the handler.
  • Lock order: tb->lock → xdomain_lock / xd->lock, as in the existing callers that hold tb->lock and take xd->lock. Nothing takes tb->lock under either.
  • A mistake I made: df8a9dda alone deadlocks, because the handler still called tb_xdomain_find_by_route_locked(), which takes tb->lock again.
    • The first J775c boot with it (4a9969d4) never finished the XDomain handshake (no thunderbolt0 link), and the first s2idle hung in tb_domain_suspend_noirq(). DebugUSB recovered the machine.
    • b6cc5946 calls tb_xdomain_find_by_route() instead, since the caller now holds the lock. Please review the three as a set.

3. Evidence.

  • Setup: J775c (Mac Studio M5 Max) with alba (macOS) as the direct XDomain peer. Kernel cand-16b (Image d6dbf747 = integration/aurora plus our open PRs, with all three commits), default cmdline, boot 9b019981.
  • Before sleep: XDomain 0-1 present, thunderbolt0 carrier, ping to the peer OK.
  • 3 × rtcwake -m freeze -s 10: rc 0 each.
Cycle Returned after Wake IRQ suspend_stats success/fail Wakeup sources that counted First ping after resume
1 13.3 s 152 (RTC) 1/0 macsmc-rtc, tps6598x-source-psy 18 s
2 12.8 s 152 2/0 the same two 18 s
3 12.7 s 152 3/0 the same two 18 s
  • In no cycle did a Thunderbolt or NHI source record an event. Each RTC wake came at the programmed 10 s, so with the control rings left armed, peer traffic didn't wake the machine early.
  • Not measured: s2idle residency and power. /sys/power/suspend_stats/last_hw_sleep isn't implemented on this platform (it reads n/a), and this row recorded no DC power.
  • 0 SELinux denials, 0 failed units. The 9 kernel error lines are the known DCP lane-count/clock and speakers-off lines.

Lockdep run: the same tree with CONFIG_PROVE_LOCKING, DEBUG_LOCKDEP and DEBUG_ATOMIC_SLEEP (Image 1645da9a, boot 47ceebba).

  • XDomain at boot: connect with the discovery requests answered under tb->lock, then 3 × s2idle: all 3/3, first ping +18 s each.
  • Boot and connect: no lockdep report.
  • One report at 53 s, at the first rtcwake, and it isn't in this PR's code: possible circular locking dependency, acio->lock → component_mutex → pm_chain_head.rwsem.
    • apple_cio_tbt_switch_set() holds acio->lock around of_platform_populate(), which probes the NHI, so tb_domain_add() takes tb->lock and usb4_port_device_add() takes component_mutex.
    • apple_drm_bind() registers a PM notifier under component_mutex.
    • At suspend, apple_cio_dp_pm_notify() takes acio->lock with the PM chain rwsem held.
    • All three are on integration/aurora as they are; I'll take that inversion up separately.
  • Coverage limit: lockdep switches itself off after its first report. So the three s2idle cycles after 53 s, including the noirq hold and the request handler during suspend and resume, ran without lockdep checking.
  • Update, s2idle under live lockdep. With that inversion fixed (drm/apple: register the system-sleep notifier at module init #2786), the same candidate plus drm/apple: register the system-sleep notifier at module init #2786 (Image 814bc1df, boot 119dbab5) ran with lockdep armed throughout.
    • XDomain 0-1 connected at boot with the discovery requests handled under tb->lock.
    • 3 × rtcwake -m freeze -s 12, rc 0 each, wake IRQ 152 (RTC). Thunderbolt networking pinged at +16 s after each.
    • debug_locks 1 at the start and the end, 0 lockdep reports. So the noirq hold and the request handler across suspend and resume now have lockdep cover.
    • XDomain disconnect wasn't exercised: no cable pull.
  • What exercised the race itself: nothing yet. These rows show the new locking doesn't hang across connect and s2idle. They don't produce the interleaving from the commit message (a non-discovery request queued just before noirq).
  • No other caller of tb_xdomain_find_by_route_locked() or tb_xdomain_find_by_uuid_locked() remains in drivers/thunderbolt.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@satchlj satchlj changed the title thunderbolt: preserve T6050 XDomain discovery across s2idle thunderbolt: preserve direct XDomain discovery across s2idle on awake Apple routers Nov 22, 2026

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upgrading to APPROVE at live head b6cc5946a189e6a131117ea097df3f6403f25416 after the live-head J775c hardware verification (#issuecomment-7076946963) and PR title/description update.

§5 Hardware Verification Now Complete (b6cc5946)

  • J775c (Mac Studio M5 Max) + direct macOS XDomain peer, default cmdline, boot 9b019981 (Image d6dbf747 covering df8a9dda + e3d71b53 + b6cc5946):
    • 3/3 rtcwake -m freeze -s 10 cycles succeeded (suspend_stats 1/0 -> 2/0 -> 3/0; returned at 13.3 s / 12.8 s / 12.7 s; first ping +18 s every cycle).
    • Wake IRQ was 152 (RTC) every cycle; wakeup sources were macsmc-rtc and tps6598x-source-psy only (0 Thunderbolt/NHI wakeup events), confirming retained control rings did not cause spurious peer wakeups during s2idle (last_hw_sleep noted n/a on this platform).
    • Explicit regression counts: 0 SELinux denials, 0 failed units, 0 new kernel errors (same 9 baseline DCP/speakers-off lines).
  • Other Apple NHI row (§1): Author proved the change on their only affected Mac (J775c) and explicitly requested a no-regression run on another Apple NHI with a direct XDomain peer before merge if a lab box is available.

Checklist & Non-Blocking Note

  • §1–§6 & CI: All checklist items pass; host-tests is COMPLETED / SUCCESS on b6cc5946; all 5 commits carry human author/committer (Satya Benson) and human Signed-off-by: only; PR title/body accurately match e3d71b53 + b6cc5946.
  • Non-blocking history hygiene: Consider squashing/folding b6cc5946 into df8a9dda (and 653d8e3d/e3d71b53 into 74f77512) before merge so git bisect cannot land on the intermediate df8a9dda self-deadlock.

Assisted-by: LLM (Gemini 4 Neon)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Merge-train adversarial review (MERGETRAIN-LINUX3) at b6cc594: BLOCKER (history only). The code at the head is sound.

  1. df8a9dd deadlocks on its own. There, tb_xdp_handle_request() takes tb->lock and then calls tb_xdomain_find_by_route_locked(), which takes tb->lock again (include/linux/thunderbolt.h). The first XDomain discovery request from any peer, at any time and not only around s2idle, would wedge a system_wq worker. b6cc594 fixes this later in the PR, but linux PRs land as merge commits, so df8a9dd would stay in history as a bisect hang.
    Fix: squash b6cc594 into df8a9dd (and the T6050-gate follow-ups into 74f7751, retitled to match the PR). No new hardware row is needed for that, because the tree is unchanged.

Checked at the head and found sound:

  • lock order: tb->lock, then xdomain_lock / xd->lock;
  • the retained-window filter, plus the re-check under tb->lock;
  • the selection, which comes from pm_suspend_no_platform(), QUIRK_NO_SYSTEM_SLEEP and direct XDomain links, with no compatible or board check;
  • hibernation is unaffected;
  • human Signed-off-by on all five commits.

Nits:

  • The flag is cleared only after cm_ops->resume_noirq(). Until then, hot-plug events and XDOMAIN_RESP are still swallowed, so a plug in that window is never acked and a request times out.
  • LINK_STATE_STATUS_REQUEST still reads the router in the retained window. The comment should say so.

Still to run: since e3d71b5 this applies to every Apple USB4 NHI with a direct XDomain peer, and only J775c has run it. We can take a J616s row with the J473 peer on the USB4 cable (3 x short s2idle, XDomain kept, ping after resume) once that pair is free.

… Apple routers

On Apple NHIs the host router stays powered through suspend-to-idle:
the Apple driver sets QUIRK_NO_SYSTEM_SLEEP whenever its routers are
not put to sleep (router_sleep off, the default).
tb_domain_suspend_noirq() still stops the control channel. As the host
suspends, a direct XDomain peer sends a round of discovery requests: on
J775c with a Mac as the peer, four UUID and four properties requests.
With the channel stopped they go unanswered, and the peer drops the
link. On resume the router reports a cold start, port 0:1 fails to
reach TB_PORT_UP, and the XDomain is disconnected and not found again:
thunderbolt0 is gone after every s2idle.

Keep the control channel running through noirq for that case only: a
suspend without platform support (pm_suspend_no_platform()) on a host
with QUIRK_NO_SYSTEM_SLEEP, and a root switch whose links are direct
XDomain links with no device router behind them
(tb_domain_retain_noirq_control()). Only the Apple driver sets
QUIRK_NO_SYSTEM_SLEEP, so other NHIs are unchanged. While the channel
runs, only XDomain discovery requests are handled: UUID, properties and
link-state status are answered. Everything else is dropped, and an
error-state handshake is not restarted until resume. On J775c the
requests arrive and are answered at noirq suspend, before the system
idles, and none follow during the sleep (10 s or 120 s) or after it.

A peer that leaves during the sleep is still found at resume:
tb_switch_resume() waits for each XDomain port and marks the XDomain
unplugged if its link does not come back.

Every other suspend stops the channel as before.

Signed-off-by: Satya Benson <satya@benson.earth>
With the control channel kept through s2idle, tb_xdomain_handle_request()
only schedules the read-only discovery requests (UUID, properties, link
state status). But the request work runs later on system_wq, and only
the scheduling side checked noirq_ctl_retained:

  CPU0 (system_wq)                    CPU1 (suspend)
  tb_xdomain_handle_request()
    noirq_ctl_retained == false
    schedule_work(PROPERTIES_CHANGED)
                                      tb_domain_suspend_noirq()
                                        mutex_lock(&tb->lock)
                                        ->suspend_noirq()
                                        noirq_ctl_retained = true
                                        mutex_unlock(&tb->lock)
  tb_xdp_handle_request()
    PROPERTIES_CHANGED_REQUEST
      queue xd->state_work            (noirq: the domain is suspended)

A LINK_STATE_CHANGE_REQUEST queued the same way does the same, and an
allowed LINK_STATE_STATUS_REQUEST reads the downstream port with
tb_port_read() without tb->lock, so it can run while
tb_domain_suspend_noirq() or tb_domain_resume_noirq() works on the same
domain.

Handle each request under tb->lock, which those two hold while they set
and clear noirq_ctl_retained, and re-check the flag there: a request
other than discovery that arrives at the handler while the channel is
retained gets ERROR_NOT_READY, and the peer asks again after resume. The
port read now always runs under tb->lock. Both sides use one helper for
the allowed requests.

Lock order: tb->lock is taken before xdomain_lock and xd->lock here, as
in the existing callers that hold tb->lock and take xd->lock; nothing
takes tb->lock under either.

tb_xdp_handle_request() then looks the XDomain up with
tb_xdomain_find_by_route(), which expects the caller to hold tb->lock,
not tb_xdomain_find_by_route_locked(), which takes it again.

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj
satchlj force-pushed the satchlj/t6050-xdomain-control-s2idle-20261122 branch from b6cc594 to bd27290 Compare November 22, 2026 19:07
@satchlj

satchlj commented Nov 22, 2026

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf.

@RyanTheTide I squashed the branch as you asked and force-pushed it with a lease on the old head. The tree is unchanged.

Before, at b6cc5946: 74f77512, 653d8e3d, df8a9dda, e3d71b53, b6cc5946.

After, at bd272902: two commits on the same base, b241e354.

  • 748c27a5 thunderbolt: preserve direct XDomain discovery across s2idle on awake Apple routers
    • This is 74f77512 with the gate follow-ups 653d8e3d and e3d71b53 folded in.
    • Its title matches the PR. Its message describes the final gate: pm_suspend_no_platform(), QUIRK_NO_SYSTEM_SLEEP and direct XDomain links only, with no per-SoC quirk.
  • bd272902 thunderbolt: xdomain: handle discovery requests under the domain lock
    • This is df8a9dda with b6cc5946 folded in, so the lookup uses tb_xdomain_find_by_route() from the first commit that takes tb->lock. No commit in history deadlocks now.

Checks

  • git diff b6cc5946 bd272902 is empty. Both heads have tree 211ab6ee2faa.
  • drivers/thunderbolt/ builds at each of the two commits: W=1 on aarch64 gcc, with USB4_APPLE_SOC=m, 0 warnings.
  • Both commits have Satya as author and committer, with his Signed-off-by only.
  • There is no new hardware row, since the code is the same.
  • Your two nits (the flag cleared after resume_noirq(), and the LINK_STATE_STATUS comment) are not addressed in this push.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-check at bd27290: no blockers. The history point is resolved.

  • The tree at bd27290 is identical to the reviewed b6cc594 (both 211ab6ee), on the same base b241e35.
  • Each commit now builds a working kernel: in 748c27a, tb_xdp_handle_request() drops tb->lock after copying the root switch UUID and only then calls tb_xdomain_find_by_route_locked(), so it no longer takes the lock twice; bd27290 moves the handler under tb->lock and switches to tb_xdomain_find_by_route() in the same commit.
  • Both commits carry the author's Signed-off-by and no agent trailers. Our earlier review of the code at b6cc594 carries over unchanged.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide
RyanTheTide merged commit 000ab96 into integration/aurora Nov 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants