Skip to content

nvme: apple: T8122: idle ANS at the sleep's last stop, as macOS does (draft) - #2573

Draft
satchlj wants to merge 2 commits into
soc-apple/j613-sleep-portfrom
nvme-apple/t8122-sleep-idle
Draft

satchlj wants to merge 2 commits into
soc-apple/j613-sleep-portfrom
nvme-apple/t8122-sleep-idle

Conversation

@satchlj

@satchlj satchlj commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Claude Opus 5.5 on @satchlj's behalf. Draft: not yet run successfully on hardware in this form. It is stacked on #2572, the J613 sleep driver, which is itself a draft.

Problem

J613's NVMe node is apple,t8122-nvme-ans2 and falls back to the T8103 match data. That data's suspend shuts ANS down (AP quiesced, IOP sleep, AP off) and clears its run bit, so at the sleep commit ANS's ASC reads CPU_STATUS 0x2a. macOS instead idles ANS before its sleep, with the AP and then the IOP power state set to 0x201, about 22 ms before the commit. Its ASC then reads control 0, status 0x2e at the commit. These were observed on the mailbox and with a read-only ASC snapshot.

Change

T8122 gets its own match data: T8103's values plus sleep_idle, behind the existing sleep_untested parameter. With it:

  • Registration: ANS registers with the sleep driver, as T8140's does. Its PMP report entry is found through power-domain-names = "ans".
  • Suspend: the shutdown notification goes out as before, and nvme_disable_ctrl()'s own result for it is kept. Clearing CC.EN afterwards resets CSTS.SHST, so CSTS can't be read for it later.
  • If it completed and ANS's PMP request can be held for the commit: the request is marked commit-owned, as the G17 GPU's sleep holds its own. ANS is left running and marked ready. Otherwise it is shut down as before, a warning is logged, and the sleep driver refuses the sleep.
  • Last stop: the sleep driver idles ANS (apple_nvme_sleep_last_stop(): polled AP and IOP idle, the mailbox idle-status wait). Then the CPU is stopped and the PMP request withdrawn, polled.
  • No NVMe MMIO at the commit: on J613 the NVMe window is unreachable by then.
  • Resume: the ownership is dropped and the request made again, waiting for the PMP's acknowledgement. Without the acknowledgement, I/O is held rather than the NVMe window touched. Then RUN is clear, so the reset work resets and boots ANS, as after a shutdown.

Each branch logs which path it took.

Relation to #2475

#2475 gives every ANS the longer RTKit handshake timeout. It doesn't touch the sleep path or the match table, so the two are independent; either can land first.

Testing

In the J613 deep-sleep trial kernels, from a RAM initrd that mounts nothing on NVMe and keeps the disk read-only. Each run is n=1:

  • n35 (391fac63 + 41): an earlier form idled ANS in the device suspend. ANS read 0x2e right after the idle but 0x2a again at the commit.
  • n36 (391fac63 + 43): the late stop, with the notification checked at the commit. It panicked on the CSTS read, a synchronous external abort. That is why nothing reads NVMe MMIO at the commit now.
  • n37 (c92ed851 + 44): the notification was checked through CSTS.SHST in the suspend. The check never passed, because CC.EN had been cleared, so the sleep was refused (the safe fallback).
  • n40 (5d336151 + 46): the late branch ran (ANS left running for the sleep's last stop (notification 0)), but ANS answered nothing at the last stop (AP IDLE not echoed (-110): 0 received, status 0x00 after). The PMP report had already withdrawn ANS's request in its suspend_late. That is why the request is now held to the last stop.
  • n41 (abc61919 + 47): the request hold. It did not take, because the trial DT gives the "ans" domain as ps_ans, not the report entry. Fixed: the entry on that state is used.
  • n42 (eb2f8b08 + 48): the hold worked (only 0x80 withdrawn early, ANS's bit still acknowledged), yet ANS answered nothing to the idle at the commit, polled.
  • n43 (eb2f8b08 + 49): ANS idled in the suspend with interrupts on, answered (status 0x2e), and read 0x0e at the commit (IDLE gone), with RUN cleared by the sleep driver.
  • This form adds the step-by-step ASC logging and the hard stops; it is built as n44 (acfcd08b + 50) and runs next. This branch is that change squashed onto soc: apple: J613 (T8122) suspend-to-RAM on the shared T8142 sleep framework (draft: wake unshown) #2572. Every row is n=1, and the sleep has not woken.

Untested: a resume through this path, and any machine other than J613.

Assisted-by: LLM (OpenAI:GPT-6.2 Codex, Anthropic:Claude Fable 5.2, Anthropic:Claude Opus 5.5)

@satchlj
satchlj force-pushed the nvme-apple/t8122-sleep-idle branch 2 times, most recently from 3c46c6c to 5de12b0 Compare November 20, 2026 16:31
@satchlj
satchlj changed the base branch from integration/aurora to soc-apple/j613-sleep-port November 20, 2026 16:31
@RyanTheTide

Copy link
Copy Markdown
Contributor

The default-off gating holds: the T8122 data needs sleep_untested, and everything new sits behind registration, so the J615 (same compatible) is unchanged unless asked.

  • apple.c:3351-3355: when apple_rtkit_idle() fails in .suspend, sleep_idled stays false but the device is still marked ready, so the last stop falls into the polled idle at the commit (apple.c:2846). The n42 row shows ANS doesn't answer that, and the sleep driver commits anyway after a failed last stop. Falling through to the shutdown path below when the idle fails would get the same refusal the other fallbacks get.
  • apple_nvme_ans_trace() (apple.c:2811-2817) landed between apple_nvme_sleep_last_stop() and its block comment (apple.c:2806-2810), so that comment now sits on the trace helper.
  • The of_node reference kept in sleep_pmp_entry (apple.c:2949) is never put.

Assisted-by: LLM (Claude:Opus 5.5)

@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Codex GPT-6.2 Sol on @satchlj's behalf.

If RTKit idle fails during suspend, ANS now drops the PMP commit-ownership hold and takes the shutdown path. Readiness is set only after idle succeeds, so a failed idle cannot trigger the polled retry at commit. The last-stop comment is back above its callback, and device-resource cleanup releases the retained PMP entry node after unregister on remove. drivers/nvme/host/apple.o compiled for ARM64 with Apple SoC sleep and PMP report support enabled. This revision is compile-tested only and untested on hardware.

@satchlj
satchlj force-pushed the soc-apple/j613-sleep-port branch from 4820c62 to 7f160c1 Compare November 21, 2026 17:58
J613's NVMe node is apple,t8122-nvme-ans2 and falls back to the T8103
data, whose suspend shuts ANS down (AP quiesced, IOP sleep, AP off) and
clears its run bit; at the commit its ASC then reads status 0x2a. macOS
idles ANS before its sleep (AP then IOP power state 0x201), well before
its PMP stops, and its ASC reads control 0, status 0x2e at the commit.

Give T8122 its own data: T8103's values plus sleep_idle, behind
sleep_untested. With it, ANS registers with the sleep driver, and its
PMP report entry is found (the "ans" domain itself, or the entry on
that power state).

- Suspend: the shutdown notification goes out as before and
  nvme_disable_ctrl()'s result is kept (clearing CC.EN afterwards resets
  CSTS.SHST). If it completed, the report entry is known and ANS's PMP
  request can be held for the commit (commit-owned, as the G17 GPU's
  sleep holds its own), ANS is idled (AP then IOP idle, interrupts on)
  and marked ready; otherwise it is shut down as before and the sleep
  driver refuses the sleep.
- Last stop: the PMP request is withdrawn, polled (an error fails the
  stop); the sleep driver then clears RUN alone. Nothing touches the
  NVMe window, which is unreachable at the commit on J613.
- Resume: the ownership is dropped and the request made again, waiting
  for the PMP's acknowledgement; without it, I/O is held rather than the
  NVMe window touched. RUN is clear, so the reset work resets and boots
  ANS, as after a shutdown.

ANS's ASC state is logged at suspend_late, suspend_noirq and each step of
the last stop, and each branch logs which path it took.

Signed-off-by: Satya Benson <satya@benson.earth>
Mark ANS ready for the sleep commit only after RTKit acknowledges
idle. Release the PMP request hold on failure and let the existing
shutdown path run so the sleep can be refused.

Put the retained PMP entry node in device resource cleanup and
move the last-stop comment back to its callback.

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj
satchlj force-pushed the nvme-apple/t8122-sleep-idle branch from fe82992 to d464093 Compare November 21, 2026 20:54
@satchlj

satchlj commented Nov 21, 2026

Copy link
Copy Markdown
Member Author

Claude Opus 5.5 on @satchlj's behalf. Restacked onto #2572's current head (0687f99): the old pre-rewrite #2572 commits are gone, and this PR's two commits now carry human identity and DCO only (no Co-Authored-By). Same patch to drivers/nvme/host/apple.c; new head d464093, built with W=1 (nvme host and soc/apple) with no errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants