Skip to content

accel/ane: register the PM notifier outside the driver lock (lockdep ABBA) - #2505

Merged
DjDeveloperr merged 2 commits into
integration/aurorafrom
ane/pm-notifier-outside-lock
Nov 20, 2026
Merged

DjDeveloperr merged 2 commits into
integration/aurorafrom
ane/pm-notifier-outside-lock

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

Problem

Both ANE backends register their PM notifier while holding their own lock, and the notifier takes that same lock when the PM chain calls it:

  • H16 (ane_h16_backend.c): ane_h16_pin() calls register_pm_notifier() with ane_h16_lock held (ane_h16_start_work() -> ane_h16_advance() -> ane_h16_scratch_boot()). ane_h16_pm_event() takes ane_h16_lock.
  • H17 (ane_fw_cold.c): ane_fw_retain() registers it under b->lock, from ane_fw_register_parked() and from the "map" stage. ane_fw_pm_event() takes b->lock in ane_fw_sleep_prepare() and ane_fw_sleep_resume().

That orders the PM chain rwsem after the driver lock in one path and before it in the other. With PROVE_LOCKING, the first system sleep after the engine comes up logs "possible circular locking dependency detected" and lockdep switches itself off for the rest of the boot.

Change

Two commits, one per backend, same shape:

  • Register the notifier once, at probe (H16) or ane_fw_staged_init() (H17), with no driver lock held, and before the start work that can pin or retain the device.
  • Until the device is pinned or retained, the notifier returns NOTIFY_DONE right away, which is what an unregistered notifier did. Pinning and retaining only set the flag now.
  • A device removed before it was pinned or retained unregisters the notifier, outside the lock.
  • H16: the debugfs stage write could also pin the device. It now takes the system sleep lock like the start work, so the pinned flag never changes inside a sleep transition (on H17 every retain path already holds it).

Testing

Lockdep kernels (PROVE_LOCKING, DEBUG_LOCKDEP, DEBUG_ATOMIC_SLEEP) on integration/aurora 2c35f50 with #2240 (f1623ba) and this branch merged:

Machine Row Result
MacBook Neo (J700, T8140, H17) Boot, ANE up, two/three-client ANE rows, one deep s2r (PMU RTC wake) with two programs held No lockdep report, debug_locks 1, replay identical after wake
MacBook Neo, same kernel with only ane_t6021.ko built without this branch Same deep s2r "possible circular locking dependency detected": &b->lock in ane_fw_pm_event vs (pm_chain_head).rwsem from register_pm_notifier in ane_fw_start_work (2 of 2 boots)
M4 Pro (J616s, T6040, H16) Boot, ANE up, two/three-client ANE rows, one s2idle (rtc1) with two programs held (cold replay) No lockdep report, debug_locks 1, replay identical after wake
M4 Pro, same kernel family without this branch (earlier run) One s2idle with two programs held "possible circular locking dependency detected": ane_h16_lock in ane_h16_pm_event vs (pm_chain_head).rwsem from register_pm_notifier in ane_h16_pin

checkpatch --strict is clean on both commits. The branch merges cleanly with #1768, #1863, #1899, #1957 and #2240.

Still to run

Nothing for this change.

Lockdep report before the change (H17, Neo)
WARNING: possible circular locking dependency detected
bash/3424 is trying to acquire lock:
 (&b->lock){+.+.}-{4:4}, at: ane_fw_pm_event+0xbc/0x11bc [ane_t6021]
but task is already holding lock:
 ((pm_chain_head).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain_robust+0x48/0xd0
the existing dependency chain (in reverse order) is:
-> #1 ((pm_chain_head).rwsem){++++}-{4:4}:
       down_write+0x54/0xa4
       blocking_notifier_chain_register+0x28/0x70
       register_pm_notifier+0x1c/0x28
       ane_fw_start_work+0x84/0x300 [ane_t6021]
-> #0 (&b->lock){+.+.}-{4:4}:
       mutex_lock_nested+0x24/0x30
       ane_fw_pm_event+0xbc/0x11bc [ane_t6021]
       notifier_call_chain+0xa0/0x228
       blocking_notifier_call_chain_robust+0x64/0xd0
       pm_notifier_call_chain_robust+0x24/0x40
       pm_suspend+0x174/0x41c

The H16 report on the M4 Pro has the same shape with ane_h16_lock, ane_h16_pin() and ane_h16_pm_event().

ANE rows on the Neo with this branch (lockdep on)
boot-lockdep: reports=0 debug_locks=1
two-a 180 s: 9183 x (load, run, close) of a 64-lane add, 0 errors, 0 mismatches; resident MobileNet 37554 runs, 0 errors
two-b 180 s, 3 clients: add loop 3544 cycles + MobileNet loop 3193 cycles + resident MobileNet 12049 runs, 0 errors
hold-sleep: add + MobileNet held across one deep s2r, replay identical, "resume ok: parked until the next request", lockdep 0
two-c 120 s after the wake: 6151 cycles + 25642 resident runs, 0 errors
every row: WARN 0, oops 0, ENOSPC 0, short unmap 0, quarantine 0, DART fault 0; AVC 0

Assisted-by: LLM (Claude:Opus 5.5)

…lock

ane_h16_pin() registered the PM notifier while ane_h16_lock was held
(ane_h16_start_work() -> ane_h16_advance() -> ane_h16_scratch_boot()),
which orders the PM chain rwsem after ane_h16_lock. ane_h16_pm_event()
runs with that rwsem held and takes ane_h16_lock, so lockdep reports a
possible circular dependency on the first system sleep after the engine
comes up:

  WARNING: possible circular locking dependency detected
  rtcwake is trying to acquire lock: ane_h16_lock, at: ane_h16_pm_event
  but task is already holding lock: (pm_chain_head).rwsem

Register the notifier once at probe, after ane_h16_lock is dropped and
before the start work can pin the device, and let it return NOTIFY_DONE
until the device is pinned, which is what an unregistered notifier did.
Pinning now only sets the flag. A device removed before it was pinned
unregisters the notifier outside the lock.

The debugfs stage write could also pin the device; it now takes the
system sleep lock like the start work, so pinning never lands inside a
sleep transition.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
ane_fw_retain() registers the PM notifier, and both of its callers hold
b->lock: the parked start (ane_fw_start_work() ->
ane_fw_register_parked()) and the "map" stage (ane_fw_stage_action()).
ane_fw_pm_event() runs with the PM chain rwsem held and takes b->lock in
ane_fw_sleep_resume() and ane_fw_sleep_prepare(), which is the same
lock inversion as on H16.

Register the notifier once in ane_fw_staged_init(), outside b->lock,
and return NOTIFY_DONE from it until the backend is retained, which is
what an unregistered notifier did. Retaining happens under the system
sleep lock on every path, so it never changes inside a transition.
ane_fw_staged_exit() unregisters it for a backend that was never
retained.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The ABBA is real: register_pm_notifier() takes the PM chain rwsem for write, and the callback runs under it for read and then takes the driver lock. Registering once at probe/init, outside the lock, fixes the order on both backends.

The early NOTIFY_DONE while not pinned or retained is safe because every first pin or retain happens under lock_system_sleep(). That covers H16 ane_h16_start_work() and the stage file (now also under it), and H17 ane_fw_start_work(), ane_fw_first_bringup() and the stage store. The flag can't flip between PREPARE and POST. H16 recovery needs program_ready, so it only runs after the first pin. The unregister paths look right: H16 remove drops the notifier only when it isn't kept, and H17 exit only when not retained. The H16 probe failure path unwinds debugfs and the drvdata.

PROVE_LOCKING rows on the Neo (H17, 0 vs 2/2 with the old module) and the J616s (H16) cover both backends.

Heads-up: #2240 (niteria) touches the same two files, so it will need a rebase after this lands.

@DjDeveloperr
DjDeveloperr merged commit 1473d15 into integration/aurora Nov 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants