Skip to content

fix(hook): give the callback watchdog the same watched-time rule - #1624

Merged
AprilNEA merged 2 commits into
masterfrom
fix/macos-callback-watchdog-freeze
Sep 30, 2026
Merged

AprilNEA merged 2 commits into
masterfrom
fix/macos-callback-watchdog-freeze

Conversation

@AprilNEA

Copy link
Copy Markdown
Owner

Summary

The stuck-callback watchdog compared the tap callback's entry time against now every 20 ms and force-exited past 200 ms. Like the lifecycle watchdog before #1282, that charges the callback for time the watchdog thread itself was not running: a callback entered just before the process-wide freeze around a sleep transition would read as stuck the moment the watchdog thawed, even though the tap thread froze with it. The window is the microseconds a callback is entered, so this has not been seen in a log yet; it is the same defect #1282 fixed for the lifecycle watchdog, with a smaller target. Refs #952.

Changes

  • openlogi-hook: the decision moves out of the poll loop into a sans-I/O CallbackWatchdog with the lifecycle watchdog's rule. A poll charges the whole gap since the previous one unless kern.sleeptime or kern.waketime moved inside it, in which case at most CALLBACK_OBSERVATION_GAP (5 polls, 100 ms) is charged; an entry is judged on the stall the watchdog was awake to see. stuck_callback is gone; CALLBACK_POLL_INTERVAL moves next to the other watchdog constants. The exit log prints stalled_ms next to watched_ms, like the lifecycle exit.
  • crates/openlogi-hook/AGENTS.md: the watched-time rule now names both watchdogs.

Normal operation is unchanged: with the thread polling on time, a callback that never returns is still exited on the first poll at or past 200 ms (callback_timeout_keeps_the_200ms_boundary drives the real 20 ms cadence and lands at 205 ms).

Testing

Run on macOS 26 (arm64) with RUSTFLAGS="-D warnings", rebased onto master after #1282:

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --document-private-items --exclude openlogi-ui --exclude openlogi-desktop --exclude openlogi-overlay --exclude openlogi-agent
  • cargo xtask ci clippy-windows

Mutation checks: without the sleep-gated credit cap, a_frozen_process_is_not_a_stuck_callback and repeated_sleep_cycles_still_accumulate_a_stuck_callback fail; a_callback_gap_with_no_sleep_in_it_is_charged_in_full pins that a poll delayed by scheduling alone still exits on that poll.

Not run: tests (linux), msrv, cargo-deny (CI). Not runtime-tested on hardware: no reproduction exists for this window, and the change does not alter behaviour outside a gap that contains a kernel sleep or wake.

The stuck-callback watchdog compared the callback's entry time against now every 20 ms and force-exited past 200 ms. Like the lifecycle watchdog before the previous two commits, that charges the callback for time the watchdog thread itself was not running: a callback entered just before the process-wide freeze around a sleep transition would read as stuck the moment the watchdog thawed, even though the tap thread froze with it. The window is the microseconds a callback is entered, so this has not been seen in a log yet; it is the same defect with a smaller target.

The decision now lives in a sans-I/O CallbackWatchdog with the lifecycle watchdog's rule: a poll charges the whole gap since the previous one unless kern.sleeptime or kern.waketime moved inside it, in which case at most CALLBACK_OBSERVATION_GAP (5 polls, 100 ms) is charged; an entry is judged on the stall the watchdog was awake to see. The exit log prints stalled_ms next to watched_ms. Normal operation is unchanged: with the thread polling on time, a callback that never returns is still exited on the first poll at or past 200 ms.

Refs #952
@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium risk] Refactors callback watchdog to use consistent timing rules.

The PR appears safe to merge; no outstanding finding or new actionable issue was identified.

Summary

The PR applies the lifecycle watchdog’s sleep-aware watched-time rule to the macOS callback watchdog and adds first-poll coverage.

  • Initializes a poll baseline before the callback watchdog’s first sleep, addressing the previously reported loss of first-poll stall time.
  • Adds callback freeze and timeout tests and documents the shared watchdog rule.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Start callback watchdog] --> B[Record time and power epoch]
  B --> C[Poll after 20 ms]
  C --> D{Power epoch changed?}
  D -->|Yes| E[Cap gap credit at 100 ms]
  D -->|No| F[Charge full gap]
  E --> G[Accumulate watched callback time]
  F --> G
  G --> H{At least 200 ms?}
  H -->|Yes, entry still active and tap armed| I[Exit agent]
  H -->|No| C
Loading

Reviews (2) · Last reviewed commit: "fix(hook): charge the callback watchdog'..."

Comment thread crates/openlogi-hook/src/macos/watchdog.rs
The callback watchdog thread sleeps one poll interval before its first poll, and a watchdog with no previous poll credited that first gap as nothing. A callback that wedged in that window, with the first poll itself delayed past the budget by scheduling, was granted a second 200 ms before the exit (review finding on #1624). The watchdog now starts with the instant it was spawned as its previous poll, so the first gap is charged like every later one; a kernel sleep or wake inside it is discounted as elsewhere.
@AprilNEA
AprilNEA merged commit c1703f6 into master Sep 30, 2026
23 checks passed
@AprilNEA
AprilNEA deleted the fix/macos-callback-watchdog-freeze branch September 30, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant