Recover VoiceInk after runtime service stalls

[bifrost-proxy]

Summary

• Rebuild process-local shortcut monitors, cloud speech transports, App Shortcut parameters, and model prewarm state after wake, session resume, or memory-pressure recovery.
• Add an AppKit event-dispatch watchdog that records diagnostics and performs a controlled idle-only relaunch after a sustained UI hang.
• Bound unstable cloud standby retries with exponential backoff, a circuit breaker, and failure reset only after demonstrated connection health or successful use.

User-visible behavior

When macOS pressure, sleep/resume, or service interruption leaves VoiceInk alive but unable to consume menu events, VoiceInk can rebuild stale runtime resources. If AppKit still does not dispatch events for 30 seconds, VoiceInk relaunches only when no protected work is active. Recording, live transcription, pending or active audio-file transcription, assistant requests, standalone retranscription or re-enhancement, model management, and update activity suppress automatic relaunch to avoid losing work.

A latest incident record is written to ~/Library/Logs/VoiceInk/runtime-recovery-latest.json. Automatic relaunch has a ten-minute cooldown and can be disabled with VOICEINK_DISABLE_HANG_RECOVERY=1. LaunchServices failures are retried for a bounded interval, and a terminal helper failure clears the cooldown marker so a later watchdog tick can recover instead of leaving VoiceInk closed.

Safety and compatibility

• No data migration, model, transcription-accuracy, or external dependency changes.
• No persistent login helper or background service is installed; the recovery helper is generated in the per-user cache with mode 0700 and is invoked only after an idle hard hang.
• Memory warning and critical states suspend optional prewarm and cloud standby work; non-cooperative prewarm inference is awaited until confirmed quiescent, and recovery happens after pressure returns to normal.
• Pressure cleanup uses one process-wide operation gate across every transcription registry and recording-time model warm load, releases local resources from every live registry, excludes newly starting sessions for the complete asynchronous release window, retains both cancelled-session leases and their managed wrappers until non-cooperative inference or streaming-provider cancellation or timed-out normal disconnect actually returns, cancels stale cleanup when pressure recovers, defers optional FluidAudio post-download preparation, waits for active post-download model warmups to stop, and drops FluidAudio's reusable Core ML cache.
• A hard hang during protected work is diagnosed but intentionally left for manual intervention.
• If persistent SwiftData initialization fails and VoiceInk falls back to ephemeral in-memory storage, hard automatic relaunch is disabled so the current session's records cannot be silently discarded; diagnostics and soft runtime recovery remain active.
• Only persisted-field changes or in-flight editor operations hold recovery protection, including nested prompt, custom-emoji, website, trigger-word, and trigger-group drafts; clean mode/custom-model editors and UI-only expansion state remain recoverable. Empty dictionary quick-add panels remain recoverable, while meaningful input and pending submissions are protected. Ark model-ID drafts and verification are protected even when the credential is already saved. Unsaved provider/API-key, onboarding, Ark model-ID, Ollama server-address, dictionary-add, dictionary-edit, vocabulary, domain, and filler-word drafts are protected from a hard relaunch. Connection verification is protected while in flight, and a verified key is not treated as saved unless Keychain persistence succeeds.
• Qwen MLX response reads run outside the runtime actor, so cancellation can enter immediately; shutdown sends a graceful write-only request, waits at most 500 ms without reading a potentially wedged stdout pipe, then terminates the bridge. Concurrent bridge requests are admitted FIFO instead of failing on normal prewarm/recording overlap, while shutdown bypasses the queue so a wedged response can still be interrupted. Concurrent shutdown callers await the same bounded cleanup. Cancellation recovery is installed only after FIFO admission, so cancelling queued work cannot stop the currently admitted request; cancelling active Qwen work initiates bounded bridge shutdown, atomically seals late cancellation at handoff, and retains FIFO admission until confirmed shutdown completes. Blocking stdin writes run outside actor isolation with throwing I/O, so pipe closure becomes a recoverable Swift error; shutdown starts its deadline before attempting a graceful write, so a full pipe cannot prevent termination.
• Successful complete-file fallback results return immediately even if a superseded local streaming task ignores cancellation; the managed wrapper and runtime lease remain retained by background cleanup until the old task actually exits.
• Settings export holds a protected-work lease across an atomic file write, so hang recovery cannot interrupt a slow destination write.
• Delayed clipboard restoration is protected synchronously from scheduling through completion, preventing hard relaunch from discarding the saved clipboard snapshot.
• For coalesced memory-pressure events, critical and warning states dominate normal; optional work resumes only after an unambiguous normal event.

Validation

• git diff --check
• make check
• xcodebuild test -project VoiceInk.xcodeproj -scheme VoiceInk -destination platform=macOS,arch=arm64 -only-testing:VoiceInkTests -parallel-testing-enabled NO CODE_SIGNING_ALLOWED=NO ENABLE_TESTABILITY=YES
  • Result: 444 test instances passed, 4 skipped, 0 failed on macOS 26.3 arm64.
• Added deterministic simulations for AppKit event-dispatch stalls, soft/hard thresholds, sleep/freeze clock gaps (including protected-work diagnostics and stale-probe replacement), protected-work suppression, pending and standalone audio work, visible assistant conversations, system-audio restoration, serialized memory-pressure transitions across suspension awaits, severity-dominant coalesced pressure events, process-wide multi-registry operation exclusion and all-registry cleanup, recording-time model-load coordination, full-window pressure-release/session exclusion, cloud and Native Apple exclusion from local-runtime pressure leases, pressure rechecks after waiting for resource release, stale in-flight release cancellation after pressure recovery, cancelled non-cooperative inference, managed-session self-retention through streaming disconnect, result delivery before non-cooperative cleanup completion, active-transcription cancellation barriers, idempotent streaming-disconnect lease retention, idle local-model release deferral, optional FluidAudio preparation deferral, prewarm and post-download warmup termination, controlled relaunch/cooldown failure, in-memory storage relaunch suppression, shortcut press handoff (including mode shortcuts, physical key-state reconciliation after sleep, and retry completion), bounded retry when any shortcut component fails to rebuild, hands-free recording through event-monitor loss, serialized runtime recovery requests, ordered cloud-target reconciliation, duplicate model-name lookup, runtime transport rebuilds, immediately closed WebSocket attempts, protected settings import, protected audio cleanup analysis and delete/save transactions, provider credentials and dashboard display-name editing, superseded non-cooperative prewarm tasks, out-of-order pressure transitions, in-flight lease invalidation, startup watchdog arming, Accessibility-loss release synthesis, helper-loss refusal, termination disarm, pre-launch heartbeat revalidation, outstanding-probe coalescing and event-creation failure retry, unsaved-editor protection (including custom-model credentials, onboarding, and dictionary drafts), failed cloud-session cleanup, lease-validation failures, and cloud retry circuit preservation, delayed clipboard-restoration protection, Qwen actor responsiveness while stdout is blocked, confirmed forced Qwen termination, hands-free shortcut-state preservation, failed audio-queue retention, non-cooperative streaming-startup cancellation barriers, persistent pressure cleanup arming, timed-out commit cleanup barriers, FIFO Qwen request admission, admitted Qwen cancellation with bounded bridge stop and successor-admission barrier, queued Qwen cancellation isolation, protected atomic settings export, Ollama server-draft protection, atomic late-cancellation sealing, blocked and closed Qwen stdin writer handling, and clean-versus-dirty editor protection, empty-versus-dirty dictionary quick-add state, UI-only mode expansion, Ark model draft/verification protection, clean-versus-dirty dashboard display-name editing, nested prompt/custom-emoji/trigger/trigger-group drafts, and modifier-release shortcut recovery.

Risk and rollback

The watchdog posts one private application-defined event every two seconds. Automatic relaunch requires 30 seconds without AppKit event dispatch, no protected work, and no recent automatic relaunch. Revert the PR merge commit to disable the change.

This PR intentionally does not merge, release, tag, or install VoiceInk.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42449b4efe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8e3068b93e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b13c762dbc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 058c2ee748

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3093b12ad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 601f91ef70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b3bb2da0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47821d993e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9bc4506171

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59274a29e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70a0dbf95a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1fe1c9f254

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a1ecbcbf2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dcb385fb81

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d36dd43018

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd96424a97

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cbfd5c3aab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review\n\nPlease review current commit 189b882, including background cleanup result delivery, throwing Qwen pipe writes, and nested mode draft protection. Local validation: 443 passed test instances, 4 skipped, 0 failed; make check and git diff --check passed.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 189b882748

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review\n\nPlease review current commit 8f7e9ef, especially trigger-group nested draft protection and disabling automatic hard relaunch for the in-memory SwiftData fallback. Local validation: 444 passed test instances, 4 skipped, 0 failed; make check and git diff --check passed.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8f7e9efa0f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bifrost-proxy]

@codex review\n\nRe-triggering review for HEAD 8f7e9ef because the previous request was not picked up. Please focus on the two latest changes: TriggerGroupEditorView draft protection and disabling automatic relaunch for in-memory SwiftData fallback.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 8f7e9efa0f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".