fix(typing): route Safari Google Docs dictation through the clipboard #960
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -106,7 +106,13 @@ final class TypingService { | |
| private static let pasteboardSessionSemaphore = DispatchSemaphore(value: 1) | ||
| private static let pasteboardRestoreQueue = DispatchQueue(label: "TypingService.PasteboardRestore", qos: .utility) | ||
| private static var focusSnapshot: FocusSnapshot? | ||
|
|
||
| private static let pasteOnlyBundleIdentifiers: Set<String> = ["com.mitchellh.ghostty"] | ||
| private static let webKitBrowserBundleIdentifiers: Set<String> = [ | ||
| "com.apple.Safari", "com.apple.SafariTechnologyPreview", | ||
| ] | ||
| private static let keypressDrivenEditorTitles = ["Google Docs", "Google Slides"] | ||
| private static let axMessagingTimeoutSeconds: Float = 0.25 | ||
|
|
||
| private var textInsertionMode: SettingsStore.TextInsertionMode { | ||
| SettingsStore.shared.textInsertionMode | ||
|
|
@@ -286,36 +292,70 @@ final class TypingService { | |
| return Self.isCurrentlyFocusedElement(element, expectedPID: pid) | ||
| } | ||
|
|
||
| private func pasteOnlyTarget(preferredTargetPID: pid_t?) -> (pid: pid_t, reason: String)? { | ||
| if let preferredTargetPID, preferredTargetPID > 0 { | ||
| guard let reason = self.pasteOnlyReason(forPID: preferredTargetPID) else { return nil } | ||
| return (preferredTargetPID, reason) | ||
| } | ||
|
|
||
| if let focusedPID = self.getSystemFocusedElementAndPID()?.pid, | ||
| let reason = self.pasteOnlyReason(forPID: focusedPID) | ||
| { | ||
| return (focusedPID, reason) | ||
| } | ||
|
|
||
| if let frontmostPID = NSWorkspace.shared.frontmostApplication?.processIdentifier, | ||
| let reason = self.pasteOnlyReason(forPID: frontmostPID) | ||
| { | ||
| return (frontmostPID, reason) | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| private func pasteOnlyReason(forPID pid: pid_t) -> String? { | ||
| guard pid > 0, let app = NSRunningApplication(processIdentifier: pid) else { return nil } | ||
| return Self.pasteOnlyReason( | ||
| bundleIdentifier: app.bundleIdentifier, | ||
| focusedWindowTitle: Self.focusedWindowTitle(forPID: pid) | ||
| ) | ||
| } | ||
|
|
||
| static func pasteOnlyReason( | ||
| bundleIdentifier: String?, | ||
| focusedWindowTitle: @autoclosure () -> String? | ||
| ) -> String? { | ||
| guard let bundleIdentifier else { return nil } | ||
|
|
||
| if Self.pasteOnlyBundleIdentifiers.contains(bundleIdentifier) { | ||
| return "bundleID=\(bundleIdentifier)" | ||
| } | ||
|
|
||
| guard Self.webKitBrowserBundleIdentifiers.contains(bundleIdentifier), | ||
| let title = focusedWindowTitle(), | ||
| let editor = Self.keypressDrivenEditorTitles.first(where: { title.hasSuffix($0) }) | ||
|
Comment on lines
+334
to
+336
There was a problem hiding this comment.
The routing predicate treats Safari’s page-controlled window title as proof that the destination is Google Docs or Slides. Any ordinary page whose title ends with one of those strings can therefore move otherwise directly typed dictation through How this was verified: Safari’s page-derived accessibility title is matched by suffix alone, after which the dictated string is written to the general pasteboard with only advisory markers. Knowledge Base Used: Dictation processing and typing Prompt To Fix With AIThis is a comment left during a code review.
Path: Sources/Fluid/Services/TypingService.swift
Line: 356-358
Comment:
**Page Title Controls Clipboard**
The routing predicate treats Safari’s page-controlled window title as proof that the destination is Google Docs or Slides. Any ordinary page whose title ends with one of those strings can therefore move otherwise directly typed dictation through `NSPasteboard.general`, where observers that do not honor the advisory transient markers can read or retain it. Verify the Google origin or another non-spoofable document identity before selecting the clipboard path.
**How this was verified:** Safari’s page-derived accessibility title is matched by suffix alone, after which the dictated string is written to the general pasteboard with only advisory markers.
**Knowledge Base Used:** [Dictation processing and typing](https://app.greptile.com/altic/-/custom-context/knowledge-base/altic-dev/fluidvoice/-/docs/dictation-processing-and-typing.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly. |
||
| else { | ||
| return nil | ||
| } | ||
| return "document=\(editor)" | ||
| } | ||
|
|
||
| private static func focusedWindowTitle(forPID pid: pid_t) -> String? { | ||
| guard AXIsProcessTrusted(), pid > 0 else { return nil } | ||
|
|
||
| let appElement = AXUIElementCreateApplication(pid) | ||
| AXUIElementSetMessagingTimeout(appElement, Self.axMessagingTimeoutSeconds) | ||
|
|
||
| guard let window = Self.copyAXElementAttribute(from: appElement, attribute: kAXFocusedWindowAttribute as CFString) | ||
| ?? Self.copyAXElementAttribute(from: appElement, attribute: kAXMainWindowAttribute as CFString) | ||
| else { | ||
| return nil | ||
| } | ||
|
|
||
| AXUIElementSetMessagingTimeout(window, Self.axMessagingTimeoutSeconds) | ||
| return Self.stringAXAttribute(from: window, attribute: kAXTitleAttribute as CFString) | ||
| } | ||
|
|
||
| /// Activation options used to restore focus to the external target app after dictation. | ||
| /// `.activateAllWindows` is intentionally omitted: raising every window of a multi-window | ||
| /// app (e.g. WebStorm) destroys the user's window layout on each dictation (issue #748). | ||
|
|
@@ -536,14 +576,14 @@ final class TypingService { | |
| self.log("[TypingService] Attempting to type text: \"\(text.prefix(50))\(text.count > 50 ? "..." : "")\"") | ||
|
|
||
| if self.textInsertionMode == .standard, | ||
| let pasteOnlyTarget = self.pasteOnlyTarget(preferredTargetPID: preferredTargetPID) | ||
| { | ||
| self.log("[TypingService] Paste-only target detected in standard mode (PID \(pasteOnlyTarget.pid), \(pasteOnlyTarget.reason)); forcing Reliable Paste path") | ||
| if self.tryReliablePasteInsertion(text, preferredTargetPID: pasteOnlyTarget.pid) { | ||
| self.log("[TypingService] SUCCESS: Paste-only Reliable Paste path completed") | ||
| return true | ||
| } | ||
| self.log("[TypingService] Paste-only Reliable Paste path fell through to direct-typing fallbacks") | ||
| } | ||
|
|
||
| if self.textInsertionMode == .reliablePaste { | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| import AppKit | ||
| @testable import FluidVoice_Debug | ||
| import XCTest | ||
|
|
||
| // Safari turns one synthesized unicode key event into a single `keypress` carrying only the first | ||
| // character, so editors that build their text from `keypress` insert "H" for "Hello world". Only | ||
| // Google Docs and Slides in a WebKit browser lose text this way, and only those targets may be | ||
| // pushed off the direct-typing path onto the clipboard. | ||
|
|
||
| final class TypingServicePasteOnlyRoutingTests: XCTestCase { | ||
| private func reason(_ bundleIdentifier: String?, _ title: String?) -> String? { | ||
| TypingService.pasteOnlyReason(bundleIdentifier: bundleIdentifier, focusedWindowTitle: title) | ||
| } | ||
|
|
||
| func testKnownBundleIdentifierMatchesWithoutAWindowTitle() { | ||
| XCTAssertEqual( | ||
| self.reason("com.mitchellh.ghostty", nil), | ||
| "bundleID=com.mitchellh.ghostty", | ||
| "apps that never accept synthesized typing must match on bundle ID alone, before any AX lookup" | ||
| ) | ||
| } | ||
|
|
||
| func testTheWindowTitleIsOnlyReadWhenTheBundleIdentifierCouldMatch() { | ||
| var lookups = 0 | ||
| func reasonCountingLookups(_ bundleIdentifier: String?) -> String? { | ||
| TypingService.pasteOnlyReason( | ||
| bundleIdentifier: bundleIdentifier, | ||
| focusedWindowTitle: { | ||
| lookups += 1 | ||
| return "Quarterly notes - Google Docs" | ||
| }() | ||
| ) | ||
| } | ||
|
|
||
| _ = reasonCountingLookups("com.apple.Notes") | ||
| XCTAssertEqual(lookups, 0, "a native app must not trigger an Accessibility round trip") | ||
|
|
||
| _ = reasonCountingLookups("com.mitchellh.ghostty") | ||
| XCTAssertEqual(lookups, 0, "a bundle ID match resolves before the title is ever needed") | ||
|
|
||
| _ = reasonCountingLookups("com.apple.Safari") | ||
| XCTAssertEqual(lookups, 1, "only a WebKit browser reads the focused window title") | ||
| } | ||
|
|
||
| func testSafariMatchesGoogleDocsAndSlides() { | ||
| XCTAssertEqual( | ||
| self.reason("com.apple.Safari", "Quarterly notes - Google Docs"), | ||
| "document=Google Docs", | ||
| "Docs in Safari drops everything after the first character and needs the clipboard path" | ||
| ) | ||
| XCTAssertEqual( | ||
| self.reason("com.apple.Safari", "Launch deck - Google Slides"), | ||
| "document=Google Slides", | ||
| "Slides truncates the same way Docs does" | ||
| ) | ||
| } | ||
|
|
||
| func testGoogleSheetsKeepsTheDirectTypingPath() { | ||
| XCTAssertNil( | ||
| self.reason("com.apple.Safari", "Budget - Google Sheets"), | ||
| "Sheets inserts the full string in Safari, so forcing it onto the clipboard would be a regression" | ||
| ) | ||
| } | ||
|
|
||
| func testChromiumAndGeckoKeepTheDirectTypingPath() { | ||
| XCTAssertNil( | ||
| self.reason("com.google.Chrome", "Quarterly notes - Google Docs"), | ||
| "Chromium fires no keypress and inserts the full string" | ||
| ) | ||
| XCTAssertNil( | ||
| self.reason("org.mozilla.firefox", "Quarterly notes - Google Docs"), | ||
| "Gecko fires one keypress per character and inserts the full string" | ||
| ) | ||
| } | ||
|
|
||
| func testOrdinaryBrowsingDoesNotMatch() { | ||
| XCTAssertNil( | ||
| self.reason("com.apple.Safari", "Apple"), | ||
| "a WebKit browser alone is not enough; only the affected documents may be rerouted" | ||
| ) | ||
| } | ||
|
|
||
| func testNativeAppsAreNeverForcedOntoTheClipboard() { | ||
| XCTAssertNil( | ||
| self.reason("com.apple.Notes", "Quarterly notes - Google Docs"), | ||
| "a matching window title in a native app must not trigger the browser rule" | ||
| ) | ||
| XCTAssertNil( | ||
| self.reason(nil, nil), | ||
| "an unidentifiable target keeps the default path" | ||
| ) | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the Google account/UI language localizes the product name (for example, Safari exposes a title ending in the localized equivalent of Google Docs), this exact English suffix check returns
nil, so the affected editor stays on the direct-typing path and continues dropping all but the first character. The routing signal needs to be independent of the localized window title, or explicitly support localized variants.Useful? React with 👍 / 👎.