Add TextCursor tool for precise caret-level text control#351
Open
JezaChen wants to merge 5 commits into
Open
Conversation
…n facade Break the ~1300-line text_cursor/__init__.py into cohesive modules (constants, errors, models, uia, worker, discovery, ranges, snapshots, operations, service) and reduce __init__.py to a small public facade. - Single test seam: collaborators are called by module-level name and patched on their owning module; drop the dependency-injection params that duplicated that seam. - Break the uia<->worker cycle: move find_caret_provider and try_get_caret_on_element into discovery.py so uia.py is a pure leaf. - Trim the facade to the intended public API (I/O + action models, errors, run_tool); stop re-exporting internal plumbing. - Remove dead code: the unused UIA_TEXT_PATTERN2_ID constant and worker.co_uninitialize. - Minor cleanups: drop a redundant int() wrap and document SelectRelativeAction.origin. Repoint tests to import internals from their owning submodules. No behavior change; full suite passes.
Add TextCursor to the README tool list and the manifest.json tools array, and update the README limitation about in-paragraph text selection now that TextCursor covers it for controls that expose the UIA TextPattern.
PR Summary by QodoAdd TextCursor tool for caret/selection control via UI Automation
AI Description
Diagram
High-Level Assessment
Files changed (17)
|
There was a problem hiding this comment.
Pull request overview
This PR introduces a new TextCursor MCP tool and supporting windows_mcp.text_cursor package to inspect and manipulate the focused Windows text control’s caret/selection via UI Automation TextPattern, enabling precise caret-level moves and exact range selection (without simulated mouse/keyboard input).
Changes:
- Added a full
windows_mcp.text_cursorimplementation (UIA wrappers, discovery, range ops, snapshots, worker-thread execution, and models). - Registered the new
TextCursortool and integrated it into tool registration, manifest, and README documentation. - Added comprehensive unit tests for snapshots/range math, cancellation behavior, and error propagation/verification failures.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_text_cursor.py | New unit tests covering snapshot/range behavior and edge cases. |
| tests/test_iserror_compliance.py | Ensures TextCursor errors surface as ToolError and verification failures raise. |
| src/windows_mcp/tools/text_cursor.py | New MCP tool wrapper/registration and tool description. |
| src/windows_mcp/tools/init.py | Adds TextCursor to tool import list and _MODULES registration. |
| src/windows_mcp/text_cursor/init.py | Package entrypoint exporting models/errors and run_tool. |
| src/windows_mcp/text_cursor/constants.py | UIA constants and payload caps (e.g., max selected-text length). |
| src/windows_mcp/text_cursor/errors.py | Defines TextCursorError and TextCursorVerificationError. |
| src/windows_mcp/text_cursor/worker.py | Dedicated single-thread executor and per-thread COM/UIA client caching. |
| src/windows_mcp/text_cursor/uia.py | Low-level UIA TextPattern/TextRange wrappers and offset helpers. |
| src/windows_mcp/text_cursor/discovery.py | Focused-element discovery and parent-walk to locate TextPattern provider. |
| src/windows_mcp/text_cursor/ranges.py | Range construction, document positioning, apply/select, and verification helpers. |
| src/windows_mcp/text_cursor/operations.py | Implements write operations for all supported modes. |
| src/windows_mcp/text_cursor/snapshots.py | Builds serializable caret/selection snapshots (context, bounds, warnings). |
| src/windows_mcp/text_cursor/service.py | Orchestrates get/write actions on the COM worker thread; verifies writes. |
| src/windows_mcp/text_cursor/models.py | Pydantic models for actions, snapshots, and tool result payloads. |
| README.md | Documents the new TextCursor tool and updates limitations section accordingly. |
| manifest.json | Adds TextCursor to the published tool manifest. |
Comments suppressed due to low confidence (2)
src/windows_mcp/text_cursor/models.py:194
exclude_ifis not a valid keyword forpydantic.Fieldin the repo's Pydantic version (2.13.x), so these field declarations will fail at import time. Removeexclude_iffrom the snapshot fields.
selection_start_units: int | None = Field(
default=None,
exclude_if=_ignore_none,
description=(
"Selection start in provider-defined UIA TextUnit_Character steps "
src/windows_mcp/text_cursor/models.py:209
exclude_ifis not supported bypydantic.Fieldin Pydantic 2.13.x; these usages will break imports. Removeexclude_iffrom these optional fields.
selected_text: str | None = Field(default=None, exclude_if=_ignore_none)
text_before: str | None = Field(default=None, exclude_if=_ignore_none)
text_after: str | None = Field(default=None, exclude_if=_ignore_none)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Code Review by Qodo
Context used✅ Compliance rules (platform):
15 rules 1. Overlong TextCursor description line
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| }, | ||
| { | ||
| "name": "TextCursor", | ||
| "description": "Inspects or manipulates the caret/selection of the focused Windows text control through UI Automation. Modes: 'get_info' (read caret/selection info), 'move_relative' and 'move_absolute' (move the caret by a signed delta or to an absolute character offset), 'select_relative' and 'select_absolute' (select a text range), 'select_all', 'collapse_selection' (collapse a selection to its start or end edge). Offsets use provider-defined UIA TextUnit_Character steps from the document start. Requires a control that exposes the UIA TextPattern." |
There was a problem hiding this comment.
1. Overlong textcursor description line 📘 Rule violation ✧ Quality
New lines exceed the 100-character maximum in manifest.json and README.md, which reduces readability and makes diffs harder to review. These long, single-line descriptions should be wrapped or shortened to meet the 100-character limit.
Agent Prompt
## Issue description
Several newly added lines exceed the 100-character maximum line length (notably the new `TextCursor` descriptions).
## Issue Context
The repo compliance checklist enforces a 100-character max for non-whitespace, non-comment lines.
## Fix Focus Areas
- manifest.json[153-153]
- README.md[732-732]
- README.md[797-797]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| element = None | ||
|
|
||
| raise RuntimeError( | ||
| f"The focused element {f'"{element_name}" ' if element_name else ''}and " |
There was a problem hiding this comment.
2. Single-quoted strings in discovery.py 📘 Rule violation ✧ Quality
The new f-string in find_caret_provider() uses single-quoted string literals ('' and `f'"{...}"
'`) which violates the double-quote-only string literal style requirement. This creates inconsistent
string quoting style within the new TextCursor implementation.
Agent Prompt
## Issue description
`src/windows_mcp/text_cursor/discovery.py` introduces single-quoted string literals inside a new f-string expression.
## Issue Context
The compliance checklist requires double quotes for all string literals where possible.
## Fix Focus Areas
- src/windows_mcp/text_cursor/discovery.py[75-77]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+30
to
+47
| def register(mcp, *, get_desktop, get_analytics): | ||
| @mcp.tool( | ||
| name="TextCursor", | ||
| description=_description, | ||
| annotations=ToolAnnotations( | ||
| title="TextCursor", | ||
| readOnlyHint=False, | ||
| destructiveHint=True, | ||
| idempotentHint=False, | ||
| openWorldHint=True, | ||
| ), | ||
| ) | ||
| @with_analytics(get_analytics(), "TextCursor-Tool") | ||
| async def text_cursor( | ||
| action: CursorAction, | ||
| ctx: Context = None, | ||
| ) -> CursorToolResult: | ||
| return await run_tool(action) |
There was a problem hiding this comment.
3. textcursor tool bypasses desktop 📘 Rule violation ⌂ Architecture
The MCP entry-point tool implementation for TextCursor does not delegate to the Desktop service layer (it never uses get_desktop and calls windows_mcp.text_cursor.run_tool directly). This violates the requirement that MCP entry-point tools remain thin adapters over Desktop services.
Agent Prompt
## Issue description
The `TextCursor` MCP tool implementation directly calls the TextCursor implementation (`run_tool`) and does not delegate through the Desktop service layer.
## Issue Context
Entry-point tools should remain thin adapters and call Desktop service methods for domain/platform logic.
## Fix Focus Areas
- src/windows_mcp/tools/text_cursor.py[30-47]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| """ | ||
|
|
||
|
|
||
| def register(mcp, *, get_desktop, get_analytics): |
There was a problem hiding this comment.
4. Missing type hints in new defs 📘 Rule violation ✧ Quality
Several newly added def functions/methods do not fully type-annotate their signatures (parameters and/or return types), violating the requirement for type hints on all function signatures. This reduces static-checking coverage for the new TextCursor feature and tests.
Agent Prompt
## Issue description
Newly added functions/methods are missing required type hints on parameters and/or return types.
## Issue Context
The compliance checklist requires explicit type annotations for every parameter and for the return type of every `def`.
## Fix Focus Areas
- src/windows_mcp/tools/text_cursor.py[30-30]
- src/windows_mcp/text_cursor/uia.py[20-27]
- src/windows_mcp/text_cursor/ranges.py[75-80]
- tests/test_iserror_compliance.py[90-115]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+90
to
+107
| async def run_tool(action: CursorAction) -> CursorToolResult: | ||
| """ | ||
| Inspect or manipulate the focused Windows text control through UIA. | ||
| Modes: | ||
| - get_info | ||
| - move_relative | ||
| - move_absolute | ||
| - select_relative | ||
| - select_absolute | ||
| - select_all | ||
| - collapse_selection | ||
| Every mode accepts `delay`, expressed in seconds. The delay occurs before | ||
| the focused UIA element is located, so the caller can focus the target | ||
| control during that interval. | ||
| Absolute move/select inputs and returned offsets both use provider-defined | ||
| UIA TextUnit_Character steps from DocumentRange start. Returned offsets can | ||
| be passed directly to absolute move/select actions. | ||
| """ |
There was a problem hiding this comment.
5. run_tool docstring not google-style 📘 Rule violation ✧ Quality
The new public run_tool() docstring does not follow Google-style structure (missing explicit Args:/Returns: sections), and register() has no docstring at all. This violates the requirement for Google-style docstrings on public functions/classes in changed files.
Agent Prompt
## Issue description
Public functions introduced/modified in this PR are missing Google-style docstrings (and some are missing docstrings entirely).
## Issue Context
Google-style docstrings should include a one-line summary and structured `Args:`/`Returns:` sections for public functions.
## Fix Focus Areas
- src/windows_mcp/text_cursor/service.py[90-107]
- src/windows_mcp/tools/text_cursor.py[30-47]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Member
|
Awesome, work Please check |
Collaborator
Author
Yeah, definitely. This is just my first-step commit — I originally built Next step is to consolidate: drop the parallel layer and move the UIA/text-range logic onto the shared uia module — reusing the |
…M layer TextCursor previously reimplemented UIA text ranges/patterns and ran on a dedicated MTA worker with its own COM client. Consolidate onto the shared windows_mcp.uia: - Promote the reusable range helpers onto uia.TextRange/TextPattern (IsDegenerate, Collapse, GetStartOffset, GetTextBefore/After, GetFirstSelection) and normalize TextRange.GetText None -> "". - Delete text_cursor's TextRange/TextPattern/UIAElement wrappers, the dedicated MTA worker, and create_automation; run on the main-thread STA and reuse the shared client via GetFocusedControl / Control, like the other UIA tools. - Inline the one-shot execute_sync dispatch into run_tool (kept async so the delay stays non-blocking and cancellable).
Comment on lines
+22
to
+28
| marker = caret_info.text_range.Clone() | ||
| marker.Collapse(toEnd=(endpoint == TextPatternRangeEndpoint.End)) | ||
|
|
||
| try: | ||
| return marker.GetStartOffset() | ||
| except (COMError, AttributeError, TypeError): | ||
| return None |
Comment on lines
+33
to
+34
| base.Collapse(toEnd=(origin == "selection_end")) | ||
| return base |
Comment on lines
+108
to
+110
| target = caret_info.text_range.Clone() | ||
| target.Collapse(toEnd=(action.edge == "end")) | ||
|
|
Collaborator
Author
|
I checked ac4f0d5 and simplified the implementation (see 100ec45) to reuse TextCursor now uses the shared UIA does not expose the caret as a first-class concept— |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As we discussed earlier, I've been working on implementing a new tool over the past few days, TextCursor, that inspects and manipulates the caret/selection of the focused Windows text control directly through UI Automation (
IUIAutomationTextPattern).Until now, working with text meant simulating mouse and keyboard input, which is imprecise for anything finer than whole-field edits — you cannot reliably place the caret or select an exact span. TextCursor operates on the control's own text range, so positioning and selection are exact and format-preserving.
What it does
Seven modes, all against the focused control:
get_info— read the current caret/selection (offsets, surrounding text, bounds)move_relative/move_absolute— move the caret by a signed delta or to an absolute character offsetselect_relative/select_absolute— select a text rangeselect_allcollapse_selection— collapse a selection to its start or end edgeOffsets use provider-defined UIA
TextUnit_Charactersteps from the document start, and returned offsets can be fed straight back into absolute actions. Write operations are verified by reading the applied range back. Requires a control that exposes the UIA TextPattern.Limitations
Demo
Using TextCursor to replace the 2nd-last and 3rd-last occurrences of the word
isbefore the caret withwas, while leaving the surrounding rich-text formatting intact:textcursor-demo4.mp4