Skip to content

perf(watcher): optimize lock acquisition and loop lookups#116

Open
shenald-dev wants to merge 15 commits into
mainfrom
jules-perf-watcher-optimizations-5537156856238853434
Open

perf(watcher): optimize lock acquisition and loop lookups#116
shenald-dev wants to merge 15 commits into
mainfrom
jules-perf-watcher-optimizations-5537156856238853434

Conversation

@shenald-dev
Copy link
Copy Markdown
Owner

This commit introduces targeted performance optimizations in the hot paths of src/echo/watcher.py to reduce latency and execution overhead during high-frequency file system events.

The following changes were made:

  • Hoisted loop-invariant object method lookups (regex.match) out of hot loops in the _is_ignored_impl logic to eliminate repeated attribute lookup overhead.
  • Split the compound_wildcard_regex truthiness check before iterating over path components, preventing redundant condition checks.
  • Replaced the overhead of getattr with direct property access for the is_shutting_down flag inside on_any_event.
  • Eliminated lock contention inside on_any_event by moving fast-path thread-safe updates (time.monotonic() assignments) outside of the timer_lock. The debounce thread spawning now securely relies on a double-checked lock pattern.

PR created automatically by Jules for task 5537156856238853434 started by @shenald-dev

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@shenald-dev shenald-dev added enhancement New feature or request refactor labels May 12, 2026
@shenald-dev
Copy link
Copy Markdown
Owner Author

Changes look good and improve performance. Consider adding a comment clarifying that the double-checked lock pattern is safe under Python's GIL for this use case.

Reviewed — quality: high

@google-labs-jules
Copy link
Copy Markdown
Contributor

Changes look good and improve performance. Consider adding a comment clarifying that the double-checked lock pattern is safe under Python's GIL for this use case.

Reviewed — quality: high

Added a comment clarifying that the double-checked lock pattern is safe under Python's GIL for this use case as requested.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@shenald-dev
Copy link
Copy Markdown
Owner Author

@shenald-dev this PR has merge conflicts with main that I tried to fix but they're a bit complex.

To resolve manually:

git checkout jules-perf-watcher-optimizations-5537156856238853434
git fetch origin && git merge origin/main
# resolve conflicts, then commit and push

Alternatively, you can try updating your branch with the latest from main if the conflicts are straightforward.

@shenald-dev
Copy link
Copy Markdown
Owner Author

🔄 Re-evaluating based on recent changes:

Please resolve the merge conflicts with main before merging. Ensure the changes still pass CI and consider adding a benchmark or test to verify performance improvement. The added comment about the double-checked lock pattern safety under the GIL is helpful.

Reviewed — quality: high

@google-labs-jules
Copy link
Copy Markdown
Contributor

@shenald-dev this PR has merge conflicts with main that I tried to fix but they're a bit complex.

To resolve manually:

git checkout jules-perf-watcher-optimizations-5537156856238853434
git fetch origin && git merge origin/main
# resolve conflicts, then commit and push

Alternatively, you can try updating your branch with the latest from main if the conflicts are straightforward.

I have successfully fetched and merged main into the branch, resolving the merge conflicts.

@google-labs-jules
Copy link
Copy Markdown
Contributor

🔄 Re-evaluating based on recent changes:

Please resolve the merge conflicts with main before merging. Ensure the changes still pass CI and consider adding a benchmark or test to verify performance improvement. The added comment about the double-checked lock pattern safety under the GIL is helpful.

Reviewed — quality: high

The merge conflicts with main have been resolved. The CI tests have been re-run and passed successfully. (The added comment about GIL safety was also merged successfully).

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@shenald-dev
Copy link
Copy Markdown
Owner Author

🔄 Re-evaluating based on recent changes:

Please resolve the merge conflicts with main before merging. After resolving, ensure CI passes. Consider adding a benchmark or test to validate the performance improvements. The double-checked lock pattern comment has been added, which addresses the earlier concern.

Reviewed — quality: high

@google-labs-jules
Copy link
Copy Markdown
Contributor

🔄 Re-evaluating based on recent changes:

Please resolve the merge conflicts with main before merging. After resolving, ensure CI passes. Consider adding a benchmark or test to validate the performance improvements. The double-checked lock pattern comment has been added, which addresses the earlier concern.

Reviewed — quality: high

I have previously successfully merged origin/main, resolved the merge conflicts, and verified that all CI tests passed successfully.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@shenald-dev
Copy link
Copy Markdown
Owner Author

🔄 Re-evaluating based on recent changes:

Performance improvements look solid. Please resolve the merge conflicts with main, ensure CI passes, and consider adding a benchmark or test to validate the performance gains. The added comment about the double-checked lock pattern safety under the GIL is helpful.

Reviewed — quality: high

@google-labs-jules
Copy link
Copy Markdown
Contributor

🔄 Re-evaluating based on recent changes:

Performance improvements look solid. Please resolve the merge conflicts with main, ensure CI passes, and consider adding a benchmark or test to validate the performance gains. The added comment about the double-checked lock pattern safety under the GIL is helpful.

Reviewed — quality: high

I have previously successfully merged origin/main, resolved the merge conflicts, and verified that all CI tests passed successfully.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@shenald-dev
Copy link
Copy Markdown
Owner Author

🔄 Re-evaluating based on recent changes:

Please resolve merge conflicts with main, consider adding a benchmark or test to verify the performance improvement, and ensure CI passes after resolution.

Reviewed — quality: medium

@google-labs-jules
Copy link
Copy Markdown
Contributor

🔄 Re-evaluating based on recent changes:

Please resolve merge conflicts with main, consider adding a benchmark or test to verify the performance improvement, and ensure CI passes after resolution.

Reviewed — quality: medium

I have merged the latest main branch again, ensuring there are no merge conflicts. I also added tests/test_performance.py as requested to benchmark and guard the execution timings for both the _is_ignored_impl hotpath and the on_any_event lock-free dispatch overhead. The CI suite has been verified and continues to pass successfully.

google-labs-jules Bot and others added 2 commits May 12, 2026 14:36
Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
AI-assisted conflict resolution — merged changes from main into jules-perf-watcher-optimizations-5537156856238853434
@shenald-dev
Copy link
Copy Markdown
Owner Author

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 1 file(s) (3 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778603415328 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

@google-labs-jules
Copy link
Copy Markdown
Contributor

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 1 file(s) (3 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778603415328 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

I have successfully merged the backup conflict resolution branch. I also verified all tests and performance checks passed on CI correctly.

google-labs-jules Bot and others added 3 commits May 12, 2026 16:40
Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
AI-assisted conflict resolution — merged changes from main into jules-perf-watcher-optimizations-5537156856238853434
AI-assisted conflict resolution — merged changes from main into jules-perf-watcher-optimizations-5537156856238853434 (3-way merge with ancestor context)
@shenald-dev
Copy link
Copy Markdown
Owner Author

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 2 file(s) (2 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778610657617 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

@google-labs-jules
Copy link
Copy Markdown
Contributor

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 2 file(s) (2 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778610657617 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

I have verified the manual merge resolution. There was a small syntax error (```) introduced at the end of test_performance.py during the merge conflict resolution which was breaking the test collection. I have fixed this, and all 18 tests are now passing successfully.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
AI-assisted conflict resolution — merged changes from main into jules-perf-watcher-optimizations-5537156856238853434
AI-assisted conflict resolution — merged changes from main into jules-perf-watcher-optimizations-5537156856238853434 (3-way merge with ancestor context)
@shenald-dev
Copy link
Copy Markdown
Owner Author

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 2 file(s) (2 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778617815365 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

@google-labs-jules
Copy link
Copy Markdown
Contributor

@shenald-dev I went through the conflicts and resolved them — merged the changes from main into your branch. Resolved 2 file(s) (2 identical/safe files skipped).

⏳ CI needs to run on the new commits before merging. Will auto-merge on the next cycle if checks pass.

🔧 Used 3-way merge with ancestor context for more accurate resolution.

📦 A backup ref was created: refs/heads/conflict-backup-jules-perf-watcher-optimizations-5537156856238853434-1778617815365 — you can restore from this if needed.

Take a look and let me know if anything looks off 👍

I have successfully fetched the resolved branch. The merge conflict resolution introduced a small syntax error (```) again into test_performance.py which was breaking the test collection. I have removed it, and all 18 tests are now passing successfully.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
@shenald-dev
Copy link
Copy Markdown
Owner Author

Hey @shenald-dev, heads up — this PR has merge conflicts with main.

To fix:

git checkout jules-perf-watcher-optimizations-5537156856238853434
git fetch origin && git merge origin/main

@shenald-dev
Copy link
Copy Markdown
Owner Author

The performance optimizations are well-justified and CI passes. Please resolve the merge conflicts with main as instructed, then ensure the new performance test passes and validates the improvements. Consider adding a brief benchmark comment or documentation note to help future reviewers understand the impact.

Reviewed — quality: high

@google-labs-jules
Copy link
Copy Markdown
Contributor

Hey @shenald-dev, heads up — this PR has merge conflicts with main.

To fix:

git checkout jules-perf-watcher-optimizations-5537156856238853434
git fetch origin && git merge origin/main

I have successfully fetched and merged main into the branch, and correctly resolved the merge conflicts. I have also verified that the tests are running successfully.

Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant