Skip to content

perf(grep): rank the first file batch before searching - #960

Merged
dmtrKovalenko merged 1 commit into
mainfrom
perf/rank-first-grep-page
Oct 26, 2026
Merged

dmtrKovalenko merged 1 commit into
mainfrom
perf/rank-first-grep-page

Conversation

@dmtrKovalenko

@dmtrKovalenko dmtrKovalenko commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Rank the first file batch before searching large candidate lists. Sort the remaining files when a search reaches them. Preserve the order of files with equal scores.


Stock Criterion compared the change with #958 on a warm 1.87-million-file repository. The content index was disabled. Each value is the median of two fresh process medians.

Literal search Before After
First result page 91.9 ms 41.6 ms
Missing literal, full scan 2.73 s 2.69 s

The first page returned 53 matches and stopped after 4,090 files. Full-scan timings overlapped. The regression test compares batch order and deep cursors against a full stable sort.

Summary by CodeRabbit

  • Search
    • Search results are ordered to prioritize files with higher usage frequency and more recent modifications.
    • Results retain a consistent order when files have matching ranking values, helping make browsing through result batches more predictable.

Rank the first file batch before searching large candidate lists. Sort the remaining files when a search reaches them. Preserve the order of files with equal scores.
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ff0d43d5-eba8-41ac-83cb-18ccf77598bf
📥 Commits

Reviewing files that changed from the base of the PR and between 7353971 and 1314fc4.

📒 Files selected for processing: 4
  • crates/fff-core/src/grep/fuzzy_grep.rs
  • crates/fff-core/src/grep/grep.rs
  • crates/fff-core/src/grep/multi_pattern.rs
  • crates/fff-core/src/grep/prefilter.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Prefiltering now returns an offset-aware RankedFiles collection. Fuzzy and regular grep search paths use its batch API to retrieve ranked file chunks.

Changes: Ranked grep batching

Layer / File(s) Summary
RankedFiles construction and batching
crates/fff-core/src/grep/prefilter.rs
RankedFiles stores candidate references, an offset, and the length of its ranked prefix. It partially ranks candidates under specified offset and page-limit conditions, and otherwise sorts the full list. Its batch method returns requested files and sorts the unranked suffix when needed. Tests compare batches with a full-sort reference across offsets, page limits, and batch boundaries.
Grep search batch integration
crates/fff-core/src/grep/fuzzy_grep.rs, crates/fff-core/src/grep/grep.rs, crates/fff-core/src/grep/multi_pattern.rs
Fuzzy and regular grep paths pass RankedFiles mutably to their search functions. Those functions use batch to select file chunks. Multi-pattern grep passes the mutable collection to perform_grep.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: kh05ifr4nd, gustav-fff

Merge Risk: ⚪ Minimal · up to 1314f

No merge-blocking issue was identified. The ranked batches appear to preserve search pagination; normal build and test checks remain appropriate before merging.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 56.25% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 16 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: ranking the first file batch before searching.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dmtrKovalenko
dmtrKovalenko merged commit d1c7dd4 into main Oct 26, 2026
53 checks passed
@dmtrKovalenko
dmtrKovalenko deleted the perf/rank-first-grep-page branch October 26, 2026 03:31
kwojcik-blockether added a commit to Blockether/fff that referenced this pull request Nov 4, 2026
Bring in 23 upstream commits, among them the bigram column reuse fix
(dmtrKovalenko#962), literal git status paths (dmtrKovalenko#922), the faster first grep batch
(dmtrKovalenko#960) and Blueprint text detection (dmtrKovalenko#931).

The fork's default ripgrep walker now uses is_binary_file, so text .blp
files stay searchable there too. The upstream post-scan test sets the
fork's respect_ignore_files field.

Vis-Session: 87db81b7-743c-4c3a-9132-78b8940cc023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant