Skip to content

grep: apply --include and --exclude to files named on the command line - #153

Open
jayhemnani9910 wants to merge 3 commits into
uutils:mainfrom
jayhemnani9910:include-exclude-command-line
Open

jayhemnani9910 wants to merge 3 commits into
uutils:mainfrom
jayhemnani9910:include-exclude-command-line

Conversation

@jayhemnani9910

@jayhemnani9910 jayhemnani9910 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

--include and --exclude only filtered files found under -r; files named on the command line were always searched. GNU skips a command-line file when a pattern matches its whole name or a part right after a path separator.

Globs are now translated to Onig regexes (src/glob.rs, from @lhecker's better-globbing prototype). That adds [:class:] support and lets one regex check every suffix of a command-line path. The glob dependency is gone.

cargo test passes (126), clippy and rustfmt are clean. --include/--exclude/--exclude-dir cases, recursive and on the command line, match GNU grep 3.12, including [[.a.]], [[=a=]], invalid patterns such as [z-a] (they match nothing) and non-UTF-8 parent directories.

Copilot AI balanced review requested due to automatic review settings October 26, 2026 11:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@codspeed

codspeed Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 3 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 10 untouched benchmarks
⏩ 17 skipped benchmarks1


Comparing jayhemnani9910:include-exclude-command-line (2ff1267) with main (e123c36)

Open in CodSpeed

Footnotes

  1. 17 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Comment thread src/lib.rs Outdated
Comment on lines 999 to 1010
/// Whether a file named on the command line is skipped. A pattern
/// matches when it matches a suffix of the name: the whole name, or a
/// part that starts right after a '/'.
pub fn skips_command_line_file(&self, path: &Path) -> bool {
let suffixes: Vec<&str> = path.to_str().map_or_else(Vec::new, |name| {
std::iter::once(name)
.chain(name.match_indices('/').filter_map(|(i, _)| {
let rest = &name[i + 1..];
(!rest.is_empty() && !rest.starts_with('/')).then_some(rest)
}))
.collect()
});

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work on Windows.

And I'm not sure I understand the benefit of collecting them suffixes when you could just iterate over suffixes and then run self.skips() on the current suffix until it's Some(bool).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the Windows part: it splits on std::path::is_separator now, and the suffixes aren't collected first.

I kept the loop over the patterns, though. Going suffix by suffix gives a different answer from GNU when two patterns match different suffixes:

$ /usr/bin/grep --exclude='t/*' --include=a.txt foo t/a.txt    # GNU grep 3.12
t/a.txt:foo

The whole name only matches t/*, so stopping at the first suffix that matches would skip the file. GNU takes the last pattern that matches any suffix, here --include=a.txt.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been thinking about this, and I think this is the wrong way to go about it. The suffix list is a workaround for glob treating * as non-recursive while we need it to be that.

I'm currently coding a version that reuses our existing onig crate and a custom glob translator.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's my prototype: https://github.com/uutils/grep/compare/better-globbing

While prototyping it I found several flaws with our current glob handling, like the lack of [:class:] support. I'm out of time to continue working on this. If you'd like to adopt it, please feel free to do so.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted it: rebased your branch onto main and pushed it here, with your commit as is. Tests, clippy and fmt pass, and 37 include/exclude cases match GNU grep 3.12; the only difference is that invalid patterns like [z-a] still error, where GNU matches nothing. Description updated.

@jayhemnani9910
jayhemnani9910 force-pushed the include-exclude-command-line branch from f43099a to 015e6eb Compare October 27, 2026 17:23
Copilot AI balanced review requested due to automatic review settings October 27, 2026 17:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

jayhemnani9910 and others added 2 commits October 29, 2026 00:59
They only filtered files found under -r, so "grep --exclude='*.min.js'
foo *.js" still searched the minified files. GNU grep skips a
command-line file when a pattern matches a suffix of its name: the whole
name, or a part that starts right after a path separator. Stdin and
missing files are not affected, as in GNU.
@jayhemnani9910
jayhemnani9910 force-pushed the include-exclude-command-line branch from 015e6eb to 92757ba Compare October 28, 2026 19:31
Copilot AI balanced review requested due to automatic review settings October 28, 2026 19:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Glob matching still mishandles valid suffixes under non-UTF-8 Unix prefixes and corrupts slash-based character ranges on Windows, causing incorrect filtering and invalid-glob errors.

2 open issues

Review effort: Balanced

Comment thread src/glob.rs Outdated
return false;
}

let name = name.to_str().map(str::as_bytes);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2ff1267: names are matched as bytes now. GNU 3.12 skips p\xff/a.txt for --exclude=*.txt, --exclude=a.txt and --exclude=p?/a.txt; this does the same, and the test checks it.

Comment thread src/glob.rs Outdated
out.push_str(r"\[");
}
}
'/' if cfg!(windows) => out.push_str(r"/\\"),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2ff1267: the backslash is added once before the closing ], so [/-0] becomes [/-0\\]. Test added in windows_path_separators.

@sylvestre

Copy link
Copy Markdown
Contributor

thanks for the work on this!
This can't land yet because [[.a.]] and [[=a=]] don't match a (details inline).
The two Copilot comments are still open too: the non-UTF-8 parent directory and the Windows [/-0] range.

Comment thread src/glob.rs
'\\' => push_literal(body.next().unwrap_or('\\'), out),
'[' => {
let rest = body.as_str();
if let Some(&delimiter @ (b':' | b'.' | b'=')) = rest.as_bytes().first()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[[.a.]] and [[=a=]] are copied as is into the onig class, where the inner [ is a literal.
So --exclude='[[.a.]]' doesn't skip a file named a, and it matches .] instead.
could you please translate them to the character itself and add them to gnu_brackets?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 2ff1267: a single character between [. .] or [= =] becomes that character, so [[.a.]-c] works as a range too. A longer name like [[.ab.]] matches nothing, as in GNU. Added to gnu_brackets.

Comment thread src/glob.rs Outdated

const SEPARATOR: &str = if cfg!(windows) {
r"[/\\]"
} else if cfg!(target_os = "uefi") {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need the uefi case? grep doesn't run there :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in 2ff1267.

Comment thread src/glob.rs Outdated
RegexOptions::REGEX_OPTION_MULTILINE,
)
.map_err(|err| {
USimpleError::new(2, format!("invalid glob '{pattern}': {}", err.message))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GNU just matches nothing for [z-a], it doesn't fail.
could we do the same instead of exiting with 2?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 2ff1267. An invalid glob is kept but never matches, so it still counts for the "first pattern is an include" default, same as GNU (--include=[z-a] searches nothing, --exclude=[z-a] skips nothing).

Comment thread tests/test_grep.rs Outdated
// A missing file is still reported.
let mut c = scene.cmd(env!("CARGO_BIN_EXE_grep"));
c.args(&["--exclude=missing", "grep", "missing"])
.fails_with_code(2);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check stderr too, otherwise any error passes

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 2ff1267.

[.a.] and [=a=] inside a bracket now stand for the character. An invalid
glob such as [z-a] matches nothing instead of exiting with 2. Paths are
matched as bytes, so a non-UTF-8 parent directory no longer hides a valid
suffix. On Windows, the backslash is added to a class with '/' as its own
member, so [/-0] stays a range.
Copilot AI balanced review requested due to automatic review settings November 15, 2026 05:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants