Repository navigation
grep: apply --include and --exclude to files named on the command line - #153
jayhemnani9910 wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.
Merging this PR will not alter performance1 warning
Comparing Footnotes
|
| /// Whether a file named on the command line is skipped. A pattern | ||
| /// matches when it matches a suffix of the name: the whole name, or a | ||
| /// part that starts right after a '/'. | ||
| pub fn skips_command_line_file(&self, path: &Path) -> bool { | ||
| let suffixes: Vec<&str> = path.to_str().map_or_else(Vec::new, |name| { | ||
| std::iter::once(name) | ||
| .chain(name.match_indices('/').filter_map(|(i, _)| { | ||
| let rest = &name[i + 1..]; | ||
| (!rest.is_empty() && !rest.starts_with('/')).then_some(rest) | ||
| })) | ||
| .collect() | ||
| }); |
There was a problem hiding this comment.
This doesn't work on Windows.
And I'm not sure I understand the benefit of collecting them suffixes when you could just iterate over suffixes and then run self.skips() on the current suffix until it's Some(bool).
There was a problem hiding this comment.
Fixed the Windows part: it splits on std::path::is_separator now, and the suffixes aren't collected first.
I kept the loop over the patterns, though. Going suffix by suffix gives a different answer from GNU when two patterns match different suffixes:
$ /usr/bin/grep --exclude='t/*' --include=a.txt foo t/a.txt # GNU grep 3.12
t/a.txt:fooThe whole name only matches t/*, so stopping at the first suffix that matches would skip the file. GNU takes the last pattern that matches any suffix, here --include=a.txt.
There was a problem hiding this comment.
I have been thinking about this, and I think this is the wrong way to go about it. The suffix list is a workaround for glob treating * as non-recursive while we need it to be that.
I'm currently coding a version that reuses our existing onig crate and a custom glob translator.
There was a problem hiding this comment.
Here's my prototype: https://github.com/uutils/grep/compare/better-globbing
While prototyping it I found several flaws with our current glob handling, like the lack of [:class:] support. I'm out of time to continue working on this. If you'd like to adopt it, please feel free to do so.
There was a problem hiding this comment.
Adopted it: rebased your branch onto main and pushed it here, with your commit as is. Tests, clippy and fmt pass, and 37 include/exclude cases match GNU grep 3.12; the only difference is that invalid patterns like [z-a] still error, where GNU matches nothing. Description updated.
f43099a to
015e6eb
Compare
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.
They only filtered files found under -r, so "grep --exclude='*.min.js' foo *.js" still searched the minified files. GNU grep skips a command-line file when a pattern matches a suffix of its name: the whole name, or a part that starts right after a path separator. Stdin and missing files are not affected, as in GNU.
015e6eb to
92757ba
Compare
| return false; | ||
| } | ||
|
|
||
| let name = name.to_str().map(str::as_bytes); |
There was a problem hiding this comment.
Fixed in 2ff1267: names are matched as bytes now. GNU 3.12 skips p\xff/a.txt for --exclude=*.txt, --exclude=a.txt and --exclude=p?/a.txt; this does the same, and the test checks it.
| out.push_str(r"\["); | ||
| } | ||
| } | ||
| '/' if cfg!(windows) => out.push_str(r"/\\"), |
There was a problem hiding this comment.
Fixed in 2ff1267: the backslash is added once before the closing ], so [/-0] becomes [/-0\\]. Test added in windows_path_separators.
|
thanks for the work on this! |
| '\\' => push_literal(body.next().unwrap_or('\\'), out), | ||
| '[' => { | ||
| let rest = body.as_str(); | ||
| if let Some(&delimiter @ (b':' | b'.' | b'=')) = rest.as_bytes().first() |
There was a problem hiding this comment.
[[.a.]] and [[=a=]] are copied as is into the onig class, where the inner [ is a literal.
So --exclude='[[.a.]]' doesn't skip a file named a, and it matches .] instead.
could you please translate them to the character itself and add them to gnu_brackets?
There was a problem hiding this comment.
Done in 2ff1267: a single character between [. .] or [= =] becomes that character, so [[.a.]-c] works as a range too. A longer name like [[.ab.]] matches nothing, as in GNU. Added to gnu_brackets.
|
|
||
| const SEPARATOR: &str = if cfg!(windows) { | ||
| r"[/\\]" | ||
| } else if cfg!(target_os = "uefi") { |
There was a problem hiding this comment.
do we really need the uefi case? grep doesn't run there :)
| RegexOptions::REGEX_OPTION_MULTILINE, | ||
| ) | ||
| .map_err(|err| { | ||
| USimpleError::new(2, format!("invalid glob '{pattern}': {}", err.message)) |
There was a problem hiding this comment.
GNU just matches nothing for [z-a], it doesn't fail.
could we do the same instead of exiting with 2?
There was a problem hiding this comment.
Done in 2ff1267. An invalid glob is kept but never matches, so it still counts for the "first pattern is an include" default, same as GNU (--include=[z-a] searches nothing, --exclude=[z-a] skips nothing).
| // A missing file is still reported. | ||
| let mut c = scene.cmd(env!("CARGO_BIN_EXE_grep")); | ||
| c.args(&["--exclude=missing", "grep", "missing"]) | ||
| .fails_with_code(2); |
There was a problem hiding this comment.
please check stderr too, otherwise any error passes
[.a.] and [=a=] inside a bracket now stand for the character. An invalid glob such as [z-a] matches nothing instead of exiting with 2. Paths are matched as bytes, so a non-UTF-8 parent directory no longer hides a valid suffix. On Windows, the backslash is added to a class with '/' as its own member, so [/-0] stays a range.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

--includeand--excludeonly filtered files found under-r; files named on the command line were always searched. GNU skips a command-line file when a pattern matches its whole name or a part right after a path separator.Globs are now translated to Onig regexes (
src/glob.rs, from @lhecker'sbetter-globbingprototype). That adds[:class:]support and lets one regex check every suffix of a command-line path. Theglobdependency is gone.cargo testpasses (126), clippy and rustfmt are clean.--include/--exclude/--exclude-dircases, recursive and on the command line, match GNU grep 3.12, including[[.a.]],[[=a=]], invalid patterns such as[z-a](they match nothing) and non-UTF-8 parent directories.