Repository navigation
fix(interp): handle brace-generated invalid glob patterns without panicking - #704
Conversation
…icking
Fuzzing the test builtin (FuzzTestNesting) found that a script whose brace
expansion synthesizes an invalid glob pattern crashes expansion. For example
`{a..A}` expands over the ASCII range that includes '[' and ']', so
`test {a..A}1-{a..A}0{a..A}` produces a field like `[1-0]` — an invalid
character-class range. mvdan.cc/sh's glob matcher (internal.ExtendedPatternMatcher)
compiles the derived pattern with regexp.MustCompile, which panics rather than
returning an error. The panic unwound through field expansion to Run's recovery
and surfaced as the generic "internal error" sentinel (a non-ExitStatus error),
failing callers that expect a normal exit code.
Field expansion now recovers that specific panic (identified by the
"regexp: Compile(" prefix of regexp.MustCompile's panic value) and converts it
into a clean expansion error; any other panic is re-raised so genuine bugs
still surface. The same guard wraps the two FieldsSeq loops in the static
checker. Valid globbing and no-match literal behavior are unchanged.
Tests: TestBraceGeneratedInvalidGlobDoesNotPanic and TestValidGlobStillExpands
(interp), plus the crashing input added as a FuzzTestNesting seed. Full
go test -race ./... passes.
Environment: Datadog workspace
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
julesmcrt
left a comment
There was a problem hiding this comment.
Reviewed f7698a64.
No blocking findings — this is a clean, correctly-scoped fix. Verified the root cause end to end: pattern.Regexp cleanly rejects literal [1-0]/[z-a], so the actual crasher is the single [1-\0] field among the 16,386 produced by {a..A}1-{a..A}0{a..A} (\, 0x5C, is inside the {a..A} range); the escape is stripped, the emitted regexp source contains the invalid 1-0 range, and internal.ExtendedPatternMatcher panics via regexp.MustCompile. The regexp: Compile( prefix guard matches that panic value exactly, and the type switch re-raises everything else unchanged.
Completeness check: every user-input→pattern-compilation path was enumerated and probed. All runtime field expansion funnels through the now-guarded fieldCollector.add (command args, for ... in, redirects, command substitution incl. nested), and both checker FieldsSeq loops now go through forEachCheckField. mvdan's other MustCompile sites are unreachable: ${x#pat}-style operators, case, and [[ ]] are all rejected by interp/validate.go; find's -name/-path matching is hand-rolled without regexp; grep/sed/awk/ntfs use error-checked regexp.Compile/filepath.Match. A 20s fuzz run (1.65M execs) of FuzzTestNesting found no remaining crashers. The converted error is a clean expansion failure (invalid glob pattern in expansion, exit 1, ExitStatus-compatible — verified the sentinel never surfaces), and the checker marks affected commands indeterminate.
Minor, non-blocking:
- P3 — guard robustness (
interp/runner_expand.go:320): the prefix match is a Go-internal panic-message contract; a future toolchain rewording would silently disable the guard and resurrect the non-ExitStatus "internal error" path. Consider pinning the contract with a unit test that triggersregexp.MustCompileon a known-bad translation (e.g.(?s)^[1-0]$) and assertsisGlobPatternPanicaccepts the value, so a toolchain change fails loudly; or make the guard two-factor (prefix +"error parsing regexp"). - P3 — comment accuracy (
interp/glob_pattern_panic_test.go:29): the test comment (and PR description) say the crasher "produces a field like[1-0]"; in fact the panicking field is[1-\0], and thedirect_invalid_class/reversed_classsubtests pin behavior that never panicked pre-fix. Worth naming the embedded escape as the essential ingredient so future tests don't reproduce a non-crasher. - P3 — non-fatal semantics (
interp/runner_expand.go:340): the converted error takes the defaultexpandErrarm, so the script continues after the skipped command (exit 1), unlike limit and "not supported" errors which abort. Benign since every later expansion re-guards itself, but a one-line comment stating the intent would help.
Tested:go vetclean;go test -race ./interp/... ./builtins/tests/testcmd/...green; new regression tests pass; seed corpus green; 20s fuzz clean. Nice fix.
Summary
Fuzzing the
testbuiltin (FuzzTestNesting) found a script whose brace expansion synthesizes an invalid glob pattern, crashing field expansion.{a..A}expands over the ASCII range that includes[and], sotest {a..A}1-{a..A}0{a..A}produces a field like[1-0]— an invalid character-class range ('1' > '0'). mvdan.cc/sh's glob matcher (internal.ExtendedPatternMatcher) compiles the derived pattern withregexp.MustCompile, which panics instead of returning an error. The panic unwound through field expansion toRunner.Run's recovery and surfaced as the genericinternal errorsentinel (a non-ExitStatuserror), which fails callers that expect a normal exit code.This was surfaced by the
rshell:fuzz:builtins/tests/testcmdjob on the rshell→datadog-agent migration (heavier fuzzing with a larger cached corpus reached the input); it reproduces on currentmainas well.Fix
Field expansion now recovers only that specific panic — identified by the
regexp: Compile(prefix ofregexp.MustCompile's panic value — and converts it into a clean expansion error; any other panic is re-raised so genuine interpreter bugs still surface. The same narrow guard wraps the twoexpand.FieldsSeqloops in the static checker (Check), where the panic could also occur. Valid globbing and no-match literal behavior are unchanged.The guard uses only allowlisted interp symbols (a type switch +
strings.HasPrefix), so it does not touch the symbol allowlist.Tests
interp/glob_pattern_panic_test.go:TestBraceGeneratedInvalidGlobDoesNotPanic— the crasher and related invalid-class inputs terminate with a normal exit status, never the recovered-panic sentinel (fails onmain, passes here).TestValidGlobStillExpands—*.txt,[ab].txt,[a-z].txt, and no-match globs still behave correctly.{a..A}1-{a..A}0{a..A}is added as aFuzzTestNestingseed so the seed-corpus CI run always exercises it.Full
go test -race ./...passes.Note
The underlying
MustCompile-panic-on-invalid-pattern is an upstream mvdan.cc/sh behavior; this change contains it at rshell's expansion boundary rather than relying onRun's generic panic recovery.🤖 Generated with Claude Code