fix(e2): match shell env harvesting across grep flags and quoting - #483
Open
MandoCodes1 wants to merge 1 commit into
Open
fix(e2): match shell env harvesting across grep flags and quoting#483MandoCodes1 wants to merge 1 commit into
MandoCodes1 wants to merge 1 commit into
Conversation
The shell arm of E2 only matched `env | grep` followed by an optional `-i` and a bare keyword, so `env | grep -i -E 'token|key|secret'`, `env | grep -iE "aws_|secret"` and `env | egrep -e password` all scored as clean. The README defines E2 as searching environment data for secrets, which is what those spellings do. Widen the pattern to accept env or printenv as the source, grep, egrep or fgrep as the filter, any number of short or long flags, optional quoting, and a keyword anywhere in the first 40 characters of the pattern argument. The keyword has to start at a name boundary so MONKEY_PATCH does not match on KEY, the argument scan stops at quotes, backticks, shell separators and comments, and `grep -v` is excluded because inverting the match is the redaction idiom. The flag run is possessive so a long run of flags cannot backtrack. Add pattern tests for eight harvesting spellings, nine ordinary or inverted lookups and a backtracking bound, plus a SKILL.md fixture with a CLI regression test. Signed-off-by: Miguel Orti Vila <miguelortivila@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #482.
E2_OTHER_PATTERNS(static_patterns_data_exfiltration.py:73) matchedenv | grep, one optional-i, then a bare keyword. Any second flag, combined flags, a quoted pattern oregrepfell through, soenv | grep -i -E 'token|key|secret' > /tmp/ctx.txtscanned clean whileenv | grep secretscored HIGH.The pattern now accepts
envorprintenvas the source,grep,egreporfgrepas the filter, any run of short or long flags, an optional opening quote, and the keyword within the first 40 characters of the pattern argument. Three guards keep the wider match from costing precision: the keyword has to start at a name boundary (AWS_SECRET_ACCESS_KEYmatches,MONKEY_PATCHdoes not), the argument scan stops at quotes, backticks,;,>,&,#and newline, and-v/--invert-matchis excluded because inverting the match is the redaction idiom rather than harvesting. The flag run is possessive, so a long run of flags is linear rather than exponential; a test pins that at 60 repetitions.Static scan,
--no-llm, oneSKILL.mdwith one fenced line:env | grep secretenv | grep -i -E 'token|key|secret' > /tmp/ctx.txtenv | grep -iE "aws_|secret"env | grep --ignore-case tokenenv | egrep -e password -e tokenenv | grep AWS_SECRET_ACCESS_KEYenv | grep PATHenv | grep MONKEY_PATCHprintenv | grep -v -E 'KEY|SECRET|TOKEN'dotenv | grep KEYTests: eight harvesting spellings, nine ordinary or inverted lookups and a backtracking bound in
tests/unit/test_patterns.py, plustests/fixtures/e2_shell_env_harvest/with a CLI regression test intests/unit/test_cli.py. Full suite 3978 to 3997 passed, same 14 skipped and 4 xfailed either side.ruff checkandruff format --checkclean.Deliberately left out:
env > file,export -pandset(ordinary debugging uses, no keyword to key on), non-grep filters such asrgandawk, and filters behind an intermediate stage such asenv | sort | greporenv | tee. Confidence stays at 0.8.