perf(rules): multi-pattern merge optimization with Aho-Corasick

[c2j]

Summary

Reduces per-file rule scanning from O(N×S) to O((L+1)×S) where N = number of rules×patterns, S = source size, L = number of literal patterns.

Changes

• Aho-Corasick batch matching: All literal patterns (no metavariables) matched in a single DFA pass via LiteralPatternMatcher
• Regex compile cache: compiled_regexes: HashMap<String, Regex> avoids re-compilation per pattern per file
• Safe fallback: Mixed-pattern and conditioned rules fall through to individual execution
• 12 new regression tests guarding multi-rule correctness invariants (combined == sum of individual, no leakage, overlapping patterns, dialect filtering)
• 2 GaussDB integration tests validating end-to-end correctness with real ogsql parsing

Test Results

astgrep-rules: 248 passed, 0 failed
integration:    7 passed, 0 failed
proptest:       5 passed, 0 failed
GaussDB IT:     2 passed

Files

File	Change
Cargo.toml	+aho-corasick workspace dep, version → 0.4.13
crates/astgrep-rules/Cargo.toml	+aho-corasick dep
crates/astgrep-rules/src/engine/traversal/text_pattern.rs	New — TextPattern + LiteralPatternMatcher
crates/astgrep-rules/src/engine/traversal/execution.rs	Batched execute_rules_sequential + 6 tests
crates/astgrep-rules/src/engine/traversal/types.rs	+classified_patterns, +compiled_regexes, +classify_patterns
crates/astgrep-rules/src/engine/traversal/pattern.rs	&self → &mut self for cache propagation
crates/astgrep-rules/src/lib.rs	3 RuleEngine-level tests
tests/lib/sql_parser_integration_tests.rs	2 GaussDB integration tests

Closes #44

Summary by CodeRabbit

• New Features

  • Improved multi-rule analysis performance by batching eligible text-pattern searches.
  • Added support for overlapping matches and accurate results across SQL statement boundaries.
  • Preserved correct behavior for mixed literal and regular-expression rules.

• Bug Fixes

  • Prevented findings from leaking between rules.
  • Fixed duplicate or missing findings when rules share patterns.
  • Improved dialect-specific matching accuracy, including GaussDB scenarios.

• Tests

  • Added coverage for multi-rule consistency, overlapping matches, and SQL dialect filtering.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cde6fa63-b576-412b-9b2f-1c8490341d96

📥 Commits

Reviewing files that changed from the base of the PR and between 2f514b5 and a99ed89.

📒 Files selected for processing (9)

• Cargo.toml
• crates/astgrep-rules/Cargo.toml
• crates/astgrep-rules/src/engine/traversal/execution.rs
• crates/astgrep-rules/src/engine/traversal/mod.rs
• crates/astgrep-rules/src/engine/traversal/pattern.rs
• crates/astgrep-rules/src/engine/traversal/text_pattern.rs
• crates/astgrep-rules/src/engine/traversal/types.rs
• crates/astgrep-rules/src/lib.rs
• tests/lib/sql_parser_integration_tests.rs

---

📝 Walkthrough

Walkthrough

The PR adds Aho-Corasick literal matching for multi-rule execution, caches compiled regexes, preserves fallback paths for complex rules, and adds unit, crate-level, and GaussDB integration coverage.

Changes

Batch literal rule matching

Layer / File(s)	Summary
Pattern classification and matching Cargo.toml, crates/astgrep-rules/Cargo.toml, crates/astgrep-rules/src/engine/traversal/{mod.rs,text_pattern.rs}	Adds the Aho-Corasick dependency, exposes text-pattern support, classifies literal and regex patterns, and scans overlapping literal matches with rule metadata.
Engine batching and regex caching crates/astgrep-rules/src/engine/traversal/{types.rs,execution.rs,pattern.rs}	Batches eligible literal rules, creates findings from match spans, handles SQL statement offsets, caches compiled regexes, and updates traversal helpers to use mutable engine state.
Multi-rule correctness coverage crates/astgrep-rules/src/engine/traversal/text_pattern.rs, crates/astgrep-rules/src/lib.rs, tests/lib/sql_parser_integration_tests.rs	Tests combined versus individual execution, dialect filtering, overlapping patterns, duplicate prevention, rule isolation, and GaussDB behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RuleExecutionEngine
  participant LiteralPatternMatcher
  participant RegexCache
  participant Findings
  RuleExecutionEngine->>RuleExecutionEngine: classify patterns
  RuleExecutionEngine->>LiteralPatternMatcher: scan eligible literal patterns
  LiteralPatternMatcher-->>RuleExecutionEngine: grouped match spans
  RuleExecutionEngine->>Findings: create literal findings
  RuleExecutionEngine->>RegexCache: retrieve or compile complex pattern regex
  RegexCache-->>RuleExecutionEngine: compiled regex
  RuleExecutionEngine->>Findings: append fallback findings

Loading

Possibly related PRs

• c2j/astgrep#8: Related traversal dispatch and compiled-regex span-matching changes.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/multi-pattern-merge

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}