Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b69b274
feat: configurable commit overview with per-author counts and line to…
LearningCircuit Sep 14, 2026
d4536b0
feat: default MAX_COMMIT_MESSAGES to 3
LearningCircuit Sep 14, 2026
d15cb3f
feat: check-run summary, configurable human-comment caps, label guidance
LearningCircuit Sep 14, 2026
3501fc8
fix: detect human-comment clipping from the source length
LearningCircuit Sep 14, 2026
ede8f84
feat: severity tags, highlighted inferences, and a Should-be-checked …
LearningCircuit Sep 14, 2026
2206dcf
fix: drop the commit-stats temp file; state truncation budget in bytes
LearningCircuit Sep 14, 2026
a7fdc09
feat: default model to z-ai/glm-5.3
LearningCircuit Sep 14, 2026
941c39e
fix: paginate comment and check-run fetches; mark message-list clips
LearningCircuit Sep 14, 2026
d4defa8
fix: mark prior-review clips, report unavailable line stats, forward …
LearningCircuit Sep 14, 2026
a78fe9a
fix: exact merge predicate, UTF-8-safe clips, marked description cut
LearningCircuit Sep 14, 2026
b1b2cb4
fix: count merges directly instead of total-minus-nonmerge
LearningCircuit Sep 14, 2026
6a705f3
fix: cap red-shard list, fetch PR object once, forward all knobs
LearningCircuit Sep 14, 2026
aff45c5
fix: reject partial pagination output; pin the UTF-8 guard
LearningCircuit Sep 14, 2026
5f6e842
fix: warn on PR-object fetch failure; correct byte wording
LearningCircuit Sep 14, 2026
e5fc1e4
chore: drop stray pyc committed by a local test run
LearningCircuit Sep 14, 2026
8a46c68
fix: use -B in documented test command; ignore __pycache__
LearningCircuit Sep 14, 2026
d88b1f4
fix: full commit-message bodies, neutral-run wording, stats warning
LearningCircuit Sep 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .github/workflows/ai-code-reviewer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,29 @@ jobs:

- name: AI Code Review
env:
# The || fallbacks deliberately mirror the script's built-in
# defaults so this block documents every knob at the call site —
# keep them in sync when changing a script default.
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
AI_MODEL: ${{ vars.AI_MODEL || 'minimax/minimax-m2.5' }}
AI_MODEL: ${{ vars.AI_MODEL || 'z-ai/glm-5.3' }}
AI_TEMPERATURE: ${{ vars.AI_TEMPERATURE || '0.1' }}
AI_MAX_TOKENS: ${{ vars.AI_MAX_TOKENS || '64000' }}
MAX_DIFF_SIZE: ${{ vars.MAX_DIFF_SIZE || '800000' }}
EXCLUDE_FILE_PATTERNS: ${{ vars.EXCLUDE_FILE_PATTERNS || '*.lock,*.min.js,*.min.css,package-lock.json,yarn.lock' }}
MAX_SUMMARY_COMMITS: ${{ vars.MAX_SUMMARY_COMMITS || '15' }}
MAX_COMMIT_MESSAGES: ${{ vars.MAX_COMMIT_MESSAGES || '3' }}
INCLUDE_COMMIT_SUMMARY: ${{ vars.INCLUDE_COMMIT_SUMMARY || 'true' }}
INCLUDE_PREVIOUS_REVIEWS: ${{ vars.INCLUDE_PREVIOUS_REVIEWS || 'true' }}
INCLUDE_HUMAN_COMMENTS: ${{ vars.INCLUDE_HUMAN_COMMENTS || 'true' }}
INCLUDE_CHECK_RUNS: ${{ vars.INCLUDE_CHECK_RUNS || 'true' }}
INCLUDE_LABELS: ${{ vars.INCLUDE_LABELS || 'true' }}
INCLUDE_PR_DESCRIPTION: ${{ vars.INCLUDE_PR_DESCRIPTION || 'true' }}
INCLUDE_COMMIT_MESSAGES: ${{ vars.INCLUDE_COMMIT_MESSAGES || 'true' }}
MAX_HUMAN_COMMENTS: ${{ vars.MAX_HUMAN_COMMENTS || '100' }}
MAX_HUMAN_COMMENT_LENGTH: ${{ vars.MAX_HUMAN_COMMENT_LENGTH || '4000' }}
MAX_HUMAN_COMMENTS_TOTAL: ${{ vars.MAX_HUMAN_COMMENTS_TOTAL || '20000' }}
STRUCTURED_OUTPUT: ${{ vars.STRUCTURED_OUTPUT || 'true' }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO_FULL_NAME: ${{ github.repository }}
FAIL_ON_REQUESTED_CHANGES: ${{ vars.FAIL_ON_REQUESTED_CHANGES || 'false' }}
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__pycache__/
*.pyc
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ This guide explains how to set up the automated AI PR review system using OpenRo
**Latest Updates:**
- **Thinking Model Support**: Now supports advanced reasoning models like Kimi K2 that use `<thinking>` tags
- **Rich Context**: Includes PR descriptions, commit messages, and human comments for comprehensive reviews
- **Commit Overview**: Tells the model how many commits are already on a PR, who authored them, and how many lines each author changed — with caps configured separately from the (token-heavy) fully quoted messages
- **Higher Token Limits**: Default 64k tokens for complete reviews without truncation
- **Smart Context Management**: Only fetches most recent AI review to save tokens
- **Enhanced Error Handling**: Robust parsing of various AI response formats
Expand Down Expand Up @@ -63,7 +64,7 @@ The review is posted as a single concise comment on your pull request with appro

The workflow is pre-configured with sensible defaults, but you can customize it by setting repository variables in **Settings** → **Secrets and variables** → **Actions** → **Variables**:

- **AI_MODEL**: Change the AI model (default: `moonshotai/kimi-k2-thinking`)
- **AI_MODEL**: Change the AI model (default: `z-ai/glm-5.3`)
- See [OpenRouter models](https://openrouter.ai/models) for options
- Recommended: Models with reasoning capabilities (Kimi K2, o1, etc.)
- **AI_TEMPERATURE**: Adjust randomness (default: `0.1` for consistent reviews)
Expand All @@ -72,9 +73,15 @@ The workflow is pre-configured with sensible defaults, but you can customize it
- For large PRs with thinking models, this prevents cut-off responses
- Adjust lower for cost savings on smaller PRs
- **MAX_DIFF_SIZE**: Maximum diff size in bytes (default: `800000` / 800KB)
- **MAX_SUMMARY_COMMITS**: How many of the PR's most recent commits the commit overview reads (default: `15`; `0` shows the commit count only). The overview tells the model how many commits are already on the PR, who made them, and each author's added/removed line totals. Each summarized commit costs one extra GitHub API call, but only a handful of numbers enter the prompt, so this cap can stay generous.
- **MAX_COMMIT_MESSAGES**: How many commit messages are fully quoted in the prompt (default: `3`). Fully quoted messages are the token-expensive part of the commit history, hence the separate, smaller cap — the overview (above) still covers many more commits.
- **INCLUDE_COMMIT_SUMMARY**: Include the "There are X commits already on this PR" overview with per-author counts and line totals (default: `true`)
- **MAX_HUMAN_COMMENTS**: How many of the newest human comments are included (default: `100`; `0` includes none at all). Comments are presented newest-first, so when this or the overall budget clips, the oldest go first — the latest feedback always survives.
- **MAX_HUMAN_COMMENT_LENGTH**: Maximum characters per human comment; longer comments are clipped and marked " […truncated]" (default: `4000`)
- **MAX_HUMAN_COMMENTS_TOTAL**: Overall byte budget for the human-comments block (`head -c`); when exceeded, the block is cut and marked (default: `20000`; `0` omits the block entirely)
- **STRUCTURED_OUTPUT**: Enforce a JSON Schema on the model's output via OpenRouter structured outputs (default: `true`)
- Makes the provider emit valid, correctly-escaped JSON instead of the model hand-writing it — the main cause of "Invalid JSON response from AI model"
- Requires a model/provider that supports `response_format` json_schema (most modern models do; e.g. Kimi K2, MiniMax M2.5)
- Requires a model/provider that supports `response_format` json_schema (most modern models do; e.g. GLM 5.3, Kimi K2, MiniMax M2.5)
- Set to `false` only if your chosen model doesn't support structured outputs
- **DEBUG_MODE**: Enable debug logging (default: `false`)
- ⚠️ Warning: Exposes code diff in workflow logs when enabled
Expand Down Expand Up @@ -103,11 +110,11 @@ This will generate a fresh review of the current PR state.

## Review Results

The AI reviews your code across all focus areas and reports actionable findings ordered by severity, with a location, failure scenario, impact, and suggested fix. It omits praise, change summaries, and empty sections. A clean review says "No actionable findings." followed by the verdict. Concise output does not lower the token budget available for reasoning and findings. The review is meant to assist human reviewers, not replace them.
The AI reviews your code across all focus areas and reports actionable findings as bullets tagged **must fix**, **should fix**, or **nit** (in that order), each with a location, failure scenario, impact, and suggested fix. Inferences are highlighted with an explicit "Inference (not verified):" label so they are never mistaken for verified facts, and anything that cannot be verified from the diff but is worth a human look is collected in a final "Should be checked" section. The review omits praise, change summaries, and empty sections; a clean review says "No actionable findings." followed by the verdict. Concise output does not lower the token budget available for reasoning and findings. The review is meant to assist human reviewers, not replace them.

## Cost Estimation

Costs with the default Kimi K2 thinking model are very affordable. Based on real usage data:
Costs are very affordable. The ranges below are estimates carried over from real usage with the previous default model (Kimi K2) — re-check against current [OpenRouter pricing](https://openrouter.ai/models) for GLM 5.3:

**Typical Costs:**
- Small PR (< 1000 lines): $0.01 - $0.02
Expand All @@ -120,7 +127,7 @@ Costs with the default Kimi K2 thinking model are very affordable. Based on real
- **Total cost: $0.01 - $0.05 per review**

**Why So Affordable:**
- Kimi K2 has competitive pricing (~$0.001-$0.003 per 1k tokens)
- GLM 5.3 has competitive pricing (see OpenRouter)
- Smart context management (only most recent AI review, limited commit history)
- Most PRs are smaller than you think in token count
- The 64k token limit is a ceiling, not typical usage
Expand All @@ -131,7 +138,7 @@ Costs with the default Kimi K2 thinking model are very affordable. Based on real
- Number of human comments and commit messages included
- OpenRouter provider routing (prices vary slightly by provider)

Check [OpenRouter pricing](https://openrouter.ai/models) for current Kimi K2 rates.
Check [OpenRouter pricing](https://openrouter.ai/models) for current GLM 5.3 rates.

## Customization

Expand Down Expand Up @@ -187,11 +194,11 @@ If you get a "Diff is too large" error:
The workflow fetches and sends these repository elements to the AI:
1. **Code Changes**: Full diff of modified files
2. **PR Description**: Title and description text from the pull request
3. **Commit Messages**: Up to 15 most recent commit messages (excluding merges)
4. **Human Comments**: Comments from human reviewers on the PR; bot comments are excluded, while human comments quoting a review header or marker are retained
5. **Labels**: All repository labels with descriptions and colors
6. **Previous AI Review**: Most recent bot-authored AI review comment only (limited to 10k chars), identified by its review header or `<!-- ai-code-review:sticky -->` marker
7. **CI/CD Status**: GitHub Actions check runs and build statuses
3. **Commit Messages**: Up to `MAX_COMMIT_MESSAGES` most recent commit messages (default 3, excluding merges), plus a compact overview stating how many commits are already on the PR, the per-author commit counts, and each author's added/removed line totals (covering up to `MAX_SUMMARY_COMMITS` most recent commits, default 15)
4. **Human Comments**: Comments from human reviewers on the PR, fetched across all pages (not just the first 30), newest first; bot comments are excluded, while human comments quoting a review header or marker are retained. Caps (`MAX_HUMAN_COMMENTS`, `MAX_HUMAN_COMMENT_LENGTH`, `MAX_HUMAN_COMMENTS_TOTAL`) clip the oldest first and mark any truncation.
5. **Labels**: All repository labels with descriptions and colors (kept complete on purpose; the prompt instructs the model to only apply genuinely useful ones)
6. **Previous AI Review**: Most recent bot-authored AI review comment only (limited to 10,000 bytes, marked when truncated), identified by its review header or `<!-- ai-code-review:sticky -->` marker
7. **CI/CD Status**: A one-line summary of GitHub Actions check runs ("N of M checks passed") plus only the non-passing runs — failures, skipped, cancelled, timed out, or still running — listed individually (capped at 20 lines with a "+K more" line); fully green matrix shards no longer flood the prompt
8. **PR Metadata**: Pull request details, head SHA, repository information
9. **Files**: May include sensitive configuration files, keys, or credentials

Expand Down Expand Up @@ -219,4 +226,4 @@ For issues with:

## Development Tests

Run `python3 -m unittest discover -s tests -v` to check the generated request and comment context filters. The tests use local substitutes for GitHub and OpenRouter, with no network requests or model calls. They require Python 3 and the script dependencies (Bash, jq, and Perl).
Run `python3 -B -m unittest discover -s tests -v` to check the generated request and comment context filters. The tests use local substitutes for GitHub and OpenRouter, with no network requests or model calls. They require Python 3 and the script dependencies (Bash, jq, and Perl). The `-B` flag keeps Python from writing `__pycache__` into the tree (CI uses it for the same reason; `.gitignore` covers it as a backstop).
Loading