fix(review-pr): trigger each review bot by its own mechanism - #19
Merged
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
rlorenzo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Owner
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
The old flow pushed every "[bot]" login through requested_reviewers, which cannot work for most of them. gh pr edit --add-reviewer handles Copilot via its @copilot special value, while CodeRabbit and Greptile are GitHub Apps that cannot be requested as reviewers at all and only respond to a mention comment. Detect which bots already reviewed the PR, trigger each by the right mechanism, and ask rather than guess a mention string when a bot is unrecognized. - Poll per bot. The old check took the newest review across all bots, so with two bots it declared success once the fastest reported. - Fix a jq bug where --paginate ran the filter per page, letting a stale page's commit satisfy the head-SHA check. - Trim the command prompts by ~250 words: restated headings, scaffolding labels, and rationale an agent cannot act on. - Check the summary agent's exit code in both review loops, which was captured and discarded, and stop printing a path to a file that was never written. - Remove em dashes from all prompts, scripts, and docs.
rlorenzo
force-pushed
the
claude/trim-command-prompts
branch
from
July 28, 2026 05:56
ea233dc to
f0f7dcb
Compare
|
Warning Review limit reached
Next review available in: 55 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (47)
📝 WalkthroughWalkthroughThe pull request updates agent skills, commands, prompts, review workflows, installer behavior, documentation, comments, and tests. It adds summary-artifact validation, refines bot-review polling, standardizes review output contracts, and preserves custom prompt files during setup. ChangesReview workflow and agent guidance
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 104f70c8-896a-4570-ab0a-a715372ced97
📒 Files selected for processing (46)
.antigravity/skills/code-refinement/SKILL.md.antigravity/skills/code-review/SKILL.md.antigravity/skills/commitmsg/SKILL.md.antigravity/skills/dependency-review/SKILL.md.antigravity/skills/efficient-orchestration/SKILL.md.antigravity/skills/review-pr/SKILL.md.claude/agents/Explore.md.claude/commands/code-refinement.md.claude/commands/code-review.md.claude/commands/commitmsg.md.claude/commands/dependency-review.md.claude/commands/efficient-orchestration.md.claude/commands/review-pr.md.codex/skills/code-refinement/SKILL.md.codex/skills/code-review/SKILL.md.codex/skills/commitmsg/SKILL.md.codex/skills/dependency-review/SKILL.md.codex/skills/efficient-orchestration/SKILL.md.codex/skills/review-pr/SKILL.md.copilot/skills/code-refinement/SKILL.md.copilot/skills/code-review/SKILL.md.copilot/skills/commitmsg/SKILL.md.copilot/skills/dependency-review/SKILL.md.copilot/skills/efficient-orchestration/SKILL.md.copilot/skills/review-pr/SKILL.md.kimi-code/skills/code-refinement/SKILL.md.kimi-code/skills/code-review/SKILL.md.kimi-code/skills/commitmsg/SKILL.md.kimi-code/skills/dependency-review/SKILL.md.kimi-code/skills/efficient-orchestration/SKILL.md.kimi-code/skills/review-pr/SKILL.md.markdownlint-cli2.yamlREADME.mdbin/code-review-loopbin/plan-review-looplib/lib-review-loopprompts/code-review-followup.mdprompts/code-review.mdprompts/plan-review-followup.mdprompts/plan-review.mdsetuptest/code-review-loop.batstest/generate.batstest/lib-review-loop.batstest/plan-review-loop.batstest/smoke
- Compute `stale` from the bots whose latest review predates HEAD and assign `triggered` from it. The poll referenced `triggered` without ever setting it, so comm found nothing pending and broke on the first pass, reporting success without waiting for any re-review. - Confirm the Copilot request by its own login. Any pending human reviewer previously satisfied the check. - Anchor test_review_clean to a verdict line. The unanchored match accepted prose that merely quoted the marker, which this repo's own review reports do. Emphasis is stripped so both the bare and bulleted Summary forms still count. - Assert the exact NO_FURTHER_FEEDBACK sentinel in the contract test rather than the first backticked all-caps token. - Tell dependency-review not to send internal package names to public registry or advisory endpoints.
rlorenzo
force-pushed
the
claude/trim-command-prompts
branch
from
July 28, 2026 07:06
a42f90e to
1465a1d
Compare
Owner
Author
|
@greptileai review |
There was a problem hiding this comment.
rlorenzo has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
There was a problem hiding this comment.
Pull request overview
This PR updates the /review-pr automation guidance so that each review bot is re-triggered using the mechanism that actually works for that bot (e.g., @copilot special reviewer value vs app-bot mention comments), and tightens several review-loop reliability edges (per-bot polling, pagination correctness, and summary/exit-code handling). It also performs a repo-wide prose/style pass aligned with the project’s house style, and adds contract tests to prevent silent sentinel/detector drift.
Changes:
- Update
/review-prworkflow instructions to (1) detect review bots from actual review activity, (2) re-trigger Copilot viagh pr edit --add-reviewer @copilot, and (3) re-trigger app-based bots (CodeRabbit/Greptile) via mention comments, with per-bot polling. - Improve both review loops by clearing stale summary artifacts, checking/carrying through summary agent exit codes, and reporting “not created” when expected outputs are missing.
- Add prompt/detector contract tests and refine verdict detection to avoid false positives from prose quoting the sentinel.
Reviewed changes
Copilot reviewed 44 out of 47 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/smoke | Adjusts smoke-test output wording/punctuation for house style. |
| test/plan-review-loop.bats | Comment style alignment for plan-review-loop tests. |
| test/lib-review-loop.bats | Adds prompt/detector contract tests for sentinels and verdict detection. |
| test/generate.bats | Comment style alignment for generator tests. |
| test/code-review-loop.bats | Comment style alignment for code-review-loop tests. |
| setup | Prose/style adjustments and clearer status/warning messaging in setup flow. |
| README.md | Updates Copilot reviewer re-request guidance and applies house-style prose edits. |
| prompts/plan-review.md | Tightens wording; keeps NO_FURTHER_FEEDBACK sentinel requirement explicit. |
| prompts/plan-review-followup.md | Style tweaks to constraints wording. |
| prompts/code-review.md | Consolidates/clarifies instructions; reiterates exact verdict sentinel requirement. |
| prompts/code-review-followup.md | Style tweaks to constraints wording. |
| lib/lib-review-loop | Updates verdict detection logic and comment style; supports contract tests. |
| bin/plan-review-loop | Adds summary file tracking/removal of stale artifacts and better post-run checks. |
| bin/code-review-loop | Adds summary file tracking/removal of stale artifacts and better post-run checks. |
| .markdownlint-cli2.yaml | Comment punctuation/style tweak. |
| .kimi-code/skills/review-pr/SKILL.md | Updates review-bot re-trigger and polling logic guidance for Kimi. |
| .kimi-code/skills/efficient-orchestration/SKILL.md | House-style prose changes. |
| .kimi-code/skills/dependency-review/SKILL.md | House-style prose changes. |
| .kimi-code/skills/commitmsg/SKILL.md | House-style prose changes. |
| .kimi-code/skills/code-review/SKILL.md | Mirrors updated code-review prompt wording and sentinel emphasis. |
| .kimi-code/skills/code-refinement/SKILL.md | House-style prose changes. |
| .gitignore | Ignores review-pr scratch files (.review-pr-ignored-*). |
| .copilot/skills/review-pr/SKILL.md | Updates review-bot re-trigger and polling logic guidance for Copilot. |
| .copilot/skills/efficient-orchestration/SKILL.md | House-style prose changes. |
| .copilot/skills/dependency-review/SKILL.md | House-style prose changes. |
| .copilot/skills/commitmsg/SKILL.md | House-style prose changes. |
| .copilot/skills/code-review/SKILL.md | Mirrors updated code-review prompt wording and sentinel emphasis. |
| .copilot/skills/code-refinement/SKILL.md | House-style prose changes. |
| .codex/skills/review-pr/SKILL.md | Updates review-bot re-trigger and polling logic guidance for Codex. |
| .codex/skills/efficient-orchestration/SKILL.md | House-style prose changes. |
| .codex/skills/dependency-review/SKILL.md | House-style prose changes. |
| .codex/skills/commitmsg/SKILL.md | House-style prose changes. |
| .codex/skills/code-review/SKILL.md | Mirrors updated code-review prompt wording and sentinel emphasis. |
| .codex/skills/code-refinement/SKILL.md | House-style prose changes. |
| .claude/commands/review-pr.md | Updates review-bot re-trigger and polling logic guidance for Claude command. |
| .claude/commands/efficient-orchestration.md | House-style prose changes. |
| .claude/commands/dependency-review.md | House-style prose changes. |
| .claude/commands/commitmsg.md | House-style prose changes. |
| .claude/commands/code-review.md | Mirrors updated code-review prompt wording and sentinel emphasis. |
| .claude/commands/code-refinement.md | House-style prose changes. |
| .claude/agents/Explore.md | House-style prose changes. |
| .antigravity/skills/review-pr/SKILL.md | Updates review-bot re-trigger and polling logic guidance for Antigravity. |
| .antigravity/skills/efficient-orchestration/SKILL.md | House-style prose changes. |
| .antigravity/skills/dependency-review/SKILL.md | House-style prose changes. |
| .antigravity/skills/commitmsg/SKILL.md | House-style prose changes. |
| .antigravity/skills/code-review/SKILL.md | Mirrors updated code-review prompt wording and sentinel emphasis. |
| .antigravity/skills/code-refinement/SKILL.md | House-style prose changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The anchored match still accepted "Verdict: good to go, but 3 High findings remain", terminating the loop on a report that was not clean. Require the verdict to end the line, allowing only trailing punctuation. Also splits a README run-on introduced when its em dash was removed.
Owner
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
requested_reviewers returns the login as "Copilot", while the review it later submits carries "copilot-pull-request-reviewer[bot]". Checking only the [bot] form reported a failed request after every successful one, which would have sent the poll into its full timeout.
Owner
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 47 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (5)
.claude/commands/review-pr.md:82
- Step 5 has an internal contradiction: it says empty
stalemeans success, but later says "none found" should prompt asking the user. Iflatestreturns no bots at all,staleis empty and the flow would incorrectly treat that as success. Clarify the no-bots-found case separately so the instructions are unambiguous.
Empty `stale` → every bot already covers `head_sha`, success, stop. Otherwise re-trigger each login in `stale`; they do not re-review a push on their own.
| Bot | Login | Re-trigger with |
| --- | --- | --- |
| Copilot | `copilot-pull-request-reviewer[bot]` | `gh pr edit {PR_NUMBER} --add-reviewer @copilot` |
.copilot/skills/review-pr/SKILL.md:82
- Step 5 has an internal contradiction: it says empty
stalemeans success, but later says "none found" should prompt asking the user. Iflatestreturns no bots at all,staleis empty and the flow would incorrectly treat that as success. Clarify the no-bots-found case separately so the instructions are unambiguous.
Empty `stale` → every bot already covers `head_sha`, success, stop. Otherwise re-trigger each login in `stale`; they do not re-review a push on their own.
| Bot | Login | Re-trigger with |
| --- | --- | --- |
| Copilot | `copilot-pull-request-reviewer[bot]` | `gh pr edit {PR_NUMBER} --add-reviewer @copilot` |
.codex/skills/review-pr/SKILL.md:82
- Step 5 has an internal contradiction: it says empty
stalemeans success, but later says "none found" should prompt asking the user. Iflatestreturns no bots at all,staleis empty and the flow would incorrectly treat that as success. Clarify the no-bots-found case separately so the instructions are unambiguous.
Empty `stale` → every bot already covers `head_sha`, success, stop. Otherwise re-trigger each login in `stale`; they do not re-review a push on their own.
| Bot | Login | Re-trigger with |
| --- | --- | --- |
| Copilot | `copilot-pull-request-reviewer[bot]` | `gh pr edit {PR_NUMBER} --add-reviewer @copilot` |
.antigravity/skills/review-pr/SKILL.md:82
- Step 5 has an internal contradiction: it says empty
stalemeans success, but later says "none found" should prompt asking the user. Iflatestreturns no bots at all,staleis empty and the flow would incorrectly treat that as success. Clarify the no-bots-found case separately so the instructions are unambiguous.
Empty `stale` → every bot already covers `head_sha`, success, stop. Otherwise re-trigger each login in `stale`; they do not re-review a push on their own.
| Bot | Login | Re-trigger with |
| --- | --- | --- |
| Copilot | `copilot-pull-request-reviewer[bot]` | `gh pr edit {PR_NUMBER} --add-reviewer @copilot` |
.kimi-code/skills/review-pr/SKILL.md:82
- Step 5 has an internal contradiction: it says empty
stalemeans success, but later says "none found" should prompt asking the user. Iflatestreturns no bots at all,staleis empty and the flow would incorrectly treat that as success. Clarify the no-bots-found case separately so the instructions are unambiguous.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
/review-prcould not actually re-request a review from most bots. Itpushed every
[bot]login through therequested_reviewersRESTendpoint, and the docs it was written against were misread: there is no
single API that covers all review bots.
@copilotspecial valueof
gh pr edit --add-reviewer(gh 2.88.0+). Passing its rawcopilot-pull-request-reviewer[bot]login either errors or exits 0having requested nothing.
requested_reviewers422s for them. A mention comment is their onlytrigger.
Step 5 now identifies which bots already reviewed the PR from their own
activity and triggers each by the right mechanism, asking rather than
guessing when a bot is unrecognized, since a wrong mention string posts
a visible no-op comment.
Bugs fixed along the way
bots, so with two bots it declared success as soon as the fastest
reported.
gh api --paginate --jqruns the filter per page,so
group_by | max_byreturned a per-page maximum and a stale page'scommit could satisfy the head-SHA check. Fixed with
--slurp+add.run_agentcall in either review loop whose exit code was captured and never
checked, and the completion banner printed the summary path whether or
not the file existed. Both loops now warn and print
not created.Prose pass
**Goal:**/**Steps:**scaffolding, and rationale an agent cannot acton. Version numbers and verbatim error strings for loud failures are
out; warnings about silent failures stayed.
efficient-orchestrationno longer hands Codex/Antigravity aClaude-only alias list; harness-specific advice is now labelled.
Verification
tools/generate --checkclean, 49 tests pass. The loop-terminationsentinels
Verdict: good to goandNO_FURTHER_FEEDBACKare unchanged,and this branch adds contract tests that feed each shipped prompt's own
sentinel to the shell detector that greps for it.