Skip to content

fix(agent): Don't let long running local checks prevent draft-PR creation#2693

Open
mp-hog wants to merge 1 commit into
mainfrom
posthog-code/draft-pr-no-gating
Open

fix(agent): Don't let long running local checks prevent draft-PR creation#2693
mp-hog wants to merge 1 commit into
mainfrom
posthog-code/draft-pr-no-gating

Conversation

@mp-hog

@mp-hog mp-hog commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Problem

I ran into a problem that a PR wasn't created and subsequently the sandbox got killed without snapshot so that all work was lost. The PR creation was blocked because of long-running checks that were not needed as the coding task was verified already.

This PR modifies the PR-creation instructions to not gate draft PRs if the user requests to create one.

More context: https://posthog.slack.com/archives/C09G8Q32R6F/p1781593261186719

Changes

Added a reusable draftPrVerificationGuidance block to buildCloudSystemPrompt() in packages/agent/src/server/agent-server.ts and wired it into every PR-capable branch (review-first connected-repo, auto-publish, and the publish-enabled no-repository clone path). It instructs the agent to:

  • Open the draft PR first, never gating creation on long-running local checks.
  • Handle verification after the draft is open, without assuming the repo has CI: rely on CI if present, otherwise tell the user and offer to run the relevant checks as a follow-up task when the agent judges it warranted.

How did you test this?

  • pnpm --filter @posthog/agent typecheck — passes
  • biome check on both changed files — clean
  • pnpm --filter @posthog/agent test agent-server.test.ts — 64/64 pass, including new assertions covering the guidance across the review-first, auto-publish, and no-repository branches (and its absence when createPr: false)

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Created with PostHog Code

The cloud agent treated "all checks green" as a hard prerequisite for opening
a draft PR — kicking off a full typecheck/test/build and polling for it, so the
draft was sometimes never created.

Add a reusable `draftPrVerificationGuidance` block to the cloud system prompt in
`buildCloudSystemPrompt()` and wire it into every PR-capable branch (review-first,
auto-publish, and the publish-enabled no-repository clone path). It instructs the
agent to open the draft FIRST, then handle verification without blocking: rely on
CI if the repo has it, otherwise offer to run checks as a follow-up task when the
agent judges it warranted. Does not assume CI exists.

Lives in the product prompt (injected into every cloud run) so the fix applies to
any user repo, not just this one.

Generated-By: PostHog Code
Task-Id: f899b653-ef6d-4af0-bad6-1bce6334f06e
@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit c0ee634.

@greptile-apps

greptile-apps Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff (1)

  1. packages/agent/src/server/agent-server.test.ts, line 1471-1479 (link)

    P2 signal_report test missing new assertions — and both tests are candidates for parameterisation

    The Slack-origin test now asserts "Open the draft PR without blocking on long local checks" and "Opening the PR — do not block on long checks", but the equivalent signal_report test doesn't. Both origins hit the same shouldAutoCreatePr branch and produce identical prompts, so the gap is just a test-coverage hole. Per the team's preference for parameterised tests, the two auto-PR tests could be combined with it.each([{ origin: "slack" }, { origin: "signal_report" }], ...) and a shared shouldContain list — similar to the no-repository block above. That would keep the two paths in sync automatically.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: packages/agent/src/server/agent-server.test.ts
    Line: 1471-1479
    
    Comment:
    **`signal_report` test missing new assertions — and both tests are candidates for parameterisation**
    
    The Slack-origin test now asserts `"Open the draft PR without blocking on long local checks"` and `"Opening the PR — do not block on long checks"`, but the equivalent `signal_report` test doesn't. Both origins hit the same `shouldAutoCreatePr` branch and produce identical prompts, so the gap is just a test-coverage hole. Per the team's preference for parameterised tests, the two auto-PR tests could be combined with `it.each([{ origin: "slack" }, { origin: "signal_report" }], ...)` and a shared `shouldContain` list — similar to the `no-repository` block above. That would keep the two paths in sync automatically.
    
    How can I resolve this? If you propose a fix, please make it concise.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
packages/agent/src/server/agent-server.test.ts:1471-1479
**`signal_report` test missing new assertions — and both tests are candidates for parameterisation**

The Slack-origin test now asserts `"Open the draft PR without blocking on long local checks"` and `"Opening the PR — do not block on long checks"`, but the equivalent `signal_report` test doesn't. Both origins hit the same `shouldAutoCreatePr` branch and produce identical prompts, so the gap is just a test-coverage hole. Per the team's preference for parameterised tests, the two auto-PR tests could be combined with `it.each([{ origin: "slack" }, { origin: "signal_report" }], ...)` and a shared `shouldContain` list — similar to the `no-repository` block above. That would keep the two paths in sync automatically.

Reviews (1): Last reviewed commit: "fix(agent): don't gate draft-PR creation..." | Re-trigger Greptile

@mp-hog mp-hog marked this pull request as ready for review June 16, 2026 09:04
@mp-hog mp-hog changed the title fix(agent): don't gate draft-PR creation on long local checks fix(agent): Don't let long running local checks prevent draft-PR creation Jun 16, 2026
@mp-hog mp-hog requested review from a team, charlesvien and georgemunyoro June 16, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant