Skip to content

feat(slack): tighten mention tokens and nudge toward PRs for code-fixable issues#2691

Merged
VojtechBartos merged 3 commits into
mainfrom
vojtab/slack-suggest-pr
Jun 17, 2026
Merged

feat(slack): tighten mention tokens and nudge toward PRs for code-fixable issues#2691
VojtechBartos merged 3 commits into
mainfrom
vojtab/slack-suggest-pr

Conversation

@VojtechBartos

@VojtechBartos VojtechBartos commented Jun 16, 2026

Copy link
Copy Markdown
Member

Two small Slack-origin system prompt tweaks in the cloud agent (both gated on getCloudInteractionOrigin() === "slack").

Companion to PostHog/posthog#64226, which restructures the Slack thread context the agent sees: messages are rendered as <@U…|displayname>: followed by an indented body, with role annotations like "Thread started by" / "Tagged the PostHog app". The labeled mention tokens that show up in that context block are what this PR teaches the agent to reuse verbatim.

1. Mention tokens — forbid fabrication

PR #2676 told the Slack-bound agent the labeled Slack mention format (<@U…|displayname>) and instructed it to copy tokens verbatim from context. In practice the agent learned the shape but still invented user IDs when the same person came up later in its reply.

Observed example: in a single reply the agent emitted <@U02DXFTVDNE|displayname> correctly in the first paragraph (copied from context), then <@displayname|displayname> in the second paragraph — substituting the display name for the U… ID. Slack does not render the second form as a real ping.

This PR tightens the rule:

  • Mention tokens must already appear in the message context; the agent reuses them rather than constructing them.
  • Substituting the display name (or any other string) for the U… ID is explicitly disallowed, with the bad and good forms shown.
  • If no labeled token exists in context, fall back to plain text — do not invent one.

2. Suggesting code changes

Adds a short instruction to the Slack-only identity block: after answering, if the question describes a plausibly code-fixable issue (bug in errors/logs, broken instrumentation, funnel step traceable to UI code, stale repo config), offer to open a PR. Skipped for pure data lookups and when no reachable repo could host the fix.

Goal: surface PostHog Code's PR-opening capability to Slack users who currently only use the analytical side.

@github-actions

github-actions Bot commented Jun 16, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit 98a4802.

@VojtechBartos VojtechBartos self-assigned this Jun 16, 2026
@greptile-apps

greptile-apps Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
packages/agent/src/server/agent-server.ts:1746-1747
The PR-nudge instruction is included in `identityInstructions` for every Slack-origin run, including those where `createPr` is `false`. In that mode the main body of the prompt explicitly forbids creating PRs ("Do NOT create a branch, commit, push, or open a pull request unless the user explicitly asks"), so the agent receives contradictory guidance: proactively offer a PR vs. never open one. The nudge should be gated on `shouldAutoCreatePr`, which is already computed above the string.

```suggestion
${shouldAutoCreatePr ? `
# Suggesting code changes
You can also open pull requests directly from this Slack thread. When the user's question describes a problem with a plausible code-side fix — a bug visible in errors or logs, missing or broken instrumentation, a broken funnel step traceable to UI code, a stale config that lives in a repo — end your reply with a one-sentence offer to open a PR for the fix and ask if they want you to proceed. Skip the offer for pure data lookups with no actionable code change (e.g. "what was DAU yesterday?"), and skip it when the fix would clearly live outside any repo you can reach.` : ""}
```

Reviews (1): Last reviewed commit: "feat(slack): nudge users toward opening ..." | Re-trigger Greptile

Comment thread packages/agent/src/server/agent-server.ts
@VojtechBartos VojtechBartos changed the title feat(slack): nudge users toward opening a PR for code-fixable issues feat(slack): tighten mention tokens and nudge toward PRs for code-fixable issues Jun 17, 2026
Tighten the Slack-origin system prompt rule on user mentions: require
that the `<@U…|displayname>` token come from the thread context
verbatim, and explicitly forbid substituting the display name (or any
other string) for the `U…` ID. Falls back to plain text when no
labeled token exists.

Follow-up to #2676: the agent learned the token's shape but invented
the user ID — observed in a Slack reply rendering `<@paul|Paul D'Ambra>`,
which Slack doesn't ping.
@VojtechBartos VojtechBartos force-pushed the vojtab/slack-suggest-pr branch from 75f1500 to b9cb6a1 Compare June 17, 2026 10:01
@VojtechBartos VojtechBartos requested a review from a team June 17, 2026 12:42
@VojtechBartos VojtechBartos marked this pull request as ready for review June 17, 2026 12:42
@greptile-apps

greptile-apps Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "chore(slack): swap real name in mention ..." | Re-trigger Greptile

@VojtechBartos VojtechBartos merged commit bbc3ace into main Jun 17, 2026
27 checks passed
@VojtechBartos VojtechBartos deleted the vojtab/slack-suggest-pr branch June 17, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants