feat(agents): post triage assessments through a Slack app so replies can thread #4898

User Story

I'm on OpenShell triage duty. The triage agent (#4897) will assess each new issue. In #openshell-triage I want to see the agent's recommendation for each issue at a glance, discuss it with other maintainers in a thread, and act on it from Slack without switching to GitHub. A channel webhook can only post top-level messages and can't hear replies, so I need a Slack bot.

Description

A Slack bot in #openshell-triage would carry each issue from assessment to disposition. The #4896 redesign (draft PR #4899) sketched it:

• One top-level post per state:triage-needed issue, written by a deterministic GitHub Action, not by the agent.

  • The post would carry only the linked issue number, the title, the submitter, and possibly a few other facts. No raw issue content would go to Slack.
  • The Action would escape &, <, and >.
  • At first the Action would use the existing channel webhook, which already works. Once the bot exists, the Action would post with the bot token instead. chat.postMessage would then return the post's ts, and the bot could edit the post, since it would be the bot's own message.

• A four-emoji code per issue in the top-level post, updated as triage progresses:

  • priority: 🔴 🔵 ⚪
  • recommendation: 👍 ❓ 👎
  • confidence: 🧠 🤔 🤷
  • evidence: 💪 👌 🤏

  While the Action still posts through the webhook, the bot can't edit the post or learn its ts. The code would wait for the switch to the bot token.

• One thread reply per triage-* skill, plus follow-ups such as a re-assessment after the reporter adds evidence.

• Maintainer commands in the thread: accept, valid, and decline, or plain-language requests.

  • decline would confirm its wording before posting.
  • The bot would make the GitHub change and update the code.
  • Slack doesn't run slash commands in threads, so maintainers would @-mention the bot (@bot accept) instead of typing /accept.

• Who may run a command: the requester must have connected the GitHub Slack app, so the bot knows their GitHub identity. That identity must also have permission on GitHub for the requested change.

• Reactions: 👀 claims an issue and ✅ marks it resolved.

• Claude Managed Agents would run the assessments and plain-language requests, as in #4897. The bot itself would be deterministic: it holds the Slack connection, starts or continues sessions, and posts their output. This work is meant to be a real-world example of Managed Agents.

• Hosting: the bot would run in an OpenShell sandbox. Its Slack tokens would come from an OpenShell provider, and the supervisor would swap them in at egress, so the sandbox sees only a placeholder.

• Duplicate events: Slack redelivers an event the app hasn't acknowledged. The bot would acknowledge each event before doing slow work and drop duplicates.

• Formatting: converting the skills' Markdown output to Slack formatting would belong to the bot, not to the skills.

None of this exists yet.

Slack app

• Socket Mode: the bot would open an outbound WebSocket to Slack, so it needs no public Request URL. It needs an app-level token with connections:write in addition to the bot token.
• Bot scopes: request all of these up front. Any scope change after approval means a reinstall and a new compliance assessment.
  • chat:write lets the bot post, reply in threads, and edit its own messages.
  • app_mentions:read delivers app_mention events, which carry thread commands and plain-language requests.
  • reactions:read delivers reaction_added events for 👀 and ✅.
  • reactions:write lets the bot acknowledge a command with a reaction.
• Not requested:
  • channels:history, which would deliver every message in the channel. The @-mention requirement avoids needing it.
  • chat:write.public and channels:join. A maintainer would invite the bot to the channel instead.
• Token rotation off. Once turned on it can't be turned off, and the standard requires rotation after an exposure, not on a schedule.

Security and compliance

NVIDIA's Agentic Slackbot Security Standard applies to any Slack bot with AI anywhere in its workflow, including webhook posts. For this bot it would mean:

• nSpect: register the bot under OpenShell's existing nSpect program. The Slack install request asks for the nSpect ID.
• Tokens:
  • Tokens must live in an approved secret store, be fetched when needed, and be held only in memory.
  • They must never be on disk, in environment variables, or in any LLM context.
  • The provider placeholder keeps the real tokens out of the sandbox.
  • The backing store for the provider is not decided (Open question 3).
• Audit log: one record per action with the requesting user, bot ID, channel, request ID, tool, approval decision, data source, environment, and data classification. No secrets, prompts, or personal data.
• Human approval: the bot would write to GitHub (labels, comments) only on a command from an authorized requester. It would label AI-written content as AI-written.
• Channels:
  • The channel allowlist would live in reviewed configuration and not be changeable at runtime.
  • The bot's code would refuse to post to any channel outside it. Sandbox network rules match a request's method and path, so they can limit which Slack API methods the tokens reach but likely not which channel a message goes to.
• Sandbox network policy: Slack endpoints would set enforcement: enforce, because the default, audit, only logs violations. Socket Mode would use protocol: websocket.
• Content neutralization: the bot would keep the checks from post-assessment.sh in #4897: strip mentions and off-repo links, and check for credentials. Known gaps to close:
  • Bare domains without a scheme pass the link filter.
  • Unicode variation selectors aren't stripped.
  • The credential check misses the ghs_ and gho_ GitHub token prefixes.

Context

• Depends on the triage agent (#4897) and the triage-* skills from #4896.

• Approval flow:

  1. A maintainer creates the app from a manifest.
  2. The maintainer requests an install to the NVIDIA Internal workspace.
  3. Slack's admin bot sends an agentic compliance assessment, which asks for the nSpect ID and a self-certification.
  4. A compliant request is usually approved automatically.

  A draft manifest and pre-filled answers exist outside the repository. They predate this scope and request chat:write only.

• Where the sandbox runs is deferred until a working bot exists. The standard names approved environments, and a team-owned cloud project isn't one of them. Socket Mode keeps the choice open.

• Unfurling (UNVERIFIED): Slack's docs say link_shared isn't sent for links in messages posted by an app. If so, the GitHub Slack app won't unfurl issue links in these posts, and the post would carry its own preview. This needs one live test.

• Prerelease CI notices also use the channel webhook. They involve no AI and would stay on the webhook.

Definition of Done

• An approved Slack app in Socket Mode, with the scopes above, is installed in #openshell-triage.
• A GitHub Action posts one top-level message per state:triage-needed issue, with no raw issue content. It posts with the bot token once the bot exists.
• The bot keeps the four-emoji code in that message current and posts one thread reply per triage-* skill under it.
• Follow-up output for an issue lands in that issue's existing thread.
• Thread commands work by @-mention:
  • accept, valid, decline, and plain-language requests run.
  • decline confirms its wording before posting.
  • GitHub writes happen only for a requester whose GitHub identity the bot knows and who has permission for the change.
• 👀 and ✅ reactions claim and resolve issues.
• The bot ignores redelivered events.
• The app's scopes, owners and collaborators, token store, rotation, and audit log are documented.

Next steps for an implementer

• Get a ruling on Open question 1 before building.
• Answer Open question 2: can the bot learn a requester's GitHub identity?
• Test whether the GitHub Slack app unfurls an issue link in a bot or webhook post.
• Register the bot in nSpect (Open question 5).
• Update the draft manifest to the scopes above.
• A maintainer with an NVIDIA Slack login would then:
  • create the app;
  • add the owners below as collaborators;
  • request the workspace install;
  • complete the compliance assessment.
• Build the top-level-post Action on the webhook.
• Build the bot:
  • threading, commands, and reactions;
  • requester checks and event deduplication;
  • Managed Agents sessions;
  • audit log and neutralization;
  • sandbox policy and provider.
• Switch the Action to the bot token, then choose where the sandbox runs.

Owners

Jim Meyer and John Myers for now. Both would be app collaborators.

Open questions

1. Internal and external data.
   • The standard's "Rule of Three" lets an agent handle internal data or untrusted external data, not both.
   • This bot would carry public GitHub issue content into internal Slack, and maintainers' Slack messages would steer sessions that read GitHub.
   • Does a maintainer command count as the human review that lets data cross that line?
   • One possible design would keep triage sessions GitHub-only, and translate thread requests into the fixed command set in a separate step with no GitHub access.
   • This needs a ruling from security.
2. Requester identity. Does the GitHub Slack app expose a user's linked GitHub account to other apps? If not, the bot would need another source, such as a reviewed Slack-to-GitHub mapping or a link flow of its own.
3. Token store. Which secret store would back the provider, and does that setup meet the standard?
4. LLM endpoint. The standard's approved LLM endpoints don't include the direct Anthropic API that Managed Agents uses. This is also a prerequisite in #4897.
5. nSpect record. Should the bot be a version under OpenShell's program or a separate program?
6. Bot token in the Action. Posting the top-level message with the bot token would put that token in GitHub Actions secrets, outside the provider. Does that meet the standard, or should the Action ask the bot to post instead?
7. Plain-language requests. Which actions could the bot take from a plain-language request, and which would need the same confirmation as decline?
8. Audit log. Where does it go, and how long is it kept?