feat(agents): triage each new issue with an agent running in an OpenShell sandbox #4897

Scope update (2026-10-24)

• Assessment content changed. The agent would no longer combine triage-issue with assess-issue-routing. #4896 now splits triage into three separately runnable skills, built in draft PR #4899: triage-submitter-profile, triage-validate-issue (renamed from triage-issue), and triage-recommend-actions (replaces assess-issue-routing; the routing check was dropped). This issue would consume those skills once they land.
• Slack layout moved out. The eventual Slack shape (a top-level post with a four-emoji code, one thread reply per skill, thread commands such as /accept, /decline, and /valid, and Slack formatting) belongs to a later Slack bot project. Whether that project is #4898 or a new issue is not decided.
• Not decided: how much of the existing local implementation survives. It was built against the old design: two skills, one top-level webhook post. See "What exists" and Open questions 1 and 2.
• Unchanged: the trigger, sandboxed execution, no GitHub writes, the credential constraints, and the per-session cost cap.

User Story

I'm on OpenShell triage duty, and the queue grows faster than a batch review can keep up with: more than 170 issues are waiting in state:triage-needed. I need each new issue assessed when it is opened, with the result posted where maintainers already look, so that triage happens continuously instead of in occasional large passes. I'd also like this to run on OpenShell itself, as a working example of an agent operating inside an OpenShell sandbox.

Description

Build an automated triage agent that runs once for each newly opened issue in NVIDIA/OpenShell and posts its assessment to the #openshell-triage Slack channel.

The intended shape:

• Trigger: a GitHub Actions workflow on issues: opened would start a Claude Managed Agents session for the new issue.
• Execution: the session would use a self-hosted environment whose tools run inside an OpenShell sandbox, following the published Managed Agents and OpenShell self-hosted sandbox example. One sandbox per session, with a network policy that allows only the routes the agent needs.
• Assessment: the agent would apply the triage-* skills from #4899. triage-validate-issue writes to GitHub (a 📋 triage-agent comment and a state:* label), so the agent would use a read-only adaptation of it, as the local branch already does for triage-issue. Which skills run, and in what order, is open.
• Output: v1 would post through the existing channel webhook, which can post only top-level messages. It would make no GitHub writes: no comments, labels, or assignees. Whether v1 puts all skill output in one top-level message or waits for threaded posting (#4898) is open.

Security constraints for v1:

• The Managed Agents environment key would stay on the worker host and never enter a sandbox.
• GitHub and Slack credentials would be injected through OpenShell providers, or held on the host, so they are not readable inside the sandbox.
• The per-session Managed Agents token is currently piped into the sandbox, following the published example. Delivering it through an OpenShell provider instead is open (Open question 3).
• Hosted web_fetch and web_search tools would be disabled, since they run outside the sandbox policy.
• The issue title and body would be treated as untrusted input.
• Each session would have a cost cap.

What exists (local branch, not pushed)

Branch feat/4897-issue-triage-agent/purp holds 15 signed commits, local to the maintainer's machine. It is stacked on the first three commits of #4899. No live session, post, or pilot run has happened.

• Workflow .github/workflows/issue-triage-agent.yml: on issues: opened, or a manual workflow_dispatch with issue_number, it would create a version-pinned session with a budget. The session message names only the repository and issue number, never issue text. Every run is skipped unless the repository variable TRIAGE_AGENT_ENABLED is exactly true.
• Worker host scripts in scripts/agents/issue-triage/worker/:
  • start.sh runs ant beta:worker poll.
  • on-work.sh creates one sandbox per session on a dedicated gateway with the github-triage provider, runs the agent under a wall-clock cap, downloads assessment.json, and deletes the sandbox.
  • post-assessment.sh and render-post.jq validate and neutralize the file and post it through the webhook, or post a failure notice the host composes. Neutralization covers mentions, links outside NVIDIA/OpenShell, credential-like strings, and the exact session token. A ledger prevents duplicate posts.
• Sandbox:
  • Dockerfile: pinned ant and gh, the skills, and a read-only repo snapshot.
  • policy.yaml: allows only the Managed Agents session routes on api.anthropic.com, with boundary.yaml checked by openshell-prover.
  • providers/github-triage.yaml: GET on NVIDIA/OpenShell REST paths, /search/issues, and GraphQL queries, for /usr/bin/gh only.
• Agent: agents/issue-triage.md, with a Sonnet-class model and the hosted web tools disabled, plus skills/triage-issue-readonly/, pinned to .agents/skills/triage-issue/SKILL.md by SHA-256.
• Tests and docs: mise run test:issue-triage-agent runs the tests, which are wired into Branch Checks. Setup, secrets, the runbook, and unverified API shapes are in scripts/agents/issue-triage/README.md and CI.md.

Context

• Depends on the triage-* skills (#4896, draft PR #4899). triage-recommend-actions reads REVIEWERS.md from #4863 and falls back when it is missing.
• Threaded replies need a Slack app rather than a webhook (#4898).
• Blocking prerequisites, none settled yet:
  • Anthropic access (Console admin): an org and Workspace with the Managed Agents beta and self-hosted environments, NVIDIA approval to send public issue content to Anthropic, a Workspace-scoped API key and spend limit, a per-session cost cap, and a pilot exit-gate cost.
  • Repository secret and variables (someone with repo-secrets write): ANTHROPIC_API_KEY_TRIAGE, plus TRIAGE_AGENT_ID, TRIAGE_AGENT_VERSION, TRIAGE_ENVIRONMENT_ID, TRIAGE_MAX_COST_CENTS, and TRIAGE_AGENT_ENABLED.
  • GitHub identity for the bot (token owner): a maintainer's fine-grained PAT, a machine account, or a GitHub App. App installation tokens expire hourly and would need a refresher.
  • Worker host (operator): a pilot laptop, where posts arrive late after sleep, or a cloud VM. The host also needs a pinned, checksum-verified ant, ant apply, the environment key minted in the Console, Docker, and a dedicated OpenShell gateway (v0.1.5 or later).
  • Webhook (webhook owner): the existing URL or a dedicated webhook, stored in a mode-0600 file on the host.

Definition of Done

• Opening an issue in NVIDIA/OpenShell produces triage output in #openshell-triage, built from the triage-* skills of #4899, in the layout settled under Open question 1.
• The agent's tools execute inside an OpenShell sandbox with a restrictive network policy.
• No GitHub write occurs during a triage session.
• Setup, required secrets, and the cost cap are documented next to the workflow.

Next steps for an implementer

• Get the maintainer's answers to Open questions 1 and 2 before changing the local branch.
• After #4899 merges, rebase only the 15 agent commits onto main (git rebase --onto main 05a7a1a07). The three assess-issue-routing commits underneath them belong to that PR, and a squash merge would not drop them automatically.
• Re-grep scripts/agents/issue-triage/ for triage-issue and assess-issue-routing, then:
  • Rename skills/triage-issue-readonly/ to match triage-validate-issue, and update its metadata.source and SHA-256 digest. The digest test fails as soon as the source moves.
  • Update those paths in agent_definition_test.sh, policy_test.sh, worker/build-image.sh, Dockerfile, README.md, and agents/issue-triage.md.
  • Replace agents/issue-triage.md:44 (it reads assess-issue-routing) and its 9-line routing_block output contract with the triage-recommend-actions output: a four-emoji code and Markdown sections (see #4896). render-post.jq, its fixtures, and post_assessment_test.sh validate the old block and would need the same change.
• Check each skill's GitHub calls against providers/github-triage.yaml (Open question 5).
• Prototype session-token delivery through a provider (Open question 3).
• Hand each blocking prerequisite to someone with the listed role.
• Before the first live run, set TRIAGE_MAX_COST_CENTS=1 and confirm in the Console that the session stopped at about $0.01. The cap's unit is unverified, and a wrong unit fails open.
• Run mise run test:issue-triage-agent and the prover check in the README. Verify every commit's signature before any push. Push and open a PR only when the maintainer asks.

Open questions

1. What survives the redesign? Does v1 still ship one top-level webhook post, now carrying triage-* output, or wait for threaded posting through #4898? The #4896 design defers this.
2. Which skills, in which order?
   • Does the agent also run triage-submitter-profile?
   • triage-recommend-actions reads the validation result from the 📋 triage-agent comment and the state:* label. A read-only agent writes neither, so every new issue would report "not validated". How should the agent's own validation reach the recommendation step?
3. Session token through a provider. Can OpenShell credential injection rewrite the Authorization: Bearer header that ant beta:worker run sends? That header form is itself unverified. Would ant accept a placeholder token, and would the host create and delete a provider for each session? Today the exact-token check before posting is the only guard.
4. Malformed IDs. on-work.sh checks the environment, session, and work IDs against formats guessed from the docs. A bad environment or work ID posts a "sandbox" failure notice; a bad session ID only writes a host log line. The maintainer has deferred reviewing this behavior.
5. GitHub identity and reach.
   • triage-submitter-profile calls users/<login> and orgs/<org>/members/<login>, which the provider allowlist does not include.
   • triage-recommend-actions checks write access through repos/NVIDIA/OpenShell/collaborators/<login>/permission, which needs a token with push access. The current design assumes a read-only token.
   • Should the policy widen, the token change, or those signals be dropped in the agent?
6. Slack security standard. Does NVIDIA's Agentic Slackbot Security Standard apply to agent-generated posts sent through the already-approved webhook? Until someone answers, the webhook URL would live in a 0600 file on the worker host.
7. Model. Sonnet-class, as in the published example, or a larger model at higher cost?
8. Unverified API shapes. The README lists them: session-create request and budget unit, session-token header, session metadata readback, and the interrupt event. Each needs one test session before the workflow is enabled.