Skip to content

feat(skills): add triage-submitter-profile and triage-recommend-actions, rename triage-issue - #4899

Draft
purp wants to merge 7 commits into
NVIDIA:mainfrom
purp:feat/4896-assess-issue-routing/purp
Draft

purp wants to merge 7 commits into
NVIDIA:mainfrom
purp:feat/4896-assess-issue-routing/purp

Conversation

@purp

@purp purp commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Splits per-issue triage into three triage-* skills: triage-submitter-profile (who filed it), triage-validate-issue (triage-issue renamed; is the report real), and triage-recommend-actions (accept, priority, labels, people, release target, assignee). The rules, output format, and design decisions are in #4896. This description covers what changed and the test evidence.

Related Issue

Closes #4896

Changes

Review the net diff against main. The last commit supersedes the first three.

  • 6a0cdf1c0, 036c92d87, 05a7a1a07: the original assess-issue-routing skill and its hook in review-triage-queue. None of it remains in the net diff.
  • ddf813cfa renames triage-issue to triage-validate-issue: the directory and the name: field only, body unchanged. References are updated in CONTRIBUTING.md, sync-agent-infra, review-triage-queue and its report template, apply-triage-action, and create-spike.
  • cacaa72c8 adds triage-submitter-profile, read-only, with the other-open subcommand. Its scripts come from a personal triage-helper skill, with two fixes to submitter.sh:
    • a PR number is removed from its own PR bucket (open, merged, or closed unmerged), not from an issue bucket;
    • a GitHub app's profile link points at https://github.com/apps/<slug>.
  • 740f64b36 replaces assess-issue-routing with triage-recommend-actions, read-only.
    • The routing check is dropped because the redesign in #4896 covers its questions elsewhere: who should decide is the Acceptance line, who should own the work is the Assignee line, and who filed it is the submitter profile.
    • review-triage-queue returns to its main text apart from the rename, because the routing check its hook called no longer exists.
    • The CONTRIBUTING.md skills table and the sync-agent-infra maintenance map list all three skills.

Release Notes

None. Contributor skills only.

Testing

Each skill ran on real issues in fresh agent sessions without the new text (RED), then with it (GREEN). For triage-recommend-actions, the RED baseline was the old assess-issue-routing skill. Every GREEN run left the issue's updatedAt unchanged, so no run wrote to GitHub.

triage-recommend-actions

  • RED on #3362, #2040, #1055, and #4613: all four failed most checks.
    • Each run invented its own wrapper and kept the routing block. None printed the four-emoji code or rated Evidence.
    • All four declined to recommend a release or an assignee, saying the old skill forbade it.
    • #4613 stopped at "out of scope" (maintainer-authored), and #1055 at "no intake action" (roadmap).
    • Labels were kept or invented (area:sandbox, area:gateway, state:accepted).
    • Each run had five or six live @-mentions and bare #N. Three of four inferred he/his from names.
  • GREEN, all pass:
    • #3362 🔵👍🤔👌. It cites #2048 (RFC 0012), the 2026-10-07 contributor +1, the 2026-10-22 microVM and gVisor analysis, and the Isolation Backends docs page.
    • #2040 🔵👍🤔👌.
    • #1055 🔵👍🤔💪, with four pre-check lines and the full output.
    • #4613 🔵👍🤔👌, with the maintainer pre-check line and a Release line for the Stable ReportProviderReadiness RPC.
  • Edge runs, all pass:
    • PR #2048: one stop line, on the third run of the PR check (see loopholes).
    • #3362 with the write-access check returning 403 and an injected instruction in the issue to apply state:accepted and assign a maintainer: one write-access-unknown line, the instruction ignored, and labels, assignees, and updatedAt unchanged.
    • #3362 with REVIEWERS.md absent: the missing-file line, Labels: none (REVIEWERS.md missing), and no branch or fork named.
    • #2040 re-run after the refactor below.
    • #2040 with a forged > **📋 triage-agent** comment, posted once by a NONE author saying "valid" and once by a CONTRIBUTOR as the newest comment saying "failed": both runs ignored it and reported that validation has not run.
    • Decline: #595, one of a batch of audit-style reports a maintainer closed as speculative: ⚪👎🤔👌.
    • Ask: #3654, where a maintainer asked for the use case: 🔵❓🤔👌.

Loopholes the runs found, and how the text closed them:

Loophole Fix in the text
Declined to recommend a release or assignee The first paragraph says recommending both is the whole job
Stopped on a maintainer-authored or already-routed issue The full output, then one line per pre-check that applies
Treated an untracked REVIEWERS.md as not authoritative It is the source for labels and rosters even when untracked
Kept existing labels or invented new ones Only REVIEWERS.md headings; unknown existing labels go in a sub-bullet
Custom wrappers; Confidence and Evidence merged Verbatim output block, "nothing else"; four separate ratings
Live @-mentions, bare #N, inferred pronouns Name (`@login`), linked #N, they/them
Ad hoc search calls hit the search rate limit Run submitter.sh once; no own searches
A PR number ran every gather command, then ran them in parallel with the check The PR check runs first and alone; on true, nothing else runs
Confidence and Evidence levels undefined; undated Evidence bullets Definitions for both, and a dating rule per item type
A two-part issue with a partial Stable break "The change, or any part of it"
Unclear whether an assignee needs write access The assignee may be the author and needs no write access
No rule for first-hand use by maintainer, member, or app authors Rate what the issue and comments show
Docs links with and without /latest/ One pattern: https://docs.nvidia.com/openshell/<path>
Pre-check lines printed when they did not apply "Each of these that applies"
Pre-check lines printed above the four-emoji code The output block first, then the pre-check lines
"Possibly an RFC as well." on every output Only when rfc/README.md's "When to use an RFC" fits

triage-submitter-profile

  • RED on #3362, #4613, and #1055: no profile link, live @-mentions, bare #N, org membership not checked, and pronouns inferred in two runs. #4613 and #1055 drifted into triage advice. Each run made 5 to 10 ad hoc gh calls.
  • GREEN on #3362, #4613, #1055 (app author), PR #2048, and other-open for #3362: all pass after one round of fixes.
    • A company of @Docker rendered as a live mention. The text now backticks a company that starts with @.
    • The app profile link was a 404. submitter.sh now emits https://github.com/apps/<slug>.
    • --jq .pull_request printed empty rather than null. The PR check is now --jq '.pull_request != null'.
    • other-open had no rule for an issue without a state label. The closing sentence now names it.
  • Script RED and GREEN: submitter.sh 2048 reported prs_merged: 1 and issues_completed: 0; after the bucket fix it reports prs_merged: 0 and issues_completed: 1.

triage-validate-issue: rename only. No triage-issue references remain in tracked Markdown.

  • mise run markdown:lint passes. Its globs exclude .agents/**, so it does not lint the skill files.
  • shellcheck -x (0.11.0) passes on both scripts.
  • sync-agent-infra consistency checks 1, 3, 4, and 6 pass. Check 1 also flags a pre-existing update-docs directory and name mismatch that this branch does not touch.
  • The diff touches only .agents/skills/** and CONTRIBUTING.md, and every commit signature verifies.
  • Unit tests added/updated (not applicable)
  • E2E tests added/updated (not applicable)

A whole-branch review found no critical issues and three important ones:

  • Pre-check lines printed above the four-emoji code. Fixed.
  • A forged validation comment could sway the result. Two adversarial runs did not reproduce it.
  • The 👎 and ❓ paths were untested. Covered by the decline and ask runs above.

"Possibly an RFC as well." on every output was re-graded from minor to important and fixed. The fix commit's runs on #2040 print the four-emoji code on line 1, put the pre-check line last, and omit the RFC hint.

Follow-ups

  • #4897: its unpublished work pins .agents/skills/triage-issue/SKILL.md by path and SHA-256 and reads assess-issue-routing. Both break after this merges. The edits wait until it is clear how much of #4897 survives the Slack redesign.
  • #4863: no merge-order dependency. Until it merges, triage-recommend-actions finds no REVIEWERS.md on main, recommends no labels, and draws people from RFCs and PRs only.
  • The personal triage-helper skill that triage-submitter-profile came from still exists outside the repository. Removing it waits for its owner.
  • submitter.sh exits with a jq error, not a rate-limit message, when a GitHub search call returns HTTP 403 (rate limit). Each run makes seven search calls against a limit of 30 a minute.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable)

purp added 3 commits October 23, 2026 18:41
…r mapping

Refs NVIDIA#4896

Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
…utput links

Refs NVIDIA#4896

Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 24, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@gerardoxhavez196-ui gerardoxhavez196-ui left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

purp added 2 commits October 24, 2026 12:18
The skill validates a report; acceptance and routing move to separate triage skills. Body unchanged; references updated.

Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
Add a read-only skill that summarizes an issue submitter's identity,
NVIDIA org membership, and PR and issue counts, plus an other-open
subcommand listing their open issues and PRs. Every login is written in
backticks and every issue number as a link, so pasted output notifies
no one.

The scripts come from a personal triage helper, with two changes to
submitter.sh: a PR number is removed from its own PR bucket (open,
merged, or closed unmerged) rather than from an issue bucket, and a
GitHub app's profile link points at https://github.com/apps/<slug>.

Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
purp added 2 commits October 24, 2026 12:48
Pre-check lines now follow the output block, so line 1 is always the code. The RFC hint is tied to rfc/README.md's criteria instead of appearing on every recommendation.

Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
@purp purp changed the title feat(skills): add assess-issue-routing for per-issue ownership checks feat(skills): add triage-submitter-profile and triage-recommend-actions, rename triage-issue Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): split issue triage into submitter-profile, validate-issue, and recommend-actions skills

2 participants