From 9ed978977bddb6d04e2df3cc2471a98938ba0449 Mon Sep 17 00:00:00 2001 From: Aashil Date: Tue, 29 Sep 2026 16:35:32 +0545 Subject: [PATCH] feat: add manual-qa-required/qa-verified labels to AI-filed issues Team asked for a way to flag which AI-filed issues need a manual look before acting, separate from the existing bug/feature type labels. Agent now decides manual-qa-required (confidence <70/100 or not directly reproduced) vs qa-verified per item, matching the confidence score and reproduction method it already writes in the issue body. Creates both labels on first use so a newly onboarded repo doesn't need them pre-created. Co-Authored-By: Claude Sonnet 5 --- prompts/crisp-triage-agent.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/prompts/crisp-triage-agent.md b/prompts/crisp-triage-agent.md index f5e23e4..81eea54 100644 --- a/prompts/crisp-triage-agent.md +++ b/prompts/crisp-triage-agent.md @@ -26,11 +26,16 @@ You are triaging one customer support conversation for the `{{REPO}}` repository **Feature requests need a stricter match than bugs.** Two customers can want different things from a similar-sounding or vaguely-titled request -- only treat it as the same request if this transcript asks for the exact same capability, not just the same general area of the product. A question about whether something already exists, or how to configure it, is a how-to question, not a match for an open feature request. -3. **For each item with a genuine match nothing tracks yet**, file one issue per item: +3. **For each item with a genuine match nothing tracks yet**, decide its QA label first: `manual-qa-required` if your confidence is below 70/100 *or* you did not directly reproduce it (transcript + code inspection alone, no actual run/test) -- otherwise `qa-verified`. Match this to the same confidence score and reproduction method you write in the issue body below; they must agree, never one saying "verified" and the other implying doubt. A repo may not have these labels yet, so create them first, ignoring the error if they already exist: ``` - gh issue create --repo {{REPO}} --title "..." --label bug-report,bug-report-triage --body-file + gh label create manual-qa-required --color fbca04 --description "AI-filed issue below the confidence/reproduction bar -- verify before acting" 2>/dev/null || true + gh label create qa-verified --color 0e8a16 --description "AI-filed issue reproduced directly with high confidence" 2>/dev/null || true ``` - (use label `feature-request,bug-report-triage` for a feature). Write the body file as actual markdown -- each section below is a real `##` heading in the file you write, not just plain text with the section name at the top of a paragraph: + Then file one issue per item: + ``` + gh issue create --repo {{REPO}} --title "..." --label bug-report,bug-report-triage, --body-file + ``` + (use label `feature-request,bug-report-triage,` for a feature). Write the body file as actual markdown -- each section below is a real `##` heading in the file you write, not just plain text with the section name at the top of a paragraph: ```markdown ## Summary