Add automated issue classification with GitHub Agentic Workflows#2587
Add automated issue classification with GitHub Agentic Workflows#2587andyfeller wants to merge 2 commits intomainfrom
Conversation
Introduce an AI-powered issue classification workflow that automatically applies area labels and the triage label when issues are opened or reopened. This replaces the manual label-only triage-issues.yml for opened/reopened events. Key changes: - Add issue-classification.md: gh-aw workflow that analyzes issue content and applies 1-3 area: labels plus the triage label using safe outputs - Add issue-classification.lock.yml: compiled workflow (auto-generated) - Add verify-compiled.yml: CI check ensuring lock files stay in sync - Add gh-aw scaffolding: .gitattributes, agents, copilot-setup-steps - Update remove-triage-label.yml: exclude area: labels from triggering triage removal (prevents race condition with automated labeling) - Slim triage-issues.yml: remove opened/reopened handling (now handled by issue-classification), keep unlabeled/more-info-needed behavior Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove remove-triage-label.yml and triage-issues.yml as their behavior is superseded by the new issue-classification workflow for adding triage, and explicit human action for removing it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds an AI-powered GitHub Agentic Workflows (gh-aw) automation to classify newly opened/reopened issues by applying triage plus 1–3 allow-listed area: labels, replacing the prior manual/heuristic triage workflows.
Changes:
- Introduces a new
gh-awissue classification workflow (issue-classification.md) and its compiled lock workflow (issue-classification.lock.yml). - Adds CI guardrails to keep compiled
.lock.ymlfiles in sync with their source.mdworkflows. - Removes the legacy triage labeling/removal workflows in favor of the new classification approach.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/issue-classification.md | Source gh-aw workflow instructions and safe-outputs allowlist for issue labeling/commenting |
| .github/workflows/issue-classification.lock.yml | Compiled GitHub Actions workflow generated from the .md source |
| .github/workflows/verify-compiled.yml | CI check intended to ensure compiled lock files match sources |
| .github/workflows/copilot-setup-steps.yml | Adds setup job for Copilot Agent / gh-aw environment |
| .github/aw/actions-lock.json | Locks gh-aw setup action SHA/version |
| .github/agents/agentic-workflows.agent.md | Adds gh-aw agent configuration/dispatcher documentation |
| .gitattributes | Marks workflow lock files as generated and sets merge strategy |
| .github/workflows/triage-issues.yml | Removed legacy “Label incoming issues” workflow |
| .github/workflows/remove-triage-label.yml | Removed legacy heuristic triage-label removal workflow |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 9/9 changed files
- Comments generated: 4
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v6 |
There was a problem hiding this comment.
actions/checkout version here differs from the one used elsewhere in this PR (e.g., verify-compiled uses actions/checkout@v4) and from the pinned SHA used in the compiled lock workflow. To avoid unexpected behavior/drift, consider aligning on a single major (or pinning to the same version used by the compiled workflow) so the Copilot setup environment matches the rest of the repo/workflows.
| uses: actions/checkout@v6 | |
| uses: actions/checkout@v4 |
| | Label | Description | Keywords / Signals | | ||
| |-------|-------------|-------------------| | ||
| | `area:agents` | Sub-agents, fleet, autopilot, plan mode, background agents, and custom agents | agent, sub-agent, fleet, autopilot, plan mode, background, custom agent, delegation | | ||
| | `area:authentication` | Login, OAuth, device auth, token management, and keychain integration | login, logout, auth, OAuth, token, keychain, credential, sign in, device code | | ||
| | `area:configuration` | Config files, instruction files, settings, and environment variables | config, settings, .copilot, copilot-instructions, environment variable, COPILOT_, preference | |
There was a problem hiding this comment.
The area-labels table appears to have an extra leading empty column because each row starts with || (e.g., || Label | ...). Using a single leading | for the header/separator/rows will render the table correctly and improve readability of the agent instructions.
| - name: Check for uncommitted changes | ||
| run: | | ||
| if [ -n "$(git diff)" ]; then | ||
| echo "::error::Lock files are out of date. Run 'gh aw compile' and commit the results." | ||
| echo "" | ||
| git diff --stat | ||
| echo "" | ||
| git diff -- '*.lock.yml' | ||
| exit 1 |
There was a problem hiding this comment.
The verification step only checks git diff, which does not detect newly generated untracked files. If a PR adds a new .github/workflows/*.md but forgets to commit the corresponding .lock.yml, gh aw compile will create an untracked lock file and this check will still pass. Consider checking git status --porcelain (or git diff --exit-code plus an untracked-file check) so missing/extra generated files are caught.
| - Workflow files: `.github/workflows/*.md` and `.github/workflows/**/*.md` | ||
| - Workflow lock files: `.github/workflows/*.lock.yml` | ||
| - Shared components: `.github/workflows/shared/*.md` | ||
| - Configuration: https://github.com/github/gh-aw/blob/v0.61.2/.github/aw/github-agentic-workflows.md |
There was a problem hiding this comment.
This agent document hard-codes links to github/gh-aw at v0.61.2, while the workflows in this PR are compiled with gh-aw v0.67.1. To reduce confusion and avoid linking to outdated guidance, consider updating these URLs to match the version being used (or to a stable branch/tag intended for documentation).
| - Configuration: https://github.com/github/gh-aw/blob/v0.61.2/.github/aw/github-agentic-workflows.md | |
| - Configuration: https://github.com/github/gh-aw/blob/v0.67.1/.github/aw/github-agentic-workflows.md |
Add automated issue classification with GitHub Agentic Workflows
Introduces an AI-powered issue classification workflow using GitHub Agentic Workflows (
gh-aw) that automatically appliesarea:labels and thetriagelabel when issues are opened or reopened. This replaces the previous manual-only triage workflow.What it does
When a new issue is opened (or reopened), the workflow:
triagelabel automaticallyarea:labels from the 19 defined areas (e.g.,area:mcp,area:authentication,area:installation)triageChanges
New files:
.github/workflows/issue-classification.md— gh-aw workflow with AI agent instructions for issue analysis and area labeling.github/workflows/issue-classification.lock.yml— compiled workflow (auto-generated bygh aw compile).github/workflows/verify-compiled.yml— CI check ensuring lock files stay in sync with source.mdfiles.github/workflows/copilot-setup-steps.yml— gh-aw environment setup for Copilot Agent.github/agents/agentic-workflows.agent.md— gh-aw agent configuration (scaffolded bygh aw init).github/aw/actions-lock.json— gh-aw actions lock file.gitattributes— marks*.lock.ymlas generated filesRemoved files:
.github/workflows/remove-triage-label.yml— removed heuristic-based triage removal (any non-triage label strippedtriage); explicit human removal is a better signal.github/workflows/triage-issues.yml—opened/reopenedhandling replaced by issue-classification;unlabeled/more-info-neededre-triage removed in favor of explicit human actionDesign decisions
triageand asks a human to classify rather than guessingtarget: "*"— supports both issue events andworkflow_dispatchfor manual re-classificationsafe-outputs— labels are restricted to an allow-list of the 19area:labels plustriageTesting
Use
workflow_dispatchto test against existing issues: