Repository navigation
ci: add an AI changelog impact review to the framework release watch - #1112
Merged
Merged
Conversation
…(§0.3, #1067) A new `ai-review` job in framework-release-watch.yml runs one leg per release, beside `evaluate`. It asks claude-code-action (pinned by SHA, claude-sonnet-5-5) whether the upstream notes touch what the adapter uses, and `alert` embeds the answer in the release's issue under "Impact review (AI, advisory)": Risk, the affected symbols with citations, a recommended action and the confidence. - scripts/review_framework_release.py builds the prompt (package, versions, release notes, compare link, adapter module, FRAMEWORKS rows, the §8 row and all of §8.1/§8.2) and validates the structured output. The renderer treats the answer as untrusted: schema fields only, Markdown escaped, no HTML or comment markers, no mentions, http(s) citations only, and no credential-shaped text. - Read-only: `contents: read`, Read/Grep/Glob plus WebFetch on github.com, raw.githubusercontent.com and pypi.org; no Bash or edit tools, /proc denied, in-repo Claude settings ignored. Only `alert` writes. - The key is an `ai-review` environment secret (develop only). The job is skipped on pull requests and never fails the run: no key, a failed action or an invalid answer leaves "impact review unavailable". - One review per (extra, package, version) via actions/cache. - A `replay` dispatch input (LOCKED..LATEST) reruns one past release as a dry run, to check the review against a known break. - Advisory only: never verification (§0.3), never a §8 row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1067
Summary
ai-reviewjob inframework-release-watch.yml: one leg per release, in parallel withevaluate,fail-fast: false, 15 min.anthropics/claude-code-actionpinned by SHA (v1.0.248), modelclaude-sonnet-5-5.alertembeds the result in the issue under "Impact review (AI, advisory)": Risk (none/low/high), affected symbols with upstream citations, recommended action (bump / adapt adapter / record §8.1 / watch) and confidence.scripts/review_framework_release.pybuilds the prompt (package, versions, release notes, GitHub compare link, adapter module,FRAMEWORKSrows, §8 row plus all of §8.1/§8.2) and validates the structured output (--json-schema). The answer is untrusted: only the schema's fields are kept, Markdown is escaped, HTML and comment markers are stripped, mentions and#refs are neutralised, only http(s) citations are kept, and an answer with credential-shaped text is dropped.contents: read, no write token; Read/Grep/Glob plus WebFetch limited to github.com, raw.githubusercontent.com and pypi.org; Bash/edit/subagent tools disallowed, reads under/procdenied,--setting-sources user. Onlyalertwrites.ai-reviewenvironment secret (develop only, no repo-level secret). The job is skipped onpull_requestand on non-develop refs. No key, a failed action or an invalid answer gives a notice plus "Impact review unavailable", and the watch stays green.actions/cache(artifacts don't persist across runs).replaydispatch input (LOCKED..LATESTwithpackage) re-runs one past release as a dry run, so the review can be checked against a known break.Test plan
nox -s typecheck-and-lint test-3.11 new-dependencies commit-identities workflow-lintgreen; actionlint and zizmor cleantests/unit/test_review_framework_release.py: prompt contents, schema, GitHub-output delimiter, validation, rendering (block markers, HTML, mentions, Markdown, table-cell breakout, credential-shaped text), record and loadtest_check_framework_releases.py:compare_url, replay, impact precedence (this run → earlier review → unavailable),--impactstest_ci_workflows.py:ai-reviewis the only job onenvironment: ai-reviewin any workflow, never runs onpull_request,contents: read, read/fetch-only tools, key named only in that job; replay is a dry runclaude -pwith the job's exact flags: anthropic 0.116.0..1.0.0 gave Risk: high (record §8.1); crewai 1.15.23..1.15.27 gave Risk: low (bump)Post-deploy steps
After merge, on
develop:gh workflow run framework-release-watch.yml --ref develop -f package=anthropic -f replay=0.116.0..1.0.0→ summary shows Risk: high.gh workflow run framework-release-watch.yml --ref develop -f package=crewai→ none/low.ANTHROPIC_API_KEYfrom theai-reviewenvironment and dispatch once → run green, issue says "Impact review unavailable"; then restore the key.🤖 Generated with Claude Code