ci: detect new framework releases and test each one in isolation against the PR lock #1066

Part of #1065.

Summary

Add a daily framework-release-watch.yml workflow. It finds every framework package whose newest stable PyPI release is newer than the PR lock. For each one, it runs the existing framework checks with only that package moved and everything else held at the lock, then opens or updates one framework-release issue per (package, version) with the results and the exact next step. The issue closes itself when the lock catches up.

Motivation

See #1065. The nightly fresh resolve catches breakage but can't name which release caused it, and nothing prompts the lock refresh ADR 0007 rule 3 requires. BG §1.8.

Current behavior (verified)

• Roster. The watch list comes from ADAPTERS (integrations/init.py:197) and each extra's direct requirements (pyproject.toml:57-84). For example, langgraph also pulls in langchain-openai and langchain-core, and adk also pulls in litellm.

• Lock. Exact pins per extra live in python/constraints/adapter-contract-<extra>-py3.12.txt, generated by compile_constraints.py (_COMBOS, compile_constraints.py:91).

• Reusable checks:

  • verify_frameworks.py --only <fw> --require-installed --json (verify_frameworks.py:70, :372-374)
  • the whole suite with DONKEY_CONTRACT_EXTRA arming the skip guard (conftest.py:47)
  • scripts/smoke_example.py <fw>

  framework-legs already chains all three (nightly-matrix.yml:175-259), but on a fresh resolve.

• Reusable PyPI client. fetch_project() and canonicalize() are stdlib-only (check_new_dependencies.py:58, :116).

• Issue pattern. The open-or-update issue logic is in alert (nightly-matrix.yml:261-299).

• Existing workflow guards:

  • every job needs a bounded timeout and every matrix fail-fast: false (test_workflow_bounds.py:45, :56)
  • an aggregate job needs every other job, and only alert writes issues (test_ci_workflows.py:38, :55). The new workflow must be registered in these parametrizations.

Proposal

1. python/scripts/check_framework_releases.py (stdlib only)

• Import, don't copy. Take fetch_project and canonicalize from check_new_dependencies.py.
• Watch set. For each ADAPTERS[k].extra, watch the direct requirements of that extra.
  • Locked version: the == pin in adapter-contract-<extra>-py3.12.txt.
  • Latest version: the newest non-yanked, non-pre-release version on PyPI.
• --json: emits a GitHub matrix of {framework, extra, package, locked, latest, bump: major|minor|patch, released_at, release_notes_url}, where release_notes_url comes from info.project_urls (Changelog / Release notes / Source). Prints [] when nothing is new.
• --constraints-for PKG==VER --out PATH: writes a copy of that extra's lock with that one pin replaced.
• --package NAME: limits the run to one package (for workflow_dispatch).
• --stale-issues: reads the open framework-release issues (gh JSON on stdin) and prints the numbers of those whose version is now <= the lock.

2. .github/workflows/framework-release-watch.yml

Triggers:

• schedule: "30 5 * * *", before the 06:00 nightly
• workflow_dispatch with an optional package input
• pull_request filtered to the workflow and script paths, as a dry run: no issue writes, and the rendered issue bodies go to the job summary

Jobs:

1. detect runs the script and outputs the matrix. Later jobs are skipped when it is empty.

2. evaluate runs once per release, with fail-fast: false, timeout-minutes: 30, Python 3.12, and matrix values bound through env::

   • Install with everything else locked: pip install -e ".[llm,local,otel,$EXTRA]" --group dev -c <rewritten lock>.
   • If that doesn't resolve, the new release needs other packages moved too. Retry unconstrained and record the outcome as needs co-bump.
   • Then run verify_frameworks.py --only $FW --require-installed --json, pytest -q with DONKEY_CONTRACT_EXTRA=$EXTRA, and smoke_example.py $FW.
   • Collect outcomes. Each step uses continue-on-error and records its outcome, together with the installed version from importlib.metadata, into a result.json artifact.

3. alert runs with if: always(), needs: every job, issues: write. It is the only job that writes. For each release, it opens or updates one issue titled framework-release: <pkg> <locked> → <latest>, labelled framework-release and dependencies (creating the label with gh label create --force). The body contains:

   • versions, bump kind, release date and release-notes link
   • a results table (signature / suite / smoke × isolated / co-bump) and the run link
   • a placeholder for the impact summary, filled by ci: add an AI changelog impact review to the framework release watch #1067
   • next action:
     • All green: "safe to bump". From python/, run python scripts/compile_constraints.py, then python scripts/verify_frameworks.py --only <fw> --emit-verified to refresh the section 8 row in docs/verified-apis.md, then the PR gate. Reference this issue in the bump PR.
     • Anything red: "adapt the adapter or record a §8.1 incompatibility, never add a ceiling (§8.4)".

   alert also closes the issues returned by --stale-issues, with a comment linking the lock commit. A re-run with unchanged results doesn't add a duplicate comment.

3. Docs

• docs/ci.md: add a section for the new workflow.
• CONTRIBUTING.md: in the lock-refresh step, name framework-release issues as the trigger.
• No ADR change: rule 3 still holds, because the lock bump stays a maintainer PR.

Cross-surface impact

• core/llm/registry/tools: not affected. integrations: read-only. ADAPTERS is the roster source, and adapter code doesn't change.
• TypeScript parity (planned): not affected. When the TS port ships, its framework packages (npm) need the same watch. Note that in the Phase 5 TS epic.
• CLI: not affected.
• Nextra docs site: not affected (that's docs: publish a generated framework version-compatibility page #1068).
• docs/verified-apis.md: no row changes in this issue. The issue body tells the maintainer how to refresh the section 8 rows with --emit-verified.

Out of scope

• The AI changelog review. That's ci: add an AI changelog impact review to the framework release watch #1067; this issue leaves a placeholder in the issue body for it.
• An automatic lock-bump PR. That's ci(deps): pin the dev and release toolchain with a lock, and make mypy independent of installed extras #763 and ci(security): harden the Actions supply chain — SHA pins, Dependabot, CODEOWNERS, least-privilege Pages #757.
• Pre-release watching (aN/bN/rcN). Possible later behind a flag; not needed now.

Acceptance criteria

• tests/unit/test_check_framework_releases.py, using a fake Fetch, covers: pre-releases and yanked versions are skipped; major/minor/patch classification; lock parsing; the single-pin constraint rewrite; --stale-issues.
• A roster-drift test asserts the watched set equals ADAPTERS × the extras' direct requirements, in the style of test_nightly_framework_legs_match_adapters (test_adapter_roster.py:152).
• The new workflow passes test_workflow_bounds.py and test_ci_workflows.py, and is registered in both.
• The PR's dry run lists today's real gaps (as of 2026-10-10: langchain-openai, langchain-core, litellm, strands-agents, agent-framework, anthropic, crewai) and renders their issue bodies in the job summary without creating issues.
• After merge, workflow_dispatch with package=crewai creates exactly one issue. A second run updates it instead of duplicating it. Bumping the lock, then running the watch again, closes it.
• pytest -q && mypy && ruff check . && lint-imports are green.