Repository navigation
ADR 0004 phase 8: unify catalogue maintenance, drift review, and feed publication #10298
Description
Objective and dependencies
Implement ADR 0004 Gate 8. Parent: #10290.
Scheduled source maintenance is required, not optional follow-up. The model-management system must not rely on a maintainer remembering to discover model changes, edit sources, or regenerate outputs.
Phase 3 (#10293) merged in #10317, providing the canonical sources/import/compiler foundation. Source refresh and update-PR automation can proceed against it. Central feed publication remains separately gated on Phase 5 loader/source-authentication integration; publication is not a prerequisite for scheduled checked-in source upkeep.
Delivered maintenance slice (2026-10-22)
- feat(ci): automate scheduled canonical source maintenance #10439 replaced the old manual/read-only endpoint updater with daily scheduled and main-only manual source maintenance, 11 approved evidence captures, scoped proposals, governed import refresh, deterministic regeneration, drift/impact reports, deduplicated review PRs, and independent evidence/bundle validation before privileged writes.
- fix(ci): unblock unattended model source maintenance #10466 fixed unattended Copilot command-permission matching using a narrowly permitted executable with seven fixed operations; no blanket node/npm permission was added.
- Run 39849323674 passed every job and created chore: refresh canonical model sources #10469, merged on 2026-10-22. All 11 sources were captured successfully; this proposal changed the governed OpenRouter import and all affected projections, not maintained native-provider/Copilot records. The committed report records 2,957 catalogue field changes and 663 runtime lookup changes.
- Offline fixtures cover update/no-op/deduplication, failed/partial retrieval, evidence receipts, source disappearance without inferred retirement, high-impact review gates and actual bundle ownership. No-op behavior and updates to an existing PR have fixture coverage; the successful run above exercised creation of a new PR, not every branch.
This phase remains open for central snapshot publication and Phase 5 loader/source-authentication integration. Do not rebuild the delivered updater. Successful documentation capture is not semantic verification of all maintained facts; human evidence/scope review remains mandatory.
Scope
Replace endpoint-only and ad hoc maintenance with one source-aware scheduled updater using existing GitHub Actions/gh-aw mechanisms:
- Run daily, with
workflow_dispatchfor on-demand reruns. - Fetch approved provider model APIs/documentation and governed upstream snapshots, retaining source/revision/capture/hash/license evidence as applicable. Validate retrieval failures explicitly; a missing page must not be interpreted as model retirement.
- Compare captured evidence with canonical source records and propose supported source changes. Preserve provider/channel/deployment boundaries, authority, confidence, and unknown states; no family-name or marketing-based compatibility inference.
- Run canonical validation/import checks,
npm run models:generate, and focused model regression checks, thennpm run models:check. - Create or update one deduplicated review PR containing source changes, governed import metadata, every affected generated projection, and machine/reviewer-readable impact reports. Never author generated outputs independently.
- No material change means a no-op, not timestamp-only churn. Retrieval/validation failures surface visibly with affected source names and last successful evidence.
Automate discovery, proposal, validation and regeneration; retain human review before merging model-fact changes. Do not auto-merge or automatically activate staged pricing, lifecycle, compatibility, or model-selection corrections. The bundled catalogue and generated retirement projection already have legacy runtime consumers: merging a reviewed refresh can change fallback accounting, lookup winners or retirement rejection, and reports/consumer regressions must expose those effects.
Scheduled drift comparison covers missing models, prices beyond documented provider-specific rounding tolerance, changed effort sets/limits, lifecycle, compatibility, harness constraints, and source health.
Publish reviewed versioned central snapshots through the same canonical pipeline only after Phase 5 integration is ready. Static package outputs remain reproducible and downloaded snapshots never rewrite package assets. A content digest is not publisher authentication.
Entry points: .github/workflows/model-api-mapping-updater.md and lock, canonical compiler/importer from Phase 3, CI/staleness checks, and existing release/publication machinery. Follow workflow-source compilation/post-processing requirements. Use existing execution/sandbox controls; no new kernel/controller/runtime-hardening architecture.
Acceptance criteria
- A checked-in recurring schedule triggers approved source retrieval and update proposal without a maintainer editing sources or remembering regeneration; manual reruns remain available. feat(ci): automate scheduled canonical source maintenance #10439/fix(ci): unblock unattended model source maintenance #10466; successful end-to-end run linked above.
- One deduplicated update PR contains canonical changes, governed imports/metadata, every affected projection, and a machine-readable impact report; generated outputs are never independent authoring targets. feat(ci): automate scheduled canonical source maintenance #10439; first merged output chore: refresh canonical model sources #10469.
- Reports cover adds/removals, lifecycle/channel/replacements, prices/thresholds/effective dates/promotion expiry, effort/limit/feature/harness changes, confidence, stale/missing sources, collisions/rejections. Generic exact-field diffs and review classifications in
scripts/model-information/maintain.cjs; chore: refresh canonical model sources #10469 includes actual runtime lookup effects. - Tests enforce explicit review for price decreases, retirement, compatibility contraction, confidence downgrade, and source disappearance.
scripts/model-information/maintain.test.ts. - Recurring drift fixtures detect missing models, tolerated versus material price changes, and effort/context drift without live external dependencies. Decimal-equivalent formatting is a no-op; tolerance is zero unless separately reviewed provider-specific tolerance is documented, not a guessed allowance.
- Unchanged upstream facts produce a no-op without timestamp-only catalogue changes or duplicate PRs. Offline no-op and existing-PR fixtures in feat(ci): automate scheduled canonical source maintenance #10439.
- Failed retrieval, invalid data, and partial source coverage are visible; they do not silently erase existing facts or count as a successful refresh. Named capture failures and independent validation in feat(ci): automate scheduled canonical source maintenance #10439.
- Runtime observations only propose maintained changes; they cannot silently promote uncertain facts or make advertised discounts billable. Marketing prose does not establish verified capability. Evidence-bound data proposals require human review; no new runtime-observation promotion mechanism was enabled.
- Governed imports capture source/revision/time/hash/importer/license/counts/collisions; malformed/ambiguous updates fail visibly. feat(ci): automate scheduled canonical source maintenance #10439/chore: refresh canonical model sources #10469; undeclared OpenRouter licenses remain explicitly unknown.
- Published snapshots carry schema/revision/publication/effective/provenance/integrity metadata and pass Phase 5 loader/source-authentication integration tests before publication is enabled.
- Obsolete endpoint-only authoring is replaced; source ownership, scheduled upkeep and review boundaries are documented in the ADR. feat(ci): automate scheduled canonical source maintenance #10439.
- Document the central publication/release procedure and verify published-snapshot consumers after Phase 5 integration; maintenance documentation does not establish publication readiness.
Non-goals
Do not auto-accept model-fact changes, bypass network controls, publish unreviewed provider observations, or trigger live workflows merely to validate this implementation. No new execution-security architecture or /reflect/v2. Agent workflows consume packaged model data; this scheduled maintenance process does not turn the offline compiler into a runtime network fetcher.
Activity
chore: refresh canonical model sources
chore: refresh canonical model sources (#10469)