ADR 0004 phase 2: define canonical schemas and provider-qualified identity #10292

Objective and readiness

Implement ADR 0004, Gate 2: canonical schemas, shared provider-qualified identity, and explicit compatibility/handoff contracts. Parent: #10290.

Phase 1 is complete: #10301 merged and closed #10291. Its merge commit is a4f8508dd7fbd2f691704c7d675bc71c8a6c0028. This phase is ready for agent assignment; read the current base and the merged baseline before implementation. Phase 1 changed tests/documentation, not runtime behavior.

User-directed ADR amendment: all new reflection data belongs under the existing /reflect endpoint. Do not introduce /reflect/v2, endpoint-version negotiation, or v2-to-v1 fallback. Update every affected ADR section consistently, preserving existing /reflect fields and meanings through additive changes. Catalogue revisions and versioned input/export schemas remain applicable.

Required Phase 1 handoff

Start with the ADR's new Gate 1 handoff and these merged fixtures:

• src/copilot-model.test.ts: host exact allowlist validation, separator/case normalization, retirement, and rejected prefixes/dates/queries/specialized variants.
• src/copilot-model-catalog-sync.test.ts: supported/priced/mapped/excluded sets, mapping-copy drift, stale exception entries, and named Add curated AI-credit prices for current Copilot models #10029/Complete routing effort metadata for current OpenAI and Anthropic models #10030 gaps.
• containers/api-proxy/model-metadata-characterization.test.js: differing proxy identity rules, exact dated overlays, pricing precedence/tier boundaries, and /reflect/persisted models.json contracts.

These tests describe the legacy baseline, not the target semantics. Do not rewrite them to make a new canonicalizer appear compatible, silently fix their characterized behavior, or make production consumers use the new API in this phase. Changes that intentionally affect legacy behavior belong to a later consumer migration with explicit comparison and approval.

Deliverables

1. Canonical schemas and validation

Implement versioned canonical model, overlay, imported-snapshot, central-feed, and resolved/export schema contracts with valid/invalid fixtures. Define reviewable provider/family source layout without migrating the maintained catalogue yet.

Cover provider-qualified keys, raw IDs, scoped aliases, exact/family specificity and exclusions, lifecycle/replacements, channel/deployment scope, harness version constraints, limits/modalities, reasoning, directed protocol features and translator revisions, tariffs, freshness, and per-field/per-component provenance. Preserve supported/unsupported/unknown/conditional states and unknown future provider reasoning values.

Tariffs must accommodate current input, output, cached-input, cache-write, and reasoning prices as well as the ADR's exact accounting units, long-context thresholds, effective dates, promotions/successors, and explicit unavailable-versus-zero distinctions. Preserve decimal precision in the new contract; do not change current accounting in this phase.

Add cross-record validation where JSON Schema alone cannot establish uniqueness, replacement resolution, alias ambiguity, or conflicting specificity.

2. Shared identity API and compatibility matrix

Implement one identity/matching API usable by host TypeScript and proxy CommonJS consumers, with JSON as the projection interchange format. Provider identity must survive normalization; raw provider IDs must remain available.

Document and test a legacy-to-canonical matrix for provider prefixes, case, separators, date suffixes, queries, exact matches, family matching, specialized variants, and provider aliases. Include host validation, proxy prefix stripping/direct resolution, runtime metadata, operator overlays, configured shared pricing catalogue, retirement lookup, and models.dev.

The merged baseline demonstrates differences that must not be erased by a universal string rewrite:

• Host validation rejects provider prefixes and dated IDs that proxy/runtime paths can recognize.
• Runtime/credit paths differ in query handling; retirement lookup lowercases but does not normalize separators.
• Operator overlays retain exact dated tariffs before family fallback; runtime pricing strips dates.
• models.dev lookup preserves case differences; equivalent slugs at different providers must not become canonical identity collisions.

Separate target identity rules from legacy adapter requirements. The new API and validators may be exercised directly by tests, but leave existing production callers unchanged.

3. Existing pricing-catalogue compatibility and authority

A configured shared pricing catalogue already exists. Inspect containers/api-proxy/model-pricing-catalog.js, its tests/callers, and the configuration path that supplies AWF_MODEL_PRICING_CATALOG; do not create a second unrelated catalogue contract.

The existing input uses schemaVersion: 1, unit: USD_PER_1M_TOKENS, catalogId, version, provider/model maps, input/output/cached-input/cache-write/reasoning rates, and optional long-context tiers.

Specify and test its mapping into the new canonical representation, preserving existing valid inputs, provider-alias matching, tier-boundary semantics, and provenance fields such as shared_catalog, catalogue ID, and version. Keep legacy parsing/acceptance behavior unchanged in production; strict rejection rules for newly introduced formats must not accidentally be imposed on this existing version-1 contract.

Phase 1 freezes this legacy precedence:

1. Operator provider/model overlay.
2. Configured shared pricing catalogue.
3. Complete runtime provider pricing.
4. Incomplete runtime pricing filled from lower-priority data.
5. Curated prices.
6. Bundled models.dev.
7. Dynamic-selector conservative fallback.
8. Configured default.
9. Built-in fallback for the unknown sentinel.

Distinguish configured pricing authority from downloaded central updates. The ADR puts fresh runtime observations above central feed data, but the existing configured shared catalogue outranks runtime pricing. Define how configured catalogue data maps to scoped configured/operator authority without lowering its current priority. A fetched central snapshot is not the same authority merely because it shares a schema. Record this distinction in the ADR and schema/compatibility documentation so Phase 5 cannot silently reorder precedence. Add contract fixtures for both classifications; do not implement the new runtime resolver here.

4. Workflow/config handoff and documentation

Define concrete versioned workflow-frontmatter-to-AWF-config payloads with github/gh-aw, including schema negotiation, workflow/operator conflict handling, sanitized provenance, older-pinned-AWF rejection, harness comparison semantics, and central source selection/offline settings.

Preserve the existing open, unversioned apiProxy.providers contract and the configured pricing-catalogue contract. Strict unknown-field rejection applies to newly introduced explicitly versioned formats, not retroactively to legacy inputs.

Keep any new format opt-in and inert for existing workflows. This phase defines and tests contracts; Phase 4 owns compiler-to-runtime wiring. Do not modify another repository without authorization. Record the agreed handoff and companion issue/PR links. If cross-repository agreement is unavailable, clearly identify the unresolved contract/blocker rather than claiming it is agreed or inventing a final independently incompatible format.

Use the repository's canonical config-schema generation mechanism if config schema changes are needed; do not hand-edit generated schema copies.

Acceptance criteria

• New schema/validator fixtures cover every record group, unknown/conditional capability states, raw unknown efforts, exact decimal/unit semantics, reasoning prices, promotion expiry, freshness, harness constraints, and directed translator revisions.
• Shared identity fixtures preserve providers/raw IDs and reject duplicate exact keys, ambiguous scoped aliases, unresolved replacements, conflicting equal-specificity rules, and unintended family capture of declared exceptions.
• A checked-in compatibility matrix ties legacy behavior to the Phase 1 fixtures and documents intended target matching separately; specialized variants do not inherit unverified family facts.
• Existing AWF_MODEL_PRICING_CATALOG version-1 fixtures map to the new contract without changing current parsing, priority, tier boundaries, units, or source/ID/version semantics.
• Configured-catalogue authority and fetched-feed authority are explicitly distinguished in the ADR/contracts, with regression fixtures protecting configured catalogue precedence over runtime pricing.
• New strict formats reject invalid/unknown fields without imposing stricter behavior on legacy provider overlays or legacy configured catalogues.
• Workflow/config schema, conflict policy, unsupported-version behavior, source selection, and harness comparison are documented with agreement/companion links; any unresolved cross-repository dependency is reported explicitly before completion.
• The ADR consistently uses additive /reflect data, not a separate endpoint, across architecture, export, validation, and rollout sections.
• All Phase 1 fixtures remain intact and passing; production identity, validation, routing, pricing/accounting, reflection, and persistence behavior are unchanged.
• Provide an implementation handoff linking schemas, identity API, compatibility matrix, validation commands/results, and downstream integration points for Phases 3-7.

Validation

Run new targeted schema/identity tests in every relevant execution environment, plus the merged baseline:

# Repository root
npm run build && npx jest --runInBand src/copilot-model.test.ts src/copilot-model-catalog-sync.test.ts
npm run type-check

# Run separately from containers/api-proxy
npm test -- --runInBand model-metadata-characterization.test.js

Run existing targeted overlay/shared-catalogue tests when touching their contracts or adding compatibility adapters. The root Jest configuration does not run proxy tests. Validate generated config-schema synchronization when applicable. No live provider access, production inference, or workflow dispatch is needed.

Boundaries and next-phase handoff

• No production consumer migration, new live pricing loader, central publication, maintained-record migration, or broad cleanup.
• Add curated AI-credit prices for current Copilot models #10029/Complete routing effort metadata for current OpenAI and Anthropic models #10030 remain unresolved model-data coverage requirements for Phase 3. Their characterized gaps and stale exception entries must not become target schema rules or be fixed incidentally here.
• Do not generate a new independent pricing/mapping source, replace the updater, or remove legacy readers.
• Phase 3 consumes these schemas/identity contracts for deterministic compilation; Phases 4-6 integrate production consumers; Phase 7 exposes resolved data additively through /reflect.