diff --git a/devlog/_plan/260914_r2l8_catalog_autorefresh/010_roadmap.md b/devlog/_plan/260914_r2l8_catalog_autorefresh/010_roadmap.md new file mode 100644 index 0000000000..342a4d4dfc --- /dev/null +++ b/devlog/_plan/260914_r2l8_catalog_autorefresh/010_roadmap.md @@ -0,0 +1,98 @@ +# R2-L8 — catalog auto-refresh and capability declarations + +Lane R2-L8 of the round-23 delivery unit. Branch `codex/260914-l8-catalog-autorefresh`, +one pull request against `dev`. Write scope is the one the lane assignment fixed: +`src/codex/catalog-refresh-status.ts`, `src/codex/convergence.ts`, `src/config.ts`, +`src/types/config.ts`, `src/types/provider.ts`, `src/server/background-lifecycle.ts` +and their tests. L8 is wave A's only config-schema owner, so the schema edits stay +additive and self-contained. + +## What the two issues actually need + +**Periodic catalog auto-refresh (issue 3630).** A running proxy only re-discovers +provider models when someone runs `ocx sync` or restarts. The reporter watched a +newly released upstream model stay absent from `/v1/models` and from the on-disk +catalog until they remembered to sync by hand. The ask is a configurable interval +that drives the same converge path `ocx sync` drives, plus visibility when the +served model set actually changes. + +**Per-model capability declarations (issue 3377).** The declaration half is already +on `dev`: `ModelCapabilities` in `src/types/provider.ts` carries `inputModalities`, +`contextTier` and `video.processing`; `src/config/provider-validation.ts` validates +and merges it; `ocx provider add`/`edit` accept it. Only the text-only axis is live — +`configuredInputModalities` in `src/codex/catalog/parsing.ts`'s neighbour +`catalog/provider-fetch.ts` reads it, and `src/vision/` honours it. `contextTier` and +`video.processing` are stored and inert, and both activation sites +(`src/providers/github-copilot-transport.ts`, `src/adapters/google.ts`, +`src/responses/schema.ts`) sit outside this lane's write scope. This lane therefore +does not close issue 3377; it pins the part it can own. + +## Design + +### Config surface + +```ts +export interface OcxCatalogAutoRefreshConfig { + enabled?: boolean; // master switch, default false + intervalMinutes?: number; // default 60, floor 15, 0 keeps the timer dormant +} +``` + +on `OcxConfig.catalogAutoRefresh`. Opt-in rather than default-on: a refresh spends a +live `/models` call against every enabled provider, and the repository's existing +optional-subsystem rule is that a default install runs no detection code. The floor +exists for the same reason `src/quota/reset-poller.ts` has one — provider catalogs +are cached for minutes upstream, so a one-minute cadence buys nothing and costs a +rate limit. + +Resolvers exported from `src/config.ts`: + +- `isCatalogAutoRefreshEnabled(config?)` — true only when the section is present and + `enabled === true`. +- `resolveCatalogAutoRefreshIntervalMs(config?)` — bounded milliseconds, or `0` when + the operator disabled polling explicitly. + +### Scheduler + +New `src/codex/catalog-auto-refresh.ts`, shaped after `src/quota/reset-poller.ts`: +a module-singleton `setInterval` that is unref'd, an in-flight guard so a slow +provider fetch cannot stack ticks, and a generation counter so a probe still in +flight when the timer stops cannot publish into the next generation. The config gate +lives in the callee, which is what lets an operator toggle the setting without a +restart. Every heavy import — the config barrel, the catalog admission snapshot, +the convergence path — is a dynamic `import()` inside the tick, so importing this +module costs nothing at startup. + +A tick captures a catalog admission snapshot and calls `convergeCodexCatalog` with +`{ scope: "catalog", action: "converge" }`, which is exactly the path `ocx sync` +uses. The existing "external provider owns config.toml" guard lives inside that +path, so it is respected by construction rather than re-implemented here. + +### Observability + +`src/codex/catalog-refresh-status.ts` gains a last-outcome record: when the refresh +ran, its normalized `CatalogDisposition`, whether the served model set changed, and +a consecutive-failure count. A tick that changes the model set logs one line. The +per-model "N new models discovered" count issue 3630 asks for is not delivered: +`convergeCodexCatalog` returns a boolean, not a diff, and widening its return type +reaches into the catalog writers this lane does not own. The dashboard surface for +this record belongs to R2-L9. + +### Registration + +`src/server/background-lifecycle.ts` starts and stops the scheduler alongside the +quota reset poller, and fires the cadence sync as a floating promise for the same +reason that one does: startup must not await an optional subsystem. + +## Cycle plan + +1. Docs cycle — this file. +2. Config schema and resolvers, with focused tests. +3. Scheduler, status record, and lifecycle registration, with focused tests. +4. Capability-declaration regression pinning that a periodic refresh preserves the + declared axes, plus the `structure/config.md` update the SSOT rule requires. + +## Verification posture + +No local suite, no typecheck, no install, no GUI build. Hosted CI at the exact final +head is the only proof this lane reports. diff --git a/devlog/_plan/260914_r2l8_catalog_autorefresh/020_issue_3377_capability_audit.md b/devlog/_plan/260914_r2l8_catalog_autorefresh/020_issue_3377_capability_audit.md new file mode 100644 index 0000000000..79e52666e7 --- /dev/null +++ b/devlog/_plan/260914_r2l8_catalog_autorefresh/020_issue_3377_capability_audit.md @@ -0,0 +1,85 @@ +# Issue 3377 capability-declaration audit + +Lane R2-L8 of the round-23 delivery unit. This note is the honest reading of the +current tree against issue 3377, not a plan to close it. The declaration half +is already on `dev`; what remains is activation, and every remaining activation +site sits outside this lane's write scope. + +## What the type actually carries + +`ModelCapabilities` in `src/types/provider.ts` is the stored declaration. An +exact model-ID entry may set `inputModalities` (`text` / `image` / `audio` / +`video`), `contextTier` (`default` / `long_context`), and `video.processing` +(`static` / `agentic`). The comment on `contextTier` is load-bearing: it is a +requested tier only, and storing it does not imply an upstream window or +activate an unverified wire. `src/config/provider-validation.ts` is the write +gate for all three axes. It rejects unknown axes, validates each vocabulary, +merges PATCH objects without sharing live rows, and treats null +map/model/axis/processing values as tombstones. File load retains valid axes +and restricts a malformed explicit modality list to text. Gather fingerprints +in `src/codex/catalog/provider-fetch.ts` include the whole map, so a periodic +catalog refresh will preserve whatever was declared; preservation is not +activation. + +## Which axes run today + +Only `inputModalities` is consumed at runtime. `configuredInputModalities` in +`src/codex/catalog/provider-fetch.ts` reads an exact `modelCapabilities[id].inputModalities` +entry before the legacy `modelInputModalities` record and writes it onto the +catalog row, including over a live `/models` vote that would otherwise win. +`src/vision/eligibility.ts` (`isModelVisionSidecarConsumer`, exported as +`isModelTextOnly`) treats a declared text-without-image list as the sidecar +consumer, and `src/vision/plan.ts` (`requiresVisionPreprocessing`) consults the +same declaration before legacy hints and vendor metadata. That is the text-only +axis issue 3377 asked for, and it is live. + +`contextTier` is stored and inert. Nothing in `src/providers/github-copilot-transport.ts` +reads it. That file still only stamps Copilot editor-fingerprint headers and +fail-closes the OAuth bearer onto an allowlisted `*.githubcopilot.com` host. A +follow-up that wants `long_context` to mean a larger Copilot window has to own +that transport and the Copilot-specific request header or body field it would +emit; the pricing `contextTier` in `src/usage/cost.ts` is a different vocabulary +and must not be mistaken for this declaration. The catalog already has +`modelContextWindows` / `contextWindow` as a separate numeric contract, and +storing `contextTier: "long_context"` does not advertise those windows. + +`video.processing` is stored and inert in the same way. The inbound and adapter +path already knows how to *carry* a video part: `src/chat/inbound.ts` translates +`video_url` into `input_video`, `src/responses/schema.ts` accepts that block, +and `src/adapters/google.ts` inlines Gemini video bytes (or a short marker for +a remote URL) whenever a `video` part is present. None of those sites reads +`modelCapabilities[id].video.processing`. Static versus agentic is therefore a +label in config, not a processing-mode switch. Activation would have to land in +those three files, and it would have to decide what "agentic" means on the +existing media-bridge loop rather than assuming the Google inline path is +enough. + +## What the surfaces already accept + +The management API already takes the full map. `src/server/management/provider-routes.ts` +validates PATCH `modelCapabilities` with tombstones allowed, merges axes onto +the live row, and on POST/PUT replacement runs the same merge so a complete +body cannot smuggle a tombstone through. Operators can therefore persist +`contextTier` and `video.processing` from the dashboard or a raw editor today; +the proxy will store them, fingerprint them, and do nothing else with them. + +The CLI is narrower. `ocx provider add` in `src/cli/provider.ts` and +`ocx provider edit` in `src/cli/provider-runtime.ts` accept `--model --text-only` +and write `inputModalities: ["text"]` for that one id, preserving sibling +declarations through `mergeModelCapabilities`. There is no `--context-tier` or +`video.processing` flag. An operator who wants those axes from the CLI has to +hand-edit `config.json` or PATCH the management API. + +## What this lane does not close + +Lane R2-L8 does not close issue 3377. Its write scope is the catalog +auto-refresh scheduler, the config section that gates it, the last-outcome +record, and the tests and structure paragraph that pin those. Every remaining +activation site — `src/providers/github-copilot-transport.ts` for the context +tier, `src/adapters/google.ts` plus `src/responses/schema.ts` plus +`src/chat/inbound.ts` for video processing, and a CLI flag surface if the +follow-up wants operator-facing declarations beyond management JSON — is +outside that scope. A later lane that actually closes 3377 has to own those +files, prove the Copilot long-context wire and the static/agentic video split +on a real request, and keep the storage contract in `src/config/provider-validation.ts` +as the write gate rather than re-implementing it. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 35059be7be..81a50f69a2 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -276,6 +276,7 @@ "bun-runtime.test.ts": "ci-workflows", "bun-stream-caps.test.ts": "lib", "cancel-body-on-abort.test.ts": "server", + "catalog-auto-refresh-scheduler.test.ts": "codex-integration", "catalog-cursor-search.test.ts": "codex-integration", "catalog-free-pricing-status.test.ts": "codex-integration", "catalog-full-picker-order.test.ts": "codex-integration", @@ -550,6 +551,7 @@ "compatibility-manifest.test.ts": "codex-integration", "compatibility-provider-equivalence.test.ts": "routing", "compatibility-version.test.ts": "ci-workflows", + "config-catalog-auto-refresh.test.ts": "config", "config-load-degrade.test.ts": "config", "config-mutation-lock.test.ts": "config", "config-ownership-uninstall.test.ts": "config", diff --git a/src/codex/catalog-auto-refresh.ts b/src/codex/catalog-auto-refresh.ts new file mode 100644 index 0000000000..6486b1fb1f --- /dev/null +++ b/src/codex/catalog-auto-refresh.ts @@ -0,0 +1,182 @@ +/** + * Opt-in periodic catalog refresh so newly released models appear without a + * manual `ocx sync` (issue #3630). + * + * This is load-bearing, not a convenience. The served model set is otherwise + * only rewritten by an explicit sync, a management mutation, or startup + * convergence, so an overnight provider release stays invisible until someone + * happens to run one of those. The overnight case is the whole reason the + * scheduler exists. + * + * Shape follows src/quota/reset-poller.ts: a module-singleton unref'd interval + * whose config gate lives in the callee, so toggling `enabled` or changing the + * cadence takes effect on the next tick without a restart (the rationale + * spelled out at src/oauth/token-guardian.ts:276). Importing this module at + * startup must cost nothing — src/server/background-lifecycle.ts loads it + * statically — so the config barrel, the admission snapshot, and the + * convergence funnel are all dynamic import()s inside the tick. + */ + +/** + * Keep these numeric literals aligned with CATALOG_AUTO_REFRESH_* in src/config.ts. + * They cannot be imported from there: this module is a static edge from + * background-lifecycle, and the config barrel is a heavy import reserved for the tick. + */ +const DEFAULT_INTERVAL_MS = 60 * 60_000; +const MIN_INTERVAL_MS = 15 * 60_000; +/** + * Commit-lock wait only. Gather already has per-provider timeouts, and automatic + * callers fail fast and defer (ConvergeRequest.mode) rather than holding the + * write lock across a slow tick. + */ +const TICK_DEADLINE_MS = 1_000; + +let timer: ReturnType | null = null; +let detachShutdownHook: (() => void) | null = null; +/** The bounded cadence the live timer was created with, so a tick can notice config drift. */ +let liveIntervalMs: number | null = null; +/** + * Bumped by every start and stop. A tick captures it on entry and re-checks before publishing, + * so a converge still in flight when the timer stops cannot publish into the next generation. + */ +let generation = 0; +/** setInterval does not skip a firing while the previous callback is still awaiting. */ +let inFlight = false; + +/** Number of ticks that have run. Test-only observability; carries no catalog data. */ +let tickCount = 0; + +function boundedInterval(value: number): number { + return Math.max(MIN_INTERVAL_MS, Math.floor(value)); +} + +/** Re-arm the timer when the operator changed the cadence since it was created. */ +function restartIfCadenceChanged(configured: number): void { + if (timer === null || boundedInterval(configured) === liveIntervalMs) return; + stopCatalogAutoRefresh(); + startCatalogAutoRefresh(configured); +} + +async function tick(): Promise { + // An interval firing while the previous converge is still awaiting would stack + // provider fetches precisely when a slow /models call is already in flight. + if (inFlight) return; + inFlight = true; + const entryGeneration = generation; + try { + const { + loadConfig, + isCatalogAutoRefreshEnabled, + resolveCatalogAutoRefreshIntervalMs, + } = await import("../config"); + const config = loadConfig(); + if (!isCatalogAutoRefreshEnabled(config)) return; + const configured = resolveCatalogAutoRefreshIntervalMs(config); + // 0 is dormant: the section stays configured but this tick must not converge, + // and the unref'd timer is left running so flipping the minutes back on is + // picked up without a process restart. + if (configured === 0) return; + // A stop or restart landed while the config resolved: this tick no longer owns the timer, + // so it must neither count as a refresh nor adopt a cadence for a generation that is gone. + if (entryGeneration !== generation) return; + // Adopt a changed cadence without a restart, which is why the config gate lives in the + // callee at all. Only while this tick still owns the timer. + restartIfCadenceChanged(configured); + tickCount += 1; + const [{ createManagementConvergeCodex }, { createCatalogConvergeRequest }] = await Promise.all([ + import("./management-convergence"), + import("./catalog-admission"), + ]); + // A stop or restart landed while the funnel was loading: the result belongs to a + // generation that no longer owns the timer, so it must not publish. + if (entryGeneration !== generation) return; + const converge = createManagementConvergeCodex(config); + const outcome = await converge(createCatalogConvergeRequest({ deadlineMs: TICK_DEADLINE_MS })); + if (entryGeneration !== generation) return; + // createManagementConvergeCodex always projects catalog-only. Any other kind is a + // funnel contract break, not something this scheduler should re-classify. + if (outcome.kind !== "catalog-only") return; + const { recordCatalogAutoRefreshOutcome } = await import("./catalog-refresh-status"); + recordCatalogAutoRefreshOutcome(outcome.catalogRefresh, outcome.changed); + if (outcome.changed) { + // Privacy scan: no provider names, model ids, paths, or account identifiers. + console.info("[catalog-auto-refresh] served model set changed"); + } + } catch { + // A failed refresh is not an error worth surfacing: the next tick tries again. + } finally { + inFlight = false; + } +} + +/** Idempotent. A second call while running is a no-op, matching startQuotaResetPoller. */ +export function startCatalogAutoRefresh(intervalMs = DEFAULT_INTERVAL_MS): void { + if (timer) return; + const bounded = boundedInterval(intervalMs); + generation += 1; + liveIntervalMs = bounded; + timer = setInterval(() => void tick(), bounded); + // Never keep the process alive for a catalog refresh. + timer.unref?.(); + void import("../lib/optional-shutdown-hooks") + .then(hooks => { + detachShutdownHook = hooks.registerOptionalShutdownHook( + "catalog-auto-refresh", + stopCatalogAutoRefresh, + ); + }) + .catch(() => { + // Without the hook the unref'd timer still cannot delay exit. + }); +} + +export function stopCatalogAutoRefresh(): void { + if (timer) { + clearInterval(timer); + timer = null; + } + liveIntervalMs = null; + generation += 1; + detachShutdownHook?.(); + detachShutdownHook = null; +} + +export function isCatalogAutoRefreshRunning(): boolean { + return timer !== null; +} + +/** + * Adopt the operator's configured cadence at startup. + * + * The caller starts the scheduler synchronously with the default interval, because + * resolving the config here would mean a static edge to ../config from a module + * background-lifecycle imports at load time. Resolving it through import() keeps + * that edge dynamic, at the cost of the timer running at the default for the few + * microtasks before this settles. + */ +export async function syncCatalogAutoRefreshCadence(): Promise { + const { loadConfig, resolveCatalogAutoRefreshIntervalMs } = await import("../config"); + const configured = resolveCatalogAutoRefreshIntervalMs(loadConfig()); + // 0 is dormant: tick() already returns before converging, and the timer stays unref'd. + if (configured === 0) return; + restartIfCadenceChanged(configured); +} + +/** Test-only: run one tick synchronously rather than waiting out the interval. */ +export async function runCatalogAutoRefreshTickForTests(): Promise { + await tick(); +} + +export function catalogAutoRefreshTickCountForTests(): number { + return tickCount; +} + +/** Test-only: the bounded cadence the live timer is running at, or null when stopped. */ +export function catalogAutoRefreshIntervalForTests(): number | null { + return liveIntervalMs; +} + +export function resetCatalogAutoRefreshForTests(): void { + stopCatalogAutoRefresh(); + tickCount = 0; +} diff --git a/src/codex/catalog-refresh-status.ts b/src/codex/catalog-refresh-status.ts index 8286e85090..719318698b 100644 --- a/src/codex/catalog-refresh-status.ts +++ b/src/codex/catalog-refresh-status.ts @@ -103,3 +103,96 @@ function normalizeCatalogFailureCause(value: unknown): CatalogFailureCause | und export function catalogRefreshIsPending(disposition: CatalogDisposition): boolean { return disposition.status !== "committed"; } + +export interface CatalogAutoRefreshOutcome { + readonly at: number; + readonly disposition: CatalogDisposition; + readonly changed: boolean; + /** + * A refresh that has failed repeatedly is the signal an operator needs, and the + * boolean disposition alone cannot express it: skipped and failed look the same + * as a one-off busy skip until this count climbs. + */ + readonly consecutiveFailures: number; +} + +let lastAutoRefreshOutcome: CatalogAutoRefreshOutcome | null = null; + +/** Rebuild and freeze so a management reader cannot mutate scheduler state. */ +function freezeCatalogDisposition(disposition: CatalogDisposition): CatalogDisposition { + if (disposition.status === "committed") { + return Object.freeze({ + status: "committed" as const, + changed: disposition.changed, + degraded: disposition.degraded, + notices: Object.freeze([...disposition.notices]), + }); + } + if (disposition.status === "skipped") { + return Object.freeze({ + status: "skipped" as const, + reason: disposition.reason, + retryable: disposition.retryable, + }); + } + const cause = disposition.cause + ? Object.freeze({ + kind: disposition.cause.kind, + ...(disposition.cause.code ? { code: disposition.cause.code } : {}), + }) + : undefined; + return Object.freeze({ + status: "failed" as const, + reason: disposition.reason, + phase: disposition.phase, + retryable: disposition.retryable, + partialWrite: disposition.partialWrite, + ...(cause ? { cause } : {}), + }); +} + +function freezeCatalogAutoRefreshOutcome( + outcome: CatalogAutoRefreshOutcome, +): CatalogAutoRefreshOutcome { + return Object.freeze({ + at: outcome.at, + disposition: freezeCatalogDisposition(outcome.disposition), + changed: outcome.changed, + consecutiveFailures: outcome.consecutiveFailures, + }); +} + +/** + * Record one auto-refresh tick. The disposition is rebuilt through + * normalizeCatalogDisposition before anything is stored: an unnormalizable + * value is exactly the case this privacy boundary exists for, so it is dropped + * rather than copied through into a management response. + */ +export function recordCatalogAutoRefreshOutcome( + disposition: CatalogDisposition, + changed: boolean, +): CatalogAutoRefreshOutcome | null { + const normalized = normalizeCatalogDisposition(disposition); + if (normalized === null) return null; + const consecutiveFailures = catalogRefreshIsPending(normalized) + ? (lastAutoRefreshOutcome?.consecutiveFailures ?? 0) + 1 + : 0; + const outcome = freezeCatalogAutoRefreshOutcome({ + at: Date.now(), + disposition: normalized, + changed: changed === true, + consecutiveFailures, + }); + lastAutoRefreshOutcome = outcome; + return freezeCatalogAutoRefreshOutcome(outcome); +} + +export function lastCatalogAutoRefreshOutcome(): CatalogAutoRefreshOutcome | null { + return lastAutoRefreshOutcome === null + ? null + : freezeCatalogAutoRefreshOutcome(lastAutoRefreshOutcome); +} + +export function resetCatalogAutoRefreshStatusForTests(): void { + lastAutoRefreshOutcome = null; +} diff --git a/src/config.ts b/src/config.ts index 601a70bbc2..4f851e4ffb 100644 --- a/src/config.ts +++ b/src/config.ts @@ -1227,6 +1227,24 @@ const quotaResetNotifySchema = z.object({ command: z.array(z.string()).optional(), }).strict(); +/** + * Catalog auto-refresh section (issue #3630). + * + * `.strict()` like its neighbour: a typo in an optional feature section should surface as a + * rejected write rather than a silently ignored key that leaves the operator believing they + * enabled something. + * + * `intervalMinutes` admits 0 (configured but dormant, no timer) and the resolver clamps + * anything between 1 and the 15-minute floor. Bounds live in the resolver rather than here + * so a hand-edited value degrades to a sane one instead of discarding the whole section. + * The 1440 ceiling keeps a hand edit from scheduling the refresh further out than a day, + * which is operator error far more often than intent. + */ +const catalogAutoRefreshSchema = z.object({ + enabled: z.boolean().optional(), + intervalMinutes: z.number().int().min(0).max(1440).optional(), +}).strict(); + const configSchema = z.object({ port: z.number().int().min(0).max(65535).default(10100), // A malformed hand edit must disable only remote-role behavior, not discard @@ -1331,6 +1349,8 @@ const configSchema = z.object({ agentTaskRecovery: agentTaskRecoverySchema.optional().catch(undefined), // Same rationale: a bad notify section must not cost the operator their providers. quotaResetNotify: quotaResetNotifySchema.optional().catch(undefined), + // Same rationale: a bad auto-refresh section must not cost the operator their providers. + catalogAutoRefresh: catalogAutoRefreshSchema.optional().catch(undefined), // These selections pre-date schema validation and used to pass through as // unknown fields. Invalid hand edits must disable only the optional // delegation/native-default feature, not reject the whole config and hide @@ -2372,6 +2392,15 @@ function malformedQuotaResetNotifyWarning(rawParsed: unknown): string | null { return `quotaResetNotify${field ? `.${field}` : ""} ignored: invalid quota-reset notification configuration`; } +function malformedCatalogAutoRefreshWarning(rawParsed: unknown): string | null { + const raw = rawConfigRecord(rawParsed); + if (!raw || !Object.hasOwn(raw, "catalogAutoRefresh")) return null; + const result = catalogAutoRefreshSchema.safeParse(raw.catalogAutoRefresh); + if (result.success) return null; + const field = result.error.issues[0]?.path.join("."); + return `catalogAutoRefresh${field ? `.${field}` : ""} ignored: invalid catalog auto-refresh configuration`; +} + /** * Same silent-in-the-wrong-direction failure as the notification block: a dropped pool policy means * the accounts the operator meant to exclude keep taking traffic, and the only visible symptom is @@ -2398,6 +2427,18 @@ function warnDegradedQuotaResetNotify(rawParsed: unknown): void { if (warning) console.warn(`⚠️ config.json ${warning}. Other settings were preserved.`); } +/** + * Warn once per load that the section was dropped. + * + * Same silent-in-the-wrong-direction failure as the notification block: a dropped section + * means the scheduler never starts, so the operator sees a stale catalog — which is exactly + * what they would see if the feature were working and no new models had shipped. + */ +function warnDegradedCatalogAutoRefresh(rawParsed: unknown): void { + const warning = malformedCatalogAutoRefreshWarning(rawParsed); + if (warning) console.warn(`⚠️ config.json ${warning}. Other settings were preserved.`); +} + /** * Warn once per load that the pool policy was dropped. * @@ -2573,6 +2614,7 @@ export function loadConfig(): OcxConfig { warnDegradedRuntimeRole(parsed); warnDegradedOptionalRemoteBlocks(parsed); warnDegradedQuotaResetNotify(parsed); + warnDegradedCatalogAutoRefresh(parsed); warnDegradedCodexPool(parsed); return withRefreshedCostOverlays(normalizeClaudeSubagentEffort(normalizeNativeSubagentSync(config, parsed), parsed)); } @@ -2615,6 +2657,7 @@ export function loadConfig(): OcxConfig { warnDegradedRuntimeRole(parsed); warnDegradedOptionalRemoteBlocks(parsed); warnDegradedQuotaResetNotify(parsed); + warnDegradedCatalogAutoRefresh(parsed); warnDegradedCodexPool(parsed); return withRefreshedCostOverlays(normalizeClaudeSubagentEffort(normalizeNativeSubagentSync(config, parsed), parsed)); } @@ -2642,6 +2685,7 @@ export function loadConfig(): OcxConfig { warnDegradedRuntimeRole(parsed); warnDegradedOptionalRemoteBlocks(parsed); warnDegradedQuotaResetNotify(parsed); + warnDegradedCatalogAutoRefresh(parsed); warnDegradedCodexPool(parsed); return withRefreshedCostOverlays(normalizeClaudeSubagentEffort(normalizeNativeSubagentSync(config, parsed), parsed)); } @@ -2788,6 +2832,8 @@ function validFileConfigDiagnostics(config: OcxConfig, rawParsed: unknown): Conf if (clientWarning) warnings.push(clientWarning); const notifyWarning = malformedQuotaResetNotifyWarning(rawParsed); if (notifyWarning) warnings.push(notifyWarning); + const catalogRefreshWarning = malformedCatalogAutoRefreshWarning(rawParsed); + if (catalogRefreshWarning) warnings.push(catalogRefreshWarning); const codexPoolWarning = malformedCodexPoolWarning(rawParsed); if (codexPoolWarning) warnings.push(codexPoolWarning); const plaintextWarning = malformedPlaintextV2AgentMessagesWarning(rawParsed); @@ -2953,6 +2999,16 @@ function quotaResetNotifyError(value: unknown): string | null { return `schema_invalid: quotaResetNotify${field ? `.${field}` : ""}: ${issue?.message ?? "invalid configuration"}`; } +function catalogAutoRefreshError(value: unknown): string | null { + const raw = rawConfigRecord(value); + if (!raw || !Object.hasOwn(raw, "catalogAutoRefresh") || raw.catalogAutoRefresh === undefined) return null; + const result = catalogAutoRefreshSchema.safeParse(raw.catalogAutoRefresh); + if (result.success) return null; + const issue = result.error.issues[0]; + const field = issue?.path.join("."); + return `schema_invalid: catalogAutoRefresh${field ? `.${field}` : ""}: ${issue?.message ?? "invalid configuration"}`; +} + /** * The read path degrades a malformed pool policy to undefined, which for an exclusion policy means * the excluded accounts quietly keep serving traffic. Reject it on write so `ocx config set` cannot @@ -3188,6 +3244,7 @@ export function validateConfigCandidate(value: unknown): { ok: true; config: Ocx ?? plaintextV2AgentMessagesError(value) ?? agentTaskRecoveryError(value) ?? quotaResetNotifyError(value) + ?? catalogAutoRefreshError(value) ?? codexPoolError(value) ?? googleAntigravityStaticCatalogVersionError(value) ?? codexAccountPrioritiesError(value) @@ -3788,6 +3845,48 @@ export function ultraFastTierEnabled(config: Pick): return config.ultraFastTier === true; } +/** + * Default cadence for the opt-in catalog auto-refresh (issue #3630): one converge pass + * per hour. Each pass spends a live /models call against every enabled provider, and + * provider catalogs are themselves cached upstream for minutes, so an hour is fresh + * enough for newly released models to appear without an `ocx sync`. + */ +export const CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS: number = 60 * 60_000; + +/** + * Floor under the configured cadence, for the same reason src/quota/reset-poller.ts has + * MIN_INTERVAL_MS: below this the refresh buys no freshness — upstream caches have not + * moved — and only multiplies the chance of a rate limit across every enabled provider. + */ +export const CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS: number = 15 * 60_000; + +/** + * Opt-in master switch, read with the house `=== true` idiom so an absent key and a + * malformed one both mean off. Pure on purpose: the scheduler calls this from a + * dynamically imported context, so it takes an explicit config slice and reads nothing + * global. + */ +export function isCatalogAutoRefreshEnabled( + config: Pick, +): boolean { + return config.catalogAutoRefresh?.enabled === true; +} + +/** + * Resolved tick interval in milliseconds. An explicit `intervalMinutes: 0` returns 0 — + * the section stays configured but the timer stays dormant — and any other value is + * clamped up to CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS so a hand edit cannot outrun the + * upstream catalog caches. Absent means the hourly default. + */ +export function resolveCatalogAutoRefreshIntervalMs( + config: Pick, +): number { + const minutes = config.catalogAutoRefresh?.intervalMinutes; + if (minutes === undefined) return CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS; + if (minutes === 0) return 0; + return Math.max(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS, Math.floor(minutes * 60_000)); +} + // --------------------------------------------------------------------------- // Hand-edit protection for the `claudeCode` subtree (devlog 260726_claude_auth_auto/040 H1). // diff --git a/src/server/background-lifecycle.ts b/src/server/background-lifecycle.ts index 17a7a7fe57..a2cf4da136 100644 --- a/src/server/background-lifecycle.ts +++ b/src/server/background-lifecycle.ts @@ -11,6 +11,11 @@ import { stopStorageCleanupScheduler, } from "../storage/policy-scheduler"; import { startQuotaResetPoller, stopQuotaResetPoller } from "../quota/reset-poller"; +import { + startCatalogAutoRefresh, + stopCatalogAutoRefresh, + syncCatalogAutoRefreshCadence, +} from "../codex/catalog-auto-refresh"; import { cancelQueuedStorageWorkerSpawns, drainStorageWorkers, @@ -70,6 +75,17 @@ function startProcessLoops(applyPolicy: PolicyApply): ProcessLoops { .catch(() => { // The next tick adopts it. }); + // Opt-in: the tick is a no-op unless catalogAutoRefresh.enabled is true, and the + // interval is unref'd, so a default install pays one dormant timer. The scheduler + // module keeps every heavy import inside its tick, so naming it statically here + // costs a module record and nothing else. + startCatalogAutoRefresh(); + // The scheduler starts at its default cadence because resolving the operator's value + // reads the config barrel. Fire-and-forget: startup must not await an optional + // subsystem, and the next tick adopts the cadence anyway. + void syncCatalogAutoRefreshCadence().catch(() => { + // The next tick adopts it. + }); // Install the delivery sink now rather than waiting out the first poll interval, which is 15 // minutes by default. Without this, an enabled install would observe nothing for its first // quarter hour — including the live request path, which is gated on the sink existing. @@ -85,6 +101,7 @@ function startProcessLoops(applyPolicy: PolicyApply): ProcessLoops { stateStoreSweeper?.stop(); stopStorageCleanupScheduler(); stopQuotaResetPoller(); + stopCatalogAutoRefresh(); setLivePolicyOwner(null); throw error; } @@ -97,6 +114,7 @@ function stopProcessLoops(): void { loops?.stateStoreSweeper.stop(); stopStorageCleanupScheduler(); stopQuotaResetPoller(); + stopCatalogAutoRefresh(); setLivePolicyOwner(null); } diff --git a/src/types/config.ts b/src/types/config.ts index 8f87281eb8..39c93571b6 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -668,6 +668,16 @@ export interface OcxConfig { * so absence is the only default state this feature has. */ quotaResetNotify?: OcxQuotaResetNotifyConfig; + /** + * Periodic provider model-catalog refresh (issue #3630). Absent means off: no timer, no + * refresh pass, no outcome record. + * + * Off by default for the same reason every optional subsystem here is: a refresh spends a + * live /models call against every enabled provider, and this repository's rule is that a + * default install runs no detection code and starts no live timer work. Not in + * `getDefaultConfig()` — absence is the only default state this feature has. + */ + catalogAutoRefresh?: OcxCatalogAutoRefreshConfig; /** Active provider context limits; native long windows remain within their supported ceilings. */ providerContextCaps?: Record; /** Last selected provider caps; retained while a cap is switched off. Not an active limit. */ @@ -1303,3 +1313,26 @@ export interface OcxQuotaResetNotifyConfig { */ command?: string[]; } + +/** + * Periodic model-catalog auto-refresh settings (issue #3630). + * + * Every field is optional and the whole section defaults to off. Each tick converges the + * served catalog the same way `ocx sync` does, which costs a live /models call against + * every enabled provider — so an install that never asked for this must run no refresh + * code and start no timer, matching the optional-subsystem rule the rest of this file + * follows. + */ +export interface OcxCatalogAutoRefreshConfig { + /** Master switch. Default false — no scheduler, no tick, no upstream calls. */ + enabled?: boolean; + /** + * Minutes between refresh ticks. Default 60, floor 15, and 0 keeps the timer dormant + * while leaving the section configured. + * + * The floor exists for the same reason src/quota/reset-poller.ts has MIN_INTERVAL_MS: + * provider catalogs are cached upstream for minutes, so a faster cadence buys no + * freshness and only risks a rate limit against every enabled provider at once. + */ + intervalMinutes?: number; +} diff --git a/structure/config.md b/structure/config.md index 7beca1118c..a4c0b97ead 100644 --- a/structure/config.md +++ b/structure/config.md @@ -304,3 +304,7 @@ Display-name validation retains prototype-shaped model IDs as data; reviewer-tar `modelCapabilities` on `src/types/provider.ts` stores exact model-ID entries with optional inputModalities, contextTier and video.processing axes. `src/config/provider-validation.ts` strictly validates writes and merges PATCH axes without sharing live objects; null map/model/axis/processing tombstones delete, while empty PATCH objects do nothing. Complete POST/PUT replacements reject tombstones. File reads retain valid axes; malformed explicit modalities restrict to text with a diagnostic. The two catalog writers receive explicit config and gather fingerprints include the map. This storage contract alone does not activate a context tier, advertise a larger window or enable video processing. The text-only consumer reads exact inputModalities declarations before legacy hints. CLI add/edit `--text-only` targets one model and preserves sibling declarations; `src/vision/eligibility.ts` routes declared text-only models into existing image-description or explicit-omission handling. Positive routed image declarations override stale candidate metadata, while native catalog authority retains its existing legacy policy. + +## Catalog auto-refresh + +`catalogAutoRefresh` on `src/types/config.ts` stores an optional `enabled` / `intervalMinutes` section that defaults off: an absent key, an explicit false, and a malformed value all leave the scheduler dormant. `src/config.ts` resolves the cadence; an explicit `intervalMinutes: 0` keeps the unref'd timer idle, and any other value is clamped up to 15 minutes because upstream `/models` caches have not moved below that and a shorter tick only multiplies rate-limit exposure. `src/codex/catalog-auto-refresh.ts` is the module-singleton interval `src/server/background-lifecycle.ts` starts beside the quota reset poller; a tick that is enabled and non-dormant drives the same catalog-only converge funnel management mutations drive. The last-outcome record lives in `src/codex/catalog-refresh-status.ts` (when the tick finished, the normalized `CatalogDisposition`, whether the served model set changed, consecutive failures) and carries no provider or account detail. diff --git a/tests/codex-integration/catalog-auto-refresh-scheduler.test.ts b/tests/codex-integration/catalog-auto-refresh-scheduler.test.ts new file mode 100644 index 0000000000..04508442da --- /dev/null +++ b/tests/codex-integration/catalog-auto-refresh-scheduler.test.ts @@ -0,0 +1,205 @@ +import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + catalogAutoRefreshIntervalForTests, + catalogAutoRefreshTickCountForTests, + isCatalogAutoRefreshRunning, + resetCatalogAutoRefreshForTests, + runCatalogAutoRefreshTickForTests, + startCatalogAutoRefresh, + stopCatalogAutoRefresh, +} from "../../src/codex/catalog-auto-refresh"; +import { lastCatalogAutoRefreshOutcome, resetCatalogAutoRefreshStatusForTests } from "../../src/codex/catalog-refresh-status"; +import type { CatalogOnlyOutcome } from "../../src/codex/convergence-types"; +import * as managementConvergence from "../../src/codex/management-convergence"; +import { + CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS, + getConfigPath, + getDefaultConfig, +} from "../../src/config"; +import { + installIsolatedCodexHome, + type IsolatedCodexHome, +} from "../helpers/isolated-codex-home"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +const COMMITTED_CATALOG_ONLY = { + kind: "catalog-only", + changed: false, + catalogRefresh: { status: "committed", changed: false, degraded: false, notices: [] }, +} as CatalogOnlyOutcome; + +let previousOpenCodexHome: string | undefined; +let openCodexHome = ""; +let isolatedCodexHome: IsolatedCodexHome | null = null; +let convergeFactoryCalls = 0; +let convergeImpl: () => Promise = async () => COMMITTED_CATALOG_ONLY; +let convergeSpy: { mockRestore(): void } | null = null; +let releaseHanging: ((outcome: CatalogOnlyOutcome) => void) | null = null; +let pendingTick: Promise | null = null; + +function writeCatalogAutoRefreshConfig(catalogAutoRefresh?: unknown): void { + const config = { + ...getDefaultConfig(), + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-responses", + baseUrl: "https://api.x.ai/v1", + }, + }, + ...(catalogAutoRefresh === undefined ? {} : { catalogAutoRefresh }), + }; + writeFileSync(getConfigPath(), JSON.stringify(config), "utf8"); +} + +beforeEach(() => { + previousOpenCodexHome = process.env.OPENCODEX_HOME; + openCodexHome = mkdtempSync(join(tmpdir(), "ocx-catalog-auto-refresh-")); + process.env.OPENCODEX_HOME = openCodexHome; + isolatedCodexHome = installIsolatedCodexHome("ocx-catalog-auto-refresh-codex-"); + resetCatalogAutoRefreshForTests(); + resetCatalogAutoRefreshStatusForTests(); + convergeFactoryCalls = 0; + convergeImpl = async () => COMMITTED_CATALOG_ONLY; + releaseHanging = null; + pendingTick = null; + // The tick's only converge seam is a dynamic import of management-convergence. + // Stub it so an enabled fixture cannot spend a live /models call or rewrite the catalog. + convergeSpy = spyOn(managementConvergence, "createManagementConvergeCodex").mockImplementation(() => { + convergeFactoryCalls += 1; + return convergeImpl; + }); +}); + +afterEach(async () => { + releaseHanging?.(COMMITTED_CATALOG_ONLY); + releaseHanging = null; + if (pendingTick) { + await pendingTick; + pendingTick = null; + } + stopCatalogAutoRefresh(); + resetCatalogAutoRefreshForTests(); + resetCatalogAutoRefreshStatusForTests(); + convergeSpy?.mockRestore(); + convergeSpy = null; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (previousOpenCodexHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousOpenCodexHome; + if (openCodexHome) removeTreeWithRetry(openCodexHome); + openCodexHome = ""; +}); + +describe("catalog auto-refresh scheduler", () => { + test("start is idempotent, clamps below the floor, unrefs the timer, and stop clears the cadence", () => { + // A live 15-minute interval would keep a test process alive if it were ref'd, which is + // the whole reason start unrefs. Spying setInterval is how the sweeper and update-job + // tests prove that property without waiting out the floor. + const timers: Array<{ delay: number; unrefCalls: number }> = []; + const setSpy = spyOn(globalThis, "setInterval").mockImplementation((( + _callback: () => void, + delay?: number, + ) => { + const timer = { + delay: delay ?? 0, + unrefCalls: 0, + unref() { + this.unrefCalls += 1; + return this; + }, + }; + timers.push(timer); + return timer; + }) as typeof setInterval); + const clearSpy = spyOn(globalThis, "clearInterval").mockImplementation(() => {}); + try { + startCatalogAutoRefresh(60_000); + startCatalogAutoRefresh(30 * 60_000); + expect(isCatalogAutoRefreshRunning()).toBe(true); + expect(timers).toHaveLength(1); + expect(timers[0]!.delay).toBe(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + expect(timers[0]!.unrefCalls).toBe(1); + expect(catalogAutoRefreshIntervalForTests()).toBe(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + + stopCatalogAutoRefresh(); + expect(isCatalogAutoRefreshRunning()).toBe(false); + expect(catalogAutoRefreshIntervalForTests()).toBeNull(); + expect(clearSpy).toHaveBeenCalledTimes(1); + } finally { + setSpy.mockRestore(); + clearSpy.mockRestore(); + } + }); + + test("resetCatalogAutoRefreshForTests leaves no live timer", () => { + startCatalogAutoRefresh(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + expect(isCatalogAutoRefreshRunning()).toBe(true); + resetCatalogAutoRefreshForTests(); + expect(isCatalogAutoRefreshRunning()).toBe(false); + expect(catalogAutoRefreshIntervalForTests()).toBeNull(); + expect(catalogAutoRefreshTickCountForTests()).toBe(0); + }); + + test("a tick with catalogAutoRefresh absent or enabled:false performs no converge", async () => { + writeCatalogAutoRefreshConfig(); + await runCatalogAutoRefreshTickForTests(); + expect(catalogAutoRefreshTickCountForTests()).toBe(0); + expect(convergeFactoryCalls).toBe(0); + expect(lastCatalogAutoRefreshOutcome()).toBeNull(); + + writeCatalogAutoRefreshConfig({ enabled: false, intervalMinutes: 60 }); + await runCatalogAutoRefreshTickForTests(); + expect(catalogAutoRefreshTickCountForTests()).toBe(0); + expect(convergeFactoryCalls).toBe(0); + expect(lastCatalogAutoRefreshOutcome()).toBeNull(); + }); + + test("a tick with intervalMinutes:0 stays dormant even when enabled", async () => { + // 0 is configured-but-idle, not a missing interval: clamping it to the floor would + // start the /models fan-out the operator declined. + writeCatalogAutoRefreshConfig({ enabled: true, intervalMinutes: 0 }); + await runCatalogAutoRefreshTickForTests(); + expect(catalogAutoRefreshTickCountForTests()).toBe(0); + expect(convergeFactoryCalls).toBe(0); + expect(lastCatalogAutoRefreshOutcome()).toBeNull(); + }); + + test("an overlapping tick returns immediately without a second converge", async () => { + writeCatalogAutoRefreshConfig({ enabled: true, intervalMinutes: 60 }); + + let release!: (outcome: CatalogOnlyOutcome) => void; + const hanging = new Promise((resolve) => { + release = resolve; + }); + releaseHanging = release; + let enteredFactory: () => void = () => {}; + const factoryEntered = new Promise((resolve) => { + enteredFactory = resolve; + }); + convergeImpl = () => { + enteredFactory(); + return hanging; + }; + + const first = runCatalogAutoRefreshTickForTests(); + pendingTick = first; + await factoryEntered; + const second = runCatalogAutoRefreshTickForTests(); + await second; + + // setInterval does not skip a firing while the previous callback is still awaiting; + // the in-flight guard is what stops a slow /models call from stacking another. + expect(convergeFactoryCalls).toBe(1); + expect(catalogAutoRefreshTickCountForTests()).toBe(1); + expect(lastCatalogAutoRefreshOutcome()).toBeNull(); + + release(COMMITTED_CATALOG_ONLY); + await first; + expect(convergeFactoryCalls).toBe(1); + expect(catalogAutoRefreshTickCountForTests()).toBe(1); + }); +}); diff --git a/tests/codex-integration/codex-catalog-refresh-status.test.ts b/tests/codex-integration/codex-catalog-refresh-status.test.ts index ed4e97ab24..5a728b05b8 100644 --- a/tests/codex-integration/codex-catalog-refresh-status.test.ts +++ b/tests/codex-integration/codex-catalog-refresh-status.test.ts @@ -1,10 +1,17 @@ -import { describe, expect, test } from "bun:test"; +import { afterEach, describe, expect, test } from "bun:test"; import { catalogRefreshIsPending, + lastCatalogAutoRefreshOutcome, normalizeCatalogDisposition, + recordCatalogAutoRefreshOutcome, + resetCatalogAutoRefreshStatusForTests, } from "../../src/codex/catalog-refresh-status"; import type { CatalogDisposition } from "../../src/codex/convergence-types"; +afterEach(() => { + resetCatalogAutoRefreshStatusForTests(); +}); + describe("catalogRefreshIsPending", () => { test("only committed catalog state is complete", () => { const committed: CatalogDisposition = { @@ -107,3 +114,118 @@ describe("normalizeCatalogDisposition", () => { expect(iteratorCalls).toBe(0); }); }); + +describe("catalog auto-refresh last-outcome record", () => { + test("consecutiveFailures climbs across pending dispositions and resets on a commit", () => { + const skipped: CatalogDisposition = { status: "skipped", reason: "busy", retryable: true }; + const failed: CatalogDisposition = { + status: "failed", + reason: "disk", + phase: "commit", + retryable: false, + partialWrite: true, + }; + const committed: CatalogDisposition = { + status: "committed", + changed: true, + degraded: false, + notices: [], + }; + + const first = recordCatalogAutoRefreshOutcome(skipped, false); + expect(first?.consecutiveFailures).toBe(1); + const second = recordCatalogAutoRefreshOutcome(failed, false); + expect(second?.consecutiveFailures).toBe(2); + expect(lastCatalogAutoRefreshOutcome()?.consecutiveFailures).toBe(2); + + const done = recordCatalogAutoRefreshOutcome(committed, true); + expect(done?.consecutiveFailures).toBe(0); + expect(lastCatalogAutoRefreshOutcome()?.consecutiveFailures).toBe(0); + expect(lastCatalogAutoRefreshOutcome()?.changed).toBe(true); + }); + + test("an unnormalizable disposition is dropped without changing the last outcome", () => { + const seeded = recordCatalogAutoRefreshOutcome({ + status: "skipped", + reason: "busy", + retryable: true, + }, false); + expect(seeded?.consecutiveFailures).toBe(1); + + let coercions = 0; + const coerciveReason = { + toString: () => { + coercions += 1; + return "disk"; + }, + }; + expect(recordCatalogAutoRefreshOutcome({ + status: "failed", + reason: coerciveReason, + phase: "commit", + retryable: false, + partialWrite: true, + } as unknown as CatalogDisposition, true)).toBeNull(); + expect(coercions).toBe(0); + + let getterReads = 0; + const accessorDisposition: Record = { + status: "failed", + phase: "commit", + retryable: false, + partialWrite: true, + }; + Object.defineProperty(accessorDisposition, "reason", { + enumerable: true, + get: () => { + getterReads += 1; + return "disk"; + }, + }); + expect(recordCatalogAutoRefreshOutcome( + accessorDisposition as unknown as CatalogDisposition, + true, + )).toBeNull(); + expect(getterReads).toBe(0); + + const last = lastCatalogAutoRefreshOutcome(); + expect(last?.consecutiveFailures).toBe(1); + expect(last?.disposition).toEqual({ status: "skipped", reason: "busy", retryable: true }); + expect(last?.changed).toBe(false); + }); + + test("lastCatalogAutoRefreshOutcome hands back a frozen value a caller cannot mutate into scheduler state", () => { + recordCatalogAutoRefreshOutcome({ + status: "committed", + changed: false, + degraded: true, + notices: ["provider-auth"], + }, false); + const last = lastCatalogAutoRefreshOutcome(); + expect(last).not.toBeNull(); + expect(Object.isFrozen(last)).toBe(true); + expect(Object.isFrozen(last!.disposition)).toBe(true); + if (last!.disposition.status === "committed") { + expect(Object.isFrozen(last!.disposition.notices)).toBe(true); + expect(() => { + (last!.disposition.notices as string[]).push("fallback"); + }).toThrow(); + } + expect(() => { + (last as { consecutiveFailures: number }).consecutiveFailures = 99; + }).toThrow(); + expect(() => { + (last as { changed: boolean }).changed = true; + }).toThrow(); + + const reread = lastCatalogAutoRefreshOutcome(); + expect(reread?.consecutiveFailures).toBe(0); + expect(reread?.changed).toBe(false); + expect(reread?.disposition).toEqual({ + status: "committed", + changed: false, + degraded: true, + notices: ["provider-auth"], + }); + }); +}); diff --git a/tests/config/config-catalog-auto-refresh.test.ts b/tests/config/config-catalog-auto-refresh.test.ts new file mode 100644 index 0000000000..dc142d2e29 --- /dev/null +++ b/tests/config/config-catalog-auto-refresh.test.ts @@ -0,0 +1,133 @@ +import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS, + CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS, + getConfigPath, + getDefaultConfig, + isCatalogAutoRefreshEnabled, + loadConfig, + resolveCatalogAutoRefreshIntervalMs, + validateConfigCandidate, +} from "../../src/config"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +let home = ""; +let previousHome: string | undefined; + +beforeEach(() => { + previousHome = process.env.OPENCODEX_HOME; + home = mkdtempSync(join(tmpdir(), "ocx-catalog-auto-refresh-config-")); + process.env.OPENCODEX_HOME = home; +}); + +afterEach(() => { + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + removeTreeWithRetry(home); +}); + +function candidate(catalogAutoRefresh: unknown) { + return { + ...getDefaultConfig(), + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-responses", + baseUrl: "https://api.x.ai/v1", + note: "keep me", + }, + }, + catalogAutoRefresh, + }; +} + +test("resolveCatalogAutoRefreshIntervalMs defaults to the hourly cadence", () => { + // Absence is the feature's only default state: no section and no intervalMinutes both + // resolve to the same hourly pass, so an operator who writes only { enabled: true } + // gets the documented cadence. + expect(resolveCatalogAutoRefreshIntervalMs({})).toBe(CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS); + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: {} })) + .toBe(CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS); + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { enabled: true } })) + .toBe(CATALOG_AUTO_REFRESH_DEFAULT_INTERVAL_MS); +}); + +test("resolveCatalogAutoRefreshIntervalMs honours 0 as configured-but-dormant", () => { + // 0 is a real configuration, not a missing one: the operator asked for the section to + // exist with no timer, and clamping it up to the floor would start work they declined. + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { intervalMinutes: 0 } })) + .toBe(0); +}); + +test("resolveCatalogAutoRefreshIntervalMs clamps below the floor and honours values above it", () => { + // Below the floor a refresh buys no freshness — upstream provider caches have not moved — + // and only multiplies rate-limit exposure, so the resolver lifts it rather than failing. + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { intervalMinutes: 1 } })) + .toBe(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { intervalMinutes: 14 } })) + .toBe(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { intervalMinutes: 15 } })) + .toBe(CATALOG_AUTO_REFRESH_MIN_INTERVAL_MS); + expect(resolveCatalogAutoRefreshIntervalMs({ catalogAutoRefresh: { intervalMinutes: 120 } })) + .toBe(120 * 60_000); +}); + +test("isCatalogAutoRefreshEnabled reads true only for an explicit enabled:true", () => { + // The house === true idiom keeps an absent key, an explicit false, and a hand-edited + // truthy string all reading off, so a malformed edit cannot start a live timer. + expect(isCatalogAutoRefreshEnabled({})).toBe(false); + expect(isCatalogAutoRefreshEnabled({ catalogAutoRefresh: {} })).toBe(false); + expect(isCatalogAutoRefreshEnabled({ catalogAutoRefresh: { enabled: false } })).toBe(false); + expect(isCatalogAutoRefreshEnabled({ catalogAutoRefresh: { enabled: "yes" as never } })).toBe(false); + expect(isCatalogAutoRefreshEnabled({ catalogAutoRefresh: { enabled: true } })).toBe(true); +}); + +test("validateConfigCandidate rejects a malformed section naming the field", () => { + const result = validateConfigCandidate(candidate({ enabled: "yes" })); + expect(result.ok).toBe(false); + if (result.ok) throw new Error("unreachable"); + expect(result.error).toContain("schema_invalid: catalogAutoRefresh.enabled"); + + const outOfRange = validateConfigCandidate(candidate({ intervalMinutes: -5 })); + expect(outOfRange.ok).toBe(false); + if (outOfRange.ok) throw new Error("unreachable"); + expect(outOfRange.error).toContain("schema_invalid: catalogAutoRefresh.intervalMinutes"); + + // .strict() like its neighbours: a typo'd key must surface as a rejected write rather + // than a silently ignored key that leaves the operator believing they enabled something. + const typo = validateConfigCandidate(candidate({ enabled: true, intervlaMinutes: 30 })); + expect(typo.ok).toBe(false); +}); + +test("validateConfigCandidate accepts a well-formed section", () => { + expect(validateConfigCandidate(candidate({ enabled: true, intervalMinutes: 30 })).ok).toBe(true); + expect(validateConfigCandidate(candidate({ intervalMinutes: 0 })).ok).toBe(true); + expect(validateConfigCandidate(candidate(undefined)).ok).toBe(true); +}); + +test("load drops only a malformed section and preserves the rest of the config", () => { + // Same silent-in-the-wrong-direction failure as quotaResetNotify: discarding the whole + // file over a bad optional section would cost the operator their providers, while + // dropping only the section leaves a working config and a visible warning. + writeFileSync(getConfigPath(), JSON.stringify(candidate({ enabled: "yes" })), "utf8"); + + const warn = spyOn(console, "warn").mockImplementation(() => {}); + try { + const loaded = loadConfig(); + expect(loaded.catalogAutoRefresh).toBeUndefined(); + expect(loaded.providers.xai.note).toBe("keep me"); + const messages = warn.mock.calls.flat().join("\n"); + expect(messages).toContain("catalogAutoRefresh.enabled ignored"); + } finally { + warn.mockRestore(); + } +}); + +test("load keeps a well-formed section intact", () => { + writeFileSync(getConfigPath(), JSON.stringify(candidate({ enabled: true, intervalMinutes: 45 })), "utf8"); + const loaded = loadConfig(); + expect(loaded.catalogAutoRefresh).toEqual({ enabled: true, intervalMinutes: 45 }); +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 0b93419775..0888b0825f 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -110,6 +110,7 @@ "bun-runtime.test.ts": "ci-workflows", "bun-stream-caps.test.ts": "lib", "cancel-body-on-abort.test.ts": "server", + "catalog-auto-refresh-scheduler.test.ts": "codex-integration", "catalog-cursor-search.test.ts": "codex-integration", "catalog-free-pricing-status.test.ts": "codex-integration", "catalog-full-picker-order.test.ts": "codex-integration", @@ -384,6 +385,7 @@ "compatibility-manifest.test.ts": "codex-integration", "compatibility-provider-equivalence.test.ts": "routing", "compatibility-version.test.ts": "ci-workflows", + "config-catalog-auto-refresh.test.ts": "config", "config-load-degrade.test.ts": "config", "config-mutation-lock.test.ts": "config", "config-ownership-uninstall.test.ts": "config",