Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions src/codex/history-job.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import type {
CodexHistoryWorkerOperation,
HistoryWorkerResult,
} from "./history-worker";
import { historyBackupPathFor } from "./history-provider";
import { resolveExistingHistoryBackupPath } from "./history-provider";
import type { CodexHistoryFailureReason, CodexHistoryVerifiedNoopProof } from "./history-provider";
import { getCodexHome, resolveCodexStateDbPath } from "./paths";

Expand All @@ -32,7 +32,7 @@ import { getCodexHome, resolveCodexStateDbPath } from "./paths";
*
* The SQLite root can differ from CODEX_HOME and both environment/config inputs
* can change between invocations. The parent resolves one exact target and hands
* those canonical paths to the Worker rather than asking the Worker to infer a
* those resolved paths to the Worker rather than asking the Worker to infer a
* possibly different environment.
*/
export function resolveCodexHistoryJobTarget(): {
Expand All @@ -45,10 +45,10 @@ export function resolveCodexHistoryJobTarget(): {
return {
canonicalCodexHome: home,
canonicalStateDbPath: stateDb,
// Derived by the provider's own rule rather than guessed: the manifest lives
// in the config directory under a hash of the state database, so a
// hand-built path would address a different file entirely.
canonicalBackupPath: historyBackupPathFor(stateDb),
// Resolve through the provider's canonical-first compatibility rule. Passing
// only the newly normalized name would hide a pre-#4442 Windows manifest from
// the Worker and make an upgrade look like an empty backup.
canonicalBackupPath: resolveExistingHistoryBackupPath(stateDb),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Resolve the manifest only after acquiring the history lock

On affected Windows homes, this existence-based choice is made before the Worker acquires withHistoryWriteSerialization; if another history job removes the legacy manifest or publishes the canonical one while this job waits, the frozen target becomes stale. A restore or migration can then read an empty/stale legacy path and report zero-row convergence even though the canonical manifest still owns pending entries. Resolve or revalidate the canonical/legacy choice inside the locked Worker callback, while still passing enough explicit path context to avoid ambient-environment drift.

Useful? React with 👍 / 👎.

};
}

Expand Down
2 changes: 1 addition & 1 deletion structure/catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ Provider `showThinkingSummary` is a Responses request default; it does not rewri

## Paginated history writer boundary

`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

Codex pool settings and their consumers follow the [reset-first ordering contract](providers/openai-tiers.md#reset-first-account-ordering), including independent-quota fallback and preserved affinity.

Expand Down
2 changes: 1 addition & 1 deletion structure/codex-home.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ Plan-based automatic exclusions leave native credential files untouched and pres

## Paginated history writer boundary

`src/codex/history-provider.ts` rejects provider-history changes with `history_paginated_requires_native_writer` when a target begins with an ordinal-bearing record or declares `history_mode=paginated`. Apply, manifest-backed restore, and explicit legacy recovery preflight all selected targets before changing database rows or manifests. The append boundary checks again. Codex owns ordinal allocation and the live projection cursor; reading the last ordinal and appending N+1 is not safe concurrent coordination. Legacy unnumbered rollouts retain their existing behavior. This guard prevents the observed stable-format corruption; it does not implement native-writer integration or guarantee a concurrent legacy-to-paginated conversion is excluded.
`src/codex/history-provider.ts` rejects provider-history changes with `history_paginated_requires_native_writer` when a target begins with an ordinal-bearing record or declares `history_mode=paginated`. Apply, manifest-backed restore, and explicit legacy recovery preflight all selected targets before changing database rows or manifests. The append boundary checks again. Codex owns ordinal allocation and the live projection cursor; reading the last ordinal and appending N+1 is not safe concurrent coordination. Legacy unnumbered rollouts retain their existing behavior. History Worker targets resolve the canonical manifest first and an existing pre-normalization Windows filename second, so passing an explicit target cannot bypass upgrade recovery. This guard prevents the observed stable-format corruption; it does not implement native-writer integration or guarantee a concurrent legacy-to-paginated conversion is excluded.

Injection preflights affected history using the normalized config candidate before writing config/profile/journal, then checks again after the complete artifact write. Native restore also rechecks after successful journal restoration or fallback removal, while exact config/profile/journal preimages and any coordinated remove transaction remain available for compensation. Detected migration restores all three preimages before returning a structured refusal, including on legacy-uncoordinated homes. A failed config restore stops catalog/history work; coordinated restore rolls back its published remove transition. Legacy first-line provider patches are bound to the validated file identity before and after writing. These compensating checks do not provide a native-writer lock or authorize external ordinal allocation.

Expand Down
3 changes: 2 additions & 1 deletion structure/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,8 @@ that prefix hashed to a different backup filename before the normalization, so t
(`history-provider.ts` for mutation, `native-residue.ts` for observation) fall back to the
legacy filename when no canonical manifest exists. When both names exist the canonical manifest
wins and the legacy file is left in place; a conflict is never resolved by silently replacing
either file.
either file. History Worker job targets use that same canonical-first lookup rather than passing a
canonical-only filename that would bypass the provider's legacy fallback.

`history-provider.ts` remains the strict mutation owner and maps shared validation failures to its
restore/no-op integrity states. `native-residue.ts` remains a read-only observer and maps the same
Expand Down
2 changes: 1 addition & 1 deletion structure/gui-and-management-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,7 @@ The provider editor field policy exposes `showThinkingSummary` as a boolean prov

## Paginated history writer boundary

`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

Codex pool settings and their consumers follow the [reset-first ordering contract](providers/openai-tiers.md#reset-first-account-ordering), including independent-quota fallback and preserved affinity. Codex account DTOs and cards expose the routing-plan exclusion separately from credential health; the [plan exclusion contract](providers/openai-tiers.md#automatic-pool-plan-exclusions) also governs CLI projection. Private pool credential metadata follows the [quota-history publication identity contract](providers/openai-tiers.md#quota-history-publication-identity); credential-only and account DTO projections omit it.

Expand Down
2 changes: 1 addition & 1 deletion structure/ops/docs-and-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,7 @@ Provider configuration documents distinguish actual summaries from raw reasoning

## Paginated history writer boundary

`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](../codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](../codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

Private pool credential metadata follows the [quota-history publication identity contract](../providers/openai-tiers.md#quota-history-publication-identity); credential-only and account DTO projections omit it.

Expand Down
2 changes: 1 addition & 1 deletion structure/providers/openai-tiers.md
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ Listener startup diagnostics follow [the runtime lifecycle contract](../runtime.

`src/codex/auth-api.ts` projects `selectionExcludedReason: "plan_excluded"` and `selectionExcludedPlan` from the routing config, even when a newer display-only WHAM plan could not be persisted. The dashboard and account CLI show the policy reason separately from credential health; renewal clears the derived fields. The automatic next-session action and badge are omitted for excluded rows.
## Paginated history writer boundary
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](../codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](../codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Expand Down
2 changes: 1 addition & 1 deletion structure/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ Responses route normalization resolves provider summary defaults from the origin

## Paginated history writer boundary

`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

Codex pool settings and their consumers follow the [reset-first ordering contract](providers/openai-tiers.md#reset-first-account-ordering), including independent-quota fallback and preserved affinity.

Expand Down
2 changes: 1 addition & 1 deletion structure/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ Final-route summary visibility is recomputed after fallback from the original Re

## Paginated history writer boundary

`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.
`src/codex/history-provider.ts` refuses external writes to paginated or migration-capable history. `src/codex/inject.ts` checks affected rows and manifest-owned restore targets before and after config/profile/journal changes, including successful journal and fallback restores, and compensates detected migration. Failed config restore stops later catalog/history work and rolls back a coordinated remove transition. History Worker targets preserve the canonical-first legacy-manifest fallback when they cross the Worker boundary. See the [history writer contract](codex-home.md#paginated-history-writer-boundary) for guarantees and concurrent-writer limits.

Codex pool settings and their consumers follow the [reset-first ordering contract](providers/openai-tiers.md#reset-first-account-ordering), including independent-quota fallback and preserved affinity.

Expand Down
29 changes: 26 additions & 3 deletions tests/codex-integration/codex-sqlite-home.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,22 @@ import { tmpdir } from "node:os";
import { join, resolve } from "node:path";

import { resolveCodexHistoryJobTarget } from "../../src/codex/history-job";
import { historyBackupPathFor } from "../../src/codex/history-provider";
import { legacyHistoryBackupPathFor, resolveExistingHistoryBackupPath } from "../../src/codex/history-provider";
import { resolveCodexLogsDbPath, resolveCodexSqliteHome, resolveCodexStateDbPath } from "../../src/codex/paths";
import { removeTreeWithRetry } from "../helpers/remove-tree";

const originalCodexHome = process.env.CODEX_HOME;
const originalSqliteHome = process.env.CODEX_SQLITE_HOME;
const originalOpenCodexHome = process.env.OPENCODEX_HOME;
const roots: string[] = [];

afterEach(() => {
if (originalCodexHome === undefined) delete process.env.CODEX_HOME;
else process.env.CODEX_HOME = originalCodexHome;
if (originalSqliteHome === undefined) delete process.env.CODEX_SQLITE_HOME;
else process.env.CODEX_SQLITE_HOME = originalSqliteHome;
if (originalOpenCodexHome === undefined) delete process.env.OPENCODEX_HOME;
else process.env.OPENCODEX_HOME = originalOpenCodexHome;
for (const root of roots.splice(0)) removeTreeWithRetry(root);
});

Expand Down Expand Up @@ -108,11 +111,31 @@ describe("Codex SQLite home resolution", () => {

const configured = resolveCodexHistoryJobTarget();
expect(configured.canonicalStateDbPath).toBe(join(configSqliteHome, "state_5.sqlite"));
expect(configured.canonicalBackupPath).toBe(historyBackupPathFor(configured.canonicalStateDbPath));
expect(configured.canonicalBackupPath).toBe(resolveExistingHistoryBackupPath(configured.canonicalStateDbPath));

unlinkSync(join(codexHome, "config.toml"));
const fromEnv = resolveCodexHistoryJobTarget();
expect(fromEnv.canonicalStateDbPath).toBe(join(envSqliteHome, "state_5.sqlite"));
expect(fromEnv.canonicalBackupPath).toBe(historyBackupPathFor(fromEnv.canonicalStateDbPath));
expect(fromEnv.canonicalBackupPath).toBe(resolveExistingHistoryBackupPath(fromEnv.canonicalStateDbPath));
});

test.skipIf(process.platform !== "win32")("history jobs preserve a pre-normalization extended-path manifest", () => {
const root = mkdtempSync(join(tmpdir(), "ocx-sqlite-home-legacy-"));
roots.push(root);
const codexHome = join(root, "codex");
const configHome = join(root, "opencodex");
const sqliteHome = join(root, "sqlite");
mkdirSync(codexHome);
mkdirSync(configHome);
mkdirSync(sqliteHome);
process.env.CODEX_HOME = codexHome;
process.env.OPENCODEX_HOME = configHome;
process.env.CODEX_SQLITE_HOME = `\\\\?\\${sqliteHome}`;

const stateDb = resolveCodexStateDbPath({ codexHome });
const legacyBackup = legacyHistoryBackupPathFor(stateDb);
writeFileSync(legacyBackup, "{}\n");

expect(resolveCodexHistoryJobTarget().canonicalBackupPath).toBe(legacyBackup);
});
});
Loading