Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/guides/codex-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -807,6 +807,8 @@ Catalog sync makes the selected sub-agent models available to Codex; see [Codex

When a ChatGPT account is added or reauthenticated, OpenCodex normally verifies it before saving with a small streaming request to the Codex Responses backend. It waits for `response.completed`, defaults to `gpt-5.6-luna`, and retries with `gpt-5.5` on HTTP 400 or HTTP 404. Public errors contain fixed failure categories rather than raw upstream response bodies.

An HTTP 429 from an attempted warmup is reported as `codex_warmup_rate_limited`. Retry after the temporary restriction clears or the usage limit resets; signing in again does not reset these limits. A failed attempted warmup does not add a new account or replace existing credentials. This differs from quota-confirmed deferred registration below, which can save a restricted account without a model request. HTTP 401/403 failures retain the authentication guidance.

If the new OAuth credential's authenticated usage lookup confirms an exhausted 5-hour, weekly, or monthly quota, the account is saved without this model request and shows **Validation pending**. It cannot serve pool requests, even after a restart or token refresh. Once quota recovers, **Refresh quotas** finishes validation: a fresh, complete usage reading with headroom permits one small model request, and only a completed response enables the account. Failed or incomplete readings and failed validation preserve the restriction. Passive account polling does not trigger deferred validation. Unknown usage during initial registration retains the normal warmup gate.

`ocx account refresh openai` and `ocx account list openai --quota --refresh` only read usage. Model validation spends quota and requires a human dashboard session: open `ocx gui` and click **Refresh quotas** after recovery. For a headless host, access its dashboard from your browser; an admin token alone does not authorize validation. Validation can complete while an account is paused without resuming or selecting it. Model authorization failures remain visible until successful validation or reauthentication clears them.
Expand Down
16 changes: 14 additions & 2 deletions src/codex/auth-api/login-flow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { clearAccountNeedsReauth, isAccountNeedsReauth, markAccountNeedsReauth }
import { clearCodexPoolRefreshFailure } from "../pool-refresh-backoff";
import { reconcileLiveStateStores } from "../../lib/state-store-registrations";
import { emailMaskingEnabled, projectEmail } from "../../lib/privacy";
import { codexWarmupFailureReason, isCodexWarmupProvisioningFailure, warmCodexAccount } from "../warmup";
import { CodexWarmupError, codexWarmupFailureReason, isCodexWarmupProvisioningFailure, warmCodexAccount } from "../warmup";
import type { CodexAccount, CodexAccountCredentials, OcxConfig } from "../../types";
import type { CatalogDisposition } from "../convergence-types";
import { isValidCodexAccountId } from "../account-id";
Expand Down Expand Up @@ -51,6 +51,17 @@ export async function verifyCodexAccountWarmup(
return { ok: true, validatedAt: Date.now() };
} catch (err) {
const reason = codexWarmupFailureReason(err);
if (err instanceof CodexWarmupError && err.code === "http_status" && err.status === 429) {
return {
ok: false,
response: jsonResponse({
error: "Codex account warmup was rate limited. Retry later or after the account's usage limit resets.",
code: "codex_warmup_rate_limited",
reason,
accountId,
}, 429),
};
}
return {
ok: false,
response: jsonResponse({
Expand Down Expand Up @@ -322,10 +333,11 @@ export async function handleCodexAuthLoginStart(req: Request, config: OcxConfig,
? { ok: true as const, validatedAt: undefined }
: await verifyCodexAccountWarmup(accountId, cred.access, oauthAccountId);
if (!warmup.ok) {
const body = await warmup.response.json().catch(() => ({})) as { error?: string; reason?: string };
const body = await warmup.response.json().catch(() => ({})) as { error?: string; code?: string; reason?: string };
setCodexLoginState(flowId, {
status: "error",
error: body.reason ? `${body.error ?? "Codex account warmup failed"} (${body.reason})` : body.error ?? "Codex account warmup failed",
code: body.code,
doneAt: Date.now(),
});
completed = true;
Expand Down
2 changes: 1 addition & 1 deletion src/codex/warmup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ async function drainErrorBody(res: Response, signal: AbortSignal): Promise<void>
fatalUtf8: true,
});
} catch (error) {
if (signal.aborted) {
if (signal.aborted && res.status !== 429) {
throw new CodexWarmupError("transport", "Codex warmup request failed", {
cause: error,
});
Expand Down
2 changes: 2 additions & 0 deletions structure/providers/openai-tiers.md
Original file line number Diff line number Diff line change
Expand Up @@ -567,3 +567,5 @@ Two call sites need the rule — the live path in `reevaluateAffinityQuota` and
`previewReusableAffinityAccount` that subagent fallback reads — and they share one helper rather
than restating it, because the suite asserts the two answer identically and a preview that
disagreed would hand fallback a different account than the request actually uses.

`src/codex/auth-api/login-flow.ts` distinguishes HTTP 429 from an attempted warmup as `codex_warmup_rate_limited` and preserves that code in OAuth status. Failed attempted warmup does not persist replacement credentials; quota-confirmed deferred registration and HTTP 401/403 handling remain separate. `src/codex/warmup.ts` retains a known 429 when bounded error-body draining times out.
23 changes: 2 additions & 21 deletions tests/codex-integration/codex-auth-api.test.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { registerWarmupRateLimitCases } from "../helpers/codex-warmup-rate-limit";
import * as usageHistoryModule from "../../src/usage/log";
import { getAccountQuotaHistory } from "../../src/codex/quota";
import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test";
Expand Down Expand Up @@ -5530,27 +5531,7 @@ describe("codex-auth API", () => {
expect(getCodexAccountCredential("quota-unknown")).toBeNull();
});

test("OAuth creation rejects a namespace claimed during warmup without persisting", async () => {
const config = makeConfig();
const result = await completeMockCodexOAuth({
config,
requestBody: { id: "oauth-race" },
oauthAccountId: "acct-oauth-race",
email: "oauth-race@example.test",
onWarmup: () => {
config.codexAccountNamespaces = { "oauth-race": "pool-a" };
},
});

expect(result.startStatus).toBe(200);
expect(result.state).toMatchObject({
status: "error",
error: "account id must not collide with a configured Codex account namespace",
});
expect(config.codexAccounts).toEqual([]);
expect(config.codexAccountNamespaces).toEqual({ "oauth-race": "pool-a" });
expect(getCodexAccountCredential("oauth-race")).toBeNull();
});
registerWarmupRateLimitCases(makeConfig, completeMockCodexOAuth);

test("OAuth creation reports a durable add when catalog convergence is pending", async () => {
const accountId = "oauth-picker-pending";
Expand Down
46 changes: 45 additions & 1 deletion tests/codex-integration/codex-warmup.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { afterEach, describe, expect, test } from "bun:test";
import { afterEach, describe, expect, spyOn, test } from "bun:test";
import { CodexWarmupError, warmCodexAccount } from "../../src/codex/warmup";

const originalFetch = globalThis.fetch;
Expand Down Expand Up @@ -152,6 +152,50 @@ describe("codex warmup", () => {
expect(performance.now() - startedAt).toBeLessThan(1_000);
});

test("preserves HTTP 429 classification when the error body stalls until the deadline", async () => {
let fetchCalls = 0;
let cancellations = 0;
const privateBody = "private upstream quota details";
const fetchSpy = spyOn(globalThis, "fetch").mockImplementation(async () => {
fetchCalls += 1;
const stalledBody = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new TextEncoder().encode(privateBody));
},
cancel() {
cancellations += 1;
return new Promise<void>(() => {});
},
});
return new Response(stalledBody, { status: 429 });
});

const startedAt = performance.now();
try {
let failure: unknown;
try {
await warmCodexAccount({
accessToken: "a",
chatgptAccountId: "c",
timeoutMs: 20,
});
} catch (error) {
failure = error;
}

expect(failure).toBeInstanceOf(CodexWarmupError);
if (!(failure instanceof CodexWarmupError)) throw new Error("expected CodexWarmupError");
expect(failure.code).toBe("http_status");
expect(failure.status).toBe(429);
expect(failure.message).not.toContain(privateBody);
expect(fetchCalls).toBe(1);
expect(cancellations).toBe(1);
expect(performance.now() - startedAt).toBeLessThan(1_000);
} finally {
fetchSpy.mockRestore();
}
});

test("accepts a completed SSE stream at the exact byte limit", async () => {
const encoder = new TextEncoder();
const terminal = 'data: {"type":"response.completed"}\n\n';
Expand Down
126 changes: 126 additions & 0 deletions tests/helpers/codex-warmup-rate-limit.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
import { expect, test } from "bun:test";
import type { OcxConfig } from "../../src/types";
import { getCodexAccountCredential, readCodexAccountRecord, saveCodexAccountCredential } from "../../src/codex/account-store";

interface WarmupOAuthOptions {
config: OcxConfig;
requestBody: { id: string; reauth?: boolean };
oauthAccountId: string;
email: string;
onWarmup: () => void;
warmupResponse?: () => Response;
}

/** Registers under the calling suite's isolated home and OAuth cleanup hooks. */
export function registerWarmupRateLimitCases(
makeConfig: (overrides?: Partial<OcxConfig>) => OcxConfig,
completeMockCodexOAuth: (options: WarmupOAuthOptions) => Promise<{
startStatus: number;
state: { status: string; error?: string; code?: string };
}>,
): void {
test("OAuth creation reports a rate-limited warmup without persisting the account", async () => {
const accountId = "warmup-rate-limited";
const config = makeConfig();
let warmupRequests = 0;

const result = await completeMockCodexOAuth({
config,
requestBody: { id: accountId },
oauthAccountId: "acct-warmup-rate-limited",
email: "warmup-rate-limited@example.test",
onWarmup: () => { warmupRequests += 1; },
warmupResponse: () => new Response("private upstream quota details", { status: 429 }),
});

expect(result.startStatus).toBe(200);
expect(result.state).toMatchObject({
status: "error",
code: "codex_warmup_rate_limited",
});
expect(result.state.error).toContain("usage limit");
expect(result.state.error).toContain("Retry");
expect(JSON.stringify(result.state)).not.toContain("private upstream quota details");
expect(warmupRequests).toBe(1);
expect(config.codexAccounts).toEqual([]);
expect(getCodexAccountCredential(accountId)).toBeNull();
expect(readCodexAccountRecord(accountId)).toBeNull();
});

test.each([401, 403])("OAuth creation keeps HTTP %s warmup failures on the authentication path", async status => {
const accountId = `warmup-auth-${status}`;
const config = makeConfig();

const result = await completeMockCodexOAuth({
config,
requestBody: { id: accountId },
oauthAccountId: `acct-warmup-auth-${status}`,
email: `warmup-auth-${status}@example.test`,
onWarmup: () => {},
warmupResponse: () => new Response("private upstream auth details", { status }),
});

expect(result.state).toMatchObject({
status: "error",
code: "codex_warmup_failed",
});
expect(result.state.error).toContain("Reauthenticate");
expect(JSON.stringify(result.state)).not.toContain("private upstream auth details");
expect(config.codexAccounts).toEqual([]);
expect(getCodexAccountCredential(accountId)).toBeNull();
});

test("OAuth reauth keeps the existing credential when warmup is rate limited", async () => {
const accountId = "warmup-rate-limited-reauth";
const config = makeConfig({
codexAccounts: [{ id: accountId, email: "existing@example.test", isMain: false }],
});
const existingCredential = {
accessToken: "existing-access",
refreshToken: "existing-refresh",
expiresAt: Date.now() + 60_000,
chatgptAccountId: "acct-warmup-rate-limited-reauth",
};
saveCodexAccountCredential(accountId, existingCredential);

const result = await completeMockCodexOAuth({
config,
requestBody: { id: accountId, reauth: true },
oauthAccountId: existingCredential.chatgptAccountId,
email: "existing@example.test",
onWarmup: () => {},
warmupResponse: () => new Response("private upstream quota details", { status: 429 }),
});

expect(result.state).toMatchObject({
status: "error",
code: "codex_warmup_rate_limited",
});
expect(getCodexAccountCredential(accountId)).toEqual(existingCredential);
expect(config.codexAccounts).toEqual([
{ id: accountId, email: "existing@example.test", isMain: false },
]);
});

test("OAuth creation rejects a namespace claimed during warmup without persisting", async () => {
const config = makeConfig();
const result = await completeMockCodexOAuth({
config,
requestBody: { id: "oauth-race" },
oauthAccountId: "acct-oauth-race",
email: "oauth-race@example.test",
onWarmup: () => {
config.codexAccountNamespaces = { "oauth-race": "pool-a" };
},
});

expect(result.startStatus).toBe(200);
expect(result.state).toMatchObject({
status: "error",
error: "account id must not collide with a configured Codex account namespace",
});
expect(config.codexAccounts).toEqual([]);
expect(config.codexAccountNamespaces).toEqual({ "oauth-race": "pool-a" });
expect(getCodexAccountCredential("oauth-race")).toBeNull();
});
}
Loading