Skip to content

fix(integration): install codex CLI + isolate codex auth from the deepseek judge#808

Merged
Yiminnn merged 1 commit into
mainfrom
fix/integration-codex-install-auth
Jun 19, 2026
Merged

fix(integration): install codex CLI + isolate codex auth from the deepseek judge#808
Yiminnn merged 1 commit into
mainfrom
fix/integration-codex-install-auth

Conversation

@Yiminnn

@Yiminnn Yiminnn commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Third follow-up to #806/#807 to get the full L3 review green. The first full run on #803 reached review-pack with plan ✓ + all 10 rollouts ✓ + deterministic grader ✓ — only codex failed.

Root causes

  1. codex binary not found: 'codex' — the review-pack job never installed the codex CLI.
  2. codex auth clobbered to DeepSeekprovider-select exports OPENAI_*=DeepSeek for the cheap Pass-1 judge; codex_review.py then wrote the codex auth.json from that key and ran codex exec against the DeepSeek base.

Fix

  • Install @openai/codex@0.141.0 in both review-pack jobs (L2 advisory, L3 required), continue-on-error so a failed install still hits codex's fail-closed verdict.
  • _codex_env in codex_review.py isolates the host codex exec: real OpenAI key via CODEX_API_KEY + DeepSeek OPENAI_BASE_URL dropped (default OpenAI endpoint). The Pass-1 deepseek judge keeps the original env. Workflows pass CODEX_API_KEY: secrets.OPENAI_API_KEY (L2 also drops its OPENAI_API_KEY override, which paired the real key with the deepseek base).

Test plan

…eepseek judge

The first full L3 run on #803 reached review-pack but failed: "codex binary not
found" (the codex CLI was never installed) and, underneath, the codex auth was
clobbered to DeepSeek by the per-rollout judge's OPENAI_* exports.

- Install `@openai/codex@0.141.0` in both review-pack jobs (L2 advisory, L3
  required) before the codex step; continue-on-error so a failed install still
  reaches the codex step's fail-closed verdict path.
- codex_review.py: new `_codex_env` isolates the host `codex exec` (Pass 2) — it
  uses the REAL OpenAI key via CODEX_API_KEY and drops the DeepSeek OPENAI_BASE_URL
  (default OpenAI endpoint), while the Pass-1 deepseek judge keeps the original
  env. Workflows pass `CODEX_API_KEY: ${{ secrets.OPENAI_API_KEY }}` (L2 also drops
  its OPENAI_API_KEY override, which had paired the real key with the deepseek base).

Plan ✓, all 10 rollouts ✓, deterministic grader ✓ on the prior run; this closes
the last gap (codex). Regression test added for _codex_env.
@Yiminnn Yiminnn temporarily deployed to pypi-internal-preview June 18, 2026 23:54 — with GitHub Actions Inactive
@greptile-apps

greptile-apps Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes two root causes that prevented codex from completing during L3 review-pack runs: the codex CLI was never installed, and the OPENAI_* environment variables were being clobbered by the DeepSeek judge config before codex could authenticate.

  • Adds npm install -g @openai/codex@0.141.0 (with continue-on-error) to both L2 and L3 review-pack jobs so the binary is available when codex_review.py tries to invoke it.
  • Introduces _codex_env() in codex_review.py to isolate the codex CLI's env: when CODEX_API_KEY is set it becomes OPENAI_API_KEY and OPENAI_BASE_URL is dropped, preventing the DeepSeek judge clobber from leaking into codex auth; write_codex_auth, has_codex_auth, and run_codex_verdict are all updated to use this isolated env.
  • Both workflow files switch from OPENAI_API_KEY: secrets.OPENAI_API_KEY to CODEX_API_KEY: secrets.OPENAI_API_KEY on the codex step, leaving Pass-1 DeepSeek variables intact in the job env.

Confidence Score: 4/5

Safe to merge; the auth-isolation logic is correct and well-tested, and both workflow files apply the fix consistently.

The _codex_env function correctly overwrites OPENAI_API_KEY and drops OPENAI_BASE_URL before spawning codex, but it leaves CODEX_API_KEY in the subprocess environment. Today's codex CLI ignores that variable, but it is a non-standard name that a future codex release might interpret, making the subprocess env slightly leakier than the doc-comment implies. Everything else — auth writing, auth checking, the fail-closed paths, and both workflow files — looks correct and consistent.

_codex_env in .github/scripts/codex_review.py — the CODEX_API_KEY key is not stripped before the env dict is handed to the codex subprocess.

Important Files Changed

Filename Overview
.github/scripts/codex_review.py Adds _codex_env() to isolate codex CLI auth from the DeepSeek judge env; main() now uses codex_env for write_codex_auth, has_codex_auth, and run_codex_verdict
.github/workflows/integration-final-review.yml Adds npm install -g @openai/codex@0.141.0 (continue-on-error) before the codex step; switches from no explicit CODEX_API_KEY to CODEX_API_KEY: secrets.OPENAI_API_KEY
.github/workflows/integration-scope.yml Adds codex CLI install step; replaces OPENAI_API_KEY: secrets.OPENAI_API_KEY with CODEX_API_KEY: secrets.OPENAI_API_KEY so the Pass-1 judge keeps the DeepSeek env while codex gets the real OpenAI key
tests/test_codex_review.py Adds two unit tests for _codex_env: one verifying the DeepSeek-isolation path and one for the backward-compatible no-CODEX_API_KEY path

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant WF as GitHub Actions Job
    participant PS as provider-select
    participant CR as codex_review.py
    participant DS as DeepSeek Judge (Pass 1)
    participant CX as codex exec (Pass 2)

    WF->>PS: run provider-select
    PS-->>WF: "writes OPENAI_API_KEY=ds-key,<br/>OPENAI_BASE_URL=deepseek to $GITHUB_ENV"

    WF->>WF: "inject CODEX_API_KEY=secrets.OPENAI_API_KEY (step env)"

    WF->>CR: "uv run codex_review.py (env includes DeepSeek OPENAI_*<br/>+ CODEX_API_KEY=real-key)"

    CR->>DS: "run_deepseek_findings(env)<br/>(uses original env — DeepSeek OPENAI_API_KEY + BASE_URL)"
    DS-->>CR: per-rollout findings

    CR->>CR: "codex_env = _codex_env(env)<br/>→ OPENAI_API_KEY=real-key, OPENAI_BASE_URL dropped"
    CR->>CR: "write_codex_auth(codex_env)<br/>→ auth.json with real OpenAI key"
    CR->>CX: "subprocess.run(codex exec, env=codex_env)"
    CX-->>CR: verdict output
    CR-->>WF: final_verdict
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant WF as GitHub Actions Job
    participant PS as provider-select
    participant CR as codex_review.py
    participant DS as DeepSeek Judge (Pass 1)
    participant CX as codex exec (Pass 2)

    WF->>PS: run provider-select
    PS-->>WF: "writes OPENAI_API_KEY=ds-key,<br/>OPENAI_BASE_URL=deepseek to $GITHUB_ENV"

    WF->>WF: "inject CODEX_API_KEY=secrets.OPENAI_API_KEY (step env)"

    WF->>CR: "uv run codex_review.py (env includes DeepSeek OPENAI_*<br/>+ CODEX_API_KEY=real-key)"

    CR->>DS: "run_deepseek_findings(env)<br/>(uses original env — DeepSeek OPENAI_API_KEY + BASE_URL)"
    DS-->>CR: per-rollout findings

    CR->>CR: "codex_env = _codex_env(env)<br/>→ OPENAI_API_KEY=real-key, OPENAI_BASE_URL dropped"
    CR->>CR: "write_codex_auth(codex_env)<br/>→ auth.json with real OpenAI key"
    CR->>CX: "subprocess.run(codex exec, env=codex_env)"
    CX-->>CR: verdict output
    CR-->>WF: final_verdict
Loading

Reviews (1): Last reviewed commit: "fix(integration): install the codex CLI ..." | Re-trigger Greptile

Comment on lines +276 to +281
out = dict(env)
codex_key = env.get("CODEX_API_KEY")
if codex_key:
out["OPENAI_API_KEY"] = codex_key
out.pop("OPENAI_BASE_URL", None)
return out

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 CODEX_API_KEY is not removed from the returned dict, so it is passed verbatim to the codex exec subprocess via run_codex_verdict. The current @openai/codex CLI relies on OPENAI_API_KEY and ignores CODEX_API_KEY, but if a future codex version interprets CODEX_API_KEY — which is a plausible variable name to adopt — the duplicate entry could override intended behaviour. Popping it keeps the subprocess env clean and consistent with the doc-comment intent of "isolated from the judge".

Suggested change
out = dict(env)
codex_key = env.get("CODEX_API_KEY")
if codex_key:
out["OPENAI_API_KEY"] = codex_key
out.pop("OPENAI_BASE_URL", None)
return out
out = dict(env)
codex_key = env.get("CODEX_API_KEY")
if codex_key:
out["OPENAI_API_KEY"] = codex_key
out.pop("OPENAI_BASE_URL", None)
out.pop("CODEX_API_KEY", None)
return out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant