Leaderboard v1: Codex / GPT-6 Astra

[c-suvamsa]

Summary

Adds one v1 leaderboard row for Codex / GPT-6 Astra (OpenAI), from the public Harbor job for run 2026-10-15__01-38-24.

• Agent / Model: Codex / GPT-6 Astra (OpenAI)
• Source Harbor job URL: https://hub.harborframework.com/jobs/60465aed-d0c4-4649-9506-bbb6ff4496b2

Run summary: 106/140 trials passed = 75.7% accuracy. 14 tasks × 10 attempts = 140 trials, run at concurrency 4, on tasks/v1 via Daytona (DaytonaEnvironment) with v1 shared MCP (.daytona/v1-mcp.json). Provider: OpenAI API (gpt-6-astra). Total 40,942,142 tokens; cost ~$123.29.

PR Checklist — Leaderboard Row Entry

Provenance & access

• Source Harbor job is PUBLIC (public URL opens unauthenticated; job name 2026-10-15__01-38-24 matches). Note: the results/trajectory table on the job page is client-side rendered, so a plain HTTP read could not fully render the per-trial trajectory viewer; reviewers should confirm trajectory replay in-browser.
• Submitter owns the job / job was uploaded via harbor upload
• Row entry YAML links to the public job URL (hub.harborframework.com/jobs/60465aed-d0c4-4649-9506-bbb6ff4496b2)
• Job ID in the header comment matches the public job

Row file conforms to the template

• Conforms to leaderboard/row-template.yaml; no unknown keys
• Filename follows convention: 2026-10-15__codex__gpt-6-astra.yaml
• All required metrics present: accuracy, display_accuracy, token breakdown (uncached_input, cached_input, output, total), avg_trial_duration_sec, pass_at_2/3/4/5/8/10, n_trials, total_cost_usd, display_total_cost_usd
• metadata block complete: agent_display_name, model_display_name, agent_org_display_name, model_org_display_name
• n_trials == number of trial_ids listed (140 == 140)
• trial_ids are unique — no duplicates within file or across existing main rows

Metrics integrity

• total_tokens reconciles: uncached (6,178,441) + cached (34,533,057) + output (230,644) = 40,942,142 = total_tokens
• display_accuracy ("75.7%") matches accuracy (75.71)
• display_total_cost_usd ("$123.29") matches total_cost_usd (123.29)
• status is display

Public job contents (each trial)

• Per-trial trajectory present in the local job (ATIF-format trajectories)
• Reward/result file per trial (reward 0.0/1.0; 106 passed, 34 failed)
• Token usage present: uncached input, cached, output
• Provider / model / agent surfaced on the job (codex / gpt-6-astra / OpenAI)

Run conditions (comparability)

• Timeout multipliers at default: timeout_multiplier=1.0, agent_timeout_multiplier=null, verifier_timeout_multiplier=null
• Setup/build timeout multipliers unchanged (null)
• No CPU / memory / storage overrides (Daytona defaults)
• Agent did not access benchmark site/repo or answers during the run
• Dataset = tasks/v1 (14 tasks); the entry is under leaderboard/entries/ on main.

Review

• Reviewer can independently re-verify: trajectories replay and reward matches submitted score

Validation

• Entry YAML validated against the main leaderboard schema and entries
• validate-leaderboard CI passes for this entry under leaderboard/entries/.

[nimit2801]

Thanks for the submission. Nimit’s intake workflow has picked up this v1 leaderboard entry for Codex / GPT-6 Astra. Review should verify public Harbor trace provenance, reconcile trial scores/pass@k and token totals, and check timeout comparability and trajectory integrity. CI is green, but the technical audit is still required; this is an acknowledgement, not approval.