fix(codex): share resume sessions across benchmark planning and execution - #5527
Conversation
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=xhigh
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
精确 head:fc42b84cedb21a56c15ca3b4af5eab0407829cf8。无阻塞发现;本次维护者已明确授权合并此 runtime/benchmark PR。
动机
同一 Agent 连续处理多个任务时,规划会话与执行会话分离,每次换 Todo 又丢失已有对话上下文。
在真实 CLI 的两个 Todo 对照中,旧版启动两个会话;新版第二个 Todo 续接原会话,两个 Todo 仍分别校验、分别结算。
规划、心跳与 Codex exec 可保持同一原生会话 ID;失配时拒绝静默换会话,显式 fresh 可恢复。
本次不调整 benchmark 评分、预算或任务语义,不扩大工具权限,也不改变 managed operation 与其他宿主的 Todo 会话边界。
改动思路
复用现有 Codex 原生会话描述符,而不是给 benchmark 另建状态。规划与心跳先观察原生 ID,普通 exec 再按 Goal/Agent 读取同一绑定。会话存储只承载上下文,不承担任务选择、权限、租约、验证或结算。每次执行仍采用当前 Todo 的签名输入与独立校验。managed operation 的 app-server、Chat/Lark 审批及其他宿主保留原 Todo 合同;Python 位于已有 provider 传输边界,类型化效果 owner 没有搬迁。
具体改动
基准为 e9e0f1bb30dc29260799f07e73dab69ab8f31e95 的 docs/reference/protocols/loopx-turn-v0.md;按接受前的合同核对 TurnEnvelope 独立校验/写回链和 Session eligibility 只作恢复元数据两项,均有实际 CLI 证据。上下文名与 Codex exec 范围是本次明确披露的有意变化,不用新文档反证旧合同。
关键代码讲解
_session_plan通过SessionBindingScope比较会话身份;只改变上下文 key,交易 key 与 Todo 结算身份保持独立。select_codex_cli_session复用 Goal lifetime fence;agent 文件损坏时拒绝静默新建。codex_session_profile_digest将工作区、home/config、可执行文件、模型、effort、sandbox 与 MCP 绑定到会话;失配要求显式 fresh。BenchmarkSessionWake被现有 planning/heartbeat worker 实际调用;stdout 必须确认一个原生 ID,timeout 保留 ID 但不声明成功,意外 fork 不替换已保存绑定。- CLI parser、run-once 与 journal retry 投影同一 scope;operation host/handoff 改用抽出的既有存储 owner,原审批 key 与锁顺序保留。Ark/DSH、Chat/Lark 测试随规范名更新;benchmark README/RUNTIME、Turn 与 DSH 文档披露上下文选择。
修复补齐了共享会话构造边界的类型标注,并整合已合入的两处摘要消费清单修复。旧 Todo 文件保持原 key;不会自动导入 agent scope。CLI 默认从 resume-if-available 改为 resume,Codex exec 从 Todo 改为 Agent;benchmark 默认仍为 fresh。旧显式名称需更新;需要隔离可使用 --session-scope todo 或 --iteration-context fresh。
对主干的风险
主要风险是把旧 Todo 完成或审批当作下一任务的授权。实际 CLI 对照证明两个 Todo 有不同 turn key/写回,只有新版第二次续接上下文;profile、Goal lifetime、损坏文件、另一个 Goal/Agent、超时及意外原生 ID 均有反例。managed operation 与 Chat/Lark 的原审批及接收者边界保留。历史上下文可能影响独立题目,fresh/todo 提供明确隔离;不宣称 benchmark 得分或付费模型质量改善。
父提交定向测试 278 项通过、15 项按平台/在线条件跳过;原生用例初次并行运行在未改动的进程清理路径出现 PermissionError,保留失败记录;隔离复跑通过,最终 head 的 10 项续接测试全部通过,包含实际 Codex CLI 0.159.2 与本地脚本化 Responses 端点的历史读回。四文件 mypy、Ruff、diff、公私边界、语义 advisory 与完整语义检查通过。用户已接受风险验证,不等待完整 CI 矩阵;这里不把跳过或合成模型响应写成在线质量通过。
语义与 CI 对齐
复用现有 Turn context 合同并规范化 resume 值,新增 provider 局部 scope 枚举;精确相等匹配,没有子串规则。错误与恢复是机器执行约束,文档明确其后果。benchmark-sensitive 自动分类仍要求维护者决定;本次明确授权覆盖这一合并决定,评分、预算、数据与 benchmark job 均未改变。
我的整体评价
APPROVE。long_horizon 和 user_experience 均改善:跨任务保留历史,失败恢复可解释,独立结果仍按 Todo 验证。范围与问题匹配;面向后续改动的简化已通过抽出一个存储 owner 落地,进一步框架化无必要。exact-scope CQS 已验证有效;风险 quick premerge 的直接检查通过,保留 benchmark-sensitive 维护者 hold,并由本次明确授权作合并决定;合并立即前复核 head/readiness。本次授权是此 PR 的维护者合并决定。
English verdict: APPROVE — fc42b84cedb21a56c15ca3b4af5eab0407829cf8. Shared Codex exec continuity preserves independent Todo validation and settlement. Real CLI baseline/head comparison, native loopback history, final-head 10 tests, scoped static/semantic checks passed; gated tests and the initial cleanup failure remain disclosed. Maintainer explicitly authorizes this merge and risk-based release qualification.
Goal And Delivered Outcome
Benchmark resume previously restarted heartbeat/planning execs and lost continuity when a Turn selected a new Todo. Planning, heartbeat and ordinary Codex Turn execution now share one Goal/Agent conversation and resume its exact native ID.
main.Author Declaration
docs/reference/protocols/loopx-turn-v0.md#cross-iteration-context-policy. The engineering boundary follows the existing long-horizon harness research RFC §11.benchmark/tests/test_resume_sessions.py, two-Todo public Turn CLI integrationresume, with no old-name alias; ordinary Codex exec defaults to agent scopeThe scope vocabulary is local to conversation binding, distinct from settlement binding. The context values extend the existing Turn policy owner. Python remains the existing Codex transport/persistence adapter; Goal and settlement decisions retain their existing typed owners. Extracting persistence from the oversized exec adapter gives both runner paths one store and updates its active private imports.
Scope And Continuation
Complete within this scope: heartbeat/Turn
resumeincludes planning and cross-Todo execution, timeout ID retention and fail-closed reuse.freshremains the benchmark default. Managed operation-equipped app-server sessions and other providers retain their existing Todo/approval contracts.This qualifies the runtime seam, not matched-budget quality gains. No scored benchmark jobs or leaderboard submissions were launched; E3/E4 research acceptance remains separate. No successor is needed for this request.
Validation
438af594b402f116abab83874ced5eec6b98e671.benchmark_sensitivemaintainer-review hold; no self-merge. Public/private scan excludes raw trajectories, credentials, local paths and generated state.Coverage includes the actual native exec transport and public CLI settlement path. Production provider behavior, online operation release qualification and scored benchmark efficacy were not claimed.
Frontend / Visual Evidence