Skip to content

test(dsh): verify observer with Cordis sessions - #5767

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
mikamikasuki:codex/loopx-5211-cordis-conformance
Oct 6, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
mikamikasuki:codex/loopx-5211-cordis-conformance

Conversation

@mikamikasuki

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

  • Outcome basis / optional anchor: Existing public task [Task][RFC]: Qualify observer-first reliability diagnostics and governed delivery #5211.
  • Goal/source and gap: The observer tests registered fake session/context objects, so they did not exercise Cordis disposal or native DSH session event publication. The task's public follow-up identifies real Cordis and DSH synthetic publication as the next bounded conformance slice.
  • Observable before → after, with the validation row that proves it: Before, the suite did not cover a live Cordis Context with DSH SessionStore lifecycle events. After, the new integration test creates a session, appends a synthetic user message, disposes the session store and context, then verifies the flushed ledger envelopes and counters; see the integration and unit validation rows.
  • Issue/task and intended base: Related to [Task][RFC]: Qualify observer-first reliability diagnostics and governed delivery #5211; base main at 3596467d6e5b3bbb7fc32603a1841e9170b7c4f6.

Author Declaration

  • Written by: model_agent — GPT-6 (OpenAI)

Implemented against

Criterion (spec clause) Disposition Symbol / path Test or command
#5211 follow-up: exercise real Cordis lifecycle and native DSH synthetic event publication implemented packages/dsh-loopx-plugin/tests/observer.spec.ts pnpm --dir packages/dsh-loopx-plugin test
RFC §12: deterministic fixture for a real harness event source; this PR adds the bounded DSH session API slice only implemented packages/dsh-loopx-plugin/tests/observer.spec.ts pnpm --dir packages/dsh-loopx-plugin test
  • Self-check before submission: Reviewed the single-file diff, task follow-up and RFC scope. Ran the package test suite and all three package TypeScript checks. This synthetic test does not claim a live DSH run or P0 qualification.

Scope And Continuation

Validation

  • Tested revision: a7056166c3218eb85ecc781817c235d63389df24
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
static passed tsc -p tsconfig.host.json --noEmit, tsc -p tsconfig.client.json --noEmit, and tsc -p tsconfig.tests.json passed.
unit passed pnpm --dir packages/dsh-loopx-plugin test: 11 files, 207 tests passed.
integration passed The new test uses actual Cordis Context and DSH SessionStore APIs with synthetic session/message data, then verifies three ledger event envelopes and zero rejected events/failures. It is not a live DSH runner test.
manual passed git diff --check passed; the final diff contains only the focused observer test.
  • Coverage and gaps: Does not qualify a live DSH run, C0/C1, observer overhead, P0 exit, deployment or retention policy. No production code or runtime behavior changes.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • States and viewports shown: N/A
  • Source data: synthetic
  • Attention review: No user interface changes.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)

  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)

  • Capability or extension (providers, adapters, skills)

  • Public docs or presentation surface (README, protocols, dashboard)

  • Build, packaging, installer, or CI

  • Host or runtime integration

  • Direction / acceptance reference, when applicable: Reliability diagnostics S10/S11/S13, bounded public test slice under [Task][RFC]: Qualify observer-first reliability diagnostics and governed delivery #5211.

Shared-authority RFC fixture impact

N/A — this change adds no shared Goal Authority fixture or provider conformance arm.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths.
  • I did not duplicate maintainer-owned benchmark work.
  • I kept the change scoped to the linked task.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer.

Signed-off-by: mika <211269698+mikamikasuki@users.noreply.github.com>
@mikamikasuki
mikamikasuki force-pushed the codex/loopx-5211-cordis-conformance branch from a705616 to e658ca1 Compare October 6, 2026 09:52

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | model=gpt-6.1-sol | provider=OpenAI | runtime_reported | reasoning_effort=xhigh

动机

维护 DSH 诊断适配器的人需要知道:宿主真实会话事件是否真的进入了诊断文件。原测试能模拟注册函数,但模拟对象可能恰好替实现提供了正确答案,遗漏 SDK 发布/清理行为。本次在现有测试里创建真实 Cordis Context 与 DSH SessionStore,发送合成消息,按明确顺序关闭并读取实际文件,补上了一个可重现的兼容性盲点。

Affected path: existing Vitest → real Cordis Context / DSH SessionStore → registerShadowObserver → actual session create/append/disposal → real file ledger. Before/after: Existing production hooks work with synthetic SDK sessions, but repository tests only use fake Context/Session registration; no real SDK+file lifecycle conformance assertion. → A new actual Cordis+DSH synthetic Store-first lifecycle test expects start/message/disposed and ledger counters; SDK and real file path pass. Production source/dependencies are byte-identical and base/head real probes match. Observable result: A new actual Cordis+DSH synthetic Store-first lifecycle test expects start/message/disposed and ledger counters; SDK and real file path pass. Production source/dependencies are byte-identical and base/head real probes match. Scope / remaining gap: No live DSH worker/model run, C0/C1 or P0 exit, measured overhead/retention or deployment. Direct Context-only shutdown still omits session_disposed in both base and head; existing #5211 owns remaining lifecycle acceptance.

改动思路

最强的反对理由是新增一个通过的集成测试可能使“完整宿主关闭已证明”的表述更可信,而测试其实先显式 dispose SessionStore,再 dispose Context。审查因此单独测试了直接 Context 关闭:旧版和新版都只有 start/message 两条,没有 disposal。这项变化的合理范围是一个 Store-first 发布测试,不能替代整个宿主生命周期或长期运行验收。

既有 typed observer 仍拥有被动事件过滤和文件写入,Cordis/DSH 仍拥有 Session 生命周期;测试复用已有 Vitest 和已锁定 SDK,没有新的 provider、control route、状态 owner 或手工同步字段。配置关闭时不注册 observer、不产生文件;启用后只有指定 session 被接受,新的其他 session 不会因处于同一容器而加入。真实写入失败增加诊断计数而不使会话发布抛错。

具体改动

全量 PR 只有 packages/dsh-loopx-plugin/tests/observer.spec.ts,+58/-2。评审 base 为 8251ec80e0c327d28d13a1fd64e3f343a2aa3c0e,head 为 e658ca1d8022736681af6586f0c10d65198ef051;生产 observer、package manifest 和 lockfile 的 Git blobs 在两端完全相同。这是测试覆盖增量,运行时没有被修复或升级。

验收依据是接受的 docs/architecture/rfcs/long-running-agent-reliability-diagnostics-governed-delivery-v0.md,固定 revision 8251ec80e0c327d28d13a1fd64e3f343a2aa3c0e(审查改动前的 §12)和 owner 创建的 #5211。贡献者评论 5871948621 提供测试方向,不能赋予阶段晋级授权。为可核对引用,这里将 §12 对应要求标为 p0-real-event-source-fixture:实际 SDK 发布的确定性 fixture 已验证;p0-no-outbound-boundary:复用的 passive/no outbound 边界已验证;p0-exit-held:C0/C1、真实宿主关闭、开销和留存/删除要求延期,现有 #5211 是剩余验收 owner。这些标签是本评审对要求的引用,不声称 RFC 原有这些 identifier。

关键代码讲解

  • observer.spec.ts:469 的 Cordis and DSH session publication integration 使用真实 SDK,创建精确 Session,append 合成 user message,显式先 sessionStoreFiber.dispose() 再 ctx.fiber.dispose(),读取实际 JSONL,断言 start/message/disposed 及 observed=accepted=3、rejected=failure=0。finally 再清理,失败路径也移除临时目录。
  • 未改动的 src/observer.ts:617 的 registerShadowObserver 注册只读 created/event/disposed hooks,Cordis effect cleanup 调用 observer.dispose。测试对这一生产 wiring 给出独立约束,而没有替它伪造 event postcondition。
  • 同文件 :372 的 ShadowObserver 保留精确 session 范围、缓冲上限、失败隔离和落盘。在私有源副本删除 session/event hook 后,真实 SDK 仅产生 start/disposed 两条,缺少 user_message;新测试的三事件断言会失败,说明它具有回归辨识价值。

对主干的风险

没有当前阻塞发现。按项目指定 pnpm 10.33.0 与 frozen lock 安装,三项 TypeScript/typecheck 检查及 11 files / 207 tests 全部通过,实际依赖为 Cordis 4.0.2、DSH-session 0.2.0-rc.2。独立 base/head 的真实 SDK+文件探针都得到 Store-first 三事件;其他 session 三事件均被拒绝且无 envelope;关闭配置无 hook/ledger;将 ledgerDir 指向文件时两次真实写入失败被隔离,宿主发布/关闭正常返回。所测 ledger 不含消息正文或 cwd。没有使用模型 worker 或把 SDK 合成会话称为真实 agent cycle。

需要明确保留失败:直接 ctx.fiber.dispose() 在 base/head 都缺少 session_disposed。这是父 #5211 原有的关闭顺序缺口,并非当前测试引入的生产行为回归;本 PR 也没有证实完整 Context 关闭。未来若要声明 host shutdown 或 C0/C1 完成,必须覆盖并修复这一反例。增加一个 SDK/file 测试会带来测试启动成本,当前没有验证其完整时间增量;但不增加生产常驻结构,也没有理由为测试增量做无关 runtime 重构。

语义与 CI 对齐

语义 advisory 后的全树 vocabulary smoke 和 diff 检查通过。没有新增词汇/协议/配置义务;复用 existing typed envelope/event vocabulary,DSH 特有术语留在 provider 测试。生产源码和依赖锁完全相同,加上真实 off/scope/failure 前后对照,证明本切片没有改变关闭配置或用户操作。observer 是诊断工具,不给予 scheduler/worker 控制权;测试通过是 conformance 证据,不是发布、权限或 C1 晋级许可。本地检查未查询或等待 CI。

我的整体评价

English verdict: APPROVE — no blocking finding at e658ca1. problem_context 为 justified_increment:long_horizon improved 指回归检测能力的具体提升;user_experience preserved 指运行时与用户旅程不变。真实边界能及时暴露 SDK 接线变化,减少假测试掩盖问题的可能;不能据此宣布模型长期可靠性、效率、吞吐或成本改善。

有界 future-facing pass 结论是无需新增生产抽象:现有测试和 typed observer 的归属已足够,已有 fake 测试仍服务确定性失败/缓冲覆盖,真实 SDK case 解决不同盲点。明确 Store-first 边界、保留实际 Context-only 两事件反例及 #5211 owner 后,这个单文件增量有正向维护价值;更广生命周期和长程净收益仍未验收。

@huangruiteng
huangruiteng merged commit 60371ce into loopx-project:main Oct 6, 2026
21 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants