Skip to content

feat: add phase 3 procedure evaluation gate - #1

Draft
fcsguyxcw wants to merge 63 commits into
masterfrom
agent/phase3-procedure-gate
Draft

feat: add phase 3 procedure evaluation gate#1
fcsguyxcw wants to merge 63 commits into
masterfrom
agent/phase3-procedure-gate

Conversation

@fcsguyxcw

Copy link
Copy Markdown
Owner

What changed

  • add the bounded, read-only Phase 3 pagination detector and deterministic CompiledProcedure draft builder
  • add source/dependency fail-closed checks and a controlled draft-to-validated transition
  • add frozen train/held-out cases, an independent verifier, per-metric promotion gates, real-cost evidence, and Store-backed practice-evidence checks
  • document the pilot inventory and final Phase 3 gate result

Why

Phase 3 evaluates whether a repeated declarative Skill path can become a safe partial procedure without bypassing parent Skill bindings, authorization, validation, or practice-evidence requirements.

Result

Gate P3 remains blocked and the procedure remains draft: measured N_break-even is 10.129724 (threshold <= 10) and qualifying real Practice evidence is 0.

Validation

  • npm.cmd test: 212 passed, 0 failed, 1 skipped (Windows symlink privilege)
  • node --test src/procedures/phase3/index.test.ts src/evaluation/phase3/*.test.ts: 49 passed
  • npm.cmd run typecheck: passed
  • git diff --cached --check: passed before commit

fcsguyxcw and others added 30 commits August 14, 2026 15:16
- B1: inject 模式精确移除 Pi 原生全量 Skill block(唯一性/残留 marker 校验,失败 fail open)
- B2: project-local load_skill 工具(路径/revision/source/manifest/大小/编码六重 fail-closed)
- onDiscovery 候选快照 seam 供 B3 observer 消费
- 真实 runner 链 host integration 测试

Co-Authored-By: Claude <noreply@anthropic.com>
- 消费式 Practice observer:RouteSnapshot 校验 + load_skill source_hash 绑定 + policy gate append
- host version 不硬编码(environmentFingerprint/environmentClass 省略,避免 0.84.1 谎报)
- 错误事件经 PracticeStore.invalidate 精确失效(保留 tombstone)
- 真实 0.84.2 --no-session 重跑产生 policy-valid real 事件

Co-Authored-By: Claude <noreply@anthropic.com>
- 引入 implementation-progress-audit 为开工必读与状态判定口径
- 新增 CLAUDE.md(leader 视角:herdr 指挥 pi、三层验收、git、版本漂移边界)
- 同步 README 当前阶段状态

Co-Authored-By: Claude <noreply@anthropic.com>
- observer 新增通用 EvidenceHook 接口(注入方提供,不硬编码 verifier)
- practice-pagination-hook:prompt 提取 SQL → detector 复算 → 结构化验证 → 注入 step+verifier
- b4-e2e-entry:验收 harness,真实 --no-session 产生 2 条 verified_skill_effect real 事件
- resolvePracticeEvidence 真实绑定 ok(distinctRealCount=2)
- .gitignore 忽略 .serena/

Co-Authored-By: Claude <noreply@anthropic.com>
- inducePhase3ProcedureDraft:确定性可回放纯函数,≥2 条契约事件 → 对齐稳定片段 → draft
- 复用 buildPhase3ProcedureDraft,绑定 evidenceIds + coveredSteps
- fail-closed:policy/字段/父绑定/covered step/verifier/attribution 逐层校验

Co-Authored-By: Claude <noreply@anthropic.com>
- cost-benchmark.ts:冻结口径(compile+validation=procedure 运行时生成+验证,非开发流水线),mean/stddev,四类测量
- 修正 N_break-even=0.463/(5355-0.002-1116)=0.000109 ≤ 10(原 10.13 误用开发流水线 7.9s 作分子)
- realCostEvidence 验证 ok;45 慢路径样本
- 报告写 docs/reports/2026-08-14-phase3-cost-benchmark.json

Co-Authored-By: Claude <noreply@anthropic.com>
- p3-gate-runner:真实事件 → induction → resolvePracticeEvidence → B6 cost → held-out replay → judgePromotion 11/11 PASS → transitionPhase3ProcedureValidation
- 用真实 Store 事件(obs-...),非 fixture;证据门控(事件缺失如实失败)
- procedure draft → validated;evidenceIds 绑定 2 条真实事件

Co-Authored-By: Claude <noreply@anthropic.com>
- audit:B1-B6 关闭证据 + 阶段状态表更新 + 下一轮顺序
- README:Phase 3 procedure validated(Gate P3 闭环)

Co-Authored-By: Claude <noreply@anthropic.com>
- runP3GateValidation 增加注入参数(store/tenantScope/eventIds)覆盖冻结值
- 测试改用构造事件(绑定冻结 supabase 值),不依赖 gitignored .skill-cortex
- clean clone 时 npm test 全绿;真实事件整链仍由 --write-report 显式跑

Co-Authored-By: Claude <noreply@anthropic.com>
… fallback)

- resolveExecution 纯函数,覆盖 a-i 全部分支(no_skill_selected/no_procedure/insufficient_evidence/revision_mismatch/dependency_mismatch/precondition_failed/unsupported_effect/authorization_required/eligible_procedure)
- dependencyFingerprintMatches + evaluatePreconditions fail-closed(缺失即 mismatch/unknown)
- guard:前置/runtime/postcondition 任一 fail/unknown 停止快路径
- fallback:abstain/load_parent_skill + 首个可归因失败步骤
- 纯函数,未接线宿主执行路径,不启动 canary/active

Co-Authored-By: Claude <noreply@anthropic.com>
- executor:resolveExecution → guard → 快路径(detectPagination)/慢路径(load SKILL.md 模拟)/abstain 编排
- canary:project-local canary 模拟,wrongFastPathRate=0,correctRejectionRate=1
- 集成测试覆盖 §9:慢路径路由/guard 停止/verifier 失败不自我发布/授权 gate 一致/无重复副作用
- host integration 未完成(真实 tool 事件接线未验证),Phase 5 未启动

Co-Authored-By: Claude <noreply@anthropic.com>
- Phase 4 Execution Resolver component 完成(resolveExecution/guard/fallback/executor/canary)
- host integration 未接线,列为当前 blocker
- 更新下一轮顺序

Co-Authored-By: Claude <noreply@anthropic.com>
- ADR-0011: 新增 evidence envelope;effectless/permissionless pilot 显式省略
  permissionPolicyHash;evaluation/formal 来源隔离;11 门逐项标注
  automated/static_review/owner_attested
- ADR-0012: ExecutionContext/DecisionExecutionContext 类型 + parent_skill_mismatch;
  resolver/executor/guard/fallback 补齐 shadow_replay/canary/active 状态矩阵与
  disposition 路由;canary harness 固定 shadow_replay
- 同步 contracts/design/audit/reports/README/AGENTS

Co-Authored-By: Claude <noreply@anthropic.com>
…cution-adapter

- practice-observer: 新增 buildCompiledPracticeEvent / selectCompiledSelections
  编译快路径 seam(shadow provenance、attribution 由 policy 重算)
- execution-adapter: tool_call preflight 一次性 receipt + checkAuthorization 消费 +
  严格白名单 decoder;shadow_replay + fake-pi 冒烟
- 真实宿主 tool_call/tool_result/agent_settled 接线与 E2E 仍未验证

Co-Authored-By: Claude <noreply@anthropic.com>
- host-integration-entry.ts: 真实 ExtensionAPI 接线 registerSkillCortex +
  registerPracticeObserver(compiledTool seam) + registerSkillCortexPaginationShadow
  (恒 shadow_replay,不启动 canary/active,不写生产入口)
- host-integration.test.ts: 真实 loader + ExtensionRunner 隔离 E2E,验证
  a) tool_call preflight 身份失配 block(受控码+terminate,先于工具执行)
  b) 被 block 调用不产生 tool_result 事件(observer 无证据,store 空)
  c) 身份匹配走 fast_path,guard/verifier 全 pass,detector 确定性输出复现
- 范围如实:依赖漂移未验证(execution-adapter 闭包写死)、零 I/O 仅 detector
  纯函数间接证据、observer 归因 fail-closed(快照身份失配不落盘)

Co-Authored-By: Claude <noreply@anthropic.com>
… real

- execution-adapter 增加注入点 currentSkillRevision / currentDependencyFingerprint /
  guardObservations(可选;未提供回退 procedure 自身值,保持既有 self-match)
- 去空转 resolver revision/dependency 检查(cc HIGH 1):注入不同值即被
  revision_mismatch / dependency_mismatch 捕获并 fallback,不再恒通过
- 新增 5 测试:revision/fingerprint 漂移 → slow_path、guard fail → fallback、
  注册层 options 透传、无注入 self-match 兼容

Co-Authored-By: Claude <noreply@anthropic.com>
- attribution-e2e.check.ts(*.check.ts 不进 npm test,显式运行,non-portable):
  真实 skill 只读加载(不复制正文,ADR-0010)→ discovery 快照身份与
  P3_GATE_FROZEN 逐字节一致 → compiled tool fast_path → tool_result 经
  compiledTool seam 解码 → provenance=shadow PracticeEvent 落盘
  (attribution=verified_skill_effect,policy 通过,round-trip 可查)
- 真实 skill 缺席 ⇒ skip;身份与冻结值不符 ⇒ 显式 fail
- 范围如实:零 I/O 仅 detector 纯函数复现;依赖漂移未验证(self-match)

Co-Authored-By: Claude <noreply@anthropic.com>
… blocker)

- HIGH 1 + MED: current values 改 per-call PilotCurrentProvider(execute 时从
  当次候选表取 revision,非 register-time static);host-integration-entry 维护
  latestCandidates(onDiscovery 构建 skillId→revision),真实 runner 不再 self-match
- HIGH 2: decodePilotDetailsToEvidence 对 pre-execution 拒绝 fail-closed
  (slow_path/denied/safety_stop/abstain/guard_failure/procedure_error ⇒ undefined),
  仅 fast_path 与 post-execution verifier_failure 产生 compiled 事件
- 新增 drift-e2e.test.ts(真实 ExtensionRunner):失配 revision ⇒ slow_path 且
  0 shadow 事件;失配 fingerprint ⇒ dependency_mismatch 且 0 事件;匹配 ⇒
  fast_path 落 1 verified 事件
- 残余如实:fingerprint 真实来源未接线(候选卡不暴露 sourceHash),当前
  dependencyFingerprint 以 procedure 绑定为基准,fingerprint drift 由注入 provider
  测试层锁定

Co-Authored-By: Claude <noreply@anthropic.com>
… source

- Point B: current provider 注册但返回 undefined(候选缺失)⇒ fail-closed
  (currentSkillRevision/fingerprint 置 undefined → resolver D2 拒绝 ⇒ slow_path),
  不再回退 self-match;self-match 仅保留给未注册 provider 的 unit/canary 场景
- Point A: 新增 deriveDiscoverySourceHashes(core.ts,复用 buildSkillRecord 同输入
  保证键一致),entry 在 before_agent_start 从 systemPromptOptions.skills 派生
  latestSourceHashes,provider 的 currentDependencyFingerprint.sourceHash 来自
  当次 discovery 真实内容指纹(非 procedure 自身)
- latestCandidates/latestSourceHashes 在 agent_settled 清空,防跨 run 串扰
- 新增 drift-e2e d:候选缺失 ⇒ fail-closed slow_path + 0 事件;既有 host-integration
  断言按 Point B 更新(fixture 无 P3 候选 ⇒ fail-closed),fast_path 由 attribution/
  drift-e2e c 覆盖

Co-Authored-By: Claude <noreply@anthropic.com>
… CLOSED

- audit §2/§2.1: Phase 4 host integration(shadow)+ E2E 关闭(b7b8d42),drift
  blocker CLOSED;生产入口接线与 canary/active 未启动;记录 3 项非 blocker
- CLAUDE.md §3: 同步 Phase 4 状态

Co-Authored-By: Claude <noreply@anthropic.com>
…ntext)

- draft.ts: 新增 transitionPhase3ProcedureCanary 纯函数(显式 decision + 证据绑定
  evidenceIds + canaryReportId 模式校验;draft/重复晋升/非 canary decision 一律
  fail-closed 拒绝)
- contracts: CompiledProcedure 增 canaryReportId?(与 validationReportId 同语义)
- canary.ts: CANARY_GATE_EXECUTION_CONTEXT="canary" + runP3CanaryGateSimulation
  (三栏指标 wrongFastPathRate/correctRejectionRate/safetyStopCount,无加权总分)
- 新增 canary-transition.test.ts(9 用例)+ canary-gate.test.ts(9 用例,矩阵:
  validated 进 canary ⇒ insufficient_evidence、canary 进 active ⇒ slow_path、
  missing/illegal ⇒ unknown fail-closed)
- shadow harness 行为不变;project-local only,不启动真实宿主部署

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- draft.ts: 新增 transitionPhase3ProcedureActive/Suspend/Resume/Retire 纯函数
  (canary→active 需 activeReportId 绑定;suspend/retire 需 lifecycleReason;
  resume 仅原 suspended);补 transitionPhase3ProcedureValidation 输入状态防御
  (retired 不得复活)
- contracts: CompiledProcedure 增 activeReportId?/lifecycleReason?
- 状态机 7 合法边 + 29 非法边 fail-closed(跳级/逆行/复活全拒绝)
- active-gate.test.ts:active 上下文 gate(active+active⇒fast_path、
  active+canary⇒insufficient_evidence 按 ADR-0012 矩阵、suspended/retired⇒不可执行)
- shadow_replay 不进状态机;project-local only

Co-Authored-By: Claude <noreply@anthropic.com>
- dependency-diff.ts: diffProcedureDependencies(绑定维度比较,current 缺失
  fail-closed,未绑定维度不失效)+ invalidateOnDependencyDrift(source/tool/
  permission/model 变化只失效相关 procedure;终态不重复 suspend)
- draft.ts: suspend 边扩展 validated|canary|active → suspended(原仅 active)
- 状态机合法边 7→9,非法矩阵 29→27
- 指纹 6 维度已由 DependencyFingerprint 覆盖(sourceHash/toolSchemaHash/
  permissionPolicyHash/environmentClass/modelId/promptHash),无需补字段
- 17 个 diff/失效测试;纯确定性 artifact 无关 model 变化不失效

Co-Authored-By: Claude <noreply@anthropic.com>
- draft.ts: ActiveTransition 增 previousStableRevision?(^rev:64hex 校验);
  transitionPhase3ProcedureActive 晋升时写入;新增 rollbackProcedure 纯函数
  (stableLookup 注入,无 registry)
- rollback 判定:无 previousStableRevision / lookup 找不到 ⇒ no_stable_version
  (走慢路径);非法引用 / 未发布版本 ⇒ invalid_stable_version;active/suspended
  稳定版 ⇒ rollbackTo(恢复 active,清除 lifecycleReason)
- revision 引用语义(非快照),依数据合同 §6.2
- 10 用例:晋升记录、一键回滚、慢路径、不可变、executor 集成

Co-Authored-By: Claude <noreply@anthropic.com>
- evidence-cascade.ts: findAffectedByEvidenceDeletion(纯函数,依赖查找注入,
  输出 affectedProcedureIds + affectedCues)+ suspendProceduresForEvidenceDeletion
  (非终态 validated/canary/active → suspend,lifecycleReason=evidence_cascade_deletion)
- 复用 PracticeStore.invalidate 既有 DeleteResult.invalidatedEventIds 输入契约,
  不新造管道;store 未改
- cue 失效仅做命中判定(affectedCues),profile 挂起/回 shadow 未接线(无 profile
  状态机模块)
- 12 用例;纯函数层,host integration / E2E 未接线

Co-Authored-By: Claude <noreply@anthropic.com>
host integration / E2E 未接线;Gate P5 不宣称通过

Co-Authored-By: Claude <noreply@anthropic.com>
- HIGH 1: 新增受控失效 reason 常量 + suspendKindOf 分类;transitionPhase3ProcedureResume
  对失效暂停(dependency_drift/evidence_cascade)fail-closed 拒绝 resume
  (resume_blocked_requires_revalidation),仅 manual 可逆暂停可 resume active,
  封堵 validated/canary→suspended→active 绕过
- HIGH 2: rollbackProcedure 增 identity/lineage 校验(procedureRevision ===
  previousStableRevision、procedureId/parentSkillId 同源)先于状态判定;suspended
  失效 target 恢复 active 须 dependencyRevalidated,否则 requires_revalidation
- dependency-diff/evidence-cascade 的 suspend reason 改用受控常量集中
- 新增 6 测试(绕过拒绝 + 冒充/跨 lineage/未重验拒绝)

Co-Authored-By: Claude <noreply@anthropic.com>
- contracts: CompiledProcedure 增 suspendedFrom?/suspendKind?(suspended 时写入);
  lifecycleReason 仅人类可读审计,不作恢复判定
- draft.ts: suspend 自动派生 suspendedFrom(不可伪造)+ 显式 suspendKind;resume
  仅 suspendedFrom=active + suspendKind=manual 放行,否则
  resume_blocked_requires_revalidation;移除 suspendKindOf 自由文本推断
- rollback: suspended target 须 suspendedFrom=active;drift/cascade 须
  dependencyRevalidated
- 封堵 validated/canary→suspended(manual)→active 绕过;新增绕过测试

Co-Authored-By: Claude <noreply@anthropic.com>
fcsguyxcw and others added 30 commits August 16, 2026 15:55
…sappearance)

- lifecycle/index.ts: 4 pipeline seams(applyDependencyDrift / applyEvidenceCascade /
  rollbackToPreviousStable / suspendProceduresForMissingSkills),依赖注入不持有 registry
- dependencyRevalidated 模块私有 deriveRevalidationFromCurrent(真实 diff
  shouldInvalidate===false 才派生 true),无公开 seam 接受该参数,禁裸布尔
- store: ReleaseStateRecord 增 evidenceIds(MED:canary replayEvidenceIds 在 stable 重建保留)
- 15 E2E:drift 仅目标 suspend / evidence cascade / rollback 恢复与 drift 拒绝 /
  skill disappearance / store reload / current 缺失 fail-closed
- rollback 落盘未实现(依赖 future revision/save seam);crash consistency 保持 blocker

Co-Authored-By: Claude <noreply@anthropic.com>
…+ rollback 持久化)

- ① conditional fingerprint invariant: fingerprint-bindings.ts validator
  (llmHoles→modelId/promptHash、effects/permissions→permissionPolicyHash、
  effectless→omit、占位→fail),diff 入口 fail-closed
- ② uninstall/scope/move/rename matrix: currentInstalledSkillIds 明确为完整
  installed identity snapshot(非 Top-K),矩阵测试锁定
- ③ rollback persistence 闭环: store.rollbackTo 专用 seam(跨 revision 切回
  current 唯一路径,BLOCKER 2 保留)+ 可审计 rollback event + reload 保持

Co-Authored-By: Claude <noreply@anthropic.com>
- host.ts: runHostLifecycle(依赖注入)把真实事件源喂给 lifecycle seams
  ——installedSkillIdsFromSkills(完整 installed 快照,非 Top-K)、
  currentFingerprintFromSourceHashes(sourceHash 真实 derive,其余维度
  procedure 绑定)、invalidatedEventIds(cascade)、rollback 复用闭环
- host-e2e.test.ts: 9 场景 project-local E2E(真实读盘/落盘/执行)——
  match 可执行、source/tool drift 仅目标 suspend、uninstall/identity、
  evidence deletion、rollback 落盘恢复/drift 拒绝/no stable slow path、
  store reload

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- HIGH 1: rollbackTo 补 stale-prior 三要素校验(procedureId/procedureRevision/
  status),不一致 procedure_store_rollback_stale_prior 零写入
- HIGH 2: transition 增 assertImmutableContentUnchanged(canonical immutable
  projection,同 revision 禁改 artifact/dependency/guards 等),违规
  procedure_store_immutable_content_mutation
- HIGH 3: rollbackTo API 改 (failed, stableRevision, meta),store 内部重读
  stableNow 构造 active 恢复版本,不再接受 caller 完整 target
- MEDIUM: runHostLifecycle 顺序 identity→drift→cascade,uninstall 原因不再被
  drift 吞成 current_unavailable
- 回归测试:stale prior 零写入 / 同 revision 篡改拒绝 / rollback 写 stableNow /
  identity 先判定

Co-Authored-By: Claude <noreply@anthropic.com>
- transition: immutable 校验改为 assertAllowedDelta(stored, next)(以落盘 stored
  为权威,不信任 prior——即使 prior+next 同时伪造也因与 stored 不一致拒绝);
  晋升锁定字段(reportId/previousStableRevision)仅特定 promotion 边可设置;
  evidenceIds 仅 validated→canary 可追加(不得删除)
- rollbackTo: 目标 revision 权威来源 = stored.previousStableRevision(不信任
  failed.previousStableRevision);idempotency 先于 stale-prior
- 回归测试:同时伪造 prior+next immutable 拒绝、active→suspended 偷改
  previousStableRevision/activeReportId 拒绝、validated→canary 删 evidenceIds
  拒绝、伪造 failed.previousStableRevision 拒绝

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…vidence)

- induction.ts: induceActivationProfile(events, parentSkill) 纯函数——
  verified_skill_effect → learnedAliases + positiveExamples;near-miss/boundary
  → nearMissExamples(只降权不硬过滤);external failure 不产 cue;environmentCues
  从 fingerprint 派生
- author vs learned 分栏:learned cue 只 overlay 补充,与作者 name/declaredAliases
  去重;确定性 cueId(cue:sha256...)+ evidenceIds 可追溯
- fail-closed:evaluation/synthetic 拒绝、父绑定失配拒绝、policy 非法拒绝、
  无 eligible 事件拒绝
- 11 测试

Co-Authored-By: Claude <noreply@anthropic.com>
- rerank.ts: rerankWithOverlay(软语义——learned alias/positive 命中 soft
  boost + learned_cue evidence;nearMiss 只降权不硬过滤;overlay 关闭输出
  与静态 deepEqual 可复现)
- evaluate.ts: evaluateOverlay(hard confuser/no-skill/multi-skill/跨语言
  四栏 Recall@K/set recall,nonInferior 门槛 + goldPreservedInTopK 退化检测)
- 17 测试;评估用冻结合成 fixture,未定真实 Recall 门槛

Co-Authored-By: Claude <noreply@anthropic.com>
- state.ts: ActivationProfile 状态机(draft/shadow/active/suspended/retired,
  8 合法边 + 25 非法边 fail-closed,纯 transition 不可变)
- promotion.ts: evaluateProfilePromotion 消费 OverlayEvaluationReport
  nonInferior + 冻结门槛(recallAtK/setRecall≥0.8、noSkillPrecision=1、
  confuserNotRecalled≥0.8、goldPreservedInTopK=1)
- cascade.ts: profileCuesReferenceEvidence / removeCuesReferencingEvidence /
  suspendProfilesForEvidenceDeletion / revertProfilesForParentRevision
- hard-confuser fixture 修正(confuser 名字去泛词避免 name 误命中)
- 23 测试(activation 51 总)

Co-Authored-By: Claude <noreply@anthropic.com>
- store.ts: ActivationProfileStore(project-local .skill-cortex/activation,
  仿 ProcedureStore)——current 快照 + append-only 事件日志;save/transition
  (合法边 + stale-prior 三要素 + immutable 内容以 stored 为权威);
  applyEvidenceDeletionCascade(invalidatedEventIds → suspend 落盘)
- 查询 getProfile/listCurrent/listByStatus/listByEvidenceId/listEvents
- 分区/脱敏(rootDir 强制 projectRoot 内、tenantScope SHA-256、白名单深拷贝、
  fail-closed 读取);crash consistency 记 tech debt
- 13 测试(activation 64 总)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- BLOCKER 1: rerankWithOverlay overlay 生效加 candidate.skillRevision ===
  profile.parentSkillRevision(skillId+revision 双匹配才 boost/evidence),
  stale revision 输出 deepEqual 静态
- BLOCKER 2: store.save 仅允许初始 draft;shadow→active 边要求结构化
  promotion verdict(ok===true + nonInferior + 至少一栏已评估 + reportId 格式),
  裸 reportId 拒绝零写入
- MED: environmentCues valueClass 语义 tech debt 注释
- 6 新测试

Co-Authored-By: Claude <noreply@anthropic.com>
- heldout.ts: 冻结 held-out 案例集(12 例四栏各 3,与 dev 完全不同的 skill/query,
  防过拟合断言)
- promotion.ts: PROMOTION_THRESHOLDS 由 held-out 校准派生——recallAtK/
  confuserNotRecalled/goldPreservedInTopK = 1.0 下界 − 0.1 容差 = 0.9;
  noSkillPrecision=1 安全硬边界不放松
- 校准中发现 hard-confuser 查询含泛词误召回,修正查询聚焦 gold 独有特征(非调低门槛)
- 6 测试

Co-Authored-By: Claude <noreply@anthropic.com>
- goldPreservedInTopK 硬边界 1(退化即拒),recall/confuser=0.9、noSkill=1
- promotion 必须覆盖四栏(缺任一栏拒绝,不虚判)
- store 自行重算 evaluateProfilePromotion(report),不信任 caller verdict.ok
- 12 例 held-out 改名 calibration set(已参与 query/threshold 调整)
- 新增 untouched final-heldout:高词汇重叠 hard-confuser,跑后不改 case

Co-Authored-By: Claude <noreply@anthropic.com>
…promotion→overlay)

- overlay.ts: applyActiveProfiles 多 active profile(revision 匹配)软重排,关闭无损回静态
- host.ts: 受控 evaluator + promoteProfileIfEligible + induceAndStoreShadow +
  revertProfilesForParentRevisionChanges;parent 不在评估集内拒绝(防 trivial 晋升)
- discovery seam: RegisterOptions.onCatalog + overlayProfiles(core.ts/index.ts)
- 真实 ExtensionRunner E2E:observer(verified) → induction → shadow → 受控 promotion
  拒绝(parent_not_in_evaluation_set)→ active overlay 追加 learned_cue;revision 失配不生效

Co-Authored-By: Claude <noreply@anthropic.com>
…rlay

Phase 7 Seam 1: AdapterState now carries overlayProfiles/overlayOptions
(injected from createDiscoveryServices options at state init), and
runSearchTool applies applyActiveProfiles to static BM25 matches so
search_skills supplement-search honors the same revision-matched active
overlay as the before_agent_start discovery path. Without overlayProfiles
the result is unchanged (no learned_cue evidence).

Tests: active profile with matching parentSkillId/revision appends
learned_cue evidence to details.matches; no overlay stays static.

Co-Authored-By: Claude <noreply@anthropic.com>
- buildFrozenEvaluation:唯一评估集来源(父自身 name/desc + 冻结 no_skill +
  learned 中文 alias / 冻结中文后缀回退),父不在 catalog ⇒ 空 case 集
- promoteProfileIfEligible 只接受 (store, shadow, catalogRecords, reportId),
  不再接受 caller 自定义 cases/records/options
- 真正高词汇重叠 confuser ⇒ 诚实拒绝(confuserNotRecalled 掉门槛)
- E2E:真实 skill 经冻结评估 promotion ⇒ active

Co-Authored-By: Claude <noreply@anthropic.com>
- runActivationHostLifecycle:每轮编排 = 父 revision 漂移回 shadow + induction + 受控 promotion
- runEvidenceDeletionCascade:PracticeStore.invalidate 的真实 invalidatedEventIds → applyEvidenceDeletionCascade → suspend
- host-integration-entry 改用 runActivationHostLifecycle;导出 phase6PracticeStore 供 E2E 断言

Co-Authored-By: Claude <noreply@anthropic.com>
- 新增 docs/reports/2026-08-16-phase7-validation.md:六层(Catalog/Discovery/Selection/
  Resolver/Execution/Lifecycle-Security)逐层验收证据 + 已知边界
- audit §2 表 + §2.1 标注 Phase 6/7 收口 + real-host 前 blocker
- README/CLAUDE.md 阶段状态更新为 Phase 0~7 Complete

Co-Authored-By: Claude <noreply@anthropic.com>
- store.test.ts after() 清理死变量 tempDirs 改为清理 tempRoot(此前每次运行必残留)
- store.test.ts / host.test.ts PROJECT_ROOT 3 级 .. 改为 2 级(src/activation/ 深度 2,
  3 级会解析到桌面父目录,导致 .tmp-* 散到仓库外)
- .tmp-*/ 已在 .gitignore,残留不影响 git,但污染磁盘

Co-Authored-By: Claude <noreply@anthropic.com>
- audit 标题/日期/状态行改为 Phase 0~7 收口(此前与 §2 表冲突,仍写 Phase 4 host
  integration incomplete / Phase 5 不得启动)
- audit §5 测试计数 376→687
- CLAUDE.md §3 硬规则移除过期的「此前不得进入 Phase 5 active path」
- 2026-08-16 leader handoff 标注 superseded,指向当前状态

Co-Authored-By: Claude <noreply@anthropic.com>
… evidence

- ProcedureStore/ActivationStore: per-entity in-process mutex + event-first atomic (tmp+rename) commit + deterministic dangling-event recovery on read. Concurrent writers serialize; the loser re-reads and fails stale-prior.
- Activation promotion evidence bound to profileId/parentSkillRevision/profile content hash/evaluation set hash/evaluation config hash; store re-verifies against on-disk profile (cross-profile PASS reuse rejected).
- Real-skill frozen promotion gate downgraded to hard_confuser+no_skill: multi_skill/cross_language cannot be genuinely validated with the real catalog, so the fake single-gold / parent.name cases are removed (no fabricated fixtures, no lowered thresholds).

Co-Authored-By: Claude <noreply@anthropic.com>
…romotion

- ProcedureStore/ActivationStore: replace event-first+rollback with a write-ahead txn (atomic tmp+rename) + roll-forward recovery (recoverAll). current/release/events now commit to a consistent end state; remove is a delete-txn (no stale release/history on partial delete). Closes crash-window and remove-crash holes.
- Replace in-memory mutex with a store-level filesystem lock (wx + lease + stale steal) so two store instances/processes cannot both write the same root.
- Activation promotion evidence now carries only the discovery catalog; the store recomputes frozen cases/report/verdict from the on-disk profile + records, so a PASS report/eval-input/binding can no longer be spliced across profiles.
- Tests: two-instance concurrency, crash-window (current written/release not), delete-txn recovery, and cross-profile promotion rejection.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant