diff --git a/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.md b/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.md index 9c3bf5f610..d8667fa338 100644 --- a/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.md +++ b/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.md @@ -5,7 +5,7 @@ - **Delivery maturity:** Proposal; existing catalog, hooks and external-evidence slices are partial prerequisites - **Authors / owners:** LoopX capability and control-plane maintainers - **Created:** 2026-09-21 -- **Last normative revision:** 2026-09-26 +- **Last normative revision:** 2026-10-10 - **Implementation baseline:** `65afc4872db67d36f74625a9e53ae63da2bc619c` - **Related contracts:** [overall roadmap](loopx-overall-roadmap-v0.md), [research exploration](research-exploration-control-plane-v0.md), @@ -445,6 +445,24 @@ Portfolio prompt section. Full catalog/history and owner joins stay behind on-demand inspect. Cache bounded declarations by their existing revision; configuration/provider/receipt drift invalidates only affected entries. +### Bounded improvement-intent feedback slice + +The existing `goal_capability_organization` provider consumes optional declared +trial feedback through `before_plan`. Its proposal digest binds Goal/Agent/Todo, +gap, candidate revision and original configuration/effect/rollback references. +When matching `failed` or `no_evidence` feedback is supplied, advice must not +suggest that unchanged trial again. Omitted feedback preserves legacy behavior; +this optional consumer cannot enforce a persistent retry prohibition. +Success requests original-owner review; changed inputs make prior feedback stale. +Independent candidates and enabled direct routes remain available; optional +feedback failure stays isolated. The [capability contract](../../../loopx/capabilities/goal_capability_organization/README.md) +defines the bounded wire format and identity semantics. + +This backend slice adds no authentication, adoption store, installer, composition +language or authority, and does not fulfill the portfolio lifecycle or connector +milestones. Qualification, useful effects, transfer and total development/evaluation +cost require separate evidence; deterministic tests are not an adoption decision. + ### Long-running responsibility, disclosure and memory evolution **Decision:** unify when context is disclosed and how an update is verified, diff --git a/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.zh-CN.md b/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.zh-CN.md index 082148e37f..cd759871bd 100644 --- a/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.zh-CN.md +++ b/docs/architecture/rfcs/goal-scoped-capability-portfolio-v0.zh-CN.md @@ -5,7 +5,7 @@ - **交付成熟度：** Proposal；现有目录、hook 与外部证据切片只是部分前置 - **作者 / Owner：** LoopX capability 与 control-plane 维护者 - **创建时间：** 2026-09-21 -- **最近一次规范修订：** 2026-09-26 +- **最近一次规范修订：** 2026-10-10 - **实现基线：** `65afc4872db67d36f74625a9e53ae63da2bc619c` - **相关契约：** [总路线图](loopx-overall-roadmap-v0.zh-CN.md)、 [研究探索](research-exploration-control-plane-v0.zh-CN.md)、 @@ -370,6 +370,20 @@ delegation 或 governed Turn。既有 turn-start/post-writeback hook 和 pending 条件。直接工作不增加 Portfolio prompt 段落；完整 catalog/history 与 owner join 留给 按需 inspect。有界声明按既有 revision 缓存，配置/provider/receipt drift 仅失效相关项。 +### 有界改进意图的反馈切片 + +既有 `goal_capability_organization` provider 经 `before_plan` 消费可选的声明 +反馈。提案 digest 绑定 Goal/Agent/Todo、缺口、候选 revision 与原配置／效果／ +回滚引用。收到匹配的 `failed` 或 `no_evidence` 反馈时，不能再次建议相同输入的 +试用。省略反馈保持旧行为，这个可选 consumer 不强制持久化禁止重试。成功只请求原 owner +评审，输入变化令旧反馈过期。独立候选与已启用直接路径仍可使用，可选反馈 +失败须隔离。[能力契约](../../../loopx/capabilities/goal_capability_organization/README.zh-CN.md) +定义有界输入格式与身份语义。 + +这个后端切片不增加认证、采用账本、安装器、组合语言或权限，也不完成 portfolio +生命周期或 connector 里程碑。资格、有效效果、迁移与开发／评估总成本须分别 +留证；确定性测试不是采用决定。 + ### 长程职责、渐进披露与记忆演化 **决策：** 统一何时披露上下文、如何验证更新，而不是把所有存储统一成万能记忆库。 diff --git a/loopx/capabilities/goal_capability_organization/README.md b/loopx/capabilities/goal_capability_organization/README.md index 43dd4b02e6..b94a4d99ca 100644 --- a/loopx/capabilities/goal_capability_organization/README.md +++ b/loopx/capabilities/goal_capability_organization/README.md @@ -58,6 +58,44 @@ References and caller observations are **not authenticated execution grants**; re-read the original owners before acting. Advice never authorizes a provider call, installation, configuration mutation, order, signing or transfer. +### Trial feedback through the same planning entry + +Trial proposals now return `trial_basis_digest`. After its original owner records +the outcome, the caller can attach optional `trial_feedback` to the same candidate +in `--capability-candidate-json`, for example: + +```json +{"outcome_ref":"owner/outcome-v1","trial_basis_digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","status":"failed"} +``` + +Replace the illustrative digest with the previous proposal's digest. The planning provider +hashes UTF-8 `JSON.stringify` of an object in this order: `goal_id`, `agent_id`, +`todo_id`, `gap_ref`, `capability_id`, `candidate_revision`, `configuration_ref`, +`effect_ref`, `rollback_ref`. Absent scope fields or revision are `null`. +`candidate_revision` is optional; it and `outcome_ref` follow the existing bounded +reference format. The digest has 64 lowercase hexadecimal characters after +`sha256:`. Existing applicability and enabled-state requirements still apply. + +| Caller-declared feedback | Advice for that candidate | +| --- | --- | +| Exact basis, `failed` or `no_evidence` | Continue work; do not propose the unchanged trial again | +| Exact basis, `succeeded` | Inspect the original owner's outcome; no utility or adoption inference | +| Different scope, revision or reference | Inspect stale feedback with the original owner | +| No feedback | Preserve the existing trial proposal behavior | +| Malformed feedback | Isolate optional advice failure; a fresh owner observation can recover it | + +A later independent candidate without feedback remains eligible. An enabled +direct candidate wins before feedback validation; off skips feedback entirely. +This deterministic advice consumer neither verifies receipts nor persists a +retry gate. The digest identifies declarations, not their authenticity. Renaming +a revision does not prove improvement; original admission and effect review +remain required. Records, retirement and rollback stay with their existing owners. + +Scope refinement references +[Rethinking the Evaluation of Harness Evolution for Agents](https://arxiv.org/html/2607.12227v3): +bounded tool feedback, task improvement, transfer and total development/evaluation +cost need separate evidence. Deterministic tests establish no empirical utility. + Retain the use/result/effect and retirement decision through existing Todo, outcome and capability owners. A useful result must improve a declared outcome or cost against its baseline; invocation, a PR or a successful test is not utility. diff --git a/loopx/capabilities/goal_capability_organization/README.zh-CN.md b/loopx/capabilities/goal_capability_organization/README.zh-CN.md index beeb1a88ac..de890003db 100644 --- a/loopx/capabilities/goal_capability_organization/README.zh-CN.md +++ b/loopx/capabilities/goal_capability_organization/README.zh-CN.md @@ -47,6 +47,40 @@ loopx agent-context --goal-id example --agent-id coordinator --phase before_plan 未给候选时可在预算内进行有界发现。引用和调用者观察**不认证执行授权**；行动前 仍读回原 owner。建议不授予 provider 调用、安装、配置变更、交易、签名或转账。 +### 经同一规划入口消费试用反馈 + +试用提案现在返回 `trial_basis_digest`。原 owner 记录结果后，调用者可在 +`--capability-candidate-json` 的同一候选中附加可选 `trial_feedback`，例如： + +```json +{"outcome_ref":"owner/outcome-v1","trial_basis_digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","status":"failed"} +``` + +示意 digest 须替换成原提案返回值。规划 provider 对按下列顺序构造的对象执行 UTF-8 +`JSON.stringify` 哈希：`goal_id`、`agent_id`、`todo_id`、`gap_ref`、 +`capability_id`、`candidate_revision`、`configuration_ref`、`effect_ref`、 +`rollback_ref`。缺失 scope 字段或 revision 为 `null`。可选 `candidate_revision` +与 `outcome_ref` 沿用有界引用格式；digest 是 `sha256:` 后接 64 位小写十六进制。 +适用性与启用状态仍须满足原要求。 + +| 调用者声明的反馈 | 对该候选的建议 | +| --- | --- | +| basis 一致，`failed` 或 `no_evidence` | 继续工作，不再次提议相同输入的试用 | +| basis 一致，`succeeded` | 检查原 owner 结果，不推断效用或采用 | +| scope、revision 或引用变化 | 与原 owner 检查已过期反馈 | +| 没有反馈 | 保留原试用提案行为 | +| 反馈格式非法 | 隔离可选建议失败；取得新的 owner 观察后可恢复 | + +后续独立且没有反馈的候选仍可使用；已启用直接候选在反馈校验前优先选中， +关闭策略时跳过反馈。这个确定性建议 consumer 不认证回执或持久化重试门禁。 +digest 只标识声明，不认证真实性。重命名 revision 不证明试用改进，原准入与 +效果评审仍须执行。记录、退役与回滚保留原 owner。 + +范围细化参考 +[Rethinking the Evaluation of Harness Evolution for Agents](https://arxiv.org/html/2607.12227v3)： +有界工具反馈、任务效果、迁移和开发／评估总成本须分别留证，确定性测试不证明 +实证效用。 + 使用／结果／效果和退役判定继续写回既有 Todo、outcome 与能力 owner。实际价值 必须相对基线改善目标结果或成本；调用、PR 和测试通过不是效用，未知效果保持未知。 经原能力 owner 回滚，不删除历史证据。空、非法或超预算建议独立失败，不能阻断 diff --git a/loopx/control_plane/capabilities/goal_capability_organization.ts b/loopx/control_plane/capabilities/goal_capability_organization.ts index 80565736d8..6beeeb1fe1 100644 --- a/loopx/control_plane/capabilities/goal_capability_organization.ts +++ b/loopx/control_plane/capabilities/goal_capability_organization.ts @@ -1,4 +1,5 @@ /** Goal-owned improvement intent; never capability enablement or admission. */ +import { createHash } from "node:crypto"; import type { AgentContextProvider } from "../agent_context.ts"; import type { JsonObject } from "../effect_program.ts"; import { jsonObject, requireJsonObject } from "../runtime_decode.ts"; @@ -42,7 +43,7 @@ export function planImprovementConfiguration(value: unknown): JsonObject { } /** Bounded caller observations are advice, not proof of execution authority. */ -export function planCapabilityImprovement(policyValue: unknown, observationValue: unknown): JsonObject { +export function planCapabilityImprovement(policyValue: unknown, observationValue: unknown, scopeValue?: unknown): JsonObject { const policy = normalizeImprovementPolicy(policyValue); const observation = jsonObject(observationValue) ?? {}; const base: JsonObject = { @@ -68,14 +69,45 @@ export function planCapabilityImprovement(policyValue: unknown, observationValue capability_id: direct.capability_id, configuration_ref: direct.configuration_ref, }; if (policy.max_trials === 0) return none("trial_budget_zero"); - const trial = candidates.find(item => identifier(item.capability_id) - && item.applicable === true && item.enabled === false - && identifier(item.configuration_ref) && identifier(item.effect_ref) && identifier(item.rollback_ref)); - if (trial) return { - ...base, recommendation: "propose_reversible_trial", reason_code: "trial_has_effect_and_rollback_basis", - capability_id: trial.capability_id, configuration_ref: trial.configuration_ref, - effect_ref: trial.effect_ref, rollback_ref: trial.rollback_ref, - }; + const scope = jsonObject(scopeValue) ?? {}; + let reviewed: JsonObject | undefined; + for (const trial of candidates) { + if (!(identifier(trial.capability_id) && trial.applicable === true && trial.enabled === false + && identifier(trial.configuration_ref) && identifier(trial.effect_ref) && identifier(trial.rollback_ref))) continue; + // This digest identifies declared trial inputs, not authenticated owner truth. + // Pin the caller and gap as well as the revision and original-owner references. + if (trial.candidate_revision != null && !identifier(trial.candidate_revision)) { + throw new Error("capability candidate revision must be a bounded reference"); + } + const trialBasis = { + goal_id: scope.goal_id ?? null, agent_id: scope.agent_id ?? null, todo_id: scope.todo_id ?? null, + gap_ref: observation.gap_ref, + capability_id: trial.capability_id, candidate_revision: trial.candidate_revision ?? null, + configuration_ref: trial.configuration_ref, effect_ref: trial.effect_ref, rollback_ref: trial.rollback_ref, + }; + const trial_basis_digest = `sha256:${createHash("sha256").update(JSON.stringify(trialBasis)).digest("hex")}`; + const refs = { capability_id: trial.capability_id, configuration_ref: trial.configuration_ref, + effect_ref: trial.effect_ref, rollback_ref: trial.rollback_ref, trial_basis_digest }; + if (trial.trial_feedback != null) { + const feedback = requireJsonObject(trial.trial_feedback, "capability trial feedback"); + if (!identifier(feedback.outcome_ref) || typeof feedback.trial_basis_digest !== "string" + || !/^sha256:[a-f0-9]{64}$/u.test(feedback.trial_basis_digest) + || typeof feedback.status !== "string" || !["succeeded", "failed", "no_evidence"].includes(feedback.status)) { + throw new Error("capability trial feedback requires an outcome reference, basis digest and receipt status"); + } + const stale = feedback.trial_basis_digest !== trial_basis_digest; + // A result is not utility. Ask the original owner to review successes and + // stale feedback; do not propose the unchanged failed trial again. + reviewed ??= { ...base, ...refs, outcome_ref: feedback.outcome_ref, + recommendation: stale || feedback.status === "succeeded" ? "inspect_trial_outcome" : "continue_current_work", + reason_code: stale ? "trial_feedback_stale" : feedback.status === "succeeded" + ? "trial_result_requires_owner_review" : feedback.status === "failed" ? "prior_trial_failed" : "prior_trial_no_evidence" }; + continue; + } + return { ...base, ...refs, recommendation: "propose_reversible_trial", + reason_code: "trial_has_effect_and_rollback_basis" }; + } + if (reviewed) return reviewed; return none("missing_applicability_or_trial_basis"); } @@ -85,9 +117,9 @@ export const improvementContextProvider: AgentContextProvider = { produce(input, config) { const policy = normalizeImprovementPolicy(config.policy); const observation = jsonObject(input.observations.capability_improvement) ?? {}; - const plan = planCapabilityImprovement(policy, observation); + const plan = planCapabilityImprovement(policy, observation, input.scope); return { - guidance: ["Use enabled capabilities directly. Inspect relevant gaps only; trials need original-owner admission, effect and rollback refs. Advice grants no authority; empty/failed advice must not stop useful work."], + guidance: ["Use enabled capabilities directly. Trials need original-owner admission and current effect/rollback refs. Feedback is caller-declared: re-read its owner; success is not utility. Refine failed trials only with changed inputs and fresh feedback. Advice grants no authority; empty/failed advice must not stop useful work."], facts: { planning_trigger: observation.trigger === "replan" ? "replan" : "before_plan", ...Object.fromEntries(Object.entries(plan).filter(([key]) => ![ diff --git a/tests/capabilities/test_capability_improvement.py b/tests/capabilities/test_capability_improvement.py index 4ed2a18192..b966494c26 100644 --- a/tests/capabilities/test_capability_improvement.py +++ b/tests/capabilities/test_capability_improvement.py @@ -105,6 +105,53 @@ def test_corrupt_policy_is_visible_fail_open_and_clearable(registry): assert configuration_summary(json.loads(registry.read_text())["goals"][0]) is None +def test_real_cli_trial_feedback_keeps_failed_inputs_out_of_retry_advice(registry): + cli(registry, "configure-goal", "--goal-id", "example", + "--capability-improvement-mode", "bounded", "--execute") + command = ["agent-context", "--goal-id", "example", "--agent-id", "coordinator", + "--phase", "before_plan", "--capability-gap-ref", "example/gap"] + candidate = {"capability_id": "example-source", "applicable": True, "enabled": False, + "configuration_ref": "owner/config-v1", "effect_ref": "experiment/baseline-v1", + "rollback_ref": "owner/rollback", "candidate_revision": "revision-v1"} + + def observe(*candidates): + args = command.copy() + for item in candidates: + args.extend(["--capability-candidate-json", json.dumps(item)]) + return cli(registry, *args) + + before = registry.read_bytes() + first = contribution(observe(candidate))["facts"] + assert first["recommendation"] == "propose_reversible_trial" + failed = {**candidate, "trial_feedback": { + "outcome_ref": "owner/outcome-v1", "trial_basis_digest": first["trial_basis_digest"], + "status": "failed"}} + facts = contribution(observe(failed))["facts"] + assert facts["reason_code"] == "prior_trial_failed" + assert facts["recommendation"] == "continue_current_work" + assert facts["outcome_ref"] == "owner/outcome-v1" + assert contribution(observe({**failed, "configuration_ref": "owner/config-v2"}))["facts"]["reason_code"] == "trial_feedback_stale" + success = {**candidate, "trial_feedback": {**failed["trial_feedback"], "status": "succeeded"}} + assert contribution(observe(success))["facts"]["recommendation"] == "inspect_trial_outcome" + fresh = {**candidate, "candidate_revision": "revision-v2"} + independent = contribution(observe(failed, fresh))["facts"] + assert independent["recommendation"] == "propose_reversible_trial" + assert independent["trial_basis_digest"] != first["trial_basis_digest"] + # Invalid feedback only loses this optional contribution. A fresh original + # owner read can recover it; neither call changes the registry or admission. + broken = {**failed, "trial_feedback": {**failed["trial_feedback"], "status": "accepted"}} + invalid = observe(broken)["agent_context"] + assert invalid["contributions"] == [] + assert invalid["failures"][0]["code"] == "context_provider_failed" + assert contribution(observe(candidate))["facts"]["recommendation"] == "propose_reversible_trial" + assert registry.read_bytes() == before + cli(registry, "configure-goal", "--goal-id", "example", + "--capability-improvement-mode", "off", "--execute") + off_before = registry.read_bytes() + assert observe(broken)["agent_context"] is None + assert registry.read_bytes() == off_before + + class SettingsHandler(GoalConfigurationRequestMixin): def __init__(self, registry, body): self.path = CHAT_GOAL_CONFIGURATION_PREVIEW_PATH diff --git a/tests/control_plane_ts/goal_capability_organization.test.ts b/tests/control_plane_ts/goal_capability_organization.test.ts index 0b51a9d148..623fd35c90 100644 --- a/tests/control_plane_ts/goal_capability_organization.test.ts +++ b/tests/control_plane_ts/goal_capability_organization.test.ts @@ -58,6 +58,64 @@ test("trial needs applicability, explicit off state, owner, effect and rollback }).reason_code, "trial_budget_zero"); }); +test("reviewed trial outcomes refine advice without becoming utility or execution authority", () => { + const candidate = { ...trial, candidate_revision: "provider/v1" }; + const plan = (value: object, caller = scope, gap = "example/gap") => + planCapabilityImprovement(policy, { gap_ref: gap, candidates: [value] }, caller); + const basis = plan(candidate).trial_basis_digest; + const feedback = { trial_basis_digest: basis, status: "failed", outcome_ref: "owner/outcome-v1" }; + const failed = plan({ ...candidate, trial_feedback: feedback }); + assert.equal(failed.recommendation, "continue_current_work"); + assert.equal(failed.reason_code, "prior_trial_failed"); + assert.equal(failed.outcome_ref, feedback.outcome_ref); + assert.equal(failed.execution_authorized, false); + assert.equal(plan({ ...candidate, trial_feedback: { ...feedback, status: "no_evidence" } }).reason_code, + "prior_trial_no_evidence"); + const success = plan({ ...candidate, trial_feedback: { ...feedback, status: "succeeded" } }); + assert.equal(success.recommendation, "inspect_trial_outcome"); + assert.equal(success.reason_code, "trial_result_requires_owner_review"); + assert.equal(success.execution_authorized, false); + assert.equal(success.adopted, undefined); + for (const patch of [{ candidate_revision: "provider/v2" }, { configuration_ref: "owner/config-v2" }, + { effect_ref: "experiment/changed-input" }, { rollback_ref: "owner/rollback-v2" }]) { + assert.equal(plan({ ...candidate, ...patch, trial_feedback: feedback }).reason_code, "trial_feedback_stale"); + } + for (const caller of [{ ...scope, goal_id: "other" }, { ...scope, agent_id: "other" }, + { ...scope, todo_id: "other" }]) { + assert.equal(plan({ ...candidate, trial_feedback: feedback }, caller).reason_code, "trial_feedback_stale"); + } + assert.equal(plan({ ...candidate, trial_feedback: feedback }, scope, "example/other-gap").reason_code, + "trial_feedback_stale"); + assert.equal(plan({ ...candidate, candidate_revision: "provider/v2" }).recommendation, "propose_reversible_trial"); +}); + +test("failed trials do not hide independent candidates or enabled direct paths", () => { + const basis = planCapabilityImprovement(policy, { gap_ref: "example/gap", candidates: [trial] }, scope).trial_basis_digest; + const failed = { ...trial, trial_feedback: { trial_basis_digest: basis, status: "failed", outcome_ref: "owner/failure" } }; + const fresh = { ...trial, capability_id: "independent-source" }; + const input = { gap_ref: "example/gap", candidates: [failed, fresh] }; + const before = structuredClone(input); + assert.equal(planCapabilityImprovement(policy, input, scope).capability_id, fresh.capability_id); + assert.deepEqual(input, before); + const direct = { ...fresh, enabled: true }; + assert.equal(planCapabilityImprovement({ ...policy, max_trials: 0 }, { + gap_ref: "example/gap", candidates: [{ ...failed, trial_feedback: "invalid" }, direct], + }, scope).recommendation, "inspect_direct_capability"); +}); + +test("malformed feedback is isolated and off advice never reads it", () => { + for (const trial_feedback of ["invalid", { status: ["failed"] }, { + status: "failed", trial_basis_digest: `sha256:${"a".repeat(64)}`, outcome_ref: "/private/result", + }]) { + const observations = { capability_improvement: { gap_ref: "example/gap", candidates: [{ ...trial, trial_feedback }] } }; + const packet = evaluateGoalAgentContext({ ...input, observations, + orchestration: { mode: "single" } })!; + assert.equal((packet.failures as any[])[0].code, "context_provider_failed"); + assert.deepEqual(packet.contributions, []); + assert.equal(planCapabilityImprovement(null, observations.capability_improvement, scope).reason_code, "improvement_off"); + } +}); + test("off preserves existing coordinator output byte for byte at every phase", () => { const orchestration = { mode: "multi_subagent", spawn_allowed: true, max_children: 2 }; for (const phase of ["before_plan", "before_delegate", "after_delegate_result"]) {