Repository navigation
feat(memory): L3 field cognition, retry failed L3 batches (MEMMY-852), feed scanned sessions into L3 - #739
feat(memory): L3 field cognition, retry failed L3 batches (MEMMY-852), feed scanned sessions into L3#739Anson-Se wants to merge 11 commits into
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
faa1762 to
5d7cded
Compare
08ec0d4 to
3a88415
Compare
3a88415 to
7cd5a20
Compare
🤖 Open Code ReviewTarget: PR #739 🔍 OpenCodeReview found 70 issue(s) in this PR.
1.
|
|
A bundle import copied each evolution_jobs and embedding_retry_queue row with the enqueue_seq it had in the source database. Importing into a database that already had queued work failed on the unique enqueue_seq index, and with conflictStrategy "replace" INSERT OR REPLACE could instead delete the local row holding that number. Imported queue rows now leave enqueue_seq out and are numbered on insert, after what is already queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the storage base for itemized machine/project L3, behind algorithm.l3FieldCognition (off by default, local SQLite only): - four new tables (l3_units, l3_items, l3_unit_batches, l3_user_projects) created alongside the existing ones without bumping SCHEMA_VERSION; old L3 tables and rows are untouched - L3FieldRepository for units, items, snapshot memories, unit deletion, inactive-item purge and the user project list; an item removed by a delete or replace keeps when that evidence happened - register the tables for bundle export/import and clear-all; a unit that already exists is kept whole on import (its unit, items, batches, jobs and snapshot from the bundle are skipped; conflictStrategy "error" fails instead; its unfinished batches this database lacks move their sessions' cursors back so those turns are frozen again), and unfinished unit batches are dropped from redacted exports - countL3Units for the per-item length limit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decide which project a path belongs to for the new L3, by reading files only (no git process): - the nearer of a user-selected folder and a git repository wins; a shared root counts once with the git identity - git identity is the normalized origin (ssh, scp-like and https forms map to host/path), else the common git dir, so a clone's worktrees and subdirectories share one project - a session folder outside any repository is a plain-folder project; a tool path only joins the session project it lands in - temp dirs, Codex dated dirs, Claude scratch workspaces, ~/.codex/memories*, Memmy task folders and the home/Desktop/ Documents/Downloads folders themselves are never projects Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The desktop app now tells Memory which folders the user picked as projects, so L3 can treat them as project roots: - POST /api/v1/l3/user-projects replaces the user's whole list and the Memmy task-folder roots; relative paths are rejected; the list is stored regardless of the L3 field cognition flag - memmy-agent's long-running gateway reads the project registry every minute and pushes only when the list changed or the last push failed; a corrupt registry is never pushed - the contract is added identically to the Memory and App copies, with a test that keeps them in sync Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cut a raw turn into evidence fragments for the new L3 so each fact can be attributed by the paths its own evidence touched: - one fragment for the user's words, one per tool call with its paired result (reusing the existing strict call/result pairing), and one per result that matches no call - paths come from file/dir arguments, cd and git -C targets and absolute command arguments; Codex code-mode snippets are read for their cmd/workdir literals; relative paths resolve against the call's workdir, else the session folder; URLs are ignored - each fragment gets the project units its paths resolve to, falling back to the session folder; long outputs keep head, tail and the error lines in between Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- machine profile: probe OS (macOS product version, not the Darwin kernel), arch, shell and node/python3/git/npm/brew versions with 2-second timeouts, and render it in a fixed format by code only - project profile for the new L3: reuse the existing scan and the shared summarizeProjectProfile model call, but feed it only what the project declares (runtime versions probed on this machine belong to the machine profile); write it into the project unit and refresh its snapshot; skip the model when the scan is unchanged and drop the result when the unit was deleted meanwhile - at most one pending profile job per unit, and a unit scanned in the last ten minutes is not scanned again (each session open asks for one); each request carries its own scan id because the immutable dedupe index never frees a key - the old project-profile pipeline is unchanged apart from calling the extracted summarizeProjectProfile Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7cd5a20 to
05815c9
Compare
The new-L3 project profile fingerprint leaves out the runtime versions probed on this machine (they belong to the machine profile), and an unchanged fingerprint skips the model even when the last summary found nothing worth a profile. The scan for it does not run the declared runtimes at all. A unit whose snapshot the user archived is not injected, and rewriting the snapshot keeps it archived. Deleting a snapshot and resetting its unit happen in one transaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wherever a task closes (session close, new task, ended topic, idle timeout), its turns are frozen for the new L3, only through that task so a task still in progress is not cut; a conversation that simply stops therefore still reaches L3. With the switch off nothing changes. Only one L3 runs at a time: the queued jobs of the other one (including its project profiles) wait in the queue until the switch turns back, and do not wake the worker. Older evidence never overturns newer state: a replace or delete whose cited evidence is older than the item's last confirmation is dropped, and what was said before a unit reset does not refill it, even when frozen or rerun later. The worker's next wake-up uses the same readiness rules as leasing (scope order, L1 summaries), so a job it cannot lease does not make it spin. A batch is shown the items removed on evidence newer than some of its own, with when (a batch rerun late, history scanned late, or one spanning the removal), so older evidence does not bring them back. Each lease of a batch job carries a token: a run that outlived its lease neither writes nor changes the job, and a run that finds its batch dead-lettered drops its result. Replace and delete skip items already gone, and ops citing a turn or L1 deleted during the model call are dropped. A machine probe that was running when the machine was reset is dropped. A snapshot is rewritten only when what it shows changed, and project snapshots move only when the machine's did, or when the machine snapshot is archived (so clients drop the cached machine part). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Temporary model failures of L3 jobs wait 1, 5, 15, 30 and 60 minutes before retrying (runAfter) and the sixth goes to dead letter; any other failure (a rejected answer, a bug) would likely repeat, so it gets one retry. Later batches then continue: no failure leaves a job that holds back its scope until restart. Used-up quota (40309) and an unreachable model endpoint (offline) wait 30 minutes without using an attempt; a bad key or endpoint is held by the queue's configuration hold until the configuration changes. L3 jobs are not leased at all while no evolution model is configured. Each dead-lettered new-L3 batch is rerun once automatically when the worker starts; a later batch whose lease ran out does not keep it waiting. Adds POST /api/v1/l3/failed-batches/retry, which resets the user's dead-letter l3_unit_update jobs and batches in place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the new L3 on, conversations first opened by an agent-source scan use the L3 v2 session protocol with Work Memory disabled, so their turns register as L3 input like Hook sessions; with it off, scans stay out of L3 as before, and a session scanned while it was on never goes to the old L3 after it is switched off: its turns wait for the new L3. Every captured turn has a durable owner: with the new L3 on, the existing Work Memory idle flush (one re-armed job per session, runAfter two hours after the last turn) is armed for every v2 session, scanned ones included, and when it comes due it freezes the session's remaining turns into L3 (trigger session_idle). A conversation that just stops therefore still reaches L3 without a close or a later turn. On worker start and after an import, open sessions with unfrozen turns get it armed and closed ones are frozen right away. The scan end is the fast path: when a scan pass finishes a source, the sessions it touched are frozen (trigger scan_pass), also when the pass stops early, and the worker is woken. Hosted imports (importManaged) do the same through POST /api/v1/l3/scan-sessions/freeze, also when an import stops early; it freezes the caller's own sessions and scanned sessions of its source. Freezing only writes rows, so neither scans nor imports wait on L3. Existing sessions and detached legacy history keep their protocol. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new L3 (field cognition) is now on unless configured off (algorithm.l3FieldCognition.enabled or MEMMY_L3_FIELD_COGNITION=0). It still runs only on local SQLite storage and never in cloud mode; there the old L3 World Model stays in use. Test suites that cover the old L3 World Model keep it explicitly: the shared LEGACY_TEST_CONFIG turns the new L3 off, as it already does for turn.judge.v1, and the turn.judge.v1 cases that check old L3 evidence do the same. Services built by the test fixture get a fixed machine id, a temporary home folder and a machine probe that never settles, so tests neither run probes on the real computer nor read its installation id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
05815c9 to
6f0f26f
Compare
|
做了什么
基于
v1.2.7(之前基于 v1.2.6,已整体搬到 1.2.7,代码不依赖 v1.2.6 独有的东西)。2026-10-30 已 rebase 到v1.2.7最新(03f915d6)。上游这期间重写了后台任务队列(#734:租约列、公平调度、统一的领取条件、失败退避和配置阻塞),并把 backend 的 Agent 来源扫描挪进了 Memory(MEMMY-533)。解决冲突时上游改动全部保留,只把本 PR 的 L3 接到上游的新机制上:lease_token和isJobLeaseHeld;mainQueueEligibility,下次唤醒时间用同一套规则;公平调度里登记了l3_unit_update的依赖等待,并注册了它的任务策略;failure_count计数;key 或地址填错交给上游的配置阻塞(改好配置后立即恢复),配额用尽(40309)和断网仍是 30 分钟后再试、不算失败次数;另外单独加了一个提交,修上游 bundle 导入的问题:导入的任务带着源库的
enqueue_seq,目标库已有任务时撞唯一约束导致整个导入失败,"替换"方式下还可能删掉本库占着同一编号的任务。现在导入的任务在目标库里重新编号。这个 PR 包含三件互相依赖的事,按提交顺序:
1. L3 场域认知改造(第 2~8 个提交)
work_memory_idle_flush(每个会话一个去重任务,每轮重新排期,会话安静 2 小时后到期),在捕获的同一个事务里排上;到期时先把这个会话还没交给 L3 的轮次冻结,再做原来的 Work Memory。所以对话直接停掉、从没发过会话关闭,最后几轮也能进 L3。algorithm.l3FieldCognition.enabled(环境变量MEMMY_L3_FIELD_COGNITION),默认打开,只在本地 SQLite 上生效,云端模式不启用。关闭时走原来的 L3,行为不变。SCHEMA_VERSION(仍为 9),只新增 4 张表,旧数据不转写,旧版 Memmy 仍能打开这个库。2. MEMMY-852 失败批次补跑(第 9 个提交)
POST /api/v1/l3/failed-batches/retry,在原任务上复位后重跑。后面批次的租约已经过期时,不会挡住重跑的前序批次。evolution_jobs队列上(用runAfter/available_at、failure_count/max_attempts、租约 CAS 和配置阻塞这些现有字段和机制),没有新建队列、定时器或重试表。3. #590 扫描会话进 L3(第 10 个提交)
agent-source-service的持久化扫描),每个来源处理完(包括中途取消)后同样把碰到的会话交给 L3;交接请求带完整信封和来源,Memory 按打开这些会话时的同一规则确认归属;交接失败只打一条 warn,不影响扫描。ingestion-service)也走同样的处理。Closes #590
改动范围
input.truncated)共 75 个文件,约 7000 行,其中测试约 3000 行。
怎么验证的
上面回放里替换/删除操作的证据时间都是递增的,这些保护在回放中不会触发,回放结果不受影响。
已知限制
🤖 Generated with Claude Code