diff --git a/.claude/skills/start-better-memory-ui/SKILL.md b/.claude/skills/start-better-memory-ui/SKILL.md index d6e63dde..fe3ff901 100644 --- a/.claude/skills/start-better-memory-ui/SKILL.md +++ b/.claude/skills/start-better-memory-ui/SKILL.md @@ -5,7 +5,7 @@ description: Use when user asks to start, launch, open, or run the better-memory # Start better-memory UI -The MCP tool `mcp__better-memory__memory_start_ui` is a stub that returns "UI not yet implemented" — but the UI itself is fully built. Launch it directly. +The MCP tool `mcp__better-memory__memory_start_ui` spawns or reuses the UI and returns `{"url", "reused"}`. This skill launches it directly instead. ## Project diff --git a/README.md b/README.md index 48065fc9..a60f4c0e 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ It's a memory layer for Claude Code that (with the default sqlite backend) runs ## How it works + 1. **As it works**, the assistant records what it tried and how it turned out — a fix that worked, an approach that failed, a preference you stated. 2. **Between sessions**, those raw notes are distilled into short, durable lessons — signal kept, noise dropped. 3. **Next session**, lessons surface exactly when they become relevant: each typed prompt is scored against the memory store (distinct-keyword evidence plus track record), and every tool call is checked against each memory's triggers, so a pitfall about Bash heredocs arrives as the heredoc is about to run and a planning rule arrives when the planning skill is invoked. Session start carries only a one-line index. No firehose, no guessing. @@ -238,7 +239,7 @@ The skills in `.claude/skills/` (every directory with a `SKILL.md`) are enumerat - **`better-memory-synthesize`** — walks Claude through the per-episode reflection synthesis loop (`memory.synthesize_next_get_context` → decide → `memory.synthesize_next_apply`). Fires when `memory.start_episode` reports `pending_synthesis.pending > 0` or when the user asks to consolidate. - **`rate-session-memories`** — classifies exposed reflections / semantic memories at session end (`cited` / `shaped` / `ignored` / `misled` / `overlooked`) via `memory.list_session_exposures` + `memory.apply_session_ratings`. Triggered by a Stop-block directive from the `session_close` hook when unrated exposures remain. -- **`start-better-memory-ui`** — launches the management UI directly (the `memory.start_ui` MCP tool is a stub; this skill runs `python -m better_memory.ui` itself). Triggered by phrasing like "start ui" / "launch the better-memory UI". +- **`start-better-memory-ui`** — launches the management UI directly (the `memory.start_ui` MCP tool spawns or reuses the UI; this skill runs `python -m better_memory.ui` itself). Triggered by phrasing like "start ui" / "launch the better-memory UI". `better-memory setup` also writes the managed CLAUDE.md block that tells Claude when to invoke `better-memory-synthesize` (the `pending_synthesis.pending > 0` trigger) — no manual CLAUDE.md editing needed. The rating skill fires from the hook directive and needs no instruction line either. @@ -304,7 +305,7 @@ Three shipped upgrades sit on top of that foundation (specs: `2026-07-23-retriev ## Observation lifecycle -Observations don't live forever. They flow through four states (`active` → `consumed_*` → `archived` → deleted) driven by two pipelines: **synthesis** (LLM-driven, runs from the UI button or automatically by `memory.start_episode`) and **retention** (manual; spec §13 explicitly defers auto-scheduling). +Observations don't live forever. They flow through four states (`active` → `consumed_*` → `archived` → deleted) driven by two pipelines: **synthesis** (LLM-driven, runs from the UI button or automatically by `memory.start_episode`) and **retention** (runs automatically at most once per 24h from `memory.retrieve`, or manually via `memory.run_retention`). ```mermaid stateDiagram-v2 @@ -340,7 +341,7 @@ stateDiagram-v2 **Prune** (opt-in, `RetentionService.run(..., prune=True, prune_age_days=N)`): hard-deletes `archived` rows older than `prune_age_days`. `dry_run=True` previews the count without deleting. Reflections are **never** auto-deleted. -Retention is invoked manually — there's no scheduler. Triggers today: +Retention runs automatically from `memory.retrieve` (`RetentionScheduler.maybe_run`, at most once per 24h) and can also be invoked manually. Triggers today: - `memory.run_retention` MCP tool, or - Direct call from a script / cron / CI step. diff --git a/better_memory/skills/memory-retrieve.md b/better_memory/skills/memory-retrieve.md index e395b8bb..6ebfc577 100644 --- a/better_memory/skills/memory-retrieve.md +++ b/better_memory/skills/memory-retrieve.md @@ -43,8 +43,8 @@ When to use: before starting any meaningful coding task, or when entering a new - `dont` — **hard constraints**. Do NOT repeat these approaches. If you're tempted to try something here, stop and reconsider. - `neutral` — general context, no strong signal either way. -4. Also read `insights` (confirmed patterns) and `knowledge` (standards, - language conventions, project docs). +4. Standards, language conventions and project docs are not in the + result — look them up with `knowledge.list` / `knowledge.search`. 5. When a returned reflection or semantic memory actually shapes your work, call `memory.credit(kind, id, class, evidence)` — see "Crediting memories you use" below. `memory.record_use` is a diff --git a/docs/agentcore-ui/agentcore-mapping.md b/docs/agentcore-ui/agentcore-mapping.md index 5e1783da..220beaa3 100644 --- a/docs/agentcore-ui/agentcore-mapping.md +++ b/docs/agentcore-ui/agentcore-mapping.md @@ -32,16 +32,16 @@ Observations and Semantic tabs — kept once, HARD). |---|---|---|---|---|---| | Reflection drawer — rating evidence receipts | Reflections | read | local `session_memory_exposure` (services/exposure_log) | Session-operational state kept in local memory.db in both modes; backend already reads/writes this exact table via the local conn. | agentcore.py:88-97; queries.py:800; design §1 | | Reflection edit form (pre-populate use_cases/hints) | Reflections | read | `get_memory_record` + `_parse_reflection_record` | Form needs only use_cases + hints from the single row, both parsed body-first off a get_memory_record fetch. Only work is decoupling from the join-heavy `reflection_detail` query. | agentcore.py:732-885, 972-983 | -| Retire reflection | Reflections | write | `retire_reflection` | Direct existing map: sets status='retired', moves record to retired namespace, handles AWS-extracted and migrated records; declared on protocol. Only route the UI write through the backend. | agentcore.py:1585-1591; protocol.py:245-247 | -| Promote reflection to general scope | Reflections | write | `promote_reflection` | Direct existing map: sets status='promoted', moves to general/reflections namespace; promotion fully modelled and admitted by the buckets fetch. Only route the UI write through the backend. | agentcore.py:1578-1583; protocol.py:241-243 | -| Episodes tab shell (page load) | Episodes | read | `supports_episodes` capability flag (False) | Shell route issues no query; the tab is hidden by the existing capability flag. Data-less shell gated by a working backend property. | app.py:205-207; agentcore.py:106-107; protocol.py:58-64 | +| Retire reflection | Reflections | write | `retire_reflection` | Direct existing map: sets status='retired', moves record to retired namespace, handles AWS-extracted and migrated records; declared on protocol. Only route the UI write through the backend. | agentcore.py:1845-1851; protocol.py:363-365 | +| Promote reflection to general scope | Reflections | write | `promote_reflection` | Direct existing map: sets status='promoted', moves to general/reflections namespace; promotion fully modelled and admitted by the buckets fetch. Only route the UI write through the backend. | agentcore.py:1838-1843; protocol.py:359-361 | +| Episodes tab shell (page load) | Episodes | read | `supports_episodes` capability flag (False) | Shell route issues no query; the tab is hidden by the existing capability flag. Data-less shell gated by a working backend property. | app.py:205-207; agentcore.py:156-157; protocol.py:58-64 | | Semantic tab shell (filter form + panel mount) | Semantic | read | none (static render) | Pure static shell, zero DB access, hardcoded initial filters; all data arrives via the /semantic/panel sub-request. Backend-agnostic. | app.py:487; templates/semantic.html | | List / filter / search semantic memories (panel) | Semantic | read | `semantic_list` (ListMemoryRecords / RetrieveMemoryRecords) | Working method fans out to list/retrieve and maps into the same SemanticMemory dataclass with every counter; Wilson ranking is Python over those counters. Exposure side-effect stays local. Only wiring: route through backend.semantic_list instead of raw SQL. | agentcore.py:1246, 1285, 1691; services/semantic.py:220 | | Semantic memory detail drawer | Semantic | read | `_get_semantic_record` (GetMemoryRecord) + local exposure ledger | Single-record read maps to GetMemoryRecord (needs a thin public get-one accessor); evidence leg reads the local session_memory_exposure ledger. Both serviceable. | agentcore.py:1193, 1285; queries.py:800 | -| Create semantic memory | Semantic | write | `semantic_observe` (BatchCreateMemoryRecords) | Direct one-to-one: inserts content as a BASE record under userPreference strategy, namespace from scope, content-hash dedup. (Sqlite mode has no local embedding write either any more — embeddings were removed repo-wide in remove-ollama-embeddings.) | agentcore.py:1199; _agentcore_strategies.py:60 | +| Create semantic memory | Semantic | write | `semantic_observe` (BatchCreateMemoryRecords) | Direct one-to-one: inserts content as a BASE record under userPreference strategy, namespace from scope, content-hash dedup. (Sqlite mode has no local embedding write either any more — embeddings were removed repo-wide in remove-ollama-embeddings.) | agentcore.py:1428; _agentcore_strategies.py:60 | | Update semantic memory text | Semantic | write | `semantic_update_text` (BatchUpdateMemoryRecords) | Direct one-to-one: get + batch_update with full metadata snapshot (reserved keys stripped), transient-404 retry. Re-embedding is AWS-managed. | agentcore.py:1383, 1012 | -| Set semantic memory scope (project<->general) | Semantic | write | `semantic_set_scope` (BatchUpdateMemoryRecords namespaces move) | Direct one-to-one: validates scope, moves record between project/general semantic namespaces. Scope is namespace membership rather than a column. | agentcore.py:1410 | -| Delete semantic memory | Semantic | write | `semantic_delete` (BatchDeleteMemoryRecords) | Direct one-to-one: removes record, AWS reclaims embedding. Minor: raises on not-found vs sqlite idempotency (trivial wrapper). | agentcore.py:1444 | +| Set semantic memory scope (project<->general) | Semantic | write | `semantic_set_scope` (BatchUpdateMemoryRecords namespaces move) | Direct one-to-one: validates scope, moves record between project/general semantic namespaces. Scope is namespace membership rather than a column. | agentcore.py:1657 | +| Delete semantic memory | Semantic | write | `semantic_delete` (BatchDeleteMemoryRecords) | Direct one-to-one: removes record, AWS reclaims embedding. Minor: raises on not-found vs sqlite idempotency (trivial wrapper). | agentcore.py:1691 | | Rating diagnostics counters (session_id_missing etc.) | Diagnostics | read | local `rating_diagnostics` | Operational/telemetry state; stays local. The rating sweep that writes these becomes real on agentcore and runs against local memory.db. | design §1, §2; protocol.py:281-288 | | Hook errors panel (grouped-by-day list) | Diagnostics | read | local `hook_errors` | Session-operational state, local in both modes; queries read local sqlite directly. Backend-independent. | design §1 (line 54) | | Hook error detail drawer | Diagnostics | read | local `hook_errors` | Single hook_errors row from local sqlite; local-only operational state. | design §1 (line 54) | diff --git a/docs/agentcore-ui/data-sources.md b/docs/agentcore-ui/data-sources.md index b2153e2a..4f5e47e2 100644 --- a/docs/agentcore-ui/data-sources.md +++ b/docs/agentcore-ui/data-sources.md @@ -32,8 +32,8 @@ In agentcore mode the **memory-content** tables (`observations`, `reflections`, content tables would see **nothing** (empty/partial content) in agentcore mode. By contrast, **operational-state** tables (`session_memory_exposure`, `audit_log`, `hook_errors`, `retention_runs`, `rating_diagnostics`) **always stay local** and remain correct in either mode. -Additional gaps: the Episodes tab is meant to be hidden when `supports_episodes=False` -(agentcore), but **no capability gating exists** in the UI today. +The Episodes tab is hidden when `supports_episodes=False` (agentcore): the episode +routes abort with 404 and the nav link is gated on `caps.supports_episodes`. > **In this inventory, `goes_through_backend = false` for every single item.** The > distinction that matters is not backend-vs-not (nothing uses the backend) but @@ -69,8 +69,8 @@ touching only **operational-state** tables are safe in agentcore mode. | Diagnostics | Overlooked total aggregate | `reflections`, `semantic_memories` | | UI infra | sqlite connection acquisition (opens local db) | — (root cause of all above) | -> The Episodes-tab content reads are **moot in agentcore mode** because the tab should be -> hidden (`supports_episodes=False`) — but the gating is not yet implemented. +> The Episodes-tab content reads are **moot in agentcore mode** because the tab is +> hidden (`supports_episodes=False`) and its routes return 404. ### Reads that bypass the backend but touch ONLY operational state (SAFE in agentcore) @@ -117,27 +117,27 @@ gap — while operational-state writes (hook-error delete/purge) are correctly l ## Episodes tab -> Whole tab is meant to be hidden in agentcore mode because +> Whole tab is hidden in agentcore mode because > `AgentCoreBackend.supports_episodes=False` (AgentCore groups events by `sessionId` and has -> no episode concept). Gating is not implemented yet, so these content reads are otherwise -> "moot in agentcore" only in intent. +> no episode concept). The episode routes abort with 404 when the flag is false, so these +> content reads are moot in agentcore. | Feature | Op | UI surface | Tables | Query / Service | Through backend? | Notes | |---------|----|-----------|--------|-----------------|------------------|-------| -| Episodes tab shell (page load) | read | `GET /episodes` → `episodes.html` | — | (none — renders shell only) | No | `app.py:205-207` renders the shell only; issues NO query. The shell's `hx-get` triggers load the banner, timeline, and drawer sub-fragments. Whole tab hidden in agentcore mode (`supports_episodes=False`). | -| Episode timeline (list grouped by day) | read | `GET /episodes/panel` → `fragments/panel_episodes.html` (+ per-row `fragments/episode_row.html`) | `episodes`, `observations`, `reflection_sources` | `queries.episode_list_for_ui` | No | `app.py:209-226` calls `queries.episode_list_for_ui(conn, project=project_name())`; scope from cwd. RAW SQL, no `StorageBackend`. Main `SELECT` from `episodes` (`WHERE project=?`, `ORDER BY started_at DESC`, LIMIT 100); `observation_count` correlated subquery over `observations` (`episode_id=e.id`); `reflection_count` correlated subquery joining `reflection_sources → observations` counting DISTINCT reflection_id. Rows grouped by `started_at[:10]` via `itertools.groupby`. All three content → AWS-side, but moot (tab hidden). | -| Unclosed-episode banner | read | `GET /episodes/banner` → `fragments/episode_banner.html` | `episodes` | `queries.unclosed_episode_count` | No | `app.py:228-236` calls `queries.unclosed_episode_count(conn, project=project_name())`. RAW SQL: `SELECT COUNT(*) FROM episodes WHERE project=? AND ended_at IS NULL`. No `StorageBackend`. Banner informational-only (spec deviation noted in template); refreshes on load, every 30s, and episode-closed event. Content; N/A in agentcore (tab hidden). | -| Episode detail drawer (open row) | read | `GET /episodes//drawer` → `fragments/episode_drawer.html` | `episodes`, `observations`, `reflections`, `reflection_sources` | `queries.episode_detail` | No | `app.py:238-246` calls `queries.episode_detail(conn, episode_id=id)`; `abort(404)` if None. RAW SQL. Runs 3 queries: `SELECT * FROM episodes WHERE id=?` (mapped via `services/episode.py row_to_episode`); `SELECT ... FROM observations WHERE episode_id=?` (ALL statuses, historical); DISTINCT `SELECT FROM reflections JOIN reflection_sources JOIN observations WHERE o.episode_id=?` (ALL reflection statuses). Drawer renders episode meta, observation list, seeded-reflection list. All content → AWS-side, but moot (tab hidden). | -| Episode close existence pre-check | read | `POST /episodes//close` → `fragments/episode_drawer.html` | `episodes`, `observations`, `reflections`, `reflection_sources` | `queries.episode_detail` | No | `app.py:265-266`: before writing, the close route calls `queries.episode_detail` to confirm the episode exists (`abort(404)` if None). Same raw-SQL read reused as a guard. Not a `StorageBackend` call. After the write it calls `episode_detail` again to re-render the closed drawer. | -| Close episode (set outcome/close_reason) | write | `POST /episodes//close?outcome=` → `fragments/episode_drawer.html` (drawer 'Close as ...' buttons) | `episodes`, `episode_sessions` | `EpisodeService.close_by_id` | No | `app.py:255-288` validates outcome against `_DEFAULT_CLOSE_REASONS` (`{success:goal_complete, partial:plan_complete, abandoned:abandoned, no_outcome:session_end_reconciled}`), then calls `app.extensions['episode_service'].close_by_id(...)`. SERVICE LAYER (`services/episode.py`), NOT `StorageBackend`, NOT `queries.py`. Runs raw SQL within a `SAVEPOINT`: `UPDATE episodes SET ended_at/close_reason/outcome/summary WHERE id=?`, and `UPDATE episode_sessions SET left_at WHERE episode_id=? AND left_at IS NULL`. ValueError on already-closed (409 error card). On success 200 with `HX-Trigger: episode-closed` so banner+timeline reload. `EpisodeService` constructed with `conn=db_conn` (`app.py:93`), bypasses `StorageBackend`/factory. Content-side, N/A in agentcore (`supports_episodes=False`, tab hidden). | +| Episodes tab shell (page load) | read | `GET /episodes` → `episodes.html` | — | (none — renders shell only) | No | `app.py:221-227` renders the shell only; issues NO query. The shell's `hx-get` triggers load the banner, timeline, and drawer sub-fragments. Whole tab hidden in agentcore mode (`supports_episodes=False`). | +| Episode timeline (list grouped by day) | read | `GET /episodes/panel` → `fragments/panel_episodes.html` (+ per-row `fragments/episode_row.html`) | `episodes`, `observations`, `reflection_sources` | `queries.episode_list_for_ui` | No | `app.py:229-249` calls `queries.episode_list_for_ui(conn, project=project_name())`; scope from cwd. RAW SQL, no `StorageBackend`. Main `SELECT` from `episodes` (`WHERE project=?`, `ORDER BY started_at DESC`, LIMIT 100); `observation_count` correlated subquery over `observations` (`episode_id=e.id`); `reflection_count` correlated subquery joining `reflection_sources → observations` counting DISTINCT reflection_id. Rows grouped by `started_at[:10]` via `itertools.groupby`. All three content → AWS-side, but moot (tab hidden). | +| Unclosed-episode banner | read | `GET /episodes/banner` → `fragments/episode_banner.html` | `episodes` | `queries.unclosed_episode_count` | No | `app.py:251-262` calls `queries.unclosed_episode_count(conn, project=project_name())`. RAW SQL: `SELECT COUNT(*) FROM episodes WHERE project=? AND ended_at IS NULL`. No `StorageBackend`. Banner informational-only (spec deviation noted in template); refreshes on load, every 30s, and episode-closed event. Content; N/A in agentcore (tab hidden). | +| Episode detail drawer (open row) | read | `GET /episodes//drawer` → `fragments/episode_drawer.html` | `episodes`, `observations`, `reflections`, `reflection_sources` | `queries.episode_detail` | No | `app.py:264-275` calls `queries.episode_detail(conn, episode_id=id)`; `abort(404)` if None. RAW SQL. Runs 3 queries: `SELECT * FROM episodes WHERE id=?` (mapped via `services/episode.py row_to_episode`); `SELECT ... FROM observations WHERE episode_id=?` (ALL statuses, historical); DISTINCT `SELECT FROM reflections JOIN reflection_sources JOIN observations WHERE o.episode_id=?` (ALL reflection statuses). Drawer renders episode meta, observation list, seeded-reflection list. All content → AWS-side, but moot (tab hidden). | +| Episode close existence pre-check | read | `POST /episodes//close` → `fragments/episode_drawer.html` | `episodes`, `observations`, `reflections`, `reflection_sources` | `queries.episode_detail` | No | `app.py:296-298`: before writing, the close route calls `queries.episode_detail` to confirm the episode exists (`abort(404)` if None). Same raw-SQL read reused as a guard. Not a `StorageBackend` call. After the write it calls `episode_detail` again to re-render the closed drawer. | +| Close episode (set outcome/close_reason) | write | `POST /episodes//close?outcome=` → `fragments/episode_drawer.html` (drawer 'Close as ...' buttons) | `episodes`, `episode_sessions` | `EpisodeService.close_by_id` | No | `app.py:284-320` validates outcome against `_DEFAULT_CLOSE_REASONS` (`{success:goal_complete, partial:plan_complete, abandoned:abandoned, no_outcome:session_end_reconciled}`), then calls `app.extensions['episode_service'].close_by_id(...)`. SERVICE LAYER (`services/episode.py`), NOT `StorageBackend`, NOT `queries.py`. Runs raw SQL within a `SAVEPOINT`: `UPDATE episodes SET ended_at/close_reason/outcome/summary WHERE id=?`, and `UPDATE episode_sessions SET left_at WHERE episode_id=? AND left_at IS NULL`. ValueError on already-closed (409 error card). On success 200 with `HX-Trigger: episode-closed` so banner+timeline reload. `EpisodeService` constructed with `conn=db_conn` (`app.py:73`), bypasses `StorageBackend`/factory. Content-side, N/A in agentcore (`supports_episodes=False`, tab hidden). | ## Semantic memories tab | Feature | Op | UI surface | Tables | Query / Service | Through backend? | Notes | |---------|----|-----------|--------|-----------------|------------------|-------| -| Tab shell (filter form + panel mount) | read | `GET /semantic` (`app.py:487 semantic()`) → `templates/semantic.html`, includes `fragments/semantic_filter_form.html`, lazy-loads panel via `hx-get semantic_panel` (`hx-trigger: load, every 30s, semantic-changed from:body`) | — | (none — static shell render, no DB call) | No | Pure shell render, no DB access. `initial_filters={scope_filter:'', search:''}` passed to template. All data arrives via `/semantic/panel`. | +| Tab shell (filter form + panel mount) | read | `GET /semantic` (`app.py:537 semantic()`) → `templates/semantic.html`, includes `fragments/semantic_filter_form.html`, lazy-loads panel via `hx-get semantic_panel` (`hx-trigger: load, every 30s, semantic-changed from:body`) | — | (none — static shell render, no DB call) | No | Pure shell render, no DB access. `initial_filters={scope_filter:'', search:''}` passed to template. All data arrives via `/semantic/panel`. | | List / filter / search (panel) | read | `GET /semantic/panel` (`app.py:495 semantic_panel()`) → `fragments/panel_semantic.html` (includes `semantic_create_form.html` + `semantic_row.html` per row) | `semantic_memories`, `session_memory_exposure`, `rating_diagnostics` | `SemanticMemoryService.list_for_project` (`services/semantic.py:220`) | No | Service runs RAW SQL directly on the connection (NOT `StorageBackend`). Reads content columns + rating counters, ranks by Wilson lower bound (useful+overlooked positives over rated exposures) in Python, tiebreak created_at DESC. CRUCIAL: this READ has WRITE side-effects — when `track_exposure=True` and `CLAUDE_SESSION_ID` set it INSERTs one `source='retrieve'` row per returned memory into `session_memory_exposure` (dedup via `NOT EXISTS`); when session id missing it bumps `rating_diagnostics metric='session_id_missing'`. Agentcore: `semantic_memories` content (AWS-side / `RetrieveMemoryRecords`); `session_memory_exposure` + `rating_diagnostics` operational (local). Scope filter: None=project rows + all general; 'project'=project only; 'general'=general only. search LIKE-escaped (`%`,`_`,`\`). | -| Detail drawer | read | `GET /semantic//drawer` (`app.py:560 semantic_drawer()`) → `fragments/semantic_drawer.html`; triggered by `hx-get` on `semantic_row.html` into `#semantic-drawer` | `semantic_memories`, `session_memory_exposure` | inline raw SQL in `app.py` + `queries.fetch_rating_evidence` (`queries.py:800`) | No | Two direct sqlite reads, no service/backend: (1) inline `SELECT` of the `semantic_memories` row (content, scope, timestamps, `useful_count`/`times_misled`/`times_overlooked`); `abort(404)` if absent. (2) `fetch_rating_evidence` reads `session_memory_exposure WHERE memory_kind='semantic' AND evidence IS NOT NULL`, newest first, LIMIT 10. Agentcore: row = content (AWS-side); exposure evidence = operational (local). Drawer also hosts the edit form posting to `semantic_update`. | +| Detail drawer | read | `GET /semantic//drawer` (`app.py:600 semantic_drawer()`) → `fragments/semantic_drawer.html`; triggered by `hx-get` on `semantic_row.html` into `#semantic-drawer` | `semantic_memories`, `session_memory_exposure` | `backend.semantic_get` + `queries.fetch_rating_evidence` (`queries.py:823`) | Partly | Two reads: (1) `backend.semantic_get(id=id)` (sqlite: `SemanticMemoryService.get`, a direct `SELECT` of the `semantic_memories` row: content, scope, timestamps, `useful_count`/`times_misled`/`times_overlooked`); `abort(404)` if None. (2) `fetch_rating_evidence` reads `session_memory_exposure WHERE memory_kind='semantic' AND evidence IS NOT NULL` directly on sqlite, newest first, LIMIT 10. Agentcore: row = content (AWS-side); exposure evidence = operational (local). Drawer also hosts the edit form posting to `semantic_update`. | | Create semantic memory | write | `POST /semantic` (`app.py:514 semantic_create()`) → `HX-Trigger 'semantic-changed'`; form in `fragments/semantic_create_form.html` (`hx-post semantic_create`, error into `#semantic-create-error`) | `semantic_memories` | `SemanticMemoryService.create` (`services/semantic.py:80`) | No | Service `INSERT`s into `semantic_memories` (id=uuid4 hex, content, project, scope, created_at/updated_at). Single commit — no embedding write any more (the `semantic_embeddings` vec0 table and its sync_embedder plumbing were removed repo-wide in remove-ollama-embeddings). Validates scope in `('project','general')` and non-empty content, ValueError → 400 card. Direct SQL, NOT `StorageBackend`. Agentcore: content AWS-side; create maps to `CreateMemoryRecord`, embeddings AWS-managed. | | Update semantic memory text | write | `POST /semantic//update` (`app.py:589 semantic_update()`) → `HX-Trigger 'semantic-changed'`; edit form in `fragments/semantic_drawer.html` | `semantic_memories` | `SemanticMemoryService.update_text` (`services/semantic.py:105`) | No | `UPDATE semantic_memories SET content, updated_at WHERE id`; `rowcount==0` → rollback + ValueError('semantic memory not found') → 400 card. No re-embed any more (`_store_embedding` / `semantic_embeddings` removed repo-wide in remove-ollama-embeddings). Direct SQL, NOT `StorageBackend`. Agentcore: content AWS-side (`UpdateMemoryRecord`), embeddings AWS-managed. | | Set scope (project ↔ general) | write | `POST /semantic//scope` (`app.py:533 semantic_scope()`) → `HX-Trigger 'semantic-changed'`; scope toggle buttons in `fragments/semantic_row.html` | `semantic_memories` | `SemanticMemoryService.set_scope` (`services/semantic.py:126`) | No | `UPDATE semantic_memories SET scope, updated_at WHERE id` (always bumps updated_at, even setting same value). `rowcount==0` → rollback + ValueError → 400. Validates scope in `('project','general')`. No embedding change. Direct SQL, NOT `StorageBackend`. Agentcore: content metadata (AWS-side). | diff --git a/docs/agentcore-ui/read-spec.md b/docs/agentcore-ui/read-spec.md index c26cb546..56e3235f 100644 --- a/docs/agentcore-ui/read-spec.md +++ b/docs/agentcore-ui/read-spec.md @@ -55,8 +55,8 @@ gives three things: min_confidence) + `useful_only`; default `status IN ('pending_review', 'confirmed')`; ordered confidence DESC, updated_at DESC, rowid DESC. - **agentcore source:** a **new flat backend method** - `AgentCoreBackend.list_reflections(*, project, status_filter, tech, phase, - polarity, min_confidence, useful_only, limit)`, built on + `AgentCoreBackend.reflection_list(*, project, tech, phase, polarity, status, + min_confidence, useful_only, limit)`, built on `self._data.list_memory_records(memoryId=episodic.memory_id, namespace=…, maxResults≤100, nextToken=…)` fanned out over **two** namespaces (`projects/{actor}/reflections/` + `general/reflections/`), plus @@ -82,7 +82,7 @@ gives three things: / `last_*_at`); SELECT #2 the provenance join `reflection_sources → observations → episodes`. - **agentcore source:** the row leg only, via a **new thin accessor** - `AgentCoreBackend.get_reflection(*, reflection_id)` wrapping + `AgentCoreBackend.reflection_get(*, reflection_id)` wrapping `self._data.get_memory_record(memoryId=episodic.memory_id, memoryRecordId=id)['memoryRecord']` (falling back to `list_memory_records` with `metadataFilters` for BASE records, as `_get_record` already does), @@ -116,7 +116,7 @@ gives three things: - **agentcore source:** **HIDDEN in agentcore mode by default** (`supports_reflection_text_edit=False`). Reflection text is AI-managed. When the flag is enabled (open decision — see Gaps), it reads only `use_cases` / - `hints` from the R2 accessor (`get_reflection` + `_parse_reflection_record`), + `hints` from the R2 accessor (`reflection_get` + `_parse_reflection_record`), decoupled from the join-heavy detail query. - **shape / field differences:** the provenance columns the sqlite form ignores are simply never fetched. @@ -126,7 +126,7 @@ gives three things: - **sqlite source:** `queries.reflection_distinct_projects` — `SELECT DISTINCT project FROM reflections`, then unioned with `project_name()`. - **agentcore source:** a **new backend method** - `AgentCoreBackend.list_projects()` returning the UNION of: + `AgentCoreBackend.distinct_projects()` returning the UNION of: (a) `self._data.list_actors(memoryId=episodic.memory_id)` → `actorSummaries[].actorId` (event-derived actors — every project that has ≥1 `observe()`); and (b) the project set parsed from the local `agentcore_migration` ledger's @@ -183,7 +183,7 @@ gives three things: (content, scope, timestamps, counters); `abort(404)` if absent. Evidence leg via `queries.fetch_rating_evidence`. - **agentcore source:** a **new public accessor** - `AgentCoreBackend.get_semantic(*, id)` wrapping the existing private + `AgentCoreBackend.semantic_get(*, id)` wrapping the existing private `_get_semantic_record` (=`get_memory_record(memoryId=semantic.memory_id, memoryRecordId=id)`), mapped by `_semantic_summary_to_model`. Evidence leg **stays local memory.db** (`fetch_rating_evidence`, `memory_kind='semantic'`). diff --git a/docs/agentcore-ui/write-spec.md b/docs/agentcore-ui/write-spec.md index 7711dc57..68fc9a10 100644 --- a/docs/agentcore-ui/write-spec.md +++ b/docs/agentcore-ui/write-spec.md @@ -41,7 +41,7 @@ repo-wide in remove-ollama-embeddings.) observation→semantic promote action, and all provenance/linked-reflection sections. - `supports_reflection_text_edit` (new; False on agentcore by default — see §Open decision) → hides the inline reflection use_cases/hints edit form. - - `supports_reflection_confirm` (new; False on agentcore) → hides the Confirm action + - `supports_reflection_review` (new; False on agentcore) → hides the Confirm action (reflections are born `active`; there is no `pending_review` gate). sqlite mode returns True for all of these, so nothing about sqlite mode changes. @@ -56,20 +56,20 @@ Legend for **destination**: **AWS** = durable content, goes to AgentCore memory | # | UI write | Current sqlite path | AgentCore-mode path (backend method → AWS API) | Dest | |---|----------|---------------------|-----------------------------------------------|------| -| R1 | **Promote reflection to general scope** (`POST /reflections//promote`) | `ReflectionService.promote_to_general` → `UPDATE reflections SET scope='general', updated_at` on raw conn | `backend.promote_reflection(reflection_id=id)` → `AgentCoreBackend.promote_reflection` (agentcore.py:1578) → `_mutate_namespace_and_status(new_namespaces=[resolve_namespace("general","reflections")], new_status="promoted")` → **UpdateMemoryRecord / BatchUpdateMemoryRecords** (moves record project→general reflections namespace, status→`promoted`). Declared on `protocol.py:241`. | AWS | -| R2 | **Retire reflection** (`POST /reflections//retire`) | `ReflectionService.retire` → `UPDATE reflections SET status='retired', updated_at` on raw conn | `backend.retire_reflection(reflection_id=id)` → `AgentCoreBackend.retire_reflection` (agentcore.py:1585) → `_mutate_namespace_and_status(new_namespaces=[resolve_namespace(actor_id,"retired")], new_status="retired")` → **UpdateMemoryRecord / BatchUpdateMemoryRecords** (moves to `projects/{project}/retired/`, status→`retired`; handles AWS-extracted and migrated records). Declared on `protocol.py:245`. | AWS | -| R3 | **Confirm reflection** (`POST /reflections//confirm`) | `ReflectionService.confirm` → `UPDATE reflections SET status='confirmed'` on raw conn | **HIDDEN/DISABLED** in agentcore mode via `supports_reflection_confirm=False`. Reason: agentcore's reflection status model is `active`/`promoted`/`retired` with **no `pending_review` gate** — reflections are born `active` (== sqlite `confirmed`). There is no confirm method on the protocol and `confirmed` would be neither produced nor read back. Route returns 404/hidden button; never wired. | — | +| R1 | **Promote reflection to general scope** (`POST /reflections//promote`) | `ReflectionService.promote_to_general` → `UPDATE reflections SET scope='general', updated_at` on raw conn | `backend.promote_reflection(reflection_id=id)` → `AgentCoreBackend.promote_reflection` (agentcore.py:1838) → `_mutate_namespace_and_status(new_namespaces=[resolve_namespace("general","reflections")], new_status="promoted")` → **UpdateMemoryRecord / BatchUpdateMemoryRecords** (moves record project→general reflections namespace, status→`promoted`). Declared on `protocol.py:359`. | AWS | +| R2 | **Retire reflection** (`POST /reflections//retire`) | `ReflectionService.retire` → `UPDATE reflections SET status='retired', updated_at` on raw conn | `backend.retire_reflection(reflection_id=id)` → `AgentCoreBackend.retire_reflection` (agentcore.py:1845) → `_mutate_namespace_and_status(new_namespaces=[resolve_namespace(actor_id,"retired")], new_status="retired")` → **UpdateMemoryRecord / BatchUpdateMemoryRecords** (moves to `projects/{project}/retired/`, status→`retired`; handles AWS-extracted and migrated records). Declared on `protocol.py:363`. | AWS | +| R3 | **Confirm reflection** (`POST /reflections//confirm`) | `ReflectionService.confirm` → `UPDATE reflections SET status='confirmed'` on raw conn | **HIDDEN/DISABLED** in agentcore mode via `supports_reflection_review=False`. Reason: agentcore's reflection status model is `active`/`promoted`/`retired` with **no `pending_review` gate** — reflections are born `active` (== sqlite `confirmed`). There is no confirm method on the protocol and `confirmed` would be neither produced nor read back. Route returns 404/hidden button; never wired. | — | | R4 | **Edit reflection text** (use_cases + hints) (`POST /reflections//edit`) | `ReflectionService.update_text` → `UPDATE reflections SET use_cases, hints, updated_at` on raw conn (no embedding write any more — removed repo-wide in remove-ollama-embeddings) | **HIDDEN/DISABLED by default** in agentcore mode via `supports_reflection_text_edit=False`. Reason: reflection text is AI-managed by the AgentCore extraction pipeline; there is **no wired reflection text-update backend method** (only `semantic_update_text` exists), and AWS-extracted records' field editability is unresolved. This is a genuine open decision (see below); DEFAULT is disable. If later enabled, it maps to a new `backend.reflection_update_text(id, use_cases, hints)` → `batch_update_memory_records` body RMW (agentcore.py:1055) → **BatchUpdateMemoryRecords**. | AWS (if enabled) | -| R5 | Rating evidence receipts re-render after any reflection write | `queries.fetch_rating_evidence` on `session_memory_exposure` | **UNCHANGED — LOCAL.** Reads `session_memory_exposure` on `app.extensions['db_connection']` in both modes (session-operational state; `protocol.py:262` list_session_exposures reads the same local table). | LOCAL | +| R5 | Rating evidence receipts re-render after any reflection write | `queries.fetch_rating_evidence` on `session_memory_exposure` | **UNCHANGED — LOCAL.** Reads `session_memory_exposure` on `app.extensions['db_connection']` in both modes (session-operational state; `protocol.py:380` list_session_exposures reads the same local table). | LOCAL | ### Semantic memories (full CRUD, all AWS in agentcore mode) | # | UI write | Current sqlite path | AgentCore-mode path (backend method → AWS API) | Dest | |---|----------|---------------------|-----------------------------------------------|------| -| S1 | **Create semantic memory** (`POST /semantic`) | `SemanticMemoryService.create` → `INSERT semantic_memories` (no local embedding write any more — removed repo-wide in remove-ollama-embeddings) | `backend.semantic_observe(content=…, project=…, scope=…)` → `AgentCoreBackend.semantic_observe` (agentcore.py:1199) → inserts a BASE record under the userPreference strategy, namespace from scope (`resolve_namespace(actor,"semantic")`), content-hash dedup → **BatchCreateMemoryRecords**. Returns record id. Embedding is AWS-managed. Declared `protocol.py:152`. | AWS | -| S2 | **Update semantic memory text** (`POST /semantic//update`) | `SemanticMemoryService.update_text` → `UPDATE semantic_memories SET content` (no local re-embed any more — removed repo-wide in remove-ollama-embeddings) | `backend.semantic_update_text(id=id, content=…)` → `AgentCoreBackend.semantic_update_text` (agentcore.py:1383) → get + batch_update with full metadata snapshot (reserved keys stripped), transient-404 retry → **BatchUpdateMemoryRecords**. Re-embedding AWS-managed. Declared `protocol.py:173`. | AWS | -| S3 | **Set scope (project ↔ general)** (`POST /semantic//scope`) | `SemanticMemoryService.set_scope` → `UPDATE semantic_memories SET scope` | `backend.semantic_set_scope(id=id, scope=…)` → `AgentCoreBackend.semantic_set_scope` (agentcore.py:1410) → validates scope, moves the record between `projects/{project}/semantic/` and `general/semantic/` namespaces (scope is namespace membership, not a column) → **BatchUpdateMemoryRecords** (namespace move). Declared `protocol.py:177`. | AWS | -| S4 | **Delete semantic memory** (`POST /semantic//delete`) | `SemanticMemoryService.delete` → `DELETE FROM semantic_memories` (idempotent) | `backend.semantic_delete(id=id)` → `AgentCoreBackend.semantic_delete` (agentcore.py:1444) → removes the record; AWS reclaims the embedding → **BatchDeleteMemoryRecords**. Minor semantic difference: agentcore raises on not-found vs sqlite idempotency — wrap so the route stays idempotent (treat not-found as success). Declared `protocol.py:181`. | AWS | +| S1 | **Create semantic memory** (`POST /semantic`) | `SemanticMemoryService.create` → `INSERT semantic_memories` (no local embedding write any more — removed repo-wide in remove-ollama-embeddings) | `backend.semantic_observe(content=…, project=…, scope=…)` → `AgentCoreBackend.semantic_observe` (agentcore.py:1428) → inserts a BASE record under the userPreference strategy, namespace from scope (`resolve_namespace(actor,"semantic")`), content-hash dedup → **BatchCreateMemoryRecords**. Returns record id. Embedding is AWS-managed. Declared `protocol.py:237`. | AWS | +| S2 | **Update semantic memory text** (`POST /semantic//update`) | `SemanticMemoryService.update_text` → `UPDATE semantic_memories SET content` (no local re-embed any more — removed repo-wide in remove-ollama-embeddings) | `backend.semantic_update_text(id=id, content=…)` → `AgentCoreBackend.semantic_update_text` (agentcore.py:1630) → get + batch_update with full metadata snapshot (reserved keys stripped), transient-404 retry → **BatchUpdateMemoryRecords**. Re-embedding AWS-managed. Declared `protocol.py:262`. | AWS | +| S3 | **Set scope (project ↔ general)** (`POST /semantic//scope`) | `SemanticMemoryService.set_scope` → `UPDATE semantic_memories SET scope` | `backend.semantic_set_scope(id=id, scope=…)` → `AgentCoreBackend.semantic_set_scope` (agentcore.py:1657) → validates scope, moves the record between `projects/{project}/semantic/` and `general/semantic/` namespaces (scope is namespace membership, not a column) → **BatchUpdateMemoryRecords** (namespace move). Declared `protocol.py:266`. | AWS | +| S4 | **Delete semantic memory** (`POST /semantic//delete`) | `SemanticMemoryService.delete` → `DELETE FROM semantic_memories` (idempotent) | `backend.semantic_delete(id=id)` → `AgentCoreBackend.semantic_delete` (agentcore.py:1691) → removes the record; AWS reclaims the embedding → **BatchDeleteMemoryRecords**. Minor semantic difference: agentcore raises on not-found vs sqlite idempotency — wrap so the route stays idempotent (treat not-found as success). Declared `protocol.py:270`. | AWS | ### Observation → semantic promotion @@ -96,7 +96,7 @@ Legend for **destination**: **AWS** = durable content, goes to AgentCore memory | Action | Gate flag | Reason | |--------|-----------|--------| -| Confirm reflection (R3) | `supports_reflection_confirm=False` | No `pending_review` gate; reflections born `active`. `confirmed` has no server-side meaning. | +| Confirm reflection (R3) | `supports_reflection_review=False` | No `pending_review` gate; reflections born `active`. `confirmed` has no server-side meaning. | | Inline reflection text-edit (R4) | `supports_reflection_text_edit=False` (DEFAULT; open decision) | Reflection text is AI/extraction-pipeline managed; no wired text-update method; AWS-extracted editability unresolved. | | Promote observation→semantic (O1) | `supports_observations=False` | Observations are immutable events with no status lifecycle — cannot mark consumed; SAVEPOINT crosses event+record planes. | | Whole Observations tab | `supports_observations=False` | No observation concept in agentcore mode (events handled transparently). Also removes all provenance/linked-reflection sections. | diff --git a/docs/hooks-setup.md b/docs/hooks-setup.md index e247e6fd..0c93be1a 100644 --- a/docs/hooks-setup.md +++ b/docs/hooks-setup.md @@ -8,30 +8,30 @@ below. | Event | Matcher / `if` | Purpose | Module | |---|---|---|---| -| `SessionStart` | — | Open/reuse a background episode; inject project + general semantic memories and reflections as `additionalContext`; run the wiring autocheck | `better_memory.hooks.session_bootstrap` | +| `SessionStart` | — | Open/reuse a background episode; inject a one-line memory index (or, in `legacy` mode, project + general semantic memories and reflections) as `additionalContext`; run the wiring autocheck | `better_memory.hooks.session_bootstrap` | | `PostToolUse` | `Write\|Edit\|Bash`, async | Capture every matched tool invocation as a spool event | `better_memory.hooks.observer` | | `Stop` | — | Mark session end for consolidation boundary detection; emit the end-of-session rating-sweep directive when unrated exposures remain | `better_memory.hooks.session_close` | | `Stop` | — (second, independent registration) | Remind Claude to record non-obvious observations before stopping | `better_memory.hooks.stop_sweep` | | `UserPromptSubmit` | — | Inject curated memories (semantic + reflections) relevant to the current prompt as `additionalContext` | `better_memory.hooks.contextual_inject` | -| `PreToolUse` | unscoped, latched to one firing/session | Same as above, keyed to the tool name + input | `better_memory.hooks.contextual_inject` | +| `PreToolUse` | unscoped | Inject memories whose declared triggers match the tool name + input (sqlite backend only) | `better_memory.hooks.contextual_inject` | | `PreToolUse` | `Bash`, `if: "Bash(git commit*)"` | Remind Claude to record an observation before a `git commit` that fixes a bug, addresses review feedback, or closes a phase | `better_memory.hooks.commit_checkpoint` | | `PreCompact` | — | Remind Claude to persist the in-flight task, decisions, and open questions before context compaction | `better_memory.hooks.pre_compact` | The `contextual_inject` hook is gated by `BETTER_MEMORY_CONTEXT_INJECT_MODE` (`userprompt` \| `pretool` \| `both` (default) \| `off`). It runs in-process -against `memory.db` and gates each candidate through a two-leg evidence -check (a BM25 match against `reflection_fts`, or — only when that leg is -structurally unavailable, or for semantic memories, which have no FTS -substrate of their own — a keyword-hit fallback), then ranks qualifiers -by reciprocal rank fusion over a Wilson-score usefulness prior; it -injects the top matches (capped at `BETTER_MEMORY_CONTEXT_MAX_ITEMS`) and +against `memory.db`. On `UserPromptSubmit` it skips non-human prompts and gates +each candidate (reflections and semantic memories alike) on distinct-hit +evidence — a memory qualifies when at least `BETTER_MEMORY_CONTEXT_MIN_HITS` +(default 2) distinct, non-ubiquitous prompt tokens appear in it — then ranks +qualifiers by reciprocal rank fusion over a Wilson-score usefulness prior; +it injects the top matches (capped at `BETTER_MEMORY_CONTEXT_MAX_ITEMS`) and never blocks a turn. On agentcore, both kinds are instead gated by the backend's own server-side `relevance_ranks`, falling back to the keyword-hit floor only when that lookup itself fails. `PreToolUse` hosts two independent registrations — -`contextual_inject`, unscoped (matches every tool) but latched to one real -firing per session (later tool calls short-circuit on a state file before -touching the DB), and `commit_checkpoint`, scoped to `Bash` and gated to +`contextual_inject`, unscoped (matches every tool) and trigger-only (it fires +on every tool call and injects memories whose declared triggers match the +tool name + input; sqlite backend only), and `commit_checkpoint`, scoped to `Bash` and gated to fire only immediately before a `git commit`. `stop_sweep` and `pre_compact` are simple, ungated reminder-injectors — no evidence gate, no DB access. @@ -147,8 +147,8 @@ Notes on the shape above (`better_memory/setup/manifest.py::hook_entry`): `pythonw.exe`. - The `if` filter on `commit_checkpoint`'s `PreToolUse` group scopes it to commands matching `git commit*`; `contextual_inject`'s `PreToolUse` - group has no matcher (fires on every tool, latched to one real firing - per session — see above) and is a separate group, not merged with + group has no matcher (fires on every tool, trigger-only — see above) + and is a separate group, not merged with `commit_checkpoint`'s. - If better-memory is installed as a system-wide package rather than a repo-local `.venv`, the `command` is whatever `python -m @@ -171,12 +171,12 @@ Notes on the shape above (`better_memory/setup/manifest.py::hook_entry`): - retrieves project-scoped + general-scope semantic memories and distilled reflections (`do` / `dont` / `neutral` buckets, capped at 20 per bucket), then renders them per `BETTER_MEMORY_INJECT_MODE`: - - `deferred` (the mode actually deployed live) — renders only - general-scope semantic memories in full, plus a one-line index + - `deferred` (the config default, and what `better-memory setup` installs) — + renders no memories in full, only a one-line index ("better-memory knows N reflections + M semantic memories...") - telling Claude to pull specifics via `memory_retrieve` with a - task query. No reflections are rendered in full at bootstrap. - - `legacy` (the config default when the env var is unset) — renders + telling Claude that relevant ones surface as it works, or to pull + specifics via `memory_retrieve` with a task query. + - `legacy` (opt-in via `BETTER_MEMORY_INJECT_MODE=legacy`) — renders up to `BETTER_MEMORY_BOOTSTRAP_TOP_N` (default 5) semantic memories and reflections in full, with the remainder listed in a compact "Index (not expanded)" section. diff --git a/website/agentcore-setup.md b/website/agentcore-setup.md index c19538d7..0bc9d45f 100644 --- a/website/agentcore-setup.md +++ b/website/agentcore-setup.md @@ -7,7 +7,7 @@ - AWS account with Bedrock AgentCore Memory available in your chosen region (`init --region` defaults to `eu-west-2`). - IAM principal (user or role) with the policy below attached. - AWS credentials discoverable by boto3 (env vars, `~/.aws/credentials`, EC2/EKS role, etc.). -- `better-memory[agentcore]` installed: `pip install 'better-memory[agentcore]'` or `uv pip install '.[agentcore]'`. The extra pins `boto3>=1.43.56` and `botocore>=1.43.56` — older versions predate the `extractionMode` parameter on `CreateEvent`, which both the Stop hook's closure event and the end-of-session rating sweep's receipt event (see [Capability table](#capability-table)) depend on. If you install boto3/botocore separately (rather than through the extra), make sure they meet that floor. +- `better-memory[agentcore]` installed: `pip install 'better-memory[agentcore]'` or `uv pip install '.[agentcore]'`. The extra pins `boto3>=1.43.56` and `botocore>=1.43.56` — older versions predate the `extractionMode` parameter on `CreateEvent`, which the end-of-session rating sweep's receipt event (see [Capability table](#capability-table)) depends on. If you install boto3/botocore separately (rather than through the extra), make sure they meet that floor. ## IAM policy @@ -60,7 +60,7 @@ To revert to sqlite: remove the `storage_backend` key from `settings.json`, or s After `init` returns, restart your Claude Code session (or the MCP server) so it picks up the new backend. The MCP server reads `settings.json`, then `agentcore.json`, and constructs an `AgentCoreBackend` instead of `SqliteBackend`. !!! note "Custom `BETTER_MEMORY_HOME`" - The installer injects `BETTER_MEMORY_HOME` only into the MCP server's env block in `~/.claude.json`; hooks run without it and fall back to `~/.better-memory`. If you use a custom home, run `init` against it (`--home` or the env var) **and** make sure hooks can see the same home — otherwise the Stop hook looks for `~/.better-memory/settings.json` and stays on sqlite. + The installer injects `BETTER_MEMORY_HOME` only into the MCP server's env block in `~/.claude.json`; hooks run without it and fall back to `~/.better-memory`. If you use a custom home, run `init` against it (`--home` or the env var) **and** make sure hooks can see the same home — otherwise the hooks look for `~/.better-memory/settings.json` and stay on sqlite. ## Verify @@ -130,7 +130,7 @@ The self-rating / learning loop is **backend-agnostic**: agentcore mode runs the ## UI capability flags -The management UI's `create_app` builds a `StorageBackend` and reads six `@property -> bool` capability flags off it through a `caps` template context processor: `supports_episodes`, `supports_observations`, `supports_provenance`, `supports_retention_runs`, `supports_reflection_review`, `supports_reflection_text_edit`. All six are `False` in agentcore mode (all `True` on sqlite). Every flag now gates a real surface — nav link, route, and template block together — so an agentcore session cannot reach a page whose data the backend does not carry, and cannot land on a page whose gated block would otherwise render empty. The capability table below lists what each flag hides. +The management UI's `create_app` builds a `StorageBackend` and reads seven `@property -> bool` capability flags off it through a `caps` template context processor: `supports_episodes`, `supports_observations`, `supports_provenance`, `supports_retention_runs`, `supports_reflection_review`, `supports_reflection_text_edit`, `supports_triggers`. All seven are `False` in agentcore mode (all `True` on sqlite). Every flag now gates a real surface — nav link, route, and template block together — so an agentcore session cannot reach a page whose data the backend does not carry, and cannot land on a page whose gated block would otherwise render empty. The capability table below lists what each flag hides. **Content routing.** The Reflections tab (list, detail, promote, retire) and the Semantic tab (create, list, detail, edit, scope toggle, delete) read and write agentcore-mode memory content through `AgentCoreBackend` instead of any local table — when the active backend is `agentcore`, the management UI shows AgentCore-resident reflections and semantic memories, not local `memory.db` rows. Reflection status shows the agentcore vocabulary (`active`/`promoted`/`retired`, no `pending_review`) rather than a sqlite-shaped one. `AgentCoreBackend.semantic_list(scope_filter=None)` fans out over both the project and `general/semantic/` namespaces and dedups by id (project wins), matching sqlite's default view (`project OR scope='general'`). diff --git a/website/architecture.md b/website/architecture.md index b868d074..f093a22f 100644 --- a/website/architecture.md +++ b/website/architecture.md @@ -40,7 +40,7 @@ The `sqlite-vec` extension is still loaded by every `memory.db` connection (`bet | Latency | Single-digit ms | 100-500 ms per AWS call | | Cost | Free | Per-API-call + per-record pricing | | Multi-machine sync | No | Yes — memory content and rating counters are shared; the exposure ledger and its evidence stay local/session-scoped on every machine | -| Closure events | N/A | `CreateEvent(role=OTHER)` from Stop hook; end-of-session rating sweeps additionally emit one best-effort `CreateEvent` (`extractionMode: SKIP`) as a durable, team-visible receipt of what was rated | +| Closure events | N/A | None — the Stop hook no longer fires a closure `CreateEvent`; end-of-session rating sweeps emit one best-effort `CreateEvent` (`extractionMode: SKIP`) as a durable, team-visible receipt of what was rated | | Episode tracking | Local `episodes` table | Internal to AgentCore (sessionId) | | Self-rating loop (exposure ledger, `memory.credit`, sweep, Wilson ranking, exploration slot) | Local, entirely in `memory.db` | Same loop — backend-agnostic. Exposure ledger writes to the SAME local `session_memory_exposure` table (session-operational state, never memory content); rating counters (`useful_count`, `ignored_count`, etc.) live on the AWS record and are genuinely shared across teammates. `query`-conditioned retrieval and the contextual-injection evidence gate use server-side semantic search (`RetrieveMemoryRecords`) in place of sqlite's BM25 leg, degrading to Wilson-only / keyword fallback only on an AWS error. | | Tool-call triggers (`triggers` column, PreToolUse trigger channel, drawer editors) | Supported (`supports_triggers = True`) | Not supported: `supports_triggers = False`, `set_triggers` raises, `triggered_candidates` returns `[]`; the UI hides the trigger editors and the channel is inactive | diff --git a/website/configuration.md b/website/configuration.md index 7f7612f4..b03b9ea3 100644 --- a/website/configuration.md +++ b/website/configuration.md @@ -75,7 +75,7 @@ Eight managed hook entries ship with better-memory, across seven hook modules (` - **`better_memory.hooks.session_bootstrap`** (SessionStart) — opens or reuses a background episode for the session and injects the project's curated context as `additionalContext` for Claude's first turn. Shape depends on `BETTER_MEMORY_INJECT_MODE`: in `legacy` (opt-in) it's project-scoped and general-scope semantic memories plus distilled reflections in `do` / `dont` / `neutral` buckets (retrieved up to 20 per bucket and ranked by a Wilson-score lower bound on positive-rated exposures, then confidence, then recency, with the bucket's last slot reserved for an under-rated memory - fewer than 3 rated exposures - when one exists; see [Architecture](architecture.md#self-rating-loop)), with only the top `BETTER_MEMORY_BOOTSTRAP_TOP_N` project-scoped items (default 5; general-scope semantic memories are always shown in full) rendered in full and the rest collapsed into a one-line index plus a `memory.retrieve` / `memory.retrieve_observations` affordance (`BETTER_MEMORY_BOOTSTRAP_TOP_N=0` restores the full dump). In `deferred` (default) it is a single one-line index and nothing else: memories surface through `contextual_inject` (typed prompts and tool-call triggers) or on-demand `memory.retrieve`. Runs in-process against `memory.db` (in agentcore mode it also opens the local `memory.db` — for the exposure ledger only, never memory content — while content reads route through the storage backend instead); failure-isolated: if bootstrap breaks, a fallback directive is injected and the failure is recorded in the `hook_errors` table. Also runs the wiring autocheck (`better_memory.setup.autocheck.maybe_repair`, see `BETTER_MEMORY_WIRING_AUTOCHECK` above) after bootstrap renders, appending at most one summary line — see [Architecture](architecture.md#self-managing-wiring). - **`better_memory.hooks.observer`** (PostToolUse) — captures tool-call snapshots into `spool/` for later observation creation. -- **`better_memory.hooks.session_close`** (Stop) — writes a session-close marker into `spool/`; also emits the rating directive described in [Self-rating loop](architecture.md#self-rating-loop). In agentcore mode (resolved via the env var, else `settings.json`) it additionally fires one `CreateEvent(role=OTHER)` closure event against the current AgentCore session so the episodic strategy extracts within minutes; failure is logged to `hook_errors` and never blocks the marker write. +- **`better_memory.hooks.session_close`** (Stop) — writes a session-close marker into `spool/`; also emits the rating directive described in [Self-rating loop](architecture.md#self-rating-loop). Failure is logged to `hook_errors` and never blocks the marker write. - **`better_memory.hooks.stop_sweep`** (Stop) — a second, independent Stop registration: prints a `systemMessage` reminding Claude to record any non-obvious observations from the session (per the CLAUDE.md mandatory triggers) before stopping. - **`better_memory.hooks.contextual_inject`** (UserPromptSubmit, every prompt; PreToolUse, every tool call) — two channels. Typed prompts (peer-agent messages, command output and system tags are skipped by prefix) pass through a distinct-hit evidence gate: at least `BETTER_MEMORY_CONTEXT_MIN_HITS` distinct, non-ubiquitous prompt tokens must appear in a memory, the same rule for reflections and semantic memories (on agentcore the backend's server-side `relevance_ranks` is the gate instead). Tool calls are matched against each memory's `triggers` (`tool:`, `skill:`, `bash:`, `path:`, `write:large`); a memory without triggers is never tool-served. Survivors are deduped per session, capped to `BETTER_MEMORY_CONTEXT_MAX_ITEMS`, injected as a `` block in `additionalContext`, and logged with source `contextual` or `trigger`. Gated by `BETTER_MEMORY_CONTEXT_INJECT_MODE`. See [Architecture](architecture.md#injection-strategies) for detail. - **`better_memory.hooks.commit_checkpoint`** (PreToolUse, matcher `Bash`, `if: "Bash(git commit*)"`) — fires only immediately before a `git commit` tool call and injects a reminder that CLAUDE.md's mandatory triggers require a `memory.observe` call first if the commit fixes a non-obvious bug, addresses reviewer feedback, or wraps a phase. diff --git a/website/mcp-tools.md b/website/mcp-tools.md index f869b760..b5c75a87 100644 --- a/website/mcp-tools.md +++ b/website/mcp-tools.md @@ -18,6 +18,7 @@ Create a new observation at a decision point. | `theme` | string | optional | Cross-cutting tag (`bug`, `decision`, `architecture`, `convention`, `gotcha`, `dependency`, `infrastructure`, `preference`). | | `trigger_type` | string | optional | What prompted the observation (`user-feedback`, `test-failure`, `review`, `deploy`). | | `tech` | string | optional | Free-text tech tag (`python`, `react`, `sqlite`). | +| `scope` | `project` / `general` | optional | `project` (default) for project-scoped observations; `general` for cross-project workflow rules that surface in every project's `memory.retrieve`. | Project scope is auto-derived from `Path.cwd()`. Returns `{"id": ""}`. @@ -57,7 +58,7 @@ Stamp reinforcement outcome on a memory after validation. | Parameter | Type | Required | |---|---|---| | `id` | string | yes | -| `outcome` | `success` / `failure` | yes | +| `outcome` | `success` / `failure` | optional | Use sparingly and only when the signal is clear. Reinforcement decays stale memories and promotes reliable ones. @@ -123,8 +124,8 @@ Spawn or reuse the management UI. Returns `{"url": "...", "reused": bool}`. Open or reuse a session episode and inject curated context as `additionalContext` markdown. Mirrors what the SessionStart hook does; callable manually for recovery, testing, or post-`/clear` re-injection. Shape depends on `BETTER_MEMORY_INJECT_MODE` (see [Architecture](architecture.md#injection-strategies)): -- **`legacy`** (default) -- project + general semantic memories and reflections. Reflections are retrieved up to 20 per polarity bucket (`do` / `dont` / `neutral`), ranked by Wilson-prior lower bound on hit-rate, ties by confidence. Only the top `BETTER_MEMORY_BOOTSTRAP_TOP_N` project-scoped items (default 5; general-scope semantic memories always render in full) are shown in full -- the rest collapse into a one-line index plus a retrieve affordance (`BETTER_MEMORY_BOOTSTRAP_TOP_N=0` renders everything in full). -- **`deferred`** -- general-scope semantic memories in full plus a single index line; project-scoped semantic memories and reflections are not rendered and surface only via the contextual channel or `memory.retrieve` / `memory.retrieve_observations`. +- **`legacy`** -- project + general semantic memories and reflections. Reflections are retrieved up to 20 per polarity bucket (`do` / `dont` / `neutral`), ranked by Wilson-prior lower bound on hit-rate, ties by confidence. Only the top `BETTER_MEMORY_BOOTSTRAP_TOP_N` project-scoped items (default 5; general-scope semantic memories always render in full) are shown in full -- the rest collapse into a one-line index plus a retrieve affordance (`BETTER_MEMORY_BOOTSTRAP_TOP_N=0` renders everything in full). +- **`deferred`** (default) -- a single index line and no memories at all, general-scope semantic memories included; semantic memories and reflections surface only via the contextual channel or `memory.retrieve` / `memory.retrieve_observations`. | Parameter | Type | Required | Notes | |---|---|---|---| diff --git a/website/observation-lifecycle.md b/website/observation-lifecycle.md index f2c5ac2c..002e5b24 100644 --- a/website/observation-lifecycle.md +++ b/website/observation-lifecycle.md @@ -36,7 +36,7 @@ Implemented by `better_memory.services.retention.RetentionService.run`, default | Rule | Archives observations where … | |---|---| -| **A** | linked only to *retired* reflections, oldest retirement ≥ retention_days old | +| **A** | linked only to *retired* reflections, most recent retirement ≥ retention_days old | | **B** | `status='consumed_without_reflection'` and the status change was ≥ retention_days ago | | **C** | belongs to an episode whose `outcome='no_outcome'` and ended ≥ retention_days ago | diff --git a/website/troubleshooting/agentcore.md b/website/troubleshooting/agentcore.md index 952dfdc3..0b4fe938 100644 --- a/website/troubleshooting/agentcore.md +++ b/website/troubleshooting/agentcore.md @@ -56,7 +56,7 @@ The full message is: boto3 is required for the agentcore storage backend. Install it with: pip install 'better-memory[agentcore]' ``` -The agentcore backend's AWS dependencies live in an optional extra so sqlite-only installs stay lean. Install the extra into the same environment the MCP server (or hook) runs from: `pip install 'better-memory[agentcore]'` or `uv pip install '.[agentcore]'`. Both the server factory and the Stop hook's lazy import raise this same hint. +The agentcore backend's AWS dependencies live in an optional extra so sqlite-only installs stay lean. Install the extra into the same environment the MCP server (or hook) runs from: `pip install 'better-memory[agentcore]'` or `uv pip install '.[agentcore]'`. The server factory raises this hint. ## `AgentCoreConfigError` — corrupt or unreadable `agentcore.json`